midiviz

Midi Visualization Experiments

Browser experiments with MIDI, piano and violin rendering, synthesis and audio analysis. How each one works, with numbers, sources and tests, is in the changelog.

MidiViz Piano - Main player with sepia theme UI

Audio analysis

#042 AugmentedNet - a trained Roman numeral network (ISMIR 2021) on onnxruntime-web, alongside #041 (details)
#041 Harmony - chords, Roman numerals, cadences and keys of a MIDI file, live (details)
#040 Rhythm - beats, tempo, meter and swing from the microphone (details)
#039 Tuner - fast, accurate chromatic and string tuner; installable, works offline (details)
#036 Transcription on ONNX - audio → MIDI with Basic Pitch and Transkun on onnxruntime-web (details)
#035 Basic Pitch - audio → MIDI with Spotify’s Basic Pitch on TensorFlow.js, live from the mic (details)

Pianist and piano

#050 Engraved Score - #045’s pianist with the score engraved by nwc-viewer over her: systems scroll in, the cursor goes from note to note and note heads light as they sound; in videos too (notes, details)
#049 Score Scan - #045’s pianist with the printed page (Bach-Gesellschaft, 1853) sliced into systems over her, bars and barlines found in the scan; in videos too (notes, details)
#047 Wardrobe - try hair, dresses, fabrics, shoes and colours on the pianist in one click each; realistic, stylized or cel-shaded, path-traced stills (notes, details)
#045 Path-traced Pianist - #032 path traced: three-gpu-pathtracer and MuTracer, stills and video, against the page’s own picture (notes, details)
#032 Pianist - a pianist at a concert grand playing any MIDI file, filmed by an automatic director; character notes, liner notes (details) and YouTube Shorts (details)
#031 RoboPianist vs #029 - MuJoCo Shadow Hands: their RL policy against our fingering (details)
#030 Concert Stage - the grand on a spotlit stage, pianist hands, lesson mode, hall reverb, MP4 export (details)
#029 Piano Fingering - automatic fingering from MIDI, shown as hands (details)
#028 Grand Piano - a physically based concert grand built in code (details)
#027 JabTunes Reimagined - the 2009 JabTunes piano in three.js (details)
#026 JabTunes (recovered) - an HTML5 port of jabtunes.com as archived in 2010 (details)
#025 Notation 3D - 3D sheet music with bouncing balls (details)

Violin and strings

#051 Duo in the Park - #038’s duo on the lawn under the tree of Sorairo’s hillside: sky, grass, wind, birds and painterly post, at any hour (details)
#048 Bowed Violin - play a physical bowed-string model with pressure, bow position, vibrato, body responses, room sound and Web MIDI (notes)
#046 Score Fingering - photograph or open a PDF of printed music, auto-detect violin/viola, choose among ranked fingerings, edit the reading, read it as a score with playback, metronome, page turns and pen ink, and export annotated pages (notes, details)
#038 Duo - the violinist and the pianist on one stage, from the score, with expression modelled on recordings; or from any MIDI file you open (details, score and expression, your MIDI files)
#037 Violinist - an anime-style violinist playing the solo part of a violin piece, from its score (details, score input)
#034 Violin Sound Lab - six violin engines to compare by ear, from physical models to samples (details)
#033 Violin & Viola Fingering - automatic fingering for violin, viola, cello and double bass (details)
Research: violin sound in the browser, commercial modelled violins, audio → MIDI

Line Rider

#044 Bosh - generate verified Line Rider tracks from MIDI, with synchronized take-offs and landings, forced inline loops, wild-track controls and .track.json export (notes)

Synthesia-style players and synths

#024 Minimal UI - the MidiViz player with a minimal interface
#023 Background Playback - AudioWorklet MIDI clock, keeps playing in background tabs (details)
#022 Multi-Synth - switch between OxiSynth, SpessaSynth and FluidWeb while playing (details)
#021 OxiSynth SF3, #020 SpessaSynth, #019 OxiSynth Worker, #018 OxiSynth - WASM soundfont synths (details)
#017 FluidWeb - RustySynth WASM with SF2 soundfonts (details)
#016 Three.js, #015 WebGL, #014 Synthesia - falling notes over an 88-key piano (details)
#013 Physical Model - MIDI playback with our own piano synthesis (details)

Early visualizations

Tools and tests

WebMscore - MuseScore’s renderer (libmscore in WebAssembly): load scores, export SVG / PDF / MIDI (details)
SF3 to SF2 Converter, FluidWeb Piano Test, FluidWeb Piano (ScriptProcessor), FluidWeb Test, Audio Test, Piano Test

More ideas

using lines
fountain rain
tonejs
note/color buckets