midiviz

Changelog

What each experiment does and how, newest first. The short list is in the README.

6 Nov 2026

Render in parts, its sound on its own, and Resume (#045, #049, #050; work in progress)

• Render → Parts (?exparts=60|120|300|600): the render goes into a folder you pick as parts of about that length, each from one of the director’s cuts to another and never while an intertitle shows; each part starts the scene 3 s before its first frame without drawing it, so the hair and the dress have caught up. The sound is one file of its own for the whole render (notation3d/video-export.js exportAudio, the same plan, room and fades as the picture), so no note or ring is cut where a part ends. The folder also holds render.json (the settings and which parts are done), parts.txt and join.sh (ffmpeg concat, no re-encoding), and the subtitles and chapters. Resume… reopens the last folder (or one picked), loads the render’s settings, and renders the parts not done; a part cut short is rendered again whole.
• video-export.js: exportPlan (the timeline both share), exportVideo’s range and warm. pathtrace/parts.js; tests: pathtrace/test/parts.test.js (2). A 20 s smoke render in three parts joined to 22.0 s with its sound; Cancel then Resume is not tested yet.

Film grain: the same at every resolution (#045, #049, #050, path-traced stills)

• The grain was one noise value a pixel: coarse at 480p, all but gone at 4K. A grain is now a cell of 1/1080 of the frame’s height (a pixel at 1080 rows, as before), smoothly interpolated above that, and below it each pixel averages several, so the grain is fainter by the rows the frame lacks, as a 1080p frame scaled down would be. Measured on a patch of the floor, each frame scaled to 540 rows: 540p 12.1 → 6.2, 1080p 6.1 → 6.1, 2160p 3.0 → 6.4.

Depth of field: her face and her hands sharp together, and a pull when they can’t be (#045, #049, #050)

• In a frame that holds her face and her hands, or both hands, the focus puller (pianist/focus-puller.js) now focuses between the nearest and the farthest of them and closes the aperture just enough that each stays sharp (a blur radius of about 1/2000 of the frame’s width). The room behind still blurs to the shader’s limit, as before. Measured on the 36 camera spots: from behind (back), her hands went from 2.2–2.5 times the sharp limit to within it, the aperture 0.8 of before; the close shots of the keys keep both hands sharp (the off hand was 3.9 times the limit), at half the aperture.
• Where the face and the hands lie too far apart in depth for that (over her shoulder: the face 5 times the limit), the focus stays on the hands and racks to her eyes while she looks up from the keys (the rig’s glances), back after, over about 0.6 s.
• Effects → Face and hands sharp (?dofboth=0 turns it off: the shot’s subject alone, as before). The Blur slider above 1 now also raises the most blur (0.006 of the frame, up to 3 times). Path-traced stills and videos take the same focus and aperture.

Dust in the light: the same at every resolution (#045, #049, #050)

• A mote was 1 to 3.5 pixels at any resolution: a large share of a 480p frame, specks at 4K. Its size is now a share of the frame’s height (0.9 to 3.2 thousandths, what it was at 1080 rows), and where that is under 1.5 pixels it is drawn 1.5 pixels wide and fainter by the area it lacks, so it gives the same light.

Goldberg Variations with repeats: 81 intertitles (liner/pieces/bach-goldberg-repeats.md)

• The 72-minute file had 10 intertitles, 2 % of the screen time (the other pieces’ notes run 1.6–3.8 a minute). It now has 79 pinned lines and the 2 floating stories, about one a minute, 16 % of the time: 2–4 a movement, each where a half starts or repeats (what to listen for on the first pass, a story or a technique on the repeat), each with its note in the panel and its source. History (the 1741 print, Goldberg, Keyserlingk, Forkel’s goblet, Bach’s own copy found in Strasbourg in 1974 with fourteen more canons, the Haussmann portrait), analysis (every canon’s interval, the groups of three, the lament bass of Var. 21), technique (the two manuals on one keyboard) and trivia (Landowska 1933, Gould 1955 and 1981, Scarlatti, Diabelli, Hannibal Lecter). No two overlap, none is dropped, with any take.

Director: Still shots, Subjects and Presets (#032, #045, #049, #050)

• Camera → Director gains Still shots (?dstill=: the share of shots that hold still after the cut), Subjects (?dmix=: screen-time targets for the hands and keys, her and the piano, in place of Favour’s tilt) and Preset (?dpreset=: every director setting at once: Automatic, Hands study, Still camera, The pianist), and Pace a Slower step (0.8). pianist/director.js planShots takes still and mix; pianist/director-menu.js wires the menus on both pages. Without them the director plans as before (its tests pass unchanged).
• Hands study on the Goldberg with repeats: 368 shots of 13 s on average, 82 % still, hands and keys 62 % of the time, her 32 %, the piano 6 %, close and medium only (Automatic: 406 shots, 7 % still, hands 39 %). A 5 s smoke test exports with it from Var. 5.

5 Nov 2026

Goldberg Variations: a tempo for each movement (midis/, #049, #050)

• 13 movements keep the Mutopia editors’ metronome marks (Var. 5, 8, 11, 12, 15, 17, 18, 19, 21, 22, 23, 26, 29). The other 18 have no mark, and LilyPond’s MIDI played them at its default, a quarter = 60: the fast variations 40-70 % too slow, the Adagio (Var. 25) almost three times too fast, and the Ouverture’s fugue at the Ouverture’s tempo. Each now takes a tempo of its character (midis/tools/bach_goldberg_tempo.mjs): the Aria ♩=56, Var. 1, 10, 20 ♩=100, 14 ♩=104, 28 ♩=96, 25 ♪=44, 16 ♩=60 then ♩.=60 from bar 17, and the rest as midis/README.md lists. A choice for these files, not an edition’s.
• bach_goldberg_voicing.mjs retimes each movement before the ornaments are written, so trills keep their speed (73 ornament notes fewer, as the fast movements’ trills are shorter). bach_goldberg.mid is now 41.0 min (was 44.0), bach_goldberg_repeats.mid 79.1 min (was 85.3); Var. 25 is 4.4 min, or 8.7 with its repeats. The standalone Aria keeps ♩=60.
• The score manifest, both liner notes and the tests follow (every heading within 0.11 s of its movement; every bar starts on a note, within 0.083 s). The crossings mode finds 43 passages (5.0 min), as faster passages join.
• Then 10 % faster, the whole work: SPEED 1.1 in bach_goldberg_tempo.mjs plays every movement 1.1 times its tempo above, the editors’ marks too (Var. 5 at ♩=143, the Aria at ♩=62, Var. 25 at ♪=48); the 2.5 s between movements stay. bach_goldberg.mid is 37.4 min, bach_goldberg_repeats.mid 72.1 min. The score manifest, both liner notes (headings within 0.12 s) and the tests follow; every bar starts on a note, within 0.076 s. Crossings mode: 41 passages (4.8 min).

The score in step with an older copy of the Goldberg (#049, #050, every page’s songs)

• After the change of the Goldberg’s endings, a browser that had the old midis/bach_goldberg.mid cached played it against the new score manifest, which leaves out the first endings: the bars ran one or two ahead from the first ending on in Var. 2, 4, 6, 16 and 25. The service worker served MIDI files from its cache first, and noticed a change only later, by size. It now fetches them from the network first, as it does pages and code (sw.js), so a song and the score and liner notes that follow it come from the same deploy; the cached copy is kept for offline.
• sheet/timeline.js checks each movement’s length against the file: where it isn’t the order given, it takes the form that has it (as printed, or the other file’s). The old file now follows the score with 988 bars played, the new one with 974; every bar starts on a note of its file (Var. 16 within 0.125 s).

Goldberg Variations: the second endings, and a version with the repeats (midis/, #049, #050)

• midis/bach_goldberg.mid played both endings of every half that has two (Var. 2, 4, 6, 16 and 25), one after the other: LilyPond’s MIDI plays a repeat as printed. It now plays the second alone, as a performance without repeats does: 121 notes and 14 bars fewer, 44.0 min. Every movement from Var. 3 on starts earlier, by up to 26 s at the da capo.
• New midis/bach_goldberg_repeats.mid, 85 min: each half twice, the first ending the first time and the second the second; the Aria da capo without its repeats. In the song menus of every page.
• midis/tools/bach_goldberg_form.mjs writes both from the file as printed, with the repeats and endings of the LilyPond sources; it is the last step of bach_goldberg_voicing.mjs, whose rerun gives the same files. A note still sounding where the music jumps ends with its bar (6 notes). bach_hands.mjs leaves the Goldberg to that rerun.
• #049 and #050 follow both: the manifest keeps each file’s order of the bars (sheet/tools/goldberg.mjs, performances). With the repeats, the panel scrolls back to the system a half starts on, and the first ending lights before the jump. Every played bar starts on a note of its file (Var. 16 within 0.125 s).
• Liner notes: the complete Goldberg’s times moved with the cut, and a copy for the repeats version (liner/pieces/bach-goldberg-repeats.md). The overview no longer says that this MIDI plays the repeats.
• The tempos stay the Mutopia editors’ metronome marks, in 13 movements; the other 18 have no mark and play at a quarter = 60.
• Tests: the order of the bars without and with repeats, and a timeline from a file’s own order (sheet/test/sheet.test.js, 13); sheet/test/browser.mjs checks the repeats file’s order of Var. 2’s bars.

Experiment #032 - Pianist: leaps and crossings prepared ahead (shared by #038, #045, #047)

• A pianist leaves for a jump as soon as the hand’s last keys are up, and gets there early. The hand here moved late and landed on time at best: it stayed until 0.28 s before the onset whatever the free time, and the smoothing of its placement through runs started it before the keys were up and landed it with a long creep. Over the first 90 s of The Entertainer, Rush E, One More Time, One More Chance, Chopin’s 15th prelude and Prelude1, an isolated leap of 12 cm or more was half done at 0.56-0.91 of its free time and 90 % done at 0.98-1.03 of it (late).
• fingering/motion.js leaps (LEAP, the pianist’s alone: #029, #030 and RoboPianist are as they were): a move of 4 cm or more leaves 30 ms after every key struck before it is up and takes 0.08 s + 5 ms a cm, landing 60 ms before the onset, or up to 0.4 s earlier where there is time; with less time, the whole free time, quicker. pianist/hand-driver.js keeps those moves out of the placement’s smoothing (it restarts on either side of each). Now an isolated leap is half done at 0.24-0.51 of its free time and 90 % done at 0.54-0.65, and it doesn’t start before the release.
• A leap written legato, the keys held up to the note it jumps to while the two are further apart than the hand spans (20 cm), had no free time at all: the hand lets go early enough for the jump, the pedal or the sound carrying it (at most 45 % of the time since the last onset; the sound unchanged). The Entertainer’s left-hand stride bass: 97 of its 99 leaps have free time, was 42.
• Her body and arms get ready for it from the same moment: until the hand leaves, the torso doesn’t look past the leap (it turned into it while the hand was still playing: Chopin’s 15th prelude, its turn done 1.21 of the way at the release, now 0.27), and from then it turns to where the hand lands, ahead of it; the lean in for a key out of reach waits for the hand to leave too. A crossing hand starts to rise for the crossing from its last key up before it (at most 0.6 s ahead, at least 60 ms), not a fixed 0.3 s before.
• One More Time, One More Chance’s right hand misses keys on 9.3 % of the frames (was 10.7 %); the others as they were.
• No snaps. Counted on the skinned bones at 60 fps, a frame that moves 1.5 cm or more and 2.5 times the frames beside it: the worst were a hand jumping 10–13 cm in one frame (One More Time, One More Chance, where a key held legato while the hand had moved on let go), an elbow flipping 17.6 cm from one side of the arm to the other (Rush E, at 19.4 s) and a crossing hand dropping back 5 cm as its keys were let go. Now the worst in the first 60 s of each is 4.3 cm (One More Time), 2.9 cm (The Entertainer) and 5.9 cm (Rush E, a little finger). A free finger leaving its key or on its way to the next follows the motion’s own path and settles into its place in the hand only as it gets there (pianist/hand-driver.js); the elbow’s swing round the arm, its lead included, turns at 5 rad/s at most (pianist/rig.js PHI_RATE); a crossing hand’s move in over its keys eases in and out.
• A finger needn’t stay on its key for the note’s whole length: once the hand has moved 3 cm on from where it struck the key, the finger lets go (after 0.1 s, and 40 % of the written length, at least), and a move of 8 cm or more that the time after the keys would make faster than 200 cm/s at its fastest lets go early enough for that, as for a leap beyond the hand’s span. The sound is unchanged. Right hand keys missed: One More Time, One More Chance 9.3 % → 5.6 %, Goldberg Var. 5–6 2.3 % → 1.0 %.
• Which hand goes over in a crossing, as a pianist decides it: the hand that keeps playing stays down on the keys, underneath, and the one that jumps across goes over it. Before, it was the hand that had left its own register, and then the hand that went over stayed over for any crossing within 3 s, so in Goldberg Variation 5 the left hand stayed on top through the passages where the right hand leaps over the left hand’s running notes: the busier hand (more notes within 0.5 s) was over on 312 of 514 crossed frames. Now at each crossing note the hand whose notes leap further from one to the next around it (the mean step within 0.6 s) goes over, decided for the passage (the majority of the crossing notes within 2.5 s) so the hands trade places only where the music hands the crossing over: the busier hand is under on 447 of 507 crossed frames, and they swap twice, where the roles change.
• The hand underneath a crossing keeps its thumb (Godowsky, as Dale Haven reported his master class in The Piano Teacher, 1963: “the hand using the thumb goes on the bottom”). The fingering kept the thumb off every note played while the hands were crossed, in both hands, as though both hung over the other; now it decides first which hand goes over (the hand that leaps across, by the rule above, moved to fingering/fingering.js crossOver so the fingering and the animation share it) and keeps the thumb off that hand’s notes alone. In Goldberg Variation 5 the hand underneath plays its crossing notes with the thumb 46 times in 150 (was 18), and the hand over 6 in 71 (was 5); Variations 8 (640–700 s) and 14 (860–900 s) alike, 11 → 32 and 11 → 32. A test plays the opening of Bach’s Partita No. 1 Gigue in outline both ways round (one hand running on in the middle, the other leaping from the bass to the treble over it), and pianist/test/wrist.mjs checks the single notes of each crossing passage: the hand over on at most 15 % of them with its thumb, and on no more than the hand underneath.
• Close calls go to the thumb. Where the hands leap about as far in a crossing passage (the median difference under 30 % of the larger, or their notes’ say within 35–65 %), the leaps don’t decide which goes over: the fingering first plays the passage charging both hands for the thumb on its crossing notes, and the hand that takes its thumb on more of its single crossing notes all the same goes under; where both do as often, the hand on more black keys goes over, as the same article advises. Then it fingers the piece again with that hand over. In the Goldberg Variations, the hand going over takes its thumb on 8 % of its crossing notes in Variation 11 (was 17 %), and the hands swap inside a passage 3 times there (was 4) and none in Variation 14 (was 3); fingering a piece with such passages takes twice as long (the Goldberg 2.9 s → 5.1 s, in the page’s worker). A test has a whole-tone scale and a hand rocking between two keys above it step alike: the scale, which passes its thumb under, stays under.

Hands: automatic, with the tracks’ hand crossings, by default (#032, #045, #049, #050)

• Performance > Hands now starts on Automatic, with the tracks’ hand crossings, first in the list. It was From the score’s tracks. For a file whose tracks name the hands (midis/’s Bach files), the fingering’s own split now plays everything but the hand crossings, which keep the edition’s. A file whose tracks don’t name the hands is automatic, as before. ?hands=tracks and ?hands=auto still choose the other two, and a link leaves out hands for the default.

3 Nov 2026

Hands: automatic, with the tracks’ hand crossings (#032, #045, #049, #050)

• Performance > Hands has a third mode, Automatic, with the tracks’ hand crossings (?hands=crossings). It is the automatic split (the one fingering.js finds easiest) except where the file’s tracks cross: where the left hand’s track plays above the right hand’s, at least 4 times with less than 2 s between, the tracks’ split holds from a second before to a second after. The label beside Hands says how many passages it found.
• On the complete Goldberg it finds 53 passages (5.6 of the 44 minutes): 49 in the variations for two manuals, 3 in Var. 1 and 1 in Var. 25, and none in the Aria, whose few left-above-right moments are a held tenor note under a brief low treble note. In those passages 99.6 % of the notes take the tracks’ hand (Automatic: 49 %, as it uncrosses the hands); outside them it agrees with Automatic on 99.8 %.
• fingering.js splitHands now takes the notes that already have a hand as where that hand is, so the split after a passage carries on from where each hand left it. Automatic is unchanged; From the tracks changes only for a file with tracks that name no hand.
• Test: the crossings on a made-up passage, the hands set in and out of it, and a file whose tracks never cross (sheet/test/sheet.test.js, 11).

2 Nov 2026

#051 Duo in the Park: #038 outdoors, in the scene of Sorairo (random-sketches/vibes/sorairo)

• Duo with ?stage=park (or Venue: Park) puts the two on a lawn by the tree of Sorairo’s hillside. park/ holds Sorairo’s sky, terrain, tree, grass, birds, ambience and post chain (light shafts, bloom, grade, anisotropic Kuwahara), with three changes: a flat lawn 4.2 m across at (-10.9, 7.1) blended into the slope over 3.5 m, grass there at a quarter height and without flowers, and a hook in the post chain that draws a second scene into the scene target before the passes run
• park/park.js joins the two scenes, the duo in centimetres and the park in metres: the park camera follows the duo camera at 1/100 scale, held above the ground, and the duo is drawn into the same colour and depth buffers, so grass hides feet and the tree hides the piano. The duo casts into the park’s sun shadow map with a matching orthographic camera, so the piano and the two shade the grass
• The stage and hall are hidden. A directional sun with shadows and a hemisphere sky light take the hour’s colours (Sorairo’s presets: morning to night, Time menu), at pi x 0.75 since Sorairo’s cel lighting doesn’t divide by pi (?lightk=); the lacquer and varnish reflect a sky gradient rebuilt when the hour changes. The room defaults to Dry
• Sorairo’s bloom takes anything above 1.35; the duo’s white keys and highlights in full sun reached 2 and bloomed into glares, so each duo material gets a soft knee above 1 (0.3 of the excess kept)
• Two views, In the grass and From the hillside, join the camera menu; the director opens on the first. Tree style and Birds are in the panel, and the URL keeps stage, tod, tree and amb

Songs: spread chords in the engraved Bach files (notes)

• A chord marked \arpeggio in the score (the wavy line, as in the Goldberg Aria’s bar 11 at 0:30) was struck as a block: strings/lilypond.js now keeps the mark (art.arp), and midis/tools/bach_ornaments.mjs rolls the chord from the bottom up, both hands’ notes together where both are marked, 7 % of a quarter between notes (30-70 ms; 70 ms in the Aria), the roll at most a third of the shortest note, each note held to its written end. 1 in the Aria, 1 in Var. 16, 5 in WTC I No. 4’s prelude, 45 in No. 8’s prelude
• The notes of realized ornaments and grace notes now carry the hand of the note they decorate into midis/tools/bach_hands.mjs (it guessed from the nearest written note, and the main note having been replaced, a trill sometimes went to the other hand: the Aria’s bar 11 trill ended in the left hand, as did right-hand trills in WTC I No. 11’s prelude); the Aria now has 1 left-hand note above the right, from 4
• bach_ornaments.mjs no longer awaits its top-level main: with bach_hands.mjs importing it, the run never finished

#045 Path-traced Pianist: three-gpu-pathtracer 0.0.28

• 0.0.28, released on 2 Nov with the r187 fix, takes the place of GitHub main at ec19f7c in pathtrace/tools/vendor.mjs, so vendor/ comes from a release again. Since ec19f7c it adds a “progress” event and stops measuring sample counts once a render has finished; engines.js reads the counts itself and needs no change. A 320x180, 16 spp WebGPU still traces with no WebGPU errors and looks as before, with and without OIDN (llvmpipe).

#045 Path-traced Pianist: MuTracer 0.3.0

• vendor/ moves to @meimu/mutracer 0.3.0 (from 0.2.0). engines.js asks for batchSamples, the name 0.3.0 gives allowMultipleSamplesPerFrame.
• 0.3.0’s WebGL sphere-light sampler doesn’t compile in a scene that also holds an emissive mesh (“‘randomSampleValue’ : undeclared identifier”), the stage among them: pathtrace/tools/vendor.mjs takes out the stray emissive branch as it vendors.
• 0.3.0 samples emissive meshes as lights, and the grand’s 88 hammer felts glow faintly for the raster picture: a 640x360, 32 spp still took 59 s to trace, against 6.7 s. Their material is marked userData.rasterEmissive, and pathtrace/snapshot.js gives the tracers such a material without emission. The still is back to 6.7 s, the same picture as 0.2.0’s (lavapipe, MuTracer WebGL).

#045 Path-traced Pianist: three-gpu-pathtracer’s WebGPU tracer on three.js r187

• 0.0.27, the last release, failed on r187: r187 binds every storage buffer as an array, and the tracer’s compute shaders no longer compiled (“cannot index into expression of type ‘array'"), so each frame logged invalid compute pipelines and nothing traced. vendor/ now holds three-gpu-pathtracer from GitHub main at ec19f7c (2 Nov), which has the fix (PR #911), with oidn-webgpu 0.6.0, the renamed oidn-web, for its OIDN denoiser (pathtrace/tools/vendor.mjs). engines.js turns off main's new adaptiveFrameBudget and adaptiveDynamicRes, which would retune the frame budget toward 16 ms frames. A 320x180, 16 spp still traces with no WebGPU errors, with and without OIDN (Electron, llvmpipe and SwiftShader WebGPU).

#049 Score Scan and #050 Engraved Score: the score stays in one place

• Automatic no longer moves the panel as the shots change. It picks one place for the song from a survey of the director’s planned shots (120 moments; pathtrace/app.js surveyFrame), measured as the intertitles are (pianist/frame-space.js), and keeps it. It picks again only for another song, director setting, view or liner-notes setting. It no longer moves up for the controls either.
• The liner notes are part of the choice. A place also costs half of what the intertitles would cover of her in the best place it leaves them. A fixed intertitle place, or captions at the bottom, rules out a panel place over it. Automatic intertitles now keep off any place the panel overlaps at all (they kept off only those it covered 15 % of, and the bottom ones could touch it). On the Aria with the automatic camera the panel goes top right and the intertitles bottom left.

#045 Path-traced Pianist and the pages that boot it (#049, #050): a lead-in before the music

• Render → Lead-in before the music: picture and silence before the first note of a render from the start of the piece. The scene comes up from black (the director’s fade starts at the head of the video, director.opts.fadeFrom) while she sits ready at the keys, then the music starts. With the title as the first intertitle, the title comes up with the picture, so it is all but in when the first note sounds. Automatic is 3 s with the title intertitle and 2 s without (it was 1 s of black before). The choices are 1, 2, 3, 4 and 6 s. A clip from later in the piece, or a Short, still starts on the music. The score panel shows the first system during the lead-in, and the subtitle files carry the title from where it shows. In a 5 s test render of the Aria, the audio is silent for the first 3.0 s.

#045 Path-traced Pianist and the pages that boot it (#049, #050): a fade to black at the end

• Render → The end: Fade to black (the default) or Cut. The picture, the score panel and the captions fade to black over the last three quarters of the tail, 2 s at most (1.5 s on the automatic 2 s tail), and the last frame is black. The sound fades with it on an equal-power curve, after the hall’s reverb, so the ring fades too. It applies only where the video ends with the piece: a clip that ends earlier keeps its handle as it is, for editing, and a Short keeps its own fades. notation3d/video-export.js exportVideo takes the fade as fadeOut (seconds); other pages don’t pass it, so their videos end as before. On the Aria’s last 5 s, the picture’s mean luma goes from 51 to 16 (black) over the last 1.5 s.

Fade length, and the score waits for the music (#045, #049, #050)

• Render → Fade length, beside The end: Automatic (three quarters of the tail, 2 s at most, as before), or 0.5, 1, 1.5, 2, 3, 4 or 6 s. A fade longer than the tail starts before the last note. On the Aria’s last 5 s with 4 s, the picture starts to darken 4 s from the end.
• In a video’s lead-in, the score panel stays off until the music: it comes up over the half second before the first note and is whole as it sounds (sheet/overlay.js start, the song’s first note). In a 5 s test render of the Aria with the 3 s lead-in, it shows from 2.5 s and is whole at 3.0 s, where the audio starts.

#049 Score Scan and #050 Engraved Score: the complete Goldberg Variations

• Both follow midis/bach_goldberg.mid (the Aria, 30 variations and the da capo, 44 min) as they follow the Aria. #049 shows the Bach-Gesellschaft’s pp. 263-308 (BSB IIIF, 46 pages, 14 MB, sheet/scores/bga988/). #050 engraves each movement as it is needed (about 0.75 s each), the one at the time asked for first.
• sheet/tools/goldberg.mjs maps the 31 movements’ 956 bars to the 271 systems. A movement starts on an indented system. Each system’s bar count adds up to the movement’s count, from the barlines found and the system’s width. 6 systems in Var. 14 and Var. 24 take barlines read by eye (sheet/scores/bach_goldberg.fixes.json).
• scan.js finds a system’s opening line up to 5 spaces in, so a pencil mark in the margin no longer hides a movement’s first system (Var. 10 and Var. 26 were missed).
• sheet/timeline.js createMovementTimeline finds each movement by the silence before it and numbers the bars through the score: the da capo plays on the Aria’s systems, and between movements the panel holds the last bar. Every bar start lands on an onset of the file (Var. 16 within 0.125 s). sheet/layout.js scanLayoutPages loads each page as its systems first show, and a still or a video loads the ones it opens on first.
• Tests: 2 more unit tests (10), and a still of each page at 1330 s (Var. 16, bar 2) in sheet/test/browser.mjs.

1 Nov 2026

Experiment #038 - Duo: One More Time, One More Chance, and MIDI files of your own

• Yamazaki - One More Time, One More Chance (arr. Never Cry), in a new Arrangements group (duo/midis). The violin plays the file’s melody track, which is written at the singer’s pitch (E3-A4), one octave up (E4-A5); the pianist plays its two piano tracks (1,530 notes, with the pedal). The file’s chord pad, choir, bass and drums are left out. There is no Mutopia score, so the bowing is inferred (226 slurred notes).
• Open MIDI… (or drop a file on the page) plays any MIDI file. The violin takes the track most like a violin melody: a violin or viola program first, then strings, leads and voices, a melody’s name, one note at a time. A file of keyboards only gives him the top line of its highest part, and her the rest. The melody moves by octaves into the violin’s range, with any note left below G3 raised an octave. Two menus appear for these pieces: Violin: any track, or the top line of any track, and Her part: the piano tracks or everything but the drums. Checked headless on the new piece, a one-track piano duet (Heart and Soul: 300 notes his, 1,023 hers), JVKE’s golden hour and a 53-track orchestral Hedwig’s Theme (the violin track, 1,992 notes).
• Mixer (button in the bottom bar, or M) in place of the violin / piano balance: a level for the violin, the piano, and each of her tracks (the two hands, a score’s staves, or a MIDI file’s tracks), from off to +6 dB. Each of her tracks plays on its own channel of the piano’s synth (14 at most; more share one), set by CC7; the pedal goes to each. ?vol=violin,piano keeps the two main levels; the old ?bal= is read as before.
• Piano sound, in the Mixer: the synth (OxiSynth, the default; SpessaSynth; RustySynth; the built-in wavetable piano), the soundfont (Full Grand Piano, as before, its compressed SF3, the Motif ES6 and Roland XP-80 pianos, and three GM fonts) and the preset, read from the soundfont’s own preset list. ?psynth=, ?psf=, ?pprog=. RustySynth and the wavetable piano take no controllers, so there the levels scale each note’s velocity and the pedal is ignored. Each synth’s engine is kept once made: disposing an OxiSynth backend left its audio output calling the freed synth (WASM “unreachable” errors).
• soundfont-engine’s SpessaSynth backend works in Chromium 130: its processor class keeps process as an instance field, which that Chromium’s registerProcessor refuses, and its AudioWorklet has no TextDecoder. worklet-compat.js, loaded before the processor, adds both.
• Her hands are the pianist’s (?hands=model for the VRoid model’s own; changing it reloads the page, since the rig is built on one skeleton or the other): pianist/hand-graft.js cuts #032’s MakeHuman hands, with 4 cm of forearm, out of its body and sets each on her wrist whole, with one rotation and one uniform scale (0.88: 14.6 cm wrist to fingertip, the wrist then her wrist’s girth). The proxy skeleton the rig drives is moved to the new hands’ joints before the rig is made, so the fingering reaches the keys with the fingers that are drawn. A first version fitted the hand bone by bone to the VRoid hand’s proportions (a palm 0.61 of MakeHuman’s, proximal phalanges 1.0-1.36 of theirs), which gave a small palm, swollen fingers and pinched joints. The VRoid hands are folded into the wrist (their bones scaled to 2 %). The new hands keep #032’s skin material.
  • Longer fingers. MakeHuman’s fingers are short for its palm: seen from the back of the hand, the middle finger from the web to the tip was 0.78 of the hand from the web to the wrist crease, where in photographs of women’s hands the two are about equal. Fingers 2–5 are lengthened 1.25× along their bones (loadCharacter(…, { fingers }), the same stretch character.js gives the thumbs: longer, not thicker); the hand is now 16.2 cm, wrist to the middle fingertip. #032’s own pianist is unchanged.
  • Cel-shaded, with ink, by default: her face’s MToon material, untextured, in the median colour of her face’s skin, with its outline. Hands: the pianist’s, realistic skin (?hands=real) keeps #032’s skin.
  • Ink you can see. The models’ MToon outlines are authored in metres (0.75 mm) and three-vrm keeps a world-coordinate outline’s width in world units, so in this centimetre scene they were 0.0075 mm and never drew. Each model’s outline widths are now scaled by its own scale (×100), so Ink draws her, him and her hands at the width they were made with.
• Violin sound from #034, in the Mixer: paste a #034 Violin Sound Lab link (or the part after its ?) and Apply. Duo takes its engine (GM, hybrid, waveguide, Helmholtz), fingering mode, room and seat, the expression and humanize sliders, Take (the same seed as #034), and the bowing, left-hand and timbre options (rosin, resonance, body, WG pressure / position, sordino and the rest), and says what it leaves out: the piece, speed, instrument and performer, which Duo has its own controls for, and the DDSP and VSCO engines. Kept in Duo’s URL as vs=. The sliders and options play on #034’s engines; the GM soundfont plays the notes as they are. A performer (modelled or recorded) shapes the phrases itself, so Phrasing counts only with Performer: as written.
• Free view starts from the hall (The two, from the hall). It had no place of its own before a drag, so ?view=free opened at the scene’s origin, looking at the floor under the piano.
• OxiSynth plays through the hall. The oxisynth backend’s WASM opened an AudioContext of its own and played straight to the speakers, so the Room menu, the master volume and the limiter never reached the GM violin or the piano. The new oxisynth-worklet backend (soundfont-engine/src/backends/oxisynth-worklet.js) runs the same synth (OxiSynthRaw from the same WASM) in an AudioWorkletNode on the page’s context, rendering each 128-frame block on the audio thread. Duo uses it for the violin’s and the piano’s OxiSynth. Checked headless: one AudioContext on the page (there were three), both synths connected to the hall’s input, both sounding (0.081 and 0.045 RMS). RustySynth still plays on a context of its own.

Experiment #032 - Pianist: octaves and repeated chords (shared by #038, #045, #047)

• What she still missed in One More Time, One More Chance was octaves: 68 % of the right hand’s missed frames were chords spanning an octave and bare octaves, most of them on the thumb or the little finger. An octave was at the very edge of what the hand could stretch, and the way it was placed made it worse: the hand sat over the middle of the octave, slid toward whichever end was furthest, and the other end fell short (E4–E5: slid 4 cm toward the little finger, the thumb fully straight and 3.4 cm off its key).
• pianist/hand-driver.js: for keys an octave or so apart the hand spreads to its limit (the thumb and the little finger to SPLAY.max, not SPLAY.held) and moves up to 3 cm in over the keys, where the thumb, whose base is well behind the knuckles, reaches sideways rather than forward. And the slide stops where the worst of the pressed fingers is least out of reach, not wherever the worst one alone wants it (worstOf, a ternary search along the slide).
• Repeated chords and rows of octaves are played from the wrist: the hand comes off the keys between them and drops onto the next one. The stroke planner already bounces the hand there, but only in a gap where the hand holds nothing, and written legato they left none: all 96 of the right hand’s repeated chords had no gap. Now every key the hand holds is let go 40 % of the time to the next one before it (40–150 ms), the sound unchanged, and the wrist lifts the hand about 2–3 cm between them: 365 wrist strokes in the right hand where there were 2.
• Fast repeated chords and rows of octaves move the hand from the wrist, not the elbow. The elbow led every lift of the hand (it rises a moment before it, rig.js elbowLead) and the shoulder rose with it, wrist bounces included: in a row of octaves every 0.23 s the elbow swung up and down with each one. pianist/stroke.js now gives the arm’s own lift apart from the wrist strokes (at(t)[3]), and the elbow and the shoulder follow that alone. One More Time, One More Chance at 81.5–82.8 s (octaves every 0.23 s): the elbow moved 24 cm in all, was 49; the hand’s bounce is the same.
• The same key struck twice at once (two voices on one note, or an empty note a doubled note-on leaves) was fingered as two keys: in One More Time an E4–E5 octave came out 1-2-3, its E4 twice. fingering/fingering.js fingerPiece now fingers the longest of them and gives the others its hand and finger: 1-5.
• Right hand, keys missed: One More Time, One More Chance 19 % → 10.8 % (5.6 % over the whole song), The Entertainer 5.8 % → 2.4 %, Rush E 2.6 % → 1.2 %, Prelude1 0.6 % → 0 %, Chopin’s 15th prelude 7.7 % → 6.7 %; Goldberg Var. 5–6 2.6 % → 3.3 %.

Experiment #032 - Pianist: longer thumbs, and held keys she can’t keep down let go (shared by #038, #045, #047)

• Her thumb was 8.6 cm from the base of its metacarpal to the tip against a 17.1 cm hand, where a woman’s of that hand length is about 10 cm. Out of reach, it was most of what the right hand missed in One More Time, One More Chance (913 of 1,341 missed notes over 120 s): in an ordinary A–C♯–F–A arpeggio it ended fully straight and still 3.5 cm short. pianist/character.js THUMB (1.17) lengthens both thumbs’ three bones as the model loads, before anything is built from it: the joints and bone heads move out along the chain and each vertex the thumb carries is stretched along the bone that carries it (longer, not thicker), so the mesh, the rig’s measurements and the solver agree.
• With a longer thumb the hand was placed further back (the knuckles were set by where the thumb lands), so the little finger’s natural landing fell in front of the keys. pianist/hand-driver.js handDepths now brings the knuckles in as far as either the thumb (just inside its key’s edge) or the little finger (1.1 cm inside) needs, and the thumb still plays by the front edge, where it reaches furthest.
• Keys are no longer always held for as long as they are written. A pianist lets go of a key the hand can’t keep down while it plays on, and the pedal or the sound carries it: letGoTimes places the hand at each moment for what it is striking now (and as it is placed for everything it holds), and an older key, struck at least 0.1 s before, more than 0.5 cm past its finger’s reach either way is let go there. The motion is made again with those releases; the sound and the written notes are unchanged. One More Time, One More Chance: 163 keys let go early in 90 s.
• pianist/test/wrist.mjs now counts a key as missed when its fingertip isn’t on it (off its width by more than 0.3 cm, or more than 1 cm over it), as it looks, in place of the solver’s own error, which didn’t see a finger its splay limit had turned off the key. Right hand, keys missed before → after: One More Time, One More Chance 52 % → 19 %, The Entertainer 15.3 % → 5.8 %, Prelude1 2.8 % → 0.6 %, Goldberg Var. 5–6 4.3 % → 2.6 %, Rush E 3.0 % → 2.6 %; left hands 1–10 % → 0–5 %. Chopin’s 15th prelude’s right hand stays at 7.7 %, most of it on chords wider than a ninth, which this hand can’t span. What’s left in One More Time is mostly a fingertip landing just beside its key (median 0.9 cm past its edge).

Experiment #049 - Score Scan and Experiment #050 - Engraved Score: the score over the path-traced pianist as she plays, two ways, on the Goldberg Aria (sheet/README.md)

• #049 cuts a scan of the printed page into systems: the Bach-Gesellschaft edition, Band 3, p. 263 (1853), the edition of IMSLP #00823. The image comes from the Bayerische Staatsbibliothek’s IIIF server, because IMSLP’s downloads need a human check. sheet/scan.js finds the staves in 16 strips, so a page bowed toward the spine still reads, and finds barlines as columns dark through both staves and the gap between them. On this page it finds 6 systems and the score’s 32 bars (6 + 6 + 5 + 5 + 5 + 5). sheet/tools/detect.mjs keeps them in the score’s manifest.
• #050 engraves the score with nwc-viewer in wrap mode: 7 systems, with the bass’s three voices laid on one staff. The cursor moves between the onsets nwc-viewer placed, and each note head glows as it sounds. The notes come from Mutopia’s LilyPond, as printed (sheet/tools/score_midi.mjs), because the performance MIDI plays its trills as notes. A song with no score is engraved from its own MIDI.
• Both show the system being played in a panel (glass, paper or ink only; one or two systems), light the bar, and scroll the next system up over the last 0.7 s before it is reached. They draw the same into exported videos.
• Bars come from sheet/timeline.js. Its bar length comes from the number of bars played, not from the file’s ppq or time signature: Mutopia’s Aria counts a quarter as 960 ticks at ppq 480, has no time signature, and plays no repeats (LilyPond’s MIDI leaves out \repeat volta).

Score panel placed automatically, bar highlight by default (#049, #050)

• Visuals → Score → Position starts on Automatic: the panel goes where the pianist and the piano fill the least of the frame, using the measure #045’s intertitles use (pianist/frame-space.js), among bottom, top and the corners. sheet/placer.js keeps it steady: it moves only after 3 s in a place, when another is clearly emptier, gliding over 0.6 s, or at once when a cut puts her behind it. Automatic intertitles keep off the panel (pathtrace/app.js reserve).
• Follow with starts on the bar alone. #049’s cursor moves by time within the bar and missed the notes. The cursor is still offered.

three.js r187, MuTracer 0.2.0, three-mesh-bvh 0.9.17 (#045, #047, #049, #050)

• The import maps and pathtrace/tools/vendor.mjs move to three.js 0.187.0 (from 0.185.1), @meimu/mutracer 0.2.0 (from 0.1.3) and three-mesh-bvh 0.9.17. three-gpu-pathtracer stays at 0.0.27, its latest; every three.js name it imports is still exported in r187. The other experiments stay on r170.
• MuTracer 0.2.0 fades spots as three.js does and flips three.js equirect rows itself, so the two workarounds for 0.1.3 are gone from pathtrace/engines.js. With both of ours dropped, mini.html’s spot scene gives 63 (three-gpu-pathtracer 62) and its sky scene 184 (183 before).
• pianist/fabric.js sets the taffeta’s thin film through iridescenceThicknessMinimum / Maximum where three.js has them (r187 renamed iridescenceThicknessRange), and through the old name on r170.
• The raster picture is unchanged (a still at 6 s: mean brightness 85.2 against 85.4). Full-stage path-traced stills can’t be compared here: on SwiftShader MuTracer’s WebGL tracer renders the stage blank before and after (a known problem), and three-gpu-pathtracer’s WebGL tracer gives a different broken picture on each run of the same code. A GPU check is still to do.

Hands from the score or automatic (#032, #045, #049, #050)

• Performance → Hands, ?hands=tracks|auto: the file’s own split where its tracks name the hands (as midis/’s Bach files do), or the split fingering.js finds easiest. On the Aria they differ on 95 of 542 notes: the left hand gets 250 notes from the score and 163 automatically, which gives the tenor line to the right hand. fingering/hands.js replaces the copies of the track-name test in #032 and #045.

#045 and its pages on three.js r187, MuTracer 0.2.0, oidn-web 0.5.0 (also #047)

• Soft shadows were drawing dark contour bands across her skin and specks on the plate. From r17x three.js keeps a VSM shadow map’s depth and depth squared in half floats (r170 packed them into 8-bit RGBA), and that is too coarse over the key light’s 250-1000 cm. The pages use PCF shadows now; ?shadows=vsm brings VSM back.
• MuTracer 0.2.0 fades spots as three.js does, reads three’s equirect rows the right way up and copies vertex colours, so engines.js no longer converts spots or flips the environment for it. pathtrace/test/mini.html on SwiftShader WebGL: ?test=spot 63 against three-gpu-pathtracer’s 60 (13 on 0.1.3 without the workaround), ?test=sky 184 against 170, ?test=vcolor 161,64,64 against 148,60,60 (white on 0.1.3).
• Path tracing: garments are lifted 0.3 cm off the skin along their normals (snapshot.js FABRIC_LIFT). The raster page draws them with a polygon offset that a tracer doesn’t have, so the skin poked through the evening gown’s collar and edges.
• The skin under a garment is kept and sunk into the body instead of hidden (pianist/shell.js coverBody), so a gap in the fabric (a collar’s rhinestones, a bodice’s top edge, an open back) shows skin, not the inside of the body or the stage behind her. Community garments sink the skin a ray from it along its normal meets within 4 cm, 0.8 cm deep, fading to nothing over two rings at the garment’s edge; the built-in dresses sink the skin their bodice and skirt cover, 2 cm deep, over one ring. Face morphs keep the sink (character.js morphGeometry), and taking a garment off restores the skin. ?skin=hide hides covered skin as before. Checked headless on SwiftShader with long-gown, halter-wine and reference at four times in the Goldberg aria: no body vertex lies outside the fabric where it is near it.

#045 as a shared app

• 045_pathtracer.html’s script moved to pathtrace/app.js (boot(plugin)), its markup to pathtrace/shell.js and its styles to pathtrace/app.css. The page is 46 lines. A plugin adds menus, follows each song and frame, and draws over exported frames. #049 and #050 are such plugins (sheet/plugin.js). A ?shot still of #045 at 6 s matches the picture before the move.
• notation3d/midi-score.js midiToScore takes clefs, a clef for each track’s staff.
• Tests: sheet/test/sheet.test.js (6: timeline, placement, the page reader on a drawn slanted page, hands, automatic placement) and sheet/test/browser.mjs (#049 and #050 headless, and a 5 s export of #049).

Experiment #032 - Pianist: hands at rest on the community outfits’ dresses (shared by #038, #045 and #047)

• The rig rests her hands on what she wears (state.restCover, the page’s dress.object), but a community garment (the long gown, the shift, the bob outfits and every other MakeHuman community dress) is skinned to her body and lives there, so dress.object was an empty group: the rig saw no dress, laid the hands 1.2 cm over her skin, and they sank into the gown. pianist/community.js now points the group at its garment (userData.cover) and pianist/rig.js reads it; the cloth may lie up to 14 cm off the skin (was 8: a gown’s skirt across the lap lies further out), and a sleeve, the vertices an arm bone carries, doesn’t count as the lap.
• pianist/test/rest.mjs reads the garment the same way, so it checks those outfits at all. In lap mode all 13 outfits now keep the hands out of her and the dress, on One More Time, One More Chance and Prelude1 (long gown: 0.8 cm into the skirt before, 0.2 cm clear after; the cocktail dress: 3.4 cm into it before). Left: the cocktail dress’s left hand rests 5.5 cm above its skirt in lap mode, and the long-sleeved updo gown’s clasped hands go 0.65 cm into the dress (0.5 allowed).
• wrist.mjs and rest.mjs take GL=llvmpipe: Mesa’s llvmpipe through ANGLE’s GL-EGL backend, which runs the wrist suite in 44 s on this 64-core machine where SwiftShader took over half an hour.

28 Oct 2026

Experiment #046 - Score fingering: 🎼 Stand mode, for a laptop turned on its side as a music stand (notes)

• The app is drawn a quarter turn round, full screen, the page upright and fitted to the tall screen; ⟲ flip for the other side (?stand=left|right). Tap the right of the page, scroll, or press → / ↓ / Page Down (page-turner pedals) for the next page; the left of the page, ← / ↑ / Page Up to go back; one turn per scroll gesture. The next PDF page is recognized in the background (Stand and Reader), so turns are immediate. Ink, popups and scrolling map pointer and wheel input into the rotated frame (score-fingering/stand.js), so annotations land under the pen and turn with the page.
• A Size slider (25–150%, remembered) scales the pencil finger numbers on the page and in the PNG / PDF exports. 100% is 48 px on the page, twice the earlier 24 px. Numbers grow away from their notes, and near the top of the page they are kept on it; tap targets follow.
• Double stops stack closer, with the same small gap between numbers at every size (centres one numeral height plus 4 px apart, was 26 px).
• Finger numbers move off printed markings such as div., dynamics and other notes: the page is sampled with staff lines removed and a number that would sit on ink moves to the nearest clear spot, a little higher or to the side; a double stop moves as one, and numbers keep clear of each other.
• A finger that repeats the one just written is not written again (three 3s in a row show one 3). It is written again on a new system, after a rest of a beat or more, after six beats without one, or when the string or position changes; repeated double stops are treated the same way. repeated fingers shows them all.
• Tests: stand.test.js (frame round trips, rotated boxes, drag / wheel axes, one turn per gesture); the browser smoke adds 10 checks (layout and fit, tap / scroll / ↓ turns, read-ahead, ink position on the turned page, flip, exit).

26 Oct 2026

Experiment #046 - Score fingering as a score reader (notes)

• 📖 Reader shows the page full width with the controls hidden (⛶ full screen); Space plays, ← → turn pages, M the metronome. A transport plays the recognized page through ScoreShift’s player (notation-ir Web Audio voice, lookahead scheduler, bar-aware metronome) from the top, from a tapped bar or from a note’s popup, with live tempo, speed and metronome; the sounding noteheads are ringed, the playing staff kept in view, and at the end of a page playback turns to the next page with music and carries on. Fingerings can be hidden for plain reading.
• ✎ Ink: pen annotations from notation-ir annotations.js, attached to the note, passage or moment under them so they follow corrections; colours, widths, eraser, undo / redo, re-attach a mark whose note was removed; kept per file and page in localStorage and drawn into the PNG / PDF exports.
• Playback now sounds like the instrument: each note is read from the University of Iowa violin or viola recordings #034 ships (the mezzo-forte take of each string, loaded on the first Play), on the string it is fingered on, with its recorded attack, a pitch-synchronous loop for held notes, delayed vibrato and a small room (score-fingering/sound.js); Sound: Piano keeps the light additive voice. ⟳ Loop repeats a passage of tapped bars, shaded on the page, optionally from 60% speed rising 5% a pass.
• Fixed: on Sibelius pages where the viola part goes divisi on two braced staves (8 of the 17 pages), the lower staves were read as a second part starting at the top of the page, so they played at the same time as the opening bars. ScoreShift now starts such a part’s bars with the bars they stand under (random-sketches score.js alignParts, with a test from page 2).
• Fixed: ink written right after turning ✎ Ink on, before the page’s music was ready (the modules load on first use), was dropped; it now waits and attaches. A stroke the browser cancels part way keeps what it wrote, and ink that cannot attach says so in the transport (the status line is hidden in Reader).
• Tests: reader.test.js (bar starts on the playback clock, divisi bars, tap-to-bar, sounding heads, which recording a note uses, ink attached and stored); the browser smoke adds 22 checks (playhead, viola recordings, divisi timing, metronome / speed, loop and speed-up, Reader layout and keys, tap-to-play, page turn while playing, ink attach / early ink / undo / persistence / export).

Experiment #032 - Pianist: crossing hands no longer pass through each other (shared by #038, #045 and #047)

• In Goldberg Variation 5 the hand going over stayed at the other’s height, its wrist only tilting up, so the palms met: measured as capsules on the bones (palm 1.6 cm, fingers 0.8–0.9), one hand was more than 1 cm into the other on 47 % of the crossed frames. Now 9 %.
• The hand over is carried up to 4.5 cm higher (pianist/hand-driver.js setLift): 2.5 cm as it comes over, plus whatever overlap is still measured on the drivers’ joints each frame (pianist/rig.js handsOverlap), remembered for a second so a passage that crosses again and again comes over at the height it needed last time. Its free fingers and thumb ride up with it rather than hang down into the other hand. The hand under drops 1.5 cm. The wrist rises faster (0.05 s instead of 0.12).
• Raised, a hand’s pressed finger reaches less far forward, so it moves in over its key (up to 5 cm), coming down steeply as a crossing hand does; only what that can’t make up comes off the height. Keys missed in Variations 5–6: left 2.6 %, right 6.3 %.
• Which hand goes over, and when to start lifting it, now comes from the score: markCrossings marks both hands’ notes (the left above the right is the right below the left), so the one going over is the one that has left its own register for the note, 7 semitones or more past the middle of what it plays around it (hands driver crossAt, rising 0.3 s before the note). Before, the hand lifted ahead was sometimes the one that stayed under.
• pianist/test/wrist.mjs fails when one hand is more than 1 cm into the other on more than 15 % of the crossed frames.
• Then lower and steadier. The hand over is carried 1.2 cm higher as it comes over and at most 3 cm in all (was 2.5 and 4.5; median 2.6 cm through Variation 5, was 4.4), the one under drops 1 cm (was 1.5), and the overlap feedback adds less per frame. And the hands no longer swap over and under in the middle of a passage: the hand that went over stays the one over for the next crossing within 3 s (either hand can be on top in any crossing, so the right hand’s run passing below the left’s held note stays under it); only a fresh crossing after a pause is decided anew, from the score. Variations 5–6: 39 swaps inside a passage before, 0 after; hands more than 1 cm into each other on 11 % of the crossed frames; keys missed left 2.0 %, right 5.1 %. wrist.mjs now also fails on more than 2 swaps inside a passage. (Rush E’s one near-crossing at 19.6 s, the left hand landing beside the right, wrists 8 cm apart, touches from the side, which lifting doesn’t clear; it has its own budget.)

25 Oct 2026

Experiment #048 - Bowed Violin (notes)

• A monophonic physical bowed-string instrument running live in an AudioWorklet: hysteretic stick/slip bow friction excites a two-part fractional-delay waveguide, with pitch-tracking loss, changing bow position, pressure, rosin grip, bow noise, portamento and slow performance variation.
• Four open-string models cover G3, D4, A4 and E5. The played note automatically chooses a string; filtered feedback delays add sympathetic resonance from all four open strings. Legato keeps the bow moving while the active string or pitch changes.
• The string passes through a selectable violin-body response and a generated stereo room. Two included body responses are fitted from the University of Iowa Musical Instrument Samples violin recordings; synthetic classic, dark and bright bodies, a dry string and any user-loaded WAV response are available too.
• The XY performance pad controls expression vertically and vibrato horizontally, with optional spring return or press-to-bow behavior. Computer keys play chromatically from G3, and Web MIDI supports notes, CC11/CC2 expression and CC1 vibrato.
• Six presets cover solo legato, sul tasto, sul ponticello, baroque, flautando and the raw string. A live display reports the selected string, pitch and stick/slip regime while waveform and spectrum traces show the output.
• Offline diagnostics render G3–E6 within 4 cents with about one slip per period. node --test physical-violin/test/*.test.mjs checks pitch, stability, page wiring and the experiment’s standalone terminology.

25 Oct 2026

Experiment #032 - Pianist: the Goldberg Variations’ hand crossings, and each hand from the score in the Well-Tempered Clavier (shared by #038, #045 and #047)

• The Goldberg files had one track, so the pianist guessed the hands, and the split (fingering.js splitHands) charges for every left-hand note above the right. Variation 5, “Hand crossing”, played none: measured against the Mutopia score, its left hand has 83 notes sounding above the right, the split 2. The same in every two-manual variation: Var. 8 44 against 2, 11 81/1, 14 91/0, 17 55/0, 20 118/1, 23 55/1, 26 87/1, 28 90/0.
• midis/tools/bach_hands.mjs aligns each movement with its LilyPond source (as the voicing does) and writes the notes into a “Right hand” and a “Left hand” track, which the page already reads (handsFromTracks). The source’s two parts are the two hands. A left hand that crosses over only changes clef, and the odd upper-part note written on the lower staff (\staffDown, Var. 5 bar 26) stays with the right hand. 17,163 of the score’s 17,349 notes are matched; the rest of the file’s 18,506 (realized ornaments) take the hand of the matched note nearest in time and pitch. Notes, timing and velocities are unchanged; bach_ornaments.mjs (which the voicing script runs) now ends by calling it.
• The rig, on that many crossings: which arm goes over is decided from where the wrists will be 0.06–0.18 s ahead (the baked placement), so the crossing wrist is already rising as it comes over rather than once it has arrived. The arm that is over stays over until the other is clearly further across (3 cm), so it doesn’t flicker as they pass. And the over hand lifts its wrist less while its thumb plays (from a high wrist the thumb, which hangs from the side of the hand, cannot reach). In Variation 5 the hands now cross on 154 frames of 1410 (before: 0), and the forearms come within 4 cm of each other on 10 (0.7 %, from 17).
• Fingering a crossing hand (fingering/fingering.js markCrossings, weight crossHand): a note played across the other hand (right below a left-hand note sounding within 0.15 s, or left above a right) costs 18 on the thumb, 12 on 5 and 6 on 4, so it goes to 2 or 3, the long fingers that reach down from a hand held over the other. Variation 5’s crossing notes, right hand: thumb 40 → 11, 2 and 3 61 → 95; left hand: thumb 44 → 13. The benchmarks are unchanged (core 15/15, scales 48/48). PARNCUTT leaves it out.
• The two hands’ fingers tangled when the left reached over to a key just above the right hand’s thumb with the wrists still apart (Var. 5 at 6:59 + 4.8 s: left on F♯4, right on A3 to E4). Three changes in pianist/rig.js: two hands whose pressed fingertips interleave count as crossed, whatever their wrists; the hand that goes over is the one that has come further across from where it plays around this moment (its baked wrist over ±0.6 s: the leaping left hand over the right hand’s run, as Bach writes it), not the one whose wrist is further across; and the crossed hands are separated in depth, the over hand playing 3.5 cm further in along the keys (a white key between the black keys on its narrow part) and the under hand 1.5 cm nearer the front (hand-driver.js setDepth). Variation 5 now misses keys on 3.4 % (left) and 3.5 % (right) of the frames, from 3.8 and 5.9.
• The same tracks for the Well-Tempered files (all 15, and bach_wtc1_11p): 97–100 % of each score’s notes matched; WTC II No. 6’s prelude has 31 left-hand notes over the right, WTC I No. 12’s fugue 17. WTC I No. 12’s prelude is written as one part whose voices move between the staves with \change Staff, which the reader doesn’t follow, so its notes go in a third track, “Piano”, for the player’s own split (split by voice instead, tenor and bass to the left, the left hand missed keys on 11 % of the frames, against 6.5 %). Both WTC files checked (I No. 12, II No. 6’s prelude) miss no more keys than with the old split, and their forearms stay at least 6.8 cm apart.
• pianist/test/wrist.mjs takes song.mid@start (or START), waits for a long piece’s fingering, and checks that the forearms never pass through each other (closer than 4 cm on more than 1 % of the frames), and now plays the Goldberg from Variation 5 by default. 032_pianist.html?shot&fingerwait=N waits N × 50 ms for the fingering (default 400: 20 s, short of the complete Goldberg’s).

24 Oct 2026

Experiment #032 - Pianist: the wrist stays within what a wrist can bend (shared by #038, #045 and #047)

• Before, the hand’s turn came only from fitting the fingertips to the keys. Its yaw was clamped to ±23° but never measured against the forearm, so with the elbow out to the side or brought forward round the body, the hand met the arm at an impossible angle. Measured on the skeleton: in Prelude1 the bend toward the little finger was 46–70° at the 99th percentile and up to 75°, and the wrist was bent back up to 70°. A real wrist manages about 30° and 70°.
• New WRIST_RANGE in pianist/hand-driver.js: 25° toward the little finger, 12° toward the thumb, 35° back and 45° down, from the character’s own rest pose (FORE_DEV and FORE_ELEV, measured by measureHand). Past that, the hand turns with the forearm, about the centre of its fingertips so they stay over their keys, and the wrist rises about the knuckles.
• Applied twice. While baking the hand’s placement, the forearm comes from the driver’s own arm IK, so the fit and the slide are planned for the yaw the hand will actually play with. Each frame, the elbow the rig placed is used (setElbow). When the arm has moved on too far for that (a leap, or the elbow leading a stroke), the rig places the hand again for this frame’s elbow (wristExcess, redo) and the arm follows it, keeping its swing. The driver and the rig now share one hand frame (handAxes).
• After the change, in Prelude1, Chopin’s prelude, The Entertainer and Rush E: about 25° at the 99th percentile and never more than 10° past the range. Fingertips short of their keys are no more frequent than before. The Entertainer’s 16 and 31 % were already there: its fingering asks for stretches this hand cannot make.
• Test: node pianist/test/wrist.mjs [song.mid …], headless, measured from the bones. It fails on more than 1 % of frames 3° past the range, any frame 10° past, or more key misses than a per-piece budget. Each piece has a time limit (PIECE_TIMEOUT, default 10 min). SECONDS sets how much of each piece and FPS the frame rate (30). A hand whose arm degenerates keeps its fitted placement (no NaN).

22 Oct 2026

Experiment #047 - Wardrobe (notes)

• The #032 pianist in a round photo studio, for trying hair, clothes and colour combinations quickly. Each rail (outfit, hair, dress or top, fabric, slits, shoes) is ‹ › or the arrow keys. A slot builds one piece at a time and skips to the latest click. Neighbouring assets are fetched during idle time.
• Colours apply as the picker moves, with no rebuild: hair (the eyebrows follow it), dress, a separate skirt colour under the community tops, the qipao trim, shoes, skin tone and backdrop. In the stylized and cel looks the colour goes through the new look.recolor() in pianist/look.js.
• Poses: arms down, hands clasped (rig.state.standArms in pianist/rig.js) and at the piano. Lights: studio, stage and daylight. Framings: full, half, face and shoes, plus a turntable. Shuffle all, shuffle colours, and a lookbook of saved looks with thumbnails (localStorage). The URL keeps the state.
• Path-traced stills through #045’s pathtrace/still.js, always in the realistic look. The tracers’ environment map is now re-baked when the page’s environment intensity changes. dress.setTrim() was added to pianist/dress.js.
• Concert outfits from research into what women soloists wear (concert black, sleeveless floor-length gowns, jewel tones, fishtail and Grecian silhouettes, a jumpsuit, black separates with wide trousers, short sequinned dresses with heels, low closed shoes, a white gown worn barefoot; sources in wardrobe/README.md). There are 12, grouped first in the Outfit rail and now the default.
• 27 new MakeHuman community assets (CC0 / CC-BY): fishtail, Grecian, column and halter gowns, strapless and one-shoulder dresses, a halter jumpsuit, camisole and bodice tops, palazzo, slim and wool trousers, a long full skirt, ballet flats, Mary Janes, T-bars, and six hairstyles. The community catalogue is offline, so they come from the asset packs through the new pianist/build/import-packs.mjs, which keeps each asset’s node id as its key. build-character.mjs gives trousers and skirts a new bottom kind and now classifies a gown as a dress whatever its title says. The Bottom rail pairs a bottom with any top; #032 and #045 leave bottoms out of their menus.
• Skins and face shapes: the Skin rail switches between MakeHuman’s young European, East Asian and African skin textures (CC0, instant). East Asian face and African face sliders blend MakeHuman’s ethnic macro targets live, on the head only (fading through the neck) and with the targets’ whole-head shift and scale taken out. Every eye, brow, lash, hair and garment part is refitted on the morphed body at build time and stored as a sparse delta, so hair follows the skull. New: setFace() / setSkin() in pianist/character.js, rig.measureEyes() in pianist/rig.js; build-character.mjs writes the morphs and skin_asian.jpg / skin_african.jpg (character.bin 2.2 → 2.9 MB).
• Nine more fabrics in pianist/fabric.js, shared with #032 and #045 (14 in all): silk charmeuse (anisotropic), chiffon, crepe, shot taffeta (iridescent ribs), damask (satin medallions on a matte ground, as a roughness map), lamé, glitter, beaded, leather. Each is material values plus, where the surface shows, a procedural tile made in the page. Concert outfits use them, plus two new outfits (an ivory damask gown and a leather jumpsuit).
• Lace and pleats (16 fabrics). Lace: corded rosettes on a hexagonal net over a darker lining; on community garments its colour tile is multiplied into the garment’s own texture. Pleats: vertical knife pleats, a normal map on our dresses (u runs round the body) and, on community garments, a shader that pleats by the angle round her body axis in the rest pose, so they hang straight whatever the UVs. A preview of all 16 on our gown is in wardrobe/README.md.
• Lace is now illusion lace: the lace over a lining in her skin tone, so it reads as see-through (a shader mixes dress and lining colour by the lace’s coverage; the tulle between the motifs tints the skin under it). A Lace lining rail picks another lining. Path-traced stills use a CPU bake of the same mix (traceMapOf(), read by pathtrace/snapshot.js). Eight new concert outfits in lace and pleats. Fabric state moved from userData to a WeakMap, so cloning a material (the looks, the tracers’ snapshots) no longer serialises its textures.
• Tests: node wardrobe/test/smoke.mjs (every rail, look, pose, light and framing headless; FULL=1, PT=1). OUTFITS=1. On SwiftShader a dress takes 0.5–0.7 s a click, hair 0.2–0.3 s, colours and fabric ≤ 16 ms.

21 Oct 2026

Experiment #046 - Score fingering (notes)

• Take or open a photograph or multi-page PDF of printed music. A dedicated worker runs ScoreShift’s staff-first OMR locally, keeps the normalized page and every notehead’s coordinates, assembles parts / measures / voices, and sends the selected part to the string fingering planner. ScoreShift is imported straight from the sibling random-sketches checkout (../random-sketches/vibes/scoreshift/, same origin on GitHub Pages) instead of a vendored copy; the key-signature and PDF fixes #046 needed (opening naturals as a cancellation or a new key, analyzeAfter for the previous page’s key, keyless sections, cancellable PDF renders) moved upstream with their tests, alongside its IMSLP fixes (fragmented sharps/flats no longer become hollow heads, a touching time-signature loop no longer cuts a key short). Run locally by serving the folder that holds both checkouts.
• Try viola PDF now opens the supplied 17-page, 720×936 pt Sibelius Symphony No. 2 viola part. Its first page exercises 9 alto-clef staves, 174 detected notes and 48 assembled measures; auto viola removes the 84 range rejections produced by forcing violin. Page 2 carries page 1’s D-major key only into staves with no conflicting printed key, restoring F♯ and the requested E♯–F♯–A fingering 2–3–1 instead of the false enharmonic 3–3–1. The score adapter also retains printed diatonic spelling, distinguishing D♭ from enharmonic C♯ so bar 5’s A–B♭–D♭–D♭–C stays in first position as 1–2–4–4–3.
• Auto (the default) chooses violin or viola from an unambiguous filename, the selected part’s clefs and viola-only low range, carrying strong evidence for that part across PDF pages; manual Violin/Viola overrides remain. Standard / beginner / expressive / fewest-shifts style, tempo-sensitive shift costs, and a part selector cover ensemble scores. Up to four exact k-best Viterbi paths under the recommended held-safe constraints are ranked for the whole passage; alternatives are filtered to distinct string, finger or position choices.
• The photographed page gets larger, slightly softened, uncircled handwritten graphite finger numbers above each complete note, including upward stems and beams; chord fingers form a vertical stack. The colored string-change and hand-position badges are hidden by default; string / position details shows them beside their noteheads. Chord notes that do not fit one player’s strings/hand frame keep ordinary independent divisi fingerings; the explanation remains in status and the note popup rather than adding warning circles. Pick a complete path, or tap one note to lock its string and re-plan around that decision. The same popup moves a recognized pitch by staff step, changes its accidental or removes a false note. recognition shows ScoreShift’s pitch reading; Annotated PNG flattens the selected fingering onto the normalized photograph.
• PDFs get lazy thumbnails and stable per-page records: navigation restores each page’s recognized model, pitch corrections, string locks, selected part, selected alternative and keyboard position without recognizing it again. A title/text page is shown unchanged; recognition failures are shown separately and block whole-file export. The 100 MB / 100-page input limit, two-page LRU raster/model cache, 64 MB encoded-export limit and one-at-a-time temporary export pages bound memory while compact corrections survive eviction. Annotated PNG names the current PDF page; Annotated PDF snapshots its settings, preserves page order and point sizes, flattens each cached selected fingering, recognizes unopened music pages with defaults, and JPEG-preserves pages with no staves.
• strings/fingering.js now exposes immutable solveFingering() and suggestFingerings() APIs while retaining fingerLine() / fingerHeld() for #033, #034, #037 and #038. Inputs and custom profiles are validated, every accepted cost is finite, rejected notes and unresolved held-string collisions have diagnostics, repeated solves leave no stale fields or generated locks, and held voices use a bounded constraint search. Ranked alternatives use an exact heap-merged k-best recurrence with compact backpointers; a 1,000-event / 24-path stress passage takes about 1.2 s here.
• Tests: 49 fingering contracts, including stable 1–2–3–2–1–3–2 continuity, the page-2 viola 2–3–1 chromatic approach and independent divisi fallbacks for over-capacity chords; score-fingering/test/adapter.test.js, document.test.js and integration.test.js cover source mapping, ties, correction-stable IDs, incompatible-lock warnings, isolated page state, PDF writer order/sizes and local OMR fixtures (the 4-staff, 96-note, 24-bar minuet plus a 30-note / 8-grace oracle checked through 260 bpm). browser.mjs checks the generated photo’s four paths, the 17-page Sibelius sample’s auto/manual instrument behavior, manual locking, recognition correction, deletion, keyboard focus/scrolling, stale and overlapping loads, worker failure/preemption, operation-owned busy state and 320 px / DPR 2 layout; its PDF path checks a title page, three thumbnails, unopened-page recognition, cached corrections/locks/alternatives, page-aware PNG and a re-readable three-page Letter PDF. Textbook quality stays 29/33 exact / 96.2% note agreement and 4/5 scales / 96.4%; TNUA is 82.1% string, 67.6% finger, 64.9% joint and 74.8% soft over 10,870 notes × 10 annotations.

Expression for pieces no one has recorded (strings/expression-model.js), and #038 from the score

• A model predicts from the score how a violinist would time a piece and shape its dynamics. It is fitted by violinist/tools/fit_expression.js on the 62 recordings in violinist/performances (37 pieces, mostly Bach’s Sonatas and Partitas) into violinist/expression-model.json (2 KB). Each onset gets 42 features read from the score alone: its length and place in the bar, where it sits in its phrase, its height against the line around it, slurs and leaps, fermatas and ornaments, the approach to an arrival (a note twice as long as the run before it), and the last bars of the piece and of each repeated section. Two ridge regressions give the onset’s local tempo (over at least a beat, as a log ratio to the player’s median) and its dynamic (the lane the recordings play with on the Source model). A learned swell shapes long notes, and the score’s written dynamics and hairpins set the level underneath.
• Checked on pieces it was not fitted on (each piece left out in turn, with all its recordings): tempo r 0.25 per onset, 0.31 for the tempo curve; dynamics r 0.39. As a ceiling, two violinists playing the same piece agree at tempo r 0.39 and dynamics r 0.36, so the dynamics are as close to a player as another player is. Through #034’s Source model, on 9 held-out recordings, the rendered notes’ levels follow the player at mean r 0.20 against 0.14 for the rules used before (higher on 6, lower on 2). Fitted to the least error, the predictions are an average player’s, flatter than any real one, so the pages play the tempo at 1.5 × and the dynamics at 2 ×; the local tempo then moves about ±5–10 % with a final ritardando, and the piece keeps its length.
• Performer: modelled from recordings is the new default in #037 and #038 for every piece; the accompaniment follows its tempo. As written (?performer=none) and the recorded performers are unchanged. With it, renderPerformance leaves out its own phrase arches and swells (expression.phrased) but keeps the intonation and vibrato rules.
• #038 Duo now plays from the score too, as #037 does: the violin from the Mutopia LilyPond source, the pianist the score’s piano (or orchestral) staves, the pedal still from the MIDI file. Notes: score / MIDI file (?src=midi). The bowing for score notes moved to bowScore() in violinist/score-play.js, shared by both pages.
• The LilyPond reader keeps fermatas (art.fermata), and planned articulation keeps grace notes, ornaments and fermatas on the note (they were dropped when the bowing was planned).
• Tests: strings/test/expression_model.test.js (a score played by the model; the held-out figures). node violinist/tools/fit_expression.js --agree prints the two-player agreement, --render the comparison through the Source model, --write refits.

Experiment #037 - Violinist: the score is the input

• #037 now plays each piece from its Mutopia LilyPond source (violinist/scores, read by strings/lilypond.js) instead of the MIDI file LilyPond made from it. violinist/score-play.js turns the score into the solo and the accompaniment; Notes: score / MIDI file switches back (?notes=midi).
• From the score: its slurs, strokes and bow signs on the notes themselves; its dynamics and hairpins (\<, \>, \cresc, ending on the next dynamic or a step away at \!) as each note’s loudness, where 43 of the 48 MIDI files give every violin note the same velocity; grace notes realised as written (appoggiaturas on the beat for half the note, two thirds when dotted; \grace and \acciaccatura about 70 ms on the beat; \afterGrace at the end of the note before), 216 grace notes over the 48 pieces; every other staff of the movement as the accompaniment with its GM instrument, and \set Staff.midiInstrument changes inside a part (the Tchaikovsky strings’ pizzicato).
• 99.55 % of the 47,324 MIDI violin notes pair with a score note (the same pitch at the same place). A recorded performer’s timing and pitch curves, and the Herrmann / Auer bowings, are both aligned to the MIDI’s notes, so they are applied there and carried over to the paired score notes; the rest (grace notes, notes LilyPond’s MIDI dropped) follow the recording’s tempo map.
• With the GM soundfont violin, the dynamics are now audible: the MIDI path played everything at the file’s velocity (often 127) and ignored its CC7 lane, so quiet passages may sound quieter than before.
• Tests: strings/test/score_play.test.js (hairpins, grace notes, all 48 sources). The 48-file LilyPond check is unchanged (99.5 % of notes).

Experiment #045 - Path-traced Pianist (notes)

• #032 moved to three.js r185, with two GPU path tracers rendering the frame the page draws: three-gpu-pathtracer 0.0.27 (WebGPU, and its deprecated WebGL tracer) and MuTracer 0.1.3 (WebGPU, WebGL 2), vendored as browser modules by pathtrace/tools/vendor.mjs
• Path trace menu: a still of the current frame by one tracer or all of them, split against the raster picture, with setup, first-sample, tracing and denoising times; OIDN or JNLM denoising; the page’s bloom, grain and vignette run on the traced picture (pathtrace/post.js). Render → Renderer exports a video with every frame traced; Camera → Inside the piano leaves out the director’s shots that look into the strings (the slowest part to trace)
• pathtrace/snapshot.js freezes the stage for both tracers: skinning on the CPU, instancing expanded, shader patches dropped, the fallboard mirror as lacquer, the strand hair as camera-facing ribbons (bakeStrands() in pianist/hair.js), the page’s falloff-free lights as inverse-square ones, the reflection panels ray cast into an equirectangular map
• MuTracer first rendered much darker, with white hair. Three causes in MuTracer 0.1.3, worked around here: soft spots never reach full strength (the stage’s peaked at 0.4–4 %), the environment map is read upside down (flipY ignored), and the three.js converter drops vertex colours. Test scene pathtrace/test/mini.html: spot 13 → 65 against three-gpu-pathtracer’s 77, sky from above 1 → 183 against 189
• Open: MuTracer’s WebGL 2 fallback renders the full stage blank on SwiftShader; no GPU timings recorded yet
• Tests: node pathtrace/test/stills.mjs, node pathtrace/test/video.mjs, pathtrace/test/mini.html?test=…

Songs: ornaments in the engraved Bach files (notes)

• The 15 Well-Tempered Clavier files and the Goldberg Variations came from Mutopia’s LilyPond MIDI, which plays no trills, mordents or turns and turns grace notes into 55 ms blips before the beat. midis/tools/bach_ornaments.mjs reads the LilyPond sources, lines them up with each file note by note (longest common subsequence on onset order and pitch: 99.6–100 % of score notes found, 92 % in BWV 857’s prelude, whose doubled unisons LilyPond merges), and writes the ornaments out after Bach’s 1720 table: trills from the upper note (from the main note when the upper note was just played), about 12 notes a second fitted to the beat, long trills starting slower; short trills, mordents, turns, trills with turn or mordent endings; appoggiaturas on the beat taking half the note (two thirds when dotted); grace notes on the beat; after-graces at the end of the note. Auxiliary notes are diatonic, chosen from the pitches heard in the same and neighbouring bars. Dynamics and timing of every other note are unchanged.
• The Well-Tempered files first gained 121 ornaments and 18 grace-note groups, the Goldberg 185 and 113 (Aria counted twice). Their note counts now come within 5 % of the pianists’ in MAESTRO.
• strings/lilypond.js keeps ornament signs on the note (art.orn), keeps grace notes with { graces: true }, and reads \scaleDurations #'(4 . 6) (Goldberg Var. 26’s 18/16 hand was too slow). Its tests and the 48-file Mutopia check are unchanged.
• Two more sources of ornaments for the Well-Tempered Clavier: the Bach-Gesellschaft edition as encoded by CCARH (humdrum-tools/bach-wtc, read by midis/tools/kern.mjs, a small Humdrum **kern reader; 99.6 % of its notes line up with Mutopia’s), which adds 22 signs Mutopia lacks (14 in the B minor prelude) and gives every trill and mordent its interval; and a subject rule for the fugues: the subject is found (the first voice until another voice brings in its first four notes), its entries are found by rhythm and intervals (tonal answers allowed), and a sign on one entry is played on all of them, 20 more (WTC II No. 1’s mordent on all 14 entries, WTC I No. 24’s cadential trill). WTC total now 164 ornaments. Still fewer than the pianists play in WTC I Nos. 15, 17 and 20 and WTC II No. 1, where they add their own
• Trills at the main cadences, where players add one though no edition prints it: the final cadence of every prelude and fugue, and perfect authentic cadences on a downbeat (bass leaping from the dominant, top line arriving on the root) at least 2 bars from Bach’s own signs and 8 from each other; the trill goes on the top line’s note over the dominant, degree 2 or 7, that resolves to the tonic. 21 trills, 14 of them final (none where the close is already ornamented, runs through, or sits over a tonic pedal); WTC total 184. --no-cadences turns them off
• strings/lilypond.js notes also carry their voice (art.vk), so the voices of a staff can be told apart
• 14 MAESTRO v3.0.0 performances of the same preludes and fugues added beside them in the song list, for comparison (CC BY-NC-SA 4.0).

20 Oct 2026

YouTube Shorts for #032 Pianist (notes)

• Render → Format → YouTube Short: 1080×1920, a clip of up to 3 minutes with its own camera plan (only spots framed for a tall screen, the hands first, cuts every 3–6 s on phrase starts, ending on the opening framing so the loop is seamless), the title and intertitles inside YouTube’s safe area, fades so the loop doesn’t click, and a ready description (-short.txt)
• A clip strip to choose it: energy, sections and phrases of the piece, edges that snap to phrase starts and ends, 15–90 s / whole-piece chips, three suggested clips with reasons (pianist/shorts.js: a strong first 3 s, the loudest passage, a breath at the end), and a looped 9:16 preview with the safe area
• Layout: one picture, or stacked: her above and her hands below (split 50 / 50, 60 / 40, 40 / 60 or 2/3 / 1/3), each with its own camera, never cutting together; or a lesson view below, #030’s falling notes in each hand’s colour with finger numbers, lit keys and sparks (pianist/lesson.js)
• Tests: bun test pianist/test (6 new); contact sheet node pianist/test/shorts_sheet.mjs

Experiment #042 - AugmentedNet

• AugmentedNet (Nápoles López, Gotham & Fujinaga, ISMIR 2021; MIT), a trained Roman numeral network, exported from Keras to ONNX (460 KB, augnet/tools/export_augmentednet.py) and run on onnxruntime-web 1.30 in a worker; its input encoding and decoding ported from inference.py to JS (augnet/augnet.js), with the music21 tables it needs precomputed, so no Python or music21 in the browser
• From MIDI: onsets snapped to sixteenths and triplets as music21 does, notes spelled in two passes (music21’s spelling, then each note nearest the local key the network found)
• Keys, chords and numerals in lanes, its frame-by-frame certainty, the voicing it predicts (bass / tenor / alto / soprano), and #041’s analysis underneath with a strip where the two agree; 43 ms in the network for BWV 846, 84 ms for Mozart K. 545 i
• On the 7 BPS-FH movements it held out: key 80.8, root 84.0, quality 80.1, inversion 74.2 (published 85.0 / 84.4 / 79.0 / 73.4); #041 on the same: 75.3 / 82.6 / 79.6 / 75.4 (notes)
• Tests: node --test augnet/test/augnet.test.js (4: the encoding matches the Python pipeline frame for frame on a Bach chorale, the decoding gives its 61 chords, onnxruntime-web matches Keras); node harmony/test/bps.js --model augnet

Experiment #041 - Harmony

• Chord and key analysis of a MIDI file as it loads and plays: chord symbols (G7/B, spelled from the key), Roman numerals (V65, V7/V, N6, Ger+6, mixture), tonic / predominant / dominant, cadences (PAC, IAC, HC, DC, PC) and local keys, on lanes over a piano roll whose notes are coloured by their role in the chord (root, third, fifth, seventh, not in the chord); a “now” card and a circle of fifths with the root’s path
• Engine harmony/harmony.js (in a worker): Pardo & Birmingham’s template scoring inside a semi-Markov Viterbi that segments and labels together, with the segment evidence Masada & Bunescu found matters (bass, metre, passing and neighbour notes, pedal), a key prior and root-motion preferences; keys by profile correlation (Aarden–Essen, Temperley–Kostka–Payne, Krumhansl–Kessler, Bellman–Budge, Sapp) and Viterbi, then a second pass from the chords (dominants that resolve)
• On BPS-FH (first movements of Beethoven’s 32 sonatas, human analyses; weights tuned on the odd sonatas), held-out even sonatas: key 76.5 %, root 80.3 %, quality 75.7 %, inversion 70.3 %, full Roman numeral 46.3 % of the time (AugmentedNet 85.0 / 84.4 / 79.0 / 73.4 / 45.4, ChordGNN 82.0 / 82.5 / 74.1 / 76.5 / 49.1 on the BPS test split); 0.4 s for BWV 846, 1 s for the Polonaise Op. 53
• Keys → AugmentedNet takes the local keys from #042’s network (on onnxruntime-web, ~150 ms) and keeps #041’s own chords: on the 7 BPS test movements key 75.3 → 80.6 %, Roman numerals 52.0 → 55.6 % (AugmentedNet alone 52.6); on 353 Bach chorales key 59.6 → 71.9 %, Roman numerals 37.7 → 46.4 %
• Non-chord tones: passing and neighbour notes, suspensions (held or restruck, with their figure: 4–3, 9–8, 7–6, 2–3) and appoggiaturas are marked from the notes and excused only when they are not in the chord being scored; marked on the roll (s, a, ap, p, n). Inside the analysts’ suspensions the root is right 54.6 % of the time instead of 48.1 % (BPS-FH, held out) and 75.2 instead of 73.3 % (371 Bach chorales from When in Rome, never used for tuning; root + quality 66.5 instead of 61.8 %)
• Triads, classical (sevenths) or pop / jazz vocabularies (sus, 6, add9, 9ths, 5); an Instant lane shows what a DAW-style chord display names onset by onset; MIDI in names the chord you hold; hover for alternatives and certainty; ← / → step by chord; export a MIREX .lab (Harte syntax) or a chord chart by bar
• Research: the theory (qualities, inversions, non-chord tones, keys, numerals, functions, cadences, spelling), the algorithms (Krumhansl–Schmuckler and later profiles, Pardo & Birmingham, Temperley’s preference rules, semi-CRFs, AugmentedNet, ChordGNN, BACHI), and how Logic, Cubase, Studio One, Dorico, MuseScore, Scaler, Ableton, tonal.js and music21 do it
• Tests: node --test harmony/test/harmony.test.js (11); node harmony/test/bps.js [--split test] [--verbose] [--tune] scores BPS-FH (GPL-3.0, clone it into the git-ignored harmony/data/); --corpus wir/chorales scores When in Rome exports (python harmony/test/wir_export.py, CC BY-SA)

Experiment #040 - Rhythm

• Beats, tempo, meter, downbeats, swing and a step grid of the rhythm from the microphone (or a demo drum loop, or an audio file), live, in an AudioWorklet
• Engine rhythm/: SuperFlux onsets on a log filterbank, a comb-filtered autocorrelation tempogram as a tempo prior, and the bar-pointer beat HMM of Krebs, Böck & Widmer (madmom’s DBN) run as a causal forward filter that predicts the next beat; a beat-level stage halves an eighth-note pulse when every other one is accented; meter (3 vs 4) from bar repetition, downbeats from the kick, onsets and harmonic change, duple / triple feel and the swing ratio from the onsets across the beat
• Scored live (128-sample blocks) on 34 songs (18 drum styles and 16 repo MIDIs, OxiSynth), clean and through a laptop-microphone model: beat F 88.3, AMLt 87.9, tempo right 78 % of the time (98 % allowing double / half), meter 33/34, 0.4–1.4 s to lock on drums; beats are reported 4–6 ms before they happen, at 1.5–3 % of a core
• Readout to 0.1 BPM, beat lamps with the downbeat, a timeline of onsets in three bands with detected and predicted beats, the tempogram, the rhythm grid; tap tempo (T), beat level Auto / Pulse / Half, tempo range, a metronome on the detected beat (M) compensated for input and output latency; settings in the address bar
• Research: onset detection, tempo, beat trackers (agents, DP, bar-pointer HMMs, particle filters, neural activations), meter and downbeats, swing, and what madmom, BeatNet, Aubio, librosa and Essentia do
• Tests: node --test rhythm/test/rhythm.test.js (10); node rhythm/test/eval.js scores the corpus; node rhythm/test/browser.mjs runs the page in Chromium on the demo loop and a fake microphone

Experiment #039 - Tuner

• A chromatic and string tuner on the Web Audio input: the microphone (echo cancellation, noise suppression and auto gain off, device picker) or a built-in test tone in Source (bowed, plucked or sine, detune in cents, vibrato; ?source=test)
• Detector tuner/pitch.js, in an AudioWorklet (tuner/tuner-worklet.js): McLeod’s NSDF by FFT autocorrelation (peaks compared after interpolation, so high notes with a weak fundamental are not an octave low), refined on the peaks at 2, 4, 8 … periods, then by a joint least-squares fit of DC and the first 8 harmonics; the window is sized to the note and trimmed to the onset after silence, with a level gate, a clarity gate, a median of 3 and smoothing in cents that jumps on note changes
• Accuracy on synthetic tones C1–C8: sine within 0.01 ¢, sawtooth within 0.15 ¢; first reading 5–10 ms after the onset above A3, 25 ms on E2, 50 ms on E1 (two periods); a reading every 5 ms (10 ms Precise); about 10 % of one core on low notes. Through Chromium’s fake microphone a 110.255 Hz file read as A2 +4.0 ¢
• Fast / Balanced / Precise response (8 / 16 / 32 periods, 20–100 ms windows), needle gauge with the in-tune band (±0.5–5 ¢), a three-row strobe (1st, 2nd, 4th harmonic) that stands still in tune, 10 s history of cents
• Instruments: chromatic, guitar, bass (4 and 5), ukulele, violin, viola, cello, double bass, mandolin, with string buttons that keep the last offset measured on each string (+6.5 ¢ / −12 ¢, taken once the note has been steady for 200 ms, so you can check every string at a glance, and a sparkline of each string over the last 8 s you played it; C clears; click to lock one) and pure fifths from A for the bowed strings; A4 400–480 Hz, B♭ / E♭ / F transposition, sharps or flats, a reference tone at the target (R) and a concert A button always on screen (A, at the A4 setting), a green smiley when in tune (Display → smiley, ?smile=1), hold (Space); settings in the address bar
• Installable and offline (PWA): 039_tuner.webmanifest and icons (tuner/icon.svg → PNGs by node tuner/tools/icons.mjs); the shared sw.js is handed the tuner’s files on the first visit (~120 KB), so it opens and tunes with no network; an Install section in the menu (an Install app button where the browser offers one, otherwise the steps for the browser in use, and the rest under Other browsers); the panel starts minimised on phones and tablets (?panel=1 opens it), with a two-column layout for landscape phones, and the stage keeps clear of notches when installed. ?sw=0 skips the service worker
• Strings only claim a reading within a semitone; between strings it reads as the chromatic note (an A3 on a guitar was “B3 −193 ¢”)
• Tests: node --test tuner/test/pitch.test.js (10)

19 Oct 2026

Liner notes for #032 Pianist

• X-Ray-style notes on a piece, written in Markdown and tied to bars (m22, m22.3), placed with the MIDI file’s own time signatures and tempo map: captions while it plays (history, analysis, trivia, things to listen for), a notes panel (I: now / about / all notes / people and terms), sections on the seek bar, and a card when hovering a song
• First piece: Bach’s Prelude in C, BWV 846 (Prelude1.mid), 6 sections, 18 notes with sources, 5 cards (Bach, the Well-Tempered Clavier, pedal point, diminished seventh, the Bach–Gounod Ave Maria)
• Intertitles: a few short lines marked for the screen (screen: Paris, 1853 | Gounod will sing… on a note, or a # Intertitles part), in cinematic type (Cormorant Garamond / Cinzel / Montserrat, letter-spaced kickers), faded in and out by the song clock with room to read; everything else stays in the panel. Or every note as a caption, or nothing
• Intertitle look: black and white by default (all white, black on light, warm gold), placed automatically away from the pianist or in a chosen corner, headings optional; the song title as an intertitle or the classic title card (off by default)
• Second piece: Beethoven’s Moonlight Sonata, III. Presto agitato (midis/mond_3.mid), 8 sections, 24 notes, 11 intertitles (7 stories, 4 about the music), 6 new cards; play: maps score bars to a file that writes out the exposition repeat (m21' is the second time) and spreads the cadenza over four bars
• Video export: the same burned into the picture, and saved next to the MP4 as two YouTube subtitle tracks (the intertitles, and every note) and chapters, timed to the video
• Stories and analysis apart: history and trivia float (aside: / ## ~), placed by a seeded generator, one every 20–30 s through the opening, then spread out; lines about the music are pinned and come when the passage they describe ends (m188-m190), in their own style (spaced Montserrat); the song title is off by default
• #038 Duo too: the same intertitles, panel and seek-bar sections (Notes menu), with notes for Gounod’s Ave Maria (violinist/midis/gounod_ave_maria.mid): 6 sections, 19 notes, 10 intertitles (6 stories, the extra bar, 3 about the music), and a Gounod card
• Tests: node --test liner/test/liner.test.js (15)

Pedal and transport fixes

• OxiSynth itself sustains correctly (CC64 ≥ 64, per channel); the players lost pedal changes. Late CCs are now always sent (a skipped one left the pedal stuck), a seek restores each channel’s pedals, modulation and expression as the song has them there, and unloading a song lifts its pedals. In playback_controls.js / midi_clock_worklet.js (#014–#024) and soundfont-engine’s MidiScheduler (#025–#038)
• #023: resume after pause continued from 0 (every play reloaded the song into the clock worklet); it now keeps its position, and a seek while paused sticks
• Tests: node --test test/playback.test.js soundfont-engine/test/scheduler.test.js (11)

18 Oct 2026

Experiment #038 - Duo

• The #037 violinist and the #032 VRoid pianist in her concert dress on one stage: he plays the violin part, she plays everything else on the #030 grand (fingered by fingering/, pedal from CC64)
• Violin sonatas (Bach, Handel, Mozart, Beethoven Op. 30 No. 2 and the Kreutzer), Gounod’s Ave Maria, and three concertos with the orchestra on the piano
• An auto camera that plans the piece’s takes in advance and follows the busier player, plus ten fixed views; menus for his model and outfit, her dress, the room and the ink outline (notes)

Experiment #037 - Violinist

• An anime-style violinist (VRoid CC0 VRMs, cel shaded, Nodame Cantabile-like stage lighting) plays the solo part of a violin MIDI; the rest plays as accompaniment
• 48 Mutopia MIDIs: all of Bach’s Sonatas & Partitas BWV 1001–1006, sonatas (Bach, Handel, Mozart, Beethoven Kreutzer), concertos (Bach, Vivaldi, Beethoven, Tchaikovsky), Paganini caprices (list and credits)
• Bowing from the score: each MIDI’s Mutopia LilyPond source is in violinist/scores (243 files, 1.2 MB), and #034 / #037 / #038 put its slurs and marks on the MIDI’s notes (99.5 % of the violin notes found; 20 % of note pairs slurred, 0 in the MIDI files; the editions carry almost no up / down bow signs); Bowing: score (default) / inferred / fixed rule in #037 and #038; for Bach’s Sonatas and Partitas also the bowings of two printed editions on IMSLP (Herrmann 1900, Auer 1917) read from their scans by optical music recognition with ScoreShift and aligned onto the MIDI notes (violinist/editions: 85–91 % of the notes found, up to 464 bow signs), and ⊓ / V signs now set the bow direction
• Rig violinist/rig.js: the #033 fingering, left hand and bow drive the body (violin hold from motion-capture angles, two-bone arm and leg IK, per-finger IK, sway, breath, head on the chin rest) (notes)
• Research: biomechanics, character and look

17 Oct 2026

Experiment #036 - Transcription models on ONNX Runtime Web

• A playground for audio → MIDI models exported to ONNX and run with onnxruntime-web 1.30 (WASM or WebGPU) in a module worker, no TensorFlow; switch models (keys 1 / 2) on the same source and compare them in a table (notes, onset F1, onset+offset F1, time, × real time)
• Basic Pitch (Spotify, ICASSP 2022, 230 KB, Apache-2.0): transcription/onnx/basic-pitch.js ports the windowing, note creation and pitch bends (activations match the tfjs package to 8e-5, identical notes); threshold sliders re-decode instantly; Live mode streams the mic (≈1.7 s behind)
• Transkun v2 (Yan & Duan, ISMIR 2024, MIT): piano notes with velocity plus sustain and soft pedal. The network is exported by transcription/onnx/tools/export_transkun.py as two graphs (55 MB: interval scores + velocity / onset / offset head); transcription/onnx/transkun.js does the rest in JS: the 6-window FFT / mel front end, the semi-CRF Viterbi, and the 16 s segment stitching. Same notes as the Python package (61/61 within 5 ms, velocity ±1)
• Chopin’s Minute Waltz (45 s, soundfont render): Transkun F1 98.7 (with offsets 67.8) vs Basic Pitch 78.5 (17.1); Transkun runs at 1.9× real time on 8 WASM threads, Basic Pitch at ≈100×
• coi-sw.js: a service worker scoped to this page adds COOP / COEP, so WASM gets threads on GitHub Pages and serve.sh (?coi=0 turns it off)
• Sources: audio files, microphone recording, Live (Basic Pitch), and test MIDIs rendered with the soundfont (OxiSynth, pedal included) or simple synthetic tones, scored against the true notes (mir_eval-style, ±50 ms onsets); piano roll over Basic Pitch’s activations, pedal lane, plays the audio and the transcription (with pedal) together; MIDI (with CC 64 / 67) and WAV export
• Tests: node --test transcription/onnx/test/*.test.js (15; the 3 end-to-end tests need onnxruntime-web, see the files)

Experiment #035 - Basic Pitch Playground

• Audio → MIDI with Spotify’s Basic Pitch on TensorFlow.js: the 0.9 MB tfjs model from @spotify/basic-pitch 1.0.1 (Apache-2.0, vendored in transcription/vendor/basic-pitch/) in a module worker, WebGL / WebGPU / WASM / CPU backends
• Sources: audio files (open or drop), microphone and another tab’s audio live, and demo MIDIs rendered offline with the soundfont (as written or forced to one GM instrument) and scored against the MIDI
• Live: complete 2 s windows are committed (~1 s behind) and a “peek” re-runs the unfinished window on what has arrived, so notes show ~0.2–0.5 s behind; stopping keeps the take for playback and export
• Frame / onset / contour heatmap under the notes; threshold, min length, tolerance, range, infer-onsets and melodia sliders re-decode instantly; reference MIDI scoring (onset and onset+offset P / R / F1, ±50 ms) with an offset slider
• transcription/basic-pitch.js: the library’s windowing (identical rows, tested), a streaming transcriber, and a linear-time port of the library’s note decoder (its melodia pass is quadratic, 40 s for 3 minutes of audio; output identical, tested)
• Original and resynthesized notes play in sync (split L / R), echo notes while live; MIDI export (optionally one channel per note with pitch bends) and the 22.05 kHz WAV the model heard
• Tests: node --test transcription/test/basic-pitch.test.js (9; the 2 model tests need BP_TFJS, see the file)

Experiment #034 - Violin Sound Lab

• One performance, six engines to compare by ear (keys 1–6 switch while playing, same spot): bowed digital waveguide, Helmholtz source-filter, Magenta’s DDSP violin, VSCO 2 CE samples (CC0), the GM soundfont #033 uses, and a hybrid sampler (research brief)
• Performance renderer strings/sound/performance.js: the #033 fingering and bow strokes become per-string curves at 250 Hz: pitch with late-starting vibrato (none on open strings) and slides on shifts, bow speed with attacks, bow changes and phrase arches, bow force kept inside Schelleng’s playable window, contact point, finger lifts and taps, and loudness for DDSP
• Physical models strings/sound/bowed.js in an AudioWorklet: four waveguide strings with stick-slip friction coupled at the bridge (sympathetic open strings), a modal violin / viola / cello body, a tone EQ fitted to the VSCO recordings (node strings/test/fit_tone.js); real time, plays MIDI keyboard and computer-keyboard notes
• DDSP strings/sound/ddsp.js: the magenta.js violin checkpoint’s decoder runs on TensorFlow.js in a worker, its harmonic + noise synthesizer is reimplemented in JS (same output, 9× faster on CPU); 5 s windows rendered ahead of the playhead, a second pass for double stops
• Expression sliders (vibrato, slides, dynamics, phrasing, bow speed, pressure, contact), the #030 hall reverb, WAV export of each engine; presets (Default / Clean / Expressive / Student / Baroque / Romantic, which also pick the fingering style) and Reset
• Humanization for all five engines after Embertone’s “Intuition” (strings/sound/humanize.js): a few ms of timing, per-note dynamics and bow variance, stopped notes landing a few cents off and settling (live notes too), round-robin samples; Timing / Intonation / Variance sliders and New take
• Hybrid sampler, engine 6 (strings/sound/hybrid.js), after Embertone’s Joshua Bell Violin: University of Iowa non-vibrato samples recorded on each string (violin, viola, cello; pp / mf / ff; 8–12 MB of Opus per instrument), played on the string #033 chose and read along our f0 curve (vibrato, slides, attack error) and loudness curve (layers crossfaded in phase); recorded attacks spliced short for detached notes, bow-change starts for connected ones, no new attack inside slurs (the old sample glides into the new one), looped sustain, the recorded release when the bow leaves; rendered progressively in JS (30–70× real time), live notes too. Bank builder: node strings/test/build_iowa.js --instrument=violin
• Measured body for the physical models (strings/test/fit_body.js, strings/sound/body.js): estimated from the Iowa recordings by comparing every recorded harmonic with the same harmonic of our string (4,500–7,200 harmonics per instrument), played as a 4,096-tap minimum-phase FIR by partitioned FFT convolution; on held-out notes 1.5–2 dB closer to the recordings per harmonic (median) and 4–7 dB at the 90th percentile than the synthetic modal body; Body: Measured (Iowa) / Measured (Philharmonia) / Measured (NBody) / Modal in the Timbre panel; Philharmonia is a second set of real violin, viola and cello bodies fitted from the Philharmonia Orchestra samples (vibrato frames as the sweep; node strings/test/fit_body_philharmonia.js, samples git-ignored, curves committed); NBody is Cook and Trueman’s hammer-measured violin (Princeton, 1998) used as an impulse response, built locally by node strings/test/build_nbody.js (no licence, so git-ignored)
• Automatic bowing after SWAM Violin 3 (Bowing panel): an Expression lane from the track’s CC11 / CC7 hairpins / CC2 moves bow speed, pressure and contact together (Mozart K379 and the Kreutzer write their dynamics only as CC7), interactive bow pressure in the waveguide (slips counted per period, a 25 ms force kick out of double slipping: Helmholtz on 359 of 369 test notes instead of 343), SWAM’s play modes (bow start, keep bow direction, bow lift off / on string, mono cross-string muting, tremolo slow / fast / tempo-synced, harmonics 2 / 3 / 4 with the touch modelled in the waveguide), sordino, rosin and string resonance; live CC11 / CC2 expression, CC1 vibrato and pitch bend from a MIDI keyboard
• Loudness matched across the six engines (strings/test/loudness.mjs): none is normalised per piece any more (GM, Samples and Hybrid were, so a quiet piece came out as loud as a loud one, live notes differed from the render, and on Bach’s Adagio the engines spanned 13 dB); each plays at a fixed level trimmed to the Hybrid at mezzo-forte (K-weighted loudness, ITU-R BS.1770): a test performance within 1.2 dB, four songs within 3–4 dB (piece-dependent: chords, soft short notes). The GM soundfont’s velocity curve (16–19 dB from p to f) is bent to 9–11 dB like the others, and DDSP’s flat 5 dB response expanded to 9; the physical models with a measured body no longer lose 9–13 dB at the top of the E string (level-by-pitch voicing fitted to the Iowa recordings on the same string, strings/test/fit_level.js)
• Articulation planner (strings/articulation.js) and score input (strings/score.js): Open… takes MuseScore .mscz / .mscx directly (3 and 4; on the OpenScore string quartets the same notes, slurs and marks as MuseScore’s own MusicXML export, 99.4–99.9 %), LilyPond .ly as Mutopia publishes it (one file, its -lys.zip or all its .ly files; 99.5 % of the notes of the MIDI files LilyPond made from the same sources) and MusicXML / .mxl (MuseScore, Finale, Sibelius, Dorico) with their slurs, staccato, bowings and ties, a Track menu for the parts (a viola or cello part switches the instrument); MIDI performances are read as SWAM does (overlap slurs, gaps re-bow); score-export MIDIs, where every note touches, get slurs inferred from the music (by step, short, in beat groups). On the Bach Violin Dataset’s editions of BWV 1001–1006 the inference agrees on 68 % of note pairs (slurred F1 50 %), the old fixed rule 60 % (F1 39 %); Articulation: Auto / Inferred / Fixed rule in the Bowing panel (research); stroke types (spiccato bounced off the string, martelé pinched and stopped), repeats and dots under a slur played in one bow with a louré / portato pulse (4–5 / 8–9 dB) or re-bowed (Repeats in slurs, Portato slider), and slides from the plan (a score’s slide marks, a performance’s quiet overlaps, about one slow legato leap in 6 s), kept on one string by the fingering
• Auto bowing (Expression block): inside every note the contact point and bow force keep moving as a violinist’s do (toward the bridge as a long note swells, lighter toward the tip, slow wander), shown on live Bow pressure / Bow position meters; and bow-force brightness, a tilt per string that follows the force, fitted per instrument to the Philharmonia pp → ff brightening (violin 521 cents; our strings had 15–80, the sawtooth-like same tone at every dynamic; node strings/test/fit_tilt.js)
• Performer: Bach’s Sonatas and Partitas played as a real violinist did (Emil Telmányi, John Garner, Ko Donghwi, MinJi Kim: 47 recordings of the Bach Violin Dataset, public domain / CC BY): each note’s timing, the recording’s level as a lane of dynamics for the bow, and each sustained note’s pitch curve (vibrato, intonation) measured from the audio (strings/expression.js, violinist/performances); the bowing still from the score. The lane is corrected by listening (rendered on the Source model and moved by the level difference, three times). Against the recordings, rendered note levels correlate 0.89 (0.21 with our rules at the same timing) and pitch curves differ by 5 cents (14) (notes); all the Bach movements are now in the Song menu
• More performers, aligned here: recordings that come with no alignment are matched to the MIDI’s notes by strings/align.js (chroma and chroma-onset features, subsequence DTW coarse to fine, every way of taking the score’s repeats): Heifetz’s Chaconne (1952), Menuhin’s Preludio (1948), and seven more public-domain / CC0 / CC BY movements from Wikimedia Commons; on the Bach Violin Dataset it agrees with the dataset’s own alignment within 32 ms (median) and finds the repeats each player took 12 / 12, and the note’s pitch sounds where it places the note 91 % of the time (the dataset’s alignment: 83 %). Kreisler and Zimbalist’s Bach double concerto (1915) and a Beethoven concerto finale give the timing, the orchestra following. The dynamics lane is now corrected for the Waveguide model as well as Source (note levels r 0.86 with its own lane, 0.84 with Source’s). Performer menus in #037 (timing and dynamics on the GM violin, the accompaniment following his tempo) and #038 (the physical violin with his pitch too, the piano following; the Bach solo movements, the double concerto and the Beethoven rondo added)
• Performers beyond Bach: a QMUL violinist’s Paganini Caprice 24 (2014, public domain; timing, dynamics and pitch), and with orchestra for their timing Georg Kulenkampff’s Beethoven Romance No. 2 (1932, two 78 sides joined), and two of Vivaldi’s Winter (Air Force Strings 1999, Philip Milman’s orchestra 2022). The aligner now tries leaving out the repeats that Mutopia’s MIDI writes out (the Caprice: 39 → 71 % of notes heard where placed, chance 13 %), and more than four repeats (all, none, each flipped alone). fetch_recordings.js fetches 78 sets side by side and Internet Archive items. #038’s song list gains the Caprices and the Romance
• Fitted to real violinists (strings/test/perf_stats.js, Bach Violin Dataset: four violinists, 401 sustained notes): vibrato on 68 % of notes (players 67 %), 6.45 Hz (6.55), ±13 cents (±13), from the start of the note (was a late bloom at 0.3 s), notes swelling 1.3 dB (1.6); A/B against Ko Donghwi’s Bach (strings/test/ab_recording.js); Source + Philharmonia, closest by ear, is the page’s default
• Tests: node --test strings/test/sound.test.js (23: Helmholtz motion and intonation over the range, stability, vibrato, slides, bowing, DDSP synthesis, real-time speed, humanization, hybrid pitch, sustain, dynamics, legato and speed, interactive bow pressure, expression lane, play modes, harmonics, tremolo, oversampling, measured body, impulse-response body, Philharmonia body; the hybrid tests need ffmpeg)

Research - Violin sound in the browser

• Samples, physical modelling (waveguide, SWAM), DDSP / MIDI-DDSP, RAVE / BRAVE, MIDI-to-audio diffusion (VIOLET, ViolinDiff), Suno-style and open text-to-music models (MusicGen, Stable Audio, ACE-Step, Magenta RealTime 2)

Research - How commercial modelled violins get their sound

• SWAM Solo Strings (pure physical and behavioural modelling: selectable bodies, interactive bow pressure, modelled player habits), Expressive E Soliste, Sound Magic Violin One, and what #034’s physical models lack

Research - Audio → MIDI / MusicXML

• Options for transcribing audio in the browser: Basic Pitch, MuScriptor, the MT3 family, piano models (Transkun, Kong, Aria-AMT), streaming / low-latency piano, monophonic pitch (SwiftF0, PESTO), source separation, performance MIDI → score, and the runtimes (onnxruntime-web, Transformers.js, tfjs, ggml)

16 Oct 2026

Experiment #033 - Violin & Viola Fingering

• Automatic fingering for violin, viola, cello and double bass (string, position and finger for every note, double stops and chords on adjacent strings; the bass with Simandl’s 1-2-4, the 4→3 switch at the crook of the neck and thumb position, research), from exercises, MIDI tracks, typed notes, TNUA files, a MIDI keyboard or clicks on the fingerboard
• Research briefs: what violinists do (positions and finger patterns, shifting, string choice, open strings, Galamian / Flesch / Cooper scale fingerings, double stops, viola vs violin) and published algorithms (Sayegh, Maezawa, Nagata’s HMM, the TNUA dataset and models, guitar tablature, bowing physics, violinist motion)
• Engine strings/fingering.js: time-aware Viterbi over hand-frame states, with distances from the fret formula (so the viola’s longer string makes stretches harder); scale-book templates for long runs; styles beginner / standard / expressive / fewest shifts; hand size; /A string locks
• Tuned on the TNUA dataset (10 violinists): 64.5 % string + finger agreement (62.5 % held out, 57.1 % before tuning; violinists agree with each other 89.8 %); textbook exercises 19/23, three-octave scales 4/5 (notes)
• 3D: procedural violin / viola / cello / double bass (the cello and bass upright on the end pin, the bass with sloping shoulders; arched plates, f-holes, scroll, bridge, strings that press to the fingerboard and blur while sounding) with the bow; left hand on the #029 finger IK with shifts, fingers kept down and vibrato; bow strokes, slurs, retakes and string crossings
• 2D: scrolling string tablature (finger colours, position changes, shifts, red marks where the result differs from the book or the violinist) above a fingerboard map
• Tests: node --test strings/test/fingering.test.js (17), node strings/test/bench.js, node strings/test/tnua.js [--tune]

15 Oct 2026

Experiment #031 - RoboPianist vs #029

• RoboPianist’s MuJoCo piano and two Shadow Hands in the browser (mujoco_wasm, scene and recorded RL actions from their demo, Apache-2.0), side by side: their per-song DroQ policy replayed vs our #029 fingering + motion driving the same hands through a damped-least-squares IK
• Scored with RoboPianist’s own frame F1: Twinkle 89.1 (RL) vs 86.5 (ours), Nocturne 84.0 vs 57.1; ours needs no training and plays any MIDI (notes)
• Shadow Hand span table for the fingering engine (spanTable option), measured with the IK; closed-loop pressing, servo feed-forward, active-set joint limits
• node robopianist/eval.js [--human] [file.mid…] runs the comparison headless

Experiment #029 - Piano Fingering

• Automatic fingering from MIDI, typed notes, PIG files or a live MIDI keyboard, shown as simulated hands on the #028 grand (3D) and as a finger-coloured falling-note roll with hands seen from above (2D)
• Research briefs: what pianists do (treatises from Couperin to Fraser, scale/arpeggio/chromatic tables, octaves, trills, repeated notes, spans, hand size) and published algorithms (Parncutt 1997, Jacobs 2001, Al Kasimi 2007, Balliauw, Nakamura’s HMMs and the PIG dataset, neural models, RoboPianist / FürElise hand motion)
• Engine fingering/fingering.js: hand split by beam search, then per hand a beam search over hand configurations (chords and held keys included), Parncutt/Jacobs costs with time awareness, leaps, repeated-note rotation, pattern consistency; the LH is solved as a mirrored RH. Whole pieces in 0.1–0.4 s
• Motion fingering/motion.js and hands fingering/hands3d.js: fingertip trajectories with lift, approach and thumb-under, a gliding hand anchor, exact coupled-joint finger IK
• Scale and chromatic templates fingering/templates.js: runs that fit a major / minor scale or the chromatic scale are pulled toward the standard fingering (soft cost, so the ergonomic model can still override it)
• Tests: node --test fingering/test/fingering.test.js (43 tests: Parncutt’s published rule points, textbook exercises, all 24 scales and the chromatic scale from every note in both hands, 3-octave scales, hand split, real pieces, PIG metrics); node fingering/test/bench.js scores the exercise suite (15/15 patterns and 48/48 scales exact; the published Parncutt model: 7/15, 14/48), test/tune.js tunes the weights
• PIG evaluation: node fingering/test/pig.js <PianoFingeringDataset folder> [--split] [--verbose] [--tune] reports M_gen / M_high / M_soft / M_rec on the 30-piece test set next to the published HMM results (the dataset is non-commercial and not redistributable: keep it outside the repo or in the git-ignored fingering/data/)
• Model switch (ours / published Parncutt), hand size, dashed red marks where it differs from the textbook, Export PIG

Experiment #030 - Concert Stage

• The #028 grand restaged after Concert Creator’s renders: black stage, one warm spotlight with dust in the beam, a back light rimming the case, and a warm bounce from the open lid onto the plate
• Piano forked to concertstage/concert-grand.js: mirror fallboard (planar reflection of the keys), red key felt, three staggered rows of coiled tuning pins on red felt, agraffes, capo bar, red stringing braid, hammered bronze-gold plate, orange-peel lacquer
• Songs: the full MidiViz player list (playback_controls.js: selections, classical, symphonic, modern, video games) with a filter, or any MIDI file
• Their UI layout: Songs / Camera / Visuals / Render pills and a bottom transport (speed, play, loop, ms clock); chrome fades out while a song plays
• Cameras from their menu: Top, First Person, Orbit, View 1–4, plus the cinematic tour from #028 (optionally synced to the song); First Person works in both modes; the fixed views glance a little toward where the hands are playing, with a slow wander
• Lesson mode: keyboard only, falling notes in left / right hand colours with L / R (or finger number) labels, glowing keys, halos and sparks; hands from a quick split, refined by fingering/ when present
• Pianist hands like Concert Creator’s avatars (concertstage/avatar-hands.js): one skinned mesh per hand (palm, thenar pad, tapered fingers with knuckles and nails, wrist, white shirt cuff, dark jacket sleeve fading toward the shoulder) on 18 bones; fingers follow the #029 fingering and motion model with its finger IK, arms are two-bone IK from a seated shoulder
• Depth of field that keeps the hands sharp (picks the hand nearer the middle of the frame or the busier one, no focus pull for single notes), bloom, film grain, vignette and a slight handheld drift; fully procedural lighting (no HDRI downloads)
• Render → offline MP4 export (any resolution, 30/60 fps, with audio) via notation3d/video-export.js
• Hall acoustics (concertstage/acoustics.js, hall-reverb.js): master volume (to 200%, with a limiter) and a room boost (±12 dB of reverb, ambience and applause against the piano); a stereo FDN reverb with early reflections in an AudioWorklet after the synth; rooms from studio to cathedral, seats from the bench to the balcony (direct/reverberant balance and air absorption), reverb / decay / tone, hall ambience, applause at the end of a song. The MP4 export runs the same DSP
• Share: the address bar holds every setting (stage, piano, colours, effects, sound, hall, render options, mode, camera view or free camera, speed, loop, song) and Share copies it with the song position; a shared song starts on the first click

14 Oct 2026

Experiment #028 - Grand Piano

• Physically based concert grand built entirely in code (grandpiano/grand-piano.js), ~211 cm model-B proportions
• S-curved rim, lid on a prop stick with fold-back flap (full / short stick / closed), cheek blocks, fallboard, music desk with sheet music, turned legs on brass casters, lyre and three pedals, artist bench
• Inside: gold plate with windows over a spruce soundboard and bridges, ~230 overstrung strings (copper bass crossing the tenor), tuning and hitch pins
• 88 keys with hammers and dampers that move with them; sustain pedal lifts all dampers and presses the pedal
• MeshPhysicalMaterial throughout: clearcoat lacquer (ebony, satin, white, walnut, mahogany, rosewood, red), metallic plate and strings, sheen felt
• Scenes: photo studio (real softbox HDRI with a dark floor, on a cyclorama), Victorian ballroom salon on 2k herringbone parquet, concert stage looking out over the hall (Poly Haven CC0 2k HDRIs, grounded skybox)
• Photo-scanned Poly Haven veneers: spruce-toned soundboard, maple bridges, walnut, sapele mahogany and figured rosewood case finishes (loaded on demand)
• Soft shadows, contact shadow, GTAO and a little bloom; AgX tone mapping with exposure control
• Cinematic tour camera: a slow 110 s loop through eleven shots on a spline (tail, under the lid, hammers, along the keys…); drag to take over, Esc hides the UI, ?view=tour starts in it
• While a song plays the tour follows it: camera position is locked to song time (pause holds the shot, seeking glides there, the loop is stretched to fit the song), it leans toward the notes being played and pushes in on loud chords (?sync=0 to turn off)
• Shareable links: the address bar always holds the current scene, finish, lid, camera angle, effects, sound and song; “Copy link” adds the song position (a shared song starts on the first click)
• Play with mouse, QWERTY or Web MIDI, or play songs through soundfont-engine; ?q=low for slower GPUs

13 Oct 2026

Experiment #027 - JabTunes Reimagined

• Real-time three.js take on the 2009 JabTunes piano: every key is extruded geometry (notched whites, sloped blacks) on its own pivot, so chords, glissandi and whole songs animate
• Mirror-under-lacquer fallboard with gold “My Music” (?title= to change); light trails rise from each played key up through the varnish (per-channel colours for songs)
• Song player: built-in MIDIs, Open… or drag-and-drop any .mid; keyboard grows to fit, speed, seek
• Sound via soundfont-engine (oxisynth, Motif ES6 + Creative GM by default, GM instrument picker) or the original 2009 samples, repitched to any range, with damping and a working pedal
• Original QWERTY map, ←/→ octave shift, Space sustain, Web MIDI keyboard input with velocity
• Notepad recorder in the old c1 d1s … notation with Play / Back / Clear, plus Share (link replays the recording)
• 25, 37, 49, 61 or 88 keys; original, player, low, top and free-orbit cameras

Experiment #026 - JabTunes (recovered)

• Faithful HTML5 port of jabtunes.com as archived on 15 Jan 2010: landing page, “Peek” bubble and fancybox popup, then the Flash stage
• Uses the assets pulled out of 3Dpiano.swf and peek.swf (decompiled with JPEXS, kept in jabtunes/): 26 pre-rendered JPEG frames swapped per key, 25 embedded 11 kHz MP3 samples, the 25 hand-placed button hit shapes, and the ActionScript, ported line by line (quirks included: one key shown at a time, samples always ring out, no glissando)
• Instrument combo, Undo / Clear notepad and Play replay as in the original; the streamed non-piano instruments were never archived, so they fall back to General MIDI patches
• ?peek opens the stage alone, ?peek&fit scales it to the window, &hits outlines the hit areas

1 Oct 2026

Experiment #025 - Notation 3D

• 3D sheet-music player inspired by “Note Bounce”: extruded SMuFL notation, bouncing balls land on every onset
• Engraving by nwc-viewer (interpreter + scroll-layout typesetter), loaded from ../nwc-viewer/ (override with ?nwc=<url>)
• Canvas2D → three.js bridge: nwc-viewer draws into a recording context; glyphs become InstancedMeshes, beams/stems/staff lines are extruded
• Own MIDI → notation builder (grand-staff split, onset-clipped durations, running accidentals, key detection, piano reduction)
• Loads MIDI, NWC and MusicXML; audio via soundfont-engine + MidiScheduler
• Views: tilted, flat, tabletop, low angle, free orbit · night ink / paper themes · bloom, ripples, particles, trails, shadows
• Layouts: single endless line, wrapped sheet, or Letter pages on a desk (nwc-viewer wrap/page modes) with a whole-page view and pan/crossfade/cut line changes
• Needs a server rooted at the parent folder locally, e.g. cd .. && python3 -m http.server

15 Mar 2026

WebMscore Reference Renderer

• Load .mscz, .mscx, MusicXML, MIDI files and render via MuseScore’s libmscore in WASM
• SVG/PNG/PDF/MIDI/MusicXML export
• Score metadata, parts, measure positions
• Interactive API console for experimentation

5 Jan 2026

Experiment #023 - Synthesia Background Playback

• AudioWorklet-based MIDI scheduling (plays in background tabs)
• Reduced audio glitches via worklet timing
• Uses new useBackgroundClock option in PlaybackControls

3 Jan 2026

Experiment #022 - Synthesia Multi-Synth

• Selectable synth engine: OxiSynth SF3, OxiSynth, SpessaSynth, FluidWeb
• Hot-swap synths during playback

Experiment #021 - Synthesia OxiSynth SF3

• OxiSynth with native SF3 support via spessasynth_core
• Decodes SF3 to SF2 in-browser before loading
• Non-blocking decompression (main thread stays responsive)

SF3 to SF2 Converter

• Browser-based SF3 to SF2 converter
• Uses Web Worker for non-blocking Vorbis decompression

Experiment #020 - Synthesia SpessaSynth

• SpessaSynth WASM synthesizer
• SF2 and SF3 soundfont support
• Soundfont stacking with bank offsets

Experiment #019 - Synthesia OxiSynth Worker

• Experimental: runs OxiSynth in Web Worker with AudioWorklet
• Keeps soundfont parsing off main thread
• Higher latency and memory usage than #018 due to message passing overhead
• Use #018 for better performance; this is for reference

Experiment #018 - Synthesia OxiSynth

• OxiSynth WASM synthesizer (Rust/oxisynth)
• SF2 and SF3 soundfont support
• Soundfont caching via font bank
• Multi-instrument via MIDI channels

2 Jan 2025

Experiment #017 - Synthesia FluidWeb

• RustySynth WASM synthesizer (SF2 only)
• Multiple .sf2 soundfont selection
• Real-time SF2 playback with AudioWorklet

FluidWeb Piano Test

• WASM SoundFont synth with AudioWorklet

FluidWeb Piano (ScriptProcessor)

• WASM SoundFont synth with ScriptProcessorNode (iOS compatible)

FluidWeb Test

• Basic FluidWeb WASM test

27 Dec 2024

Experiment #016 - Synthesia Three.js

• Three.js 3D version of Synthesia
• Orthographic (default) and perspective camera views
• 88-key 3D piano with proper key positioning

Experiment #015 - Synthesia WebGL

• WebGL2 GPU-accelerated version of Synthesia
• Blur-based bloom effect with screen blending
• Batched rendering for better performance

Experiment #014 - Synthesia

• Synthesia-style falling notes visualization
• 88-key piano at bottom with key lighting
• Particle effects on note impact
• Grid lines for octaves and measures
• Multiple instrument support (6 synths)
• Bloom toggle for performance

Experiment #013 - Physical Model

• MIDI playback with custom piano synthesis
• Ball visualization from earlier experiments

Audio Test

• Audio-only performance test (no visuals)
• For benchmarking synth performance

Piano Test

• Interactive 3-octave keyboard
• Test piano synthesis directly

Piano Synthesis Library:

• piano.js - Modal synthesis with harmonic profiles (PianoForte-inspired)
• wavetable_piano.js - PeriodicWave wavetable (musical.js-inspired)
• fast_piano.js - Pre-rendered samples for max polyphony
• electric_piano.js - Additive synthesis with inharmonicity
• harpsichord.js - Karplus-Strong plucked string
• silent_piano.js - 4’33” by John Cage (for testing)

26 Jan 2021

Experiment #012 - Merry Spiral

• Modified from 011, particles moves from inside out
• feels like a merry go round

Experiment #011 - Sucking star

• Modified from 010, but clamp to the center

Experiment #010 - Fountain rain

• Modified from 003, but arranged in a circle
• notes move from the edge to the center

Experiment #009 - Lighted piano roll

• Extension of 001, but I wanted it to light up when notes are being played

4 Feb 2018

Experiment #008 - Spiral path

• Extension of 007, with the trails
• Follow cursor isn’t the best, feels like a drawing app.
• Maybe it should have a life, and let interactivitiy move it.

23 Jan 2018

Experiment #007 - Spiral

• Draw notes as they place around in a spiral
• Fade notes away after drawing
• More ideas: follow cursor / trail / random,
• predraw vs continous simulation

21 Jan 2018

Experiment #006 - Hammers

• Visualize notes by abstract piano hammers
• also test easing algorithms

Experiment #005 - Keyboard

• Visualize notes on Keyboard

Experiment #004 - SoundFont

• Use of soundfont.js for audio samples

20 Jan 2018

Experiment #003 - Balls with controls

• Balls + Controls

Experiment #002 - Balls

• Balls

Experiment #001 - Piano Roll

• Classic visualizations