Two ways to show the score while the #045 path-traced pianist plays, on the Goldberg Aria (midis/bach_goldberg_aria.mid) and on the complete Goldberg Variations (midis/bach_goldberg.mid: the Aria, the 30 variations and the Aria da capo, 37 min, and midis/bach_goldberg_repeats.mid, the same with every repeat, 72 min):
../nwc-viewer/, the engine #025 uses), from the score’s own notes.Both draw the system being played in a panel over the picture, light the bar being played, and scroll the next system up into the panel over the last 0.7 s before the music reaches it. The same drawing goes into every frame a video export makes (Render). In a video’s lead-in (Render → Lead-in before the music) the panel waits for the music: it comes up over the half second before the first note. It is drawn after the frame source, so path-traced videos get it too.
Visuals → Score: Show the score (key S), Style (glass, paper, ink only), Position (automatic, the default; bottom, top or a corner), Systems shown (1 or 2), Size, Follow with (the bar, the default; bar and cursor, cursor, nothing) and In videos. The cursor is off by default because in #049 it moves by time within the bar and doesn’t land on each note.
Automatic picks one place for the whole song and keeps the panel there. placer.js asks the page for a survey (pathtrace/app.js surveyFrame). The survey runs a copy of the director over the song’s planned shots, 120 moments spread over its length (about 135 ms on SwiftShader), and at each moment measures how much the pianist and the piano fill each place. It uses the same measure #045’s intertitles use (pianist/frame-space.js: the piano’s surfaces and her skeleton projected into a 48 × 27 grid, her body counting double and her head three times). On a fixed view it measures the camera as it is. The places are bottom and top centre, plus the four corners where the panel leaves a third of the width beside it. A place’s cost is what it covers of her, plus half of what the intertitles would cover in the emptiest of their places it leaves free. Where the intertitles have a fixed place, or every note shows as a caption at the bottom, the panel keeps off that place. The panel holds its part of the frame, and automatic intertitles keep off any place it overlaps. It is chosen again only when the survey’s answer would change: another song, a director setting, another view, or the liner notes arriving. It doesn’t move while the song plays, or when the controls show: they pass over it, as they don’t appear in a video. On the Aria with the automatic camera, it goes top right, and the intertitles go bottom left.
Performance → Hands (in #032, #045, #049 and #050) chooses who plays what: From the score’s tracks (?hands=tracks) where the file names its tracks for the hands, Automatic (?hands=auto), the split that fingering.js finds easiest for the fingers, or the two together (the default, below). On the Aria the two differ on 95 of its 542 notes: the score gives the left hand 250 notes, the automatic split 163, because it moves the tenor line into the right hand. Automatic, with the tracks’ hand crossings (?hands=crossings, the default since 5 Nov) is the automatic split except where the tracks cross: a passage where the left hand’s track plays above the right hand’s, at least 4 times with less than 2 s between, keeps the tracks’ split from a second before to a second after. The automatic split carries on from where each hand left the passage. On the complete Goldberg it finds 41 such passages, 4.8 min of the 37: 38 in the variations for two manuals (5, 8, 11, 14, 17, 20, 23, 26, 28, 29), 2 in Var. 1 and 1 in Var. 25, and none in the Aria. In the passages it plays 99.6 % of the notes with the tracks’ hand, where Automatic plays 49 %, which uncrosses the hands. Outside them it agrees with Automatic on 99.8 % of the notes. A file whose tracks don’t name the hands is automatic in every mode. fingering/hands.js does this for every page that plays.
| File | What it does |
|---|---|
| timeline.js | Song seconds to (score bar, place in the bar). createMovementTimeline does it for a work of many movements: it finds each movement by the silence (1.5 s or more) before it, and numbers the bars through the whole score, so the da capo is on the Aria’s bars. Between two movements it holds the last bar. Where a file plays the bars in its own order, the manifest’s performances gives it, file by file: midis/bach_goldberg.mid leaves out the first of two endings (974 bars played), and midis/bach_goldberg_repeats.mid plays every repeat (1918), so at a repeat the panel scrolls back to the system the half starts on. Each movement is checked against the file: where its length isn’t the one that order gives, the order of the form that has its length is taken (as printed, or the other file’s), so an older copy of the file, as a browser’s cache may hold, is still followed bar by bar. On the complete Goldberg every bar start of the manifest lands on an onset of the file (Var. 16 within 0.125 s). The bar length comes from the number of bars the file plays: Mutopia’s Aria counts a quarter as 960 ticks at ppq 480, and has no time signature. A form lists the printed bars in the order played, for files that play the repeats. This one doesn’t: LilyPond’s MIDI leaves out \repeat volta, so the Aria is 32 bars, 96 s. |
| scan.js | Reads a page scan in plain JS: Otsu threshold; staff lines found in 16 vertical strips and followed across the page, so a page bowed toward the spine still reads; a straight-line fit per staff drops strips that took a ledger line or a beam for a staff line; barlines are columns dark through every staff of the system and, where most of them do, through the gap between the staves too, which leaves out the time signature and stems that line up. |
| tools/detect.mjs | Runs scan.js headless on an image and writes what it found into the score’s manifest; --png marks the bars on the page. On p. 263 it finds 6 systems with 6, 6, 5, 5, 5 and 5 bars, the score’s 32. The page can also detect at load where a manifest has no detected. |
| engrave.js | Engraves a MIDI with nwc-viewer in wrap mode (systems 1300 units wide) and draws it on a canvas. Its staves, barlines and note heads give the systems, the bars and each onset’s heads, so the cursor moves from note to note and the heads glow as they sound. Tracks named <staff> · voice <n> are laid on one staff, with the highest voice’s stems up, the lowest’s down, and the later voices’ rests hidden. A song with no score in scores.js is engraved from its own MIDI. The complete Goldberg is engraved a movement at a time, about 0.75 s each: the one at the time asked for first, the rest after it, one by one, and a movement’s page is drawn again when one of its systems first shows (3 kept). |
| tools/score_midi.mjs | Writes scores/bach_goldberg_aria.score.mid from Mutopia’s LilyPond (read by strings/lilypond.js): the notes as printed, a track a voice, 3/4 in G. It leaves out the 27 grace notes and the realized ornaments. The performance in midis/ plays trills as notes, which would engrave as runs of 32nds. |
| tools/goldberg.mjs | Builds the complete Goldberg’s manifest, scores/bach_goldberg.json. It reads the 31 movements from Mutopia’s LilyPond (bar starts, meters, keys; 956 bars, Var. 16 in two meters), detects the 46 pages headless (tools/detect-pages.mjs; --detect again, --marked dir marks them) and assigns the movements to the 271 systems by dynamic programming. A movement starts on an indented system, which in this edition is exactly the 31 under a heading. Each system’s bar count is chosen to add up to the movement’s count, close to the barlines found and closer still to the system’s width over the movement’s mean bar. Barlines narrower than 0.3 of the mean (time signatures, repeat signs, stems in line) join a neighbour. Where that still misses, scores/bach_goldberg.fixes.json gives the barlines read by eye (6 systems, in Var. 14 and Var. 24). It writes a score MIDI per movement to scores/goldberg/ for #050. |
| layout.js | The model both approaches produce: a page image, its systems’ rectangles, and each bar’s x range and onsets. place() puts the cursor between onsets where the layout has them, and in proportion to time within the bar where it doesn’t (the scan). inkOf() makes a system’s ink a mask, so a scan’s paper drops out and the ink can be tinted. scanLayoutPages() makes one layout of many pages: each system keeps its page, which loads as it first shows (with the page after it; 6 kept), and the overlay skips a system until its page is in. |
| overlay.js | Draws the panel, the bar, the cursor, the glowing heads and the scroll on any 2D context. |
| placer.js | Automatic placement: one place for the song, from a survey of the director’s shots, leaving the liner notes their room. |
| plugin.js | Joins it to pathtrace/app.js: the Score menu, a canvas over the picture, and the export overlay. |
| scores/ | The Aria’s manifest (meter, bars, sources, and the detected systems), the page scan, and the score MIDI; the complete Goldberg’s manifest, pages, fixes and score MIDIs. |
045_pathtracer.html was a 2,650-line page. Its script is now pathtrace/app.js, with its markup in pathtrace/shell.js and its styles in pathtrace/app.css, and the page boots it. A page that builds on #045 passes a plugin to boot():
boot({ title, href, defaultSong,
onReady(app), // after the page is built: add menus (app.addVisuals), keep settings in the link (app.addState)
onSong(song, midi, app), // a song loaded (also for ?shot stills)
onFrame(t, app), // each frame, song time t (and once for a ?shot still)
overlay(app) }) // returns (g, w, h, t) => {} to draw over each exported frame, or null
app gives pickPlace(places, prefer, margin) (the emptiest part of the frame now, as the intertitles choose it), surveyFrame(places) and surveyKey() (the same over the song’s planned shots, with the liner notes’ places), planReady, reserve(rect) (a part of the frame the intertitles keep off), songNow(), seekTo(), song, songFile, params, playing, exporting, renderer, camera and scene. #049 and #050 are each about 50 lines of HTML.
node --test sheet/test/sheet.test.js: the timeline (bar length from bars played, repeats), placement between onsets, the page reader on a drawn and slanted page, the hands option (the three modes, and the crossings found on a made-up passage), automatic placement (one place a song, room for the liner notes), the timeline of movements (found by their silences, the da capo on the same bars, held between movements) and the layout of many pages.node sheet/test/browser.mjs [out dir] (the parent folder served, as for pathtrace/test/stills.mjs) checks #049 and #050 headless. Both boot, find their 32 bars, and draw the panel at the bottom with 16.4 s in bar 6. A 5 s 480p export of #049 finishes, and ffmpeg pulls a frame from it. Then a still of the complete Goldberg in each, at 1024 s (Var. 16, bar 2). EXPORT=0 skips the video, FULL=0 the complete Goldberg.