Repository navigation
feat(live): optional tooling to livestream a model's storm days to X - #4
Merged
Merged
Conversation
…the stream, paced on slots under a spend guard scripts/live_runner.py plays airops-occ3's joint days one `agent-env run` at a time, as the board's pilot plays one (occ.sweep's bundle, caps, rows, days and transcripts under results/runs/<name>), in the order of scripts/live_runner.toml: the 44 training days no model played on the board, ranked by storm (GDP hours plus twice the ground-stop hours at JFK and BOS, plus one per ten flights cancelled), a storm's next day after the rest and the seasons interleaved, then the eval days. A day starts on a grid of slots, after an intermission until the last one's slot ends, only while the spend so far plus a worst day stays within the budget and its slot ends inside the window; while it runs, its desks' "$X spent" log lines give a live estimate, and the day is stopped if that reaches the budget or the day outruns the longest allowed. A failed attempt is tried once more, then the day is skipped; an audit refusal is final. current.json gives the live server the running day, its instance as soon as agent-env records it (the store's newest instance of the bundle task), its env once deployed, the finished day's record, the next day and the history. stop.flag or a SIGTERM finishes the day, a second SIGTERM stops it, and a restarted runner picks a running day up. The config is read again before every day; --dry-run prints the schedule and `order` writes the ranking. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… as it is decided, with a director, a live panel and the day's end web/live.js polls the live server, grows the page in place (the cards, the room, the 3D airport's traffic) and paces each newly decided hour to the desks' watches, never past what is decided; web/director.js cuts between the room, the airport and the map on the day's events; the day ends on its score card, its highlights and the next storm, and the page reloads onto the next day. &stream lays it out for a 1920x1080 broadcast. scripts/live_fake.py serves recorded days as the live server would, and occ_qa.py --live checks the live mode on it. Recorded playback is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… wall redraws and rests the hidden airport The viewer polls /live/state.json, fetches the episode as more is decided and once more for the late sidecar, reads the desks' moves from live.feed, the spend from spent_usd and total_usd, and plays the day's end only as far as it is decided. The director varies its fillers, films the terminal only where people are and keeps highlights inside the day; the stream has a title; the sound (window.airops.sound) is updated from the live frames. The fake serves the contract's endpoints on its own port. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ce the next is on, and reloads after a lost GPU view The day's briefing (hub, date, network, weather, the FAA, what it is scored against) shows while the clock holds at the day's start for the desks' first decisions; the director captions the desks' turns and favours the map once the day is all decided; fetches time out, and a lost WebGL context or a heap past 3 GB reloads the page onto the day. Node tests for the live clock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ally, serve the day so far occ/onair.py plays the env's calls again as its record grows (each result and seq checked against the env's) and builds the viewer's episode of the day so far: the room as it stands, the closed watches as they were left, the flights, crews and passengers on the open watch's projection, the projected cost, no grade; once done, the recorded episode itself (episode.episode on the replay), equal to occ_viewer.py's. scripts/live_server.py: prebuild (a day's news, hub traffic and radar frames), serve (follows the runner's current.json through starting, playing, done and intermission, catching up two watches at a time, reading the day file when the env is gone; /live/state.json, /live/episode.json with its sidecars, finished days, the assets and the viewer, CORS open), feed (a fake runner and env revealing recorded days a watch at a time, with failing calls and a restart) and attach (current.json for a plain agent-env run). episode.episode takes its news when already read; export.sidecar its traffic; episode.passengers' rows and held_minutes are shared with the projection's passengers. Recorded episodes build byte for byte as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r room, made from what is on screen web/audio.js plays the airport outdoors (its bed and the eight loudest aircraft by phase: idle and taxi, pushback with its tug and APU, the takeoff roar with Doppler, approach, reverse thrust and spool-down, by distance and bearing), the terminal's hall and crowd with a chime and the jets muffled, the room's tone, keys and clicks, the map faint, and rain, wind and thunder from the stage's weather, levelled near -20 LUFS under a sample-peak limiter on the audio thread. The recordings in web/audio/ are Freesound CC0, each listed in SOURCES.md. The viewer loads it after the episode is up: off behind a speaker button by default, on with &sound=1 or &stream, absent with ?sound=0 and in record=1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d history, and records the desks' summary in a day's result; days take 80-minute slots Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ioWorklet, and the limiter at -2.5 dBFS An http origin other than localhost has no AudioWorklet; there the mix still can't pass -2 dBFS. The worklet's ceiling moves to -2.5 dBFS so thunder's inter-sample peaks stay under -1.5 dBTP. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # web/app.js
…ver replays the day, and ignores its empty first state Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, and centres the intermission card Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e stream with its own card; the sound's limiter sits at -3.5 dBFS Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…S peaks to -0.6 dBTP Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and the rehearsal's day, after the other training days Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…oes on from it in seconds serve pickles the replay (the Follower and its Played day, the feed, the desks' last calls, the watches' wall times, the env's grade) to LIVE/ckpt/<run_id>.pkl after each batch of new calls at a new watch, once done, and at most every 30 s between, tmp + rename. The pack's world is named by its objects' places in it, not copied: it holds closures, and the restarted server has the same one. A server started on a run with a checkpoint loads it when it is that run's (run id, task, mode, attempt, instance) on the same package files, publishes the resumed day at once with its status, archives a done day again, and follows data/get from there; a missing, torn or foreign checkpoint means the full replay as before. A new day removes the other runs' checkpoints. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s after the day ends are in the final episode The server stopped following the env at the replay's end of day, so calls the env logs after it (the desks' last resigns, a few seconds later) and, behind a fake feed, the env's grade were missing from the final episode: on b6-20230729 it had 632 steps to the record's 633, and the server test failed about one run in three. It now follows until it has read the runner's day file to its end, building the final episode again when calls come after the end and once the grade is in; the record's digest is checked once per digest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ard desks' after the rest of it Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…PU box, every account detail a setting The runner box, the live server, the streamer and the stale-picture watchdog as they ran, with the AWS and SSM helpers reading live/live.env, the contracts and the runbook. Nothing in it ships in the wheel. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Optional tooling for livestreaming a model playing AirOpsEnv v3 storm days, live in 3D with airport sound, to X or any RTMP target. Normal use of the plugin is unchanged:
live/sits outside the package and isn't in the wheel or the sdist, live mode only runs with?live=, and sound is off behind a speaker button.How it works
One AWS GPU box (g6.4xlarge, NVIDIA L4) runs everything on localhost:
scripts/live_runner.pyplays the model's days back to back, with a spend guard and time slots.scripts/live_server.pyfollows the env's data/get and replays the calls exactly in its own day, then serves the day so far as Episode JSON. It checkpoints each watch, so a restart resumes in seconds.web/live.jsplays the day as it is decided, never past what is decided.web/director.jscuts between the room, the airport, the terminal and the map. A live panel shows the desks' latest moves. Between days come a score card, highlights and a next-storm card.web/audio.jsplays CC0 Freesound recordings plus synthesis, levelled to -20 LUFS.live/stream/runs Chrome on Xorg, with ffmpeg NVENC at 1080p30 into a local relay and recorder, then the push. A supervisor restarts what dies.live/onair/box/fresh.pycompares the stream's picture with Chrome's screenshot.live/README.mdis the runbook: settings, setup, a $0 scripted test, a one-model-day rehearsal, arming for a start time and budget, going live, monitoring and teardown. Every account- or deployment-specific value is a setting inlive/live.env, and secrets are read from SSM.Other changes on the branch
onair.Followerchecks the env's digest once per digest.episode.episode(news=),episode.ledger_rowsandexport.sidecar(traffic=)are new, used by the live server. Recorded episodes are built byte for byte as before.Verified
occ_qaon recorded v1, v2 and v3 episodes, same-origin and cross-origin, with 0 failures. Stills match v0.5.0.tests/occandtests/viewerpass (138), andruff check .is clean.exact, 0 mismatches);live/: the scripts were changed only in their settings and paths, and checked withbash -n, shellcheck and a stubbed dry run. They haven't run against AWS since the move.🤖 Generated with Claude Code