fix (audio): cut output latency, and pin the pacing policy down with tests by JayBeest · Pull Request #74 · murkantor/Project-Midas

JayBeest · 2026-10-22T03:27:18Z

Follow-up to the silent-system fix (#69). Once audio worked, field
testing against the N64 original turned up a second, older problem: sound
arrives roughly 100 ms late. This patch reduces that to ~35–52 ms, and makes
the render-pacing rules testable so neither the silence nor the latency can
come back unnoticed.

This could potentially fix audio and gameplay bugs mentioned in #71

Symptom

Reported by @ltm8plays-sudo and me, comparing Project Midas side by side with real hardware:

Notice where the weapon reload sound starts. On real hardware, it starts at
the frame where the weapon starts to lower. But on this build, the sound
starts after the weapon is off-screen.

The same lag is visible in a release-era livestream, so it predates the
silence fix and is not a regression from it.

Root cause

Three buffers sit between an event being triggered and the first sample
reaching a speaker, and all three were sized generously for a bring-up build.
At 22050 Hz stereo s16 that is 88200 B/s, so bytes convert to time directly:

Queue watermark — PORT_AUDIO_QUEUE_LOW was 4 * 1024 B = 46.4 ms
(port/src/audio_pc.c). The pump refills only while the queue is below
this, so the queue always holds at least that much audio. It is the floor of
the steady-state backlog, and therefore a direct addition to latency.
Render quantum — one full g_FrameSize of 736 samples = 33.4 ms
(src/audi.c:342-355: (22050 << 1) / 60, rounded up to a multiple of 16).
An event posted between pumps waits up to one of these for its first
rendered sample.
Device period — want.samples = 512 = 23.2 ms, the cushion the sound
card owns.

The queue oscillates in [watermark, watermark + one quantum] = 46–80 ms;
add the device period and the range is 63–103 ms. That is the whole
reported delay. Weapon lowering is a handful of 60 Hz frames — roughly
100–150 ms — so a ~100 ms audio lag puts the reload sound right about where
the weapon leaves the frame, exactly as reported.

One term is not a bug: the SFX player's own clock granularity is
AL_USEC_PER_FRAME_30FPS (33 ms, src/snd.c:368), so a PLAY event waits up
to that for the next client wake-up. Hardware had that too; this patch leaves
it alone.

Changes

1. Smaller buffers, in all three terms

PORT_AUDIO_QUEUE_LOW 4096 → 2048 B (46.4 → 23.2 ms).
Render quantum becomes half a field, 368 samples (33.4 → 16.7 ms).
Throughput is unchanged — two quanta per pump at 147 Hz still cover the
2944 samples a field needs — only the granularity is finer.
want.samples 512 → 256 (23.2 → 11.6 ms).

Measured worst edge: ~52 ms, down from ~103 ms.

2. The device, not the clock, sets the pace

The previous patch made the pump render whenever the wall clock owed a frame,
added to the existing queue-driven catch-up. At steady state the two coincide,
which is why field testing showed no regression — but any device consuming
even slightly slower than real time would let the clock keep rendering, the
queue would creep toward PORT_AUDIO_QUEUE_MAX (16 KB = 185.8 ms), and the
extra frames would be discarded as glitches.

Pacing authority is now explicit and exclusive:

device present and draining → refill to the watermark, then stop. The DAC
sets the rate, so the backlog cannot drift upward.
no device, or a stalled one → the wall clock paces, so game audio time
still advances. Samples are discarded.

Neither branch can be blocked by the device. That distinction is the whole
point: the earlier bug was device state stopping the synthesizer; the latency
bug was device state failing to stop it.

3. Stall detection measures consumption, not change

The previous detector treated an unchanged queue as a dead sink. That is wrong
at these pump rates: at 147 Hz the pump runs every 6.8 ms while a 256-sample
device period lasts 11.6 ms, so a perfectly healthy queue is legitimately
byte-identical across a pump or two. The detector now measures real
consumption — bytes left at the end of the last pump, minus bytes present
now — and only after PORT_AUDIO_DRAIN_PUMPS (8, ~54 ms) consecutive empty
pumps does it stop trusting the device as the pacing authority. Reopening
still needs PORT_AUDIO_STALL_PUMPS (120, ~0.8 s), so a merely hesitant
device falls back to clock pacing long before it is torn down.

4. The policy moved somewhere it can be tested

port/src/audio_pace.c + port/include/audio_pace.h now hold the pacing
decisions as pure functions: no SDL, no Ultra64 headers, no globals, no clock
reads. portAudioPump() gathers observations and obeys the verdict.

This is not cosmetic. A test written against a reimplementation proves only
that the test agrees with itself — I wrote exactly such a model first and it
had a variable-shadowing bug that made a healthy sink look perma-stalled. The
shipped functions are what port/tests/audio_pace.c exercises.

Tests

port/tests/audio_pace.c, ROM-free and device-free, following the
port/tests/ + CHECK convention:

cmake -B build -DGE_PORT_TESTS=ON
cmake --build build --target port_tests
ctest --test-dir build -R audio_pace

Or standalone, since the unit has no dependencies:

cc -std=c99 -Iport/include port/tests/audio_pace.c port/src/audio_pace.c -o audio_pace_test
./audio_pace_test

Coverage:

Render quantum: half a field, 16-aligned, plus degenerate arguments
(0 field, 0 divisor, too-small, too-large, uncapped).
Live device paces to the watermark — including the case that matters most:
a large clock debt must not override a full-enough device queue.
No device → the wall clock paces, so the synthesizer keeps running.
Stalled device → falls back to the clock instead of freezing.
Drain tracker: empty queue counts as live; a hesitation that recovers clears
the streak; a dead sink is detected within PORT_AUDIO_DRAIN_PUMPS and
reopened at PORT_AUDIO_STALL_PUMPS, and the two thresholds are ordered so
a hesitant device is never closed.
A simulated pump loop against a device that consumes in whole periods —
the bursty pattern that broke the previous detector — asserting the backlog
never leaves the watermark band over ten seconds, and that render volume
tracks consumption exactly (feeding it more would grow latency, less would
starve it).
Latency arithmetic asserted rather than eyeballed: ≤55 ms end to end.

Mutation-checked, because a test that cannot fail is decoration. Each of these
deliberate regressions is caught by a specific assertion:

portAudioShouldRender always returns 1 (no watermark) → band test fails.
Drain threshold lowered to 1 (any gap = dead) → healthy-sink test fails.
No-device branch returns 0 (the original silence bug) → clock-pacing test
fails.

Verification

port/tests/audio_pace.c passes, standalone and through CMake/ctest.
port/src/audio_pace.c compiles standalone with -Wall -Wextra, no
warnings, no SDL or Ultra64 headers in scope.
port/src/audio_pc.c compiles clean with the macOS build's exact flags
(-fsyntax-only -std=gnu11 -arch arm64 -DGE_NATIVE_64=1 -DGE_VIDEO=1 …).

Verified om M5 macOS(27.0.1 (26A434)):
To my eye and ear this seems like this is (at least partly) underway towards a fix.

delay.mp4

Taken into account that both recording and encoding audio/video might not be perfect. THIS SHOULD BE TESTED LOCALLY instead of relying on video evidence.

What to listen for

Weapon reload should now land as the weapon starts to lower, not after it
leaves the frame.
No crackle on map load or during a busy firefight. If there is, raise
PORT_AUDIO_DEV_SAMPLES back to 512 first: it is a single constant and the
cheapest of the three knobs to give back (~12 ms).
ge: audio: device stalled ... should not appear on a healthy machine. If it
does, the drain threshold is too aggressive for that backend.

Follow-ups

SDL_AudioStream fallback so a device refusing 22050 Hz is accepted at its
own rate (the synth rate is baked into g_FrameSize, src/audi.c:344, so
the request cannot simply be relaxed).
portAcmdValidate's whole-frame bail-out still discards every frame of a
run while reporting only through GE_DUMP_BOND. That remains the leading
suspect for the Windows No audio on Windows 11 — silent SDL capture #56 report, which showed no event-queue starvation
at all.
An SDL callback backend would cut the remaining latency to one device period
plus one render chunk, by removing the queue from the path entirely. Worth
doing only if ~50 ms still reads as late in a side-by-side comparison.

This contains

Bug fix (non-breaking change which fixes an issue)
Tests

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce.

macOS build. running on macOS 27.0.1 (26A434)
linux build. running on arch linux 7.2.11-zen1-1-zen kernel -> PulseAudio

Checklist

My code has been thoroughly tested and is free of bugs
I have performed a (again, semi, not an audio expert) self-review of my code
I have commented my code, particularly in hard-to-understand areas
The commits I've made a clean, small and easy to understand
My changes generate no compiler warnings
My Code is AI-Generated or AI-assisted (please specify which AI model you are using)
My PR contains no AI-generated images, textures, models, sound or music (these are never accepted)
I am an AI (please specify which AI model you are)
I have reviewed my PR to make sure the above is truthful and accurate. I understand my PR can be rejected if it doesn't meet this project's standards.

Two silent-system reports (murkantor#19 Batocera, murkantor#56 Windows 11) share a cause that looks nothing like "no sound": the AL synthesizer was never executed. portAudioPump is the only caller of alAudioFrame, and alAudioFrame is the only caller of every AL client handler (synthesizer.c:189), so the SFX event queue, the sound-state list and the three music sequencers advance only while the pump renders. The pump had two device-gated exits - a permanent s_AudioDead flag set by any SDL_OpenAudioDevice failure, and a per-pump break while the playback queue sat above PORT_AUDIO_QUEUE_LOW - so a device that could not be opened, or that opened and then stopped consuming, froze game audio state outright. The Linux report shows the result exactly: the SFX queue pinned at its capacity of 256 (0x40 x GE_SNDSCALE 2 x MUSIC_SFX_EVENT_HEADROOM 2), 3328 events dropped, no voice ever allocated, and the API heartbeat still alive. No system has 22050 Hz s16 stereo natively; it works only because every healthy backend converts behind SDL. Where a 32-bit client cannot reach that layer, the open failed and the game went mute with no message - this was the only SDL subsystem in the port failing without logging, and the Windows build opens no console for anyone to notice. Changes, all in port/src/audio_pc.c: - portAudioInit returns success/failure and logs the driver and the obtained format, or the failing operation with SDL_GetError(). A failed open is retried every 2 s instead of being permanent, so a sink that appears or moves after boot (headset plugged in, PipeWire restart, late 32-bit client stack) is picked up. Failures are rate-limited so a permanently mute machine cannot bury the rest of the log. - The render pace is wall time, with the playback queue as a catch-up trigger rather than a gate: a frame renders when either the audio clock owes one or the queue is below PORT_AUDIO_QUEUE_LOW, so no device state can stall the synthesizer. The clock is kept in samples x1000 so a 147 Hz pump's fractional share of a frame cannot round to zero, and catch-up is clamped to 100 ms per pump so a debugger pause or a long load cannot replay minutes of audio back to back. An over-cap queue now discards that frame, not the render. - An unmoving full queue for ~1 s is treated as a dead sink: close it and let the retry path reopen whatever is there now. This is the one heuristic piece and the part most worth watching in field reports. - portAudioPumpForce renders exactly one frame, as its comment always claimed. It previously bypassed the queue gate per iteration, so a forced call could render eight, and music.c's stop-wait spins it up to 512 times. The healthy path is unchanged: queue-driven catch-up still runs and the wall clock only adds a floor, so systems where audio already worked render at the same pace as before. printf is declared locally rather than pulling <stdio.h>: PR/os.h declares sprintf itself, which collides with the hosted header's fortified macros on macOS, and this TU includes ultra64.h before any libc header. Same workaround as src/snd.c and port/src/printf_port.c. Verified: compiles clean with the macOS build's exact flags (-fsyntax-only, -std=gnu11, -arch arm64, -DGE_NATIVE_64=1 -DGE_VIDEO=1 ...) and adds no -Wall -Wextra diagnostics in the changed region. Not verified: no full build and no runtime test here - upstream is 32-bit and this machine is arm64 macOS - so frame cadence at 60/120/147 Hz pumps and the stall detector still want a real play session. Follow-ups, deliberately not in this commit: - An SDL_AudioStream fallback so a device that refuses 22050 Hz is accepted at its own rate (the synth rate is baked into g_FrameSize, audi.c:344, so the request cannot simply be relaxed). - portAcmdValidate's whole-frame bail-out discards every frame of a run while reporting only through GE_DUMP_BOND. That is invisible in normal logs and is the leading suspect for the Windows murkantor#56 report, which shows no event-queue starvation at all. - Trim the queue back to target instead of pinning it at the cap, so a long session cannot accumulate fixed audio latency.

Second audio patch, after the silent-system fix (murkantor#19, murkantor#56). With sound working, side-by-side testing against real hardware showed a delay that had been there all along: the weapon reload sound starts after the weapon has left the screen, when on N64 it starts as the weapon begins to lower. Roughly 100 ms. The same lag is visible in release-era footage, so it predates the silence fix. It is the output buffer depth, and the three terms are all auditable. At 22050 Hz stereo s16 (88200 B/s) bytes convert straight to time: - PORT_AUDIO_QUEUE_LOW was 4 KB = 46.4 ms. The pump refills only below this, so it is the floor of the steady-state backlog and a direct addition to latency. - One render quantum was a full g_FrameSize, 736 samples = 33.4 ms (audi.c:342, (22050 << 1) / 60 rounded up to 16). An event waits up to one of those for its first rendered sample. - want.samples was 512 = 23.2 ms, the cushion the sound card owns. The queue oscillates in [watermark, watermark + quantum] = 46-80 ms; add the device period and it is 63-103 ms. Weapon lowering is 100-150 ms at 60 Hz, so the reload sound lands right where the weapon leaves the frame. The SFX player's own 33 ms clock granularity (snd.c:368, AL_USEC_PER_FRAME_30FPS) is vanilla and deliberately untouched. Changes: - Watermark 4096 -> 2048 B (46.4 -> 23.2 ms), render quantum halved to 368 samples (33.4 -> 16.7 ms; two per pump at 147 Hz still cover a field, so throughput is unchanged), device period 512 -> 256 (23.2 -> 11.6 ms). Worst edge is now ~52 ms. - The pacing authority is now explicit and exclusive. The previous patch let the wall clock drive renders alongside queue-driven catch-up; at steady state the two coincide, which is why it tested clean, but a device consuming slightly slower than real time would let the clock keep rendering until the queue reached its 16 KB cap (185.8 ms) and the extra frames were discarded as glitches. Now a draining device sets the pace (refill to the watermark, then stop); with no device, or a stalled one, the clock paces so game audio time still advances. Neither branch can be blocked by the device - the earlier bug was device state stopping the synthesizer, this one was device state failing to stop it. - Stall detection measures consumption instead of change. An unchanged queue is not evidence of a dead sink at these rates: the pump runs every 6.8 ms while a 256-sample period lasts 11.6 ms, so a healthy queue is legitimately byte-identical across a pump or two. It now compares bytes left last pump against bytes present now, stops trusting the device after 8 empty pumps (~54 ms), and only reopens after 120 (~0.8 s), so a merely hesitant device falls back to clock pacing long before it is torn down. - The policy moved into port/src/audio_pace.c + port/include/audio_pace.h as pure functions: no SDL, no Ultra64 headers, no globals, no clock. That is not cosmetic. A test written against a reimplementation proves only that the test agrees with itself - the first model I wrote had a variable-shadowing bug that made a healthy sink look permanently stalled. The shipped functions are what the test drives. Tests, in port/tests/audio_pace.c, under a new GE_PORT_TESTS option: cmake -B build -DGE_PORT_TESTS=ON cmake --build build --target port_tests ctest --test-dir build -R audio_pace or standalone, since the unit has no dependencies: cc -std=c99 -Iport/include port/tests/audio_pace.c port/src/audio_pace.c \ -o audio_pace_test && ./audio_pace_test Covered: quantum geometry and degenerate arguments; device paces to the watermark, including the case that matters most (a large clock debt must not override a full-enough device queue); no device means the clock paces, so the synthesizer keeps running; a stalled device falls back to the clock instead of freezing; the drain tracker treats an empty queue as live, clears the streak when a hesitation recovers, detects a dead sink within DRAIN_PUMPS and reopens at STALL_PUMPS; and a simulated pump loop against a device draining in whole periods - the bursty pattern that broke the previous detector - asserting the backlog never leaves the watermark band over ten seconds and that render volume tracks consumption exactly, since more would grow latency and less would starve it. Latency arithmetic is asserted at <=55 ms rather than eyeballed. The suite is mutation-checked, because a test that cannot fail is decoration: always-render fails the band test, a drain threshold of 1 fails the healthy-sink test, and restoring the original no-device-means-no-render bug fails the clock-pacing test. The test target builds with explicit compiler flags rather than add_executable: the directory-wide add_compile_options() carries -m32/-msse2 for the x86-32 game, and this unit is plain C99 that should build on any host able to run it. Verified: tests pass standalone and through CMake/ctest; audio_pace.c compiles standalone with -Wall -Wextra and no warnings; audio_pc.c compiles clean with the macOS build's exact flags. Not verified: no runtime test here - upstream is -m32 and this machine is arm64 macOS - so the reduced device period's underrun headroom in a busy firefight still needs a real play session. That is the main thing to watch on merge; PORT_AUDIO_DEV_SAMPLES is the cheapest knob to give back (~12 ms) if a backend crackles. Follow-ups: an SDL_AudioStream fallback for devices that refuse 22050 Hz; portAcmdValidate's still-silent whole-frame bail-out, the leading suspect for the Windows murkantor#56 report; and a callback backend if ~50 ms still reads as late side by side.

Exercises port/src/audio_pace.c, which holds the render-pacing decisions split out of the SDL layer for exactly this reason: no cart, display or sound device needed, and the suite runs in well under a second. Covers the invariants both audio fixes depend on - the synthesizer keeps running with no device, a live device keeps setting the pace so the backlog cannot drift into latency, and a healthy bursty sink is never misread as dead (the previous detector's failure mode, since at 147 Hz a correct queue is legitimately byte-identical across pumps). Latency arithmetic is asserted rather than eyeballed. Mutation-checked, because a test that cannot fail is decoration: always-render fails the backlog-band test, a drain threshold of 1 fails the healthy-sink test, and the original no-device-means-no-render bug fails the clock-pacing test. New GE_PORT_TESTS option, off by default: cmake -B build -DGE_PORT_TESTS=ON cmake --build build --target port_tests ctest --test-dir build -R audio_pace Built with explicit compiler flags rather than add_executable: the directory-wide add_compile_options() carries -m32/-msse2 for the x86-32 game, and this unit is plain C99 that should build on any host able to run it. Also verified standalone: cc -std=c99 -Iport/include port/tests/audio_pace.c port/src/audio_pace.c \ -o audio_pace_test && ./audio_pace_test

JayBeest · 2026-10-22T06:00:48Z

@murkantor and others, I added a summary of my latest findings:

README

Doing another read of the artifacts, I saw this. And it dawned on me:

I don't know if the team has any opinion on (multi)threading the audio? (and/or other parts of the engine for that matter) But that sounds/seems like the 'proper way' forward? no?

When i think about it more, it is probably not needed at all. I think we should explore other options first.
Then again a thread-safe implementation for audio should not be that complicated.

jcorneli added 5 commits October 21, 2026 23:46

docs: Add high-level scoping notes for 32-bit ABI and macOS constraints

a59581f

docs: added findings

4963163

JayBeest force-pushed the audio-latency branch from a9779a3 to 4963163 Compare October 22, 2026 05:58

Navigation Menu

Repository navigation

fix (audio): cut output latency, and pin the pacing policy down with tests - #74

fix (audio): cut output latency, and pin the pacing policy down with tests#74
JayBeest wants to merge 5 commits into
murkantor:mainfrom
JayBeest:audio-latency

JayBeest commented Oct 22, 2026 •

edited

Loading

Uh oh!

JayBeest commented Oct 22, 2026 •

edited

Loading

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

JayBeest commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Symptom

Root cause

Changes

1. Smaller buffers, in all three terms

2. The device, not the clock, sets the pace

3. Stall detection measures consumption, not change

4. The policy moved somewhere it can be tested

Tests

Verification

What to listen for

Follow-ups

This contains

How Has This Been Tested?

Checklist

Uh oh!

JayBeest commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

JayBeest commented Oct 22, 2026 •

edited

Loading

JayBeest commented Oct 22, 2026 •

edited

Loading