Skip to content

Read only the newest raw audio in the streaming melspectrogram - #356

Open
qwertey6 wants to merge 1 commit into
dscripka:mainfrom
qwertey6:stream-tail
Open

qwertey6 wants to merge 1 commit into
dscripka:mainfrom
qwertey6:stream-tail

Conversation

@qwertey6

Copy link
Copy Markdown

AudioFeatures._streaming_melspectrogram runs on every streamed chunk. It does list(self.raw_data_buffer)[-n_samples-160*3:], which turns the whole raw audio deque (maxlen=sr*10, 160,000 samples) into a list just to keep the newest n_samples + 480. This change reads those samples from the end of the deque instead, so a call costs time in proportion to the chunk, not the 10 s buffer.

Nothing else changes: the buffer, its size and _buffer_raw_data are untouched.

Same output. I compared it against the current code with hey_jarvis_v0.1 and alexa_v0.1 on 60 s of speech and noise, seeding numpy before each model, since AudioFeatures fills its feature buffer with random audio:

  • predictions, melspectrogram_buffer and feature_buffer were identical for 80 ms chunks, and for mixed chunk sizes from 333 samples to 1 s;
  • also identical after reset();
  • also identical for 80 ms chunks with speex noise suppression on.

Speex with uneven chunk sizes differs from run to run even on the current code, so that case can't be compared.

Faster. Median predict() time per 80 ms chunk, speex on, both models, pinned to one core in a Linux aarch64 container on Apple silicon:

median per chunk
current 1.67 ms
this change 1.26 ms

On a Raspberry Pi CM5 the current code measured 7.9 ms per chunk with a single wake word model. I haven't timed this change there yet.

tests/test_models.py passes (16 tests), and flake8 is clean at the project's 140-character limit.

🤖 Generated with Claude Code

_streaming_melspectrogram copied the whole 10 s raw audio deque into a
list on every call, then used only the newest n_samples + 480 samples.
Read those from the end of the deque instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant