ci: run the pipeline on 3.12, fix the gates it was hiding, and cover dev - #47
Conversation
The CI workflow pinned Python 3.9 while the backend ships on 3.12 (backend/Dockerfile). Test collection died on PEP 604 syntax, so pytest ran nothing and reported 31% coverage against a 60% floor; mypy never reached apps/ at all. Every backend job now runs 3.12, the runtime the code targets. Backend: - mypy.ini python_version 3.9 -> 3.12. At 3.9 mypy aborts inside site-packages (anyio uses match statements) before checking a single project file. - backend-typecheck gains DEBUG/SECRET_KEY. The django-stubs plugin imports config.settings, which refuses to load without them; the job failed with ImproperlyConfigured before mypy produced any output. - FolderSerializer installs its user-scoped `parent` field through get_fields() instead of a class attribute. DRF's own Field.parent owns that name on the base class, so the declarative form reads as an override of an unrelated attribute. Same field map, same IDOR scoping. - VoiceNoteListSerializer.folder gets the element-type annotation PrimaryKeyRelatedField needs when it is read-only. Frontend: `npm run build` under CI=true treats eslint warnings as errors, and there were 17 of them (npm run lint saw only 12 -- its glob is not recursive). All fixed in the source, none suppressed: - Dropped imports and bindings nothing read: DocumentTextIcon, MicrophoneIcon (x2), ExclamationTriangleIcon, PerformanceIndicator and its never-rendered showPerformanceIndicator prop, getPerformanceStatusColor, and four leftovers from the disabled performance manager in useAudioRecorder. - RegisterPage's two href="#" anchors became plain text. The app has no /terms or /privacy route, and LoginPage already states the same notice as text. - speech.d.ts declares SpeechRecognitionConstructor instead of redeclaring the SpeechRecognition name as both interface and var. - PerformanceManager reads its CPU-benchmark accumulator so the timed loop cannot be optimised away, and drops an unused getCurrentTier() call. - startRecording's dependency array is now accurate: state.recordingTime and updateAudioLevel move behind refs. Adding recordingTime directly would rebuild the callback every second, and updateAudioLevel is declared below it. The ref also fixes the diagnostic that always logged a duration of 0. - MicrophonePermission's mount probe declares its dependency; the callback stays stable via a ref because RecordPage passes a fresh handler identity each render, which would otherwise loop. Triggers now include dev on push and pull_request, so PRs into dev get checks. playwright.config.js ignores production-sweep.spec.js everywhere. It drives https://clio.chadacus.dev and registers real accounts there; the e2e job would have run it against production.
comprehensive-harness.spec.js expected an h1 of "Record Voice Note". No such
string exists in frontend/src -- RecordPage.tsx renders "New Voice Note"
(line 168) -- so the assertion failed against a correct page. The two sibling
assertions in the same block ("Ready to record" from RecorderControls,
"Recording Tips" from RecordPage) do match the source and are unchanged.
production-sweep.spec.js carries the same stale assertion but is excluded from
automated runs, so it is left alone.
|
Additional fix pushed as 11e33b8.
The two sibling assertions in the same block were checked against the source and
Verification: |
The e2e suite's registration and login tests share the anon bucket (60/minute) with the health check, so the health spec failed with 429 once the whole suite ran in one worker. A liveness probe must never be throttled; uptime checks hit it too. Adds a test that pins the exemption.
conftest disables throttling for the suite, so the first version passed with or without the exemption. Restore the production throttle on the base view for this test, tighten the anon rate, and prove with a control request that the throttle bites before asserting the probe stays 200.
… volume mount - Plumb THROTTLE_ANON_RATE through docker-compose.yml and raise it for the CI e2e job only; one runner address sends the suite's anonymous traffic inside the production 60/minute window. Production default unchanged. - celery now waits for the backend to be healthy: both images carry /app/media and both mounted the empty media_files volume at once, which failed with 'mkdir ...: file exists' on the second CI run. - playwright.config.js comment now says what testIgnore really does. - Folder update path gets the IDOR regression test review asked for.
The response interceptor retried every 401 through /auth/refresh/, including the refresh call's own 401, so a visitor with no session looped profile -> refresh -> refresh ... and never left the loading state. Locally the anon throttle (60/minute) turned the loop into a 429 within a second, which is why the redirect-to-login tests passed; the raised CI throttle exposed it. The refresh request is now excluded from the retry path, so its 401 rejects and the existing redirect runs.
What this fixes
.github/workflows/ci.ymlhas never been green, and it never ran for PRs intodev.The root cause is one line: the workflow pinned
python-version: '3.9'while thebackend ships on Python 3.12 (
backend/Dockerfile). On 3.9,backend/tests/test_mcp_queries.pyfails to import (PEP 604X | None), so pytestexecuted nothing and reported 31% coverage against the
--cov-fail-under=60gate. The typecheck job never got as far as a type error. Moving CI to the real
runtime exposed the checks that were hiding behind that failure, and this PR fixes
each of them in the source. No gate was lowered and nothing was suppressed.
Backend
python-version: '3.9'->'3.12'in all three backend jobsmypy.inipython_version = 3.9->3.12site-packages(anyio/_core/_tasks.pyusesmatch) with "errors prevented further checking" and never analyses a single project file.anyiois a transitive dep with no pin, so CI resolves the same version. Raising the target is not a loosening -- it is the only setting under which mypy checksapps/at all, and it is stricter about the syntax the code really runs.backend-typecheckgainsDEBUG/SECRET_KEYenvmypy_django_pluginimportsconfig.settings, which raisesImproperlyConfigured: You must set a secure SECRET_KEY when DEBUG=False. This is the actual failure in run 28346913057 -- mypy produced no output at all. Same two valuesbackend-testalready uses.FolderSerializer.parentmoves into aget_fields()overrideField.parentoccupies that attribute name on the base class, so the declarativeparent = UserScopedFolderField(...)reads to mypy as an override of an unrelated attribute ([assignment]).get_fields()is a public DRF extension point and produces the identical field map, so the user-scoped queryset (the IDOR guard) is unchanged.VoiceNoteListSerializer.foldergetsPrimaryKeyRelatedField[Folder][var-annotated]: aread_only=Truerelated field has no queryset for the stubs to infer the element type from. Quoted so the subscript is never evaluated at runtime.No
# type: ignorewas added anywhere.Frontend
npm run buildunderCI=trueturns eslint warnings into errors. There were 17,not the 12 that
npm run lintreports -- the lint script iseslint src/**/*.{ts,tsx}run undersh, which has noglobstar, so**collapses to one directory level and files like
src/components/AudioRecorder/*.tsxare never linted. The build output is theauthoritative list. (The glob itself is left alone; it is out of scope for this PR
but worth a follow-up.)
Every warning is fixed in the code. No
eslint-disablewas added.Dead imports and bindings removed:
DocumentTextIcon(Layout),MicrophoneIcon(LoginPage, RegisterPage),ExclamationTriangleIcon(MicrophonePermission),
PerformanceIndicatorplus itsshowPerformanceIndicatorprop (RecorderControls -- the component was neverrendered and no caller passes the prop),
getPerformanceStatusColor(PerformanceIndicator), and four leftovers from the disabled performance manager
in
useAudioRecorder(enablePerformanceManagementbinding,lastAudioLevelUpdate,getAudioAnalysisInterval,mediaRecorderOptionsandthe loop that filled it -- the MediaRecorder creation loop below already walks
the same config list directly).
jsx-a11y/anchor-is-validx2 (RegisterPage): both anchors werehref="#".The app has no
/termsor/privacyroute, so aLinkwould point nowhere anda
<button>would have nothing to do.LoginPagealready renders the samenotice as plain text; RegisterPage now matches. Visible wording is unchanged.
no-redeclare(speech.d.ts):SpeechRecognitionwas both aninterfaceanda
declare var. The constructor now has its own named type,SpeechRecognitionConstructor, used by theWindowinterface. Nothing referencedthe bare global as a value --
AudioDebugPanelgoes throughwindow.SpeechRecognition-- so the type surface is preserved.PerformanceManager: the CPU-benchmark accumulator was written and never read,
which is exactly what lets an engine delete the loop the benchmark is timing. It
is now read after the loop. The unused
getCurrentTier()call indowngradeQuality()is a pure classifier and was dropped.react-hooks/exhaustive-depsinuseAudioRecorder(the interesting one):startRecordingwas missingstate.recordingTimeandupdateAudioLevel.Neither could simply be added:
state.recordingTimeincrements every second while recording, so listing itwould rebuild
startRecordingon every tick. It was only read for aconsole.errordiagnostic, and the closure meant that diagnostic always logged0. It now readsrecordingTimeRef.current, alongside theisRecordingRef/isPausedRefpattern the hook already uses. This fixes the diagnostic as well.updateAudioLevelis declared belowstartRecording, so naming it in thedependency array would be a TDZ
ReferenceErrorat render time. It is invokedthrough
updateAudioLevelRef.current().With those two resolved the rule then reported
mimeType,sampleRateandshouldEnableDebugLoggingas unnecessary -- the first two reach the callbackonly via the memoised
getEffective*getters, which are already dependencies,and the third is not read at all. Removed.
react-hooks/exhaustive-depsinMicrophonePermissionandRecorderControls: the mount-time permission probe is now auseCallbackthatthe effect depends on.
RecordPage.handlePermissionChangeis not memoised, sodepending on the prop directly would re-run the probe ->
setState-> re-render-> new prop identity -> loop. The prop is read through a ref instead, which also
means
permissionStatus.onchangereports to the current handler rather than theone captured at mount.
RecorderControlsdestructures the hook's stablecheckMicrophonePermissionrather than depending on a property access the rulecannot verify.
Triggers
on.push.branchesandon.pull_request.brancheswere[main]only, so nothing onthe
devline was ever checked. Both are now[main, dev].e2e
frontend/e2e/tests/production-sweep.spec.jsdriveshttps://clio.chadacus.devandregisters real accounts on it. The
chromiumproject declares notestMatch, so itmatched everything under
testDir-- thee2e-testjob would have run aregistration sweep against production as soon as the earlier jobs went green.
playwright.config.jsnow sets a top-leveltestIgnorefor that spec, with acomment. It is unconditional rather than gated on
CI, so no environment variablecan turn it back on. The spec is not deleted.
npx playwright test --list --project=chromiumdrops from 11 files to 10 (50 tests); the spec was neverexecuted locally.
Verification
Run locally against the exact CI commands, on a Python 3.12 venv
(
uv pip install -r requirements.txt, plusruff==0.1.9as CI pins) and athrowaway
postgres:15container.(
pytest.inialready sets-q, so the CI invocation is effectively-qqandsuppresses the count line. The same suite run as
pytest -o addopts= -m "not live" --tb=short -qreports139 passed, 6 deselected, 204 warnings in 6.80s.)Verification on this PR
CI runs on this PR:
pull_requestresolves the workflow from the merge ref, so the trigger change applies to the PR itself. The checks on the PR head are the evidence for every job, includinge2e-test.Follow-ups landed after review: the health probe is exempt from the anonymous throttle (a liveness probe must never 429),
THROTTLE_ANON_RATEis plumbed throughdocker-compose.ymland raised for the CI e2e job only, and the celery service waits for the backend so both containers do not race to create the shared media volume path.