Repository navigation
feat: audio accessibility alternative - #3123
Merged
Merged
Conversation
Adds `CaptchaType.iconOrder`: a frame of procedurally generated icons with a
legend naming which of them to click, and in what order.
The answer never leaves the provider. Icon positions and the required order
are written to the challenge record before the response is sent; the widget
receives a composited frame and a legend strip and nothing else, so there is
no coordinate for a client to echo back. Grading is strict on order and uses
a hit radius that scales with each icon's own size.
New packages:
- @prosopo/icon-order-assets — glyph vocabulary, placement, compositing and
the ordered legend. Reuses puzzle-assets' PRNG, background generator and
encoders, and shares its pre-generated background buffer at runtime.
- @prosopo/procaptcha-icon-order — the widget: click capture, numbered
markers, reset/confirm.
Also extracts InteractiveCaptchaManager, the server-verify pipeline both
interactive types now share (replay and recency checks, client-session
correlation, access policies, spam rules, traffic filter, IP validation and
the decision machine). PuzzleCaptchaManager moves onto it — 996 lines down to
484 — with its existing suite unchanged and green.
Ranking places icon-order above puzzle and below image wherever captcha types
are ordered by harshness. Coercion only reaches for icon-order when
icon-order was asked for: the puzzle and image fallbacks are untouched, so no
existing site's behaviour changes.
…odule commit The lockfile was regenerated while the nested fingerprintjs submodule was checked out at a commit other than the one this repo records, which pinned @prosopo/config@3.3.1 against a submodule that asks for 3.3.3 and made `npm ci` refuse the tree.
…ages lint:refs requires each tsconfig's `references` to mirror its package.json workspace dependencies.
`name` and `email` come straight off the form, so interpolating them into markup lets anything typed into those fields run as HTML. CodeQL flags it as js/xss-through-dom.
The icon-order demo pages existed but pointed at the puzzle sitekey, and
nothing seeded a sitekey configured for the new type.
- `getDefaultSiteKeys` seeds one, so `npm run setup` registers
`DEV_PHRASE//iconOrder` with `captchaType: iconOrder` and writes
`PROSOPO_SITE_KEY_ICONORDER` into the env files. It is ordered before
`puzzle` deliberately: `updateDemoHTMLFiles` rewrites the sitekey in every
demo HTML file once per seeded type, so appending would have silently
repointed the android/ios webview demos from puzzle to icon-order.
- Both demo pages, the vite `define` block and the env templates use the new
key, and the injected code sample now names the variable that actually
exists.
- The nav splits camelCase type names, so the entry reads "Icon Order".
Also fixes the /frictionless short-circuit, which threw
"Unhandled configured captchaType" for an icon-order sitekey: every widget
enters through /frictionless, so a configured icon-order site could not
obtain a session at all. Adds the missing `sendIconOrderCaptcha` alongside
the image/pow/puzzle helpers.
…env files `updateEnvFiles` only replaced variables that were already present, so the site key for a newly added captcha type never reached a developer's existing `.env`: `copyEnvFile` seeds from the template only when the file is absent, so any machine set up before the type existed kept a file without the variable. The demo page then rendered with an undefined site key and the widget fell through to the wrong captcha type. A variable named like a site key is now written whenever the file already tracks site keys at all. Files that track none are still left untouched, so this doesn't scatter keys into unrelated env files.
…radient A smooth mesh gradient was the wrong frame for this captcha. It has almost no edges of its own, so every icon stroke was the strongest local signal in the picture and a single edge-detection pass found all of them. The frame is now a collage: colour panels blocking it into regions, concentric ripple families, and heavy opaque bars slicing across. Panel lightness is drawn from dark / near-white / mid bands and hues jump around the wheel rather than staying analogous, so neighbouring regions actually contrast — which is the boundary an icon stroke can hide against. Post-rasterisation grain raises the noise floor those edges have to clear. `haloOpacity` goes up to keep icons findable against it. `backgroundClutter` scales every element family at once, so one operator knob moves the frame from clean to busy; 0 renders a plain single-colour frame as the escape hatch. The collage is vector work rasterised natively, cheap enough to draw per request, so icon-order no longer borrows the puzzle type's pre-generated background buffer — that buffer exists only because the puzzle's mesh gradient is per-pixel JS. Dropping it also means every frame is unique without a buffer having to guarantee consume-once.
HughParry
force-pushed
the
feat/header-restriction
branch
from
September 2, 2026 11:12
9f2b6ad to
df08a0c
Compare
Conflict resolutions of note:
- CaptchaType / ClientApiPaths / server.ts docs: both sides added members;
kept both (iconOrder and authenticated).
- captcha-severity: main extracted the per-type severity table that this
branch had extended in blacklistRequestInspector and checkTrafficFilter.
Dropped both local tables in favour of the shared package and added the
iconOrder tier there (image > iconOrder > puzzle > pow > frictionless),
so icon-order policies keep their ranking instead of falling to 0.
- provider tsconfig{,.cjs}.json: union of both reference lists. Also fixed
the icon-order-assets entry in the cjs config, which pointed at the ESM
tsconfig.
- getIconOrderCaptchaChallenge: main added a required requestHeaders
argument to getPrioritisedAccessPolicies; the icon-order handler now
passes normalizeHeadersForMatching(req.headers) like its siblings.
- package.json: took main's dependency versions and re-added the two new
workspace deps. The two new packages pinned pre-merge sibling versions
(@prosopo/puzzle-assets 0.1.3, @prosopo/api 4.1.3, vitest 4.1.10, ...),
which resolved them off the registry instead of the workspace; realigned
to the post-merge versions.
- package-lock.json: reconciled in place from main's lockfile rather than
regenerated, so every platform's optional deps survive.
…kage
- `verifyIconOrderCaptchaSolution` claimed the challenge's single submission
by reading `userSubmitted` off the record it had just fetched. Concurrent
submitters all read the same unclaimed record, so each was graded and
handed a verdict. `claimIconOrderCaptchaSubmission` does it under a
`userSubmitted: { $ne: true }` filter instead, so exactly one wins.
- Lower the `iconOrderTolerance` ceiling from 20 to 12, the smallest value
that still lets the Cypress specs make any click on the frame count. The
arithmetic is written onto the field schema so it is not raised casually.
- Drop `private: true` from `@prosopo/icon-order-assets`: the published
`@prosopo/provider` depends on it at runtime.
- Document the icon-order flow and the shared `InteractiveCaptchaManager`
base in architecture.mmd, which still described only puzzle.
Release v3.8.8 bumped every workspace package while this branch was open. Took main's package.json versions and re-added the two deps this branch introduces (`@prosopo/procaptcha-icon-order` on procaptcha-frictionless, `@prosopo/icon-order-assets` on provider), then realigned the two new packages' own `@prosopo/*` pins to the released versions so npm links the workspace copies rather than resolving them from the registry. package-lock.json reconciled in place from main's, not regenerated.
The frame bound both onClick and onTouchEnd. A tap on a touch device fires touchend followed by a synthesised compatibility click at the same coordinates, so each tap appended two points to the answer. Three correct taps submitted six clicks against three targets and gradeClicks rejected them on length before comparing a single position — icon-order could not be solved on any phone. Staging records confirm it: every mobile submission holds six clicks in near-identical pairs, each pair within its target's hit radius. Switch to onPointerUp/onPointerMove, which cover mouse, touch and pen and fire once per interaction. touch-action: none was already set on the frame and does not suppress the synthesised click. The regression test dispatches the full tap sequence — pointerup, touchend, then the synthesised click — and asserts one marker.
forgetso
force-pushed
the
feat/audio-captcha
branch
from
September 11, 2026 11:43
6639945 to
61ccc64
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
forgetso
force-pushed
the
feat/audio-captcha
branch
from
September 11, 2026 11:49
61ccc64 to
f77439b
Compare
forgetso
marked this pull request as ready for review
September 11, 2026 11:55
Adds `CaptchaType.audio`: a challenge that speaks five digits and asks the user to type them. It is directly selectable and routable like image, pow and puzzle, and doubles as the accessibility path — a per-site `audioAccessibilityEnabled` flag adds a "use audio instead" control to the image and puzzle widgets. Speech is synthesised procedurally rather than recorded or generated by a TTS dependency. `@prosopo/audio-assets` implements a source-filter (Klatt-style) formant synthesiser: glottal pulse train or band-limited noise, three interpolated formants, lip radiation. A fixed corpus is a finite set, and anything finite can eventually be collected; a continuous parameter space has no such set. The transcript never leaves the provider. `GetAudioCaptchaResponse` has no field it could be assigned to, which is the structural half of the guarantee; the puzzle captcha shipped its target coordinates to the client once, and any caller could echo them back and pass without rendering anything. Challenges are single-use, so a wrong answer issues a fresh challenge rather than reoffering the spent one. Grading is exact match after non-digits are stripped, so "1 2 3 4 5" and "12345" both pass; there is no edit-distance tolerance. The render defaults are tuned for intelligibility rather than difficulty, and the surrounding signals do the gatekeeping. Difficulty work is expected to come from varying what the challenge asks rather than from distorting the audio further, so the challenge task is a per-challenge parameter from the start. Also fixes a pre-existing bug found on the way: `settings.puzzle` was read by the provider and offered in the portal but had no path in the mongoose schema, so every operator override was silently discarded on save. Both `puzzle` and `audio` now have paths, and the persistence test asserts each field round-trips. Scope is English digits only. Letters are excluded because several of the letter names are too easily confused, and the 32 locale files carry the English string as a placeholder. Tracked on the issue. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rebases the audio work onto main, which has moved on considerably: the captcha-type harshness tables the branch edited have been extracted into `@prosopo/captcha-severity`, `downgradePuzzleIfUnavailable` is now `coerceToEnabledCaptchaType`, main has landed its own (better-bounded) mongoose paths for the puzzle render settings, and `CaptchaType` has gained `authenticated`. Audio is folded into each of those rather than restating them, and `coerceToEnabledCaptchaType` gains an audio case: the synthesiser has no native dependency, so unlike puzzle rendering it cannot be unavailable on a provider that has the code. Brings audio to parity with the other types on client-session correlation. It was the only type that did not bind a solve to the session id the site rendered the widget with, so a token earned in one session could be verified against another. The widget now records the id, the provider compares it at verify time, and both the API client and the dapp-server dispatch pass it through. Adds the missing tests. `@prosopo/procaptcha-audio` shipped with none at all, which also made `npm run test:all` fail outright, and the provider's audio manager had none either. 159 tests now cover the widget, the player, the manager and the provider-side issue / grade / verify path. Two defects those tests found, in the player: - the clip swap on a retry never paused the outgoing audio, because React has already committed the new `src` by the time the effect runs, so the `src !== clip` guard was always false. The user heard the end of the challenge they had just failed over the new one. - a replay was only counted when playback had advanced far enough to move `currentTime`, so pressing play twice in quick succession reported zero replays. The count now tracks presses, which is what it is documented to mean. Also aligns the audio widget's `execute()` wiring with the other widgets, which gained container-targeted dispatch: without it a bound button could not drive a visible audio widget. Trims the stats and citations out of the comments and the commit message. This is a public repository, and the arithmetic on guess probabilities and the pointers to published attack results were doing more for an attacker than for a maintainer. The engineering rationale stays; the numbers go. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`audio-explicit.html` wrote the name and email straight into `innerHTML`, which CodeQL flags as a high-severity `js/xss-through-dom`. The three sibling audio pages already use `textContent`; this one was copied from an older demo that did not. Same output, whitespace preserved with CSS. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
forgetso
force-pushed
the
feat/audio-captcha
branch
from
September 11, 2026 12:09
4b8d896 to
6d9b3dc
Compare
`cypress.image.config.js` has no `specPattern`, so it runs every spec that is not explicitly excluded, and `audio.cy.ts` was never added to that list. The image step therefore ran the audio spec with the image sitekey, the image demo page, and without the `audioAnswer` cy.task that only the audio config registers — so it failed before reaching any audio behaviour, and took the whole cypress job down with it before the audio step could run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Vite's `%VAR%` HTML substitution and `import.meta.env.X` both read the env object that `define` populates, and `PROSOPO_SITE_KEY_AUDIO` was never added there. Every audio demo page therefore sent the literal string `%PROSOPO_SITE_KEY_AUDIO%` to the provider as its sitekey, which answered API.INVALID_SITE_KEY — so the widget never got as far as asking for an audio challenge and the cypress spec timed out waiting for one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The provider consumes a frictionless session when it issues a challenge, so the widget's retry — `manager.start()` on the same session — could only ever answer NO_SESSION_FOUND. A user who mistyped a digit got a 400 instead of another clip. Uses the reload delegation the image widget already has: the manager asks the wrapper to mint a fresh session and re-mount against it, rather than restarting frictionless itself (which drops the user back to an unticked checkbox) or refetching against a spent one. A direct-React consumer, which has no wrapper to ask, keeps the old behaviour. Also adds `audio` to the demo dapp server's derived-keypair loop. It walks the captcha types to find the key a token was minted under, and without an audio entry every audio token was rejected as "site key does not match the server configuration" before it reached the verify call. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Dispose the manager on destroy so expiry timers and late answers cannot reach a dead widget, show checkbox errors in the widget's language, and await render() on the explicit demo pages as main's pages now do.
…and trim comments Move the code icon-order copied from puzzle into procaptcha-common: the lazy mount wrapper, manager expiry/dispose lifecycle, spent-session guard, behavioural-data encryption and trusted click coordinates. Puzzle and icon-order now both use them. Frictionless mounts the reloadable solvers from one table. Ignore non-primary mouse buttons on the icon-order frame, fix frictionless's CJS project reference, and cut the comments down to the ones that explain a non-obvious why.
…end, fix three icon-order gaps Puzzle and icon-order now share one challenge handler, one solution handler, one verify route builder, one submit pipeline in InteractiveCaptchaManager and one set of database record methods; the icon-order files are thin specs over them. Fixes found on the way: icon-order event arrays were unbounded, icon-order solves were not counted by the per-email submission limit, and routing-machine icon-order overrides were never written to the session. Comments cut to the ones that explain a non-obvious why.
…end on the shared interactive code Audio now sits on the shared interactive-captcha backend: thin challenge, solution and verify specs, a database descriptor, the shared API client helpers, and AudioCaptchaChallenge built from RenderedAudioClip. The "offers audio" check used by both frictionless paths is one helper. Fixes: - verified audio solves were left out of the per-email submission count (covered by countCommitmentsByNormalisedEmail.integration.test.ts) - audioEvents was validated item by item before its length cap, so a huge array produced one error per item (covered by requestArrayLimits.test.ts) Also: audio appended after icon-order in saveCaptchas so icon-order keeps its position, unused audio request-body aliases removed, the maintenance audio builder moved off the puzzle builder's comment, comment-only edits to powTasks/captchaTypeSelection reverted, audio-assets dead code and unused exports removed, tests that duplicated the shared pipeline or the grader dropped, a server dispatch test for audio tokens added, and comments cut to the ones that explain a non-obvious why.
…et on the shared plumbing Move the audio widget onto the helpers the icon-order review put in procaptcha-common: lazyMount for the lazy loader, createManagerLifecycle, createSpentSessionGuard and encryptBehavioralDataForSubmit in the manager, trustedClickCoords and PROCAPTCHA_EXECUTE_EVENT in the widget, with the same reportError/returnToCheckbox/beginChallenge helpers. Its test harnesses now match icon-order's (defaultProcaptchaState, typed deferred<T>()). "Use audio instead": the image component and the canvas footer share one slot that shows or hides the button, and the button takes a Theme rather than a light/dark string the footer had to derive. Frictionless keeps the user's audio choice as one flag until a re-mint has no visual challenge to swap, instead of re-arming a one-shot flag in every recovery path, and mounts audio through the same branch as the other reloadable solvers. The audio demo pages are rebuilt from the icon-order pages (dead onLoggedIn handler, bogus fetch option and narration gone), the cypress spec shares one helper to reach audio, and comments across the slice are cut to the ones that explain a non-obvious why.
HughParry
changed the title
# Conflicts: # package-lock.json # packages/procaptcha-frictionless/package.json # packages/provider/package.json # packages/provider/src/tasks/puzzleCaptcha/puzzleTasks.ts
# Conflicts: # packages/provider/src/api/captcha/getPuzzleCaptchaChallenge.ts # packages/types-database/src/types/provider.ts # packages/types/src/provider/database.ts
# Conflicts: # packages/types/src/decisionMachine/index.ts
Icon-order is now switched on per site by Prosopo through captchaTypeFeatureFlags.iconOrder, which defaults to false. frictionlessTypes.iconOrder becomes an ordinary owner preference that defaults to true, like image and puzzle, and resolveAllowedCaptchaTypes allows icon-order only when both agree. A site pinned to icon-order is held to the flag alone. The icon-order challenge endpoint's own opt-in check is gone: isValidRequest's feature-flag check already refuses icon-order, with or without a session, on a site without the flag.
Audio is now served only when the site owner has audioAccessibilityEnabled on AND Prosopo has switched on captchaTypeFeatureFlags.audio, which defaults to false. isAudioAlternativeEnabled holds that rule and feeds both the /frictionless "audio alternative available" advert and isValidRequest, which now refuses audio up front, before the visual session it trades is spent. That replaces the audio endpoint's own isEnabled check. Audio stays out of frictionlessTypes, so it is still never selectable or routable.
# Conflicts: # packages/provider/src/api/verify.ts
# Conflicts: # packages/procaptcha-frictionless/src/procaptchaFrictionless.ts
# Conflicts: # packages/procaptcha-frictionless/src/procaptchaFrictionless.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an audio challenge, offered only as an accessibility alternative (like reCAPTCHA's audio option). The user hears a short sequence of synthesised spoken digits and types them in.
Stacked on #3165 (icon order). Until that merges, this diff includes it.
What it does
audioAccessibilityEnabled. Then image, puzzle and icon-order challenges show a "Use audio instead" button, and pressing it swaps in the audio challenge. A wrong answer keeps the user on audio with a fresh clip.captchaType, traffic-filter categories, Restrict rules and the site-key CLI all rejectaudio; routing, PoW escalation and severity tiers leave it out. The provider only serves audio against the visual session the user already has, and refuses a request without a session.@prosopo/audio-assetssynthesises the speech (no recorded clips to collect), and@prosopo/procaptcha-audiois the widget.Built on the shared code
@prosopo/procaptcha-common.Tests
Known gap
After a PoW challenge escalates to image or puzzle, the audio button isn't offered. The
/frictionlessresponse for a PoW session doesn't carry the flag. Fixing it needs a small design decision, so it's a follow-up.