Skip to content

feat(settings): run the QR promo value proposition experiment - #21112

Open
vbudhram wants to merge 1 commit into
mainfrom
fxa-14273
Open

feat(settings): run the QR promo value proposition experiment#21112
vbudhram wants to merge 1 commit into
mainfrom
fxa-14273

Conversation

@vbudhram

Copy link
Copy Markdown
Contributor

Because

  • The mobile promo in the auth flow has never been A/B tested, so we do not know which value proposition converts desktop users to app downloads
  • FxA has had Nimbus plumbing for a while but no experiment has ever used it; this is the first one
  • The current promo fits one line of copy, so the longer test strings have nowhere to go

This pull request

  • Declares the promo-qr-mobile feature in configs/nimbus.yaml with two variables, enabled and branch. Every channel defaults to the control, so a Cirrus failure is a no-op
  • Adds branches.ts, mapping each branch slug to its heading and its own QR code. One slug drives both, because each QR scans to a distinct Bitly link and a mismatched pair would misattribute the download. An unknown slug falls back to the control
  • Declares all 9 headings as literal ids in en.ftl so l10n extraction finds them, and picks one at runtime with useFtlMsgResolver. The CTA moves below the QR and changes wording, so it takes a new promo-qr-mobile-description-v2 id
  • Rebuilds index.tsx to the updated design. Drops the logo above the heading, since the Bitly QR already embeds the Firefox glyph. Replaces the 230KB qr-mobile-kit.svg with fox-body.svg, fox-tail.svg, and a per-branch QR, roughly a 10x reduction
  • Adds branch and nimbus_user_id extra keys to promo_qr_mobile.view. Cirrus reports enrollment itself, so the app must stamp nimbus_user_id on its own events for analysis to join them
  • Exports NimbusContext so tests and stories can supply a branch without a network fetch

Issue that this pull request solves

Closes: https://mozilla-hub.atlassian.net/browse/FXA-14273

Checklist

  • My commit is GPG signed.
  • If applicable, I have modified or added tests which pass locally.
  • I have added necessary documentation (if appropriate).
  • I have verified that my changes render correctly in RTL (if appropriate).
  • I have manually reviewed all AI generated code.

Other information

npx jest src/components/PromoQrMobile: 52 passed, 0 failed.
npx jest src/lib/glean src/models/contexts: 188 passed, 0 failed.

How to test:

  1. Set NIMBUS_ENABLED=true in the root .env and bring up infrastructure so the cirrus pm2 process runs
  2. Create the experiment in Experimenter against app accounts.cirrus, feature promo-qr-mobile, 9 branches
  3. Load the app with ?nimbusPreview=true at 1024px or wider, on /, /signin, or /signup
  4. Confirm the heading and QR change together per branch, and that promo_qr_mobile.view carries branch and nimbus_user_id

Branch records come from Remote Settings, so enrollment cannot be hand-authored locally. Storybook has Control, TreatmentB, and TreatmentA stories covering 1, 2, and 3-line headings without a Nimbus fetch.

Needs a decision before merge: the CTA reads "Scan to download mobile app" per the Figma design, but the ticket table says "Scan to get the app" in all 9 rows. The CTA is constant across branches so it does not confound the experiment, but it does change shipped copy in every branch including the control.

Needs data review: the two new Glean extra keys.

Add the first FxA Nimbus A/B test. Test 8 value proposition headings against
the current control on the desktop QR promo in the authentication flow, and
ship the updated promo design that makes room for the longer strings.

Declare the promo-qr-mobile feature in the FML with two variables, enabled
and branch. A single branch slug drives the heading and the QR code
together, because each QR scans to its own Bitly link and a mismatched pair
would misattribute the download. Every channel defaults to the control, so a
Cirrus failure is a no-op, and an unknown slug falls back to the control, so
a typo in Experimenter cannot break the promo.

Declare all 9 headings as literal FTL ids so the l10n extraction finds them,
and pick one at runtime with useFtlMsgResolver. The CTA moves below the QR
and changes wording, so it takes a new -v2 id and translators re-translate
rather than keep the old string.

Rebuild the component to the updated design. The logo above the heading is
gone, because the Bitly QR already embeds the Firefox glyph in its centre.
The 230KB qr-mobile-kit.svg gives way to real fox vectors and a per-branch
QR, about a 10x reduction. The heading used text-md, which is not in the
font scale and so set no size at all; it is now text-sm font-bold.

Report branch and nimbus_user_id on promo_qr_mobile.view. Cirrus reports
enrollment itself, so the app must stamp nimbus_user_id on its own events
for the analysis to join them. A CMS-supplied image carries no branch, so
those users get the control copy and report no branch. The render waits on
Nimbus so the control does not paint and then swap.
@vbudhram
vbudhram requested review from a team as code owners August 26, 2026 14:30
@vbudhram vbudhram changed the title Fxa 14273 feat(settings): run the QR promo value proposition experiment Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants