Conversation
Components and Storybook stories only — no route, no server function, no schema. The data contract in components/ads/types.ts is what server/ads.ts will have to satisfy. Fixtures are real captured ChatGPT ad payloads from Elmo's own tracked prompts, so the layouts are exercised against the shape and sparsity the live data actually has rather than a tidier invention.
Drops the three variants for the one that was chosen, with the auction board folded in after the two ranked lists it cross-tabulates. Google AI Mode ads join ChatGPT's, which the page has to handle rather than average away: the two surfaces have different payload shapes, AI Mode often gives a display name where a domain belongs, and one of them is currently silent. So advertisers key on an explicit advertiserKey with a nullable domain, and each platform gets its own ad rate. Empty is the common case here and splits four ways — no ad-capable platform, no answers, no ads ever, and ads that stopped. The last is the one that matters: reading it as "nobody is buying" when a scraper lost the surface is the expensive mistake.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The cliff is 2026-08-25, not the 28th, and ads were not the only thing lost: every field that only exists when the collector completes a real browsing turn went with them. web_search_triggered and search_sources came back on 09-01; model, web_search_query and ads did not. Which means ChatGPT query fan-out has been answering "unavailable" for three weeks — a live regression, same root cause, worth the same ticket. Ruled out: our own code (the only provider change in the window landed a day after the cliff and is a pure refactor), OpenAI pulling ads (they expanded), and geo (a US-pinned commercial probe rendered the shopping carousel and still returned no ads).
Follow-up: traced the ChatGPT blackout, and it's worse than "ads stopped"Answering the three questions directly. Did we change something? No.The only change to The real cliff is 2026-08-25, and ads weren't the only casualtyFive fields collapsed on one day, while Changelogs and news: nothing announced, because the field was never documented
Ruled out geo too — and found a knob we should turn anywayChatGPT ads were US-only for most of 2026 and we've never pinned a country, so this looked like the answer. It isn't. The sharpest probe: We should send The ask
Also worth asking whether Full write-up in |
Not our extractor: August payloads carried web_search_query under the same name we read, nothing replaced it, and reportedWebQueries already degrades to the unavailable sentinel rather than claiming zero fan-out. Not BrightData alone either — DataForSEO's fan_out_queries is null on a live probe too, which points at a ChatGPT UI change both DOM readers lost rather than one vendor's bug. Oxylabs reads the SSE stream instead of the DOM and still has both the queries and the model, so moving the ChatGPT target there restores fan-out with no code change. It does not bring ads back: the stream carries no ad markers at all.
Query fan-out: broken on their end, not ours — and recoverable todayI over-attributed this to BrightData in the last comment. The fan-out half is broader than that, and there's a fix already written. Not our extractor
Not BrightData alone eitherDataForSEO's ChatGPT LLM Scraper returns Oxylabs still has itOxylabs reads ChatGPT's SSE conversation stream rather than the rendered DOM, and the data is right there: "metadata": {
"search_model_queries": { "queries": ["best noise cancelling headphones 2026 …"] },
"resolved_model_slug": "gpt-5-6"
}A live Oxylabs run on 09-16 returned This also explains the shape of the outage: Ads don't come back this wayThe SSE stream carries no ad markers at all — no Revised actions
|
Ads did not stop: trackers have ChatGPT ad penetration at ~51% of US replies and holding, so our zero is a capture failure rather than the market moving. Eligibility is what makes it invisible — ads ride the conversation stream as a type:"ads" event on /backend-anon/, served only to free or logged-out sessions in a handful of English-speaking markets, and nothing in the payload says when a session fails that test. Cloro sells this deliberately behind include.ads. Our cloro.ts already builds that include object for ChatGPT and asks for two of its six fields; ads and shopping are both there and both omitted. The same provider already sends include.searchQueries, so one flag would fix the fan-out regression too. BrightData has no equivalent flag — the echoed input confirms the whole accepted set — and its ads field is undocumented, which is worth knowing before building on it.
Correcting the record: ads never stopped, and there is a flag — on CloroThree answers, and the first one revises what I said above. 1. Yes, plenty of people have noticed — ads are at ~51% of US repliesI was wrong to frame this as "ChatGPT ads stopped". They didn't:
So our zero across ~5,000 runs is a capture failure, not the market. Two caveats worth carrying into the design: the rate is genuinely volatile (26.5% in late May → 0.05% → ~51% by July, "still visibly tuning") and it's an English-speaking-market product, "roughly zero" elsewhere. What it takes to be served an ad — and why this is invisible to usAds ride the conversation SSE stream itself — a Eligibility is narrow, and is probably the mechanism:
A scraper that drifts into an authenticated session, a paid tier, or a non-US exit returns a perfectly valid answer with no ads, forever, and nothing in the payload says why. That also explains Oxylabs: its SSE stream had no ad event, but that session's 2. Per-provider ad support
No 3. The flag:
|
ChatGPT was the only model producing fan-out at all — every other tracked model contributes zero, by design or by provider — so its drop from 58% to 1.1% of runs leaves the page showing its empty state for every cloud brand rather than merely thinning out. Nothing in our code is broken: the extractor reads the right fields, the sentinel records "searched but would not say what", and the page says so in as many words. The data just stops arriving. Cloro charges one +2 surcharge for any combination of raw response, fan-out, ads and shopping. We already send searchQueries, so ads and shopping are free.
Cloro pricing, and exactly what's broken with fan-outAdding
|
| Model | Aug 1–24 runs | with fan-out | Sept 1+ runs | with fan-out |
|---|---|---|---|---|
| chatgpt | 3,245 | 1,895 (58%) | 5,054 | 58 (1.1%) |
| claude | 3,128 | 0 | 4,410 | 0 |
| google-ai-mode | 3,104 | 0 | 1,902 | 0 |
| google-ai-overview | 3,105 | 0 | 4,438 | 0 |
| gemini | 10 | 0 | 2,550 | 0 |
| perplexity | 9 | 0 | 10 | 0 |
Every other tracked model contributes zero both before and after, by design or by provider. So Query Fan-Out isn't partially degraded — it shows the empty state for every cloud brand. ChatGPT was carrying it alone and dropped to ~1%.
DataForSEO's fan_out_queries is null on a live probe too, so it isn't BrightData-specific.
The country test is still running
Running it as an A/B rather than more one-off samples: 8 high-commercial-intent prompts with country: "US" vs. the same 8 with no country, recording model / web_search_query / search_sources / shopping / ads per run. At the ~51% penetration trackers report, zero-of-eight would be p≈0.004 — decisive either way, and the control isolates country as the variable.
Incidental finding while running it: an 8-input batch snapshot has been >45 min on BrightData, against ~4 min for the single-input snapshots production sends. Not a production issue (we send one input per snapshot, under a 12-minute poll timeout), but worth knowing before anyone batches inputs to save on trigger overhead.
Tested:
|
| prompt | country | model | searched | web_search_query |
sources | shopping | ads |
|---|---|---|---|---|---|---|---|
| best project management software for remote teams | US | null | true | null | 3 | 0 | no |
| best noise cancelling headphones | US | null | true | null | 4 | 3 | no |
| best running shoes for beginners | US | null | true | null | 4 | 6 | no |
| best mattress for back pain | US | null | true | null | 3 | 4 | no |
| best travel credit card | US | null | true | null | 4 | 0 | no |
| best web hosting for a small ecommerce store | US | null | true | null | 4 | 0 | no |
| best crm for small business | US | null | true | null | 5 | 0 | no |
| cheapest car insurance quotes | US | null | true | null | 5 | 0 | no |
0 of 8. Against the ~51% US penetration trackers report, p ≈ 0.004 — not a sampling accident.
Two details that make this conclusive rather than merely negative:
- Three of the eight rendered a shopping carousel (3, 6 and 4 products). The collector reached the monetizable part of the answer and still returned no ad — so this isn't "the run failed", it's specifically the ad unit that's missing.
modelandweb_search_querystayed null on all eight. Same regression, unmoved by geo.
country is accepted and echoed back correctly; it just changes nothing that matters here.
Still send it. One field, doesn't fix this, but today the exit country is whatever BrightData picks — a variable worth removing now that ads serve in more markets.
A no-country control arm on the same eight prompts is still running; it can only confirm the negative, so it doesn't change the conclusion. I'll append it when it lands.
docs/ads-plan.md §5 updated (80c81bc).
GET /datasets/v3/scrapers returns full input and output schemas. The ChatGPT Search dataset declares 29 output fields and ads is one of them, alongside model and web_search_query — all three declared, all three now always null, which is a contract violation rather than a courtesy field quietly retired. Its whole input schema is url, prompt, country, index, require_sources, additional_prompt, web_search, geolocation; nothing requests ads or selects fields, and collect_by_url is its only mode. Of all 1,078 scrapers on the account exactly two declare an ads field, and they are the two we already run.
Every remaining parameter tested one snapshot each. Forcing web_search was the best hypothesis left — we set it on every run, a deviation from their default nobody had questioned — and it is not the cause: omitting it changes nothing, and setting it false still returns web_search_triggered true, so the flag does not control searching at all. Shopping rendered on most of the runs, so the collector keeps reaching the monetizable part of the answer. Against the ~51% US penetration trackers report, fourteen consecutive misses is p of about 0.00006. Also: geolocation needs numeric lat/lon/radius despite declaring type text, and the account has only a serp zone, so the Browser API route that would let us read the ads event off the stream ourselves is not provisioned.
Exhausted BrightData:
|
| variant | country | model | searched | web_search_query |
sources | shopping | ads |
|---|---|---|---|---|---|---|---|
web_search omitted (their default) |
US | null | true | null | 3 | 4 | no |
web_search: false |
US | null | true | null | 4 | 3 | no |
require_sources: true |
US | null | true | null | 5 | 3 | no |
geolocation NYC (40.7128,-74.0060,10000) |
US | null | true | null | 6 | 4 | no |
Forcing web_search was the best hypothesis left — we set it true on every run, a deliberate deviation from their default nobody had questioned, and it plausibly routes ChatGPT into a search-mode UI. It isn't the cause: omitting it changes nothing, and setting it false still returns web_search_triggered: true, so the flag doesn't even control whether a search happens.
Running total: 14 US-pinned runs, zero ads, shopping rendering on at least seven. Against ~51% US penetration, p ≈ 0.00006.
Two incidental findings
geolocationis not free text despite declaringtype: "text"— it needs numericlatitude, longitude, radiusor the crawler errors out.- The account has exactly one zone:
sdk_serp(typeserp). The Browser API / Web Unlocker route isn't provisioned.
What's actually left on BrightData
Driving chatgpt.com ourselves through Browser API / Web Unlocker and reading the type: "ads" event off the SSE stream — the way Oxylabs gets search_queries. That's a different product, a new zone, and a real integration, not a flag. It's also the only option that survives them breaking the parsed field again.
Versus Cloro, which returns ads deliberately behind include.ads, costs us nothing extra (we already pay the +2 for searchQueries), and fixes fan-out in the same call. That comparison hasn't changed — it's just better evidenced now.
docs/ads-plan.md §5 updated (aa6fbe5).
1 participant
Closes #168.
Plan in
docs/ads-plan.md, plus the page built as components and Storybook stories (Pages/Ads). No route, no server function, no migration —apps/web/src/components/ads/types.tsis the contractserver/ads.tswill have to satisfy. Nothing user-facing yet, so no changeset.We already have the data, and it is retroactive
BrightData's ChatGPT scraper returns an
adsobject on every run and we persist it verbatim inprompt_runs.raw_output. History goes back to 2026-05-11 — 5,900+ ad impressions across 400+ advertisers. Google AI Mode carries its ownadsarray and is live and growing today.Against Elmo's own tracked prompts: AirOps, Profound, Searchable and Ahrefs are buying against our exact prompt set, and we can read their ad copy. All four are already on the competitor list, so attribution works with no extra config.
ChatGPT ads stopped on 2026-08-28, and the data is nowhere else
I probed all three ChatGPT scrapers live against production credentials on 2026-09-16:
adsfield{"carousel_cards": null}answer_html— nobzrcdn.openai.com/tessera, noutm_medium=paid, no "Sponsored"raw_response, which is ChatGPT's own SSE conversation streamSo: not a parsing loss on our side, and not recoverable from another provider. The stored data points at the cause — ads only ever appeared on runs where BrightData also reported the
modelfield, and that session type collapsed at exactly the right moment:That reads as a BrightData capture regression rather than OpenAI withdrawing ads — a vendor ticket, not a code change. It doesn't block this: the May–August history earns the page on its own and AI Mode keeps it live. But it is exactly why the page must not render "0 ads" as "nobody is buying".
This should land on top of #711
Ads are an L1 extraction artifact in that PR's terms — read straight from
raw_output, identical for every tenant, with brand/competitor attribution applied at read time exactly ascategorizeDomainalready does for citations. So ads need no deriver, and more importantly no backfill job:extractRun()returns{ textContent, citations }and is the single extraction entry point. Ads become a third key.reprocessalready replaces a run's citations wholesale whenextractor_versionis stale.replaceAdImpressionsis the same five lines.EXTRACTOR_VERSION1 → 2 is detected on worker start and replays every stored run at low concurrency behind live processing.#711's own plan doc names this case: "a new deriver such as ads detection is a version bump plus the rows it needs, never a one-off script." On
maintoday it would instead need a bespoke chained job over 1.8 GB of TOAST.Rollup tables aren't needed for v1 and that's a measurement, not a preference:
ad_impressionsis ~9k rows / <10 MB againstcitations' 1.1M rows / 673 MB. When volume warrants it,rollup_ad_advertisersis one entry inrebuildRange's table list, oneINSERT … SELECT … GROUP BY, and aROLLUP_VERSIONbump. One wrinkle recorded in the plan: it must key onadvertiser_key, not domain — same reason competitor mentions key by name (#709), plus AI Mode genuinely has no domain on some rows.Performance
Reading ads out of
raw_outputper request is a non-starter:prompt_runsis 2 GB of TOAST for 74k rows, a single de-TOASTing 30-day aggregate takes ~800 ms warm, and the competitor-join version timed out at 120 s. A normalizedad_impressionstable with a covering index mirroringcitations_brand_created_analytics_idxis ~1% of the Citations row count, so it should be faster than the Citations page, not merely as fast.The page
Order: stats → platform strip → ad-rate trend → advertisers → contested prompts → auction board → movement → creatives.
The board sits after both ranked lists rather than leading. It's the densest thing on the page and reads far better once the names in it are familiar.
Platform filtering reuses the existing
FilteredListShellfilter bar withtrackedTargetsnarrowed to the newAD_CAPABLE_MODELS— same URL-backed control as every other page, and the filter and the denominator read the same list so they can't drift. Each advertiser row and creative card carries its platform's icon, because the same advertiser on both surfaces is two different buys.Empty is the common case
Ads are sparse, so "nothing here" gets four distinct answers rather than one: no ad-capable platform tracked, no answers in the window, no ads and none ever seen, and — the one that matters — ads that used to appear and have stopped. That last case carries a warning on the platform's row with the date of the last ad, because "nobody is buying" and "our scraper lost the surface" produce an identical empty table and mean opposite things.
Notes
aclkredirect in its URL field and sometimes a display name ("Avnet", "Amsive") where a host belongs, so rows key onadvertiserKeyand the domain-dependent affordances — favicon, host label, "track as competitor" — are conditional.paid-vs-earned-card.tsxis built but not placed — it's the only section needing acitationsjoin, and it earns less than the six that ship.Verification
pnpm lint,pnpm turbo check-types, and all unit suites pass (cli 32, config 106, deployment 47, lib 571, web 545).Open workspace in Conductor