Skip to content

task: make the assistant's existing telemetry reportable (GA4 custom dimensions, gtag after GTM swap, unanswered-question review) #2307

Description

@GigaHierz

The docs assistant already emits GA4 events and already logs questions server-side. Neither is reportable yet, for two separate reasons. Both are configuration, not code.

What already exists

Client, widget.js in celo-org/docs-ai-assistant — seven events through the page's existing window.gtag:

Event Parameters
assistant_question answered, escalated, truncated
assistant_error from_api, status
assistant_citation_click href
assistant_opened, assistant_escalate, assistant_new_chat, assistant_copy

Server, app/api/chat/route.tslogQuestion() pushes {question, model, citedUrls, answered, timestamp, refused} onto the Upstash Redis list docs-assistant:questions, trimmed to the most recent 10,000.

Gap 1: the event parameters are not registered as custom dimensions

GA4 drops event parameters that are not registered. The events arrive; answered, escalated, truncated, from_api, status and href cannot be broken out in any report until they are added as event-scoped custom dimensions in the GA4 property for G-0CXEKQ81V2.

answered is the one that matters — it is the share of questions the assistant could not answer from the docs, which is the docs-gap rate.

Gap 2: track() is a silent no-op without window.gtag

function track(name, params) {
  if (typeof window.gtag === 'function') window.gtag('event', name, params || {});
}

window.gtag is provided today by integrations.ga4 in docs.json. #2300 replaces that with integrations.gtm. GTM's Google tag should still define the global, but this exact configuration has already produced one silent failure — the container shipped with its own ID in the Tag ID field, so it loaded, fired, errored nowhere, and sent nothing to GA4. Verify rather than assume.

Gap 3: nothing reads the Redis list

The list is the actual docs-gap signal — the uncited questions in it are the pages that need writing — and there is no routine for reading it. A monthly pull of entries where answered: false would close the loop that #2250 used to justify building the assistant instead of paying for Mintlify Pro.

Acceptance criteria

  • The six assistant parameters registered as event-scoped custom dimensions in GA4
  • After feat(analytics): route GA4 through GTM and instrument engagement events #2300 merges, typeof window.gtag === 'function' confirmed on a published page, and one assistant_question event seen in GA4 Realtime
  • A stated cadence and owner for reviewing docs-assistant:questions where answered: false
  • ANALYTICS.md records whichever of these lands

Refs #2250, #2286, #2300.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions