Skip to content

feat(chat): chat history — index, admin UI, realtime (Phases 1–3)#8

Open
maksymhryzodub-prog wants to merge 4 commits into
mainfrom
feat/chat-history
Open

feat(chat): chat history — index, admin UI, realtime (Phases 1–3)#8
maksymhryzodub-prog wants to merge 4 commits into
mainfrom
feat/chat-history

Conversation

@maksymhryzodub-prog

Copy link
Copy Markdown
Contributor

Replaces mazda-ai's "chats" with durable, browsable chat history across all
channels (web/Bridle, Telegram, Slack). Message content stays in the agents'
append-only S3 JSONL (source of truth); this adds a queryable Postgres index
so chats are listable/filterable without fanning out to agent pods and remain
available when a pod is down.

Architecture

  • Postgres = index only (metadata + monotonic counts). Transcript content is read
    live from S3 via IFileGateway.
  • Index populated two ways: S3 reconciliation (ChatSyncService) + realtime
    session_activity over the always-on agent↔hub socket.
  • The agent runtime stays host-agnostic (no DB dependency) — see companion runtime PR.

Phase 1 — API core (chat slice)

  • ChatSession + ChatFeedback Prisma models (FK cascade to Agent) + migration.
  • ChatSyncService reconciles from S3 (IFileGateway.listdata/sessions/*.jsonl);
    env-gated setInterval (CHAT_SYNC_INTERVAL_SEC) + manual POST /chats/sync.
    Counts are monotonic (seed/raise, never lower — compaction shrinks the file).
  • Endpoints (Owner/Admin): GET /chats, /chats/:id, /chats/:id/messages, POST /chats/sync.
  • Shared TranscriptReaderService (agent/file): surfaces compaction summary events
    as a collapsed marker instead of a silent gap, and filters synthetic loop-control
    noise (continuation prompts + the duplicated pre-cutoff partial). Bridle's transcript
    endpoint now delegates to it (byte-identical).

Phase 2 — Admin UI (chat slice)

  • Chats list: search / channel / archived / internal filters, server pagination, Sync.
  • Chat detail: meta header, tail-first transcript with scroll-up "load older",
    "show tool events" debug toggle, collapsible summary marker, assistant markdown.
  • Sidebar "Chats" nav + a "Chats" tab on the agent detail page (scoped by agentId).
  • Regenerated admin + app OpenAPI clients (ChatsService).

Phase 3 — Realtime

  • IChatGateway.recordActivity: atomic increment (updateMany, race-free), dedup by
    eventId, create fallback with a retry on the concurrent-create unique violation.
    Realtime owns the monotonic counts.
  • bridleAgentWs.handler subscribes to session_activityrecordActivity, taking
    agentId from the authenticated socket (never the payload).
  • Fixed a DI cycle (Bridle → Chat → File → Bridle) with forwardRef.

On the "source of truth"

Compaction windows long chats down (the JSONL is a rolling window, not a full ledger).
Product decision: history = gist, not a verbatim log — compaction is untouched and
the summary is surfaced instead.

Verification

  • tsc --noEmit clean. Unit specs: chat gateway (reconcile monotonic counts;
    recordActivity create/increment/dedup/create-race), transcript reader (hygiene,
    summary, pagination), reconcile.
  • Live e2e vs local Postgres: POST /chats/sync indexes S3 sessions; emitting
    session_activity updates the row live (count/dedup/freshness) with no Sync.

Deploy notes

  • Run the migration (migrate:prod) — additive (two new tables, FKs, indexes).
  • Set CHAT_SYNC_INTERVAL_SEC to enable periodic reconcile (else chats index only on
    manual Sync).
  • Realtime updates require agents on runtime ≥ 0.27.0 (companion PR); without it,
    reconcile still populates the list.

Not included (later phases)

Insights/summaries (Phase 4), feedback + export (Phase 5), app "my history" (Phase 6).

maksymhryzodub-prog and others added 4 commits July 15, 2026 15:16
Add a `chat` slice: a Postgres INDEX over agents' S3 JSONL session
transcripts, so chats are listable/filterable without fanning out to
agent pods and remain available when a pod is down. Message content
stays in S3 (source of truth); Postgres only indexes.

- ChatSession + ChatFeedback models (FK cascade to Agent) + migration.
- ChatSyncService reconciles the index from S3 (IFileGateway.list).
  Counts are monotonic lifetime totals — reconcile seeds/raises, never
  lowers (compaction shrinks the file; realtime will own the true total).
- Endpoints (Owner/Admin): GET /chats, /chats/:id, /chats/:id/messages,
  POST /chats/sync.
- Shared TranscriptReaderService (agent/file): surfaces compaction
  `summary` events as collapsed markers instead of a silent gap, and
  filters synthetic loop-control noise (continuation prompts + the
  duplicated pre-cutoff partial assistant chunk). The bridle transcript
  endpoint now delegates to it (byte-identical: user/assistant, no
  hygiene), removing ~90 lines of duplicated reader logic.
- 11 unit specs (reader hygiene/summary/pagination, monotonic counts,
  reconcile); tsc clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Admin slice to browse the Phase 1 chat index: a filterable list, a chat
detail with a tail-first transcript, and a "Chats" tab on the agent page.

- slices/chat: Pinia store (ChatsService + {success,data} unwrap); chats
  list page + ChatListProvider (search / channel / archived / internal
  filters, server pagination, Sync button, agent-scopable via :agent-id);
  chat detail + ChatDetailProvider (meta header, scroll-up "load older"
  paging, "show tool events" debug toggle); read-only ChatMessageBubble
  (assistant markdown via bridle's renderMarkdown, collapsible summary
  marker, compact tool_call/tool_result blocks).
- Sidebar "Chats" nav item (+ MessageSquare in iconMap).
- Agent detail: "Chats" tab rendering <ChatListProvider :agent-id>.
- Regenerate admin + app OpenAPI clients from the new chat endpoints
  (ChatsService: getChats/getChat/getChatMessages/syncChats) — additive only.

Verified: nuxt build green, store tsc clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The hub records every session_activity signal from the agent runtime into
the chat index, so the admin list updates live (no reconcile wait).

- IChatGateway.recordActivity: atomic increment via updateMany (race-free,
  no read-modify-write), dedup by eventId (NOT-filter), create fallback with
  a retry on the concurrent-create unique violation. Realtime owns the
  monotonic counts.
- bridleAgentWs.handler subscribes to `session_activity` -> recordActivity,
  taking agentId from the authenticated socket (never the payload).
- BridleModule imports ChatModule; ChatModule forwardRefs File/AgentModule to
  break the Bridle -> Chat -> File -> Bridle DI cycle.
- recordActivity specs (create / increment / dedup / create-race).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CI (`bun run lint` = `eslint --fix`) failed on no-redundant-type-constituents:
`insights: unknown | null` → `unknown` (unknown already subsumes null). This is
the only non-auto-fixable error; the prettier violations CI was silently fixing
each run are now applied to the committed chat slice + touched files, so the
source is lint-clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant