Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,15 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html)

## [Unreleased]

### Added
- **`ccx me`: what the prompt record says about the human at the keyboard.** Every other command describes the agent; this one reads only what you typed and counts it (`ccx.me.v1`, no model, no judgment): working hours and median day start/end, the longest streak and every break of 2+ idle days, and per month the prompt length, CJK / question / command / late-night shares, workspaces touched per day and new ones started. Steering is measured too: slash commands, `/rewind` read as "I take that back" (how many seconds after the prompt, and how often the same prompt comes back edited — longer or shorter), near-verbatim resends within the hour, and how often consecutive prompts change workspace. `--term name=regex` trends any habit of speech by month, so ccx ships no word list. The source is prompt history, not session files: history outlives session cleanup, so a 14-month profile of 21k prompts takes 2 s where a session scan takes 100 s and can only see the transcripts still on disk (on one store, January: 1,953 prompts on record, 332 with a transcript). A day runs 05:00-05:00, so a night session is not split into two days. (dogfood 2026-09-30)
- **`log -C/--context N` shows what a match interrupted.** With `--kind` or `--match`, keeps N records on each side of every match from the same transcript, flagged `"context": true` (text view: `ctx <kind>`); `records_matched` and the aggregates still count matches only. An interrupt alone reads "turn aborted: interrupted"; with `-C 2` it shows what the agent was doing and what the human typed next. (dogfood 2026-09-30)
- **`ccx doctor` checks that the record is whole.** Two records of the same work have different lifetimes: Claude Code deletes transcripts older than `cleanupPeriodDays` (default 30, by file mtime) and never touches `history.jsonl`, so a store can lose months without one error — every command just returns fewer sessions. `doctor` now reports the retention setting (warning when it is the default, or 0) and compares the sessions named in prompt history against the transcripts on disk, per provider and month, listing each month where more than a tenth of the prompts have no transcript (`2026-01 1953 prompts, 332 on disk; 77 sessions, 7 on disk`). Codex is checked too, including `archived_sessions/`. 0.2 s; no transcript is read. (dogfood 2026-09-30: a five-month-old hole nobody could see from inside ccx)

### Fixed
- **`log --kind user_prompt` counted subagent briefs as human prompts.** A subagent's task prompt is user-role text written by the parent agent; inline (`isSidechain`) or in a `subagents/` file it was classified `user_prompt`, so "the humans in the loop" and `metrics.user_prompts` / `days[].user_prompts` ran a fifth to a half high (3,079 of 16,829 on one store; in a month whose parent transcripts were gone, the surviving subagent files made up most of the "prompts"). They are now kind `agent_prompt`. `insight` already excluded them and is unchanged.
- `log --since X` without `--until` was rejected ("both required"); "since X" now means until now, in `log` and `insight`, and the reconstructed `insight` command no longer prints an empty `--until`.

## [0.17.0] - 2026-09-14

### Added
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ ccx reads session files from `~/.claude/`, `~/.codex/`, and `~/.grok/` and gives
- **Context trace** — `ccx trace` emits evidence for Context Folding
- **Time-sliced logs** — `ccx log` cuts through long-running session JSONL by timestamp
- **Window digest** — `ccx insight` answers "what was worked on this week, everywhere": per session the human's prompts, the agent's final answer, edits, commits, interventions and cost (with coverage), rolled up by workspace, day, provider and model; JSON for skills, HTML cockpit for humans
- **The human's profile** — `ccx me` reads only what you typed (prompt history, which outlives session cleanup): working hours, streaks and breaks, rewinds and resends, workspaces held per day, any phrase trended by month with `--term`
- **Provider filter** — `--provider cc`, `cx`, or `gx` on any command
- **Date filter** — `--after 2026-03-01 --before 2026-04-01`
- **Keyboard shortcuts** — `j/k` scroll, `/` search, `z` fold, `r` refresh, `d` theme
Expand Down Expand Up @@ -85,6 +86,7 @@ ccx log --scope yesterday --tz +8 --all --json # Time-sliced log evidence
ccx log --scope today --all --kind user_prompt # Every human prompt today, across sessions
ccx insight --scope week --all --json # Per-session digest of the week: prompts, answers, edits, cost
ccx insight --scope month --all # Same digest as an HTML cockpit (ccx web -> /insights)
ccx me --tz +8 # How you work: hours, streaks, rewinds, workspaces, by month
ccx search "auth bug" # Search across sessions + memory
ccx search --content -w --sort first goose # Whole-word content hits, earliest first
ccx search --content -w --hits goose # Every mention, quoted + anchored (time, session, message id)
Expand Down
15 changes: 14 additions & 1 deletion docs/schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ they do not know.
| `ccx.turn.v1` | `ccx trace --turn N` | One turn with full step evidence, plus the sidechain entries that turn references, plus warnings. |
| `ccx.trace.v2` | `ccx trace --full` | Complete evidence bundle: all turns/steps, sidechains, git correlation, workspace context, `related` sessions, stats, warnings. Large. |
| `ccx.related.v1` | `ccx related --json`, `GET /api/related/<project>/<session>` | The anchor session's connections to the other sessions of its workspace: `related[]` of `{session_id, provider, summary, start, end, strength, relations[]}`, plus `total`/`shown`. Each relation is `{kind, count?, paths?, evidence[], truncated?}`; evidence items are `{session_id, message_id, time, path?, quote?}`. Kinds: `forked_from`/`fork_of`, `mentions`/`mentioned_by`, `handoff_from`/`handoff_to`, `builds_on`/`built_on_by`, `overlaps`, `previous`/`next`. Strength is `strong`/`medium`/`weak`. |
| `ccx.log.v1` | `ccx log --json` | Time-scoped records across sessions with pre-computed `days[]` / `providers[]` / `workspaces[]` aggregates. Record `kind` is provider-normalized: `user_prompt` and `assistant_message` are the visible conversation only (Claude command markers/echoes/notifications are `command`/`command_output`/`notification`, injected meta is `meta`, compaction carriers `compact_summary`; Codex 0.147 raw `response_item` messages are `model_input`/`model_output` and duplicated legacy events `legacy_message`). `tool_call` records carry `tool`, `path` (when the call named one) and a `Tool: argument` `text`. With `--kind`/`--match`, `metrics.records` stays scope-wide, `metrics.records_matched` reports the narrowed count before any limit, and `truncated` is measured against the matched count. |
| `ccx.log.v1` | `ccx log --json` | Time-scoped records across sessions with pre-computed `days[]` / `providers[]` / `workspaces[]` aggregates. Record `kind` is provider-normalized: `user_prompt` and `assistant_message` are the visible conversation only (a subagent's brief — user-role text on a sidechain or in a `subagents/` file, written by the parent agent — is `agent_prompt`; Claude command markers/echoes/notifications are `command`/`command_output`/`notification`, injected meta is `meta`, compaction carriers `compact_summary`; Codex 0.147 raw `response_item` messages are `model_input`/`model_output` and duplicated legacy events `legacy_message`). `tool_call` records carry `tool`, `path` (when the call named one) and a `Tool: argument` `text`. With `--kind`/`--match`, `metrics.records` stays scope-wide, `metrics.records_matched` reports the narrowed count before any limit, and `truncated` is measured against the matched count. `-C N` adds up to N neighbors per match from the same transcript, flagged `"context": true`; they are returned but never counted in `records_matched` or the aggregates. |
| `ccx.insight.v1` | `ccx insight --json` | The window digest: `sessions[]` (one per logical session — a Claude session and its subagent files fold together) with `prompts[]` (`{time, line, text}`, first 6 / last 2, `prompts_omitted`), `final_answer`, `edits` + `edited_paths[]` (capped, `edited_paths_truncated`), `commits`, `interrupts`, `denials`, `tokens`, `cost_usd`, `unpriced_tokens`, `cost_status`, `model`, `summary`, `relation`; rollups `workspaces[]` (with `session_ids[]`), `days[]`, `providers[]`, `models[]`; `metrics` including `cost_coverage` (priced tokens / all tokens), `priced_sessions`, `unpriced_sessions`. `records[]` only with `--records`. Cost and tokens are whole-session figures from the provider's accounting, never sliced to the window. |
| `ccx.me.v1` | `ccx me --json` | The human's profile, counted from prompt history alone (no session files, no model): `sources[]` (the history files read), `span` (first/last, prompts, `calendar_days`, `active_days`, `providers[]`), `tempo` (`hours[24]` by clock hour, weekday/weekend prompts per active day, `median_day_start`/`median_day_end`, `longest_streak`, `breaks[]` of 2+ idle days), `months[]` (prompts, active days, workspaces, `new_workspaces`, `median_workspaces_per_day`, `median_chars`, `cjk_share`/`question_share` over prose, `command_share`/`late_share` over all prompts, `median_day_end`, `terms{}`), `commands[]`, `repeats[]` (short prompts typed at least twice), `openers[]` (first word), `rewinds` (`count`, `paired`, `resent`, `resent_longer`/`resent_shorter`, `median_seconds_before`), `resends`, `workspaces` (`total`, `lifespans` by active days, `switches`, `top[]`), `terms[]` (one per `--term`). Shares are fractions 0-1. A day runs 05:00-05:00 (`tempo.day_start_hour`), so clock strings past `24:00` are after midnight. Workspace figures cover Claude Code only: Codex history records no workspace. |

## Versioning policy

Expand Down Expand Up @@ -76,6 +77,18 @@ correction/sentiment/importance flags — v1 shipped keyword-based
user pushback. Interpretation belongs to the consumer reading
`user_text` with comprehension.

### Two records of the human

Session files and prompt history are different records with different
lifetimes. `log` and `insight` read session files. Claude Code deletes
those after `cleanupPeriodDays` (default 30, by file mtime) and never
touches `history.jsonl`; on one store that ran on the default until May,
January has 1,953 prompts on record and a transcript for 332 of them.
`me` reads history only, so its counts are whole where the
session-derived ones have holes. `ccx doctor` lists the holes. Do not
reconcile the two records by adding them; a prompt in a surviving
session is in both.

### Evidence budgets

Text fields are bounded (user text 2000 runes, narration 1200) with
Expand Down
199 changes: 199 additions & 0 deletions internal/cmd/coverage.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,199 @@
package cmd

import (
"encoding/json"
"fmt"
"io/fs"
"os"
"path/filepath"
"sort"
"strings"
"text/tabwriter"
"time"

"github.com/thevibeworks/ccx/internal/me"
)

// Transcript coverage: does the session store still hold what the
// prompt history says happened?
//
// The two records have different lifetimes. Claude Code deletes
// transcripts older than cleanupPeriodDays (default 30, by file mtime)
// and never touches history.jsonl, so a store can lose months without a
// single error: every command simply returns fewer sessions. History
// names the session each prompt was typed into, which makes the loss
// countable without reading a transcript.

const (
coverageMinPrompts = 20 // smaller months are noise, not a gap
coverageFloor = 0.9 // below this share on disk, a month is a gap
defaultCleanupDays = 30
)

// coverageRow is one provider-month of prompt history, split by
// whether the transcript of the prompt's session is on disk.
type coverageRow struct {
Provider string
Month string
Prompts int // prompts that name a session
PromptsOnDisk int
Sessions int
SessionsOnDisk int
Unattributed int // prompts with no session id: cannot be checked
}

func (r coverageRow) gap() bool {
return r.Prompts >= coverageMinPrompts && float64(r.PromptsOnDisk) < coverageFloor*float64(r.Prompts)
}

// transcriptCoverage buckets prompts by provider and month and checks
// each named session against the ids of the transcripts on disk.
func transcriptCoverage(prompts []me.Prompt, onDisk map[string]bool, loc *time.Location) []coverageRow {
type acc struct {
row coverageRow
sessions map[string]bool
}
byKey := map[string]*acc{}
for _, p := range prompts {
if p.Time.IsZero() {
continue
}
month := p.Time.In(loc).Format("2006-01")
key := p.Provider + "\x00" + month
a := byKey[key]
if a == nil {
a = &acc{row: coverageRow{Provider: p.Provider, Month: month}, sessions: map[string]bool{}}
byKey[key] = a
}
id := strings.ToLower(p.SessionID)
if id == "" {
a.row.Unattributed++
continue
}
a.row.Prompts++
if onDisk[id] {
a.row.PromptsOnDisk++
}
if !a.sessions[id] {
a.sessions[id] = true
a.row.Sessions++
if onDisk[id] {
a.row.SessionsOnDisk++
}
}
}
rows := make([]coverageRow, 0, len(byKey))
for _, a := range byKey {
rows = append(rows, a.row)
}
sort.Slice(rows, func(i, j int) bool {
if rows[i].Provider != rows[j].Provider {
return rows[i].Provider < rows[j].Provider
}
return rows[i].Month < rows[j].Month
})
return rows
}

// transcriptIDs lists the session ids that have a transcript on disk.
// Claude Code names the file after the session; Codex ends the rollout
// file name with it, in sessions/ and in archived_sessions/.
func transcriptIDs(claudeHome, codexHome string) map[string]bool {
ids := map[string]bool{}
if claudeHome != "" {
matches, _ := filepath.Glob(filepath.Join(claudeHome, "projects", "*", "*.jsonl"))
for _, m := range matches {
ids[strings.ToLower(strings.TrimSuffix(filepath.Base(m), ".jsonl"))] = true
}
}
if codexHome != "" {
for _, dir := range []string{"sessions", "archived_sessions"} {
_ = filepath.WalkDir(filepath.Join(codexHome, dir), func(path string, d fs.DirEntry, err error) error {
if err != nil || d.IsDir() {
return nil // a missing or unreadable dir holds no transcripts
}
name := strings.TrimSuffix(d.Name(), filepath.Ext(d.Name()))
if len(name) >= 36 {
ids[strings.ToLower(name[len(name)-36:])] = true
}
return nil
})
}
}
return ids
}

// claudeRetention reads cleanupPeriodDays from Claude Code's user
// settings. set is false when the file or the key is absent, which
// means the 30-day default is in force.
func claudeRetention(claudeHome string) (days int, set bool, err error) {
data, err := os.ReadFile(filepath.Join(claudeHome, "settings.json"))
if err != nil {
if os.IsNotExist(err) {
return defaultCleanupDays, false, nil
}
return 0, false, err
}
var settings struct {
CleanupPeriodDays *int `json:"cleanupPeriodDays"`
}
if err := json.Unmarshal(data, &settings); err != nil {
return 0, false, err
}
if settings.CleanupPeriodDays == nil {
return defaultCleanupDays, false, nil
}
return *settings.CleanupPeriodDays, true, nil
}

// reportCoverage prints the gaps and returns one warning per provider
// that has any. Nothing is printed when the store matches history.
func reportCoverage(rows []coverageRow) []string {
var warnings []string
for _, provider := range []string{"claude-code", "codex"} {
var gaps []coverageRow
var unattributed, missingPrompts, missingSessions int
var first, last string
for _, r := range rows {
if r.Provider != provider {
continue
}
unattributed += r.Unattributed
if !r.gap() {
continue
}
gaps = append(gaps, r)
missingPrompts += r.Prompts - r.PromptsOnDisk
missingSessions += r.Sessions - r.SessionsOnDisk
if first == "" {
first = r.Month
}
last = r.Month
}
if len(gaps) == 0 {
fmt.Printf("[OK] %s transcripts: every month of prompt history is on disk\n", provider)
} else {
fmt.Printf("[!!] %s transcripts: prompt history names sessions that are not on disk\n", provider)
w := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', tabwriter.AlignRight)
fmt.Fprintln(w, " MONTH\tPROMPTS\tON DISK\tSESSIONS\tON DISK\t")
for _, r := range gaps {
fmt.Fprintf(w, " %s\t%d\t%d\t%d\t%d\t\n", r.Month, r.Prompts, r.PromptsOnDisk, r.Sessions, r.SessionsOnDisk)
}
w.Flush()
// Claude Code keeps transcripts under projects/, Codex under sessions/.
dir := "projects"
if provider == "codex" {
dir = "sessions"
}
warnings = append(warnings, fmt.Sprintf(
"%s: %d prompts in %d sessions (%s to %s) have no transcript on disk. "+
"What was typed survives in prompt history (ccx me, ccx search --content); what the agent did does not. "+
"A backup restores it: rsync -a --ignore-existing <backup>/%s/ <home>/%s/",
provider, missingPrompts, missingSessions, first, last, dir, dir))
}
if unattributed > 0 {
fmt.Printf(" %d %s prompts predate session ids in history and cannot be checked\n", unattributed, provider)
}
}
return warnings
}
Loading
Loading