Follow service logs during startup and query debugging

[danieljhkim]

Task

DANI-10284 — Follow service logs during startup and query debugging

Description

At origin/main 3131dd0, internal/cli/logs.go:115-139 renders one snapshot and exits; only --lines controls output. Add logs --follow to observe startup/query progress without repeatedly invoking the CLI or reconstructing service paths. Reuse existing service selection and path mapping; require no Hadoop/Hive discovery. Source-based feature gap, not a claimed runtime bug. DANI-10201 added selection/tail length and10207 optimized snapshot reads; all-status follow/streaming search found no existing coverage. Coordinate with active DANI-10280 lint cleanup using normal task selectors, not manual overlapping edits.

Acceptance Criteria

• logs hive --follow --lines 20 prints the initial suffix then new content from both Hive logs with clear source attribution.
• Files absent at startup are followed once created; truncation and replacement continue predictably without replaying discarded history.
• Cancellation exits promptly and resource/memory use stays bounded; finite snapshot mode retains existing behavior.
• Deterministic file append, rotation/replacement, missing-file, and cancellation tests plus a real built-CLI demonstration validate behavior.
• Help/docs explain follow semantics and documented repository checks pass.

Execution Summary

Click to expand

Outcome: success
Changes:

• Added logs --follow, which renders the existing per-file suffix first and then polls the same selected service log paths for bounded new-content chunks with source labels.
• Tracks file identity and offsets, following late-created files and restarting at byte zero after truncation or replacement without replaying discarded initial history.
• Added deterministic append/both-Hive-log attribution, late-file, truncate, replacement, cancellation, and command-cancellation tests; documented follow behavior in README and command help.
  Assessment: Follow state is bounded to one file identity and offset per selected log, opens files only per poll to avoid descriptor retention, and caps each read at 32 KiB.
  Design weaknesses / risks:
• Severity: low. Polling has up to 100 ms delivery latency and source headers are emitted per bounded chunk; this intentionally avoids platform-specific watcher dependencies.
  Validation:
• go test ./internal/cli
• make test
• go vet ./internal/cli
• git diff --check
• Built /tmp/local-data-dani-10284 with go build -o /tmp/local-data-dani-10284 ./cmd/local-data; a real logs hive --follow --lines 1 run printed initial suffixes and appended content from both Hive logs with source labels. The Go command emitted a non-fatal sandbox denial while attempting to write its module stat cache, but the build and demonstration completed successfully.

Validation

• Not reported

Branch Freshness

• Base ref: origin/main
• Head ref: orbit/DANI-10284-6a9cd3a9
• Behind base: 0
• Ahead of base: 1