Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
00c2c14
feat: DAG-based post-processing + taxonomy handler
cedricvidal Jun 9, 2026
e9bf6ad
feat: add taxonomy API route and archive bundling (Phase 3)
cedricvidal Jun 9, 2026
2273286
fix: taxonomy handler Dockerfile and docker-compose dev setup
cedricvidal Jun 9, 2026
4507fb8
feat: add taxonomy tab to portal run detail view
cedricvidal Jun 9, 2026
3756ccc
refactor: move handler registration from migration to worker registra…
cedricvidal Jun 9, 2026
d02ec68
refactor: YAML-driven handler registration via scheduler endpoint
cedricvidal Jun 9, 2026
02900bd
feat: scheduler triggers report generation when handler DAG drains
cedricvidal Jun 9, 2026
c6cd9b2
feat(taxonomy): add dev CLI, API-based ATIF fetch, and default to gpt…
cedricvidal Jun 10, 2026
07127ae
chore: remove taxonomy changes (split to #1053)
cedricvidal Jun 24, 2026
ca11ff0
Merge remote-tracking branch 'origin/main' into cedricvidal-microsoft…
cedricvidal Jun 24, 2026
d1ea709
merge: bring in origin/main
cedricvidal Jun 25, 2026
e87bb85
Merge remote-tracking branch 'origin/main' into cedricvidal-microsoft…
cedricvidal Jun 26, 2026
8901933
Merge remote-tracking branch 'origin/main' into cedricvidal-microsoft…
cedricvidal Jun 29, 2026
688fbb5
fix(scheduler): allow autoBackfill to re-dispatch runs with outdated …
cedricvidal Jul 1, 2026
d3e5d78
Merge remote-tracking branch 'origin/main' into cedricvidal-microsoft…
cedricvidal Jul 1, 2026
7288b65
fix(scheduler): ignore stale handler completion notifications
cedricvidal Jul 1, 2026
5c64dea
fix(post-processing): enforce worker-entrypoint idempotency
cedricvidal Jul 1, 2026
fe5b9ee
docs(post-processing): clarify enqueue and claim boundaries
cedricvidal Jul 1, 2026
87df1b0
perf(scheduler): cache handler topology with short TTL
cedricvidal Jul 1, 2026
a56c0be
perf(scheduler): raise handler topology cache TTL to 5min, make it en…
cedricvidal Jul 1, 2026
b55299c
docs: call out single-replica limitation of the handler topology cache
cedricvidal Jul 1, 2026
ab696a2
feat(scheduler): cap notify/register request body size (64 KB)
cedricvidal Jul 1, 2026
726e0cd
fix(post-processing): let worker claim scheduler-queued runs
cedricvidal Jul 1, 2026
8b86d2d
Merge remote-tracking branch 'origin/main' into cedricvidal-microsoft…
cedricvidal Jul 8, 2026
0531a0a
chore(db-migrations): renumber handler-DAG migrations to 025-027
cedricvidal Jul 10, 2026
90dfca4
Merge origin/main into cedricvidal-microsoft/pp-dag-orchestration
cedricvidal Oct 11, 2026
2ca370b
Merge remote-tracking branch 'origin/feat/portal-copilot-ai' into ced…
cedricvidal Oct 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions ENV_VARIABLES.md
Original file line number Diff line number Diff line change
Expand Up @@ -926,12 +926,37 @@ conflicts closed.

How often the post-processor dispatcher polls for completed runs needing post-processing. This is a **backfill/catch-up** mechanism — the primary dispatch path is event-driven (coder workers enqueue directly on run completion). The 30s default keeps idle RU consumption low while still catching missed events or version-upgrade backfills within a reasonable window. Reduce temporarily for large backfills.

### SCHEDULER_HANDLER_CACHE_TTL_MS
**Default:** `300000`
**Type:** integer (milliseconds)

TTL for the DAG dispatcher's in-memory cache of handler topology (the `services` collection, `type: "post-process-handler"`). `loadHandlers()` runs on the notify/poll hot path (2–3 reads per notify plus every poll cycle) but topology only changes on deploys, so caching it eliminates nearly all of those reads (meaningful on CosmosDB where each query costs RUs). Cache coherence comes primarily from `HandlerDispatcher.registerHandler` invalidating the cache the moment topology changes — the scheduler is a singleton (`replicas: 1`, `Recreate`) and the sole writer of the collection — so this TTL is only a coarse backstop for out-of-band writes or a future scale-out to more than one replica. Kept well above `SCHEDULER_PP_POLL_INTERVAL_MS` so the poll loop doesn't force a fresh read every cycle.

### QUEUE_NAME_POST_PROCESSOR
**Default:** `post-processor-queue`
**Type:** string

Azure Storage Queue name used by both the scheduler (to enqueue post-processing work) and the post-processor worker (to dequeue). Must match between the two services.

### SCHEDULER_URL
**Default:** (not set)
**Type:** URL string

Optional base URL for the scheduler's HTTP notification endpoints. When set, coder workers call `POST /notify/run-terminal` after a run reaches a terminal state, and post-processing handlers call `POST /notify/handler-complete` after a handler finishes. If unset, workers fall back to the legacy direct post-processor queue enqueue path and the scheduler's polling loop catches up.

- **Docker Compose:** `http://scheduler:8080`
- **Kubernetes:** `http://scheduler-service.scoped.svc.cluster.local:8080`
- **Local dev:** Leave unset to rely on legacy queue fallback unless you are also running the scheduler locally

### API_URL
**Default:** `http://api:80`
**Type:** URL string

Base URL the **scheduler** uses to reach the API's `POST /api/v1/reports/trigger` endpoint. When a run's post-processing handler DAG drains (every registered handler reaches a terminal state), the scheduler calls this endpoint so the API can evaluate each report template's trigger and create the templated report job(s). Reports are deliberately not DAG handlers — see [docs/architecture/post-processing.md](docs/architecture/post-processing.md#report-triggering-on-dag-drain).

- **Docker Compose:** `http://api:80`
- **Kubernetes:** `http://api.scoped.svc.cluster.local:80`

### SCOPE_REAPER_ENABLED
**Default:** `false`
**Type:** boolean (`true` to enable)
Expand Down
Loading
Loading