Skip to content

feat(billing): reconcile legacy consumption by credit type - #1236

Merged
think-in-universe merged 11 commits into
mainfrom
fix/legacy-credit-reconciliation
Oct 13, 2026
Merged

think-in-universe merged 11 commits into
mainfrom
fix/legacy-credit-reconciliation

Conversation

@think-in-universe

@think-in-universe think-in-universe commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Addresses the balance and future credit-allocation issue in #1235: existing organizations can have usage recorded before credit-type attribution was introduced. When postpay is later added, that historical spend remains unattributed and does not reduce the correct per-type balance. Future usage can therefore consume prepaid credits that were already spent and receive an incorrect credit-type allocation.

Adds POST /v1/admin/organizations/{org_id}/credit/reconciliations to move reviewed legacy consumption into per-type counters. For example, a $100 payment limit, $200 postpay limit and $80 unknown spend can be reconciled to payment remaining $20 and postpay remaining $200, preserving aggregate capacity at $220.

  • Supports dry-run before/after snapshots, per-organization idempotency keys, and an append-only audit ledger with the admin actor and reason.
  • Uses the existing organization accounting lock and one transaction; returns retryable 409s for lock contention, normalizes keys, and rejects unsupported currencies, insufficient per-type capacity, over-reconciliation and conflicting retries. Versioned snapshots retain consumption for expired credit types.
  • Coordinates with historical discounts: wait for applying corrections before reconciliation; reject new historical corrections after reconciliation. Future-only discounts remain supported.
  • Exposes stored legacy_unattributed_amount in balance responses, including organizations without active credit types.
  • Documents the safe postpay transition order and every reconciliation response field; already-attributed usage keeps its recorded type.
  • Documents the operation and registers its OpenAPI schema. Admin sessions and read-write tokens are permitted; read-only tokens cannot preview or commit.

The migration only creates the audit table. No organization is reconciled automatically. Original usage logs and saved allocations remain unchanged; this addresses the balance/future-allocation portion of #1235, not historical invoice attribution or allocation-policy changes. Partial reconciliation leaves remaining history unresolved.

Validation:

  • cargo fmt --all -- --check

  • cargo check -p api --all-targets --offline

  • cargo clippy -p api -p database --all-targets --all-features --offline -- -D warnings

  • PostgreSQL 16: cargo test -p database --test credit_allocation --offline -- --test-threads=4 — 26 passed, including lock-timeout conflicts, retired-credit snapshots, normalized retries, mixed/partial preview, concurrent writes and discount regression cases.

  • PostgreSQL 16: cargo test -p api --test e2e_all admin_credit_reconciliation --offline -- --test-threads=1 — 1 passed, covering authentication, token permissions, validation, preview, commit and audit identity.

  • cargo test -p database --lib credit_reconciliation --offline — 1 passed, covering count validation. Unreleased snapshot compatibility code has been removed.

  • PostgreSQL 16: cargo test -p api --test e2e_all database_encryption:: --offline -- --test-threads=1 — 10 passed. The audit table’s textual/JSON fields have explicit restricted billing-data classification and coverage.

  • cargo check -p api --all-targets --all-features --locked — passed after updating transitive quinn-proto from 0.11.15 to patched 0.11.18, fixing the five advisories reported by CI.

Refs #1235, #704.

Copilot AI balanced review requested due to automatic review settings October 12, 2026 15:54
@claude

claude Bot commented Oct 12, 2026

Copy link
Copy Markdown

Review: legacy credit reconciliation

I found no critical issues. The writes run in a single transaction and take the existing org accounting lock, so concurrent writers are serialized. The idempotency replay is checked before any mutable state, dry runs roll back, and the post-write load_position == preview check stops any balance drift from being committed. The route is merged before admin_middleware is layered on (lib.rs:2454), so it is authenticated, and the e2e test confirms read-only tokens get a 403. Logging records only the error, with no amounts or reasons.

Minor, non-blocking:

  • Rolling deploy window: old pods still running put_usage_discount don't know about the new organization_credit_reconciliations check. While a deploy is in progress, a historical discount could still be saved for an org that was just reconciled. Wait until the rollout finishes before running any reconciliations, or note this in the runbook.
  • load_position drops some consumption: it only lists credit types that have an active organization_limits_history row. Consumption on a type whose limit has expired is silently left out of before/after. This is harmless for the aggregate-invariant check, but it can make the audit snapshot misleading.
  • Error message: validate_request returns "Specify at least one credit type" when amounts.len() > 4 too. Return a separate message for that case.
  • Audit field: created_by has no FK. That's fine for keeping the audit append-only, but say so in the migration comment.

✅

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The migration introduces unclassified plaintext and JSON columns that break database-encryption verification.

2 open findings
What changed in this PR

Adds an audited admin workflow for assigning reviewed legacy consumption to specific credit types while preserving aggregate capacity.

Changes:

  • Adds transactional reconciliation with previews and idempotency.
  • Coordinates reconciliation with historical discounts.
  • Adds API routing, OpenAPI documentation, tests, and operational guidance.
File Description
docs/​credit-allocation.md Documents reconciliation semantics and usage.
crates/​database/​tests/​credit_allocation.rs Tests balances, retries, concurrency, and discounts.
crates/​database/​src/​repositories/​organization_usage_discount.rs Blocks incompatible historical discounts.
crates/​database/​src/​repositories/​mod.rs Exports the repository module.
crates/​database/​src/​repositories/​credit_reconciliation.rs Implements transactional reconciliation.
crates/​database/​src/​migrations/​sql/​V0093__legacy_credit_reconciliations.sql Adds the audit ledger.
crates/​api/​tests/​e2e_all/​main.rs Registers the E2E module.
crates/​api/​tests/​e2e_all/​admin_credit_reconciliation.rs Tests authentication and API behavior.
crates/​api/​src/​routes/​mod.rs Exports the route module.
crates/​api/​src/​routes/​admin_credit_reconciliation.rs Defines request, response, and handler logic.
crates/​api/​src/​openapi.rs Registers endpoint schemas.
crates/​api/​src/​lib.rs Mounts the admin endpoint.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread crates/database/src/repositories/credit_reconciliation.rs
@ironloopai

ironloopai Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Review · Status

🟩 Completed

IronLoop completed the review and posted it to GitHub.

Result

Open submitted review →

Run details
  • Run: c5fd97de-2fa2-4be8-862c-daa75f52c624
  • Base: main at d7b537a
  • Head: fix/legacy-credit-reconciliation at c4e7e42
  • Created: 2026-10-12 15:59 UTC
  • Updated: 2026-10-12 16:12 UTC

Automatic trigger · attempt 1 of 3 · completed in 13m

@ironloopai ironloopai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review · Summary

🟢 No actionable findings

No new actionable findings identified in the reviewed change.

Validation
  • ✅ Static review — Reviewed the route, authorization, accounting transaction and locking, allocation integration, discount coordination, migration, tests, and documentation.
Review details
  • Run: c5fd97de-2fa2-4be8-862c-daa75f52c624
  • Attempts: 1

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OpenCodeReview found 9 issue(s) in this PR.

  • ✅ 9 posted as inline comment(s)
  • 📝 0 posted as summary

Comment thread crates/api/src/lib.rs Outdated
Comment thread crates/api/tests/e2e_all/admin_credit_reconciliation.rs
Comment thread crates/database/src/repositories/credit_reconciliation.rs
Comment thread crates/database/src/repositories/credit_reconciliation.rs
Comment thread crates/database/src/repositories/credit_reconciliation.rs
Comment thread crates/database/src/repositories/credit_reconciliation.rs Outdated
Comment thread crates/database/src/repositories/credit_reconciliation.rs
Comment thread crates/database/src/repositories/organization_usage_discount.rs
@think-in-universe
think-in-universe deployed to Cloud API test env October 12, 2026 18:26 — with GitHub Actions Active
@think-in-universe
think-in-universe deployed to Cloud API test env October 12, 2026 18:43 — with GitHub Actions Active
@think-in-universe
think-in-universe deployed to Cloud API test env October 12, 2026 18:59 — with GitHub Actions Active

@hanakannzashi hanakannzashi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after current-head review. Verified transactional reconciliation, idempotency, accounting locking, discount coordination, and successful CI.

@think-in-universe
think-in-universe deployed to Cloud API test env October 13, 2026 06:33 — with GitHub Actions Active
@PierreLeGuen

Copy link
Copy Markdown
Contributor

The logic looks right to me: one transaction under the organization accounting lock, replay checked before any mutable state, and aggregate capacity is the same before and after. Two things before merge, and one question.

1. Drop the compatibility code for the earlier snapshot shape. organization_credit_reconciliations is created by this PR, so no stored row can have that shape. Keep schema_version, and remove:

  • the ceiling alias and the serde defaults for active and schema_version in credit_reconciliation.rs
  • the original_snapshot_shape_remains_readable test
  • the "Older snapshots using ceiling are still readable" sentence in docs/credit-allocation.md

2. Add the step order for moving an existing organization to postpay to the doc. The section is titled "before enabling postpay", but every selected type needs an active limit with enough remaining capacity. So:

  • assigning history to postpay needs the postpay limit to exist first
  • assigning it to grant or payment has to happen before that limit is lowered

A short numbered sequence (dry run, commit, then the limit changes) would avoid a 409, or an order that leaves the organization without capacity.

Question. Usage posted since the allocation ledger went live is already allocated per request, to grant when the credit line was booked as a grant (example 1 in #1235). This endpoint only moves legacy_unattributed_amount, so that part keeps its type after the switch. Is that the intended end state?

Note: #1237 also adds a V0094 migration, so whichever merges second renumbers.

Comment thread crates/api/src/routes/admin_credit_reconciliation.rs
Comment thread crates/api/src/routes/admin_credit_reconciliation.rs

Copy link
Copy Markdown
Contributor Author

Addressed in d2356f6:

  • Removed the unreleased ceiling alias, active/schema_version defaults, compatibility helper and original-shape test. Kept the explicit schema version.
  • Documented the transition order: complete historical discounts, establish approved active capacity (including postpay first if assigning history there), dry run, commit, then reduce old limits while preserving ongoing capacity.
  • Expanded response-field documentation with units, formulas, inactive-type behavior, preview/commit semantics and replay behavior.

Yes, already-attributed usage intentionally keeps its recorded type. Grant allocations written since the ledger went live remain grant allocations after a postpay switch. This endpoint only assigns legacy_unattributed_amount; correcting previously attributed usage needs a separate audited reclassification operation. This scope is now explicit in the documentation.

#1237 is still open; the V0094 migration must be renumbered in whichever PR merges second.

Validation: formatting, all-target/all-feature API compilation, and the remaining reconciliation unit test pass. The earlier balance-field change also passed the PostgreSQL reconciliation endpoint test. Latest CI is running.

@think-in-universe
think-in-universe deployed to Cloud API test env October 13, 2026 07:22 — with GitHub Actions Active
@think-in-universe
think-in-universe merged commit 530e3d4 into main Oct 13, 2026
11 checks passed
@think-in-universe
think-in-universe deleted the fix/legacy-credit-reconciliation branch October 13, 2026 08:10

This branch was successfully deployed

1 active deployment
Cloud API test env — d2356f6a Deployed Oct 13, 2026 by think-in-universe via E2E tests #3752
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reconcile unattributed historical usage when enabling postpay for existing organizations

4 participants