Repository navigation
fix: close underpayment paths and harden the defaults - #13
Merged
Merged
Conversation
…imate Flat mode always sent estimatedCostSats to toll-booth, so --price only appeared in the banner. A flat price of 0 now leaves the routes unpriced rather than issuing 0-sat invoices.
The proxy only injected stream_options.include_usage when the client sent no stream_options, so a client could send include_usage: false and fall back to the cheaper chunk-count estimate.
When a stream contained reasoning chunks, the counter ignored the reported completion_tokens and billed only visible content, so thinking models were underbilled when streaming but not when buffered. Reported usage now always wins, reasoning chunks count towards the fallback estimate, and usage figures that are not non-negative numbers are ignored.
The token counter parsed each network read on its own, so an event split across two reads failed to parse. When that event was the final usage report, billing fell back to the chunk estimate with no prompt tokens. Partial lines are now carried over, and data fields without a space or with CRLF endings are accepted.
toll-booth tracks the estimated cost per payment hash, so when two requests on one credential overlapped the second overwrote the first and went unreconciled. satgate now settles each request against a hold it owns, drawing on the credit balance directly, and never calls the engine's reconcile. Requests that fail before inference (invalid JSON, a non-object body, capacity) now refund the hold as well; before, only upstream failures did. Per-request payments, sessions and the free tier are fixed holds that cannot grow or be refunded; USD credentials stay flat.
Pricing looks models up by name, so a name the upstream resolves to a priced model but pricing does not recognise, such as registry.ollama.ai/library/gemma3:4b, was served as that model and billed at the default price. Requests must now name an auto-detected or configured model, compared case-insensitively with Ollama's implicit :latest tag spelt out; anything else is refused with a 400 and the hold refunded. Pricing also matches an untagged name to its :latest entry.
Nothing bounded a completion, so a request could generate far more than the fixed hold covered and the reconciliation charge was clamped at the remaining balance. max_tokens is now clamped to an operator cap (--max-tokens, SATGATE_MAX_TOKENS or maxTokens; default 2048) and set when absent, and n/best_of are limited to 8. In per-token mode the proxy holds the worst case before calling the upstream: every body byte as a prompt token plus max_tokens per choice. Credit balances are debited for it and refunded to the actual cost, and a balance that cannot cover it gets a 402. Per-request payments (IETF Payment charge), sessions and the free tier cannot grow, so max_tokens is shrunk to fit what was paid, and a prompt larger than that is refused. The default hold now covers a 4 KiB body plus a full max_tokens completion at the dearest configured price.
Rails are tried in order and the charge rail claims every Authorization: Payment header, so session credentials were only ever verified as charges and refused.
createHonoTollBooth was built with the engine alone, so --trust-proxy did nothing: every client resolved to 0.0.0.0 and shared one free-tier bucket and one pending-invoice count. trustProxy is now passed through, and trusted proxy hops can be named with --trusted-proxies, TRUSTED_PROXIES or trustedProxies (naming them implies trustProxy).
Payment auth was mounted on /v1/*, so any method on any /v1 path could mint an invoice. It now runs only for POST to /v1/chat/completions, /v1/completions and /v1/embeddings; other methods and paths are a 404 and GET /v1/models stays free. HEAD is answered by satgate as a free price probe, so a credential sent with it is never verified or debited.
Without trustProxy the toll-booth Hono adapter files every client under 0.0.0.0, so a directly exposed satgate had one free-tier allowance and one pending-invoice count for everyone. satgate now supplies the TCP peer address in that case.
Anyone could mint Lightning invoices without limit, through 402 challenges or POST /create-invoice. Each client IP may now hold 20 unpaid invoices before getting a 429 (--max-pending-invoices, SATGATE_MAX_PENDING_INVOICES or maxPendingInvoicesPerIp; 0 disables). createTollBooth runs no housekeeping, so satgate now prunes invoices unpaid after a day, and expired Cashu claims, every hour; close() stops the timer.
With no --lightning, satgate runs with no payment or auth, yet the Cloudflare tunnel defaulted on, so the quick start published an open inference proxy. The tunnel now defaults on only when payment or an allowlist is required; --tunnel opts in explicitly. The README, examples/basic.sh and llms.txt quick starts now pass --lightning, and no longer claim the bare command accepts payments.
The hidden setup typed an absolute home-directory PATH and checkout path. It now changes to the repository root from wherever vhs is run and expects Node on PATH.
tsc emits only TypeScript and the package publishes dist/, so npx satgate served a 404 at /. The build now copies src/page/index.html to dist/page/index.html, where the compiled server looks for it, and the Docker image takes it from the build output instead of copying it separately.
--version and the banner read ../package.json relative to the module, which from dist/src is dist/package.json: --version printed 'unknown version' and the banner fell back to v0.1.0. Both now find satgate's own package.json from source or compiled layouts. The banner also printed 'nwc (undefined)', since nwc has no backend URL.
Storage defaulted to memory and the macaroon root key was regenerated on every start, so a restart lost every paid balance and invalidated every credential. With any payment rail configured, storage now defaults to SQLite, and a generated root key is saved beside the database (satgate.root-key, mode 0600) and reused. Choosing memory storage with payments enabled prints a loud warning.
MAX_CONCURRENT defaulted to unlimited, so a burst of paid or free requests queued without bound on the GPU. Beyond 8 in flight clients now get a quick 503; --max-concurrent 0 restores the old behaviour.
satgate sent no credentials upstream, so only local or keyless OpenAI-compatible endpoints worked. A bearer key can now be set with UPSTREAM_API_KEY, --upstream-key-file or upstreamKeyFile in the config file; it is sent on inference and model-list requests, and the client's own Authorization header is never forwarded.
Allowlist auth compared the NIP-98 u tag with the URL the server saw, which behind a TLS-terminating proxy is http://127.0.0.1:..., so no client signing the real https URL could authenticate. The u tag may now also match the request rebuilt from --public-url, or from X-Forwarded-Proto and X-Forwarded-Host when the proxy is trusted.
/llms.txt always claimed Lightning and NWC, listed per-token rates in flat mode, and priced tagged models at the default. It now names only the configured rails (Lightning, Cashu, LNURLcash, x402), the credential header for each, the flat price when flat pricing is on, and model rates through the same lookup billing uses.
openapi.json always said version 1.0.0 and declared only L402. It now carries the package version, declares the IETF Payment scheme alongside L402 when Lightning is configured, describes x402, Cashu and LNURLcash as the headers they are, and lists every accepted scheme on the inference endpoints.
…ent correctly The playground promised 250 free tokens whatever the server offered, and held clients to that figure client-side. It now reads the free-tier credit from /.well-known/l402, shows it in sats, and shows nothing when there is no free tier. The description no longer calls IETF Payment a way to pay (it is an authentication scheme over Lightning), and the social card is 'summary' since the page has no og:image.
The release workflow inserts each new entry after the first top-level heading, which was the old 1.15.0 entry, so 1.15.3 to 1.17.2 landed between that heading and its body. Entries are now in descending version order, the 1.15.0 notes are back under their heading, and a '# Changelog' title gives future entries the right insertion point.
pnpm-lock.yaml resolved toll-booth from a sibling checkout (file:../toll-booth) and had drifted from package-lock.json, which npm, CI and the Docker build actually use. It is now untracked and ignored.
The image exposed 3002 while satgate listens on 3000 unless PORT is set. The production deploy sets PORT=3002 itself, so it is unaffected.
Drop the '~400 lines' and 'GDPR-safe out of the box' claims, describe billing as it now works (reported usage, worst-case hold, what cannot be refunded) instead of 'never short-changed', note UPSTREAM_API_KEY for keyed upstreams, document the new defaults, and update the TypeScript badge to 6.
x402 is supported, not coming; --price is a flat per-request price, not a per-token default; --free-tier grants sats of credit, not requests; the estimatedCostSats default and trustProxy behaviour are described as they now work; LNURLcash and the nwc backend are listed; and NWC is described as a Lightning backend rather than a way for clients to pay. Also documents the flat 1 sat default when no pricing is set, and points to the forgesworn toll-booth and 402-mcp repositories.
With --lnurlcash-mints and no --lightning or Cashu mints, auth mode was inferred as open, so the payment middleware was never mounted and inference was served free. LNURLcash now selects the bearer-ecash mode like Cashu does.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pre-announcement fixes from the audit. Each code fix has a regression test that fails without it.
--priceis honoured;include_usageis always forced; reasoning tokens are billed; a usage event split across reads is no longer lost; each request settles against its own hold, so concurrent requests on one credential are billed properly; unknown model names are refused;max_tokensis capped and the worst case is held before the upstream call.trustProxy/trustedProxieswired; production trusts only the local Caddy); payment auth runs only on POST to the inference endpoints; unpaid invoices are capped per IP and pruned;MAX_CONCURRENTdefaults to 8;--lnurlcash-mintsalone no longer serves free inference.--lightning.--version, the banner and openapi.json; llms.txt derived from config; SECURITY.md; CHANGELOG ordered; stale lockfile and.idea/untracked; README claims corrected.Tests: 309 to 383.