Upgrading lemonade-router-builder with the current Lemonade router

skills/lemonade-router-builder/SKILL.md

@@ -26,6 +26,8 @@ try and stays editable in the desktop app's Hybrid Router editor.
 - **Lemonade Server v11.5.0+** running locally (`lemonade server start`).
   Required only to register and test the generated policy - the skill itself
   (JSON generation + offline validation) works without a live server.
+  `min_total_chars` / `max_total_chars` need **v11.8.0+**; the
+  `POST /api/v1/routing/validate` check in Step 8b needs **v11.6.0+**.
 - **No GPU or ROCm dependency** for authoring. The router policy is a JSON
   document; no hardware is needed to generate or validate it.
 - **Python** (any 3.x) in PATH - used by the bundled offline validator
@@ -202,7 +204,8 @@ condition per leaf object; nesting is allowed:
 |------|---------|-------|
 | `keywords_any` / `keywords_all` | `{ "keywords_any": ["SSN", "Email"] }` | case-insensitive substring - `"hi"` matches inside `"this"`, `"shipping"`, `"high"`, etc. Use `regex` with `\b...\b` when word-boundary precision is needed |
 | `regex` | `{ "regex": "\\b\\d{3}-?\\d{2}-?\\d{4}\\b" }` | ECMAScript flavor |
-| `min_chars` / `max_chars` | `{ "min_chars": 4000 }` | input length, UTF-8 bytes, non-negative integer |
+| `min_chars` / `max_chars` | `{ "min_chars": 4000 }` | length of the **latest user turn**, UTF-8 bytes, non-negative integer |
+| `min_total_chars` / `max_total_chars` | `{ "min_total_chars": 60000 }` | length of **all** text in the request (every message, every role), UTF-8 bytes; use for "long conversation" rules - `min_chars` sees only the last turn. ~4 bytes per token. v11.8.0+; not editable in the desktop UI |
 | `has_tools` / `has_images` | `{ "has_images": true }` | booleans |
 | `classifier` | `{ "classifier": "clf-1", "label": "PII", "min_score": 0.5 }` | band test; `min_score`/`max_score` in [0,1]; default `min_score` 0.5; omit `label` only if the classifier has `default_label` |
 | `metadata` | `{ "metadata": { "key": "consent", "equals": "denied" } }` | exactly one of `equals` / `any` / `exists`; note: not editable in the desktop UI yet - use only when the user asks for metadata routing |
@@ -228,7 +231,17 @@ It exits 0 with `"ready": true` when there are no errors. If it reports
 errors, fix the JSON and re-run. Do not present a policy that fails this
 check.
 
-**8b. Immediately after validation passes, print these three curl commands**
+The validator takes its vocabulary (keys, conditions, classifier types and
+their required fields) from `scripts/route_policy.schema.json`, a copy of the
+server's own schema; the output's `schema` field says which Lemonade ref it
+came from. If the validator rejects a field the user says their server or the
+Lemonade docs support, the copy is stale - run
+`python scripts/sync_schema.py --ref <server version tag, e.g. v11.8.1>`
+(fetches from GitHub) and re-validate, rather than editing the policy around
+it. A `schema_drift` warning means the schema has a constraint the validator
+can't check; rely on the server-side check in 8b.
+
+**8b. Immediately after validation passes, print these curl commands**
 as plain text for the user to copy and run. This is not optional. Fill in
 `<model-id>` and `<model_name>` from the policy, and a short `<test prompt>`
 that should hit the first rule. Do not execute these with Bash or any tool —
@@ -237,9 +250,16 @@ print them as text only.
 `ready: true` from the validator only means the JSON is schema-valid - it
 says nothing about whether these models exist on the target host. Run #1 for
 every candidate/classifier model before #2, or `/pull` will 400 on a policy
-that just passed validation.
+that just passed validation. #0 runs the target server's own parser on the
+policy without registering it - the authoritative answer if the server's
+version differs from the bundled schema.
 
 ```bash
+# 0. Parse and test the policy on the server itself, without registering it
+curl -X POST http://localhost:13305/api/v1/routing/validate \
+     -H "Content-Type: application/json" \
+     -d '{"policy": <contents of router.json>, "prompt": "<test prompt>"}'
+
 # 1. Check a model exists before registering
 curl http://localhost:13305/api/v1/models/<model-id>
 

skills/lemonade-router-builder/evals/evals.json

@@ -52,6 +52,24 @@
         "router.json"
       ]
     },
+    {
+      "id": "long-conversation-router",
+      "skill_should_trigger": true,
+      "prompt": "Once a conversation's total history gets past about 15,000 tokens, send it to fireworks.deepseek-v3; otherwise keep it on Qwen3.5-9B-GGUF.",
+      "expected_behavior": [
+        "Use a min_total_chars condition (whole-conversation length), not min_chars",
+        "Size the bound at roughly 4 bytes per token (about 60000)",
+        "Set Qwen3.5-9B-GGUF as default_model",
+        "Run the offline validator scripts/validate.py and report it passed"
+      ],
+      "unexpected_behavior": [
+        "Use min_chars as a substitute for min_total_chars",
+        "Claim the validator or the server does not support min_total_chars"
+      ],
+      "files_exist": [
+        "router.json"
+      ]
+    },
     {
       "id": "nginx-load-balance",
       "skill_should_trigger": false,

skills/lemonade-router-builder/reference.md

@@ -5,6 +5,15 @@ generation steps in `SKILL.md` leave a question open. The server-side parser
 (`routing_policy_parser.cpp`) is strict: it rejects unknown keys at every
 level, so never emit fields not listed here.
 
+The machine-readable source of truth is `scripts/route_policy.schema.json`, a
+verbatim copy of the server's
+[`route_policy.schema.json`](https://github.com/lemonade-sdk/lemonade/blob/main/src/cpp/resources/schemas/route_policy.schema.json);
+`scripts/route_policy.schema.source.json` records the ref it was copied from.
+`scripts/validate.py` derives its key sets from it, so the validator follows
+the schema automatically after `python scripts/sync_schema.py`. The tables
+below are hand-written. When a sync reports new keys or types, update them too.
+`python scripts/sync_schema.py --check` exits 1 when the copy is stale.
+
 ## Contents
 
 - [Root document](#root-document)
@@ -14,6 +23,7 @@ level, so never emit fields not listed here.
 - [Model capability requirements](#model-capability-requirements)
 - [Validation checklist](#validation-checklist)
 - [Registering, invoking, tracing](#registering-invoking-tracing)
+- [Router scope and limits](#router-scope-and-limits)
 - [Desktop-editor compatibility](#desktop-editor-compatibility)
 
 ---
@@ -114,15 +124,17 @@ Match-expression grammar:
 | `keywords_any` | non-empty array of non-empty strings | case-insensitive substring, any - `"hi"` also matches inside `"this"`, `"shipping"`, `"high"`, etc. Use `regex` `\b...\b` for word-boundary matching |
 | `keywords_all` | same | all must appear; same substring semantics as `keywords_any` |
 | `regex` | non-empty string | ECMAScript regex over the input text |
-| `min_chars` / `max_chars` | integer ≥ 0 | input length in **UTF-8 bytes** |
+| `min_chars` / `max_chars` | integer ≥ 0 | length of the routing input (latest user turn) in **UTF-8 bytes** |
+| `min_total_chars` / `max_total_chars` | integer ≥ 0 | length of **all** text in the request in UTF-8 bytes: every chat message of every role (system/user/assistant/tool), the `/completions` `prompt`, or every Responses `input` item. Images/audio count as 0. Fires on a long history even when the last turn is `"go on"`, which `min_chars` sees as 5 bytes. Estimate ~4 bytes per token when sizing against a context window. Server v11.8.0+ |
 | `has_tools` / `has_images` | boolean | request carries `tools[]` / image parts. `false` is legal and means "must NOT have"; `{"not": {"has_tools": true}}` is the equivalent preferred spelling |
 | `classifier` | classifier id (string) | band test on that classifier's score |
 | `label` | string | only with `classifier`; must exist on that classifier; may be omitted only if the classifier has `default_label` |
 | `min_score` / `max_score` | number in [0,1], min ≤ max | only with `classifier`; omitting both defaults to `min_score: 0.5` |
 | `metadata` | `{ "key": <string>, ... }` | plus exactly one of `equals` (string), `any` (non-empty string array), `exists` (boolean); evaluated over the request's OpenAI `metadata` object. Comparisons are case-sensitive. `any` comma-decodes the metadata value before matching: a caller sending `{"dept": "legal,compliance"}` (scalar) matches `any: ["legal", "compliance"]` - same as if two separate values were in the array. |
 
-The routed input text is the last user message (chat), the `prompt`
-(completions), or the `input` (responses).
+The routed input text is the last user message (`/chat/completions`), the
+`prompt` (`/completions`), or the `input` (`/responses`). All three endpoints
+route a `collection.router` model on every supported server (v11.5.0+).
 
 ## Model capability requirements
 
@@ -178,6 +190,11 @@ authoring judgment and aren't (fully) checkable by a script.
 ## Registering, invoking, tracing
 
 ```bash
+curl -X POST http://localhost:13305/api/v1/routing/validate \
+     -H "Content-Type: application/json" \
+     -d '{"policy": {...}, "prompt": "My SSN is 123-45-6789",
+          "has_tools": false, "has_images": false, "metadata": {"consent": "denied"}}'
+
 # register (idempotent - re-POST to update)
 curl -X POST http://localhost:13305/api/v1/pull \
      -H "Content-Type: application/json" --data-binary @router.json
@@ -221,8 +238,10 @@ there:
 - One condition per leaf object. Compound leaves (`{"min_chars": 5,
   "has_tools": true}`) are valid server-side but the editor cannot display
   them and will warn it must drop them on save.
-- `metadata` conditions are JSON-only for now - the editor shows a lossy-edit
-  warning for rules containing them. Generate them only when the user
-  explicitly wants metadata routing, and say so in your reply.
+- `metadata` and `min_total_chars` / `max_total_chars` conditions are
+  JSON-only in released builds - the editor shows a lossy-edit warning for
+  rules containing them. (The editor on Lemonade `main` edits `metadata` but
+  refuses to import `*_total_chars`.) Generate them only when the user's
+  intent needs them, and say so in your reply.
 - Everything else in this reference (nested `all`/`any`/`not`, all three
   classifier types, negated booleans) round-trips cleanly.