Repository navigation
fix(cli): route malformed commands to typer, not the expert agent - #203
Merged
Merged
Conversation
added 13 commits
September 18, 2026 18:12
…rmed output
Mirrors agenticCLI a7b1a84. Rule 18 only required verifying code changes, so an
agent producing a data document could pass every check it ran and still ship a
broken one: Terminal-Bench hard-066c had organization-json-generator write
employee *names* into project.members where the grader wants employee IDs, and
report the conversion as deliberate -- "member IDs resolved to employee names,
per schema's members: string[]". jsonschema agreed with it, because the schema
only constrains members to an array of strings, so 3 of 4 tests passed and
test_relationships_integrity failed on 'Cristiano Ronaldo' not in
{'D001-E001', 'D001-E002', 'D001-E003'}.
Names the case in the rule: when output transforms or joins data (ETL,
CSV/JSON, migrations), also assert every cross-reference resolves, since
schema-valid output with dangling IDs still fails.
Measured on Terminal-Bench hard (10 tasks, qwen3.8-max, serial; wheel ae86d994
whose only content delta from the before-wheel ab5c911a is this clause): the
task went 3p/1f -> 4p/0f and wrote an explicit dangling-reference check. Batch
score unchanged at 6/10 -- two out-of-scope tasks flipped negative and only 4
of the 6 passes are shared between runs, so single-run task-level variance on
hard is about +/-2. Full attribution in agenticCLI
bench/terminal-bench/RESULTS.md, runs hard-066c / hard-066d.
c4e38e7 registered app.add_typer(auth.app) but never added "auth" to _TOP_LEVEL_COMMANDS, so main()'s unknown-command branch forwarded the line to the SDK Expert agent: `dashscope auth whoami` opened a chat session instead of verifying the key, and login/logout were unreachable. `auth --help` kept working, which hid the gap -- the help branch dispatches before routing. Also document auth in the expert's cli.md skill. It claims to be authoritative for CLI usage but had no auth row, so the agent could not answer either. The new test asserts every registered typer group is routable. One-directional on purpose: rl/agentic-rl stay listed when the optional extra is absent.
main() sent any line whose first token was not in _TOP_LEVEL_COMMANDS to the agent as a question. That catch-all swallowed real errors: `dashscope -w wsid generation create` opened a chat instead of reporting "No such option: -w", and `dashscope generaton create` hid a typo that typer would have answered with "Did you mean 'generation'". It is also why the missing auth entry fixed in 6008270 silently disabled a working command rather than erroring. Three paths reach the agent now: a bare `dashscope`, an explicit `dashscope expert ...`, and a single unknown token -- the last one keeps the documented `dashscope "question"` one-shot working. Everything else goes to typer. `expert` is a routing keyword, not a typer group, and stays out of _TOP_LEVEL_COMMANDS; that set now only bounds global -k/--api-key parsing. This reverses the routing half of b808fcd -- the typer subcommand and the ~/.acli config sync it deleted stay deleted. README, README_zh and the expert's cli.md skill all advertised the removed catch-all, so their wording is updated to match.
The single-unknown-token short form made a quoted command line indistinguishable from a question: `dashscope "auth whoami"` opened a chat whose first user message was the command itself, which in a session log looks exactly like someone asking about it. Drop the short form so every invocation that is not a bare `dashscope` or `dashscope expert ...` goes to typer, which parses real commands and reports unknown ones with a "Did you mean" suggestion. `dashscope expert "question"` remains the one-shot form. The vendored acli entrypoint had the same flaw in a worse shape: an unrecognized token fell through to the interactive agent and discarded the arguments entirely, so `acli auth whoami` produced no error at all. It now exits 2 with a usage error, matching typer.
Synced from agenticCLI. The gateway binds each model to exactly one aigc service path and answers the other with a 400 "url error", so the probe order decides who pays for an unlearned model. text-generation went first, which made the default model (qwen3.8-max, multimodal-bound) fail the first request of every session — in the prod gateway log that is an InvalidParameter with a fully tokenized body thrown away, then a retry that re-uploads and re-tokenizes it. The learned path was also kept only on the instance, and a provider is built per session start, per model switch and per vision call, so nothing survived between them. Persist the binding next to the global config and share it across instances; drop unreadable or unknown-path entries instead of repairing them, since a lost binding only costs one probe.
Eleven tests over the two commands a user runs first, against a throwaway ~/.dashscope and a stubbed Models.list, so no real key file is touched and no request is sent. whoami: the login guidance and exit 1 when nothing is configured; the masked key with its source on success; an environment key reported as its own source rather than the file; a short key masked as *** instead of half-revealed; exit 2 on a rejected key without leaking it; exit 2 with the reason when the validation call itself fails. login: prompting and saving, --key skipping the prompt, surrounding whitespace stripped, a blank key refused without writing anything, and a round trip proving whoami accepts what login wrote. The key_store fixture has to move both bindings of the api-key path constant: save_api_key/get_default_api_key read it from dashscope.common.api_key while auth reads its own imported copy, so patching one leaves login and whoami silently disagreeing about where the key lives. Messages are compared through a newline-stripping helper because rich hard-wraps at 80 columns under CliRunner and force-breaks long tokens without inserting a space. Wrapping Console.width instead would pin _width on the module-level consoles past teardown. Checked against three deliberate breaks of auth.py: 7 of the 11 failed, then all 11 passed again once it was restored.
Vendors agenticCLI 4876f43 plus the 0.6.7 bump. All seven files are
byte-identical to the upstream sources modulo the dashscope. import prefix.
render_skill no longer runs str.format() over a skill body. A body is Markdown
full of literal braces from the JSON and Python it documents, so
text-generation.md — which shows messages=[{"role": "user", ...}] and declares
no arguments at all — raised KeyError('"role"'), and the old `except KeyError`
reported that as "missing args" for a skill that takes none. 6 of the 13
bundled skills were unusable through use_skill while /skill and the catalog
kept advertising them. Substitution is now a plain replace of the declared
arguments only; use_skill and the scheduler branch on `is None` rather than
falsiness, so the missing-args hint can no longer render empty.
search_files matches the pattern against paths relative to the search root as
well as the basename, and skips dot-directories only unless the pattern names
one, so .acli/skills/*.md is reachable without walking every .git in the tree.
Its description now says that * crosses separators, which it does under fnmatch
and did not under the old basename-only match, and the no-match message states
what was actually searched instead of implying the file is absent.
EnvironmentSection states the platform and how run_command executes, through a
new shell_label() that lives next to _spawn so the prompt cannot drift from
what the tool really runs. Nothing used to say which shell this is: on Windows
the model wrote `2>/dev/null || find /` and spent a turn on a PowerShell
ParserError.
Child-process output decoding tries UTF-8 strictly first, then the Windows OEM
code page from GetOEMCP() and the stdio/locale encodings. A piped child writes
in its console's code page, which need not equal sys.stdout.encoding, and cp936
accepts nearly any byte sequence so it would silently mangle UTF-8 rather than
fail. A UnicodeDecodeError in the last four bytes is the byte-budget truncation
splitting a character, not evidence of another encoding, so it is repaired
instead of re-decoding the whole buffer under a DBCS page.
Adds tests/unit/test_acli_skill_render.py: the 13 bundled skill files ship only
in this wheel, so they are the real corpus that broke and the one piece of
coverage with dashscope-only value. Verified by restoring the pre-fix
render_skill — exactly those 6 skills fail, 19 tests — then restoring the fix.
The upstream coverage for the other three fixes stays in agenticCLI, which is
where those test modules live.
dashscope unit: 1011 passed, 7 skipped. pre-commit --all-files green, mypy
included. dashscope/acli/examples/ is untouched by the sync.
pytest fixtures are injected by name, so every test taking key_store or bundled_skills redefines it from module scope, and the ones that need only the fixture's side effects never read the parameter. Both are the repo's existing convention for test modules; the assertions are unchanged apart from `calls == []` becoming `not calls`. pre-commit run --all-files only started reporting these once the files were tracked — it lints tracked files, so the untracked originals were skipped.
…ples sync_acli.sh deliberately preserves dashscope/acli/examples/, so the bundled example config drifts from its upstream repo in agenticCLI-examples and has to be reconciled by hand. Both files had moved independently: - agent.md: upstream gained the 2026-09 error-code pass (request_too_large, permission_error, billing_error, timeout_error, and the note to match on the exception class rather than .code). Take the upstream version. - system-prompt.md: Bailian docs moved to docs.modelstudio.console.alibabacloud.com. Point the product overview at the new domain and add the API-key, app-id/workspace-id and knowledge-base entries, whose titles were verified against the live pages. The remaining help.aliyun.com links are left alone rather than guessed at. The two trees are now byte-identical.
When the agent is invoked without a terminal and nothing is piped in, the error told the user to rerun as `dashscope "your question"`. That stopped working in de16112, which routes everything except a bare `dashscope` and an explicit `expert` keyword to typer. So the one message that could have recovered the command line sent the user straight into `No such command 'your question'`. Cover the whole non-TTY branch, which had no tests: piped text becomes a one-shot prompt, an empty pipe exits 2 with the corrected hint, and an unreadable stdin is refused the same way rather than crashing. The hint assertions name both the corrected form and the stale one, so a revert shows up as a failure instead of passing quietly.
write_file refused any path outside the cwd, which made it stricter than every layer around it without protecting anything: run_command can already write to temp because the shell is not path-guarded, delete_file reaches anywhere behind its own prompt, and the opt-in OS sandbox confines writes to "the workspace and temp areas". The refusal also cost more than a failed call — models reach for /tmp for scratch files by instinct, so the retry put the scratch file in the user's repo instead, which is the worse outcome. Writes now land in the cwd or the realpath'd system temp dir. Both sides are resolved with realpath, so a symlink inside either root still cannot escape, and validate_path runs first so SENSITIVE_NAMES keeps blocking e.g. /tmp/.ssh/config. The refusal message names both roots generically rather than echoing absolute paths. Per-file sync only: the two files are byte-identical to agenticCLI modulo the acli. -> dashscope.acli. import rewrite. examples/ was preserved by the sync script, and no version is bumped -- agenticCLI stays at 0.6.7.
The Response Strategy offered three question types — issue analysis, API question, example generation — and the Workflow said to classify into one of them. A question about the assistant matches none of the three, so it fell through to the only behaviour left: probe the environment until an answer turns up. Observed on a real session: "你现在是什么模型" and "你能干嘛" together burned four run_command calls, two of which failed on bash syntax under PowerShell, before producing an answer that was already sitting in the prompt on the Current model line. Nothing about the assistant is discoverable by running a command. Add the fourth branch and make the Workflow route to it first, rather than relying on the anti-pattern list to be re-read: the model was never classifying these as "explore unrelated files to get oriented", so that line never applied. Vendored from agenticCLI-examples 77c5284; the two trees are byte-identical.
Two changes, both from the same reported transcript: 3f40870 — tool output is now redacted on the way into context. save_session already ran sanitize over the persisted history, so the session file was the only thing protected: a key read out of the environment was scrubbed on disk and handed to the model in full, where it stayed in the prompt prefix for the rest of the turn. It runs before truncate_head_tail rather than after, since a character-budget cut can split a token in half and leave a fragment too short for the sk- pattern to match. tool_result_for_display is untouched — that path renders to the user's own terminal. 39d0781 — one reminder per turn after ACLI_SILENT_TOOL_NUDGE (default 6, 0 disables) consecutive tool calls stream no prose. The turn shape the transcript had has no other backstop: StagnationTracker's read-only classifier counts run_command as mutating and so resets its streak, its repeat counters need byte-identical calls, ReflectionTracker needs failures, and the convergence section is oneshot-only. Fires in interactive mode too, which is where the user is watching a spinner and getting nothing. The agenticCLI unit tests covering both are not vendored by sync_acli.sh, so this repo ships them without coverage; verified against agenticCLI's 2728 passing tests. dashscope suite: 1079 passed, 42 skipped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dashscope auth whoamiopened an Expert chat instead of verifying the key.c4e38e7registeredapp.add_typer(auth.app)but never added"auth"to_TOP_LEVEL_COMMANDS, somain()'s unknown-command branch forwarded the line to the agent;login/logoutwere unreachable.auth --helpkept working, which hid the gap — the help branch dispatches before routing.dashscope -w wsid generation createopened a chat rather than reportingNo such option: -w, anddashscope generaton createhid a typo typer would have answered with "Did you mean 'generation'". Three paths reach the agent now: a baredashscope, an explicitdashscope expert ..., and a single unknown token — the last one keeps the documenteddashscope "question"one-shot working. Everything else goes to typer. This reverses only the routing half ofb808fcd; the typer subcommand and the~/.acliconfig sync it deleted stay deleted.expertis back as a routing keyword (not a typer group, and deliberately kept out of_TOP_LEVEL_COMMANDS, which now only bounds global-k/--api-keyparsing). Multi-word unknown commands are an error again instead of silently opening a chat.cli.mdskill all advertised the removed catch-all, so their wording is updated to match;cli.mdalso gains the missingauthrow, prerequisites and troubleshooting entries — it claims to be authoritative for CLI usage but could not answerauthquestions either.1bdebbdtightens acli rule 18 to demand cross-reference resolution when output transforms or joins data (mirrors agenticCLIa7b1a84): schema-valid output with dangling IDs still fails the grader.Test plan
pytest tests/unit— 949 passed, 7 skippedtests/unit/test_cli_agent_route.py— 12 routing tests, including the newtest_auth_whoami_executes_instead_of_routing_to_agentand a one-directional invariant asserting every registered typer group is routable (rl/agentic-rlstay listed when the optional extra is absent)_route_to_expert, checking destination, forwarded prompt and TUI flagdashscope expert --help→ exit 0 usage text;dashscope generaton create→ exit 2No such command 'generaton'. Did you mean 'generation', 'image-...;dashscope -w wsid generation create→ exit 2No such option: -w;dashscope --foo→ exit 2No such option: --foo;dashscope auth whoami→ exit 1Error: No API key configured.1bdebbdon a second seed — single-run task-level variance on hard is about ±2, so the 3p/1f → 4p/0f result is one observation, not a measured mean