Activity

1. chuckmeyer commented on Aug 26, 2026

   

   chuckmeyer

   on Aug 26, 2026

   ContributorAuthor

   More actions

   The destroyed config keys are the agent's guardrails

   Inspected a dashboard-created agent (47d6a15c) with limits set to real values rather than defaults:

   {
     "suggestions": {"enabled": true},
     "feedback": {"enabled": true},
     "memory": {"enabled": true},
     "entryPoints": {"searchBar": {"aiModeEnabled": true, "promptSuggestions": {"sourceIndex": "tcg_cards_msft-build-2026"}}},
     "max_tokens": 3000,
     "max_iterations": 15,
     "thread_depth": {"max_messages": 10},
     "enableAlgoliaMcp": true
   }

   max_tokens, max_iterations and thread_depth.max_messages are the agent's runtime limits. memory is a config key not present on any previously inspected agent, so this list should not be assumed complete.

   A dry-run of the reference config against that agent:

     config:
       - enableAlgoliaMcp: True (will be removed)
       - entryPoints: {...} (will be removed)
       - feedback: {"enabled": true} (will be removed)
       - max_iterations: 15 (will be removed)
       - max_tokens: 3000 (will be removed)
       - memory: {"enabled": true} (will be removed)
       - thread_depth: {"max_messages": 10} (will be removed)
   

   All seven, including every limit.

   Why this raises the priority

   The CLI has no model for any config key — it forwards the local file's block verbatim and nothing else. Combined with PATCH's replace semantics, the practical situation is:

   • these settings can only be created in the dashboard
   • the CLI's only possible effect on them is deletion

   Any user who sets limits in the dashboard and later runs algolia-agent update silently loses them. That argues for option 1 or 2 (merge / preserve-unrecognized) over option 3 (--allow-remove), since a flag still defaults an unaware user into data loss on their first update.

   An interim mitigation independent of the option chosen: have examples/tcg/agent-config.json carry the full config block instead of only suggestions, so the reference config is not itself a footgun.
2. chuckmeyer commented on Aug 26, 2026

   

   chuckmeyer

   on Aug 26, 2026

   ContributorAuthor

   More actions

   This is ongoing schema drift, not a fixed set of gaps

   Three settings were configured on a test agent (47d6a15c) and are reported to be new dashboard options:

   Setting	API representation	Previously seen
   Dynamic mode	tools[].mode: "dynamic"	never — all 14 agents were "static"
   Allow unlisted indices	tools[].allowUnlistedIndices: true	never — all 14 were false
   Display results	algolia_display_results tool, non-default params (isTerminal: false, maxGroups: 3, maxResultsPerGroup: 6, minGroups: 1, minResultsPerGroup: 3)	tool seen, these values not

   build_tool() emits exactly ['indices', 'name', 'type'], so none of the three appear in an update payload. The dry-run in #13 does not flag mode or allowUnlistedIndices either, because it compares only fields present in the outgoing payload — correct for suppressing API-expansion noise, wrong for fields with replace semantics.

   Combined with the dashboard shipping ahead of both the CLI and the OpenAPI spec (which the team reports is out of sync), the pattern generalizes: each new dashboard option arrives as a field the CLI cannot express and will destroy, and stays invisible until someone patches build_tool(). enhancedDescription, config.memory, and config.widgetType all appeared the same way.

   Consequence for the options above

   Option 3 (--allow-remove) should be dropped. A confirmation flag only guards against losses the tool can enumerate, and the defining property of this problem is that the next loss is one nobody has modelled yet. It also defaults an unaware user into data loss on their first update.

   Recommend option 1 or 2 — preserve by default. Build the payload by starting from the GET response and overlaying only what the config file specifies, for tools, tool-level fields, and config alike. Properties:

   • a new dashboard option survives an update from a CLI that has never heard of it
   • no ongoing maintenance obligation to chase the dashboard
   • the cost is that removing something requires an explicit mechanism, which is the safer direction for the failure to lie

   Still open

   Whether omitted tool-level fields are preserved or reset to defaults. The earlier probe could not distinguish these: it used mode: "static" / allowUnlistedIndices: false, which are the defaults, so "unchanged after omission" was ambiguous. Now that non-default values exist, a throwaway-agent probe (create with mode: "dynamic", PATCH omitting it, read back, delete) would settle it.

   If they revert, update silently turns dynamic mode off — and that belongs in this issue's summary.
3. chuckmeyer commented on Aug 26, 2026

   

   chuckmeyer

   on Aug 26, 2026

   ContributorAuthor

   More actions

   Correction: index-level searchControls is destroyed, not preserved

   An earlier comment in this issue and in #13 stated that index-level searchControls survives omission from the payload. That was wrong. The first probe checked only whether the searchControls key was present after the PATCH — the key survived carrying a null value, and key presence was misread as preservation.

   A second probe using non-default values settles it. Created a throwaway draft with mode: "dynamic", allowUnlistedIndices: true, searchControls on the primary index only, and an algolia_display_results tool; then PATCHed exactly what build_tool() emits (['indices', 'name', 'type']).

   Field	Before	After
   mode	"dynamic"	"static"	reverted to default
   allowUnlistedIndices	true	false	reverted to default
   primary searchControls	set	null	wiped
   algolia_display_results	present	gone	deleted
   config (sent in full)	5 keys	5 keys	preserved
   index enhancedDescription	set	set	regenerated by platform
   tool description	set	set	regenerated by platform

   So the destroyed set is larger than this issue originally recorded, and it includes the search-tool configuration itself.

   Two useful side results:

   • Per-index searchControls is accepted on create. The replica stayed null while the primary kept its controls, so the fix proposed in config model cannot express per-index searchControls (or the full searchControls schema) #16 is viable against the real API.
   • Known tool-field defaults: mode = "static", allowUnlistedIndices = false. A field already at its default loses nothing by being omitted, which matters for distinguishing real losses from noise.

   Effect on #13

   Two false negatives, both in the destructive direction, are fixed in ffbf9ee:

   • the new_has_sc gate skipped searchControls comparison whenever the payload had no searchControls key — precisely when the update wipes them
   • matched tools compared only fields present in the payload, so an omitted mode/allowUnlistedIndices was silent

   The live dry-run against a dynamic-mode agent now reports all of it:

     searchControls:
       tcg_cards_msft-build-2026: {...} → none (will be removed)
     tools:
       - algolia_display_results
       ~ algolia_search_index.allowUnlistedIndices: True → default (not sent)
       ~ algolia_search_index.mode: 'dynamic' → default (not sent)
     config:
       - enableAlgoliaMcp: True (will be removed)
       ...
   

   This strengthens the case for preserve-by-default over a confirmation flag: the destroyed set grew twice during investigation, and both times the tooling had been silent about the addition.
4. chuckmeyer commented on Aug 27, 2026

   

   chuckmeyer

   on Aug 27, 2026

   ContributorAuthor

   More actions

   Plan: snapshot-based configuration

   Supersedes the three options in the issue body. Probing settled the design questions that made those options hard to choose between.

   Findings that shape the design

   Probe 3 (throwaway draft agent, created and deleted):

   Question	Result
   Is the GET response a valid PATCH body, verbatim?	Yes, and the agent round-trips to identical state
   Read-only fields (id, createdAt, updatedAt, lastUsedAt) in the body	Accepted and ignored, not rejected
   description, systemPrompt, templateType	All writable — not read-only, just unexpressible in the config model
   enhancedDescription	Ignored / regenerated — effectively read-only

   Two consequences:

   1. No defaults registry is needed. Omitting a field already at its default is a no-op (verified: mode: "static" omitted stays "static"). Loss only occurs when a non-default value is omitted. So writing back everything you read is sufficient for correctness, and the server remains the single authority on defaults — nothing to hand-maintain, nothing to go stale.
   2. The config model does not need to be a designed schema. It can be the agent JSON. Any field the dashboard ships in future is covered with no code change.

   This also removes the need for the merge layer, the null-means-remove convention, and the deep-vs-shallow merge decision. Removal becomes "delete the key from the file."

   Design

   Two config formats

   Format	Shape	For
   friendly (existing)	index, replicas, provider name, instructions file path, {{vars}}	provisioning many agents from one spec — the TCG workflow
   native (new)	the agent JSON as returned by GET	faithfully managing one agent

   Detected by the presence of a tools array. Both remain supported; the friendly format stays exactly as it is, so nothing breaks and templated provisioning keeps working.

   New command

   algolia-agent snapshot <agent_id> [-o FILE]     # default: agent-config.json
   

   Writes the GET response minus cosmetic fields (id, createdAt, updatedAt, lastUsedAt, enhancedDescription). Those are excluded for tidiness, not correctness — the API ignores them either way, which means a mistake in that list fails loudly (phantom diff) rather than silently.

   Guard on legacy configs

   update must refuse to send a payload that would drop non-null server-side state, pointing at snapshot as the remedy, with --force to override. _diff() already computes exactly this after #13 — the removal lines it now emits ((will be removed), → none (will be removed), → default (not sent)) are the signal. The guard must gate update itself, not only --dry-run, since the destructive path is the one without the flag.

   Phases

   1 — Guard. Refuse destructive updates unless --force. Small, independent, stops the bleeding immediately regardless of what follows. Ships on its own.

   2 — snapshot command. Write the native format. Acceptance criterion, and the reason this design is self-verifying:

   algolia-agent snapshot <id> -o snap.json
   algolia-agent update <id> --config snap.json --dry-run   # must print "No changes detected."
   

   Any field the writer misses shows up immediately as a phantom diff. Run it against 47d6a15c (dynamic mode, per-index searchControls, display-results tool) and one two-tool TCG agent.

   3 — Native format in update / create. Send the native payload as-is; skip build_tool(), skip provider-name resolution (providerId is already in the file). The friendly path is untouched.

   4 — Retire the guard's reason for existing. With snapshot available, the destructive case is only reachable from a hand-written partial config, which is then an explicit choice plus --force.

   Out of scope, tracked elsewhere

   • config model cannot express per-index searchControls (or the full searchControls schema) #16 — per-index searchControls in the friendly format. The native format gets it for free, so config model cannot express per-index searchControls (or the full searchControls schema) #16 shrinks to "friendly format parity" and the README schema docs.
   • predefinedSearchParameters — phase 2 resolves this incidentally: a snapshot of any real agent will show whether the field exists on the server at all. It appears on zero of 14 agents at tool level, while index entries carry searchParameters.

   Risks

   • Drift. Snapshot, someone edits in the dashboard, then update reverts them. Correct for a declarative model but surprising; the honest dry-run from fix(diff): eliminate dry-run false negatives and false positives #13 makes it visible before it happens. Worth a line in the README.
   • Verbosity. A full snapshot carries all 8 searchControls parameters, 9 config keys and 5 display-results params. Verbose and correct beats terse and lossy; a --minimal variant can come later if anyone wants it.
   • Two formats to maintain. Justified because they serve genuinely different jobs, and the friendly path needs no changes.
5. chuckmeyer

   added 3 commits that reference this issue on Aug 27, 2026

   feat(update): refuse updates that would delete existing configuration (…

   ...

   95a3ee9

   feat(snapshot): write a full config from an agent's current state (#19)

   ...

   7d9f5d2

   feat(create): accept a native config, making snapshots a clone source (…

   ...

   a3269d1
6. chuckmeyer commented on Aug 27, 2026

   

   chuckmeyer

   on Aug 27, 2026

   ContributorAuthor

   More actions

   Closing: phases 1–3 shipped

   Phase	PR	What landed
   1	#18	update refuses to delete configuration the config model cannot express, unless --force
   2	#19	snapshot <agent_id> writes the API's own representation; update sends a native config back verbatim
   3	#20	create accepts a native config, making a snapshot a complete clone source

   Phase 4 needed no code — with snapshot available, the destructive path requires a hand-written partial config plus --force.

   Correction to this issue's premise

   The three options originally listed here — merge, preserve-unrecognised, or a confirmation flag — were all superseded once probing established that the API accepts its own GET response as a PATCH body. That removed the need for a defaults registry, a merge layer, and a null-means-remove convention in one go: nothing has to be reconstructed, so nothing can be left out, and removal is deleting a key from the file. The server stays the only authority on its own defaults.

   Credit where due: the snapshot approach came from review, not from this issue's analysis. It is a better design than what was proposed here, primarily because it is self-verifying — snapshot → update --dry-run → "No changes detected" proves the config model covers everything the service stores. Verified clean for all 12 agents on the account.

   That oracle earned its place immediately: the first sweep failed on 3 of 14 agents and found a real bug — native configs were being template-rendered, which broke every agent carrying literal {{...}} in its stored prompt. Agent Studio's own templates ship such placeholders, so that was a whole class of agent, not an edge case. No amount of code reading would have surfaced it.

   Scope of the guard — narrower than earlier comments implied

   Worth stating precisely, because an earlier comment and #20's description were looser than the truth:

   • The guard blocks deletion of state the config model cannot express: a dropped tool, a reverted mode/allowUnlistedIndices, wiped per-index searchControls, removed config keys.
   • It does not block a partial config from overwriting fields it does specify. A config naming only index and instructions will still replace those values; --dry-run reports it, nothing refuses it.

   That distinction is deliberate — silently dropping a tool you never mentioned is different from changing a value you did write — but "update is now safe against partial configs" would be an overstatement. Confirmed the hard way: a partial config applied to a live agent replaced its instructions and index description without triggering the guard, correctly, because those were changes rather than removals. Restored from a snapshot taken moments earlier, which is itself the clearest argument for snapshot existing.

   Established about the API along the way

   None of this is in the docs; all of it was probed with throwaway agents, created and deleted.

   • PATCH replaces config, the tools array, and per-index searchControls; omitted tool fields revert to their defaults (mode → "static", allowUnlistedIndices → false)
   • Preserved when omitted: description, systemPrompt, templateType; enhancedDescription and the tool-level description are regenerated by the platform
   • Read-only fields (id, createdAt, updatedAt, lastUsedAt) are accepted and ignored, not rejected — so the snapshot exclusion list is tidiness, not correctness
   • templateType is an unvalidated provenance label, not a selector: any string is accepted, setting it has no side effects, and there is no /templates endpoint
   • description, systemPrompt and templateType are all writable

   Follow-ups

   • config model cannot express per-index searchControls (or the full searchControls schema) #16 — searchControls schema documentation (docs: complete the searchControls parameter table #21). The native format already delivers per-index searchControls.
   • predefinedSearchParameters still appears on zero agents at tool level while index entries carry searchParameters. Snapshotting any real agent is now the cheap way to confirm it is dead code.
   • merge semantics, and the query/custom search controls, remain unobserved.
7. chuckmeyer

   closed this as completedon Aug 27, 2026