[WIP] Ability for agents to create/modify extensions (custom functions/objects/behaviors) - #9094
Merged
Merged
Conversation
Phase 6 of the AI extensions support plan (gameplay tests on nested
custom objects):
- `getObjects(name, { childrenDepth })` and `getNearby(..., { childrenDepth })`
replace the boolean children flag of the snapshot builder: children are
snapshotted recursively (default depth 1, capped at 8), so the children of
a nested custom object are reachable
(`children.TankTop_Combined[0].children.TankCanon[0]`).
- Positions of every level are converted to scene coordinates (2D affine
transformation of the parent, Z offset and Z scale in 3D) and every
descendant reports the layer of the top custom object.
- Malformed snapshot options throw instead of being ignored.
- Tests on a nested custom object fixture (depth 2, depth 0, cap, rounding,
Z with a scaled parent, every instance of a child object).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
…_extension Phase 1 of the AI extensions support plan (read path): - The simplified project sent to the AI gets an `extensions` section (declarations only, never events): extension settings, variables, dependencies, tests, free functions, custom behaviors (properties, shared properties, functions) and custom objects (properties, functions, child objects, object groups, layers, variants) with the implicit `Object` / `Behavior` parameters marked and the store extensions flagged `isFromStore`. - New `inspect_extension` editor function: drill-down read at the extension, custom object, custom behavior, function and variant levels. Call forms are computed from the generated platform metadata (parameter names read from the declared parameters with a cursor that accounts for code-only parameters, operator + value pairs and getters of `ActionWithOperator`). Store extensions are reported read-only. - `formatPropertiesList`, `getSimplifiedInstance` and helpers move to `Utils.js`; the typed outputs fixture vendors `inspect_extension` and the shared `SimplifiedFunction` / `SimplifiedProperty` / `SimplifiedChildObject` types. - Fixtures: `TankConfiguration` (nested custom objects) and `LeaderboardDialog` (store extension). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
…ensions and custom object variants
Add the `scope` argument shared by the AI editor functions (`EditorFunctions/Scope.js`): a scene, the
instances of an external layout, an extension, a custom behavior, a custom object or one of its
variants ("" = default). The legacy `scene_name` maps to a scene scope, a disagreement between the
two is refused, and the 15 scene-bound functions (objects, behaviors, instances, layers, effects,
groups, variables, events reads) now resolve their containers through it. Structural edits on a
named variant are refused (the default variant propagates with complyVariantsToEventsBasedObject),
extensions installed from the store are read-only, and outside-editor notifications carry their
target (scene, external layout, custom object variant) so the matching editor tabs refresh.
The typed outputs fixture is regenerated from the backend shapes of tools v18.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
… behaviors, functions) and editor regeneration Add the AI functions creating and changing extensions, custom objects (settings, properties, variants, forwarded child functions), custom behaviors (properties and shared properties) and functions (types, parameters, sentences, lifecycle rules), with the shared property and parameter change helpers, the refactorers run before every rename, usage checks before deletions, and the read-only rule for extensions installed from the store. The editor is notified of the changed extensions once per batch: the flush regenerates the extensions (or only their metadata), refreshes the open extension, custom object and scene editors, closes the tabs of deleted items and renames the tabs of renamed ones. Deleting a custom object variant resets the objects using it to the default variant through a new Core refactorer (`WholeProjectRefactorer::RemoveEventsBasedObjectVariant`). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
…ensions - generate_events accepts a scope (extension, custom behavior, custom object) and a function_name: the function's events are serialized as the existing events, the changes are applied to the function's events list, variables and behaviors are only added on the child objects of a custom object, then the extension is regenerated and the extension editor notified. - read_events_source on a function prepends a scope summary (parameters, properties, child objects, extension variables) as comment lines. - The extensions summary uploaded for a generation includes the private members of the extension being edited. - The extensions summary is refreshed before every events generation. - Specs: GenerateEventsInFunctions, ReadEventsSourceInFunctions (Health extension fixture), Scope, ScopeEventsAndVariables, SimplifiedProject, ApplyEventsChanges. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
…er, source view and applied changes - EventScriptRenderer renders BuiltinCommonInstructions::JsCode events as `js """ ... """` fenced blocks exactly like the backend serializer (shared conformance fixtures, byte-identical), with a rendering error for a code line closing the fence early. - The events source view never cuts inside a fence: a js event is kept or dropped whole, and says so when nothing of it fits. - Generated JsCode events are inserted and replaced like any event (specs). - Instruction summaries always carry isAsync, so the backend can refuse `await` on an action that is not asynchronous. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
…ted snapshots - change_custom_function validates the whole call before applying it, keeps lifecycle functions callable, refuses parameter edits on a setter (its parameters come from its getter) and updates the setters of a renamed getter; call forms are read from freshly reloaded metadata. - Functions returning a value declare what they return (expression_type / the expressionType setting). - The events of a setter see the parameters of its getter and Value (parametersForEvents), in the source view and the generation. - Private members are offered to the events generation with the editor's own visibility rule (free functions in their extension, methods while authoring their behavior or object). - generate_events re-resolves its target after the generation and fails cleanly when it disappeared; a replaced JavaScript event keeps its flags. - Gameplay test snapshots of nested custom objects use the complete transformation of every level (world positions and bounds, local sizes). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
… child's own rotation, return type rule Core: - `WholeProjectRefactorer::RenameParameter` looks the parameter up in the parameters the events resolve against (the scoped containers' list, built from `GetParametersForEvents`) instead of the function's own declaration: the events of an `ActionWithOperator` use the parameters of its getter. - `Move*EventsFunctionParameter` apply the same offset of two to an `ActionWithOperator` as to an `ExpressionAndCondition`: their generated instructions take an operator and a value before the parameters of the function. - `ParameterMetadataContainer::GetParameterPosition` is const. Tests for the setter cases (object, number and behavior parameters renamed, type changed, moves for free, behavior and object functions). Editor (AI functions): - `generate_events` checks its target, awaits the extensions refresh, then resolves the target again and uses only that second resolution. - `change_custom_function` on a getter applies renames, type changes and moves of its parameters, and a change of what it returns, to the events and calls of every `ActionWithOperator` reading it; `Value` is refused as a parameter name of such a getter; taking the value away from it is refused (an already inconsistent getter stays editable); `create_custom_function` refuses `parameters` on an `ActionWithOperator`; a copy under a lifecycle name is checked with the type and parameters of its source. - One return-type rule: a value-returning type change keeps the return type, `StringExpression` forces a text, `expressionType` in the same call wins, a function that stops returning a value drops it (a table of eight transitions shared with the mock editor of the prompts). GDJS gameplay-test runner: - `worldBounds` of a child starts from the child's own complete transformation (inner area through its own transformation for a custom object, unrotated box rotated with the renderer's Euler around its center for a 3D object, `getAABB()` for a 2D object) before the transformations of its parents; karma cases for X/Y and X+Z rotations (with a THREE oracle), a rotated parent, a 2D child, a nested 3D custom object with a custom rotation center and a 2D custom object child. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
`applyParameterChanges` only composed `planParameterChanges` and `applyPlannedParameterChanges` for its spec, which now does it itself. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
… metadata The hint telling the agent how to call an `ActionWithOperator` from EventScript placed the operator and the value after the parameters of its getter, while the generated metadata (MetadataDeclarationHelper) inserts them before: `Set(Object, =, Value, Param)`. The declaration-based fallback (used when the extension has no generated metadata yet) now follows that layout, and the "read the value with" hint of a setter names the parameters of its getter instead of a generic `Value`. Both are covered by a spec on a getter taking a parameter. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
…, redundant expression type accepted A variable set on an instance with `add_or_edit_variable` is declared on its object when missing, in a scene and in a variant of a custom object: the child object of the default variant gets it (the owner of the structure) and the variants comply, so an instance of a named variant is covered too. Same helper and messages as the fix made for scenes on the independent branch. `create_custom_function` accepts `expression_type: "string"` on a `StringExpression`, which only repeats what the function returns; a contradicting type is still refused, with the type to use instead. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
…on the object master carries the scene version of the instance variable declaration of `add_or_edit_variable` (squash of claude/ai-object-variable-fixes). The branch already had the same helper and messages, plus the child objects of a custom object and the variants compliance: its version of `resolveVariablesContainers` and of the launch function is kept, and the helper that both sides had added is kept once. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
…ject itself holds properties A variable asked on the custom object itself (or any name that is not a child object) is refused with the child objects, and now with the reason the custom object cannot be the target: its own state is its properties. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
…t lacks `add_behavior` only checked that the behavior type exists (and the object type it may be restricted to). A behavior requiring a capability of its object - a hidden behavior objects of a kind have by default, like the 3D one of 3D objects - was added anyway, along with the capability behavior itself: the IDE hides such a behavior for these objects, and the game breaks at runtime (a 3D physics behavior on an object without depth crashed the preview with `this.owner3D.getDepth is not a function` when an agent put it on a custom object not rendered in 3D). The function now mirrors the IDE check: a required behavior that is hidden must already be on the object, otherwise the object is skipped with a warning naming the capability, the behavior type and the object type. Visible required behaviors keep being added along with the behavior. Test fixtures gain a hidden fake 3D capability, a fake 3D model having it by default, and two fake behaviors requiring a capability (the 3D one, the animatable one), so the specs cover a refusal, an addition, another capability, and a group with compatible and incompatible objects. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
… 3D child A custom object draws its children with a 2D or a 3D renderer, chosen by its `isRenderedIn3D` setting. An agent created a tank without `is_3d`, added three 3D cubes as children and put a 3D physics behavior on it: the preview crashed and the user had to enable the 3D rendering by hand. `create_or_replace_object` now marks a custom object left to its default rendering as rendered in 3D when its first child is a 3D object (from scratch, from the asset store or duplicated), tells the editor to regenerate the extension code, and says so in its result. A 2D first child keeps the 2D rendering whatever comes next. The refusal of `add_behavior` for a missing capability also names, for a custom object of the project, the setting giving it (`isRenderedIn3D`, `isAnimatable`, `isTextContainer`) and the tool changing it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
… in the store registry `create_or_replace_object` and `add_behavior` asked the store installer for any `Ext::Type`, and only skipped it for an extension already loaded in the platform. A custom object or behavior authored a moment earlier in the same script is not loaded yet: adding `Tank::TurretAssembly` as a child of `Tank` failed with "Extension Tank does not exist in the extension registry", while the identical call succeeded in the next script. A type of one of the project's extensions now regenerates the extensions and never goes to the registry; the other types are installed as before. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
Mirror what the editor enforces in its behavior list: a behavior marked as irrelevant for child objects (the physics engines, the NavMesh pathfinding), or requiring one, cannot be added to a child of a custom object by a script. The refusal points to the objects placed in a scene (the custom object itself among them). The fake Physics3D behavior of the tests is marked the same way. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCzEriwrG2Uhi8AyZkMUSJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.