diff --git a/python/a2ui_a2a/README.md b/python/a2ui_a2a/README.md index f05a570445..929c0450b1 100644 --- a/python/a2ui_a2a/README.md +++ b/python/a2ui_a2a/README.md @@ -22,9 +22,14 @@ This installs `a2ui-agent-sdk` and `a2a-sdk` as dependencies. when the client asks for it. `get_a2ui_extension_uri` and `A2UI_EXTENSION_BASE_URI` give the extension URI for a protocol version. - Part helpers: `create_a2ui_part`, `is_a2ui_part` and `get_a2ui_datapart` - wrap and unwrap A2UI messages in A2A `DataPart`s. `parse_response_to_parts`, - `parse_stream_chunk_to_parts`, `stream_response_to_parts` and - `parse_content_to_parts` turn model output into A2A parts. + wrap and unwrap A2UI messages in A2A `DataPart`s. `to_a2a_parts` converts + the parts that `A2uiRequestProcessor.parse_response()` returns, and + `stream_response_to_parts` reads a streamed response through a + `ResponseStream`, such as one from `A2uiRequestProcessor.open_stream()`. + Both leave validation errors to the caller. `parse_content_to_parts` + parses a whole response with a `Parser`, such as + `A2uiRequestProcessor.parser`, and logs and drops one that fails. A canvas + agent passes it the `RendererState` it keeps with `state=`. ```python from a2ui.a2a import create_a2ui_part, get_a2ui_agent_extension diff --git a/python/a2ui_agent/CHANGELOG.md b/python/a2ui_agent/CHANGELOG.md index 3d75f4ff4d..ae97220549 100644 --- a/python/a2ui_agent/CHANGELOG.md +++ b/python/a2ui_agent/CHANGELOG.md @@ -1,5 +1,9 @@ ## Unreleased +- Add `A2UI_RENDERER_CAPABILITIES_KEY` (`"a2uiRendererCapabilities"`) to + `a2ui.constants`, the message metadata key under which the v1.0 A2A + extension sends the renderer capabilities. v0.8 and v0.9 use + `A2UI_CLIENT_CAPABILITIES_KEY`. - Add `RendererState` to `a2ui.utils`. A `RendererState` is the agent's mirror of the surfaces a renderer holds: it wraps one `MessageProcessor` and checks each outbound message against everything sent before it, so a canvas agent @@ -83,6 +87,9 @@ per response are removed. To migrate: - `Parser.parse_chunk(chunk, wrapped)`: open a stream with `parser.open_stream(wrapped=...)` and call `feed(chunk)`, then `close()`. + - `A2uiRequestProcessor.create_parser()`: use the `processor.parser` + property, which returns the one parser the processor reuses, or + `processor.parse_response` and `processor.open_stream`. - The `parser` property of `DirectJsonFormat`, `ExpressFormat`, `ElementalFormat` and `AtomFormat`: use `create_parser()`. - `validate_payload(..., surface_catalog_ids=...)`: check the payload diff --git a/python/a2ui_agent/README.md b/python/a2ui_agent/README.md index bbbb154d35..25a3a01e9a 100644 --- a/python/a2ui_agent/README.md +++ b/python/a2ui_agent/README.md @@ -10,14 +10,15 @@ The following directories contain the base protocol logic, parsing, and schema o ### Processor & Catalog Configuration (`src/a2ui/processor`) - **`generator.py`**: Defines `A2uiGenerator`, the main entry point for configuring an agent's catalogs and default inference format factory. -- **`processor.py`**: Defines `A2uiRequestProcessor`, which binds the active catalogs negotiated for a request to an inference format and exposes prompt generation and response parsing. +- **`processor.py`**: Defines `A2uiRequestProcessor`, which binds the active catalogs negotiated for a request to an inference format and exposes prompt generation and response parsing. An agent creates one for each request with `A2uiGenerator.create_processor`, from the capabilities that the client sent. - **`catalog_config.py`**: Defines `CatalogConfig`, which binds a `Catalog` (or loads one from a path via `CatalogConfig.from_path`) with an optional sequence of `CatalogTransformer` instances into `transformed_catalog`. - **`catalog_providers.py`**: Defines `CatalogProvider`, `FileSystemCatalogProvider`, and `InMemoryCatalogProvider` for loading catalogs. ### Constants (`src/a2ui/constants.py`) - Protocol and SDK string constants such as `A2UI_OPEN_TAG`, - `A2UI_CLOSE_TAG` and `A2UI_CLIENT_CAPABILITIES_KEY`. Protocol versions are + `A2UI_CLOSE_TAG`, `A2UI_CLIENT_CAPABILITIES_KEY` and + `A2UI_RENDERER_CAPABILITIES_KEY`. Protocol versions are `a2ui.core.schema.ProtocolVersion` members. The module has no dependencies, so callers that only need a constant can import `a2ui.constants` without loading the rest of the SDK. @@ -44,17 +45,21 @@ The following directories contain the base protocol logic, parsing, and schema o `prune_common_types_schema` reduce the protocol schemas to the allowed messages and the common types that are referenced. - **`renderer_state.py`**: `RendererState` is the agent's mirror of the - surfaces a renderer holds, and checks each outbound message against - everything sent before it. `validate_payload` checks one payload the way a - renderer holding the catalogs would, on a short-lived `RendererState`, and - raises `A2uiValidationError` if the renderer would reject it. + surfaces a renderer holds. It wraps one `MessageProcessor` and checks each + outbound message against everything sent before it. A chat agent uses a new + state per turn, which is what a parser does when it is given none. A canvas + agent keeps one state for the whole conversation. `validate_payload` checks + one payload the way a renderer holding the catalogs would, on a short-lived + `RendererState`, and raises `A2uiValidationError` if the renderer would + reject it. ### Parser (`src/a2ui/parser`) -- **`parser.py`**: Defines the `Parser` abstract base class (`wrap`, `unwrap`, `compile`, `decompile`, `parse_response`, and `parse_chunk` for formats whose `supports_streaming` is true). +- **`parser.py`**: Defines the `Parser` abstract base class. A format implements `compile` and `decompile`, which translate between its notation and A2UI messages, and `wrap` and `unwrap`, which handle its tags. The base class reads model responses with `parse_response`. `open_stream` reads a response chunk by chunk; the base class raises `NotImplementedError`, and only formats that stream, whose `supports_streaming` is true, implement it. Both check every message against a `RendererState`. A parser holds no per-response state, so one parser can be reused across turns. +- **`response_stream.py`**: Defines the `ResponseStream` that `open_stream` returns. - **`response_part.py`**: Defines `ResponsePart` (`TextPart` / `A2uiPart`) and `RawResponsePart` (`RawA2uiPart`). - **`lexer.py`**: Defines `BlockLexer` for sentinel-tag tokenization (`...`). -- **`errors.py`**: Defines `A2uiParseError` and `A2uiTargetNotFoundError`. +- **`errors.py`**: Defines `A2uiCompilationError` and its subclasses `A2uiCompilationParseError` and `A2uiCompilationValidationError`. - **`messages.py`**: `to_message_models` and `to_message_dicts` convert between message dictionaries and typed models. - **`payload_fixer.py`**: Utilities to automatically correct common LLM output issues in A2UI payloads. @@ -63,7 +68,7 @@ The following directories contain the base protocol logic, parsing, and schema o - **`prompt/generator.py`**: Defines the `PromptGenerator` (`A2uiPromptGenerator`) abstract base class (`generate_base_rules`, `generate_catalog_instructions`, `generate_examples`, `generate`). - **`inference_format.py`**: Defines `InferenceFormat` and `InferenceFormatFactory`. -- **`inference_formats/`**: Format implementations and factories for Direct JSON (`direct_json/`), Express (`express/`), and experimental Elemental and Atom (`experimental/`) formats. Direct JSON is the format that streams: `DirectJsonParser.parse_chunk` heals the partial JSON of each chunk (`direct_json/json_reader.py`) and emits the messages that are ready. +- **`inference_formats/`**: Format implementations and factories for Direct JSON (`direct_json/`), Express (`express/`), and experimental Elemental and Atom (`experimental/`) formats. Direct JSON is the format that streams: the stream from `DirectJsonParser.open_stream` heals the partial JSON of each chunk (`direct_json/json_reader.py`) and emits the messages that are ready. ## Agent framework integrations diff --git a/python/a2ui_agent/agent_development.md b/python/a2ui_agent/agent_development.md index d3745dd69f..325d97f6f7 100644 --- a/python/a2ui_agent/agent_development.md +++ b/python/a2ui_agent/agent_development.md @@ -15,9 +15,15 @@ The `agent_sdk` revolves around four main classes: - **`A2uiGenerator`**: Holds the agent's supported `CatalogConfig` instances, optional few-shot example turns, and format factory, and negotiates an `A2uiRequestProcessor` for a client's renderer capabilities. -- **`A2uiRequestProcessor`**: Request-scoped facade that binds the active - negotiated catalogs to an `InferenceFormat` (defaulting to `DirectJsonFormat`) - and exposes `prompt_snippet`, `create_parser()`, and `parse_response()`. +- **`A2uiRequestProcessor`**: Request-scoped facade that binds the catalogs + negotiated for one request to an `InferenceFormat` (defaulting to + `DirectJsonFormat`). It exposes `prompt_snippet` for the system prompt, and + `parse_response()` and `open_stream()`, which parse and validate the model's + response. + +The agent creates its generators once. Each request gets its own processor, +negotiated from the capabilities that the client sent with the request, so the +prompt and validation use the catalogs that this client renders. ## Prerequisites @@ -29,38 +35,37 @@ The `agent_sdk` revolves around four main classes: ### Step 1: Set up the Generator and Request Processor -The first step in any A2UI-enabled agent is loading your catalogs into an -`A2uiGenerator` and creating an `A2uiRequestProcessor`. +Load your catalogs into an `A2uiGenerator` when the agent starts. When a +request arrives, negotiate an `A2uiRequestProcessor` for the client's renderer +capabilities. ```python from a2ui.core.basic_catalog import BasicCatalog from a2ui.core.schema import ProtocolVersion -from a2ui.processor import A2uiGenerator, A2uiRequestProcessor, CatalogConfig +from a2ui.processor import A2uiGenerator, CatalogConfig -# Define your catalogs (basic or bring your own) -catalog_configs = [ +# At startup: the catalogs that the agent supports (basic or bring your own) +catalogs = [ CatalogConfig(BasicCatalog(ProtocolVersion.V0_9)), CatalogConfig.from_path("path/to/catalog.json"), ] +generator = A2uiGenerator(catalogs) -# Initialize the agent-level generator with your catalogs -generator = A2uiGenerator(catalog_configs) - -# Negotiate a request processor for the client's renderer capabilities +# Per request: the capabilities that the client sent, keyed by protocol version processor = generator.create_processor({ - "v0.9": { - "supportedCatalogIds": [ - c.transformed_catalog.catalog_id for c in catalog_configs - ] - } + "v0.9": {"supportedCatalogIds": [BasicCatalog(ProtocolVersion.V0_9).catalog_id]} }) - -# Or create a processor directly when using a static catalog list -static_processor = A2uiRequestProcessor( - [config.transformed_catalog for config in catalog_configs] -) ``` +The client sends its capabilities in the metadata of the A2A message, under +`a2uiClientCapabilities` (`A2UI_CLIENT_CAPABILITIES_KEY` in `a2ui.constants`), +keyed by protocol version as above, so the agent passes them to +`create_processor` unchanged. v0.8 capabilities may also be the entry itself, +`{"supportedCatalogIds": [...]}`, as the v0.8 specification defines them. A web client built on `@a2ui/web_core` can get +them from its renderer with `processor.getRendererCapabilities({versions: +['v0.9']})`. `create_processor` raises an `A2uiError` if the client sent no +capabilities, or if they are malformed or name none of the agent's catalogs. + Notes: - A processor needs at least one active catalog, and all of its catalogs must @@ -68,83 +73,147 @@ Notes: of them. - To shape a catalog, such as pruning components or functions, pass `transformers` to `CatalogConfig`. `transformed_catalog` applies them in order. -- When `examples` (a sequence of message turns) are passed to `A2uiGenerator` or - `A2uiRequestProcessor`, each example turn is validated against - `processor.active_catalogs` when the processor is created (raising - `A2uiValidationError` if invalid) and rendered into `processor.prompt_snippet`. -- The provided catalogs must be freestanding, i.e. they should not reference any - external schemas or components, except for the common types. -- If you have a modular catalog that references other catalogs, refer - to [Freestanding Catalogs](../../docs/public/concepts/catalogs.md#freestanding-catalogs) - for more information. -- You can define multiple `A2uiGenerator` instances (one for each protocol version) - and select the active one at runtime based on the client request. +- When `examples` (a sequence of message turns) are passed to `A2uiGenerator`, + each example turn is validated against `processor.active_catalogs` when a + processor is created (raising `A2uiValidationError` if invalid) and rendered + into `processor.prompt_snippet`. +- `A2uiRequestProcessor(catalogs)` can also be created directly from catalogs, + for example in a script that prints the prompt for a set of catalogs. +- A catalog may reference the protocol's common types and its peer catalogs, + for example to reuse a component or a property of another catalog with a + `$ref`. The SDK resolves these references against the other catalogs it + holds: for `A2uiGenerator`, every registered catalog, including ones the + renderer doesn't support; for `A2uiRequestProcessor(catalogs)`, the catalogs + it is given. A reference that matches no catalog can't be resolved, and a + response that uses it fails validation. See + [Catalog Linking](../../docs/public/concepts/catalogs.md#catalog-linking). +- You can define one `A2uiGenerator` for each protocol version and select the + one for the version that the request activates. See [Multiple Version Support](#3-multiple-version-support) for more details. -### Step 2: Generate System Prompt +#### Choose an Inference Format + +The inference format is the notation the model writes A2UI in. The generator's +format factory decides it, and every processor the generator creates uses it +for both the prompt snippet and the parser. -Use `processor.prompt_snippet` to produce the A2UI format and schema prompt -snippet, and combine it with your agent's role/workflow instructions and -optional few-shot examples loaded via `load_examples`. +- **Direct JSON** (`DirectJsonFormatFactory`, the default): the model writes + A2UI messages as JSON inside `` tags. It is the only format that + streams: `processor.open_stream()` emits an `updateComponents` or + `updateDataModel` as soon as part of it reads, and again each time it grows, + and any other message once its object closes (see + [Option B](#option-b-incremental-streaming-parsing-advanced)). +- **Express** (`ExpressFormatFactory`): the model writes a compact notation + inside `` tags, which the SDK compiles into A2UI messages. It needs far + fewer output tokens than JSON, so a whole response arrives sooner. An Express + block resolves references across its whole body, so it doesn't stream: + parse the response with `processor.parse_response()`. ```python -from a2ui.utils import load_examples +from a2ui.inference_formats import DirectJsonFormatFactory, ExpressFormatFactory -examples = load_examples( - processor.active_catalogs, - "path/to/examples", - validate=True, -) +# The default, written out. +generator = A2uiGenerator(catalogs, inference_format_factory=DirectJsonFormatFactory()) + +# Express, for lower latency on whole responses. +generator = A2uiGenerator(catalogs, inference_format_factory=ExpressFormatFactory()) +``` + +`create_processor` also takes an `inference_format_factory` that overrides the +generator's for one request. + +### Step 2: Generate System Prompt + +Use the request's `processor.prompt_snippet` to produce the A2UI part of the +system prompt, and combine it with your agent's role and workflow +instructions. The snippet describes the format, the catalogs negotiated for +the request (including inline catalogs that the client sent, when the +generator was created with `accepts_inline_catalogs=True`), and the +generator's examples, so build the prompt per request. + +```python instruction = "\n\n".join([ "You are a helpful assistant...", "# Workflow\nAnalyze the request and return UI...", - "# UI Description\nUse the following components...", processor.prompt_snippet, - examples, ]) ``` -To save tokens, prune the catalogs before building the processor rather than -when generating the prompt. +Give few-shot examples to the generator, as message turns, rather than adding +them to the prompt yourself. Creating a processor checks each example against +the request's catalogs, and the snippet writes them in the format's notation, +so the same examples serve Direct JSON and Express. ```python -from a2ui.catalog_transformers import ComponentPruningTransformer -from a2ui.inference_formats.direct_json import schema_to_prompt -from a2ui.processor import A2uiRequestProcessor, CatalogConfig +import json -# Keep only some components in the catalog that the prompt describes. -config = CatalogConfig( - BasicCatalog(ProtocolVersion.V0_9), - transformers=[ComponentPruningTransformer(["Text", "Button", "Column"])], -) -processor = A2uiRequestProcessor([config.transformed_catalog]) +from a2ui.inference_formats import to_message_models -# Keep only some messages in the agent-to-renderer schema. -schema_text = schema_to_prompt( - processor.active_catalogs, - allowed_messages=["CreateSurfaceMessage", "UpdateComponentsMessage"], +with open("path/to/examples/contact_card.json", encoding="utf-8") as f: + contact_card = to_message_models(json.load(f)) + +generator = A2uiGenerator(catalogs, examples=[contact_card]) +``` + +To save tokens, shape what the snippet describes when you set up the +generator: + +```python +from a2ui.catalog_transformers import ComponentPruningTransformer +from a2ui.inference_formats import DirectJsonFormatFactory +from a2ui.processor import A2uiGenerator, CatalogConfig + +generator = A2uiGenerator( + [ + # Keep only some components of the catalog. + CatalogConfig( + BasicCatalog(ProtocolVersion.V0_9), + transformers=[ComponentPruningTransformer(["Text", "Button", "Column"])], + ) + ], + # Keep only some messages of the agent-to-renderer schema. + inference_format_factory=DirectJsonFormatFactory( + allowed_messages=["createSurface", "updateComponents"], + ), ) ``` +`ExpressFormatFactory` takes `allowed_messages` too. + ### Step 3: Build an LLM Agent with the System Prompt Configure your `LlmAgent` using the generated system instructions. This agent serves as the core logic for interpreting user queries and deciding when to generate rich UI responses. +Since the instruction belongs to the request, build the `LlmAgent` and its +`Runner` per request. Share one session service across the runners, so that a +conversation keeps its history from one request to the next. + ```python from google.adk.agents.llm_agent import LlmAgent from google.adk.models.lite_llm import LiteLlm +from google.adk.runners import Runner +from google.adk.sessions import InMemorySessionService +# At startup +session_service = InMemorySessionService() + +# Per request agent = LlmAgent( model=LiteLlm(model=LITELLM_MODEL), name="Your agent name", description="Your agent description.", instruction=instruction, - tools=[Your tools], + tools=[], # Your tools ) +runner = Runner(app_name=APP_NAME, agent=agent, session_service=session_service) ``` +An agent that uses the ADK `SendA2uiToClientToolset` can instead put the +request's catalog and examples into the session state; see +[Dynamic Schemas](#2-dynamic-schemas-context-aware). + ### Step 4: Process Request and Stream UI The final step is to build an executor (or a custom streaming handler) that @@ -160,15 +229,12 @@ from a2a.server.agent_execution import AgentExecutor class MyAgentExecutor(AgentExecutor): - def __init__(self, agent: LlmAgent, ...): + def __init__(self, agent: LlmAgent): self.agent = agent - ... + # ... -agent_executor = MyAgentExecutor( - agent=agent, - ... -) +agent_executor = MyAgentExecutor(agent=agent) ``` #### 4b. A2UI Extension Version Negotiation @@ -177,13 +243,15 @@ Before processing a request, negotiate which version of the A2UI extension to ac ```python from a2ui.a2a import try_activate_a2ui_extension +from a2ui.constants import A2UI_CLIENT_CAPABILITIES_KEY # In your request handler: -activated_version = try_activate_a2ui_extension(context, agent_card) +version = try_activate_a2ui_extension(context, agent_card) -if activated_version: - # Use the activated version to create a request processor from its generator - processor = generators[activated_version].create_processor(capabilities) +if version: + # Negotiate the request's processor with the generator of that version + capabilities = (context.message.metadata or {}).get(A2UI_CLIENT_CAPABILITIES_KEY) + processor = generators[version].create_processor(capabilities) ``` #### 4c. Select a Parsing Strategy @@ -196,7 +264,7 @@ Use this approach if you wait for the LLM to finish its entire response before p **1. Parse, Validate, and Fix** -`processor.parse_response(full_text)` parses, heals simple JSON formatting errors (e.g., trailing commas), and validates every A2UI block against `processor.active_catalogs` in a single call, raising `A2uiValidationError` if a renderer holding the catalogs would reject the payload. +`processor.parse_response(full_text)` parses, heals simple JSON formatting errors (e.g., trailing commas), and validates every A2UI block against `processor.active_catalogs` in a single call. It raises an `A2uiError`, such as `A2uiValidationError`, if a renderer holding the catalogs would reject the payload. Validation always runs; there is no option to skip it. ```python from a2ui.parser import A2uiPart @@ -210,28 +278,34 @@ for part in response_parts: ... ``` -**2. Stream the A2UI Payload** - -Wrap the validated payloads in an A2A `DataPart` with the correct MIME type (`application/a2ui+json`) and stream it. - -**Recommendation:** Use the `create_a2ui_part` helper. +**2. Convert to A2A Parts** -**3. Complete Agent Output Structure (Helper)** - -The `parse_response_to_parts` helper is the most efficient way to split text, extract JSON, validate, and wrap into A2A `Part` objects in one go. +`to_a2a_parts` from `a2ui.a2a` turns the parts into A2A `Part` objects: a +`TextPart` for text and a `DataPart` with the A2UI MIME type for each message. +Catch the error to retry with a prompt that says what was wrong: ```python -from a2ui.a2a import parse_response_to_parts +from a2ui.a2a import to_a2a_parts +from a2ui.core import A2uiError -yield { - "is_task_complete": True, - "parts": parse_response_to_parts(full_text), -} +try: + parts = to_a2a_parts(processor.parse_response(full_text), version=version) +except A2uiError as e: + # Ask the model again, quoting the error. + ... +else: + yield {"is_task_complete": True, "parts": parts} ``` +To wrap single messages yourself, use the `create_a2ui_part` helper. It takes a +message dictionary, so convert the models in `A2uiPart.a2ui` with +`to_message_dicts` from `a2ui.inference_formats` first. + ##### Option B: Incremental Streaming Parsing (Advanced) -Use this approach for sub-second UI updates. The `DirectJsonParser` **automatically parses, validates, and fixes (heals)** the JSON payload chunks _incrementally_ as they arrive from the LLM stream. It yields valid UI messages _before_ the entire JSON block is complete by automatically closing open quotes and braces. Create it with `processor.create_parser()`, so that it validates against `processor.active_catalogs` and heals the format's progressive keys. From v1.0 on, the parser holds all of the processor's active catalogs and checks each component against the catalog that the component or its surface's `createSurface` names. +Use this approach for sub-second UI updates. It needs the Direct JSON format, the only one that streams; `open_stream()` raises `NotImplementedError` for Express. The stream parses, heals and checks the JSON _incrementally_ as chunks arrive from the LLM. It emits an `updateComponents` or `updateDataModel` (v0.8's `surfaceUpdate` or `dataModelUpdate`) as soon as part of it reads, by closing the open braces and brackets, and emits it again each time it grows. Any other message, such as `createSurface`, `deleteSurface` or `callRendererFunction`, waits until its object closes, and the messages after it wait with it. Each chunk is read once: the stream goes on from where the previous chunk left off and checks a growing message only for what it gains, so streaming a response takes time in proportion to its length. Open the stream with `processor.open_stream()`, so that it checks each message against `processor.active_catalogs`: each component is checked against the catalog its surface uses or, from v1.0 on, the catalog the component names. + +A string still being written is emitted only for the format's progressive keys, whose open strings the stream closes where they stand. The default `DirectJsonFormatFactory()` has none, so a message with an open string waits until the string closes. To show text as it is written, pass `progressive_keys`, for example `DirectJsonFormatFactory(progressive_keys=DEFAULT_PROGRESSIVE_KEYS)` with `DEFAULT_PROGRESSIVE_KEYS` from `a2ui.constants`. > [!IMPORTANT] > **Prerequisite**: To use incremental streaming, your agent executor must support streaming mode. In ADK, enable this using `RunConfig`: @@ -243,46 +317,138 @@ Use this approach for sub-second UI updates. The `DirectJsonParser` **automatica > ``` ```python -from a2ui.a2a import create_a2ui_part -from a2ui.parser import A2uiPart, TextPart +from a2ui.a2a import to_a2a_parts -parser = processor.create_parser() +# One stream per response. +stream = processor.open_stream() # Inside your LLM stream loop: for chunk in llm_response_stream: - # Process text chunks as they arrive - response_parts = parser.parse_chunk(chunk.text) - - for part in response_parts: - if isinstance(part, A2uiPart): - # Yield partial UI updates immediately - yield { - "is_task_complete": False, - "parts": [create_a2ui_part(p) for p in part.a2ui] - } - elif isinstance(part, TextPart) and part.text: - # Yield conversational text - yield { - "is_task_complete": False, - "parts": [DataPart(text=part.text, mime_type="text/plain")] - } + # Yield text and partial UI updates as soon as they are ready. + if parts := to_a2a_parts(stream.feed(chunk.text), version=version): + yield {"is_task_complete": False, "parts": parts} + +# After the last chunk, close the stream. It emits the text still held back as a +# possible tag, and raises if a block was left open. +if parts := to_a2a_parts(stream.close(), version=version): + yield {"is_task_complete": False, "parts": parts} +``` + +Always call `close()` after the last chunk: without it, trailing text that +could still have started a tag is never emitted, and a block the model left +open goes unreported. + +`feed` and `close` raise an `A2uiError` when a message is invalid. The renderer +already has the messages emitted before the error, and they may leave a surface +incomplete: references and the root are checked only when a block closes. Read +the retry with `processor.open_stream(state=stream.state)`, so it is checked +against what the renderer holds. + +`stream_response_to_parts` from `a2ui.a2a` runs the same loop, closes the +stream after the last token, and converts the parts to A2A parts: + +```python +from a2ui.a2a import stream_response_to_parts +from a2ui.core import A2uiError + +stream = processor.open_stream() +try: + async for part in stream_response_to_parts(stream, token_stream, version=version): + yield {"is_task_complete": False, "parts": [part]} +except A2uiError as e: + # Ask the model again, quoting the error, and read the retry with + # processor.open_stream(state=stream.state). + ... ``` > [!TIP] -> `DirectJsonParser` performs content-based change detection to ensure components are only re-yielded if their content changes, minimizing bandwidth usage. +> The Direct JSON stream re-emits a message only when its content changes. Change detection is per message, so a growing `updateComponents` is re-sent with all of its components so far, not only the changed ones. + +#### 4d. Chat and Canvas UIs + +The processor checks each response against a `RendererState`, the agent's +mirror of the surfaces the renderer holds. How long that state lives is what +separates the two ways an agent shows UI. + +In a chat, each turn shows new UI. Pass no state: each call checks the response +against an empty state, so from v0.9 on, a response that updates a surface it +did not create is rejected. (A v0.8 renderer buffers updates to a surface until +its `beginRendering`, so a v0.8 response may send updates before +`beginRendering`, but one that never sends it is rejected too.) An agent that +updates surfaces from earlier turns is a canvas. + +```python +parts = processor.parse_response(text) +``` + +In a canvas, turns update the same surfaces. Keep one state for the whole +conversation and pass it every turn, streamed or not. A turn that refers to a +component no turn sent, or that creates a cycle with components from earlier +turns, is then rejected. + +```python +state = processor.create_state() + +# Turn 1, parsed whole: applied all or nothing. +parts = processor.parse_response(text, state=state) + +# Turn 2, streamed: a new stream that applies what it emits to the same state. +stream = processor.open_stream(state=state) +for chunk in llm_response_stream: + send(stream.feed(chunk.text)) +send(stream.close()) +``` + +A response parsed whole is applied to the state only if all of it is valid. A +streamed response cannot be undone, since the renderer has already received +what was emitted, so after a failure the state keeps the emitted messages. The +next turn is not required to make the components they added reachable, since +no message removes a component, but it must resolve their dangling references +and give the surface its root. + +To avoid passing the state on every call, bind it when creating the +processor. Calls that pass no state then use it: + +```python +processor = generator.create_processor(capabilities, state=state) +parts = processor.parse_response(text) +stream = processor.open_stream() +``` + +The state checks every component a turn adds or replaces: it must be reachable +from the root. A component that a turn leaves unreachable, by rewriting its +parent's children, is accepted, since the protocol has no message that removes +one component. + +The state holds the surfaces as they now stand, not the messages that built +them, and lives in memory, so keep it with the conversation session, for +example in a dict keyed by session id. A state rejects an update to a surface +it never saw created, as the renderer would. Since the processor is created per +request, the state must hold the request's catalogs in the same order, or the +processor raises `A2uiCatalogError`. The active catalogs follow the order of +the client's `supportedCatalogIds`, so a client that reorders its list needs a +new state too. Check with `processor.accepts_state(state)`, and start a new +state with `processor.create_state()` when it returns False: + +```python +state = states.get(session_id) +if state is None or not processor.accepts_state(state): + state = states[session_id] = processor.create_state() +``` ## Use Cases ### 1. Simple Agents with Static Schemas -For agents with a fixed set of UI capabilities, simply use `processor.prompt_snippet` -to generate the system instruction. +For agents with a fixed set of catalogs, build the system instruction from the +request's `processor.prompt_snippet`, and the agent around it. **Example Samples:** [restaurant_finder](../../samples/agent/adk/restaurant_finder) ```python -# Generate system prompt +# Per request +processor = generator.create_processor(capabilities) instruction = "\n\n".join([ "You are a helpful assistant...", "# Workflow\nAnalyze the request and return UI...", @@ -291,7 +457,7 @@ instruction = "\n\n".join([ ]) # Use with your LLM framework (e.g., ADK) -agent = LlmAgent(instruction=instruction, ...) +agent = LlmAgent(model=..., name="...", instruction=instruction) ``` ### 2. Dynamic Schemas (Context-Aware) @@ -305,13 +471,14 @@ e.g., Charts vs. Maps). #### 2a. Injecting Catalogs into Session State -In a dynamic scenario, you don't provide a static catalog to the agent. Instead, +In a dynamic scenario, the agent's instruction doesn't name a catalog. Instead, you configure an `A2uiGenerator` once with `CatalogConfig(catalog)` and `accepts_inline_catalogs=True`, and call `generator.create_processor(capabilities)` -at runtime (e.g., during session preparation) to negotiate the active catalogs -and store the selected catalog in the session state. +for each request (e.g., during session preparation) to negotiate the active +catalogs and store the selected catalog in the session state. ```python +from a2ui.constants import A2UI_CLIENT_CAPABILITIES_KEY from a2ui.utils import load_examples # In your AgentExecutor subclass @@ -320,7 +487,7 @@ async def _prepare_session(self, context, run_request, runner): # 1. Determine client capabilities from metadata, keyed by protocol version, # for example {"v0.9": {"supportedCatalogIds": [...]}} - capabilities = context.message.metadata.get("a2ui_client_capabilities") + capabilities = (context.message.metadata or {}).get(A2UI_CLIENT_CAPABILITIES_KEY) # 2. Negotiate a request processor for the client capabilities and load examples processor = self.generator.create_processor(capabilities) @@ -380,40 +547,38 @@ When the LLM calls the UI tool, the toolset uses the dynamic catalogs to: ### 3. Multiple Version Support -To support multiple protocol versions (e.g., v0.8 and v0.9), pre-configure `A2uiGenerator`, `A2uiRequestProcessor`, and `LlmAgent` instances for each version during your agent's initialization. At runtime, use `try_activate_a2ui_extension` to negotiate the version and select the pre-configured processor or runner. +To support multiple protocol versions (e.g., v0.8 and v0.9), create one +`A2uiGenerator` for each version when the agent starts, and list each in the +agent card. At runtime, use `try_activate_a2ui_extension` to negotiate the +version, and create the request's processor with that version's generator. ```python -# During Initialization (Setup mapping for each supported version) +# During initialization: one generator per supported version generators = { ProtocolVersion.V0_8: A2uiGenerator([CatalogConfig(BasicCatalog(ProtocolVersion.V0_8))]), ProtocolVersion.V0_9: A2uiGenerator([CatalogConfig(BasicCatalog(ProtocolVersion.V0_9))]), } -processors = { - version: gen.create_processor({ - version.value: { - "supportedCatalogIds": [ - c.transformed_catalog.catalog_id for c in gen.catalogs - ] - } - }) - for version, gen in generators.items() -} -ui_runners = { - ProtocolVersion.V0_8: build_runner(build_agent(processors[ProtocolVersion.V0_8])), - ProtocolVersion.V0_9: build_runner(build_agent(processors[ProtocolVersion.V0_9])), -} +extensions = [ + get_a2ui_agent_extension( + version, + supported_catalog_ids=[ + c.transformed_catalog.catalog_id for c in generator.catalogs + ], + ) + for version, generator in generators.items() +] -# Runtime Stream Handling (Select based on negotiation) +# Per request: select the generator for the negotiated version version = try_activate_a2ui_extension(context, self.agent_card) if version: - # Select the pre-configured agent runner and request processor - runner = ui_runners[version] - processor = processors[version] + capabilities = (context.message.metadata or {}).get(A2UI_CLIENT_CAPABILITIES_KEY) + processor = generators[version].create_processor(capabilities) + runner = build_runner(build_instruction(processor)) else: - # Fallback to standard text agent runner - runner = text_runner + # Fall back to the text agent processor = None + runner = text_runner ``` ### 4. Orchestration and Delegation @@ -432,7 +597,7 @@ supported_catalog_ids = set() for subagent in subagents: # ... fetch subagent_card ... for extension in subagent_card.capabilities.extensions: - if extension.uri == A2UI_EXTENSION_URI: + if extension.uri == get_a2ui_extension_uri(ProtocolVersion.V0_9): supported_catalog_ids.update( extension.params.get("supportedCatalogIds") or []) @@ -517,7 +682,7 @@ async def set_proxy_capabilities_interceptor( message.metadata[A2UI_CLIENT_CAPABILITIES_KEY] = { "v0.9": { "supportedCatalogIds": [ - "https://a2ui.org/specification/v0_9/basic_catalog.json" + "https://a2ui.org/specification/v0_9/catalogs/basic/catalog.json" ], }, } diff --git a/python/a2ui_agent/src/a2ui/constants.py b/python/a2ui_agent/src/a2ui/constants.py index 5a59ae5f71..d17bc968db 100644 --- a/python/a2ui_agent/src/a2ui/constants.py +++ b/python/a2ui_agent/src/a2ui/constants.py @@ -38,6 +38,9 @@ # A2UI Metadata A2UI_CLIENT_CAPABILITIES_KEY = "a2uiClientCapabilities" +# The message metadata key of the renderer capabilities in the v1.0 A2A +# extension. v0.8 and v0.9 use `A2UI_CLIENT_CAPABILITIES_KEY`. +A2UI_RENDERER_CAPABILITIES_KEY = "a2uiRendererCapabilities" A2UI_CLIENT_DATA_MODEL_KEY = "a2uiClientDataModel" A2UI_CLIENT_DATA_MODEL_SURFACES_KEY = "surfaces"