Add provider error taxonomy, model capabilities gating, and telemetry fixes

docs/framework/v2-extraction-roadmap.md

@@ -0,0 +1,118 @@
+# v2 extraction roadmap — what the platform taught us belongs in the framework
+
+The ActiveAgents platform (activeagents/activeagents) has been the field lab
+for the gems: every gap in `activeagent` or `solid_agent` shows up there as
+app-level code. This document is the audit of that code, drawing the line for
+the three-layer architecture:
+
+- **activeagent** — execution: agents, providers, tools, telemetry
+- **solid_agent** — persistence: contexts, generations, tool streams, memory,
+  pricing
+- **the platform** — accounts, billing, quotas, hosted UI, multi-tenancy
+
+The solid_agent side of this audit already landed (enriched tool persistence,
+`ModelPricing`, memory/tool-cache contracts). What follows is the
+framework-shaped remainder, proposed for v2 — each item exists today as
+platform code that any serious consumer of the gem would have to rebuild.
+
+## Absorb from the platform
+
+### 1. Per-model capability gating — ✅ shipped
+`ActiveAgent::ModelCapabilities` strips parameters the target model rejects
+(temperature/top_p on thinking-first Claude, OpenAI o-series/GPT-5) from
+prepared prompt parameters before they reach the provider. Extensible via
+`ModelCapabilities.register(pattern, unsupported:)`; disable with
+`ModelCapabilities.enabled = false`. Remaining for v2: max_tokens vs
+max_completion_tokens switching and reasoning-effort awareness.
+
+### 2. Provider model catalogs
+The platform's `/api/provider_models` queries Ollama's live model list, the
+Anthropic Models API, and OpenRouter's catalog, with curated fallbacks.
+v2: `Provider#models` on the provider contract (vendor SDKs all expose a
+listing endpoint), so model pickers and validation stop being app problems.
+
+### 3. Tool-loop safety — ✅ shipped (turns)
+`max_tool_turns` (default 25, per agent/prompt override) now bounds the
+tool-calling recursion; hitting the cap emits
+`tool_turns_exceeded.active_agent` and returns the messages gathered so far.
+Remaining for v2: a token/cost budget alongside the turn cap.
+
+### 4. Agent-to-agent delegation
+`tools_function` only routes back to `self`. The platform built `call_agent`
+(sub-agent invocation with a `Thread.current` depth cap) as an app tool.
+v2: a first-class delegation primitive — invoke another agent class/instance
+as a tool, with depth limits and shared trace/context correlation.
+
+### 5. Provider error taxonomy + fallback — ✅ taxonomy shipped
+`ActiveAgent::Providers::Errors` (RateLimited, ContextLengthExceeded,
+AuthenticationFailed, ContentFiltered, ServiceUnavailable, InvalidRequest)
+now normalizes vendor exceptions in `with_exception_handling` — classified
+by SDK class name, HTTP status, and message heuristics, original preserved
+as `#cause` — so `rescue_from` policy is portable across providers.
+Remaining for v2: the `generate_with ... fallback: [:anthropic, :ollama]`
+chain the taxonomy makes expressible.
+
+### 6. A real MCP story
+Today MCP is pass-through only: `mcps:` options are normalized into each
+vendor's *remote* MCP format (the LLM vendor's servers do the connecting).
+There is no MCP client (stdio/HTTP, `tools/list` discovery → routable
+actions) and no server facade. The platform built an MCP server over its
+agents (`run_<slug>` tools, `agent://` resources, Bearer auth) as a
+controller. v2: both halves — a client that turns any MCP server's tools
+into agent actions, and a mountable engine that presents agents as an MCP
+server.
+
+### 7. Tool DSL / schema derivation
+`lib/active_agent.rb`'s docstring advertises a `tool def get_weather(...)`
+macro that does not exist; tools are hand-written JSON Schema hashes.
+solid_agent's `HasTools` (DSL + JSON view templates) already fills this —
+v2 should either absorb it or bless it as the canonical declaration path,
+not leave two half-standards.
+
+### 8. Server-side tool implementations
+The platform's `AgentToolbox` (safe `fetch_url` with SSRF guard + redirect
+caps, `web_search`, a no-eval `Calculator`, allowlisted `browse_page`) is
+generic execution code with zero app coupling. It belongs beside the
+framework's tool routing, not in a dashboard app — persistence/caching of
+results stays solid_agent's.
+
+## Fix in place (bugs and dead seams found during the audit)
+
+- ✅ **Fixed**: `telemetry/instrumentation.rb` — provider/model/message-count
+  attributes now record (was: nonexistent `generation_provider`, private
+  helpers hidden from `respond_to?`, nonexistent `Base#messages`); the dead
+  `around_generate` registration is removed.
+- ✅ **Fixed**: `redact_attributes` is now consumed — span and span-event
+  attribute values matching the configured patterns become `[REDACTED]` in
+  `build_trace_payload`, covering both transmission and local storage.
+  `capture_bodies` remains reserved (telemetry spans capture no bodies yet;
+  the raw response on the ActiveSupport::Notifications payload is
+  in-process only).
+- `Observers`/`Interceptors` call `Prompt.register_observer` on an
+  `ActiveAgent::Prompt` class that doesn't exist; nothing in the generation
+  path notifies them. Either implement the ActionMailer-style seam
+  (persistence layers want it) or delete it.
+- The dashboard engine ships orphaned platform-shaped models
+  (`Dashboard::Agent`, `AgentRun`, `SandboxSession`, jobs, migrations) with
+  no controllers or routes. Decide: wire them (framework-level run
+  orchestration — run records, status, cancellation would pair with the
+  tool-loop limits above) or drop them from the gem.
+
+## Stays in the platform
+
+Accounts/users, billing and plan quotas, encrypted API/provider key storage,
+trace retention by plan, the hosted React dashboard, sandboxes/session
+recordings, and the account-scoped `TelemetryTrace` subclass. These touch
+tenancy and money; the gems should expose seams (auth hooks, quota
+callbacks), never implementations.
+
+## solid_agent follow-ups (tracked there, listed for completeness)
+
+- ✅ **Shipped**: `agent_runs` + persisted run progress events — the install
+  generator now ships an `AgentRun` model with lifecycle, `append_event`,
+  trace correlation, and `SolidAgent::RunFingerprint` instruction cohorts.
+- Evaluation datasets — `docs/agent-md-spec.md` already specifies
+  `*.test.yml` cases; the platform's rule-criteria scorer and LLM-judge
+  plumbing are the reference implementation.
+- Fold the platform's drifted model copies back onto the generator
+  templates once the app's Gemfile.lock reaches solid_agent 0.2.

lib/active_agent.rb

@@ -105,6 +105,7 @@ module ActiveAgent
   autoload :Preview, "active_agent/concerns/preview"
   autoload :Previews, "active_agent/concerns/preview"
   autoload :GenerationJob
+  autoload :ModelCapabilities
   autoload :Observers, "active_agent/concerns/observers"
   autoload :Provider, "active_agent/concerns/provider"
   autoload :Rescue, "active_agent/concerns/rescue"

lib/active_agent/base.rb

@@ -304,6 +304,10 @@ def prepare_prompt_parameters
       # Render out proc/lamda attributes before rendering templates
       parameters.deep_transform_values! { _1.respond_to?(:call) ? _1.call : _1 }
 
+      # Strip parameters the target model rejects (e.g. temperature/top_p
+      # on thinking-first models) before they reach the provider.
+      ModelCapabilities.sanitize!(parameters)
+
       # Apply Callbacks
       parameters.merge!(
         trace_id: prompt_options[:trace_id] || SecureRandom.uuid,

lib/active_agent/model_capabilities.rb

@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+module ActiveAgent
+  # Per-model capability quirks, applied before a request reaches the
+  # provider. Vendors ship models that reject otherwise-standard sampling
+  # parameters (thinking-first models steered by prompting/effort instead)
+  # with an API 400 — this registry strips those parameters up front so an
+  # agent configured with a shared temperature keeps working across model
+  # switches.
+  #
+  # The built-in rules cover the known families; apps can extend the
+  # registry for new or self-hosted models:
+  #
+  # @example Register a custom rule
+  #   ActiveAgent::ModelCapabilities.register(/\Amy-reasoning-model/, unsupported: [:temperature, :top_p])
+  #
+  # @example Disable sanitization entirely
+  #   ActiveAgent::ModelCapabilities.enabled = false
+  module ModelCapabilities
+    SAMPLING_PARAMS = [ :temperature, :top_p ].freeze
+
+    # Model families that reject sampling parameters with a 400:
+    # - Anthropic thinking-first models (Opus 4.7+, Opus 5, Sonnet 5,
+    #   Fable 5 / Mythos 5)
+    # - OpenAI reasoning models (o-series, GPT-5 family)
+    BUILTIN_RULES = [
+      { pattern: /\Aclaude-(opus-5|opus-4-[78]|sonnet-5|fable-5|mythos-5)/, unsupported: SAMPLING_PARAMS },
+      { pattern: /\A(o1|o3|o4)(-|$)/, unsupported: SAMPLING_PARAMS },
+      { pattern: /\Agpt-5/, unsupported: SAMPLING_PARAMS }
+    ].freeze
+
+    class << self
+      # Master switch; on by default. Set false to send parameters through
+      # untouched (the vendor then enforces its own rules).
+      attr_writer :enabled
+
+      def enabled
+        return @enabled unless @enabled.nil?
+
+        true
+      end
+
+      # Registers an app-defined capability rule ahead of the built-ins.
+      #
+      # @param pattern [Regexp] matched against the model name
+      # @param unsupported [Array<Symbol>] parameter keys the model rejects
+      def register(pattern, unsupported:)
+        custom_rules << { pattern: pattern, unsupported: unsupported.map(&:to_sym) }
+      end
+
+      def custom_rules
+        @custom_rules ||= []
+      end
+
+      def reset!
+        @custom_rules = []
+        @enabled = nil
+      end
+
+      # @return [Array<Symbol>] parameter keys the model rejects
+      def unsupported_params(model)
+        return [] if model.nil?
+
+        (custom_rules + BUILTIN_RULES).each do |rule|
+          return rule[:unsupported] if model.to_s.match?(rule[:pattern])
+        end
+        []
+      end
+
+      def sampling_supported?(model)
+        (unsupported_params(model) & SAMPLING_PARAMS).empty?
+      end
+
+      # Strips parameters the model rejects, in place. Returns the removed
+      # keys (empty when nothing applied).
+      #
+      # @param parameters [Hash] prepared prompt parameters (must carry :model)
+      # @return [Array<Symbol>] removed parameter keys
+      def sanitize!(parameters)
+        return [] unless enabled
+        return [] unless parameters.is_a?(Hash)
+
+        removed = unsupported_params(parameters[:model]).select { |key| parameters.key?(key) }
+        removed.each { |key| parameters.delete(key) }
+        removed
+      end
+    end
+  end
+end

lib/active_agent/providers/_base_provider.rb

@@ -55,7 +55,13 @@ class ProvidersError < StandardError; end
                     :request, :message_stack,        # Runtime
                     :stream_broadcaster, :streaming, # Callback (Streams)
                     :tools_function,                 # Callback (Tools)
-                    :usage_stack                     # Usage Tracking
+                    :usage_stack,                    # Usage Tracking
+                    :max_tool_turns, :tool_turns     # Tool-loop safety
+
+      # Upper bound on tool-calling round-trips within one generation. A
+      # model that keeps emitting tool calls otherwise recurses until the
+      # provider stops it — override per agent/prompt with max_tool_turns:.
+      DEFAULT_MAX_TOOL_TURNS = 25
 
       # @return [String] e.g., "Anthropic", "OpenAI"
       def self.service_name
@@ -106,6 +112,8 @@ def initialize(kwargs = {})
         self.stream_broadcaster = kwargs.delete(:stream_broadcaster)
         self.streaming          = false
         self.tools_function     = kwargs.delete(:tools_function)
+        self.max_tool_turns     = kwargs.delete(:max_tool_turns) || DEFAULT_MAX_TOOL_TURNS
+        self.tool_turns         = 0
         self.options            = options_klass.new(kwargs.extract!(*options_klass.keys))
         self.context            = kwargs
         self.message_stack      = []
@@ -344,7 +352,7 @@ def process_prompt_finished(api_response = nil)
           message_stack.push(*api_messages)
         end
 
-        if (tool_calls = process_prompt_finished_extract_function_calls)&.any?
+        if (tool_calls = process_prompt_finished_extract_function_calls)&.any? && tool_turn_allowed?
           process_function_calls(tool_calls)
           resolve_prompt
         else
@@ -373,6 +381,19 @@ def process_prompt_finished(api_response = nil)
         end
       end
 
+      # Counts a tool round-trip against the per-generation cap. When the
+      # cap is hit the loop finishes cleanly with the messages gathered so
+      # far (a partial result) instead of recursing indefinitely.
+      #
+      # @return [Boolean] whether another tool round-trip may run
+      def tool_turn_allowed?
+        self.tool_turns += 1
+        return true if max_tool_turns.nil? || tool_turns <= max_tool_turns
+
+        instrument("tool_turns_exceeded.active_agent", limit: max_tool_turns)
+        false
+      end
+
       # @abstract
       # @param api_response [Object]
       # @return [Array<Message>, nil]

lib/active_agent/providers/concerns/exception_handler.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require_relative "../errors"
+
 module ActiveAgent
   module Providers
     # Provides exception handling for provider operations.
@@ -54,7 +56,16 @@ def configure_exception_handler(exception_handler: nil)
       def with_exception_handling(&block)
         yield
       rescue => exception
-        rescue_with_handler(exception) || raise
+        # Vendor API failures are normalized into the framework taxonomy
+        # (Errors::RateLimited, Errors::ContextLengthExceeded, ...) so
+        # rescue_from policy is portable across providers; the original
+        # exception is preserved as #cause. Anything unrecognizable —
+        # including ordinary Ruby errors — passes through untouched.
+        exception = Errors::Taxonomy.normalize(
+          exception,
+          provider_tag: (tag_name if respond_to?(:tag_name))
+        )
+        rescue_with_handler(exception) || raise(exception)
         nil # Discard handler return value to prevent polluting raw_response
       end