$ref/$defs rejected in the extraction schema

[Cyrille-Lighton]

Bug Report

Reported By: User

Affected Environment: Prod

Affected Release: Console 1.1.X

Description

As a developer integrating the LightOn Console API, when I submit a nested pydantic model to POST /api/v3/extract (whose model_json_schema() automatically emits $ref/$defs as soon as it contains a sub-model), the request is rejected because the SDK does not resolve these references before sending. This results in a 422 validation_error, with the error pointing to the $ref field (e.g. schema.properties.addr.$ref: invalid). The documented happy path ("pass a pydantic model") therefore only works for strictly flat models; the behavior is identical in sync and async mode, since schema validation happens before the job is queued (no silent failure). Confirmed under real conditions (valid API key + real PDF, multipart upload).

Steps to Reproduce

1. Define a pydantic model with a nested sub-model (e.g. Company containing Address)
2. Call client.extract(schema=Company, path=...) or send the equivalent raw JSON Schema
3. Observe the 422 response with fields: {"schema.properties.<field>.$ref": ...}
4. Compare with the same schema manually inlined (no $ref/$defs) → 200

Expected Behavior

The API should accept any valid JSON schema produced by a pydantic model, including nested models. Either the SDK should resolve (inline/dereference) $ref/$defs before sending, or the /api/v3/extract endpoint should support standard JSON Schema references, so that the documented happy path ("pass a pydantic model") works for nested models and not only strictly flat ones.

Actual Behavior

Any schema containing $ref/$defs is rejected with a 422 validation_error. Since model_json_schema() automatically generates these as soon as a model contains a sub-model, and the SDK does not resolve them before sending, nested models fail. The error points to the reference field (e.g. schema.properties.addr.$ref: invalid). Behavior is identical in sync and async mode, as schema validation happens before the job is queued (no silent failure).

Workaround

Manually dereference the schema before sending (inline the $defs into properties), tested and working. Ideally fixed on the SDK side (lighton-sdk), since it already owns the pydantic → JSON Schema conversion.

Additional Information

[Screenshot or Screencast of user's experience]

Error Code: Error Code: 422 ; {"error":"validation_error","fields":{"schema.properties.<field>.$ref":[{"error":"invalid","detail":"..."}]}}

Workaround Suggested By: Support

For the workaround itself: flatten the schema before sending, either by resolving/inlining the $ref/$defs references (dereferencing the schema) or by defining a strictly flat pydantic model with no sub-models.

[MightyGoldenOctopus]

Fix merged, will ship in release 1.1.0