Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions .agents/skills/loom-framework/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -2203,8 +2203,13 @@ filter, and serialization rules belong here.
do not infer body presence from direct or inherited field counts. Constructor
legality and response presence are separate obligations; keep the imported
headerless, selected and inherited empty-object wire/spec controls aligned.
- `expr.explicitHTTPResponseBody` prepares both runtime and documentation-only
response shapes. Preserve authored zero-field objects; only the `Empty` sentinel
- `expr.explicitHTTPResponseBody` prepares runtime response shapes.
Documentation bodies own a copied schema graph and finalize it directly;
they must not acquire runtime wrappers or forced canonical component names.
`attributeValidator` separates schema visits from runtime codec admission.
Schema validation preserves contract constraints while omitting generated
untagged codec shape restrictions; never share its visited cache with runtime
validation. Keep the documentation-schema model and raw-body specimen aligned. Preserve authored zero-field objects; only the `Empty` sentinel
means no explicit body. Field-count inference belongs to the separate inferred
body projection. Response plans, codecs and OpenAPI consume the prepared shape;
never compensate for discarded presence in individual generators. Keep the
Expand Down Expand Up @@ -2732,6 +2737,7 @@ go fmt ./...
make lint
make test
make coverage-ratchet # protected consumer-aware boundaries
make test-ci # full root suite + scoped ratchet, without duplicate tests
make openapi-contract # OpenAPI work
make generated-code-quality # generated Go/output work
make test-testdata-compile # all exported design fixtures: gen, example, build, vet
Expand All @@ -2747,6 +2753,15 @@ consumer-aware profile. Treat any reduction as an explicit reviewed contract
change and document its reason; aggregate coverage remains an inspection
artifact, not the ratchet.

`make ci` and `make ci-local` use `test-ci` instead of running `test` followed
by `coverage-ratchet`. The coverage runner expands the configured consumer set
and the complete root-module inventory, runs each package once, and checks the
floors using only the configured consumers. Remaining packages cannot inflate
the ratchet. It publishes the combined `cover.out` only after both groups and
the coverage floors pass. Standalone `test` and `coverage-ratchet` remain useful
for local iteration; do not chain them in CI. Race/shuffle, generated-design,
contract and integration tiers retain their separate checks.

## Local and Remote Source Modes

HTTP and JSON-RPC temp-module tests share a worktree-local source selector:
Expand Down
4 changes: 4 additions & 0 deletions .agents/skills/loom-framework/references/formal-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ stated predicates under its hypotheses; it does not establish handwritten Go
refinement, correct plan extraction, arbitrary codec behavior or generated-code
correctness. Keep proved, tested, bounded-checked and assumed claims distinct.

The [documentation schema admission model](../../../../expr/tla/documentation_schema/README.md)
separates valid documentation schemas from unsupported generated codecs and
checks that validation visits cannot leak between those contexts.

## Compilation value ownership

The [service semantic ownership model](../../../../codegen/service/tla/semantic_ownership/README.md)
Expand Down
6 changes: 6 additions & 0 deletions .agents/skills/loom/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -1047,6 +1047,12 @@ clients and servers after upgrading to adopt typed Tag binding.
stream. Declare `ContentType(...)` on the response. The generated server
applies that static media type before it streams the body. Use
`OpenAPIBody(...)` only to document the schema.
- Documentation body modifiers are local to their occurrence. Use
`OpenAPIBody(OneOf(String, Document), Untagged)` or the equivalent
`OpenAPIRequestBody` call for untagged scalar/object or nested union schemas.
Shared named runtime types keep their codecs. Apply `Untagged` at the body,
not on a named type that violates runtime branch rules; nested unions retain
their authored representation. Regenerate OpenAPI to adopt this support.
- String-backed path, query, header, and cookie fields with
`Meta("struct:field:type", ...)` decode through `encoding.TextUnmarshaler`.
CLI flags accept raw text through the same parser; omitted optional fields
Expand Down
13 changes: 11 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,17 @@ jobs:
go-version-file: go.mod
id: go

- name: Build
run: make ci
- name: Install dependencies
run: make depend

- name: Static checks
run: make lint

- name: Unit tests and coverage ratchet
run: make test-ci

- name: Transport integration tests
run: make integration-test

- name: Upload test coverage for deep source
uses: actions/upload-artifact@v7
Expand Down
18 changes: 15 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ PROTOC_GEN_GO_GRPC_VERSION?=v1.6.2
PROTOC_BIN=protoc
PROTOC_DEST=$(GOBIN_DIR)/$(PROTOC_BIN)

.PHONY: all all-tests ci ci-local clean coverage-baseline coverage-ratchet depend fmt install-hooks lint lint-docs lint-filesize lint-gofmt lint-json-v2 lint-legacy-middleware lint-namescope lint-test-sources lint-toolchain test test-race test-pulse-redis test-testdata-compile integration-test integration-test-fast generated-code-quality openapi-contract value-contract-proof value-contract-conformance build-loom build-loom-cached loom-local loom-remote loom-status release release-promote
.PHONY: all all-tests ci ci-local clean coverage-baseline coverage-ratchet depend fmt install-hooks lint lint-docs lint-filesize lint-gofmt lint-json-v2 lint-legacy-middleware lint-namescope lint-test-sources lint-toolchain test test-ci test-race test-pulse-redis test-testdata-compile integration-test integration-test-fast generated-code-quality openapi-contract value-contract-proof value-contract-conformance build-loom build-loom-cached loom-local loom-remote loom-status release release-promote
.NOTPARALLEL: release ci-local

# Only list test and build dependencies
Expand All @@ -46,7 +46,9 @@ all: lint test integration-test

all-tests: lint test integration-test

ci: depend all coverage-ratchet
# test-ci combines the full unit suite and the scoped coverage ratchet. Running
# all + coverage-ratchet would execute the expensive generator suites twice.
ci: depend lint test-ci integration-test

# Hermetic targets must never reach a real Redis server: the pulse tests flush
# the server named by LOOM_PULSE_REDIS_ADDR. Only test-pulse-redis passes it on.
Expand All @@ -59,7 +61,7 @@ unexport LOOM_PULSE_REDIS_ADDR
# rsync, and network access are also required by the external contract gates.
# Formal verification is opt-in design work and must not enter this dependency tree.
# The source mode is intentionally inherited from the worktree or LOOM_DIR.
ci-local: all coverage-ratchet test-race openapi-contract generated-code-quality test-testdata-compile
ci-local: lint test-ci integration-test test-race openapi-contract generated-code-quality test-testdata-compile

# Install protoc
PROTOC_VERSION=36.2
Expand Down Expand Up @@ -166,6 +168,16 @@ endif
coverage-ratchet:
go run ./scripts/coveragecheck -test-timeout=$(LOOM_TEST_TIMEOUT)

# Every root test runs exactly once. Protected consumers retain their scoped
# coverage instrumentation; all remaining packages contribute ordinary coverage.
# The aggregate cover.out is published only after tests and ratchets pass.
test-ci:
ifneq ($(GOOS),windows)
PATH="$(GOBIN_DIR):$$PATH" go run ./scripts/coveragecheck -suite -coverprofile=cover.out -test-timeout=$(LOOM_TEST_TIMEOUT)
else
go run ./scripts/coveragecheck -suite -coverprofile=cover.out -test-timeout=$(LOOM_TEST_TIMEOUT)
endif

# Rewrite the checked-in floors from the current consumer-aware profile. Review
# every reduction and document why the protected boundary lost coverage.
coverage-baseline:
Expand Down
13 changes: 13 additions & 0 deletions docs/dsl-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -1119,6 +1119,19 @@ form, multipart, optional JSON body, and non-raw request modes.
same request schema for multiple media types. Loom does not decode these
formats. The service must inspect the content type and decode the raw stream.

Documentation-only bodies accept scalar/object unions and branches containing
nested unions. For example, `OpenAPIBody(OneOf(String, Document), Untagged)`
publishes a `oneOf` without a discriminator envelope. The equivalent request
form is `OpenAPIRequestBody(OneOf(String, Document), "application/json", true, Untagged)`.
A modifier applies only to that body occurrence, even when its argument is a
shared named type. Nested unions retain their own authored tagged or untagged
representation. Required fields, constraints, and branch identities are still
validated. Generated runtime untagged codecs retain their narrower branch rules;
a named type declared with `Untagged()` must still satisfy those rules. Apply
`Untagged` on the documentation body to describe a broader raw wire contract.
Regenerate OpenAPI artifacts to adopt this support; raw stream handling remains
application-owned.

`Link(...)` inside an HTTP `Response(...)` publishes OpenAPI response links.
Use `LinkOperation(...)` or `LinkOperationRef(...)` to identify the target
operation, `LinkParam(...)` to map link parameters from runtime expressions,
Expand Down
7 changes: 7 additions & 0 deletions docs/http-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -674,6 +674,13 @@ generated handler sets this static header before it streams the body.
encoding. Raw body modes are HTTP-only. You cannot combine them with gRPC or
streaming payload or result methods.

Documentation bodies can describe untagged scalar/object unions and nested
unions, for example `OpenAPIBody(OneOf(String, Manifest), Untagged)`. The same
schema works with `OpenAPIRequestBody`. These modifiers are local to the body;
sharing a named type with a typed method does not change that method's codec.
The application remains responsible for producing or consuming the documented
raw bytes. Runtime untagged union restrictions still apply to typed methods.

Use `FileResponse` for seekable downloads that need standard range and
conditional request behavior. It is HTTP-only and accepts explicit `GET` and
`HEAD` routes; Loom does not synthesize a `HEAD` route.
Expand Down
2 changes: 2 additions & 0 deletions dsl/attribute_oneof.go
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ func OneOf(arg any, args ...any) expr.DataType {
// branch must be a concrete named object or array containing primitives, named
// objects, or arrays of these shapes. Encoding and decoding require the same
// unique branch in both the JSON schema and the generated Go representation.
// In OpenAPIBody and OpenAPIRequestBody, Untagged describes only the schema:
// scalar and nested union branches are allowed, and runtime types are unchanged.
func Untagged() {
attribute, ok := eval.Current().(*expr.AttributeExpr)
if !ok {
Expand Down
14 changes: 11 additions & 3 deletions dsl/http_endpoint_body.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,10 @@ func Body(args ...any) {
// the generated OpenAPI contract only and does not change runtime transport
// encode/decode behavior.
//
// OpenAPIBody accepts the same arguments as Body.
// OpenAPIBody accepts the same arguments as Body, plus direct data types.
// Its schema modifiers affect only this documentation occurrence. Untagged
// unions may contain scalar, collection, object, or nested union branches;
// the generated runtime codec restrictions do not apply to documentation.
func OpenAPIBody(args ...any) {
body(args, true)
}
Expand All @@ -86,7 +89,9 @@ func OpenAPIBody(args ...any) {
// The body argument may be a payload attribute name, data type, or user type.
// The contentType argument is the request media type, and required controls the
// OpenAPI requestBody.required value. An optional DSL function may set schema
// details such as Description, Example, Format, or Meta.
// details such as Description, Example, Format, Meta, or Untagged. Modifiers
// affect only this documentation occurrence. Untagged unions admit scalar and
// nested branches without enabling generated codecs for those shapes.
//
// Example:
//
Expand Down Expand Up @@ -231,13 +236,16 @@ func resolveBodyAttribute(args []any, ref *expr.AttributeExpr, kind string, open
attr, ok := resolveNamedBodyAttribute(ref, kind, a)
return attr, fn, ok
case expr.UserType:
if openAPIOnly {
a = expr.Dup(a).(expr.UserType)
}
return &expr.AttributeExpr{Type: a}, fn, true
case expr.DataType:
if !openAPIOnly {
eval.InvalidArgError("attribute name, user type or DSL", a)
return nil, nil, false
}
return &expr.AttributeExpr{Type: a}, fn, true
return &expr.AttributeExpr{Type: expr.Dup(a)}, fn, true
case func():
if ref == nil {
if !openAPIOnly {
Expand Down
20 changes: 20 additions & 0 deletions expr/attribute_schema.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
package expr

import "github.com/CaliLuke/loom/eval"

func (a *AttributeExpr) validateSchema(ctx string, parent eval.Expression) *eval.ValidationErrors {
validator := attributeValidator{schemaOnly: true, seen: make(map[*AttributeExpr]bool)}
return validator.validate(a, ctx, parent)
}

// finalizeSchema preserves authored documentation without runtime body wrappers.
// An explicit occurrence name still claims a component, including inline schemas.
func (a *AttributeExpr) finalizeSchema() {
if name, ok := a.Meta.Last("openapi:typename"); ok && name != "" {
a.AddMeta("openapi:typename:canonical", "true")
if _, named := a.Type.(UserType); !named && a.Type != Empty {
a.Type = &UserTypeExpr{TypeName: name, AttributeExpr: DupAtt(a)}
}
}
a.Finalize()
}
64 changes: 64 additions & 0 deletions expr/attribute_schema_validation_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
package expr

import (
"testing"

"github.com/stretchr/testify/require"
)

func TestSchemaValidationDoesNotAdmitUnsupportedCodecs(t *testing.T) {
for _, schemaFirst := range []bool{false, true} {
name := "runtime-first"
if schemaFirst {
name = "schema-first"
}
t.Run(name, func(t *testing.T) {
union := &AttributeExpr{Type: &Union{TypeName: "ScalarChoice", Untagged: true, Values: []*NamedAttributeExpr{
{Name: "text", Attribute: &AttributeExpr{Type: String}},
{Name: "number", Attribute: &AttributeExpr{Type: Int}},
}}}
attribute := &AttributeExpr{Type: &Map{KeyType: &AttributeExpr{Type: String}, ElemType: &AttributeExpr{
Type: &Array{ElemType: &AttributeExpr{Type: &Object{{Name: "choice", Attribute: union}}}},
}}}
checkSchema := func() {
require.Empty(t, attribute.validateSchema("documentation", attribute).Errors)
}
checkRuntime := func() {
require.ErrorContains(t, attribute.Validate("runtime", attribute), "must be a concrete named object or array type")
}
if schemaFirst {
checkSchema()
checkRuntime()
} else {
checkRuntime()
checkSchema()
}
})
}
}

func TestSchemaValidationPreservesContractChecks(t *testing.T) {
for _, tc := range []struct {
name string
attribute *AttributeExpr
message string
}{
{
name: "required field",
attribute: &AttributeExpr{Type: &Object{}, Validation: &ValidationExpr{Required: []string{"missing"}}},
message: `required field "missing" does not exist`,
},
{
name: "duplicate union identity",
attribute: &AttributeExpr{Type: &Union{TypeName: "Choice", Untagged: true, Values: []*NamedAttributeExpr{
{Name: "same", Attribute: &AttributeExpr{Type: String}},
{Name: "same", Attribute: &AttributeExpr{Type: Int}},
}}},
message: "repeats logical name",
},
} {
t.Run(tc.name, func(t *testing.T) {
require.ErrorContains(t, tc.attribute.validateSchema("documentation", tc.attribute), tc.message)
})
}
}
38 changes: 26 additions & 12 deletions expr/attribute_validate.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,27 @@ import (
"github.com/CaliLuke/loom/eval"
)

// attributeValidator keeps schema-only visits separate from runtime admission.
// A documentation occurrence must not mark a shared codec contract as checked.
type attributeValidator struct {
schemaOnly bool
seen map[*AttributeExpr]bool
}

// Validate tests whether the attribute required fields exist. Since attributes
// are unaware of their context, additional context information can be provided
// to be used in error messages. The parent definition context is automatically
// added to error messages.
func (a *AttributeExpr) Validate(ctx string, parent eval.Expression) *eval.ValidationErrors {
if validated[a] {
validator := attributeValidator{seen: validated}
return validator.validate(a, ctx, parent)
}

func (v *attributeValidator) validate(a *AttributeExpr, ctx string, parent eval.Expression) *eval.ValidationErrors {
if v.seen[a] {
return nil
}
validated[a] = true
v.seen[a] = true
verr := new(eval.ValidationErrors)
if err := a.resolveTypeRef(); err != nil {
verr.Add(parent, "%s", err.Error())
Expand All @@ -39,7 +51,7 @@ func (a *AttributeExpr) Validate(ctx string, parent eval.Expression) *eval.Valid
verr.Merge(a.validatePresence(ctx, parent))
verr.Merge(a.validateStructTagKeys(ctx, parent))
verr.Merge(validateStructMeta(ctx, a.Meta, parent))
verr.Merge(a.validateChildTypes(ctx, parent))
verr.Merge(v.validateChildTypes(a, ctx, parent))
verr.Merge(a.validateViewReference(ctx, parent))

return verr
Expand Down Expand Up @@ -287,30 +299,32 @@ func (a *AttributeExpr) validatePkgPath(pkgPath string, t DataType) *eval.Valida
return nil
}

func (a *AttributeExpr) validateChildTypes(ctx string, parent eval.Expression) *eval.ValidationErrors {
func (v *attributeValidator) validateChildTypes(a *AttributeExpr, ctx string, parent eval.Expression) *eval.ValidationErrors {
verr := new(eval.ValidationErrors)
if o := AsObject(a.Type); o != nil {
verr.Merge(a.validateObjectChildren(ctx, parent, o))
verr.Merge(v.validateObjectChildren(a, ctx, parent, o))
return verr
}
if ar := AsArray(a.Type); ar != nil {
verr.Merge(ar.ElemType.Validate(appendValidationContext(ctx, "[]"), a))
verr.Merge(v.validate(ar.ElemType, appendValidationContext(ctx, "[]"), a))
return verr
}
if mapping := AsMap(a.Type); mapping != nil {
verr.Merge(mapping.KeyType.Validate(appendValidationContext(ctx, "[key]"), a))
verr.Merge(mapping.ElemType.Validate(appendValidationContext(ctx, "[value]"), a))
verr.Merge(v.validate(mapping.KeyType, appendValidationContext(ctx, "[key]"), a))
verr.Merge(v.validate(mapping.ElemType, appendValidationContext(ctx, "[value]"), a))
return verr
}
if u := AsUnion(a.Type); u != nil {
verr.Merge(validateUnionBranchIdentities(u, ctx, parent))
if u.Untagged {
verr.Merge(validateUntaggedBranches(u, ctx, parent))
if !v.schemaOnly {
verr.Merge(validateUntaggedBranches(u, ctx, parent))
}
} else {
verr.Merge(validateUnionWireKeys(ctx, parent, u))
}
for _, ut := range u.Values {
verr.Merge(ut.Attribute.Validate(appendValidationContext(ctx, "<"+ut.Name+">"), parent))
verr.Merge(v.validate(ut.Attribute, appendValidationContext(ctx, "<"+ut.Name+">"), parent))
}
if hasInlineUnionCycle(u) {
verr.Add(parent, "%srecursive OneOf branch cycle in %q is not supported; place the recursive reference inside an object field", ctx, u.Name())
Expand Down Expand Up @@ -359,7 +373,7 @@ func attributeJSONTag(attribute *AttributeExpr) (string, bool) {
// the service types, which name a field declared as "n:m" after its attribute
// name "n". HTTPEndpointExpr.validateBodyElementNames checks the element names
// that HTTP and JSON-RPC bodies use.
func (a *AttributeExpr) validateObjectChildren(ctx string, parent eval.Expression, obj *Object) *eval.ValidationErrors {
func (v *attributeValidator) validateObjectChildren(a *AttributeExpr, ctx string, parent eval.Expression, obj *Object) *eval.ValidationErrors {
verr := new(eval.ValidationErrors)
for _, name := range a.AllRequired() {
if a.Find(name) == nil {
Expand All @@ -375,7 +389,7 @@ func (a *AttributeExpr) validateObjectChildren(ctx string, parent eval.Expressio
for _, nat := range *obj {
verr.Merge(a.validatePkgPath(pkgPath, nat.Attribute.Type))
fieldCtx := appendValidationContext(ctx, nat.Name)
verr.Merge(nat.Attribute.Validate(fieldCtx, parent))
verr.Merge(v.validate(nat.Attribute, fieldCtx, parent))
name := AttributeName(nat.Name)
wireName := JSONFieldName(name, nat.Attribute)
if wireName == "-" {
Expand Down
Loading
Loading