Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ The versioned guides in [`docs/`](docs/README.md) are published at the [document
| Write a Go application client | [Go client library](docs/client.md) |
| Write a debuglet | [Write a debuglet](docs/debuglets.md) |
| Build visually or with an MCP client | [Workflow and experiment authoring](docs/development/workflow-authoring.md) |
| Share a lab's experiments and results | [Communities](docs/communities.md) |
| Integrate with the HTTP API | [OpenAPI contract](api/openapi.yaml) and [API guide](docs/api.md) |
| Understand the code and protocol | [Architecture](docs/architecture.md) |
| Operate a service | [Managed services](docs/operations/services.md) |
Expand Down
2,767 changes: 2,731 additions & 36 deletions api/openapi.yaml

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions api/spec.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ var OpenAPI []byte
const (
// Version is the HTTP API contract version as major.minor. It equals the
// info.version field of OpenAPI.
Version = "1.22"
Version = "1.23"
// Major is the compatibility number of Version. Clients that require a
// different major version are answered with an explicit incompatibility
// error instead of a best-effort response.
Expand Down Expand Up @@ -48,7 +48,8 @@ const (
// 1.20 added admission and unavailable drain status to account-owned executors.
// 1.21 added authenticated historical attribution exports.
// 1.22 added owner-managed executor presentation labels.
Minor = 22
// 1.23 added communities, shared experiments and funded community measurements.
Minor = 23
// VersionHeader carries the contract version a client requires on requests
// and the version the dispatcher implements on responses.
VersionHeader = "Debuglet-API-Version"
Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ This directory is the versioned source for the [Debuglet documentation site](htt
- [CLI reference](cli.md) — commands, local demo, and scriptable workflows.
- [Go client library](client.md) — submit debuglets and read results from an application.
- [Reusable and coordinated measurements](measurements.md) — saved profiles, retries and experiments across executors.
- [Communities](communities.md) — shared projects, editable experiments, results and group funding.
- [Portable results](results.md) — export retained output and admission facts for offline analysis.
- [Controlled latency experiment](research/latency-evaluation.md) — compare native and WASM probes under known local network faults.
- [Write a debuglet](debuglets.md) — execution model and Go authoring interface.
Expand Down
86 changes: 85 additions & 1 deletion docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ A `TEST` intent has no payment backend: its transaction row is the intent, writt

Every intent records the name of the rule that priced it in the `pricing_rule` column of its transaction; the rule above is `bw-s-ceil-ms-v1`, and a transaction written before the column existed carries an empty name and was priced by the rule of the release that wrote it. A `USDC` intent is a local record of the price, the auth key and the addresses the payer uses, created without any chain call, so an intent that fails leaves nothing external to reconcile.

Creating a `USDC` intent requires `refund_address` to be a Sui address: `0x` followed by one to 64 hexadecimal digits; short addresses such as `0x1` are accepted. An omitted, empty or malformed address returns HTTP 400 `invalid_request`, with `field_errors` naming `refund_address` and code `invalid_address`, before the intent or any order is written. While blockchain payments are disabled, the method still returns HTTP 503 `payments_disabled` first. `TEST` does not validate this field and accepts an omitted, empty or arbitrary value. `POST /payment/quote` does not evaluate `refund_address`.
Creating a personal `USDC` intent requires `refund_address` to be a Sui address: `0x` followed by one to 64 hexadecimal digits; short addresses such as `0x1` are accepted. An omitted, empty or malformed address returns HTTP 400 `invalid_request`, with `field_errors` naming `refund_address` and code `invalid_address`, before the intent or any order is written. While blockchain payments are disabled, the method still returns HTTP 503 `payments_disabled` first. `TEST` does not validate this field and accepts an omitted, empty or arbitrary value. `POST /payment/quote` does not evaluate `refund_address`. A community-funded intent ignores it: refunds return to that community's prepaid budget.

## Quotes and economics (API 1.15)

Expand Down Expand Up @@ -171,6 +171,90 @@ with a new schedule and payment intent. Template references name an immutable
version and matching program digest. Existing runs are independent of later
profile edits or deletion. See OpenAPI for field and account limits.

## Communities (API 1.23)

Communities group individually authenticated accounts. They own projects,
versioned workflow or experiment source, and references to shared measurements.
They do not create a shared login or give members dispatcher operator authority.
See the [community guide](communities.md) for the console flow, funding model and
disclosure before sharing.

| Routes | Purpose |
| --- | --- |
| `GET/POST /communities`, `GET/PATCH /communities/{id}` | List, create, read or rename a community |
| `GET /communities/{id}/members` | List current members |
| `POST /communities/{id}/invitations`, `POST /community-invitations/accept` | Create and accept a one-use, seven-day invitation |
| `DELETE /communities/{id}/members/{user_id}` | Owner removes a member; the creator cannot be removed |
| `GET/POST /communities/{id}/projects`, `GET/PATCH /community-projects/{id}` | Read, create or rename shared projects |
| `GET/POST /community-projects/{id}/experiments` | List experiment metadata or save source and an optional matching bundle |
| `GET/PUT/DELETE /community-experiments/{id}` | Read source, save an immutable next revision, or delete an unused experiment |
| `GET/POST /community-projects/{id}/measurements` | List references or explicitly share an owned measurement |
| `DELETE /community-projects/{id}/measurements/{measurement_id}` | Sharer or community owner removes the reference |
| `GET /communities/{id}/wallet`, `POST /communities/{id}/wallet/top-ups` | Read the shared budget or create an owner-signed USDC deposit intent |

All routes require an authenticated account, even in the local development
profile. Membership is checked on each request; hidden resources return 404.
Community creation and renaming, membership management and top-ups require a
browser session. Scoped API credentials can read community work with
`measurements:read`, read the wallet with `account:read`, and change projects,
experiments and measurement references with `measurements:write`. Cookie
mutations require the normal CSRF proof.

Lists are JSON arrays bounded by the resource quotas in the community guide.
Community IDs are UUIDs; measurement IDs retain their existing transaction ID
format. Community timestamps are UTC RFC3339 strings. Names use 1–120 trimmed
UTF-8 bytes and descriptions at most 2000 bytes.

Experiment creation takes `{name, kind, document, bundle?, description?, collection?}`, where `kind` is
`workflow` or `experiment` and the document uses the existing authoring format.
Responses include the saved `revision`. Updates additionally supply the current
revision; a stale value returns 409 without changing the record.
`GET /community-experiments/{id}?revision=N` reads an immutable historical
revision. Lists omit the document and bundle. Omitting the bundle on an update
removes it from the new revision. The entire save request is limited to 24 MiB;
workflow source remains limited to 1 MiB and experiment source to 4 MiB.
Experiment descriptions use at most 4000 UTF-8 bytes and trimmed collection
labels at most 120 bytes.
`DELETE /community-experiments/{id}` requires current membership and either the
experiment's creator or the community owner. It returns 204 after deleting the
experiment and all its revisions, freeing their storage usage. A measurement
reference to any revision prevents deletion with 409; another member receives
403 and a non-member receives 404. Measurement records and output are unchanged.

Sharing takes `{measurement_id, experiment_id?, experiment_revision?}`. The
optional experiment fields must name a revision in that project. It grants
current members read access through `GET /measurements/{id}` and
`GET /debuglet/{id}/detail`, `/state`, `/logs`, `/result`, `/recovery` and
`/cancellation`. Reads include the original configuration, retained output,
provenance and per-run costs, including private executor addressing and host
platform observations when recorded. They do not expose payment authorization
keys, refund wallets or private payment history. Sharing never grants run
mutation authority: `can_manage` on measurement and run details is false for a
community reader. Existing limits, output finality and payload tombstones apply.

For a new community-funded run, send `community_project_id` in the normal
payment quote and intent request. The quote checks membership and prices the
request without reserving funds. Creating an intent atomically reserves the
community's budget and returns `method: "COMMUNITY"`, its `currency`, a
submission `intent` and the confirmed `quote`. An already-recorded retry may
omit the quote, as in the personal retry flow. Submit with the returned
authorization using the normal debuglet API; no additional external-wallet
payment is required for that run. The submitting member remains the run owner.
Insufficient funds returns 409, without creating the intent. Personal payment
requests that omit the project retain their existing behavior.
Optional `community_experiment_id` and `community_experiment_revision` associate
the new measurement with a saved revision in that project.

USDC top-ups take `{amount, refund_address}`, where `amount` is an exact positive
decimal string of micro-USDC, at most 9007199254740991. Only the community owner creates and signs the
returned Sui payment intent. A deposit intent cannot authorize a measurement;
only a confirmed receipt adds the deposit once. Deposits are funds held by the
dispatcher, not permission to debit the owner's external wallet. In TEST mode,
the community uses the creator's existing allowance; there is no top-up or
automatic grant, and disabled allowances disable community spending.
The supported chain profile is Sui Testnet; there is no endpoint to withdraw
unused community deposits.

## Measurement history

API 1.12 adds account-owned `GET /measurements` and `GET /measurements/{id}`.
Expand Down
166 changes: 166 additions & 0 deletions docs/communities.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
# Work together in a community

A community gives a research group a shared workspace for projects, editable
experiments and measurement results. Everyone signs in with their own account.
The person who creates the community owns it, invites members and funds its
shared measurement budget.

Projects organize related work. A saved experiment contains either one workflow
or a coordinated experiment with several participants. Measurements are actual
runs, with their original executor choices, configuration, output and provenance.
Saving an experiment does not execute it.

## Set up a group

Open **Communities** in the console and choose **Create community**, then create
a project for an investigation or study. The **Projects**, **People** and
**Funding** tabs keep shared work, membership and the budget together. Under
**People**, the owner can choose **Invite someone** and send the link privately.
The recipient signs in with their own account, then chooses **Join community**.

An invitation expires after seven days and can be used once. It is a credential
that lets its recipient join the group, so do not publish it in a public issue or
repository. Owners can remove members. A removed member loses subsequent access
to shared work and cannot reserve new community spending; previously downloaded
copies cannot be recalled.

| Action | Member | Community owner |
| --- | --- | --- |
| Read shared projects, experiments and results | Yes | Yes |
| Create or edit projects and experiments | Yes | Yes |
| Delete an unused saved experiment | Their own experiment | Any unused experiment |
| Share a personally owned measurement | Yes | Yes |
| Submit using the available community budget | Yes | Yes |
| Remove a personally funded measurement reference | Their own share | Any personal share |
| Rename the community, invite or remove members | No | Yes |
| Deposit money into the community budget | No | Yes |

Community ownership does not make someone a dispatcher operator or give them
control over another member's runs. The creator remains the owner; ownership
transfer is not part of this interface.

## Save and reuse experiments

Choose **Save to community** in a workflow or experiment builder, select the
community project and save the editable source. **Include compiled program**
also saves its matching bundle. Members can then inspect the blocks, choose
their own executors and review a new measurement.
Optional collection labels group experiments within a project; descriptions
record the question or procedure being studied. Open an experiment to download
its source or bundle and see the measurements associated with its revisions.

A source-only experiment is editable but still needs compilation before it can
run. A saved compiled bundle can be reused without a compiler while its source
remains unchanged. Editing the source invalidates that compiled artifact; supply
a matching rebuilt bundle to run the edited version. The hosted console does
not compile source. See [workflow authoring](development/workflow-authoring.md)
for local compilation and bundle import.

**Save new revision** updates an existing shared experiment; **Save a copy**
creates another experiment. Each revision stays immutable. If two members edit the same revision,
the second save is refused as a conflict rather than overwriting the first.
Reload the latest revision, review the difference and save again. A measurement
reference can name the exact experiment revision used.

The experiment's creator or community owner can delete an unused saved experiment
to free the storage occupied by all its revisions. Deletion is refused while a
measurement reference names any revision, preserving the source linked to that
result. Deleting an experiment does not delete measurements or their output.

Bundles contain executable programs. The dispatcher checks source structure,
generated-source consistency and hashes, but those checks cannot prove that
arbitrary uploaded WASM was built from its claimed source. Review the publisher
and rebuild unfamiliar source before using it.

## Share measurements deliberately

Personal measurement history stays personal. Choose **Share with community**
on an existing measurement, or select a community project when submitting a new run.
The project collects references to those measurements; it does not create a
second copy of their output or change their original submitting account.

Sharing exposes the complete retained measurement to current community members:
program arguments, destination policy, output, timing, per-run cost and admission
provenance. That provenance can include private executor addresses and host
platform observations. Check what the measurement contains before sharing it.
Session credentials, payment authorization keys and private payment history are
not shared.

Members can inspect a shared group's participants, read output and export
results. They cannot cancel another person's run, delete its payload or reuse
its payment authorization. A personally funded measurement can be unshared by
its sharer or the community owner. Removing that reference removes its sharing
grant; another project reference can still grant access.

Community-funded measurements remain in the funding project's history and cannot
be unshared; attempting to remove their references returns 409. Expired
reservations that admitted no run are cleaned up automatically. Retention limits
and payload deletion continue to apply to the original measurement, so a
reference does not guarantee that output remains available forever.

## Pay from the community budget

The owner funds a community so members can submit under its umbrella without
asking the owner to sign each measurement. The **Funding** tab shows funded,
reserved, spent and available amounts. A member selects the project before
reviewing the measurement's quote, then chooses **Run with community funds**.

In a deployment with chain payments enabled, the owner creates a USDC top-up
and approves its payment in their external Sui wallet. Creating a top-up or
knowing a wallet address is not permission to spend from that wallet. Only a
confirmed payment credits the community budget, and the same receipt is credited
once.

The supported chain profile is Sui Testnet. This interface does not yet withdraw
unused deposits. Use test assets for this flow; it is not a production custody
service for real funds.

Deposited funds are held by the dispatcher's payment account. Debuglet does not
store the owner's wallet private key. Members spend the deposited budget; the
dispatcher reserves the full quoted amount atomically before returning an intent,
refuses insufficient funds, and settles or returns the reservation through the
normal run lifecycle. The submitting member remains recorded. This is a shared
budget without individual member spending caps, so invite only members allowed
to use it. See [chain payments](operations/payments.md) for network configuration
and the supported Testnet profile.

In a TEST deployment, the budget draws on the creator's existing TEST allowance.
TEST units are bookkeeping credits, not money. Creating a community grants no
credits, and TEST top-ups are unavailable. If allowances are disabled, community
funding is unavailable. The same allowance covers the creator's personal and
community spending, so separate communities are not separate TEST grants. A run
that executes consumes TEST allowance even if it fails, following the existing
allowance rules; unadmitted expired reservations are released. Personal
measurements keep their existing payment flow.

## Limits and deployment

| Resource | Limit |
| --- | --- |
| Communities owned by one account | 16 |
| Communities joined by one account | 128 |
| Members in one community | 128 |
| Outstanding invitations in one community | 8 |
| Projects in one community | 64 |
| Experiments in one project | 128 |
| Revisions of one experiment | 32 |
| Measurement references in one project | 1024 |
| Source and optional bundle in one save request | 24 MiB |
| Retained experiment revision bytes in one community | 256 MiB |

Workflow source retains its 1 MiB limit and coordinated experiment source its
4 MiB limit. Community and project names are trimmed and use 1–120 UTF-8 bytes;
descriptions use at most 2000 bytes.
Experiment descriptions use at most 4000 bytes and collection labels at most
120 bytes.

The collaboration API requires HTTP contract 1.23 and the matching console.
Upgrade the dispatcher database explicitly before starting the new dispatcher,
using the deployment's [upgrade and backup procedure](operations/configuration.md#stored-state).
Community records and funding records belong to dispatcher state; no executor
schema or control-protocol change is needed for collaboration itself.
Installing updated source or a console build does not publish a package or update
an existing deployment.

The [HTTP guide](api.md#communities-api-123) describes the routes and access
rules. The [OpenAPI document](../api/openapi.yaml) is the wire contract.
3 changes: 3 additions & 0 deletions docs/development/workflow-authoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ Choose **Workflow builder** to write one executor's program. Choose
workflow. The same graph can be edited visually, authored by an MCP client or
compiled from the command line with `dbl-studio`.

Use [communities](../communities.md) to save editable source and compiled bundles
in a shared project, retain revisions and collect the resulting measurements.

Conditions, calculations and repeats execute inside the compiled WASI program.
The dispatcher coordinates experiment readiness and stores results; it does not
compile source or orchestrate individual blocks. See the
Expand Down
Loading