diff --git a/docs/task-plugin-host.md b/docs/task-plugin-host.md new file mode 100644 index 00000000..df0871a9 --- /dev/null +++ b/docs/task-plugin-host.md @@ -0,0 +1,45 @@ +# Task Plugin 宿主基础 / Host foundation + +`pkg/taskplugin` 提供内部宿主基础,尚未注册公开管理或转发路由。`PackageDocument` 是 MAX-API 的数据包封装,不是 New API JavaScript 插件的直接安装格式;`source` 保持不透明,加载不会执行代码。本阶段不包含 JS VM、插件市场、管理界面或真实插件渠道接入。 + +`pkg/taskplugin` provides an internal host foundation. It does not register public management or relay routes. `PackageDocument` is a MAX-API data envelope, not a drop-in JavaScript plugin installation format. Source remains opaque and is never evaluated during loading. A JS VM, marketplace, management UI and live plugin channels are outside this stage. + +## 加载和任务身份 / Loading and task identity + +- `LoadPackage` 最多读取 1 MiB,校验整个文档的 SHA-256。期望摘要必须由宿主从可信渠道提供,不能仅根据上传文件自行计算后当作来源认证。 +- `Host.Install` 明确激活所选版本并批量发布插件和路由,包括操作员主动选择的版本回退;任一验证或路由冲突失败时整批不生效。同一 key/version 不允许替换不同内容。 +- `Host.Retain` 仅恢复历史包,不改变活动版本和路由;可以一次保留同一 key 的多个版本。补载历史任务所需的包时使用此接口,不使用 `Install`。摘要冲突、撤销和总容量限制仍适用;恢复流量前仍须恢复撤销记录。 +- 路由只匹配 manifest 声明的上游模型;省略 `route.Models` 表示继承 manifest 的范围,指定时进一步收窄。缺失或未知模型拒绝绑定。 +- `BindMapped` 固定插件版本、包摘要、原始及映射后模型,以及宿主拥有的 usage contract;`Resolve` 使用历史包恢复任务,不自动切换最新版本。 +- `AttachTaskPluginBinding` 校验用户、Token、渠道、请求、模型及冻结用量契约,写入 `Task.PrivateData`。调用方必须在发起付费请求前通过现有事务保存任务;该函数自身不执行数据库写入或上游提交。 +- 重启时必须重新加载所需历史包,并在开放流量前恢复宿主的撤销记录。缺失历史包、无效契约、身份变化或已撤销版本均拒绝恢复,不自动重提任务。当前没有自动保存插件目录或撤销清单的管理服务。 + +The loader accepts at most 1 MiB and verifies an externally trusted SHA-256 of the whole document. `Host.Install` explicitly activates the chosen versions, including operator-selected rollbacks, and publishes packages and routes together; a conflict leaves the previous generation intact. `Host.Retain` restores history without changing active versions or routes, and accepts several versions of one key in a batch. Use `Retain` rather than `Install` when loading historical task packages. Digest conflicts, revocations and the total retention limit still apply; revocations must be restored before traffic resumes. A key/version cannot silently change contents. Routes match only manifest-declared upstream models; an omitted `route.Models` inherits that set, while an explicit list narrows it. Missing or unknown models cannot bind. Bindings freeze package, usage contract, client model alias and upstream model. Recovery resolves historical packages rather than the latest version. + +The service validates task ownership and the frozen usage contract before attaching private metadata. The caller must persist it in the existing transaction before any billable request. The host must reload retained historical packages and revocations before serving traffic after restart. Missing or revoked identities fail closed and never cause automatic submission. Automatic catalog/revocation storage is not implemented here. + +## HTTP 能力 / HTTP capability + +`Session.NewFetchScope` 只提供 GET:HTTPS、主渠道 origin 与 manifest 额外 host 白名单、禁止环境代理与重定向、每次解析全部 IP 后直连已检查的 IP。回环、私有、链路本地及保留网段均拒绝。渠道 Authorization 只发往精确的渠道 origin,额外 host 不携带该凭据。插件不能提供任意请求头、请求体或 Cookie。 + +每个 hook 创建并关闭独立 scope:最多 4 次调用、每次最多 15 秒、scope 总期限 30 秒、响应体最多 1 MiB,响应头最多 64 KiB。网络错误不会包含原始 URL、查询参数或认证信息。此能力不支持内网/明文上游,也不用于提交付费任务;既有 Go adaptor 行为不变。 + +Each hook gets a separate, closed-after-use GET capability: HTTPS only, exact channel origin plus declared additional hosts, no environment proxy, redirects, arbitrary headers, body or cookies. All DNS answers are checked before dialing a validated literal IP. Private, loopback, link-local and reserved addresses are rejected. Channel authorization is attached only to the exact channel origin. + +Limits are four calls, 15 seconds per call, a 30-second scope deadline, 1 MiB response bodies and 64 KiB response headers. Network errors omit URLs, query strings and credentials. Private/plaintext upstreams and billable submission are not supported by this read-only capability. Existing Go adaptors are unchanged. + +## 计费与验证 / Billing and validation + +`ApplyTaskPluginUsage` 只附加经宿主契约验证的用量事实。显式零与缺失用量保持不同;不增加价格、额度扣减、退款或任务终态写入。结算继续使用冻结的计费计划、`task::finalize`、资金事务和独立日志/receipt effects。 + +`ApplyTaskPluginUsage` attaches validated facts only and preserves explicit zero versus missing usage. It does not price, debit, refund or publish task terminal state. The existing frozen plan, operation key, funding transaction and separate effects remain authoritative. + +```powershell +go test ./pkg/taskplugin ./pkg/taskusage -count=1 +go test ./service -run TestTaskPluginHost -count=1 +go test ./model -run TestTaskPluginHost -count=1 +``` + +这些合成测试不代表真实供应商、JS 隔离、多节点撤销传播或 MySQL/PostgreSQL 实库验收已通过。 + +Synthetic tests do not establish live-provider, JS isolation, multi-node revocation propagation or MySQL/PostgreSQL acceptance. diff --git a/model/task.go b/model/task.go index 5be04384..a23f5e8f 100644 --- a/model/task.go +++ b/model/task.go @@ -140,8 +140,9 @@ func (m Properties) Value() (driver.Value, error) { } type TaskPrivateData struct { - Key string `json:"key,omitempty"` - UpstreamTaskID string `json:"upstream_task_id,omitempty"` // 上游真实 task ID + PluginBinding *types.TaskPluginBinding `json:"plugin_binding,omitempty"` // host-issued immutable plugin identity; never client-visible + Key string `json:"key,omitempty"` + UpstreamTaskID string `json:"upstream_task_id,omitempty"` // 上游真实 task ID // AwaitingUpstreamID distinguishes a newly persisted placeholder from // legacy rows where TaskID itself is the provider task identifier. AwaitingUpstreamID bool `json:"awaiting_upstream_id,omitempty"` diff --git a/model/task_plugin_host_test.go b/model/task_plugin_host_test.go new file mode 100644 index 00000000..b9a8a8a9 --- /dev/null +++ b/model/task_plugin_host_test.go @@ -0,0 +1,100 @@ +/* +Copyright (C) 2023-2026 MAX-API-Next + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +For commercial licensing, please contact https://github.com/MAX-API-Next/MAX-API/issues +*/ + +package model + +import ( + "bytes" + "crypto/sha256" + "fmt" + "path/filepath" + "testing" + + "github.com/MAX-API-Next/MAX-API/common" + "github.com/MAX-API-Next/MAX-API/pkg/taskplugin" + "github.com/MAX-API-Next/MAX-API/pkg/taskusage" + "github.com/MAX-API-Next/MAX-API/types" + "github.com/glebarez/sqlite" + "github.com/stretchr/testify/require" + "gorm.io/gorm" +) + +func TestTaskPluginHostBindingSurvivesSQLiteFundingAndEffectReplay(t *testing.T) { + db, err := gorm.Open(sqlite.Open(filepath.Join(t.TempDir(), "plugin.db")), taskBillingTestGORMConfig()) + require.NoError(t, err) + sqlDB, err := db.DB() + require.NoError(t, err) + t.Cleanup(func() { require.NoError(t, sqlDB.Close()) }) + prepareTaskBillingContractDB(t, db, "sqlite") + f := newTaskBillingContractFixture(t, db, 9992, BillingSettlementSourceWallet, -20) + manifest := taskplugin.Manifest{APIVersion: 1, Key: "ledger-test", Name: "Ledger test", Version: "1.0.0", Models: []string{"contract-model"}, Routes: []taskplugin.Route{{Method: "GET", Path: "/plugin/:id", Type: "query", Render: "render"}}} + data, err := common.Marshal(taskplugin.PackageDocument{FormatVersion: 1, Manifest: manifest, Source: "original fixture, never executed"}) + require.NoError(t, err) + hash := sha256.Sum256(data) + p, err := taskplugin.LoadPackage(bytes.NewReader(data), fmt.Sprintf("%x", hash)) + require.NoError(t, err) + host := taskplugin.NewHost() + _, err = host.Install(p) + require.NoError(t, err) + actor := taskplugin.Actor{UserID: f.user.Id, TokenID: f.token.Id, ChannelID: f.channel.Id, RequestID: "plugin-submission"} + binding, err := host.Bind(actor, "GET", "/plugin/one", "contract-model", taskusage.MiniMaxH3Contract()) + require.NoError(t, err) + f.task.PrivateData.PluginBinding = &binding + f.task.PrivateData.TokenId = f.token.Id + f.task.PrivateData.BillingRequestId = actor.RequestID + require.NoError(t, db.Save(&f.task).Error) + var stored Task + require.NoError(t, db.First(&stored, f.task.ID).Error) + require.NotNil(t, stored.PrivateData.PluginBinding) + restarted := taskplugin.NewHost() + currentManifest := manifest + currentManifest.Version = "2.0.0" + currentData, err := common.Marshal(taskplugin.PackageDocument{FormatVersion: 1, Manifest: currentManifest, Source: "current fixture, never executed"}) + require.NoError(t, err) + currentHash := sha256.Sum256(currentData) + currentPackage, err := taskplugin.LoadPackage(bytes.NewReader(currentData), fmt.Sprintf("%x", currentHash)) + require.NoError(t, err) + _, err = restarted.Install(currentPackage) + require.NoError(t, err) + _, err = restarted.Retain(p) + require.NoError(t, err) + session, err := restarted.Resolve(*stored.PrivateData.PluginBinding, actor) + require.NoError(t, err) + require.Equal(t, "1.0.0", session.Binding().Version) + require.Equal(t, p.Digest(), session.Binding().PackageDigest) + newBinding, err := restarted.Bind(actor, "GET", "/plugin/one", "contract-model", taskusage.MiniMaxH3Contract()) + require.NoError(t, err) + require.Equal(t, "2.0.0", newBinding.Version, "restoring a persisted task must not switch the active version") + duration, images := int64(5_000), int64(0) + envelope, err := session.UsageEnvelope(&types.TaskUsage{OutputDurationMs: &duration, InputImageCount: &images, Source: types.TaskUsageSourceProviderResponse, Completeness: types.TaskUsageCompletenessComplete}) + require.NoError(t, err) + stored.PrivateData.BillingContext.TaskUsageEnvelope = envelope + require.NoError(t, db.Save(&stored).Error) + for attempt := 0; attempt < 2; attempt++ { + applied, replay, err := ApplyBillingSettlementOnce(f.input) + require.NoError(t, err) + require.EqualValues(t, -20, applied) + require.Equal(t, attempt > 0, replay) + require.NoError(t, ProcessBillingSettlementEffect(f.input.OperationKey)) + } + f.assertProjection(t, db, 80, 1) + require.NoError(t, db.First(&stored, f.task.ID).Error) + require.Equal(t, binding, *stored.PrivateData.PluginBinding) + require.Equal(t, envelope.EvidenceDigest, stored.PrivateData.BillingContext.TaskUsageEnvelope.EvidenceDigest) +} diff --git a/pkg/taskplugin/fetch.go b/pkg/taskplugin/fetch.go new file mode 100644 index 00000000..84e8b2ac --- /dev/null +++ b/pkg/taskplugin/fetch.go @@ -0,0 +1,219 @@ +/* +Copyright (C) 2023-2026 MAX-API-Next + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +For commercial licensing, please contact https://github.com/MAX-API-Next/MAX-API/issues +*/ + +package taskplugin + +import ( + "context" + "crypto/tls" + "errors" + "io" + "net" + "net/http" + "net/url" + "strconv" + "strings" + "sync/atomic" + "time" + + "github.com/MAX-API-Next/MAX-API/common" +) + +const ( + MaxFetchBytes = 1 << 20 + MaxFetchCalls = 4 +) + +var ErrFetchRejected = errors.New("task plugin fetch rejected or failed") + +// FetchScope is a read-only HTTP capability for one hook invocation. It has no +// cookie jar, proxy, redirects, body, caller headers, pricing or write access. +type FetchScope struct { + session *Session + ctx context.Context + cancel context.CancelFunc + client *http.Client + baseOrigin string + allowed map[string]bool + authorization string + calls atomic.Uint32 +} + +type FetchResult struct { + StatusCode int + Body []byte +} + +// NewFetchScope takes credentials from the host channel configuration. They are +// sent only to the exact HTTPS channel origin, never to extra allowed hosts. +// A new scope must be closed by the host after each hook, even on hook failure. +func (session *Session) NewFetchScope(ctx context.Context, channelBaseURL, authorization string) (*FetchScope, error) { + if err := session.validate(); err != nil { + return nil, err + } + if ctx == nil || validateHTTPBaseURL(channelBaseURL) != nil || strings.ContainsAny(authorization, "\r\n") { + return nil, ErrFetchRejected + } + base, err := url.Parse(channelBaseURL) + if err != nil || base.Scheme != "https" { + return nil, ErrFetchRejected + } + bounded, cancel := context.WithTimeout(ctx, 30*time.Second) + scope := &FetchScope{session: session, ctx: bounded, cancel: cancel, allowed: map[string]bool{}, authorization: authorization, baseOrigin: httpsOrigin(base)} + if scope.baseOrigin == "" { + cancel() + return nil, ErrFetchRejected + } + scope.allowed[scope.baseOrigin] = true + for _, host := range session.plugin.manifest.AllowedHosts { + parsed, err := url.Parse("https://" + host) + if err != nil || httpsOrigin(parsed) == "" { + cancel() + return nil, ErrFetchRejected + } + scope.allowed[httpsOrigin(parsed)] = true + } + transport := &http.Transport{ + Proxy: nil, + DialContext: strictPluginDial, + TLSClientConfig: &tls.Config{MinVersion: tls.VersionTLS12}, + TLSHandshakeTimeout: 5 * time.Second, + ResponseHeaderTimeout: 10 * time.Second, + MaxResponseHeaderBytes: 64 << 10, + DisableKeepAlives: true, + } + scope.client = &http.Client{Transport: transport, Timeout: 15 * time.Second, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }} + return scope, nil +} + +func (scope *FetchScope) Close() { + if scope != nil { + scope.cancel() + scope.client.CloseIdleConnections() + } +} + +// Fetch never follows a redirect and never returns network errors containing a +// signed URL, query, header or credential. The original context error is safe. +func (scope *FetchScope) Fetch(rawURL string) (FetchResult, error) { + if scope == nil || len(rawURL) > 8192 { + return FetchResult{}, ErrFetchRejected + } + if err := scope.session.validate(); err != nil { + return FetchResult{}, err + } + if err := scope.ctx.Err(); err != nil { + return FetchResult{}, err + } + if scope.calls.Add(1) > MaxFetchCalls { + return FetchResult{}, ErrFetchRejected + } + u, err := url.Parse(rawURL) + if err != nil || u.User != nil || u.Fragment != "" || u.Opaque != "" || !isASCII(u.Host) || strings.ContainsAny(rawURL, "\\\r\n") { + return FetchResult{}, ErrFetchRejected + } + origin := httpsOrigin(u) + if !scope.allowed[origin] || origin == "" { + return FetchResult{}, ErrFetchRejected + } + request, err := http.NewRequestWithContext(scope.ctx, http.MethodGet, u.String(), nil) + if err != nil { + return FetchResult{}, ErrFetchRejected + } + request.Header.Set("Accept", "application/json") + if origin == scope.baseOrigin && scope.authorization != "" { + request.Header.Set("Authorization", scope.authorization) + } + response, err := scope.client.Do(request) + if err != nil { + if ctxErr := scope.ctx.Err(); ctxErr != nil { + return FetchResult{}, ctxErr + } + return FetchResult{}, ErrFetchRejected + } + defer func() { _ = response.Body.Close() }() + if response.ContentLength > MaxFetchBytes { + return FetchResult{}, ErrFetchRejected + } + body, err := io.ReadAll(io.LimitReader(response.Body, MaxFetchBytes+1)) + if err != nil || len(body) > MaxFetchBytes { + return FetchResult{}, ErrFetchRejected + } + if err := scope.session.validate(); err != nil { + return FetchResult{}, err + } + return FetchResult{StatusCode: response.StatusCode, Body: body}, nil +} + +func httpsOrigin(u *url.URL) string { + if u == nil || u.Scheme != "https" || u.Hostname() == "" || u.User != nil || !isASCII(u.Host) { + return "" + } + port := u.Port() + if port == "" { + port = "443" + } + number, err := strconv.Atoi(port) + if err != nil || number <= 0 || number > 65535 { + return "" + } + return "https://" + net.JoinHostPort(strings.ToLower(u.Hostname()), strconv.Itoa(number)) +} + +// Resolve and validate every address, then dial the chosen literal IP. This +// avoids a second DNS lookup between the SSRF check and the actual connection. +func strictPluginDial(ctx context.Context, network, address string) (net.Conn, error) { + return pluginDial(ctx, network, address, net.DefaultResolver.LookupIPAddr, (&net.Dialer{Timeout: 5 * time.Second}).DialContext) +} + +func pluginDial(ctx context.Context, network, address string, lookup func(context.Context, string) ([]net.IPAddr, error), dial func(context.Context, string, string) (net.Conn, error)) (net.Conn, error) { + host, portText, err := net.SplitHostPort(address) + if err != nil { + return nil, ErrFetchRejected + } + port, err := strconv.Atoi(portText) + if err != nil { + return nil, ErrFetchRejected + } + protection := &common.SSRFProtection{AllowPrivateIp: false, DomainFilterMode: false, IpFilterMode: false, AllowedPorts: []int{port}, ApplyIPFilterForDomain: true} + if err := protection.ValidateNetworkTarget(host, port); err != nil { + return nil, ErrFetchRejected + } + var addresses []net.IPAddr + if ip := net.ParseIP(host); ip != nil { + addresses = []net.IPAddr{{IP: ip}} + } else { + addresses, err = lookup(ctx, host) + if err != nil || len(addresses) == 0 { + return nil, ErrFetchRejected + } + } + for _, address := range addresses { + if address.Zone != "" || protection.ValidateResolvedIP(host, address.IP) != nil { + return nil, ErrFetchRejected + } + } + for _, address := range addresses { + connection, err := dial(ctx, network, net.JoinHostPort(address.IP.String(), portText)) + if err == nil { + return connection, nil + } + } + return nil, ErrFetchRejected +} diff --git a/pkg/taskplugin/fetch_test.go b/pkg/taskplugin/fetch_test.go new file mode 100644 index 00000000..f76515b5 --- /dev/null +++ b/pkg/taskplugin/fetch_test.go @@ -0,0 +1,152 @@ +/* +Copyright (C) 2023-2026 MAX-API-Next + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +For commercial licensing, please contact https://github.com/MAX-API-Next/MAX-API/issues +*/ + +package taskplugin + +import ( + "context" + "io" + "net" + "net/http" + "strings" + "testing" + + "github.com/MAX-API-Next/MAX-API/pkg/taskusage" + "github.com/stretchr/testify/require" +) + +type pluginRoundTrip func(*http.Request) (*http.Response, error) + +func (f pluginRoundTrip) RoundTrip(request *http.Request) (*http.Response, error) { return f(request) } + +func testFetchScope(t *testing.T) *FetchScope { + t.Helper() + host := NewHost() + manifest := validManifest() + manifest.AllowedHosts = []string{"assets.vendor.example:443"} + _, err := host.Install(testPackage(t, manifest, "original")) + require.NoError(t, err) + actor := Actor{UserID: 1, TokenID: 2, ChannelID: 3, RequestID: "generation-one"} + binding, err := host.Bind(actor, "POST", "/vendor/v1/jobs", "vendor-model", taskusage.DoubaoVideoContract()) + require.NoError(t, err) + session, err := host.Resolve(binding, actor) + require.NoError(t, err) + scope, err := session.NewFetchScope(context.Background(), "https://channel.vendor.example", "Bearer synthetic-only") + require.NoError(t, err) + t.Cleanup(scope.Close) + transport := scope.client.Transport.(*http.Transport) + require.Nil(t, transport.Proxy) + require.NotNil(t, transport.DialContext) + require.False(t, transport.TLSClientConfig.InsecureSkipVerify) + require.Nil(t, scope.client.Jar) + return scope +} + +func TestPluginFetchSeparatesCredentialsAndDoesNotFollowRedirects(t *testing.T) { + scope := testFetchScope(t) + calls := 0 + scope.client.Transport = pluginRoundTrip(func(request *http.Request) (*http.Response, error) { + calls++ + require.Equal(t, "GET", request.Method) + if request.URL.Host == "channel.vendor.example" { + require.Equal(t, "Bearer synthetic-only", request.Header.Get("Authorization")) + } else { + require.Empty(t, request.Header.Get("Authorization")) + } + return &http.Response{StatusCode: 302, Header: http.Header{"Location": []string{"https://blocked.invalid/token"}}, Body: io.NopCloser(strings.NewReader("redirect"))}, nil + }) + for _, target := range []string{"https://channel.vendor.example/status", "https://assets.vendor.example/status"} { + result, err := scope.Fetch(target) + require.NoError(t, err) + require.Equal(t, 302, result.StatusCode) + } + require.Equal(t, 2, calls) + for _, target := range []string{"https://blocked.invalid", "http://channel.vendor.example/status"} { + _, err := scope.Fetch(target) + require.ErrorIs(t, err, ErrFetchRejected) + } + require.Equal(t, 2, calls) +} + +func TestPluginFetchBudgetsRevocationAndRedactedErrors(t *testing.T) { + t.Run("body", func(t *testing.T) { + scope := testFetchScope(t) + scope.client.Transport = pluginRoundTrip(func(*http.Request) (*http.Response, error) { + return &http.Response{StatusCode: 200, Header: http.Header{}, ContentLength: -1, Body: io.NopCloser(strings.NewReader(strings.Repeat("x", MaxFetchBytes+1)))}, nil + }) + _, err := scope.Fetch("https://channel.vendor.example/status?token=synthetic-only") + require.ErrorIs(t, err, ErrFetchRejected) + require.NotContains(t, err.Error(), "synthetic-only") + }) + t.Run("calls", func(t *testing.T) { + scope := testFetchScope(t) + calls := 0 + scope.client.Transport = pluginRoundTrip(func(*http.Request) (*http.Response, error) { + calls++ + return &http.Response{StatusCode: 200, Header: http.Header{}, Body: io.NopCloser(strings.NewReader("ok"))}, nil + }) + for i := 0; i < MaxFetchCalls; i++ { + _, err := scope.Fetch("https://channel.vendor.example/status") + require.NoError(t, err) + } + _, err := scope.Fetch("https://channel.vendor.example/status") + require.ErrorIs(t, err, ErrFetchRejected) + require.Equal(t, MaxFetchCalls, calls, "exhausted scope must reject before transport") + }) + t.Run("revoked", func(t *testing.T) { + scope := testFetchScope(t) + require.NoError(t, scope.session.host.Revoke("vendor", "1.0.0")) + _, err := scope.Fetch("https://channel.vendor.example/status") + require.ErrorIs(t, err, ErrBindingRejected) + }) + t.Run("canceled", func(t *testing.T) { + scope := testFetchScope(t) + scope.Close() + _, err := scope.Fetch("https://channel.vendor.example/status") + require.ErrorIs(t, err, context.Canceled) + }) +} + +func TestPluginDialRejectsPrivateAndRebindingTargetsBeforeDial(t *testing.T) { + for _, ips := range [][]string{{"127.0.0.1"}, {"169.254.169.254"}, {"::1"}, {"::ffff:127.0.0.1"}, {"8.8.8.8", "10.0.0.1"}} { + called := false + _, err := pluginDial(context.Background(), "tcp", "allowed.example:443", func(context.Context, string) ([]net.IPAddr, error) { + var addresses []net.IPAddr + for _, ip := range ips { + addresses = append(addresses, net.IPAddr{IP: net.ParseIP(ip)}) + } + return addresses, nil + }, func(context.Context, string, string) (net.Conn, error) { called = true; return nil, nil }) + require.ErrorIs(t, err, ErrFetchRejected) + require.False(t, called) + } + var actual string + left, right := net.Pipe() + defer left.Close() + defer right.Close() + connection, err := pluginDial(context.Background(), "tcp", "allowed.example:443", func(context.Context, string) ([]net.IPAddr, error) { + return []net.IPAddr{{IP: net.ParseIP("8.8.8.8")}}, nil + }, func(_ context.Context, _ string, address string) (net.Conn, error) { + actual = address + return left, nil + }) + require.NoError(t, err) + require.Same(t, left, connection) + require.Equal(t, "8.8.8.8:443", actual) +} diff --git a/pkg/taskplugin/history_test.go b/pkg/taskplugin/history_test.go new file mode 100644 index 00000000..5f27ee1f --- /dev/null +++ b/pkg/taskplugin/history_test.go @@ -0,0 +1,170 @@ +/* +Copyright (C) 2023-2026 MAX-API-Next + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +For commercial licensing, please contact https://github.com/MAX-API-Next/MAX-API/issues +*/ + +package taskplugin + +import ( + "fmt" + "github.com/MAX-API-Next/MAX-API/pkg/taskusage" + "github.com/stretchr/testify/require" + "sync" + "testing" +) + +func TestHostRetainRestoresHistoryWithoutChangingActive(t *testing.T) { + actor := Actor{UserID: 1, TokenID: 2, ChannelID: 3, RequestID: "retained-task"} + previous := testPackage(t, validManifest(), "original-v1") + original := NewHost() + _, err := original.Install(previous) + require.NoError(t, err) + binding, err := original.Bind(actor, "POST", "/vendor/v1/jobs", "vendor-model", taskusage.MiniMaxH3Contract()) + require.NoError(t, err) + + latest := validManifest() + latest.Version = "2.0.0" + current := testPackage(t, latest, "active-v2") + restarted := NewHost() + _, err = restarted.Install(current) + require.NoError(t, err) + before := restarted.state.Load() + _, err = restarted.Retain(previous) + require.NoError(t, err) + restored, err := restarted.Resolve(binding, actor) + require.NoError(t, err) + require.Equal(t, previous.Digest(), restored.Binding().PackageDigest) + next, err := restarted.Bind(actor, "POST", "/vendor/v1/jobs", "vendor-model", taskusage.MiniMaxH3Contract()) + require.NoError(t, err) + require.Equal(t, "2.0.0", next.Version, "restoring a historical task must preserve the chosen active version") + require.Equal(t, before.routes, restarted.state.Load().routes) +} + +func TestHostRetainMultipleVersionsDoesNotActivateRoutes(t *testing.T) { + host := NewHost() + first := testPackage(t, validManifest(), "first") + secondManifest := validManifest() + secondManifest.Version = "2.0.0" + second := testPackage(t, secondManifest, "second") + generation, err := host.Retain(second, first) + require.NoError(t, err) + require.Equal(t, generation, host.state.Load().generation) + require.Len(t, host.state.Load().history, 2) + require.Empty(t, host.state.Load().active) + actor := Actor{UserID: 1, TokenID: 2, ChannelID: 3, RequestID: "history-only"} + _, err = host.Bind(actor, "POST", "/vendor/v1/jobs", "vendor-model", taskusage.MiniMaxH3Contract()) + require.ErrorIs(t, err, ErrBindingRejected) + + _, err = host.Install(second) + require.NoError(t, err) + binding, err := host.Bind(actor, "POST", "/vendor/v1/jobs", "vendor-model", taskusage.MiniMaxH3Contract()) + require.NoError(t, err) + require.Equal(t, "2.0.0", binding.Version) + // Install is an explicit activation decision, including a chosen rollback. + _, err = host.Install(first) + require.NoError(t, err) + binding, err = host.Bind(actor, "POST", "/vendor/v1/jobs", "vendor-model", taskusage.MiniMaxH3Contract()) + require.NoError(t, err) + require.Equal(t, "1.0.0", binding.Version) +} + +func TestHostRetainInvalidBatchLeavesGenerationUntouched(t *testing.T) { + for _, invalid := range []string{"empty", "nil", "unverified", "duplicate", "digest conflict", "revoked", "capacity"} { + t.Run(invalid, func(t *testing.T) { + host := NewHost() + original := testPackage(t, validManifest(), "original") + _, err := host.Install(original) + require.NoError(t, err) + additional := testPackage(t, registryManifest("historical", "/historical/:id"), "historical") + packages := []*Package{additional} + switch invalid { + case "empty": + packages = nil + case "nil": + packages = append(packages, nil) + case "unverified": + packages = append(packages, &Package{}) + case "duplicate": + packages = append(packages, additional) + case "digest conflict": + packages = append(packages, testPackage(t, validManifest(), "different bytes")) + case "revoked": + require.NoError(t, host.Revoke(original.manifest.Key, original.manifest.Version)) + packages = append(packages, original) + case "capacity": + // Exercise the shared accounting limit without allocating a + // 64 MiB fixture; LoadPackage's 1 MiB cap is tested separately. + additional.size = MaxRetainedPackageBytes + } + before := host.state.Load() + _, err = host.Retain(packages...) + require.Error(t, err) + require.Same(t, before, host.state.Load(), "failed batches must publish nothing") + require.NotContains(t, host.state.Load().history, packageIdentity(additional.manifest.Key, additional.manifest.Version)) + }) + } + for _, host := range []*Host{nil, {}} { + _, err := host.Retain() + require.Error(t, err) + } +} + +func TestHostConcurrentRetainPreservesActivationAndRevocation(t *testing.T) { + host := NewHost() + activeManifest := validManifest() + activeManifest.Version = "3.0.0" + _, err := host.Install(testPackage(t, activeManifest, "active")) + require.NoError(t, err) + previous := testPackage(t, validManifest(), "previous") + _, err = host.Retain(previous) + require.NoError(t, err) + + const count = 8 + packages := make([]*Package, count) + for i := range packages { + manifest := validManifest() + manifest.Version = fmt.Sprintf("1.0.%d", i+1) + packages[i] = testPackage(t, manifest, "historical") + } + results := make(chan error, count+2) + var workers sync.WaitGroup + for _, p := range packages { + workers.Add(1) + go func() { + defer workers.Done() + _, err := host.Retain(p) + results <- err + }() + } + unrelated := testPackage(t, registryManifest("unrelated", "/unrelated/:id"), "unrelated") + workers.Add(2) + go func() { defer workers.Done(); _, err := host.Install(unrelated); results <- err }() + go func() { defer workers.Done(); results <- host.Revoke(previous.manifest.Key, previous.manifest.Version) }() + workers.Wait() + close(results) + for err := range results { + require.NoError(t, err) + } + require.Len(t, host.state.Load().history, count+3) + require.Len(t, host.state.Load().active, 2) + require.Equal(t, "3.0.0", host.state.Load().active[activeManifest.Key].manifest.Version) + for _, p := range packages { + require.Equal(t, p.Digest(), host.state.Load().history[packageIdentity(p.manifest.Key, p.manifest.Version)].Digest()) + } + _, err = host.Retain(previous) + require.ErrorContains(t, err, "revoked") +} diff --git a/pkg/taskplugin/host.go b/pkg/taskplugin/host.go new file mode 100644 index 00000000..8a1b9706 --- /dev/null +++ b/pkg/taskplugin/host.go @@ -0,0 +1,255 @@ +/* +Copyright (C) 2023-2026 MAX-API-Next + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +For commercial licensing, please contact https://github.com/MAX-API-Next/MAX-API/issues +*/ + +package taskplugin + +import ( + "errors" + "sort" + "strings" + "sync/atomic" + + "github.com/MAX-API-Next/MAX-API/pkg/taskusage" + "github.com/MAX-API-Next/MAX-API/types" +) + +var ErrBindingRejected = errors.New("task plugin binding unavailable or unauthorized") + +// Actor comes from the authenticated host context, not plugin/client metadata. +type Actor struct { + UserID, TokenID, ChannelID int + RequestID string +} + +type hostState struct { + generation uint64 + active map[string]*Package + history map[string]*Package + revoked map[string]bool + routes Snapshot +} + +// Host atomically publishes a complete route/package generation. Historical +// packages are retained so an upgrade cannot reinterpret an in-flight task. +type Host struct{ state atomic.Pointer[hostState] } + +const MaxRetainedPackageBytes = 64 << 20 + +func NewHost() *Host { + host := &Host{} + host.state.Store(&hostState{active: map[string]*Package{}, history: map[string]*Package{}, revoked: map[string]bool{}, routes: NewRegistry().Snapshot()}) + return host +} + +func packageIdentity(key, version string) string { return key + "@" + version } + +// Install explicitly activates a batch all-or-nothing, including operator-chosen +// rollbacks. Use Retain to restore history without changing the active version. +func (host *Host) Install(packages ...*Package) (uint64, error) { + return host.publishPackages(packages, true) +} + +// Retain restores immutable historical packages without activating their routes +// or changing the active version. Several versions of one key may be retained +// together. Digest conflicts, revocations and the shared retention limit apply. +func (host *Host) Retain(packages ...*Package) (uint64, error) { + return host.publishPackages(packages, false) +} + +func (host *Host) publishPackages(packages []*Package, activate bool) (uint64, error) { + if host == nil || host.state.Load() == nil || len(packages) == 0 { + return 0, errors.New("task plugin host or package batch is empty") + } + seen := map[string]bool{} + for _, p := range packages { + if p == nil || p.digest == "" { + return 0, errors.New("invalid or duplicate task plugin package") + } + identity := packageIdentity(p.manifest.Key, p.manifest.Version) + if activate { + identity = p.manifest.Key + } + if seen[identity] { + return 0, errors.New("invalid or duplicate task plugin package") + } + seen[identity] = true + } + for { + current := host.state.Load() + next := cloneHostState(current) + for _, p := range packages { + id := packageIdentity(p.manifest.Key, p.manifest.Version) + if old := current.history[id]; old != nil && old.digest != p.digest { + return 0, errors.New("task plugin key/version source changed") + } + if current.revoked[id] { + return 0, errors.New("task plugin package is revoked") + } + next.history[id] = p + if activate { + next.active[p.manifest.Key] = p + } + } + total := 0 + for _, p := range next.history { + total += p.size + } + if total > MaxRetainedPackageBytes { + return 0, errors.New("task plugin retained package limit reached; historical packages were preserved") + } + if activate { + registry := NewRegistry() + keys := make([]string, 0, len(next.active)) + for key := range next.active { + keys = append(keys, key) + } + sort.Strings(keys) + for _, key := range keys { + if _, err := registry.Publish(next.active[key].manifest); err != nil { + return 0, errors.New("task plugin batch has conflicting routes") + } + } + next.routes = registry.Snapshot() + } + if host.state.CompareAndSwap(current, next) { + return next.generation, nil + } + } +} + +// Revoke disables both new bindings and old sessions for one immutable version. +// A process restart must reapply host-owned revocations before serving traffic. +func (host *Host) Revoke(key, version string) error { + if host == nil || host.state.Load() == nil { + return ErrBindingRejected + } + id := packageIdentity(key, version) + for { + current := host.state.Load() + if current.history[id] == nil { + return ErrBindingRejected + } + if current.revoked[id] { + return nil + } + next := cloneHostState(current) + next.revoked[id] = true + if host.state.CompareAndSwap(current, next) { + return nil + } + } +} + +func cloneHostState(current *hostState) *hostState { + next := &hostState{generation: current.generation + 1, active: map[string]*Package{}, history: map[string]*Package{}, revoked: map[string]bool{}, routes: current.routes} + for k, v := range current.active { + next.active[k] = v + } + for k, v := range current.history { + next.history[k] = v + } + for k, v := range current.revoked { + next.revoked[k] = v + } + return next +} + +// Bind selects a route and freezes its package and host-owned usage contract. +func (host *Host) Bind(actor Actor, method, path, model string, contract types.TaskUsageContract) (types.TaskPluginBinding, error) { + return host.BindMapped(actor, method, path, model, model, contract) +} + +// BindMapped preserves the client's model alias while resolving the declared +// upstream model. Neither identity may be replaced during later polling. +func (host *Host) BindMapped(actor Actor, method, path, model, upstreamModel string, contract types.TaskUsageContract) (types.TaskPluginBinding, error) { + if host == nil || host.state.Load() == nil || !validActor(actor) { + return types.TaskPluginBinding{}, ErrBindingRejected + } + state := host.state.Load() + match, ok := state.routes.MatchRoute(method, path, upstreamModel) + if !ok { + return types.TaskPluginBinding{}, ErrBindingRejected + } + p := state.active[match.PluginKey] + if state.revoked[packageIdentity(p.manifest.Key, p.manifest.Version)] || model == "" || model != strings.TrimSpace(model) || !declaresModel(p.manifest, upstreamModel) { + return types.TaskPluginBinding{}, ErrBindingRejected + } + digest, err := taskusage.ContractDigest(contract) + if err != nil { + return types.TaskPluginBinding{}, ErrBindingRejected + } + if _, err := taskusage.ResolveContract(contract.SourceID, contract.SchemaVersion, digest); err != nil { + return types.TaskPluginBinding{}, ErrBindingRejected + } + return types.TaskPluginBinding{PluginKey: p.manifest.Key, Version: p.manifest.Version, PackageDigest: p.digest, SourceID: contract.SourceID, SchemaVersion: contract.SchemaVersion, ContractDigest: digest, Model: model, UpstreamModel: upstreamModel, UserID: actor.UserID, TokenID: actor.TokenID, ChannelID: actor.ChannelID, RequestID: actor.RequestID}, nil +} + +// Resolve uses persisted identity, never the currently active version. Missing +// history or revoked code fails closed; it never retries a provider submission. +func (host *Host) Resolve(binding types.TaskPluginBinding, actor Actor) (*Session, error) { + if host == nil || host.state.Load() == nil || !validActor(actor) || binding.UserID != actor.UserID || binding.TokenID != actor.TokenID || binding.ChannelID != actor.ChannelID || binding.RequestID != actor.RequestID { + return nil, ErrBindingRejected + } + state := host.state.Load() + id := packageIdentity(binding.PluginKey, binding.Version) + p := state.history[id] + if p == nil || state.revoked[id] || binding.PackageDigest != p.digest || binding.Model == "" || binding.Model != strings.TrimSpace(binding.Model) || !declaresModel(p.manifest, binding.UpstreamModel) { + return nil, ErrBindingRejected + } + contract, err := taskusage.ResolveContract(binding.SourceID, binding.SchemaVersion, binding.ContractDigest) + if err != nil { + return nil, ErrBindingRejected + } + return &Session{host: host, binding: binding, actor: actor, contract: contract, plugin: p}, nil +} + +func validActor(actor Actor) bool { + return actor.UserID > 0 && actor.TokenID > 0 && actor.ChannelID > 0 && actor.RequestID != "" && actor.RequestID == strings.TrimSpace(actor.RequestID) +} + +// Session holds private, immutable host context; it is not a JavaScript VM. +type Session struct { + host *Host + binding types.TaskPluginBinding + actor Actor + contract types.TaskUsageContract + plugin *Package +} + +func (session *Session) validate() error { + if session == nil { + return ErrBindingRejected + } + _, err := session.host.Resolve(session.binding, session.actor) + return err +} + +func (session *Session) Binding() types.TaskPluginBinding { return session.binding } +func (session *Session) UsageContract() types.TaskUsageContract { + return cloneUsageContract(session.contract) +} + +// UsageEnvelope accepts facts only. Funding and task state stay in MAX-API's +// existing settlement and polling layers, including explicit-zero semantics. +func (session *Session) UsageEnvelope(usage *types.TaskUsage) (*types.TaskUsageEnvelope, error) { + if err := session.validate(); err != nil { + return nil, err + } + return taskusage.BuildEnvelope(types.TaskUsageProducerKindTaskPlugin, session.contract, types.TaskUsageSourceProviderResponse, usage) +} diff --git a/pkg/taskplugin/host_test.go b/pkg/taskplugin/host_test.go new file mode 100644 index 00000000..e8909831 --- /dev/null +++ b/pkg/taskplugin/host_test.go @@ -0,0 +1,186 @@ +/* +Copyright (C) 2023-2026 MAX-API-Next + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +For commercial licensing, please contact https://github.com/MAX-API-Next/MAX-API/issues +*/ + +package taskplugin + +import ( + "bytes" + "crypto/sha256" + "fmt" + "strings" + "sync" + "testing" + + "github.com/MAX-API-Next/MAX-API/common" + "github.com/MAX-API-Next/MAX-API/pkg/taskusage" + "github.com/MAX-API-Next/MAX-API/types" + "github.com/stretchr/testify/require" +) + +func testPackage(t *testing.T, manifest Manifest, source string) *Package { + t.Helper() + data, err := common.Marshal(PackageDocument{FormatVersion: 1, Manifest: manifest, Source: source}) + require.NoError(t, err) + hash := sha256.Sum256(data) + p, err := LoadPackage(bytes.NewReader(data), fmt.Sprintf("%x", hash)) + require.NoError(t, err) + return p +} + +func TestLoadPackagePinsWholeDocumentAndClonesMetadata(t *testing.T) { + manifest := validManifest() + p := testPackage(t, manifest, "opaque source - never evaluated") + require.Equal(t, manifest.UsageSchema, p.Manifest().UsageSchema) + copy := p.Manifest() + copy.Models[0] = "changed" + copy.UsageSchema["resolution"] = UsageField{} + require.Equal(t, manifest, p.Manifest()) + _, err := LoadPackage(strings.NewReader("{}"), strings.Repeat("0", 64)) + require.ErrorContains(t, err, "digest mismatch") + _, err = LoadPackage(strings.NewReader(strings.Repeat("x", MaxPackageBytes+1)), strings.Repeat("0", 64)) + require.ErrorContains(t, err, "size limit") + _, err = LoadPackage(strings.NewReader("{}"), "") + require.Error(t, err) +} + +func TestHostBatchAtomicityAndVersionPinning(t *testing.T) { + host := NewHost() + actor := Actor{UserID: 1, TokenID: 2, ChannelID: 3, RequestID: "request-one"} + first := testPackage(t, validManifest(), "first") + _, err := host.Install(first) + require.NoError(t, err) + binding, err := host.Bind(actor, "POST", "/vendor/v1/jobs", "vendor-model", taskusage.MiniMaxH3Contract()) + require.NoError(t, err) + manifest := validManifest() + manifest.Version = "2.0.0" + _, err = host.Install(testPackage(t, manifest, "second")) + require.NoError(t, err) + session, err := host.Resolve(binding, actor) + require.NoError(t, err) + require.Equal(t, first.Digest(), session.Binding().PackageDigest) + _, err = host.Install(testPackage(t, validManifest(), "silently changed")) + require.ErrorContains(t, err, "source changed") + conflict := registryManifest("conflict", "/vendor/v1/jobs/:id") + newPlugin := registryManifest("new", "/new/:id") + before := host.state.Load().generation + _, err = host.Install(testPackage(t, newPlugin, "new"), testPackage(t, conflict, "conflict")) + require.Error(t, err) + require.Equal(t, before, host.state.Load().generation) + require.NotContains(t, host.state.Load().active, "new") + _, err = host.Bind(actor, "POST", "/vendor/v1/jobs", "undeclared-model", taskusage.MiniMaxH3Contract()) + require.ErrorIs(t, err, ErrBindingRejected) + other := actor + other.TokenID++ + _, err = host.Resolve(binding, other) + require.ErrorIs(t, err, ErrBindingRejected) + corrupt := binding + corrupt.ContractDigest = strings.Repeat("0", 64) + _, err = host.Resolve(corrupt, actor) + require.ErrorIs(t, err, ErrBindingRejected) + + // Rebuild after a process restart from retained, externally pinned packages. + restarted := NewHost() + _, err = restarted.Install(first) + require.NoError(t, err) + _, err = restarted.Resolve(binding, actor) + require.NoError(t, err) + require.NoError(t, host.Revoke(binding.PluginKey, binding.Version)) + _, err = session.UsageEnvelope(nil) + require.ErrorIs(t, err, ErrBindingRejected) +} + +func TestHostConcurrentInstallPreservesEveryPackage(t *testing.T) { + host := NewHost() + const count = 8 + packages := make([]*Package, count) + for i := range packages { + packages[i] = testPackage(t, registryManifest(fmt.Sprintf("p%d", i), fmt.Sprintf("/p%d/:id", i)), "original") + } + var group sync.WaitGroup + errors := make(chan error, count) + for _, p := range packages { + group.Add(1) + go func() { defer group.Done(); _, err := host.Install(p); errors <- err }() + } + group.Wait() + close(errors) + for err := range errors { + require.NoError(t, err) + } + require.Len(t, host.state.Load().active, count) +} + +func TestHostEnvelopePreservesExplicitZeroAndRejectsInvalidFacts(t *testing.T) { + host := NewHost() + _, err := host.Install(testPackage(t, validManifest(), "original")) + require.NoError(t, err) + actor := Actor{UserID: 1, TokenID: 2, ChannelID: 3, RequestID: "generation-one"} + binding, err := host.Bind(actor, "POST", "/vendor/v1/jobs", "vendor-model", taskusage.DoubaoVideoContract()) + require.NoError(t, err) + session, err := host.Resolve(binding, actor) + require.NoError(t, err) + zero := int64(0) + envelope, err := session.UsageEnvelope(&types.TaskUsage{CompletionTokens: &zero, TotalTokens: &zero, Source: types.TaskUsageSourceProviderResponse, Completeness: types.TaskUsageCompletenessComplete}) + require.NoError(t, err) + require.Equal(t, types.TaskUsagePresencePresentZero, envelope.Presence) + _, err = session.UsageEnvelope(&types.TaskUsage{Source: types.TaskUsageSourceRequestEstimate, Completeness: types.TaskUsageCompletenessComplete}) + require.Error(t, err) +} + +func TestHostMappedBindingPreservesOriginAndUpstreamIdentity(t *testing.T) { + host := NewHost() + _, err := host.Install(testPackage(t, validManifest(), "original")) + require.NoError(t, err) + actor := Actor{UserID: 1, TokenID: 2, ChannelID: 3, RequestID: "mapped-request"} + binding, err := host.BindMapped(actor, "POST", "/vendor/v1/jobs", "client-alias", "vendor-model", taskusage.MiniMaxH3Contract()) + require.NoError(t, err) + require.Equal(t, "client-alias", binding.Model) + require.Equal(t, "vendor-model", binding.UpstreamModel) + _, err = host.Resolve(binding, actor) + require.NoError(t, err) + binding.UpstreamModel = "unknown" + _, err = host.Resolve(binding, actor) + require.ErrorIs(t, err, ErrBindingRejected) +} + +func TestHostOverlappingRoutesRespectManifestModels(t *testing.T) { + host := NewHost() + alpha := registryManifest("alpha", "/jobs/:id") + alpha.Models = []string{"model-a"} + beta := registryManifest("beta", "/:kind/latest") + beta.Models = []string{"model-b"} + _, err := host.Install(testPackage(t, alpha, "alpha"), testPackage(t, beta, "beta")) + require.NoError(t, err) + actor := Actor{UserID: 1, TokenID: 2, ChannelID: 3, RequestID: "overlapping-path"} + for _, item := range []struct{ model, key string }{{"model-a", "alpha"}, {"model-b", "beta"}} { + binding, err := host.Bind(actor, "GET", "/jobs/latest", item.model, taskusage.MiniMaxH3Contract()) + require.NoError(t, err) + require.Equal(t, item.key, binding.PluginKey) + } + _, err = host.Bind(actor, "GET", "/jobs/latest", "unknown-model", taskusage.MiniMaxH3Contract()) + require.ErrorIs(t, err, ErrBindingRejected) +} + +func TestLoadPackageRejectsUnsupportedEnvelopeFields(t *testing.T) { + data, err := common.Marshal(map[string]any{"format_version": 1, "manifest": validManifest(), "source": "original", "enabled": true}) + require.NoError(t, err) + hash := sha256.Sum256(data) + _, err = LoadPackage(bytes.NewReader(data), fmt.Sprintf("%x", hash)) + require.ErrorContains(t, err, "unsupported task plugin package fields") +} diff --git a/pkg/taskplugin/package.go b/pkg/taskplugin/package.go new file mode 100644 index 00000000..3c9c8e2e --- /dev/null +++ b/pkg/taskplugin/package.go @@ -0,0 +1,92 @@ +/* +Copyright (C) 2023-2026 MAX-API-Next + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +For commercial licensing, please contact https://github.com/MAX-API-Next/MAX-API/issues +*/ + +package taskplugin + +import ( + "crypto/sha256" + "encoding/hex" + "errors" + "io" + "unicode/utf8" + + "github.com/MAX-API-Next/MAX-API/common" +) + +const MaxPackageBytes = 1 << 20 + +// PackageDocument is MAX-API's data-only transport envelope, not executable +// JavaScript metadata. Source is retained for a future sandbox, never evaluated. +type PackageDocument struct { + FormatVersion int `json:"format_version"` + Manifest Manifest `json:"manifest"` + Source string `json:"source"` +} + +// Package has already passed an administrator-supplied digest check. The hash +// covers the entire document, including routing and network permissions. +type Package struct { + manifest Manifest + source string + digest string + size int +} + +// LoadPackage accepts exactly one bounded document and requires a trusted hash +// from outside the document. Computing a hash from an untrusted upload alone +// does not establish publisher authenticity. +func LoadPackage(reader io.Reader, expectedSHA256 string) (*Package, error) { + expected, err := hex.DecodeString(expectedSHA256) + if reader == nil || err != nil || len(expected) != sha256.Size { + return nil, errors.New("task plugin package requires a SHA-256 pin") + } + data, err := io.ReadAll(io.LimitReader(reader, MaxPackageBytes+1)) + if err != nil || len(data) > MaxPackageBytes || !utf8.Valid(data) { + return nil, errors.New("task plugin package unreadable or exceeds size limit") + } + actual := sha256.Sum256(data) + if hex.EncodeToString(expected) != hex.EncodeToString(actual[:]) { + return nil, errors.New("task plugin package digest mismatch") + } + var fields map[string]any + if common.Unmarshal(data, &fields) != nil || len(fields) != 3 { + return nil, errors.New("unsupported task plugin package fields") + } + for _, name := range []string{"format_version", "manifest", "source"} { + if _, ok := fields[name]; !ok { + return nil, errors.New("unsupported task plugin package fields") + } + } + var document PackageDocument + if err := common.Unmarshal(data, &document); err != nil { + return nil, errors.New("invalid task plugin package document") + } + if document.FormatVersion != 1 || document.Source == "" || !utf8.ValidString(document.Source) { + return nil, errors.New("unsupported task plugin package format or invalid source") + } + if err := ValidateManifest(document.Manifest); err != nil { + // Validation errors can contain attacker-supplied metadata. Keep source + // and credentials out of any caller's logs and API error response. + return nil, errors.New("invalid task plugin package manifest") + } + return &Package{manifest: cloneManifest(document.Manifest), source: document.Source, digest: hex.EncodeToString(actual[:]), size: len(data)}, nil +} + +func (p *Package) Manifest() Manifest { return cloneManifest(p.manifest) } +func (p *Package) Digest() string { return p.digest } diff --git a/pkg/taskplugin/registry.go b/pkg/taskplugin/registry.go index 0dc42aa9..16a2ad3d 100644 --- a/pkg/taskplugin/registry.go +++ b/pkg/taskplugin/registry.go @@ -159,7 +159,7 @@ func (snapshot Snapshot) MatchRoute(method, requestPath, model string) (RouteMat continue } manifest, ok := snapshot.state.plugins[binding.pluginKey] - if !ok { + if !ok || !declaresModel(manifest, model) { continue } return RouteMatch{PluginKey: binding.pluginKey, Manifest: cloneManifest(manifest), Route: cloneRoute(binding.route)}, true @@ -207,6 +207,15 @@ func routeAllowsModel(route Route, model string) bool { return false } +func declaresModel(manifest Manifest, model string) bool { + for _, candidate := range manifest.Models { + if candidate == model { + return true + } + } + return false +} + func routeMatches(pattern, requestPath string) bool { parsed, err := url.ParseRequestURI(requestPath) if err != nil || parsed.Path != requestPath { @@ -254,8 +263,9 @@ func cloneManifest(manifest Manifest) Manifest { manifest.AllowedHosts = append([]string(nil), manifest.AllowedHosts...) manifest.Upstreams = append([]string(nil), manifest.Upstreams...) if manifest.UsageSchema != nil { + schema := manifest.UsageSchema manifest.UsageSchema = make(map[string]UsageField, len(manifest.UsageSchema)) - for key, field := range manifest.UsageSchema { + for key, field := range schema { field.Enum = append([]string(nil), field.Enum...) manifest.UsageSchema[key] = field } diff --git a/service/task_plugin_host.go b/service/task_plugin_host.go new file mode 100644 index 00000000..31ca372f --- /dev/null +++ b/service/task_plugin_host.go @@ -0,0 +1,94 @@ +/* +Copyright (C) 2023-2026 MAX-API-Next + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +For commercial licensing, please contact https://github.com/MAX-API-Next/MAX-API/issues +*/ + +package service + +import ( + "github.com/MAX-API-Next/MAX-API/model" + "github.com/MAX-API-Next/MAX-API/pkg/taskplugin" + "github.com/MAX-API-Next/MAX-API/pkg/taskusage" + relaycommon "github.com/MAX-API-Next/MAX-API/relay/common" + "github.com/MAX-API-Next/MAX-API/types" +) + +// AttachTaskPluginBinding prepares private task metadata for the caller's +// existing persistence transaction, before upstream acceptance. It does not +// reserve, settle, refund, submit or retry anything. +func AttachTaskPluginBinding(host *taskplugin.Host, task *model.Task, binding types.TaskPluginBinding) error { + if !taskPluginBindingMatches(task, binding) { + return taskplugin.ErrBindingRejected + } + if _, err := host.Resolve(binding, taskplugin.Actor{UserID: task.UserId, TokenID: task.PrivateData.TokenId, ChannelID: task.ChannelId, RequestID: task.PrivateData.BillingRequestId}); err != nil { + return err + } + if task.PrivateData.PluginBinding != nil && *task.PrivateData.PluginBinding != binding { + return taskplugin.ErrBindingRejected + } + copy := binding + task.PrivateData.PluginBinding = © + task.IncludePrivateDataInUpdate() + return nil +} + +func taskPluginBindingMatches(task *model.Task, binding types.TaskPluginBinding) bool { + if task == nil || task.PrivateData.BillingContext == nil { + return false + } + plan := task.PrivateData.BillingContext.TaskBillingPlan + upstream := task.Properties.UpstreamModelName + if upstream == "" { + upstream = task.Properties.OriginModelName + } + return plan != nil && binding.UserID > 0 && binding.UserID == task.UserId && binding.TokenID > 0 && binding.TokenID == task.PrivateData.TokenId && binding.ChannelID > 0 && binding.ChannelID == task.ChannelId && binding.RequestID != "" && binding.RequestID == task.PrivateData.BillingRequestId && binding.Model != "" && binding.Model == task.Properties.OriginModelName && binding.UpstreamModel == upstream && binding.SourceID == plan.UsageSourceID && binding.SchemaVersion == plan.UsageSchemaVersion && binding.ContractDigest == plan.UsageContractDigest +} + +// ResolveTaskPluginSession validates persisted ownership and frozen billing +// identity. Restoring a binding never falls back to the latest plugin version. +func ResolveTaskPluginSession(host *taskplugin.Host, task *model.Task) (*taskplugin.Session, error) { + if task == nil || task.ID <= 0 || task.PrivateData.PluginBinding == nil || !taskPluginBindingMatches(task, *task.PrivateData.PluginBinding) { + return nil, taskplugin.ErrBindingRejected + } + return host.Resolve(*task.PrivateData.PluginBinding, taskplugin.Actor{UserID: task.UserId, TokenID: task.PrivateData.TokenId, ChannelID: task.ChannelId, RequestID: task.PrivateData.BillingRequestId}) +} + +// ApplyTaskPluginUsage only attaches validated provider facts. All terminal +// decisions and money mutations remain in the existing polling/settlement path. +func ApplyTaskPluginUsage(host *taskplugin.Host, task *model.Task, result *relaycommon.TaskInfo, usage *types.TaskUsage) error { + if result == nil { + return taskplugin.ErrBindingRejected + } + session, err := ResolveTaskPluginSession(host, task) + if err != nil { + return err + } + envelope, err := session.UsageEnvelope(usage) + if err != nil { + return err + } + candidate := *result + if envelope.SourceID != taskusage.SourceIDDoubaoVideo || envelope.Completeness == types.TaskUsageCompletenessComplete { + if err := applyLegacyTaskUsageFields(&candidate, envelope); err != nil { + return err + } + } + candidate.UsageEnvelope = types.CloneTaskUsageEnvelope(envelope) + candidate.Usage = types.CloneTaskUsage(envelope.Usage) + *result = candidate + return nil +} diff --git a/service/task_plugin_host_test.go b/service/task_plugin_host_test.go new file mode 100644 index 00000000..fd40906b --- /dev/null +++ b/service/task_plugin_host_test.go @@ -0,0 +1,75 @@ +/* +Copyright (C) 2023-2026 MAX-API-Next + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +For commercial licensing, please contact https://github.com/MAX-API-Next/MAX-API/issues +*/ + +package service + +import ( + "bytes" + "crypto/sha256" + "fmt" + "testing" + + "github.com/MAX-API-Next/MAX-API/common" + "github.com/MAX-API-Next/MAX-API/model" + "github.com/MAX-API-Next/MAX-API/pkg/taskplugin" + "github.com/MAX-API-Next/MAX-API/pkg/taskusage" + relaycommon "github.com/MAX-API-Next/MAX-API/relay/common" + "github.com/MAX-API-Next/MAX-API/types" + "github.com/stretchr/testify/require" +) + +func TestTaskPluginHostRequiresPersistedOwnerAndFrozenUsageContract(t *testing.T) { + manifest := taskplugin.Manifest{APIVersion: 1, Key: "host-test", Name: "Host test", Version: "1.0.0", Models: []string{"original-model"}, Routes: []taskplugin.Route{{Method: "POST", Path: "/plugin/jobs", Type: "submit", Decode: "decode", Render: "render"}}} + data, err := common.Marshal(taskplugin.PackageDocument{FormatVersion: 1, Manifest: manifest, Source: "not executed"}) + require.NoError(t, err) + hash := sha256.Sum256(data) + p, err := taskplugin.LoadPackage(bytes.NewReader(data), fmt.Sprintf("%x", hash)) + require.NoError(t, err) + host := taskplugin.NewHost() + _, err = host.Install(p) + require.NoError(t, err) + actor := taskplugin.Actor{UserID: 1, TokenID: 2, ChannelID: 3, RequestID: "submission-one"} + contract := taskusage.DoubaoVideoContract() + binding, err := host.BindMapped(actor, "POST", "/plugin/jobs", "client-alias", "original-model", contract) + require.NoError(t, err) + task := &model.Task{ID: 99, UserId: 1, ChannelId: 3, Properties: model.Properties{OriginModelName: "client-alias", UpstreamModelName: "original-model"}, PrivateData: model.TaskPrivateData{TokenId: 2, BillingRequestId: "submission-one", BillingContext: &model.TaskBillingContext{TaskBillingPlan: &types.TaskBillingPlan{UsageSourceID: binding.SourceID, UsageSchemaVersion: binding.SchemaVersion, UsageContractDigest: binding.ContractDigest}}}} + require.NoError(t, AttachTaskPluginBinding(host, task, binding)) + // Roundtrip exactly the JSON value that model.Task stores in private_data. + persisted, err := common.Marshal(task.PrivateData) + require.NoError(t, err) + task.PrivateData = model.TaskPrivateData{} + require.NoError(t, common.Unmarshal(persisted, &task.PrivateData)) + zero := int64(0) + usage := &types.TaskUsage{CompletionTokens: &zero, TotalTokens: &zero, Source: types.TaskUsageSourceProviderResponse, Completeness: types.TaskUsageCompletenessComplete} + result := &relaycommon.TaskInfo{Status: string(model.TaskStatusSuccess)} + require.NoError(t, ApplyTaskPluginUsage(host, task, result, usage)) + require.Equal(t, types.TaskUsagePresencePresentZero, result.UsageEnvelope.Presence) + require.Equal(t, types.TaskUsageProducerKindTaskPlugin, result.UsageEnvelope.ProducerKind) + task.PrivateData.TokenId = 100 + before := *result + require.ErrorIs(t, ApplyTaskPluginUsage(host, task, result, usage), taskplugin.ErrBindingRejected) + require.Equal(t, before, *result, "invalid ownership must not mutate polling evidence") + task.PrivateData.TokenId = 2 + task.PrivateData.BillingContext.TaskBillingPlan.UsageContractDigest = "tampered" + require.ErrorIs(t, ApplyTaskPluginUsage(host, task, result, usage), taskplugin.ErrBindingRejected) + public, err := common.Marshal(task) + require.NoError(t, err) + require.NotContains(t, string(public), "plugin_binding") + require.NotContains(t, string(public), binding.PackageDigest) +} diff --git a/types/task_plugin.go b/types/task_plugin.go new file mode 100644 index 00000000..ce327d1d --- /dev/null +++ b/types/task_plugin.go @@ -0,0 +1,38 @@ +/* +Copyright (C) 2023-2026 MAX-API-Next + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +For commercial licensing, please contact https://github.com/MAX-API-Next/MAX-API/issues +*/ + +package types + +// TaskPluginBinding is host-issued private task metadata. It carries no +// credentials or pricing and must never be accepted from a client body. +// Persist it with the task before sending any billable upstream request. +type TaskPluginBinding struct { + PluginKey string `json:"plugin_key"` + Version string `json:"version"` + PackageDigest string `json:"package_digest"` + SourceID string `json:"source_id"` + SchemaVersion int `json:"schema_version"` + ContractDigest string `json:"contract_digest"` + Model string `json:"model"` + UpstreamModel string `json:"upstream_model"` + UserID int `json:"user_id"` + TokenID int `json:"token_id"` + ChannelID int `json:"channel_id"` + RequestID string `json:"request_id"` +}