Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions docs/task-plugin-host.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Task Plugin 宿主基础 / Host foundation

`pkg/taskplugin` 提供内部宿主基础,尚未注册公开管理或转发路由。`PackageDocument` 是 MAX-API 的数据包封装,不是 New API JavaScript 插件的直接安装格式;`source` 保持不透明,加载不会执行代码。本阶段不包含 JS VM、插件市场、管理界面或真实插件渠道接入。

`pkg/taskplugin` provides an internal host foundation. It does not register public management or relay routes. `PackageDocument` is a MAX-API data envelope, not a drop-in JavaScript plugin installation format. Source remains opaque and is never evaluated during loading. A JS VM, marketplace, management UI and live plugin channels are outside this stage.

## 加载和任务身份 / Loading and task identity

- `LoadPackage` 最多读取 1 MiB,校验整个文档的 SHA-256。期望摘要必须由宿主从可信渠道提供,不能仅根据上传文件自行计算后当作来源认证。
- `Host.Install` 明确激活所选版本并批量发布插件和路由,包括操作员主动选择的版本回退;任一验证或路由冲突失败时整批不生效。同一 key/version 不允许替换不同内容。
- `Host.Retain` 仅恢复历史包,不改变活动版本和路由;可以一次保留同一 key 的多个版本。补载历史任务所需的包时使用此接口,不使用 `Install`。摘要冲突、撤销和总容量限制仍适用;恢复流量前仍须恢复撤销记录。
- 路由只匹配 manifest 声明的上游模型;省略 `route.Models` 表示继承 manifest 的范围,指定时进一步收窄。缺失或未知模型拒绝绑定。
- `BindMapped` 固定插件版本、包摘要、原始及映射后模型,以及宿主拥有的 usage contract;`Resolve` 使用历史包恢复任务,不自动切换最新版本。
- `AttachTaskPluginBinding` 校验用户、Token、渠道、请求、模型及冻结用量契约,写入 `Task.PrivateData`。调用方必须在发起付费请求前通过现有事务保存任务;该函数自身不执行数据库写入或上游提交。
- 重启时必须重新加载所需历史包,并在开放流量前恢复宿主的撤销记录。缺失历史包、无效契约、身份变化或已撤销版本均拒绝恢复,不自动重提任务。当前没有自动保存插件目录或撤销清单的管理服务。

The loader accepts at most 1 MiB and verifies an externally trusted SHA-256 of the whole document. `Host.Install` explicitly activates the chosen versions, including operator-selected rollbacks, and publishes packages and routes together; a conflict leaves the previous generation intact. `Host.Retain` restores history without changing active versions or routes, and accepts several versions of one key in a batch. Use `Retain` rather than `Install` when loading historical task packages. Digest conflicts, revocations and the total retention limit still apply; revocations must be restored before traffic resumes. A key/version cannot silently change contents. Routes match only manifest-declared upstream models; an omitted `route.Models` inherits that set, while an explicit list narrows it. Missing or unknown models cannot bind. Bindings freeze package, usage contract, client model alias and upstream model. Recovery resolves historical packages rather than the latest version.

The service validates task ownership and the frozen usage contract before attaching private metadata. The caller must persist it in the existing transaction before any billable request. The host must reload retained historical packages and revocations before serving traffic after restart. Missing or revoked identities fail closed and never cause automatic submission. Automatic catalog/revocation storage is not implemented here.

## HTTP 能力 / HTTP capability

`Session.NewFetchScope` 只提供 GET:HTTPS、主渠道 origin 与 manifest 额外 host 白名单、禁止环境代理与重定向、每次解析全部 IP 后直连已检查的 IP。回环、私有、链路本地及保留网段均拒绝。渠道 Authorization 只发往精确的渠道 origin,额外 host 不携带该凭据。插件不能提供任意请求头、请求体或 Cookie。

每个 hook 创建并关闭独立 scope:最多 4 次调用、每次最多 15 秒、scope 总期限 30 秒、响应体最多 1 MiB,响应头最多 64 KiB。网络错误不会包含原始 URL、查询参数或认证信息。此能力不支持内网/明文上游,也不用于提交付费任务;既有 Go adaptor 行为不变。

Each hook gets a separate, closed-after-use GET capability: HTTPS only, exact channel origin plus declared additional hosts, no environment proxy, redirects, arbitrary headers, body or cookies. All DNS answers are checked before dialing a validated literal IP. Private, loopback, link-local and reserved addresses are rejected. Channel authorization is attached only to the exact channel origin.

Limits are four calls, 15 seconds per call, a 30-second scope deadline, 1 MiB response bodies and 64 KiB response headers. Network errors omit URLs, query strings and credentials. Private/plaintext upstreams and billable submission are not supported by this read-only capability. Existing Go adaptors are unchanged.

## 计费与验证 / Billing and validation

`ApplyTaskPluginUsage` 只附加经宿主契约验证的用量事实。显式零与缺失用量保持不同;不增加价格、额度扣减、退款或任务终态写入。结算继续使用冻结的计费计划、`task:<id>:finalize`、资金事务和独立日志/receipt effects。

`ApplyTaskPluginUsage` attaches validated facts only and preserves explicit zero versus missing usage. It does not price, debit, refund or publish task terminal state. The existing frozen plan, operation key, funding transaction and separate effects remain authoritative.

```powershell
go test ./pkg/taskplugin ./pkg/taskusage -count=1
go test ./service -run TestTaskPluginHost -count=1
go test ./model -run TestTaskPluginHost -count=1
```

这些合成测试不代表真实供应商、JS 隔离、多节点撤销传播或 MySQL/PostgreSQL 实库验收已通过。

Synthetic tests do not establish live-provider, JS isolation, multi-node revocation propagation or MySQL/PostgreSQL acceptance.
5 changes: 3 additions & 2 deletions model/task.go
Original file line number Diff line number Diff line change
Expand Up @@ -140,8 +140,9 @@ func (m Properties) Value() (driver.Value, error) {
}

type TaskPrivateData struct {
Key string `json:"key,omitempty"`
UpstreamTaskID string `json:"upstream_task_id,omitempty"` // 上游真实 task ID
PluginBinding *types.TaskPluginBinding `json:"plugin_binding,omitempty"` // host-issued immutable plugin identity; never client-visible
Key string `json:"key,omitempty"`
UpstreamTaskID string `json:"upstream_task_id,omitempty"` // 上游真实 task ID
// AwaitingUpstreamID distinguishes a newly persisted placeholder from
// legacy rows where TaskID itself is the provider task identifier.
AwaitingUpstreamID bool `json:"awaiting_upstream_id,omitempty"`
Expand Down
100 changes: 100 additions & 0 deletions model/task_plugin_host_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
/*
Copyright (C) 2023-2026 MAX-API-Next

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.

For commercial licensing, please contact https://github.com/MAX-API-Next/MAX-API/issues
*/

package model

import (
"bytes"
"crypto/sha256"
"fmt"
"path/filepath"
"testing"

"github.com/MAX-API-Next/MAX-API/common"
"github.com/MAX-API-Next/MAX-API/pkg/taskplugin"
"github.com/MAX-API-Next/MAX-API/pkg/taskusage"
"github.com/MAX-API-Next/MAX-API/types"
"github.com/glebarez/sqlite"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
)

func TestTaskPluginHostBindingSurvivesSQLiteFundingAndEffectReplay(t *testing.T) {
db, err := gorm.Open(sqlite.Open(filepath.Join(t.TempDir(), "plugin.db")), taskBillingTestGORMConfig())
require.NoError(t, err)
sqlDB, err := db.DB()
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, sqlDB.Close()) })
prepareTaskBillingContractDB(t, db, "sqlite")
f := newTaskBillingContractFixture(t, db, 9992, BillingSettlementSourceWallet, -20)
manifest := taskplugin.Manifest{APIVersion: 1, Key: "ledger-test", Name: "Ledger test", Version: "1.0.0", Models: []string{"contract-model"}, Routes: []taskplugin.Route{{Method: "GET", Path: "/plugin/:id", Type: "query", Render: "render"}}}
data, err := common.Marshal(taskplugin.PackageDocument{FormatVersion: 1, Manifest: manifest, Source: "original fixture, never executed"})
require.NoError(t, err)
hash := sha256.Sum256(data)
p, err := taskplugin.LoadPackage(bytes.NewReader(data), fmt.Sprintf("%x", hash))
require.NoError(t, err)
host := taskplugin.NewHost()
_, err = host.Install(p)
require.NoError(t, err)
actor := taskplugin.Actor{UserID: f.user.Id, TokenID: f.token.Id, ChannelID: f.channel.Id, RequestID: "plugin-submission"}
binding, err := host.Bind(actor, "GET", "/plugin/one", "contract-model", taskusage.MiniMaxH3Contract())
require.NoError(t, err)
f.task.PrivateData.PluginBinding = &binding
f.task.PrivateData.TokenId = f.token.Id
f.task.PrivateData.BillingRequestId = actor.RequestID
require.NoError(t, db.Save(&f.task).Error)
var stored Task
require.NoError(t, db.First(&stored, f.task.ID).Error)
require.NotNil(t, stored.PrivateData.PluginBinding)
restarted := taskplugin.NewHost()
currentManifest := manifest
currentManifest.Version = "2.0.0"
currentData, err := common.Marshal(taskplugin.PackageDocument{FormatVersion: 1, Manifest: currentManifest, Source: "current fixture, never executed"})
require.NoError(t, err)
currentHash := sha256.Sum256(currentData)
currentPackage, err := taskplugin.LoadPackage(bytes.NewReader(currentData), fmt.Sprintf("%x", currentHash))
require.NoError(t, err)
_, err = restarted.Install(currentPackage)
require.NoError(t, err)
_, err = restarted.Retain(p)
require.NoError(t, err)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
session, err := restarted.Resolve(*stored.PrivateData.PluginBinding, actor)
require.NoError(t, err)
require.Equal(t, "1.0.0", session.Binding().Version)
require.Equal(t, p.Digest(), session.Binding().PackageDigest)
newBinding, err := restarted.Bind(actor, "GET", "/plugin/one", "contract-model", taskusage.MiniMaxH3Contract())
require.NoError(t, err)
require.Equal(t, "2.0.0", newBinding.Version, "restoring a persisted task must not switch the active version")
duration, images := int64(5_000), int64(0)
envelope, err := session.UsageEnvelope(&types.TaskUsage{OutputDurationMs: &duration, InputImageCount: &images, Source: types.TaskUsageSourceProviderResponse, Completeness: types.TaskUsageCompletenessComplete})
require.NoError(t, err)
stored.PrivateData.BillingContext.TaskUsageEnvelope = envelope
require.NoError(t, db.Save(&stored).Error)
for attempt := 0; attempt < 2; attempt++ {
applied, replay, err := ApplyBillingSettlementOnce(f.input)
require.NoError(t, err)
require.EqualValues(t, -20, applied)
require.Equal(t, attempt > 0, replay)
require.NoError(t, ProcessBillingSettlementEffect(f.input.OperationKey))
}
f.assertProjection(t, db, 80, 1)
require.NoError(t, db.First(&stored, f.task.ID).Error)
require.Equal(t, binding, *stored.PrivateData.PluginBinding)
require.Equal(t, envelope.EvidenceDigest, stored.PrivateData.BillingContext.TaskUsageEnvelope.EvidenceDigest)
}
219 changes: 219 additions & 0 deletions pkg/taskplugin/fetch.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,219 @@
/*
Copyright (C) 2023-2026 MAX-API-Next

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.

For commercial licensing, please contact https://github.com/MAX-API-Next/MAX-API/issues
*/

package taskplugin

import (
"context"
"crypto/tls"
"errors"
"io"
"net"
"net/http"
"net/url"
"strconv"
"strings"
"sync/atomic"
"time"

"github.com/MAX-API-Next/MAX-API/common"
)

const (
MaxFetchBytes = 1 << 20
MaxFetchCalls = 4
)

var ErrFetchRejected = errors.New("task plugin fetch rejected or failed")

// FetchScope is a read-only HTTP capability for one hook invocation. It has no
// cookie jar, proxy, redirects, body, caller headers, pricing or write access.
type FetchScope struct {
session *Session
ctx context.Context
cancel context.CancelFunc
client *http.Client
baseOrigin string
allowed map[string]bool
authorization string
calls atomic.Uint32
}

type FetchResult struct {
StatusCode int
Body []byte
}

// NewFetchScope takes credentials from the host channel configuration. They are
// sent only to the exact HTTPS channel origin, never to extra allowed hosts.
// A new scope must be closed by the host after each hook, even on hook failure.
func (session *Session) NewFetchScope(ctx context.Context, channelBaseURL, authorization string) (*FetchScope, error) {
if err := session.validate(); err != nil {
return nil, err
}
if ctx == nil || validateHTTPBaseURL(channelBaseURL) != nil || strings.ContainsAny(authorization, "\r\n") {
return nil, ErrFetchRejected
}
base, err := url.Parse(channelBaseURL)
if err != nil || base.Scheme != "https" {
return nil, ErrFetchRejected
}
bounded, cancel := context.WithTimeout(ctx, 30*time.Second)
scope := &FetchScope{session: session, ctx: bounded, cancel: cancel, allowed: map[string]bool{}, authorization: authorization, baseOrigin: httpsOrigin(base)}
if scope.baseOrigin == "" {
cancel()
return nil, ErrFetchRejected
}
scope.allowed[scope.baseOrigin] = true
for _, host := range session.plugin.manifest.AllowedHosts {
parsed, err := url.Parse("https://" + host)
if err != nil || httpsOrigin(parsed) == "" {
cancel()
return nil, ErrFetchRejected
}
scope.allowed[httpsOrigin(parsed)] = true
}
transport := &http.Transport{
Proxy: nil,
DialContext: strictPluginDial,
TLSClientConfig: &tls.Config{MinVersion: tls.VersionTLS12},
TLSHandshakeTimeout: 5 * time.Second,
ResponseHeaderTimeout: 10 * time.Second,
MaxResponseHeaderBytes: 64 << 10,
DisableKeepAlives: true,
}
scope.client = &http.Client{Transport: transport, Timeout: 15 * time.Second, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
return scope, nil
}

func (scope *FetchScope) Close() {
if scope != nil {
scope.cancel()
scope.client.CloseIdleConnections()
}
}

// Fetch never follows a redirect and never returns network errors containing a
// signed URL, query, header or credential. The original context error is safe.
func (scope *FetchScope) Fetch(rawURL string) (FetchResult, error) {
if scope == nil || len(rawURL) > 8192 {
return FetchResult{}, ErrFetchRejected
}
if err := scope.session.validate(); err != nil {
return FetchResult{}, err
}
if err := scope.ctx.Err(); err != nil {
return FetchResult{}, err
}
if scope.calls.Add(1) > MaxFetchCalls {
return FetchResult{}, ErrFetchRejected
}
u, err := url.Parse(rawURL)
if err != nil || u.User != nil || u.Fragment != "" || u.Opaque != "" || !isASCII(u.Host) || strings.ContainsAny(rawURL, "\\\r\n") {
return FetchResult{}, ErrFetchRejected
}
origin := httpsOrigin(u)
if !scope.allowed[origin] || origin == "" {
return FetchResult{}, ErrFetchRejected
}
request, err := http.NewRequestWithContext(scope.ctx, http.MethodGet, u.String(), nil)
if err != nil {
return FetchResult{}, ErrFetchRejected
}
request.Header.Set("Accept", "application/json")
if origin == scope.baseOrigin && scope.authorization != "" {
request.Header.Set("Authorization", scope.authorization)
}
response, err := scope.client.Do(request)
if err != nil {
if ctxErr := scope.ctx.Err(); ctxErr != nil {
return FetchResult{}, ctxErr
}
return FetchResult{}, ErrFetchRejected
}
defer func() { _ = response.Body.Close() }()
if response.ContentLength > MaxFetchBytes {
return FetchResult{}, ErrFetchRejected
}
body, err := io.ReadAll(io.LimitReader(response.Body, MaxFetchBytes+1))
if err != nil || len(body) > MaxFetchBytes {
return FetchResult{}, ErrFetchRejected
}
if err := scope.session.validate(); err != nil {
return FetchResult{}, err
}
return FetchResult{StatusCode: response.StatusCode, Body: body}, nil
}

func httpsOrigin(u *url.URL) string {
if u == nil || u.Scheme != "https" || u.Hostname() == "" || u.User != nil || !isASCII(u.Host) {
return ""
}
port := u.Port()
if port == "" {
port = "443"
}
number, err := strconv.Atoi(port)
if err != nil || number <= 0 || number > 65535 {
return ""
}
return "https://" + net.JoinHostPort(strings.ToLower(u.Hostname()), strconv.Itoa(number))
}

// Resolve and validate every address, then dial the chosen literal IP. This
// avoids a second DNS lookup between the SSRF check and the actual connection.
func strictPluginDial(ctx context.Context, network, address string) (net.Conn, error) {
return pluginDial(ctx, network, address, net.DefaultResolver.LookupIPAddr, (&net.Dialer{Timeout: 5 * time.Second}).DialContext)
}

func pluginDial(ctx context.Context, network, address string, lookup func(context.Context, string) ([]net.IPAddr, error), dial func(context.Context, string, string) (net.Conn, error)) (net.Conn, error) {
host, portText, err := net.SplitHostPort(address)
if err != nil {
return nil, ErrFetchRejected
}
port, err := strconv.Atoi(portText)
if err != nil {
return nil, ErrFetchRejected
}
protection := &common.SSRFProtection{AllowPrivateIp: false, DomainFilterMode: false, IpFilterMode: false, AllowedPorts: []int{port}, ApplyIPFilterForDomain: true}
if err := protection.ValidateNetworkTarget(host, port); err != nil {
return nil, ErrFetchRejected
}
var addresses []net.IPAddr
if ip := net.ParseIP(host); ip != nil {
addresses = []net.IPAddr{{IP: ip}}
} else {
addresses, err = lookup(ctx, host)
if err != nil || len(addresses) == 0 {
return nil, ErrFetchRejected
}
}
for _, address := range addresses {
if address.Zone != "" || protection.ValidateResolvedIP(host, address.IP) != nil {
return nil, ErrFetchRejected
}
}
for _, address := range addresses {
connection, err := dial(ctx, network, net.JoinHostPort(address.IP.String(), portText))
if err == nil {
return connection, nil
}
}
return nil, ErrFetchRejected
}
Loading
Loading