Skip to content

Commit 5c29f6c

Browse files
committed
feat: async execution with 202 response and executions query API
- Rule files can opt into async: true to return HTTP 202 immediately and run matched actions in a background goroutine - Global backpressure via server.max_async_tasks (default 32); excess requests are rejected synchronously with 429 - Execution policies (block/cooldown) are still enforced at accept time; the running mark is released when the background task ends - New internal/execstore ring buffer keeps the last 100 async records, queryable via GET /api/executions?limit=N - Graceful shutdown waits up to 30s for in-flight async tasks before closing rule loggers; shutdown order adjusted accordingly - Docs updated: configuration/usage/README in both languages, example.yaml gains an async comment
1 parent bd5abca commit 5c29f6c

17 files changed

Lines changed: 807 additions & 22 deletions

File tree

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,7 @@ $ curl -X POST http://localhost:9000/webhook/github-auto-deploy \
8686
- **Flexible Auth** — Token (Header/Query) + HMAC Signature (GitHub/GitLab) + IP whitelist with AND relationship
8787
- **Multi-Condition Filters** — Match against Header / Query / Body with operators: `eq` `ne` `contains` `regex`
8888
- **Execution Policies** — Three modes: `block` (prevent concurrency), `always` (always execute), `cooldown` (rate limiting)
89+
- **Async Execution** — Per-file `async: true` returns HTTP 202 instantly and runs actions in the background, queryable via `/api/executions`
8990
- **Policy Inheritance** — File-level → Rule-level override
9091
- **File-Level Filters** — Global constraints applied to all rules, AND with rule-level filters
9192
- **Parameter Passing** — Template variables (`{{.body.ref}}`) and `pass_args` to inject request data into commands

README_zh.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,7 @@ $ curl -X POST http://localhost:9000/webhook/github-auto-deploy \
8686
- **灵活验证** — Token(Header/Query)+ HMAC 签名(GitHub/GitLab)+ IP 白名单,AND 组合
8787
- **多条件过滤** — 支持 Header / Query / Body 匹配,操作符:`eq` `ne` `contains` `regex`
8888
- **执行策略** — 三种模式:`block`(防并发)、`always`(始终执行)、`cooldown`(冷却限频)
89+
- **异步执行** — 文件级 `async: true` 立即返回 HTTP 202,后台执行 actions,可通过 `/api/executions` 查询执行记录
8990
- **策略继承** — 文件级 → 规则级逐层覆盖
9091
- **文件级过滤** — 全局约束应用于所有规则,与规则级 filters 为 AND 组合
9192
- **参数传递** — 模板变量(`{{.body.ref}}`)、`pass_args` 和环境变量注入(`env_from`)将请求数据传入命令/脚本

cmd/hookrun/main.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@ func runServer() error {
125125
}
126126

127127
// Init engine
128-
eng := engine.New(configMgr.Rules(), log, cfg.Log.Mode, cfg.Log.RetentionDays, cfg.Log.MaxSizeMB)
128+
eng := engine.New(configMgr.Rules(), log, cfg.Log.Mode, cfg.Log.RetentionDays, cfg.Log.MaxSizeMB, cfg.Server.MaxAsyncTaskLimit())
129129

130130
// Init relay registry (if enabled)
131131
if cfg.Server.IsRelayRegistryEnabled() {

docs/configuration.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ HookRun uses two levels of YAML configuration:
1717
| `server.route` | string | `/webhook` | Base webhook endpoint path |
1818
| `server.allow_all` | bool | `false` | Allow base route (`/webhook`) to iterate all config files |
1919
| `server.max_body_size_mb` | int | `10` | Max request body size in MB. `0` = unlimited |
20+
| `server.max_async_tasks` | int | `32` | Max concurrent background async tasks. Excess requests get HTTP 429 |
2021
| `server.relay_registry_token` | string | `""` | Relay registry API auth token. Non-empty enables registry |
2122
| `server.max_relay_ttl` | int | `0` (unlimited) | Max TTL cap for registered targets (seconds) |
2223
| `server.max_registry_entries` | int | `100` | Max number of registered targets |
@@ -169,6 +170,7 @@ Each YAML file in `config_dir` defines a rule set. The **filename** (without `.y
169170
| Field | Type | Required | Description |
170171
|-------|------|----------|-------------|
171172
| `name` | string | Yes | Rule set name, used in logs and responses |
173+
| `async` | bool | No | `true` = return HTTP 202 immediately and run actions in background (default `false`) |
172174
| `auth` | object | No | Authentication settings (AND relationship) |
173175
| `execution` | object | No | File-level execution policy |
174176
| `filters` | array | No | File-level global filters (AND with rule-level) |
@@ -283,6 +285,30 @@ Priority: **Rule-level > File-level > Default (block)**
283285

284286
---
285287

288+
### 2.2.5 `async` — Asynchronous Execution
289+
290+
When `async: true`, a matched request is accepted immediately with HTTP 202 while the actions run in the background:
291+
292+
```yaml
293+
name: "deploy"
294+
async: true # return 202, execute in background
295+
execution:
296+
policy: "block"
297+
rules:
298+
- name: "deploy-main"
299+
actions:
300+
- type: "command"
301+
cmd: "deploy.sh"
302+
```
303+
304+
Behavior details:
305+
306+
- The 202 response carries a `request_id` used to correlate logs and execution records
307+
- Execution policies (`block` / `cooldown`) are still enforced — policy rejections (409/429) are returned synchronously
308+
- Backpressure: when the number of running background tasks reaches `server.max_async_tasks`, new requests are rejected with 429
309+
- Recent async executions are queryable via `GET /api/executions` (see Usage docs)
310+
- On shutdown, in-flight async tasks get a 30-second grace period before forced exit
311+
286312
### 2.3 `filters` — File-Level Filters
287313

288314
File-level filters act as **global constraints** applied to ALL rules in the config. They use **AND** logic with rule-level filters: both must match for a rule to execute.

docs/configuration_zh.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ HookRun 使用两级 YAML 配置:
1717
| `server.route` | string | `/webhook` | Webhook 基础路由路径 |
1818
| `server.allow_all` | bool | `false` | 是否允许基础路由 `/webhook` 遍历所有配置文件 |
1919
| `server.max_body_size_mb` | int | `10` | 请求体大小上限 MB。`0` = 不限制 |
20+
| `server.max_async_tasks` | int | `32` | 后台异步任务最大并发数,超出时新请求返回 HTTP 429 |
2021
| `server.relay_registry_token` | string | `""` | 注册池 API 鉴权 token。非空时启用注册池 |
2122
| `server.max_relay_ttl` | int | `0`(不限) | 注册目标的最大 TTL 上限(秒) |
2223
| `server.max_registry_entries` | int | `100` | 注册池最大容量 |
@@ -169,6 +170,7 @@ log:
169170
| 字段 | 类型 | 必填 | 说明 |
170171
|------|------|------|------|
171172
| `name` | string | 是 | 规则集名称,用于日志和响应 |
173+
| `async` | bool | 否 | `true` = 立即返回 HTTP 202,后台执行 actions(默认 `false`) |
172174
| `auth` | object | 否 | 认证设置(AND 关系) |
173175
| `execution` | object | 否 | 文件级执行策略 |
174176
| `filters` | array | 否 | 文件级全局过滤条件(与规则级 AND 组合) |
@@ -283,6 +285,30 @@ rules:
283285

284286
---
285287

288+
### 2.2.5 `async` — 异步执行
289+
290+
配置 `async: true` 后,匹配成功的请求立即返回 HTTP 202,actions 在后台执行:
291+
292+
```yaml
293+
name: "deploy"
294+
async: true # 返回 202,后台执行
295+
execution:
296+
policy: "block"
297+
rules:
298+
- name: "deploy-main"
299+
actions:
300+
- type: "command"
301+
cmd: "deploy.sh"
302+
```
303+
304+
行为说明:
305+
306+
- 202 响应携带 `request_id`,用于关联日志与执行记录
307+
- 执行策略(`block` / `cooldown`)依然生效 — 策略拒绝(409/429)同步返回
308+
- 背压保护:后台运行任务数达到 `server.max_async_tasks` 时,新请求返回 429
309+
- 最近的异步执行记录可通过 `GET /api/executions` 查询(见使用文档)
310+
- 关闭时在途异步任务有 30 秒宽限期,超时后强制退出
311+
286312
### 2.3 `filters` — 文件级过滤条件
287313

288314
文件级 filters 作为**全局约束**,应用于该文件下的所有规则。与规则级 filters 为 **AND** 关系:两者必须同时匹配才能执行规则。

docs/usage.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -303,6 +303,37 @@ When relay is configured, an additional `relay` field is included:
303303
{"status": "ok", "uptime": "2h30m15s", "rules": 3, "version": "x.y.z", "relay": {"role": "upstream+downstream", "upstream_targets": 3, "downstream_connected": true}}
304304
```
305305

306+
### Async Execution Records
307+
308+
```
309+
GET /api/executions?limit=20
310+
```
311+
312+
Lists recent asynchronous executions (newest first), kept in memory (last 100). No authentication, consistent with `/health`. `limit` defaults to 20 and is capped at 100.
313+
314+
```bash
315+
curl http://localhost:9000/api/executions?limit=5
316+
```
317+
318+
```json
319+
{
320+
"executions": [
321+
{
322+
"request_id": "1720000000000-a1b2c3d4",
323+
"config": "deploy",
324+
"rule": "deploy-main",
325+
"status": "succeeded",
326+
"started_at": "2026-08-05T10:00:00Z",
327+
"finished_at": "2026-08-05T10:00:12Z",
328+
"duration": "12.05s",
329+
"exit_code": 0
330+
}
331+
]
332+
}
333+
```
334+
335+
`status` is one of `running`, `succeeded`, or `failed`. Failed records carry `exit_code` (`-1` when not applicable) and an `error` message.
336+
306337
### Relay API
307338

308339
#### `GET /api/relay/status` — Comprehensive Relay Status (always available, no auth)
@@ -432,6 +463,22 @@ When `policy: "cooldown"` and the cooldown window is active:
432463
}
433464
```
434465

466+
### Accepted Asynchronously (202)
467+
468+
When the matched config has `async: true`, the request is accepted immediately and actions run in the background:
469+
470+
```json
471+
{
472+
"code": 202,
473+
"message": "Accepted, executing asynchronously",
474+
"config": "deploy",
475+
"rule": "deploy-main",
476+
"request_id": "1720000000000-a1b2c3d4"
477+
}
478+
```
479+
480+
Use `request_id` to correlate log entries and execution records.
481+
435482
### Base Route Disabled (400)
436483

437484
When `allow_all: false` and a request is sent to `/webhook`:

docs/usage_zh.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -303,6 +303,37 @@ GET /health
303303
{"status": "ok", "uptime": "2h30m15s", "rules": 3, "version": "x.y.z", "relay": {"role": "upstream+downstream", "upstream_targets": 3, "downstream_connected": true}}
304304
```
305305

306+
### 异步执行记录
307+
308+
```
309+
GET /api/executions?limit=20
310+
```
311+
312+
列出最近的异步执行记录(最新在前),保存在内存中(最近 100 条)。无需认证,与 `/health` 一致。`limit` 默认 20,上限 100。
313+
314+
```bash
315+
curl http://localhost:9000/api/executions?limit=5
316+
```
317+
318+
```json
319+
{
320+
"executions": [
321+
{
322+
"request_id": "1720000000000-a1b2c3d4",
323+
"config": "deploy",
324+
"rule": "deploy-main",
325+
"status": "succeeded",
326+
"started_at": "2026-08-05T10:00:00Z",
327+
"finished_at": "2026-08-05T10:00:12Z",
328+
"duration": "12.05s",
329+
"exit_code": 0
330+
}
331+
]
332+
}
333+
```
334+
335+
`status` 取值为 `running``succeeded``failed`。失败记录携带 `exit_code`(无退出码时为 `-1`)和 `error` 信息。
336+
306337
### Relay API
307338

308339
#### `GET /api/relay/status` — 综合 Relay 状态(始终可用,无需认证)
@@ -432,6 +463,22 @@ curl -H "Authorization: Bearer your-registry-secret" \
432463
}
433464
```
434465

466+
### 异步接受(202)
467+
468+
当匹配的配置文件设置了 `async: true` 时,请求立即被接受,actions 在后台执行:
469+
470+
```json
471+
{
472+
"code": 202,
473+
"message": "Accepted, executing asynchronously",
474+
"config": "deploy",
475+
"rule": "deploy-main",
476+
"request_id": "1720000000000-a1b2c3d4"
477+
}
478+
```
479+
480+
使用 `request_id` 关联日志条目和执行记录。
481+
435482
### 基础路由已禁用(400)
436483

437484
`allow_all: false` 时请求 `/webhook`

hooks/example.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,10 @@ execution:
2020
policy: "block" # "block" | "always" | "cooldown"
2121
# cooldown_seconds: 300 # only for "cooldown" policy
2222

23+
# Async execution (optional): return HTTP 202 immediately and run actions
24+
# in the background. Results are logged and queryable via GET /api/executions.
25+
# async: true
26+
2327
# Rule-level independent log file (dual-write: global + this file)
2428
# log:
2529
# path: "./logs/github-auto-deploy.log"

internal/config/loader.go

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -87,8 +87,8 @@ func (m *Manager) ValidateAll() []error {
8787
}
8888

8989
// Check for duplicate config names and routing file names
90-
seenNames := make(map[string]string) // config name -> source file
91-
seenFiles := make(map[string]string) // file name (routing key) -> source file
90+
seenNames := make(map[string]string) // config name -> source file
91+
seenFiles := make(map[string]string) // file name (routing key) -> source file
9292
for _, r := range m.rules {
9393
if prev, dup := seenNames[r.Name]; dup {
9494
errs = append(errs, fmt.Errorf("duplicate config name '%s' in '%s' and '%s'", r.Name, prev, r.FilePath))

internal/config/types.go

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ type ServerConfig struct {
2020
Route string `yaml:"route"`
2121
AllowAll *bool `yaml:"allow_all,omitempty"` // allow /webhook to iterate all configs (default: false)
2222
MaxBodySizeMB *int `yaml:"max_body_size_mb,omitempty"` // max request body in MB, 0 = unlimited (default: 10)
23+
MaxAsyncTasks *int `yaml:"max_async_tasks,omitempty"` // max concurrent background async tasks (default: 32)
2324
RelayRegistryToken string `yaml:"relay_registry_token,omitempty"` // registry API auth token (empty = registry disabled)
2425
MaxRelayTTL int `yaml:"max_relay_ttl,omitempty"` // max TTL cap in seconds, 0 = unlimited
2526
MaxRegistryEntries int `yaml:"max_registry_entries,omitempty"` // registry pool capacity (default: 100)
@@ -41,6 +42,7 @@ type RuleLogConfig struct {
4142
// RuleConfig represents a single hooks/*.yaml file structure.
4243
type RuleConfig struct {
4344
Name string `yaml:"name"`
45+
Async bool `yaml:"async,omitempty"` // true = return 202 and execute actions in background
4446
Auth *AuthConfig `yaml:"auth,omitempty"`
4547
Execution *ExecutionConfig `yaml:"execution,omitempty"`
4648
Filters []Filter `yaml:"filters,omitempty"` // file-level global filters (AND with rule-level)
@@ -206,6 +208,9 @@ func (g *GlobalConfig) Validate() error {
206208
if *g.Server.MaxBodySizeMB < 0 {
207209
return fmt.Errorf("server.max_body_size_mb must be >= 0, got %d", *g.Server.MaxBodySizeMB)
208210
}
211+
if g.Server.MaxAsyncTasks != nil && *g.Server.MaxAsyncTasks < 0 {
212+
return fmt.Errorf("server.max_async_tasks must be >= 0, got %d", *g.Server.MaxAsyncTasks)
213+
}
209214
if g.Server.MaxRegistryEntries < 0 {
210215
return fmt.Errorf("server.max_registry_entries must be >= 0, got %d", g.Server.MaxRegistryEntries)
211216
}
@@ -248,6 +253,15 @@ func (s *ServerConfig) IsAllowAll() bool {
248253
return *s.AllowAll
249254
}
250255

256+
// MaxAsyncTaskLimit returns the effective max concurrent async task count
257+
// (nil or non-positive values fall back to the default of 32).
258+
func (s *ServerConfig) MaxAsyncTaskLimit() int {
259+
if s.MaxAsyncTasks == nil || *s.MaxAsyncTasks <= 0 {
260+
return 32
261+
}
262+
return *s.MaxAsyncTasks
263+
}
264+
251265
// IsRelayRegistryEnabled returns true when the relay registry is activated.
252266
func (s *ServerConfig) IsRelayRegistryEnabled() bool {
253267
return s.RelayRegistryToken != ""

0 commit comments

Comments
 (0)