Repository navigation
test: add SQLite/MySQL acceptance and task plugin foundation #62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
1bc3f4a
test: verify pre3 SQLite startup and settlement recovery
CSCITech 1c9efa8
Merge pull request #57 from MAX-API-Next/sqlite-upgrade-acceptance
CSCITech 70b892a
feat(ops): add safe sqlite archive rehearsal tooling
CSCITech 14a531f
test(sqlite): rehearse restored business snapshots
CSCITech 46c1482
test(mysql): gate startup migration on isolated empty schema
CSCITech a17d695
test(mysql): allow explicitly named dedicated schema
CSCITech ed2c181
test(mysql): audit initialized schema compatibility
CSCITech 4c853d3
test(mysql): pin TLS for external billing contract
CSCITech 1c4f9ff
feat(mysql): support pinned TLS certificates
CSCITech 3520870
test(mysql): allow explicitly authorized billing schema
CSCITech 9c5aa92
feat(taskplugin): validate plugin manifest contract
CSCITech 5cc3c11
feat(taskplugin): add atomic plugin registry
CSCITech c9b0445
test(taskplugin): add deterministic mock driver
CSCITech 853ccc9
test(taskplugin): exercise sqlite settlement replay
CSCITech 1348246
fix: address CodeRabbit task plugin and TLS findings
CSCITech dc34eab
fix: close remaining PR62 review findings
CSCITech f00fdde
fix: make sqlite operations publication atomic
CSCITech File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| # SQLite 运维备份与恢复 / SQLite Operations Backup and Restore | ||
|
|
||
| `tools/sqliteops` 只处理**已停写**的 SQLite 文件副本。它不会打开数据库、执行迁移、复制配置内容或修改业务数据。 | ||
|
|
||
| `tools/sqliteops` operates only on a **quiesced** SQLite file set. It does not open a database, run migrations, copy configuration contents, or modify business data. | ||
|
|
||
| ## 成套备份 / Complete archive | ||
|
|
||
| 先停止 API 进程、后台写入任务和结算恢复任务,确认没有新的请求或日志写入,然后执行: | ||
|
|
||
| ```powershell | ||
| go run ./tools/sqliteops backup ` | ||
| --quiesced ` | ||
| --primary C:\data\ledger.sqlite ` | ||
| --log C:\data\logs.sqlite ` | ||
| --config C:\data\runtime.env ` | ||
| --destination C:\archives\max-api-20261016 | ||
| ``` | ||
|
|
||
| `--log` 省略或与 `--primary` 相同时,主库同时作为日志库归档。SQLite 的 `-wal` 和 `-shm` 文件存在时会随对应数据库一起归档。配置文件只记录文件名、大小和 SHA-256,**不会复制配置内容或密钥**。 | ||
|
|
||
| The `--log` argument may be omitted, or may point to the primary file when the application uses a shared database. Existing SQLite `-wal` and `-shm` sidecars are archived with their database. Configuration is fingerprinted by name, size, and SHA-256 only; its contents and secrets are never copied. | ||
|
|
||
| ## 验证与隔离恢复 / Verify and isolated restore | ||
|
|
||
| 归档不得覆盖已有目录。验证归档清单和所有文件哈希: | ||
|
|
||
| ```powershell | ||
| go run ./tools/sqliteops verify --backup C:\archives\max-api-20261016 | ||
| ``` | ||
|
|
||
| 恢复到全新的空目录: | ||
|
|
||
| ```powershell | ||
| go run ./tools/sqliteops restore ` | ||
| --backup C:\archives\max-api-20261016 ` | ||
| --target C:\rehearsals\max-api-20261016 | ||
| ``` | ||
|
|
||
| 验证会拒绝缺失、篡改、路径逃逸和未列入清单的额外文件。恢复只复制已验证的数据库文件和 WAL sidecar;配置仍需人工按安全渠道准备。恢复副本必须先使用当前版本执行初始化、重启和业务快照核对,不能直接接入生产流量。 | ||
|
|
||
| Verification rejects missing, modified, path-escaping, and unlisted files. Restore copies only verified database files and WAL sidecars; configuration must be supplied separately through a secure channel. Run initialization, restart, and business-snapshot checks against the restored copy before any traffic is admitted. | ||
|
|
||
| ## 业务快照验收 / Business snapshot rehearsal | ||
|
|
||
| 仓库中的合成验收会使用当前模型创建主库和独立日志库,执行归档与两次隔离恢复,然后重新打开恢复文件,比较用户额度、Token、渠道用量、订阅周期与预消费、settlement、日志和 receipt;主库还会在保持连接期间写入已提交但未检查点的真实 WAL 标记,并在恢复后核对。它只验证文件归档及当前 SQLite 模型数据的可读性,不代表真实生产库、跨库原子性或外部支付状态已经验证: | ||
|
|
||
| ```powershell | ||
| go test ./model -run TestSQLiteOperationsAcceptancePreservesBusinessSnapshot -count=1 | ||
| ``` | ||
|
|
||
| The repository rehearsal creates a synthetic primary database and independent log database from the current models, archives them, restores them twice into isolated directories, reopens both copies, and compares wallet quota, token and channel usage, subscription/pre-consume state, settlement, logs, and receipts. It also keeps the primary connection open while a committed, uncheckpointed WAL marker is archived and verifies that marker after restore. It validates the archive and current SQLite model readability only; it does not establish production-database, cross-database atomicity, or external-payment correctness. | ||
|
|
||
| ## 边界 / Boundaries | ||
|
|
||
| - `--quiesced` 是强制门禁;运行中复制单个 SQLite 文件不能证明恢复点一致。 | ||
| - 归档不代表主库、日志库和外部支付状态已经形成跨系统原子快照。 | ||
| - manual、unknown outcome、跨订阅周期和外部已接受但本地未完成的结算必须单独对账。 | ||
| - 新版本写入不保证旧版本可以直接降级;回退使用完整升级前归档,并核对期间新增写入。 | ||
| - 本工具不验证 MySQL/PostgreSQL、Redis、多主机、真实渠道、OAuth、掉电恢复或浏览器流程。 | ||
| - 合成业务快照测试不能替代真实进程启动、真实计费/渠道调用或 MySQL/PostgreSQL 兼容性测试。 | ||
|
|
||
| - `--quiesced` is mandatory; copying one SQLite file while it is live does not establish a consistent restore point. | ||
| - An archive is not an atomic snapshot across the business database, log database, and external payment state. | ||
| - Manual, unknown-outcome, cross-period, and externally accepted but locally incomplete settlements require separate reconciliation. | ||
| - New-version writes are not guaranteed to support direct downgrade; rollback uses the complete pre-upgrade archive and reconciles writes made during the rehearsal. | ||
| - This tool does not validate MySQL/PostgreSQL, Redis, multi-host operation, real providers, OAuth, power loss, or browser flows. | ||
| - The synthetic business-snapshot test does not replace live-process startup, real billing/provider calls, or MySQL/PostgreSQL compatibility testing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| # SQLite 升级验收 / SQLite upgrade acceptance | ||
|
|
||
| 备份、清单校验和隔离恢复命令见[SQLite 运维备份与恢复](sqlite-operations.md)。 | ||
|
|
||
| See [SQLite operations backup and restore](sqlite-operations.md) for the archive, manifest verification, and isolated restore commands. | ||
|
|
||
| 升级前停止接收新请求,在原数据库完成待处理资金结算及日志副作用,停止服务后一起备份业务库、日志库和配置。SQLite 使用 WAL 时,保留尚未合并的 WAL 文件;运行中的单个数据库文件复制不能作为完整备份。 | ||
|
|
||
| Before upgrading, stop new traffic, finish pending funding settlements and log effects against their original databases, stop the service, and back up the business database, log database, and configuration together. Preserve uncheckpointed WAL files. Copying only a database file while it is in use does not establish a complete backup. | ||
|
|
||
| 先对备份副本运行升级版本,核对钱包、Token、订阅用量与周期、历史日志、`billing_settlements` 和 `billing_log_receipts`,再重复启动并确认恢复没有再次扣费、退款或生成重复日志。待人工处理、未知结果或跨周期结算保留其核对状态,不能视为升级后自动成功。 | ||
|
|
||
| Run the upgraded version against an isolated backup copy first. Check wallet and token balances, subscription usage and period, historical logs, `billing_settlements`, and `billing_log_receipts`. Restart again and confirm recovery does not debit, refund, or project a log twice. Manual, unknown-outcome, and cross-period records retain their reconciliation boundaries; upgrading does not imply their successful settlement. | ||
|
|
||
| ## 日志文件 / Log files | ||
|
|
||
| 默认同库和旧 `LOG_SQL_DSN=local` 可以继续使用。新增 `LOG_SQLITE_PATH` 时按[独立日志数据库指引](sqlite-log-database.md)准备目标文件;这个参数不复制或合并旧日志。完成原库的 pending effects,并保持日志和 receipt 一起保留,是切换前的必要准备。新空日志文件的启动验证只适用于已完成这些准备的场景。 | ||
|
|
||
| Existing shared-database and `LOG_SQL_DSN=local` settings remain supported. Follow the [separate log database guide](sqlite-log-database.md) when configuring `LOG_SQLITE_PATH`. The setting does not copy or merge old logs. Finish effects against the original file and retain each log with its receipt before switching. Fresh-file startup acceptance assumes these preparations are complete. | ||
|
|
||
| ## 自动化覆盖 / Automated coverage | ||
|
|
||
| `model/sqlite_upgrade_test.go` 使用 MAX-API `v2.0.0-smartops.pre3` 固定提交 `48260ce6517a9ab68851a7ac499477981d78da63` 的真实 `InitDB/InitLogDB` 生成 schema fixture(39 张表、160 个索引),并填入合成数据。当前版本在独立测试进程中执行主节点迁移及恢复,覆盖共享、旧 local、已有独立目标与完成结算后的新空目标,多次启动、64 位余额、明确零费用、原资金来源与周期、旧 receipt 去重及目标文件的日志标记回填。LF 和 Windows CRLF 格式均验证。fixture 只有 schema,不包含用户数据、密钥或运行时二进制。 | ||
|
|
||
| The fixture comes from real `InitDB/InitLogDB` execution at the pinned MAX-API pre3 source revision: 39 tables and 160 indexes, with synthetic data added by the tests. Fresh test processes run current master migrations and recovery for shared, legacy-local, existing separate, and quiesced fresh log targets. Coverage includes repeated startup, 64-bit balances, explicit zero usage, original funding and periods, old receipt deduplication, and target-specific retry-marker backfill. Both LF and Windows CRLF inputs are checked. The committed fixture contains schema only, with no user data, credentials, or runtime binary. | ||
|
|
||
| 这个基线不等于验证了所有历史部署。更老版本、真实旧镜像与用户库、MySQL/PostgreSQL、混合引擎、Redis 多节点、在线切库、实盘掉电、保留清理策略及实际渠道仍需对应环境验收。升级后写入的数据不保证可直接降级;需要回退时使用成套升级前备份,并核对外部已接受请求和未完成结算。 | ||
|
|
||
| This baseline does not establish compatibility with every historical deployment. Older releases, actual old images and user databases, MySQL/PostgreSQL, mixed engines, Redis clusters, online switching, power interruption, retention cleanup, and real channels require their own acceptance. New-version writes are not guaranteed to support direct downgrade. Restore the complete pre-upgrade backup for rollback and reconcile externally accepted requests and incomplete settlements. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.