diff --git a/.changeset/tidy-eagles-shout.md b/.changeset/tidy-eagles-shout.md new file mode 100644 index 000000000..880d0a914 --- /dev/null +++ b/.changeset/tidy-eagles-shout.md @@ -0,0 +1,34 @@ +--- +"@cartesi/cli": minor +--- + +Add a `defineConfig` function, and configure applications from +`cartesi.config.ts` instead of `cartesi.toml`. + +An application is now configured by a `cartesi.config.ts` file that exports its +configuration through `defineConfig`, imported from `@cartesi/cli/config`. The +helper does nothing at runtime, and exists so the configuration is type checked +and completed by the editor. A configuration file may also export a function, +which receives the command being run and the mode, so a project can configure +itself differently for `cartesi build` and `cartesi run`. + +Applications not written in TypeScript or JavaScript describe the same +configuration as plain data, in a `cartesi.config.json`, `cartesi.config.yaml` +or `cartesi.config` file. + +The configuration gained a `run` section with the project defaults of the local +development environment (`epochLength`, `services`, `blockTime`, `forkUrl`, +...), so they do not have to be repeated on every `cartesi run`. Command line +options take precedence over it. + +Nvrams, the accounts drive markers and the `withdrawal` overrides are +configured the same way, with camelCase keys. As in the TOML parser, a key that +no section reads is an error, so a misspelled option is reported rather than +silently ignored. + +`cartesi.toml` keeps working, and is read when a project has no other +configuration file, but it is deprecated and now prints a warning. + +`resolveConfig` is now asynchronous, since a configuration file has to be +imported, and the `config` option of the API functions accepts a configuration +written inline, in the same shape a `cartesi.config.ts` file exports. diff --git a/CLAUDE.md b/CLAUDE.md index 8194b85d0..8a488926e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -52,7 +52,7 @@ The CLI build pipeline (`apps/cli`): `clean` → `codegen` (wagmi ABI generation - **`builder/`** — Drive builder implementations (directory, docker, tar, empty, none). Each builder produces ext2 or SquashFS filesystems for Cartesi Machine drives. - **`compose/`** — Docker Compose service definitions generated as TypeScript objects (anvil, node, bundler, database, paymaster, proxy, explorer, etc.). - **`exec/`** — Wrappers around subprocess execution (cartesi-machine, rollups) using `execa`. -- **`config.ts`** — Parses `cartesi.toml` (TOML-based project config) into typed `Config` objects. Defines drive configs, machine configs, and SDK versions. +- **`config.ts`, `config/`** — `config.ts` is the barrel the rest of the CLI imports; `config/` holds the types, defaults, errors and the parsers. An application is configured by a `cartesi.config.ts` file exporting `defineConfig({ ... })`, or by the same configuration as plain data in `cartesi.config.json`, `cartesi.config.yaml` or `cartesi.config`; `config/load.ts` finds and reads it, `config/normalize.ts` validates it, and `config/toml/` parses the deprecated `cartesi.toml`. - **`contracts.ts`** — Generated contract addresses and ABI bindings (via `@wagmi/cli`). - **`wallet.ts`** — Wallet utilities using `viem` for Ethereum interaction. diff --git a/apps/cli/README.md b/apps/cli/README.md index 3290c21e9..2711edab4 100644 --- a/apps/cli/README.md +++ b/apps/cli/README.md @@ -18,6 +18,115 @@ cartesi --help More documentation at [https://docs.cartesi.io](https://docs.cartesi.io). +## Configuration + +An application is configured by a `cartesi.config.ts` file at its root, which +exports its configuration through `defineConfig`: + +```ts +import { defineConfig } from "@cartesi/cli/config"; + +export default defineConfig({ + drives: { + root: { builder: "docker", dockerfile: "Dockerfile" }, + data: { builder: "empty", size: "64Mi", mount: "/mnt/data" }, + }, + machine: { + entrypoint: "dapp", + ramLength: "256Mi", + }, + run: { + epochLength: 10, + services: ["explorer"], + }, +}); +``` + +`defineConfig` does nothing at runtime: it exists so the configuration is type +checked and completed by the editor, without any annotation. A configuration +file is free to compute its configuration, and to export a function instead of +an object when it depends on what is being run: + +```ts +export default defineConfig(async ({ command, mode }) => ({ + machine: { envFile: `.env.${mode}` }, + run: { epochLength: command === "run" ? 10 : 720 }, +})); +``` + +`mode` comes from `CARTESI_ENV` (or `NODE_ENV`), and defaults to +`development`. + +The `run` section holds the defaults of the local development environment +started by `cartesi run`, so a project does not have to repeat the same +options on every invocation. Command line options always take precedence over +it. + +Nvrams, and the drive or nvram that holds the accounts of emergency +withdrawal, are configured the same way: + +```ts +export default defineConfig({ + nvrams: { + accounts: { accountsDrive: true, size: "4Ki" }, + state: { size: "8Ki", shared: true }, + }, + withdrawal: { account_size: 64 }, +}); +``` + +An application has no obligation to have a configuration file: without one, it +is built from a `Dockerfile` of its directory, which is what most applications +need. + +### Applications not written in TypeScript + +The very same configuration can be written as plain data, for applications not +written in TypeScript or JavaScript. The keys are the ones above, plus +`$schema`, which points an editor at a schema and is otherwise ignored: + +```yaml +# cartesi.config.yaml +machine: + entrypoint: dapp + ramLength: 256Mi +drives: + data: + builder: empty + size: 64Mi + mount: /mnt/data +run: + epochLength: 10 +``` + +The configuration file of a project is the first of these that exists: + +| File | Format | +| ---------------------------------------------------------- | ---------------------------------------- | +| `cartesi.config.ts`, `.mts`, `.cts`, `.js`, `.mjs`, `.cjs` | module exporting `defineConfig({ ... })` | +| `cartesi.config.json` | JSON | +| `cartesi.config.yaml`, `cartesi.config.yml` | YAML | +| `cartesi.config` | YAML, which accepts JSON as well | +| `cartesi.toml` | deprecated | + +TypeScript configuration files are read by the runtime itself, so there is no +build step and no extra dependency: the standalone binaries run on Bun, and +node strips the types from version 24, which this package requires. + +A key that no section reads is an error rather than a typo that silently does +nothing, in every format, which is why a `cartesi.config.json` may carry +`$schema` but nothing else the CLI does not know. + +### Migrating from `cartesi.toml` + +`cartesi.toml` still works, and is read when a project has no other +configuration file, but it is deprecated and prints a warning. The new formats +describe the same configuration, with one difference: keys are camelCase, as +the resolved configuration already is, so `extra_size` becomes `extraSize`, +`boot_args` becomes `bootargs` and `accounts_drive` becomes `accountsDrive`. +The keys of the `withdrawal` section keep the names the `cartesi-rollups-cli` +expects (`account_size`, `withdrawal_output_builder`). + ## Library Every command of the CLI is also available as a function, so applications can be @@ -57,9 +166,15 @@ The following functions are available: `addressBook`, `build`, `clean`, A few things to keep in mind: - functions operate on the current working directory, just like the CLI, and - read `cartesi.toml` from it by default. Functions that take a configuration - accept a path, a list of paths (merged in order), or an already parsed - `Config` object; + look the configuration file of the project up in it by default. Functions + that take a configuration accept a path, a list of paths (merged in order), + or the configuration written inline, in the same shape a `cartesi.config.ts` + file exports: + + ```ts + await build({ config: { machine: { ramLength: "256Mi" } } }); + await build({ config: "cartesi.config.production.ts" }); + ``` - functions are silent, and never write to the terminal. Pass `progress: "default"` (or `"verbose"`) to get the same output as the CLI; - functions throw on error, and never terminate the process; @@ -73,5 +188,7 @@ A few things to keep in mind: `DepositError` (`InsufficientBalanceError`, `InvalidAmountError` or `TokenNotFoundError`). -The package is typed, and the types of the configuration file (`Config`, -`DriveConfig`, `MachineConfig`, ...) are exported as well. +The package is typed, and the types of the configuration file (`UserConfig`, +`Config`, `DriveConfig`, `MachineConfig`, `RunConfig`, ...) are exported as +well, along with `defineConfig`, `loadConfig`, `findConfigFile`, +`normalizeConfig` and `mergeConfig`. diff --git a/apps/cli/build.ts b/apps/cli/build.ts index 6b6dccb92..6b8dfea9c 100644 --- a/apps/cli/build.ts +++ b/apps/cli/build.ts @@ -20,7 +20,7 @@ await Bun.build({ await Bun.build({ define, - entrypoints: ["./src/lib.ts"], + entrypoints: ["./src/lib.ts", "./src/defineConfig.ts"], minify: true, outdir: "dist", sourcemap: true, diff --git a/apps/cli/package.json b/apps/cli/package.json index c04a4984f..20351e5af 100644 --- a/apps/cli/package.json +++ b/apps/cli/package.json @@ -15,6 +15,10 @@ "types": "./dist/types/lib.d.ts", "default": "./dist/lib.js" }, + "./config": { + "types": "./dist/types/defineConfig.d.ts", + "default": "./dist/defineConfig.js" + }, "./package.json": "./package.json" }, "repository": "cartesi/cli", diff --git a/apps/cli/src/api/build.ts b/apps/cli/src/api/build.ts index c7d71185c..e00e12dc3 100644 --- a/apps/cli/src/api/build.ts +++ b/apps/cli/src/api/build.ts @@ -153,8 +153,8 @@ export const build = async ( ): Promise => { const { debug = false, drivesOnly = false, progress = "silent" } = options; - // get application configuration, from a Config object or 'cartesi.toml' - const config = resolveConfig(options.config); + // get application configuration, written inline or read from a file + const config = await resolveConfig(options.config, "build"); // destination directory for image and intermediate files const destination = path.resolve(getContextPath()); diff --git a/apps/cli/src/api/run.ts b/apps/cli/src/api/run.ts index c095cf1bd..8c32fbe45 100644 --- a/apps/cli/src/api/run.ts +++ b/apps/cli/src/api/run.ts @@ -13,6 +13,7 @@ import { getContextPath, getMachineHash, getProjectName } from "../base.js"; import { DEFAULT_SDK_VERSION, PREFERRED_PORT, + type RunConfig, type WithdrawalConfig, } from "../config.js"; import { @@ -35,6 +36,11 @@ import { resolveConfig, } from "./types.js"; +/** + * Options of {@link run}. Every option that is not about this particular + * invocation can also be set in the `run` section of the application + * configuration, which these take precedence over. + */ export type RunOptions = ConfigOptions & ProgressOptions & { /** @@ -267,6 +273,39 @@ const deployMachine = async (options: { return application; }; +/** + * Layer the three sources of every option of the environment: the options + * given to {@link run} (which is where the command line options end up), the + * `run` section of the application configuration, and the defaults. + * + * @param options options given to {@link run} + * @param config `run` section of the application configuration + * @returns the option values the environment is started with + */ +export const resolveRunOptions = ( + options: RunOptions, + config: RunConfig = {}, +) => ({ + blockTime: options.blockTime ?? config.blockTime ?? 2, + claimStagingPeriod: + options.claimStagingPeriod ?? config.claimStagingPeriod ?? 0, + cpus: options.cpus ?? config.cpus, + defaultBlock: options.defaultBlock ?? config.defaultBlock ?? "latest", + epochLength: options.epochLength ?? config.epochLength ?? 720, + forkBlockNumber: options.forkBlockNumber ?? config.forkBlockNumber, + forkUrl: options.forkUrl ?? config.forkUrl, + memory: options.memory ?? config.memory, + // a port of zero is not a port, so the first free one is resolved later + port: options.port || config.port, + projectName: options.projectName ?? config.projectName, + prt: options.prt ?? config.prt ?? false, + runtimeVersion: + options.runtimeVersion ?? config.runtimeVersion ?? DEFAULT_SDK_VERSION, + services: options.services ?? config.services ?? [], + verbose: + options.verbose ?? config.verbose ?? options.progress === "verbose", +}); + /** * Run a local Cartesi node for the application, and deploy to it the machine * snapshot built at `.cartesi/image`, if there is one. @@ -279,32 +318,46 @@ const deployMachine = async (options: { */ export const run = async (options: RunOptions = {}): Promise => { const { - blockTime = 2, - claimStagingPeriod = 0, - cpus, - defaultBlock = "latest", deploy: deployOnStart = true, detach = true, dryRun = false, - epochLength = 720, - memory, progress = "silent", - prt = false, - runtimeVersion = DEFAULT_SDK_VERSION, - services = [], } = options; - const verbose = options.verbose ?? progress === "verbose"; - - // project name explicitly defined or the current directory name - const projectName = getProjectName(options); - // get application configuration (e.g. use withdrawal config if present) - const applicationConfig = resolveConfig(options.config); + const applicationConfig = await resolveConfig(options.config, "run"); + + // the 'run' section of the configuration defines project level defaults, + // which the options given to this function always take precedence over + const { + blockTime, + claimStagingPeriod, + cpus, + defaultBlock, + epochLength, + memory, + prt, + runtimeVersion, + services, + verbose, + ...resolved + } = resolveRunOptions(options, applicationConfig.run); + + // project name explicitly defined, defined by the configuration, or the + // current directory name + const projectName = getProjectName(resolved); + + if (defaultBlock !== "finalized" && progress !== "silent") { + console.warn( + chalk.yellow( + `WARNING: default block is set to '${defaultBlock}', production configuration will likely use 'finalized'`, + ), + ); + } // resolve port number, using the first free port in a range, unless explicitly set const port = - options.port || + resolved.port || (await getPort({ port: portNumbers(PREFERRED_PORT, PREFERRED_PORT + 10), })); @@ -313,7 +366,7 @@ export const run = async (options: RunOptions = {}): Promise => { const url = `${host}:${port}`; // configure optional anvil fork - const forkConfig = await configureFork(options); + const forkConfig = await configureFork(resolved); if (forkConfig) { await assertForkConfig(forkConfig, { includePRT: prt }); @@ -322,14 +375,16 @@ export const run = async (options: RunOptions = {}): Promise => { // emergency withdrawal configuration, derived from the accounts drive of the built // machine. The configuration is resolved again, as a rebuild may have changed it. const imagePath = getContextPath("image"); - const resolveWithdrawalConfig = () => - getWithdrawalConfig(resolveConfig(options.config), imagePath, { - fork: forkConfig !== undefined, - }); + const resolveWithdrawalConfig = async () => + getWithdrawalConfig( + await resolveConfig(options.config, "run"), + imagePath, + { fork: forkConfig !== undefined }, + ); // fail before starting the environment if it can't be derived if (fs.existsSync(imagePath)) { - resolveWithdrawalConfig(); + await resolveWithdrawalConfig(); } // run compose environment @@ -373,7 +428,7 @@ export const run = async (options: RunOptions = {}): Promise => { // the accounts drive may have moved with a rebuild, resolved before // undeploying in case it is now invalid - const withdrawalConfig = resolveWithdrawalConfig(); + const withdrawalConfig = await resolveWithdrawalConfig(); // keep the consensus of the previous deployment, if there is one const consensus = deployment?.consensus; diff --git a/apps/cli/src/api/shell.ts b/apps/cli/src/api/shell.ts index 65dd9aa24..d413086d0 100644 --- a/apps/cli/src/api/shell.ts +++ b/apps/cli/src/api/shell.ts @@ -32,8 +32,8 @@ export type ShellOptions = ConfigOptions & { export const shell = async (options: ShellOptions = {}): Promise => { const { command = "/bin/sh", runAsRoot = false } = options; - // get application configuration, from a Config object or 'cartesi.toml' - const config = resolveConfig(options.config); + // get application configuration, written inline or read from a file + const config = await resolveConfig(options.config, "shell"); // destination directory for image and intermediate files const destination = path.resolve(getContextPath()); diff --git a/apps/cli/src/api/types.ts b/apps/cli/src/api/types.ts index 3b3fe59cb..09f4e2fb7 100644 --- a/apps/cli/src/api/types.ts +++ b/apps/cli/src/api/types.ts @@ -1,5 +1,10 @@ -import { getApplicationConfig } from "../base.js"; -import type { Config } from "../config.js"; +import { + type Config, + type ConfigCommand, + loadConfig, + normalizeConfig, + type UserConfig, +} from "../config.js"; /** * Verbosity of the progress information written to the terminal while an API @@ -23,38 +28,47 @@ export type ProgressOptions = { }; /** - * Application configuration, either already parsed, or a path (or list of - * paths) of TOML configuration files to be read and merged, in order. + * Application configuration, either written inline in the same shape a + * `cartesi.config.ts` file exports, or a path (or list of paths) of + * configuration files to be read and merged, in order. Files of any of the + * supported formats are accepted. An already resolved {@link Config} is a + * valid inline configuration as well. */ -export type ConfigInput = Config | string | string[]; +export type ConfigInput = UserConfig | string | string[]; export type ConfigOptions = { /** - * Application configuration, or path of the configuration file(s). - * @default "cartesi.toml" + * Application configuration, or path of the configuration file(s). When + * not given, the configuration file of the project is looked up by name. */ config?: ConfigInput; }; /** * Resolve the application configuration from the several ways it can be - * provided to the API: an already parsed {@link Config}, one configuration file - * path, a list of configuration file paths, or nothing (which falls back to - * `cartesi.toml` of the current directory). + * provided to the API: a configuration written inline, one configuration file + * path, a list of configuration file paths, or nothing (which looks the + * configuration file of the project up, and falls back to the defaults when + * there is none). * @param config configuration or path of configuration file(s) - * @returns parsed application configuration + * @param command command being run, given to a configuration file that exports + * a function + * @returns resolved application configuration */ -export const resolveConfig = (config?: ConfigInput): Config => { +export const resolveConfig = async ( + config?: ConfigInput, + command: ConfigCommand = "build", +): Promise => { if (config === undefined) { - return getApplicationConfig(["cartesi.toml"]); + return loadConfig({ command }); } if (typeof config === "string") { - return getApplicationConfig([config]); + return loadConfig({ command, files: [config] }); } if (Array.isArray(config)) { - return getApplicationConfig(config); + return loadConfig({ command, files: config }); } - return config; + return normalizeConfig(config); }; /** diff --git a/apps/cli/src/base.ts b/apps/cli/src/base.ts index 256408114..88cba9271 100644 --- a/apps/cli/src/base.ts +++ b/apps/cli/src/base.ts @@ -2,6 +2,7 @@ import { InvalidArgumentError } from "@commander-js/extra-typings"; import { execa } from "execa"; import fs from "node:fs"; import path from "node:path"; +import { loadConfig } from "./config/load.js"; import { type Address, getAddress, @@ -11,7 +12,6 @@ import { zeroHash, } from "viem"; import { foundry } from "viem/chains"; -import { type Config, parse } from "./config.js"; import { applicationFactoryAddress, authorityFactoryAddress, @@ -44,7 +44,7 @@ export const getContextPath = (...paths: string[]): string => { * SDK image of the project, which built its machine snapshot. An invalid cartesi.toml is * reported rather than falling back to the default image, whose emulator may not load it. */ -const getProjectSdk = (): string => getApplicationConfig(["cartesi.toml"]).sdk; +const getProjectSdk = async (): Promise => (await loadConfig()).sdk; /** * Read the hash of the cartesi machine snapshot, if one exists. Without a @@ -57,25 +57,12 @@ export const getMachineHash = async (options?: { }): Promise => { const imagePath = getContextPath("image"); if (fs.existsSync(imagePath)) { - const image = options?.sdk ?? getProjectSdk(); + const image = options?.sdk ?? (await getProjectSdk()); return await cartesiMachineStoredHash.computeHash(imagePath, { image }); } return undefined; }; -export const getApplicationConfig = (configPaths: string[]): Config => { - const tomls = configPaths.map((configPath) => { - if (fs.existsSync(configPath)) { - return fs.readFileSync(configPath).toString(); - } - if (configPath === "cartesi.toml") { - return ""; - } - throw new Error(`Config file ${configPath} does not exist`); - }); - return parse(tomls); -}; - export const getProjectName = (options: { projectName?: string }) => { return options.projectName ?? path.basename(process.cwd()); }; diff --git a/apps/cli/src/commands/build.ts b/apps/cli/src/commands/build.ts index d9728011d..c19af5305 100755 --- a/apps/cli/src/commands/build.ts +++ b/apps/cli/src/commands/build.ts @@ -7,11 +7,16 @@ export const createBuildCommand = () => { .description( "Build application by building Cartesi machine drives, configuring a machine and booting it.", ) - .option( - "-c, --config ", - "path to the configuration file", - (value, prev) => prev.concat([value]), - ["cartesi.toml"], + .addOption( + new Option( + "-c, --config ", + "path to the configuration file", + ) + .argParser((value, prev) => prev.concat([value])) + .default( + [] as string[], + "the configuration file of the project", + ), ) .addOption( new Option( diff --git a/apps/cli/src/commands/run.ts b/apps/cli/src/commands/run.ts index e310afc27..7cbae650f 100755 --- a/apps/cli/src/commands/run.ts +++ b/apps/cli/src/commands/run.ts @@ -7,7 +7,6 @@ import { build } from "../api/build.js"; import { logs } from "../api/logs.js"; import { run, type RunResult } from "../api/run.js"; import { nodeAllowedEnvironmentVariables } from "../compose/node.js"; -import { DEFAULT_SDK_VERSION } from "../config.js"; import { AVAILABLE_SERVICES, getDeployments, @@ -52,7 +51,7 @@ const reportApplicationStatus = (deployment: RollupsDeployment) => { const shell = async (options: { config: string[]; node: RunResult; - verbose: boolean; + verbose?: boolean; }) => { const { config, node, verbose } = options; const { projectName } = node; @@ -166,19 +165,12 @@ const shell = async (options: { export const createRunCommand = () => { return new Command("run") .description("Run a local cartesi node for the application.") - .addOption( - new Option( - "--prt", - "deploy application with PRT consensus", - ).default(false), - ) + .addOption(new Option("--prt", "deploy application with PRT consensus")) .addOption( new Option( "--block-time ", - "interval between blocks (in seconds)", - ) - .argParser(Number) - .default(2), + "interval between blocks (in seconds) (default: 2)", + ).argParser(Number), ) .addOption( new Option( @@ -189,10 +181,8 @@ export const createRunCommand = () => { .addOption( new Option( "--default-block ", - "default block to be used when fetching new blocks.", - ) - .choices(["latest", "safe", "pending", "finalized"]) - .default("latest"), + "default block to be used when fetching new blocks. (default: latest)", + ).choices(["latest", "safe", "pending", "finalized"]), ) .option("--dry-run", "show the docker compose configuration", false) .option( @@ -216,33 +206,32 @@ export const createRunCommand = () => { .addOption( new Option( "--epoch-length ", - "length of an epoch (in blocks)", - ) - .argParser(Number) - .default(720), + "length of an epoch (in blocks) (default: 720)", + ).argParser(Number), ) .option("-p, --port ", "port to listen on", Number) .addOption( new Option( "--claim-staging-period ", - "claim staging period (in blocks). Number of blocks between a claim being submitted and accepted", - ) - .argParser(Number) - .default(0), + "claim staging period (in blocks). Number of blocks between a claim being submitted and accepted (default: 0)", + ).argParser(Number), ) - .option( - "-c, --config ", - "Path to the configuration file (.toml)", - (value, prev) => prev.concat([value]), - ["cartesi.toml"], + .addOption( + new Option( + "-c, --config ", + "path to the configuration file", + ) + .argParser((value, prev) => prev.concat([value])) + .default( + [] as string[], + "the configuration file of the project", + ), ) .addOption( new Option( "--runtime-version ", "version for Cartesi Rollups Runtime to use", - ) - .default(DEFAULT_SDK_VERSION) - .hideHelp(), + ).hideHelp(), ) .option( "--project-name ", @@ -252,9 +241,8 @@ export const createRunCommand = () => { "--services ", `optional services to start, comma separated list from [${AVAILABLE_SERVICES.join(", ")}]`, commaSeparatedList, - [], ) - .option("-v, --verbose", "verbose output", false) + .option("-v, --verbose", "verbose output") .action(async (options) => { const { prt, @@ -288,14 +276,6 @@ export const createRunCommand = () => { return; } - if (defaultBlock !== "finalized") { - console.warn( - chalk.yellow( - `WARNING: default block is set to '${defaultBlock}', production configuration will likely use 'finalized'`, - ), - ); - } - // if TTY is not attached, run on foreground (not detached) const detach = !!process.stdin.isTTY; diff --git a/apps/cli/src/commands/shell.ts b/apps/cli/src/commands/shell.ts index 7131971b6..81ac94598 100755 --- a/apps/cli/src/commands/shell.ts +++ b/apps/cli/src/commands/shell.ts @@ -1,14 +1,19 @@ -import { Command } from "@commander-js/extra-typings"; +import { Command, Option } from "@commander-js/extra-typings"; import { shell } from "../api/shell.js"; export const createShellCommand = () => { return new Command("shell") .option("--command ", "shell command to run", "/bin/sh") - .option( - "-c, --config ", - "path to the configuration file", - (value, prev) => prev.concat([value]), - ["cartesi.toml"], + .addOption( + new Option( + "-c, --config ", + "path to the configuration file", + ) + .argParser((value, prev) => prev.concat([value])) + .default( + [] as string[], + "the configuration file of the project", + ), ) .option("--run-as-root", "run as root user", false) .action(async (options) => { diff --git a/apps/cli/src/config.ts b/apps/cli/src/config.ts index ad0795837..618abccf1 100644 --- a/apps/cli/src/config.ts +++ b/apps/cli/src/config.ts @@ -1,14 +1,15 @@ -import { parse as parseToml, type TomlTable } from "smol-toml"; -import { assertAccountsDrive } from "./config/accounts.js"; -import { DEFAULT_SDK_IMAGE, DEFAULT_SDK_VERSION } from "./config/constants.js"; -import { DuplicateLabelError } from "./config/errors.js"; -import { parseDrives } from "./config/toml/drives.js"; -import { parseMachine } from "./config/toml/machine.js"; -import { parseNvrams } from "./config/toml/nvrams.js"; -import { assertKnownKeys, mergeTomlTables } from "./config/toml/tables.js"; -import { parseString } from "./config/toml/values.js"; -import { parseOptionalWithdrawal } from "./config/toml/accounts.js"; -import type { Config } from "./config/types.js"; +/** + * Configuration of a Cartesi application. + * + * An application is configured by a `cartesi.config.ts` file that exports a + * configuration built with `defineConfig`, which gives it type checking and + * editor completion. Applications not written in TypeScript or JavaScript + * describe the same configuration as plain data, in a `cartesi.config.json`, + * `cartesi.config.yaml` or `cartesi.config` file. + * + * The deprecated `cartesi.toml` file is still read when no other configuration + * file is present. + */ export { type AccountsDrive, @@ -20,6 +21,7 @@ export { export { DEFAULT_SDK_IMAGE, DEFAULT_SDK_VERSION, + MAX_NVRAMS, NVRAM_ALIGNMENT, PREFERRED_PORT, } from "./config/constants.js"; @@ -27,49 +29,40 @@ export { defaultConfig, defaultMachineConfig, defaultRootDriveConfig, + defaultRunConfig, } from "./config/defaults.js"; export * from "./config/errors.js"; -export { splitSize } from "./config/size.js"; +export { + CONFIG_FILES, + findConfigFile, + LEGACY_CONFIG_FILE, + loadConfig, + type LoadConfigOptions, + loadConfigFile, +} from "./config/load.js"; +export { mergeConfig } from "./config/merge.js"; +export { normalizeConfig } from "./config/normalize.js"; +export { parseBytes, parseSize, splitSize } from "./config/size.js"; export { getDriveFormat } from "./config/toml/drives.js"; +export { parse } from "./config/toml/parse.js"; +export { DEFAULT_BLOCKS } from "./config/types.js"; export type * from "./config/types.js"; - -const CONFIG_KEYS: readonly string[] = [ - "drives", - "machine", - "nvrams", - "sdk", - "withdrawal", -]; - -export const parse = (str: string[]): Config => { - let toml: TomlTable = {}; - for (const s of str) { - toml = mergeTomlTables(toml, parseToml(s)); - } - assertKnownKeys(toml, CONFIG_KEYS, "at the top level"); - - const drives = parseDrives(toml.drives); - const nvrams = parseNvrams(toml.nvrams); - - // drives and nvrams share the DTB /aliases namespace, so labels cannot collide - for (const label of Object.keys(nvrams)) { - if (drives[label] !== undefined) { - throw new DuplicateLabelError(label); - } - } - - const config: Config = { - withdrawal: parseOptionalWithdrawal(toml.withdrawal), - drives, - machine: parseMachine(toml.machine), - nvrams, - sdk: parseString( - toml.sdk, - `${DEFAULT_SDK_IMAGE}:${DEFAULT_SDK_VERSION}`, - ), - }; - - assertAccountsDrive(config); - - return config; -}; +export { + type ConfigCommand, + type ConfigEnv, + defineConfig, + type Size, + type UserAccountsDriveMarker, + type UserConfig, + type UserConfigExport, + type UserConfigFn, + type UserDirectoryDriveConfig, + type UserDockerDriveConfig, + type UserDriveConfig, + type UserEmptyDriveConfig, + type UserExistingDriveConfig, + type UserMachineConfig, + type UserNvramConfig, + type UserTarDriveConfig, + type UserWithdrawalOptions, +} from "./config/user.js"; diff --git a/apps/cli/src/config/defaults.ts b/apps/cli/src/config/defaults.ts index 67dd95b78..440b5e9af 100644 --- a/apps/cli/src/config/defaults.ts +++ b/apps/cli/src/config/defaults.ts @@ -4,7 +4,7 @@ import { DEFAULT_SDK_IMAGE, DEFAULT_SDK_VERSION, } from "./constants.js"; -import type { Config, DriveConfig, MachineConfig } from "./types.js"; +import type { Config, DriveConfig, MachineConfig, RunConfig } from "./types.js"; export const defaultRootDriveConfig = (): DriveConfig => ({ builder: "docker", @@ -29,10 +29,13 @@ export const defaultMachineConfig = (): MachineConfig => ({ user: undefined, }); +export const defaultRunConfig = (): RunConfig => ({}); + export const defaultConfig = (): Config => ({ drives: { root: defaultRootDriveConfig() }, machine: defaultMachineConfig(), nvrams: {}, + run: defaultRunConfig(), sdk: `${DEFAULT_SDK_IMAGE}:${DEFAULT_SDK_VERSION}`, withdrawal: undefined, }); diff --git a/apps/cli/src/config/errors.ts b/apps/cli/src/config/errors.ts index 46ae03465..3c3fba5ae 100644 --- a/apps/cli/src/config/errors.ts +++ b/apps/cli/src/config/errors.ts @@ -1,9 +1,8 @@ -import type { TomlValue } from "smol-toml"; import { MAX_NVRAMS, NVRAM_ALIGNMENT } from "./constants.js"; // smol-toml creates tables with a null prototype, which can't be coerced to a // string by template literals, so format objects (tables, arrays) as JSON. -export const formatValue = (value: TomlValue): string => +export const formatValue = (value: unknown): string => typeof value === "object" && value !== null && !(value instanceof Date) ? JSON.stringify(value, (_, v) => typeof v === "bigint" ? v.toString() : v, @@ -11,42 +10,42 @@ export const formatValue = (value: TomlValue): string => : String(value); export class InvalidBuilderError extends Error { - constructor(builder: TomlValue) { + constructor(builder: unknown) { super(`Invalid builder: ${formatValue(builder)}`); this.name = "InvalidBuilder"; } } export class InvalidDriveFormatError extends Error { - constructor(format: TomlValue) { + constructor(format: unknown) { super(`Invalid drive format: ${formatValue(format)}`); this.name = "InvalidDriveFormatError"; } } export class InvalidEmptyDriveFormatError extends Error { - constructor(format: TomlValue) { + constructor(format: unknown) { super(`Invalid empty drive format: ${formatValue(format)}`); this.name = "InvalidEmptyDriveFormatError"; } } export class InvalidStringValueError extends Error { - constructor(value: TomlValue) { + constructor(value: unknown) { super(`Invalid string value: ${formatValue(value)}`); this.name = "InvalidStringValueError"; } } export class InvalidBooleanValueError extends Error { - constructor(value: TomlValue) { + constructor(value: unknown) { super(`Invalid boolean value: ${formatValue(value)}`); this.name = "InvalidBooleanValueError"; } } export class InvalidNumberValueError extends Error { - constructor(value: TomlValue, key?: string) { + constructor(value: unknown, key?: string) { super( `Invalid number value: ${formatValue(value)}${key ? ` for key: ${key}` : ""}`, ); @@ -55,7 +54,7 @@ export class InvalidNumberValueError extends Error { } export class InvalidAddressValueError extends Error { - constructor(value: TomlValue, key?: string) { + constructor(value: unknown, key?: string) { super( `Invalid address value: ${formatValue(value)}${key ? ` for key: ${key}` : ""}`, ); @@ -64,14 +63,14 @@ export class InvalidAddressValueError extends Error { } export class InvalidBytesValueError extends Error { - constructor(value: TomlValue) { + constructor(value: unknown) { super(`Invalid bytes value: ${formatValue(value)}`); this.name = "InvalidBytesValueError"; } } export class RequiredFieldError extends Error { - constructor(key: TomlValue) { + constructor(key: unknown) { super(`Missing required field: ${formatValue(key)}`); this.name = "RequiredFieldError"; } @@ -85,7 +84,7 @@ export class InvalidStringArrayError extends Error { } export class InvalidEnvError extends Error { - constructor(value: TomlValue) { + constructor(value: unknown) { super(`Invalid env configuration: ${formatValue(value)}`); this.name = "InvalidEnvError"; } @@ -158,10 +157,70 @@ export class UnknownConfigKeyError extends Error { } export class InvalidTableError extends Error { - constructor(location: string, value: TomlValue) { + constructor(location: string, value: unknown) { super( `Invalid ${location}: expected a table, got ${formatValue(value)}`, ); this.name = "InvalidTableError"; } } + +/** + * Thrown when a field only accepts a fixed set of values, and is given + * something else. + */ +export class InvalidEnumValueError extends Error { + constructor(key: string, value: unknown, allowed: readonly string[]) { + super( + `Invalid value for '${key}': ${formatValue(value)}, must be one of ${allowed.join(", ")}`, + ); + this.name = "InvalidEnumValueError"; + } +} + +/** + * Thrown when a configuration file explicitly asked for does not exist. + */ +export class ConfigFileNotFoundError extends Error { + constructor(filename: string) { + super(`Config file ${filename} does not exist`); + this.name = "ConfigFileNotFoundError"; + } +} + +/** + * Thrown when a configuration file has an extension the CLI cannot read. + */ +export class UnsupportedConfigFormatError extends Error { + constructor(filename: string) { + super(`Unsupported configuration file format: ${filename}`); + this.name = "UnsupportedConfigFormatError"; + } +} + +/** + * Thrown when a TypeScript or JavaScript configuration file does not export a + * configuration as its default export. + */ +export class InvalidConfigExportError extends Error { + constructor(filename: string) { + super( + `Configuration file ${filename} must have a default export with the application configuration, ` + + "as in 'export default defineConfig({ ... })'", + ); + this.name = "InvalidConfigExportError"; + } +} + +/** + * Thrown when the runtime cannot import a TypeScript configuration file. + */ +export class TypeScriptNotSupportedError extends Error { + constructor(filename: string) { + super( + `Cannot import ${filename}: this runtime does not strip TypeScript types. ` + + "Run on node 24 or later without '--no-experimental-strip-types', or write the configuration as cartesi.config.js, .json or .yaml", + ); + this.name = "TypeScriptNotSupportedError"; + } +} diff --git a/apps/cli/src/config/load.ts b/apps/cli/src/config/load.ts new file mode 100644 index 000000000..e70ebebba --- /dev/null +++ b/apps/cli/src/config/load.ts @@ -0,0 +1,258 @@ +import chalk from "chalk"; +import fs from "node:fs"; +import path from "node:path"; +import { pathToFileURL } from "node:url"; +import { parse as parseYaml } from "yaml"; +import { + ConfigFileNotFoundError, + InvalidConfigExportError, + TypeScriptNotSupportedError, + UnsupportedConfigFormatError, +} from "./errors.js"; +import { mergeConfig } from "./merge.js"; +import { normalizeConfig } from "./normalize.js"; +import { parse as parseTomlConfig } from "./toml/parse.js"; +import type { Config } from "./types.js"; +import type { + ConfigCommand, + ConfigEnv, + UserConfig, + UserConfigFn, +} from "./user.js"; + +/** + * Names of the configuration files looked up in the project directory, in + * order of precedence. The first one that exists is the configuration of the + * application. + * + * TypeScript and JavaScript files export their configuration with + * `defineConfig`. The remaining formats are for applications not written in + * TypeScript or JavaScript, and describe the very same configuration as plain + * data. A file with no format in its name (`cartesi.config`) is read as YAML, + * which also makes it valid JSON. + */ +export const CONFIG_FILES = [ + "cartesi.config.ts", + "cartesi.config.mts", + "cartesi.config.cts", + "cartesi.config.js", + "cartesi.config.mjs", + "cartesi.config.cjs", + "cartesi.config.json", + "cartesi.config.yaml", + "cartesi.config.yml", + "cartesi.config", + "cartesi.toml", // deprecated +] as const; + +/** Name of the deprecated configuration file. */ +export const LEGACY_CONFIG_FILE = "cartesi.toml"; + +const MODULE_EXTENSIONS = [".ts", ".mts", ".cts", ".js", ".mjs", ".cjs"]; +const TYPESCRIPT_EXTENSIONS = [".ts", ".mts", ".cts"]; + +const isLegacyConfigFile = (filename: string): boolean => + path.extname(filename) === ".toml"; + +let legacyWarned = false; + +/** + * Warn, once per process, that the legacy configuration file is deprecated. + */ +const warnLegacyConfig = (filename: string): void => { + if (legacyWarned) { + return; + } + legacyWarned = true; + console.warn( + chalk.yellow( + `WARNING: the TOML configuration format (${path.basename(filename)}) is deprecated, migrate to cartesi.config.ts, or to cartesi.config.yaml for applications not written in TypeScript`, + ), + ); +}; + +/** + * Find the configuration file of the application. + * @param cwd directory to look the configuration file up at + * @returns absolute path of the configuration file, or `undefined` if the + * application has no configuration file, in which case the defaults apply + */ +export const findConfigFile = ( + cwd: string = process.cwd(), +): string | undefined => { + for (const name of CONFIG_FILES) { + const filename = path.resolve(cwd, name); + if (fs.existsSync(filename)) { + return filename; + } + } + return undefined; +}; + +/** + * Whether the runtime can import a TypeScript file directly. Bun always can, + * and so does node, which this package requires at version 24 or later, unless + * type stripping was explicitly turned off. + */ +const canImportTypeScript = (): boolean => + process.versions.bun !== undefined || Boolean(process.features.typescript); + +/** + * Import a TypeScript or JavaScript configuration file and return its default + * export. The file is imported by the runtime itself, so there is no build + * step: the standalone binaries run on Bun, and node strips the types. + */ +const importConfigFile = async (filename: string): Promise => { + if ( + TYPESCRIPT_EXTENSIONS.includes(path.extname(filename)) && + !canImportTypeScript() + ) { + throw new TypeScriptNotSupportedError(filename); + } + + // the query string busts the module cache, so a configuration file edited + // between two builds of the same process is picked up + const url = `${pathToFileURL(filename).href}?t=${Date.now()}`; + const module: unknown = await import(url); + + // a CommonJS file assigning to 'module.exports' also lands on 'default' + return (module as { default?: unknown })?.default; +}; + +/** + * Read one configuration file, of any of the supported formats, without + * applying any default. + * @param filename path of the configuration file + * @param env context given to a configuration file that exports a function + * @returns configuration as written in the file + */ +export const loadConfigFile = async ( + filename: string, + env: ConfigEnv, +): Promise => { + const extension = path.extname(filename); + + if (MODULE_EXTENSIONS.includes(extension)) { + const exported = await importConfigFile(filename); + if (exported === undefined || exported === null) { + throw new InvalidConfigExportError(filename); + } + + // a configuration file may export the configuration itself, a promise + // of one, or a function computing it from the environment + const config = + typeof exported === "function" + ? await (exported as UserConfigFn)(env) + : await exported; + + if (typeof config !== "object" || config === null) { + throw new InvalidConfigExportError(filename); + } + return config as UserConfig; + } + + const contents = fs.readFileSync(filename, "utf8"); + + switch (extension) { + case ".json": + return (JSON.parse(contents) ?? {}) as UserConfig; + case ".yaml": + case ".yml": + // a file with no format in its name, such as 'cartesi.config', is read + // as YAML, which accepts JSON as well + case ".config": + return (parseYaml(contents) ?? {}) as UserConfig; + case ".toml": + warnLegacyConfig(filename); + return parseTomlConfig([contents]); + default: + throw new UnsupportedConfigFormatError(filename); + } +}; + +export type LoadConfigOptions = { + /** + * Command being run, given to a configuration file that exports a + * function. + * @default "build" + */ + command?: ConfigCommand; + + /** + * Directory the configuration is resolved from. + * @default process.cwd() + */ + cwd?: string; + + /** + * Configuration files to read, merged in order. When empty, the + * configuration file of the project is looked up by name. + */ + files?: string[]; + + /** + * Mode the application is being built or run in. + * @default `CARTESI_ENV`, `NODE_ENV`, or "development" + */ + mode?: string; +}; + +/** + * Load the configuration of an application. + * + * Unless configuration files are explicitly given, the project directory is + * searched for one of the {@link CONFIG_FILES}. An application without a + * configuration file is perfectly valid, and gets the default configuration. + * + * @param options where to load the configuration from + * @returns resolved application configuration + */ +export const loadConfig = async ( + options: LoadConfigOptions = {}, +): Promise => { + const cwd = options.cwd ?? process.cwd(); + const env: ConfigEnv = { + command: options.command ?? "build", + cwd, + mode: + options.mode ?? + process.env.CARTESI_ENV ?? + process.env.NODE_ENV ?? + "development", + }; + + let files: string[]; + if (options.files?.length) { + files = options.files.map((file) => path.resolve(cwd, file)); + for (const file of files) { + if (!fs.existsSync(file)) { + throw new ConfigFileNotFoundError(file); + } + } + } else { + const found = findConfigFile(cwd); + files = found ? [found] : []; + } + + if (files.length === 0) { + // an application without a configuration file uses the defaults + return normalizeConfig(undefined); + } + + if (files.every(isLegacyConfigFile)) { + // legacy files are merged as TOML tables before being parsed, which is + // how the deprecated format has always behaved + for (const file of files) { + warnLegacyConfig(file); + } + return parseTomlConfig( + files.map((file) => fs.readFileSync(file, "utf8")), + ); + } + + let config: UserConfig = {}; + for (const file of files) { + config = mergeConfig(config, await loadConfigFile(file, env)); + } + return normalizeConfig(config); +}; diff --git a/apps/cli/src/config/merge.ts b/apps/cli/src/config/merge.ts new file mode 100644 index 000000000..8007fe852 --- /dev/null +++ b/apps/cli/src/config/merge.ts @@ -0,0 +1,63 @@ +import type { UserConfig } from "./user.js"; + +/** + * Checks if a value is a plain object, and not an array, a class instance or + * any other exotic object. + */ +const isPlainObject = (value: unknown): value is Record => { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + return false; + } + const prototype = Object.getPrototypeOf(value); + return prototype === Object.prototype || prototype === null; +}; + +const merge = ( + base: Record, + other: Record, +): Record => { + const result: Record = { ...base }; + + for (const [key, otherValue] of Object.entries(other)) { + if (otherValue === undefined) { + // an undefined value does not override anything, so a partial + // configuration can be built with optional fields left out + continue; + } + + const baseValue = result[key]; + if (isPlainObject(baseValue) && isPlainObject(otherValue)) { + result[key] = merge(baseValue, otherValue); + } else { + // arrays and scalars are replaced, not concatenated + result[key] = otherValue; + } + } + + return result; +}; + +/** + * Deep merge two configurations, with the values of `other` taking precedence + * over the ones of `base`. Objects are merged recursively, while arrays and + * scalars are replaced. Useful to share a base configuration between + * applications, or to layer environment specific overrides: + * + * ```ts + * export default defineConfig( + * mergeConfig(base, { machine: { ramLength: "256Mi" } }), + * ); + * ``` + * + * @param base base configuration + * @param other configuration merged on top of `base` + * @returns a new merged configuration + */ +export const mergeConfig = ( + base: T, + other: UserConfig, +): T => + merge( + base as Record, + other as Record, + ) as T; diff --git a/apps/cli/src/config/normalize.ts b/apps/cli/src/config/normalize.ts new file mode 100644 index 000000000..2a132a716 --- /dev/null +++ b/apps/cli/src/config/normalize.ts @@ -0,0 +1,645 @@ +import { type Address, getAddress, isAddress } from "viem"; +import { + assertAccountsDrive, + LOG2_LEAF_SIZE, + log2OfPowerOfTwo, +} from "./accounts.js"; +import { + DEFAULT_FORMAT, + DEFAULT_RAM, + DEFAULT_SDK_IMAGE, + DEFAULT_SDK_VERSION, + MAX_NVRAMS, + NVRAM_ALIGNMENT, +} from "./constants.js"; +import { defaultRootDriveConfig } from "./defaults.js"; +import { + DuplicateLabelError, + formatValue, + InvalidAddressValueError, + InvalidBooleanValueError, + InvalidBuilderError, + InvalidDriveFormatError, + InvalidEmptyDriveFormatError, + InvalidEnumValueError, + InvalidEnvError, + InvalidNumberValueError, + InvalidNvramSizeError, + InvalidStringArrayError, + InvalidStringValueError, + InvalidTableError, + InvalidWithdrawalConfigError, + MissingNvramSourceError, + RequiredFieldError, + TooManyNvramsError, + UnknownConfigKeyError, +} from "./errors.js"; +import { parseBytes, parseSize } from "./size.js"; +import { getDriveFormat } from "./toml/drives.js"; +import { + type AccountsDriveMarker, + type Builder, + type Config, + DEFAULT_BLOCKS, + type DefaultBlock, + type DriveConfig, + type DriveFormat, + type MachineConfig, + type NvramConfig, + type RunConfig, + type WithdrawalOptions, +} from "./types.js"; + +/** + * Validation and defaulting of a configuration written by a developer, for + * every format but the deprecated TOML one: a `cartesi.config.ts` file through + * `defineConfig`, or the same configuration as plain data. + * + * A `cartesi.config.ts` file is type checked where it is written, but a JSON or + * YAML one is not, so every value is checked here, and, as the TOML parser + * does, a key no section reads is an error rather than a typo that does + * nothing. + */ + +const isRecord = (value: unknown): value is Record => + typeof value === "object" && value !== null && !Array.isArray(value); + +const asTable = ( + where: string, + value: unknown, + known: readonly string[], +): Record => { + if (value === undefined || value === null) { + return {}; + } + if (!isRecord(value)) { + throw new InvalidTableError(where, value); + } + const unknown = Object.keys(value).filter((key) => !known.includes(key)); + if (unknown.length > 0) { + throw new UnknownConfigKeyError(unknown, known, `in ${where}`); + } + return value; +}; + +const asBoolean = (value: unknown, defaultValue: boolean): boolean => { + if (value === undefined) { + return defaultValue; + } + if (typeof value === "boolean") { + return value; + } + throw new InvalidBooleanValueError(value); +}; + +const asOptionalBoolean = (value: unknown): boolean | undefined => { + if (value === undefined) { + return undefined; + } + if (typeof value === "boolean") { + return value; + } + throw new InvalidBooleanValueError(value); +}; + +const asString = (value: unknown, defaultValue: string): string => { + if (value === undefined) { + return defaultValue; + } + if (typeof value === "string") { + return value; + } + throw new InvalidStringValueError(value); +}; + +const asOptionalString = (value: unknown): string | undefined => { + if (value === undefined) { + return undefined; + } + if (typeof value === "string") { + return value; + } + throw new InvalidStringValueError(value); +}; + +const asRequiredString = (value: unknown, key: string): string => { + if (value === undefined) { + throw new RequiredFieldError(key); + } + if (typeof value === "string") { + return value; + } + throw new InvalidStringValueError(value); +}; + +const asOptionalStringBoolean = ( + value: unknown, +): string | boolean | undefined => { + if (value === undefined) { + return undefined; + } + if (typeof value === "string" || typeof value === "boolean") { + return value; + } + throw new InvalidStringValueError(value); +}; + +const asStringArray = (value: unknown): string[] => { + if (value === undefined) { + return []; + } + if (typeof value === "string") { + return [value]; + } + if (Array.isArray(value)) { + return value.map((v) => { + if (typeof v === "string") { + return v; + } + throw new InvalidStringValueError(v); + }); + } + throw new InvalidStringArrayError(); +}; + +const asOptionalStringArray = (value: unknown): string[] | undefined => + value === undefined ? undefined : asStringArray(value); + +/** + * Coerce a table of environment variables into a record of strings, since the + * value of an environment variable is always a string. + */ +const asStringRecord = (value: unknown): Record => { + if (value === undefined) { + return {}; + } + if (!isRecord(value)) { + throw new InvalidEnvError(value); + } + return Object.entries(value).reduce>( + (acc, [key, val]) => { + if (typeof val === "string") { + acc[key] = val; + } else if ( + typeof val === "number" || + typeof val === "bigint" || + typeof val === "boolean" + ) { + acc[key] = String(val); + } else { + throw new InvalidStringValueError(val); + } + return acc; + }, + {}, + ); +}; + +const asOptionalNumber = (value: unknown, key?: string): number | undefined => { + if (value === undefined) { + return undefined; + } + if (typeof value === "number" && Number.isFinite(value)) { + return value; + } + if (typeof value === "bigint") { + return Number(value); + } + throw new InvalidNumberValueError(value, key); +}; + +const asOptionalBigInt = (value: unknown): bigint | undefined => { + if (value === undefined) { + return undefined; + } + if (typeof value === "bigint") { + return value; + } + if (typeof value === "number" && Number.isInteger(value)) { + return BigInt(value); + } + throw new InvalidNumberValueError(value); +}; + +const asOptionalAddress = ( + value: unknown, + key: string, +): Address | undefined => { + if (value === undefined) { + return undefined; + } + if (typeof value === "string" && isAddress(value)) { + return getAddress(value); + } + throw new InvalidAddressValueError(value, key); +}; + +const asBuilder = (value: unknown): Builder => { + if (value === undefined) { + return "docker"; + } + if (typeof value === "string") { + switch (value) { + case "directory": + case "docker": + case "empty": + case "none": + case "tar": + return value; + } + } + throw new InvalidBuilderError(value); +}; + +const asFormat = (value: unknown): DriveFormat => { + if (value === undefined) { + return DEFAULT_FORMAT; + } + if (value === "ext2" || value === "sqfs") { + return value; + } + throw new InvalidDriveFormatError(value); +}; + +const asEmptyFormat = (value: unknown): "ext2" | "raw" => { + if (value === undefined) { + return DEFAULT_FORMAT; + } + if (value === "ext2" || value === "raw") { + return value; + } + throw new InvalidEmptyDriveFormatError(value); +}; + +const ACCOUNTS_DRIVE_KEYS: readonly string[] = [ + "accountsDrive", + "accountsDriveSize", +]; + +const DRIVE_KEYS: readonly string[] = [ + ...ACCOUNTS_DRIVE_KEYS, + "builder", + "mount", + "shared", + "user", +]; + +/** The keys each builder reads, on top of DRIVE_KEYS */ +const BUILDER_KEYS: Record = { + directory: ["directory", "extraSize", "format"], + docker: [ + "buildArgs", + "context", + "dockerfile", + "extraSize", + "format", + "image", + "tags", + "target", + ], + empty: ["format", "size"], + none: ["filename", "format"], + tar: ["extraSize", "filename", "format"], +}; + +const NVRAM_KEYS: readonly string[] = [ + ...ACCOUNTS_DRIVE_KEYS, + "filename", + "shared", + "size", + "user", +]; + +const MACHINE_KEYS: readonly string[] = [ + "assertRollingTemplate", + "bootargs", + "entrypoint", + "env", + "envFile", + "maxMCycle", + "ramImage", + "ramLength", + "useDockerEnv", + "useDockerWorkdir", + "user", +]; + +const RUN_KEYS: readonly string[] = [ + "blockTime", + "claimStagingPeriod", + "cpus", + "defaultBlock", + "epochLength", + "forkBlockNumber", + "forkUrl", + "memory", + "port", + "projectName", + "prt", + "runtimeVersion", + "services", + "verbose", +]; + +const WITHDRAWAL_KEYS: readonly string[] = [ + "account_size", + "guardian", + "withdrawal_output_builder", +]; + +// '$schema' is not configuration, it points a JSON or YAML editor at the schema +const CONFIG_KEYS: readonly string[] = [ + "$schema", + "drives", + "machine", + "nvrams", + "run", + "sdk", + "withdrawal", +]; + +/** + * Reads the keys that mark a drive or nvram as the accounts drive, leaving them + * out when unset, so they do not show up as an explicit `undefined`. + */ +const normalizeAccountsDriveMarker = ( + drive: Record, +): AccountsDriveMarker => { + const accountsDrive = asOptionalBoolean(drive.accountsDrive); + const accountsDriveSize = parseSize(drive.accountsDriveSize); + return { + ...(accountsDrive !== undefined && { accountsDrive }), + ...(accountsDriveSize !== undefined && { accountsDriveSize }), + }; +}; + +const normalizeDrive = (name: string, value: unknown): DriveConfig => { + if (value !== undefined && !isRecord(value)) { + throw new InvalidTableError(`drives.${name}`, value); + } + const builder = asBuilder((value as Record)?.builder); + const implicit = + (value as Record)?.builder === undefined + ? ", by default" + : ""; + const drive = asTable( + `drives.${name} (builder "${builder}"${implicit})`, + value, + [...DRIVE_KEYS, ...BUILDER_KEYS[builder]], + ); + + const marker = normalizeAccountsDriveMarker(drive); + const mount = asOptionalStringBoolean(drive.mount); + const shared = asOptionalBoolean(drive.shared); + const user = asOptionalString(drive.user); + + switch (builder) { + case "directory": + return { + ...marker, + builder: "directory", + directory: asRequiredString(drive.directory, "directory"), + extraSize: parseBytes(drive.extraSize, 0), + format: asFormat(drive.format), + mount, + shared, + user, + }; + case "docker": + return { + ...marker, + builder: "docker", + buildArgs: asStringArray(drive.buildArgs), + context: asString(drive.context, "."), + dockerfile: asString(drive.dockerfile, "Dockerfile"), + extraSize: parseBytes(drive.extraSize, 0), + format: asFormat(drive.format), + image: asOptionalString(drive.image), + mount, + shared, + tags: asStringArray(drive.tags), + target: asOptionalString(drive.target), + user, + }; + case "empty": + return { + ...marker, + builder: "empty", + format: asEmptyFormat(drive.format), + mount, + shared, + size: parseBytes(drive.size, 0), + user, + }; + case "tar": + return { + ...marker, + builder: "tar", + extraSize: parseBytes(drive.extraSize, 0), + filename: asRequiredString(drive.filename, "filename"), + format: asFormat(drive.format), + mount, + shared, + user, + }; + case "none": { + const filename = asRequiredString(drive.filename, "filename"); + return { + ...marker, + builder: "none", + filename, + // the format of an existing drive comes from its extension, + // unless it is explicitly given + format: + drive.format === undefined + ? getDriveFormat(filename) + : asFormat(drive.format), + mount, + shared, + user, + }; + } + } +}; + +const normalizeDrives = (value: unknown): Record => { + if (value !== undefined && !isRecord(value)) { + throw new InvalidTableError("drives", value); + } + const drives = Object.entries( + (value as Record) ?? {}, + ).reduce>((acc, [name, drive]) => { + acc[name] = normalizeDrive(name, drive); + return acc; + }, {}); + + if (drives.root === undefined) { + // every machine needs a root drive, add a default one + drives.root = defaultRootDriveConfig(); + } + return drives; +}; + +const normalizeNvram = (label: string, value: unknown): NvramConfig => { + const nvram = asTable(`nvrams.${label}`, value, NVRAM_KEYS); + const size = parseSize(nvram.size); + const filename = asOptionalString(nvram.filename); + + if (size === undefined && filename === undefined) { + throw new MissingNvramSourceError(label); + } + if (size !== undefined && (size <= 0 || size % NVRAM_ALIGNMENT !== 0)) { + throw new InvalidNvramSizeError(label, size); + } + + return { + ...normalizeAccountsDriveMarker(nvram), + filename, + size, + shared: asOptionalBoolean(nvram.shared), + user: asOptionalString(nvram.user), + }; +}; + +const normalizeNvrams = (value: unknown): Record => { + if (value !== undefined && !isRecord(value)) { + throw new InvalidTableError("nvrams", value); + } + const entries = Object.entries((value as Record) ?? {}); + if (entries.length > MAX_NVRAMS) { + throw new TooManyNvramsError(entries.length); + } + return entries.reduce>( + (acc, [label, nvram]) => { + acc[label] = normalizeNvram(label, nvram); + return acc; + }, + {}, + ); +}; + +const normalizeMachine = (value: unknown): MachineConfig => { + const machine = asTable("machine", value, MACHINE_KEYS); + return { + assertRollingTemplate: asOptionalBoolean(machine.assertRollingTemplate), + bootargs: asStringArray(machine.bootargs), + entrypoint: asOptionalString(machine.entrypoint), + env: asStringRecord(machine.env), + envFile: asOptionalString(machine.envFile), + maxMCycle: asOptionalBigInt(machine.maxMCycle), + ramLength: asString(machine.ramLength, DEFAULT_RAM), + ramImage: asOptionalString(machine.ramImage), + useDockerEnv: asBoolean(machine.useDockerEnv, true), + useDockerWorkdir: asBoolean(machine.useDockerWorkdir, true), + user: asOptionalString(machine.user), + }; +}; + +const asDefaultBlock = (value: unknown): DefaultBlock | undefined => { + if (value === undefined) { + return undefined; + } + if (DEFAULT_BLOCKS.includes(value as DefaultBlock)) { + return value as DefaultBlock; + } + throw new InvalidEnumValueError("defaultBlock", value, DEFAULT_BLOCKS); +}; + +const normalizeRun = (value: unknown): RunConfig => { + const run = asTable("run", value, RUN_KEYS); + return { + blockTime: asOptionalNumber(run.blockTime, "blockTime"), + claimStagingPeriod: asOptionalNumber( + run.claimStagingPeriod, + "claimStagingPeriod", + ), + cpus: asOptionalNumber(run.cpus, "cpus"), + defaultBlock: asDefaultBlock(run.defaultBlock), + epochLength: asOptionalNumber(run.epochLength, "epochLength"), + forkBlockNumber: asOptionalNumber( + run.forkBlockNumber, + "forkBlockNumber", + ), + forkUrl: asOptionalString(run.forkUrl), + memory: asOptionalNumber(run.memory, "memory"), + port: asOptionalNumber(run.port, "port"), + projectName: asOptionalString(run.projectName), + prt: asOptionalBoolean(run.prt), + runtimeVersion: asOptionalString(run.runtimeVersion), + services: asOptionalStringArray(run.services), + verbose: asOptionalBoolean(run.verbose), + }; +}; + +/** + * Checks `account_size`, which divides the accounts drive into accounts and so + * must be a power of two of at least one 32-byte leaf. + */ +const asAccountSize = (value: unknown): number => { + const size = parseSize(value); + if ( + size === undefined || + size < 2 ** LOG2_LEAF_SIZE || + log2OfPowerOfTwo(size) === undefined + ) { + throw new InvalidWithdrawalConfigError( + `account_size ${formatValue(value)} must be a power of two of at least 32 bytes, such as 32 or 64`, + ); + } + return size; +}; + +const normalizeWithdrawal = (value: unknown): WithdrawalOptions | undefined => { + if (value === undefined || value === null) { + return undefined; + } + const withdrawal = asTable("withdrawal", value, WITHDRAWAL_KEYS); + return { + guardian: asOptionalAddress(withdrawal.guardian, "guardian"), + withdrawal_output_builder: asOptionalAddress( + withdrawal.withdrawal_output_builder, + "withdrawal_output_builder", + ), + account_size: + withdrawal.account_size === undefined + ? undefined + : asAccountSize(withdrawal.account_size), + }; +}; + +/** + * Validate a configuration written by a developer and resolve it into the fully + * defaulted {@link Config} the rest of the CLI works with. + * + * An already resolved {@link Config} is a valid input, and normalizing it again + * yields an equivalent configuration. + * + * @param input configuration as written, or `undefined` for the defaults + * @returns resolved application configuration + */ +export const normalizeConfig = (input: unknown): Config => { + const table = asTable("the configuration", input, CONFIG_KEYS); + + const drives = normalizeDrives(table.drives); + const nvrams = normalizeNvrams(table.nvrams); + + // drives and nvrams share the DTB /aliases namespace, so labels cannot collide + for (const label of Object.keys(nvrams)) { + if (drives[label] !== undefined) { + throw new DuplicateLabelError(label); + } + } + + const config: Config = { + drives, + machine: normalizeMachine(table.machine), + nvrams, + run: normalizeRun(table.run), + sdk: asString(table.sdk, `${DEFAULT_SDK_IMAGE}:${DEFAULT_SDK_VERSION}`), + withdrawal: normalizeWithdrawal(table.withdrawal), + }; + + assertAccountsDrive(config); + + return config; +}; diff --git a/apps/cli/src/config/size.ts b/apps/cli/src/config/size.ts index 9f98c8ae0..1926969d0 100644 --- a/apps/cli/src/config/size.ts +++ b/apps/cli/src/config/size.ts @@ -1,3 +1,5 @@ +import { InvalidBytesValueError } from "./errors.js"; + /** * Splits a number of bytes into the largest IEC unit that divides it exactly, e.g. 4194304 is * 4 "Mi", and 1000 is 1000 "". @@ -22,3 +24,66 @@ export const formatSize = (bytes: number): string => { const { value, unit } = splitSize(bytes); return `${value}${unit}`; }; + +const IEC_MULTIPLIERS: Readonly> = { + "": 1, + b: 1, + k: 1024, + ki: 1024, + kb: 1024, + kib: 1024, + m: 1024 ** 2, + mi: 1024 ** 2, + mb: 1024 ** 2, + mib: 1024 ** 2, + g: 1024 ** 3, + gi: 1024 ** 3, + gb: 1024 ** 3, + gib: 1024 ** 3, + t: 1024 ** 4, + ti: 1024 ** 4, + tb: 1024 ** 4, + tib: 1024 ** 4, + p: 1024 ** 5, + pi: 1024 ** 5, + pb: 1024 ** 5, + pib: 1024 ** 5, +}; + +/** + * Parses a byte size of a drive or nvram, accepting the IEC suffixes used by cartesi-machine + * ("4Ki", "1MiB") and their decimal spellings ("4kb", "100Mb"), all as powers of 1024, up to + * petabytes. Sizes are numbers, so the ones beyond 2^53 bytes, which they can't hold exactly, + * are rejected. + */ +export const parseSize = (value: unknown): number | undefined => { + if (value === undefined) { + return undefined; + } + let size: number | undefined; + if (typeof value === "number") { + size = value; + } else if (typeof value === "bigint") { + size = Number(value); + } else if (typeof value === "string") { + const match = /^\s*(\d+(?:\.\d+)?)\s*([a-z]*)\s*$/i.exec(value); + const multiplier = match + ? IEC_MULTIPLIERS[match[2].toLowerCase()] + : undefined; + if (match && multiplier !== undefined) { + size = Number(match[1]) * multiplier; + } + } + // a whole number of bytes, e.g. "1.5Ki" is but "1.1Ki" isn't + if ( + size !== undefined && + Number.isInteger(size) && + size <= Number.MAX_SAFE_INTEGER + ) { + return size; + } + throw new InvalidBytesValueError(value); +}; + +export const parseBytes = (value: unknown, defaultValue: number): number => + parseSize(value) ?? defaultValue; diff --git a/apps/cli/src/config/toml/parse.ts b/apps/cli/src/config/toml/parse.ts new file mode 100644 index 000000000..1a0c31b37 --- /dev/null +++ b/apps/cli/src/config/toml/parse.ts @@ -0,0 +1,61 @@ +import { parse as parseToml, type TomlTable } from "smol-toml"; +import { assertAccountsDrive } from "../accounts.js"; +import { DEFAULT_SDK_IMAGE, DEFAULT_SDK_VERSION } from "../constants.js"; +import { defaultRunConfig } from "../defaults.js"; +import { DuplicateLabelError } from "../errors.js"; +import type { Config } from "../types.js"; +import { parseOptionalWithdrawal } from "./accounts.js"; +import { parseDrives } from "./drives.js"; +import { parseMachine } from "./machine.js"; +import { parseNvrams } from "./nvrams.js"; +import { assertKnownKeys, mergeTomlTables } from "./tables.js"; +import { parseString } from "./values.js"; + +/** + * Parser of the deprecated `cartesi.toml` configuration file, superseded by + * `cartesi.config.ts` and its plain data equivalents. It uses snake_case keys + * and has no `[run]` section, and is kept working so existing applications + * keep building. + */ + +const CONFIG_KEYS: readonly string[] = [ + "drives", + "machine", + "nvrams", + "sdk", + "withdrawal", +]; + +export const parse = (str: string[]): Config => { + let toml: TomlTable = {}; + for (const s of str) { + toml = mergeTomlTables(toml, parseToml(s)); + } + assertKnownKeys(toml, CONFIG_KEYS, "at the top level"); + + const drives = parseDrives(toml.drives); + const nvrams = parseNvrams(toml.nvrams); + + // drives and nvrams share the DTB /aliases namespace, so labels cannot collide + for (const label of Object.keys(nvrams)) { + if (drives[label] !== undefined) { + throw new DuplicateLabelError(label); + } + } + + const config: Config = { + withdrawal: parseOptionalWithdrawal(toml.withdrawal), + drives, + machine: parseMachine(toml.machine), + nvrams, + run: defaultRunConfig(), + sdk: parseString( + toml.sdk, + `${DEFAULT_SDK_IMAGE}:${DEFAULT_SDK_VERSION}`, + ), + }; + + assertAccountsDrive(config); + + return config; +}; diff --git a/apps/cli/src/config/toml/values.ts b/apps/cli/src/config/toml/values.ts index 101444a86..95065aa90 100644 --- a/apps/cli/src/config/toml/values.ts +++ b/apps/cli/src/config/toml/values.ts @@ -3,7 +3,6 @@ import { type Address, getAddress, isAddress } from "viem"; import { InvalidAddressValueError, InvalidBooleanValueError, - InvalidBytesValueError, InvalidEnvError, InvalidNumberValueError, InvalidStringArrayError, @@ -12,6 +11,8 @@ import { } from "../errors.js"; import { isTomlTable } from "./tables.js"; +export { parseBytes, parseSize } from "../size.js"; + export const parseBoolean = ( value: TomlValue, defaultValue: boolean, @@ -154,64 +155,3 @@ export const parseOptionalNumber = (value: TomlValue): bigint | undefined => { } throw new InvalidNumberValueError(value); }; - -const IEC_MULTIPLIERS: Readonly> = { - "": 1, - b: 1, - k: 1024, - ki: 1024, - kb: 1024, - kib: 1024, - m: 1024 ** 2, - mi: 1024 ** 2, - mb: 1024 ** 2, - mib: 1024 ** 2, - g: 1024 ** 3, - gi: 1024 ** 3, - gb: 1024 ** 3, - gib: 1024 ** 3, - t: 1024 ** 4, - ti: 1024 ** 4, - tb: 1024 ** 4, - tib: 1024 ** 4, - p: 1024 ** 5, - pi: 1024 ** 5, - pb: 1024 ** 5, - pib: 1024 ** 5, -}; - -/** - * Parses a byte size of a drive or nvram, accepting the IEC suffixes used by cartesi-machine - * ("4Ki", "1MiB") and their decimal spellings ("4kb", "100Mb"), all as powers of 1024, up to - * petabytes. Sizes are numbers, so the ones beyond 2^53 bytes, which they can't hold exactly, - * are rejected. - */ -export const parseSize = (value: TomlValue): number | undefined => { - if (value === undefined) { - return undefined; - } - let size: number | undefined; - if (typeof value === "number") { - size = value; - } else if (typeof value === "string") { - const match = /^\s*(\d+(?:\.\d+)?)\s*([a-z]*)\s*$/i.exec(value); - const multiplier = match - ? IEC_MULTIPLIERS[match[2].toLowerCase()] - : undefined; - if (match && multiplier !== undefined) { - size = Number(match[1]) * multiplier; - } - } - // a whole number of bytes, e.g. "1.5Ki" is but "1.1Ki" isn't - if ( - size !== undefined && - Number.isInteger(size) && - size <= Number.MAX_SAFE_INTEGER - ) { - return size; - } - throw new InvalidBytesValueError(value); -}; - -export const parseBytes = (value: TomlValue, defaultValue: number): number => - parseSize(value) ?? defaultValue; diff --git a/apps/cli/src/config/types.ts b/apps/cli/src/config/types.ts index 92941f049..de054643b 100644 --- a/apps/cli/src/config/types.ts +++ b/apps/cli/src/config/types.ts @@ -124,10 +124,100 @@ export type WithdrawalOptions = { account_size?: number; // in bytes, default is 32 }; +/** Block used by the node when fetching new blocks. */ +export type DefaultBlock = "latest" | "safe" | "pending" | "finalized"; + +export const DEFAULT_BLOCKS: readonly DefaultBlock[] = [ + "latest", + "safe", + "pending", + "finalized", +]; + +/** + * Configuration of the local development environment started by `cartesi run`. + * + * Every field is optional: it defines a project level default for the + * equivalent command line option, which always takes precedence when given. + */ +export type RunConfig = { + /** + * Interval between blocks, in seconds. + * @default 2 + */ + blockTime?: number; + + /** + * Number of blocks between a claim being submitted and accepted + * (Authority/Quorum only). + * @default 0 + */ + claimStagingPeriod?: number; + + /** Number of cpu limits for the rollups-node. */ + cpus?: number; + + /** + * Block used when fetching new blocks. + * @default "latest" + */ + defaultBlock?: DefaultBlock; + + /** + * Length of an epoch, in blocks. + * @default 720 + */ + epochLength?: number; + + /** Block number to fork from. */ + forkBlockNumber?: number; + + /** RPC URL to fork from. */ + forkUrl?: string; + + /** Memory limit for the rollups-node, in MB. */ + memory?: number; + + /** + * Port to listen on. + * @default first free port from 6751 + */ + port?: number; + + /** + * Name of the project, used by docker compose and by the rollups node. + * @default basename of the current working directory + */ + projectName?: string; + + /** + * Deploy the application with PRT consensus. + * @default false + */ + prt?: boolean; + + /** Version of the Cartesi Rollups Runtime to use. */ + runtimeVersion?: string; + + /** + * Optional services to start. The single value `all` starts every optional + * service. + * @default [] + */ + services?: string[]; + + /** + * Increase the log level of the environment services. + * @default false + */ + verbose?: boolean; +}; + export type Config = { drives: Record; machine: MachineConfig; nvrams: Record; + run: RunConfig; sdk: string; withdrawal?: WithdrawalOptions; }; diff --git a/apps/cli/src/config/user.ts b/apps/cli/src/config/user.ts new file mode 100644 index 000000000..564b51c53 --- /dev/null +++ b/apps/cli/src/config/user.ts @@ -0,0 +1,344 @@ +import type { Address } from "viem"; +import type { DriveFormat, RunConfig, WithdrawalOptions } from "./types.js"; + +/** + * Input types of the application configuration: the shape a developer actually + * writes, either in a `cartesi.config.ts` file through {@link defineConfig}, or + * in one of the plain data formats (`cartesi.config.json`, + * `cartesi.config.yaml`, `cartesi.config`). + * + * Everything that has a default is optional here. Sizes accept human readable + * strings (`"64Mi"`) as well as a number of bytes. + */ + +/** A size, either a number of bytes or a human readable string like `"64Mi"`. */ +export type Size = number | string; + +/** + * Keys that mark the drive or nvram holding the accounts of emergency + * withdrawal. At most one drive or nvram may carry them. + */ +export type UserAccountsDriveMarker = { + /** Whether this drive or nvram holds the accounts of emergency withdrawal. */ + accountsDrive?: boolean; + + /** + * Keep the accounts in this many bytes at the beginning of the drive, + * leaving the rest for other application state. + * @default all of the drive + */ + accountsDriveSize?: Size; +}; + +/** A drive built from a directory of the host filesystem. */ +export type UserDirectoryDriveConfig = { + builder: "directory"; + + /** Directory to copy into the drive. */ + directory: string; + + /** + * Extra free space to add to the drive, beyond the size of its contents. + * @default 0 + */ + extraSize?: Size; + + /** @default "ext2" */ + format?: DriveFormat; +}; + +/** A drive built from the filesystem of a docker image. */ +export type UserDockerDriveConfig = { + /** @default "docker" */ + builder?: "docker"; + + /** + * Arguments passed to the docker build, as `NAME=value` strings. + * @default [] + */ + buildArgs?: string[]; + + /** @default "." */ + context?: string; + + /** @default "Dockerfile" */ + dockerfile?: string; + + /** + * Extra free space to add to the drive, beyond the size of its contents. + * @default 0 + */ + extraSize?: Size; + + /** @default "ext2" */ + format?: DriveFormat; + + /** Use an existing image instead of building one from a Dockerfile. */ + image?: string; + + /** + * Tags applied to the image built. + * @default [] + */ + tags?: string[]; + + /** Stage to build, defaults to the last stage of a multi-stage build. */ + target?: string; +}; + +/** An empty drive, to be used as writable storage by the application. */ +export type UserEmptyDriveConfig = { + builder: "empty"; + + /** @default "ext2" */ + format?: "ext2" | "raw"; + + /** + * Size of the drive. + * @default 0 + */ + size?: Size; +}; + +/** A drive that already exists as a filesystem image on disk. */ +export type UserExistingDriveConfig = { + builder: "none"; + + /** Name of the drive file, its extension defines the format. */ + filename: string; + + /** @default inferred from the extension of `filename` */ + format?: DriveFormat; +}; + +/** A drive built from the contents of a tar archive. */ +export type UserTarDriveConfig = { + builder: "tar"; + + /** Name of the tar file. */ + filename: string; + + /** + * Extra free space to add to the drive, beyond the size of its contents. + * @default 0 + */ + extraSize?: Size; + + /** @default "ext2" */ + format?: DriveFormat; +}; + +export type UserDriveConfig = ( + | UserDirectoryDriveConfig + | UserDockerDriveConfig + | UserEmptyDriveConfig + | UserExistingDriveConfig + | UserTarDriveConfig +) & + UserAccountsDriveMarker & { + /** Where the drive is mounted inside the machine. */ + mount?: string | boolean; + + /** Whether writes to the drive are visible to the host. */ + shared?: boolean; + + /** User that owns the files of the drive. */ + user?: string; + }; + +/** + * An nvram of the machine: a raw range of bytes exposed to the guest as a + * `/dev/uio*` device, with no filesystem and no mount point. Either `size` or + * `filename` must be given. + */ +export type UserNvramConfig = UserAccountsDriveMarker & { + /** Path of an existing raw image with the initial contents. */ + filename?: string; + + /** Size of the nvram, a positive multiple of 4Ki. */ + size?: Size; + + /** Whether writes by the guest are persisted to the host. */ + shared?: boolean; + + /** User that owns the image of the nvram. */ + user?: string; +}; + +/** Configuration of the Cartesi machine of the application. */ +export type UserMachineConfig = { + assertRollingTemplate?: boolean; + + /** + * Extra arguments appended to the kernel command line. + * @default [] + */ + bootargs?: string[]; + + /** Command the machine runs on boot, overriding the docker image one. */ + entrypoint?: string; + + /** + * Environment variables injected into the machine. Values that are not + * strings are coerced to strings. + * @default {} + */ + env?: Record; + + /** Path of a `.env` file with environment variables injected into the machine. */ + envFile?: string; + + /** Maximum number of machine cycles to run. */ + maxMCycle?: bigint | number; + + /** + * Amount of RAM of the machine. + * @default "128Mi" + */ + ramLength?: string; + + /** Kernel image of the machine. */ + ramImage?: string; + + /** + * Inject the `ENV` of the docker image of the root drive into the machine. + * @default true + */ + useDockerEnv?: boolean; + + /** + * Inject the `WORKDIR` of the docker image of the root drive into the machine. + * @default true + */ + useDockerWorkdir?: boolean; + + /** User the entrypoint runs as. */ + user?: string; +}; + +/** + * Overrides of the emergency withdrawal configuration, which otherwise is + * derived from the accounts drive of the built machine. Only applies to an + * application that marks a drive or an nvram with `accountsDrive: true`. + * + * The keys are the ones of the resolved configuration, which the + * cartesi-rollups-cli expects in snake_case. + */ +export type UserWithdrawalOptions = { + /** @default a devnet account */ + guardian?: Address; + + /** @default the devnet TestUsdWithdrawalOutputBuilder */ + withdrawal_output_builder?: Address; + + /** + * Size of one account, a power of two of at least 32 bytes. + * @default 32 + */ + account_size?: Size; +}; + +/** + * Configuration of an application, as written by a developer. + * + * @see {@link defineConfig} + */ +export type UserConfig = { + /** + * Drives of the Cartesi machine. A `root` drive built from a `Dockerfile` + * of the current directory is used when none is defined. + */ + drives?: Record; + + /** Configuration of the Cartesi machine. */ + machine?: UserMachineConfig; + + /** Nvrams of the Cartesi machine, at most eight. */ + nvrams?: Record; + + /** + * Project level defaults of the local development environment started by + * `cartesi run`. Command line options always take precedence. + */ + run?: RunConfig; + + /** + * Docker image with the build tools used to build the drives and the + * machine. + * @default "cartesi/sdk:" + */ + sdk?: string; + + /** Overrides of the emergency withdrawal configuration. */ + withdrawal?: UserWithdrawalOptions | WithdrawalOptions; +}; + +/** Command being run, given to a configuration file that exports a function. */ +export type ConfigCommand = "build" | "run" | "shell"; + +/** + * Context given to a configuration file that exports a function instead of an + * object, so the configuration can depend on what is being run. + */ +export type ConfigEnv = { + /** Command being run. */ + command: ConfigCommand; + + /** Directory the configuration is being resolved from. */ + cwd: string; + + /** + * Mode the application is being built or run in, from `CARTESI_ENV` or + * `NODE_ENV`. + * @default "development" + */ + mode: string; +}; + +/** A configuration file that computes its configuration from the environment. */ +export type UserConfigFn = (env: ConfigEnv) => UserConfig | Promise; + +/** Anything a configuration file is allowed to export as its default export. */ +export type UserConfigExport = UserConfig | Promise | UserConfigFn; + +/** + * Define the configuration of a Cartesi application. + * + * This is an identity function that exists purely to give a `cartesi.config.ts` + * file type checking and editor completion, without the developer having to + * annotate the exported value: + * + * ```ts + * import { defineConfig } from "@cartesi/cli/config"; + * + * export default defineConfig({ + * drives: { + * root: { builder: "docker", dockerfile: "Dockerfile" }, + * }, + * machine: { ramLength: "256Mi" }, + * run: { epochLength: 10 }, + * }); + * ``` + * + * A function can be exported instead, to configure the application differently + * depending on the command being run: + * + * ```ts + * export default defineConfig(({ command }) => ({ + * run: { epochLength: command === "run" ? 10 : 720 }, + * })); + * ``` + * + * The function may be asynchronous. + * + * @param config configuration object, promise of one, or a function that + * computes one from the {@link ConfigEnv} + * @returns the configuration, unchanged + */ +export function defineConfig(config: UserConfig): UserConfig; +export function defineConfig(config: Promise): Promise; +export function defineConfig(config: UserConfigFn): UserConfigFn; +export function defineConfig(config: UserConfigExport): UserConfigExport; +export function defineConfig(config: UserConfigExport): UserConfigExport { + return config; +} diff --git a/apps/cli/src/defineConfig.ts b/apps/cli/src/defineConfig.ts new file mode 100644 index 000000000..49a59aa79 --- /dev/null +++ b/apps/cli/src/defineConfig.ts @@ -0,0 +1,68 @@ +/** + * Entrypoint of `@cartesi/cli/config`, the module a `cartesi.config.ts` file + * imports: + * + * ```ts + * import { defineConfig } from "@cartesi/cli/config"; + * + * export default defineConfig({ + * machine: { ramLength: "256Mi" }, + * }); + * ``` + * + * It only carries the configuration types and the helpers that operate on + * them, so importing it from a configuration file does not pull in the rest of + * the CLI. + */ + +export { mergeConfig } from "./config/merge.js"; +export type { + AccountsDriveMarker, + Config, + DefaultBlock, + DirectoryDriveConfig, + DockerDriveConfig, + DriveConfig, + DriveFormat, + EmptyDriveConfig, + ExistingDriveConfig, + ImageInfo, + MachineConfig, + NvramConfig, + RunConfig, + TarDriveConfig, + WithdrawalConfig, + WithdrawalOptions, +} from "./config/types.js"; +export { + MAX_NVRAMS, + NVRAM_ALIGNMENT, + DEFAULT_SDK_IMAGE, + DEFAULT_SDK_VERSION, + PREFERRED_PORT, +} from "./config/constants.js"; +export { + defaultConfig, + defaultMachineConfig, + defaultRootDriveConfig, + defaultRunConfig, +} from "./config/defaults.js"; +export { + type ConfigCommand, + type ConfigEnv, + defineConfig, + type Size, + type UserAccountsDriveMarker, + type UserConfig, + type UserConfigExport, + type UserConfigFn, + type UserDirectoryDriveConfig, + type UserDockerDriveConfig, + type UserDriveConfig, + type UserEmptyDriveConfig, + type UserExistingDriveConfig, + type UserMachineConfig, + type UserNvramConfig, + type UserTarDriveConfig, + type UserWithdrawalOptions, +} from "./config/user.js"; diff --git a/apps/cli/src/lib.ts b/apps/cli/src/lib.ts index 21f9e3595..88a9a74f0 100644 --- a/apps/cli/src/lib.ts +++ b/apps/cli/src/lib.ts @@ -21,26 +21,54 @@ */ export * from "./api/index.js"; -// configuration of an application ('cartesi.toml') +// configuration of an application ('cartesi.config.ts' and friends) export { + CONFIG_FILES, type Config, + type ConfigCommand, + type ConfigEnv, DEFAULT_SDK_IMAGE, DEFAULT_SDK_VERSION, + type DefaultBlock, defaultConfig, defaultMachineConfig, defaultRootDriveConfig, + defaultRunConfig, + defineConfig, type DirectoryDriveConfig, type DockerDriveConfig, type DriveConfig, type DriveFormat, type EmptyDriveConfig, type ExistingDriveConfig, + findConfigFile, type ImageInfo, + LEGACY_CONFIG_FILE, + loadConfig, + type LoadConfigOptions, + loadConfigFile, type MachineConfig, + mergeConfig, + normalizeConfig, type NvramConfig, parse as parseConfig, + parseSize, PREFERRED_PORT, + type RunConfig, + type Size, type TarDriveConfig, + type UserConfig, + type UserConfigExport, + type UserConfigFn, + type UserDirectoryDriveConfig, + type UserDockerDriveConfig, + type UserDriveConfig, + type UserEmptyDriveConfig, + type UserExistingDriveConfig, + type UserMachineConfig, + type UserNvramConfig, + type UserTarDriveConfig, + type UserWithdrawalOptions, type WithdrawalConfig, type WithdrawalOptions, } from "./config.js"; diff --git a/apps/cli/tests/unit/api/run.test.ts b/apps/cli/tests/unit/api/run.test.ts new file mode 100644 index 000000000..ebc0df733 --- /dev/null +++ b/apps/cli/tests/unit/api/run.test.ts @@ -0,0 +1,104 @@ +import { describe, expect, it } from "bun:test"; +import { resolveRunOptions } from "../../../src/api/run.js"; +import { DEFAULT_SDK_VERSION } from "../../../src/config.js"; + +describe("api/run", () => { + describe("resolveRunOptions", () => { + it("should fall back to the defaults", () => { + expect(resolveRunOptions({})).toEqual({ + blockTime: 2, + claimStagingPeriod: 0, + cpus: undefined, + defaultBlock: "latest", + epochLength: 720, + forkBlockNumber: undefined, + forkUrl: undefined, + memory: undefined, + port: undefined, + projectName: undefined, + prt: false, + runtimeVersion: DEFAULT_SDK_VERSION, + services: [], + verbose: false, + }); + }); + + it("should take the values of the configuration", () => { + const resolved = resolveRunOptions( + {}, + { + blockTime: 1, + defaultBlock: "finalized", + epochLength: 10, + forkUrl: "https://rpc.example.com", + port: 8080, + projectName: "my-app", + prt: true, + services: ["explorer"], + }, + ); + expect(resolved).toMatchObject({ + blockTime: 1, + defaultBlock: "finalized", + epochLength: 10, + forkUrl: "https://rpc.example.com", + port: 8080, + projectName: "my-app", + prt: true, + services: ["explorer"], + }); + // not set anywhere, so still the default + expect(resolved.claimStagingPeriod).toBe(0); + }); + + it("should let the options take precedence over the configuration", () => { + const resolved = resolveRunOptions( + { + blockTime: 5, + epochLength: 100, + projectName: "from-option", + services: [], + }, + { + blockTime: 1, + epochLength: 10, + projectName: "from-config", + services: ["explorer"], + }, + ); + expect(resolved).toMatchObject({ + blockTime: 5, + epochLength: 100, + projectName: "from-option", + // an empty list is a value, and does not fall back + services: [], + }); + }); + + it("should let a falsy option override the configuration", () => { + expect( + resolveRunOptions( + { blockTime: 0, prt: false }, + { blockTime: 1, prt: true }, + ), + ).toMatchObject({ blockTime: 0, prt: false }); + }); + + it("should resolve a free port later when none is given", () => { + // zero is not a usable port, and neither is it a request for one + expect(resolveRunOptions({ port: 0 }, { port: 0 }).port).toBe(0); + expect(resolveRunOptions({}, {}).port).toBeUndefined(); + }); + + it("should be verbose when the progress is verbose", () => { + expect(resolveRunOptions({ progress: "verbose" }).verbose).toBe( + true, + ); + expect( + resolveRunOptions({ progress: "verbose" }, { verbose: false }) + .verbose, + ).toBe(false); + expect(resolveRunOptions({}, { verbose: true }).verbose).toBe(true); + }); + }); +}); diff --git a/apps/cli/tests/unit/api/types.test.ts b/apps/cli/tests/unit/api/types.test.ts index 3c055456a..3c596537f 100644 --- a/apps/cli/tests/unit/api/types.test.ts +++ b/apps/cli/tests/unit/api/types.test.ts @@ -8,19 +8,27 @@ const fixture = (...paths: string[]) => describe("api/types", () => { describe("resolveConfig", () => { - it("should default to the default configuration", () => { - // there is no cartesi.toml at the root of the repository - expect(resolveConfig()).toEqual(defaultConfig()); + it("should default to the default configuration", async () => { + // there is no configuration file at the root of the repository + expect(await resolveConfig()).toEqual(defaultConfig()); }); - it("should return a configuration object as is", () => { + it("should normalize a configuration given inline", async () => { + const config = await resolveConfig({ sdk: "my/sdk:1.0.0" }); + expect(config).toEqual({ + ...defaultConfig(), + sdk: "my/sdk:1.0.0", + }); + }); + + it("should accept an already resolved configuration", async () => { const config = defaultConfig(); config.sdk = "my/sdk:1.0.0"; - expect(resolveConfig(config)).toBe(config); + expect(await resolveConfig(config)).toEqual(config); }); - it("should read a configuration file", () => { - const config = resolveConfig(fixture("drives", "rives.toml")); + it("should read a configuration file", async () => { + const config = await resolveConfig(fixture("drives", "rives.toml")); expect(Object.keys(config.drives)).toEqual([ "root", "doom", @@ -29,8 +37,8 @@ describe("api/types", () => { expect(config.nvrams).toEqual({}); }); - it("should merge a list of configuration files", () => { - const config = resolveConfig([ + it("should merge a list of configuration files", async () => { + const config = await resolveConfig([ fixture("drives", "rives.toml"), fixture("withdrawal", "config.toml"), ]); @@ -42,9 +50,9 @@ describe("api/types", () => { expect(config.nvrams.accounts?.accountsDrive).toBe(true); }); - it("should fail for a configuration file that does not exist", () => { - expect(() => resolveConfig("undefined.toml")).toThrow( - "Config file undefined.toml does not exist", + it("should fail for a configuration file that does not exist", async () => { + expect(resolveConfig("undefined.toml")).rejects.toThrow( + "does not exist", ); }); }); diff --git a/apps/cli/tests/unit/config.test.ts b/apps/cli/tests/unit/config.test.ts index e9338ac43..57bcddd4d 100644 --- a/apps/cli/tests/unit/config.test.ts +++ b/apps/cli/tests/unit/config.test.ts @@ -48,6 +48,8 @@ describe("when parsing the fixtures", () => { it("should parse every key of full.toml", () => { expect(parse(loadFixture("full.toml"))).toStrictEqual({ sdk: "my-sdk:dev", + // the deprecated format has no [run] section, so it always defaults + run: {}, machine: { assertRollingTemplate: true, bootargs: ["no4lvl", "quiet"], diff --git a/apps/cli/tests/unit/config/fixtures/files/bare.config b/apps/cli/tests/unit/config/fixtures/files/bare.config new file mode 100644 index 000000000..151d74fa8 --- /dev/null +++ b/apps/cli/tests/unit/config/fixtures/files/bare.config @@ -0,0 +1,3 @@ +# a configuration file with no format in its name, read as YAML +machine: + ramLength: 256Mi diff --git a/apps/cli/tests/unit/config/fixtures/files/cartesi.config.js b/apps/cli/tests/unit/config/fixtures/files/cartesi.config.js new file mode 100644 index 000000000..90e7cac32 --- /dev/null +++ b/apps/cli/tests/unit/config/fixtures/files/cartesi.config.js @@ -0,0 +1,3 @@ +export default { + machine: { ramLength: "512Mi" }, +}; diff --git a/apps/cli/tests/unit/config/fixtures/files/cartesi.config.json b/apps/cli/tests/unit/config/fixtures/files/cartesi.config.json new file mode 100644 index 000000000..82089bd5d --- /dev/null +++ b/apps/cli/tests/unit/config/fixtures/files/cartesi.config.json @@ -0,0 +1,5 @@ +{ + "$schema": "https://cartesi.io/schema/cartesi.config.json", + "machine": { "ramLength": "256Mi" }, + "run": { "epochLength": 10 } +} diff --git a/apps/cli/tests/unit/config/fixtures/files/cartesi.config.ts b/apps/cli/tests/unit/config/fixtures/files/cartesi.config.ts new file mode 100644 index 000000000..c3ff59446 --- /dev/null +++ b/apps/cli/tests/unit/config/fixtures/files/cartesi.config.ts @@ -0,0 +1,10 @@ +import { defineConfig } from "../../../../../src/defineConfig.js"; + +export default defineConfig({ + drives: { + data: { builder: "empty", size: "64Mb" }, + }, + machine: { entrypoint: "dapp", ramLength: "256Mi" }, + run: { epochLength: 10 }, + sdk: "my/sdk:1.0.0", +}); diff --git a/apps/cli/tests/unit/config/fixtures/files/cartesi.config.yaml b/apps/cli/tests/unit/config/fixtures/files/cartesi.config.yaml new file mode 100644 index 000000000..22a6a92af --- /dev/null +++ b/apps/cli/tests/unit/config/fixtures/files/cartesi.config.yaml @@ -0,0 +1,10 @@ +machine: + ramLength: 256Mi +drives: + data: + builder: empty + size: 64Mb +run: + epochLength: 10 + services: + - explorer diff --git a/apps/cli/tests/unit/config/fixtures/files/empty.config b/apps/cli/tests/unit/config/fixtures/files/empty.config new file mode 100644 index 000000000..e69de29bb diff --git a/apps/cli/tests/unit/config/fixtures/files/function.config.ts b/apps/cli/tests/unit/config/fixtures/files/function.config.ts new file mode 100644 index 000000000..a0ba6f99f --- /dev/null +++ b/apps/cli/tests/unit/config/fixtures/files/function.config.ts @@ -0,0 +1,6 @@ +import { defineConfig } from "../../../../../src/defineConfig.js"; + +export default defineConfig(async ({ command, mode }) => ({ + machine: { entrypoint: `${command}:${mode}` }, + run: { epochLength: command === "run" ? 10 : 720 }, +})); diff --git a/apps/cli/tests/unit/config/fixtures/files/invalid.config.ts b/apps/cli/tests/unit/config/fixtures/files/invalid.config.ts new file mode 100644 index 000000000..ddfdfea9d --- /dev/null +++ b/apps/cli/tests/unit/config/fixtures/files/invalid.config.ts @@ -0,0 +1 @@ +export const config = { sdk: "my/sdk:1.0.0" }; diff --git a/apps/cli/tests/unit/config/fixtures/files/override.yaml b/apps/cli/tests/unit/config/fixtures/files/override.yaml new file mode 100644 index 000000000..4b00cb5dd --- /dev/null +++ b/apps/cli/tests/unit/config/fixtures/files/override.yaml @@ -0,0 +1,4 @@ +machine: + entrypoint: overridden +run: + epochLength: 20 diff --git a/apps/cli/tests/unit/config/fixtures/files/unsupported.ini b/apps/cli/tests/unit/config/fixtures/files/unsupported.ini new file mode 100644 index 000000000..3b8d5eb22 --- /dev/null +++ b/apps/cli/tests/unit/config/fixtures/files/unsupported.ini @@ -0,0 +1 @@ +ram_length = 256Mi diff --git a/apps/cli/tests/unit/config/fixtures/project-legacy/cartesi.toml b/apps/cli/tests/unit/config/fixtures/project-legacy/cartesi.toml new file mode 100644 index 000000000..b866db650 --- /dev/null +++ b/apps/cli/tests/unit/config/fixtures/project-legacy/cartesi.toml @@ -0,0 +1,2 @@ +[machine] +ram_length = "256Mi" diff --git a/apps/cli/tests/unit/config/fixtures/project-none/.keep b/apps/cli/tests/unit/config/fixtures/project-none/.keep new file mode 100644 index 000000000..e69de29bb diff --git a/apps/cli/tests/unit/config/fixtures/project-yaml/cartesi.config.yaml b/apps/cli/tests/unit/config/fixtures/project-yaml/cartesi.config.yaml new file mode 100644 index 000000000..3f01502a8 --- /dev/null +++ b/apps/cli/tests/unit/config/fixtures/project-yaml/cartesi.config.yaml @@ -0,0 +1,4 @@ +machine: + ramLength: 256Mi +run: + epochLength: 10 diff --git a/apps/cli/tests/unit/config/load.test.ts b/apps/cli/tests/unit/config/load.test.ts new file mode 100644 index 000000000..967efd348 --- /dev/null +++ b/apps/cli/tests/unit/config/load.test.ts @@ -0,0 +1,178 @@ +import { describe, expect, it } from "bun:test"; +import * as path from "node:path"; +import { + CONFIG_FILES, + ConfigFileNotFoundError, + defaultConfig, + findConfigFile, + InvalidConfigExportError, + loadConfig, + loadConfigFile, + UnsupportedConfigFormatError, +} from "../../../src/config.js"; + +const fixtures = path.join(__dirname, "fixtures"); +const files = path.join(fixtures, "files"); +const file = (name: string) => path.join(files, name); + +const env = { command: "build", cwd: files, mode: "test" } as const; + +describe("findConfigFile", () => { + it("should find a configuration file by name", () => { + expect(findConfigFile(path.join(fixtures, "project-yaml"))).toBe( + path.join(fixtures, "project-yaml", "cartesi.config.yaml"), + ); + }); + + it("should find the deprecated configuration file", () => { + expect(findConfigFile(path.join(fixtures, "project-legacy"))).toBe( + path.join(fixtures, "project-legacy", "cartesi.toml"), + ); + }); + + it("should return undefined when the project has no configuration file", () => { + expect(findConfigFile(path.join(fixtures, "project-none"))).toBe( + undefined, + ); + }); + + it("should prefer the first of the supported names", () => { + // the fixture directory has a file of every supported format + expect(findConfigFile(files)).toBe(file(CONFIG_FILES[0])); + }); +}); + +describe("loadConfigFile", () => { + it("should load a TypeScript configuration file", async () => { + const config = await loadConfigFile(file("cartesi.config.ts"), env); + expect(config).toEqual({ + drives: { data: { builder: "empty", size: "64Mb" } }, + machine: { entrypoint: "dapp", ramLength: "256Mi" }, + run: { epochLength: 10 }, + sdk: "my/sdk:1.0.0", + }); + }); + + it("should load a JavaScript configuration file", async () => { + const config = await loadConfigFile(file("cartesi.config.js"), env); + expect(config).toEqual({ machine: { ramLength: "512Mi" } }); + }); + + it("should call a configuration file that exports a function", async () => { + const config = await loadConfigFile(file("function.config.ts"), { + command: "run", + cwd: files, + mode: "production", + }); + expect(config).toEqual({ + machine: { entrypoint: "run:production" }, + run: { epochLength: 10 }, + }); + }); + + it("should load a JSON configuration file", async () => { + const config = await loadConfigFile(file("cartesi.config.json"), env); + expect(config).toMatchObject({ + machine: { ramLength: "256Mi" }, + run: { epochLength: 10 }, + }); + }); + + it("should load a YAML configuration file", async () => { + const config = await loadConfigFile(file("cartesi.config.yaml"), env); + expect(config).toEqual({ + drives: { data: { builder: "empty", size: "64Mb" } }, + machine: { ramLength: "256Mi" }, + run: { epochLength: 10, services: ["explorer"] }, + }); + }); + + it("should read a file with no format in its name as YAML", async () => { + expect(await loadConfigFile(file("bare.config"), env)).toEqual({ + machine: { ramLength: "256Mi" }, + }); + }); + + it("should load an empty configuration file", async () => { + expect(await loadConfigFile(file("empty.config"), env)).toEqual({}); + }); + + it("should fail for a module without a default export", async () => { + expect(loadConfigFile(file("invalid.config.ts"), env)).rejects.toThrow( + new InvalidConfigExportError(file("invalid.config.ts")), + ); + }); + + it("should fail for an unsupported format", async () => { + expect(loadConfigFile(file("unsupported.ini"), env)).rejects.toThrow( + new UnsupportedConfigFormatError(file("unsupported.ini")), + ); + }); +}); + +describe("loadConfig", () => { + it("should use the defaults when the project has no configuration file", async () => { + const config = await loadConfig({ + cwd: path.join(fixtures, "project-none"), + }); + expect(config).toEqual(defaultConfig()); + }); + + it("should find and resolve the configuration file of the project", async () => { + const config = await loadConfig({ + cwd: path.join(fixtures, "project-yaml"), + }); + expect(config.machine.ramLength).toBe("256Mi"); + expect(config.run.epochLength).toBe(10); + }); + + it("should still read the deprecated configuration file", async () => { + const config = await loadConfig({ + cwd: path.join(fixtures, "project-legacy"), + }); + expect(config.machine.ramLength).toBe("256Mi"); + }); + + it("should resolve and validate an explicit configuration file", async () => { + const config = await loadConfig({ + cwd: files, + files: ["cartesi.config.ts"], + }); + expect(config.sdk).toBe("my/sdk:1.0.0"); + expect(config.machine.ramLength).toBe("256Mi"); + expect(config.run.epochLength).toBe(10); + // sizes written as strings are resolved to a number of bytes + expect(config.drives.data).toMatchObject({ size: 64 * 1024 * 1024 }); + // a root drive is always present + expect(config.drives.root).toMatchObject({ builder: "docker" }); + }); + + it("should merge a list of configuration files, in order", async () => { + const config = await loadConfig({ + cwd: files, + files: ["cartesi.config.ts", "override.yaml"], + }); + expect(config.machine.entrypoint).toBe("overridden"); + // untouched by the override + expect(config.machine.ramLength).toBe("256Mi"); + expect(config.run.epochLength).toBe(20); + }); + + it("should give the command and the mode to a configuration file", async () => { + const config = await loadConfig({ + command: "run", + cwd: files, + files: ["function.config.ts"], + mode: "production", + }); + expect(config.machine.entrypoint).toBe("run:production"); + }); + + it("should fail for a configuration file that does not exist", async () => { + expect( + loadConfig({ cwd: files, files: ["missing.config.ts"] }), + ).rejects.toThrow( + new ConfigFileNotFoundError(file("missing.config.ts")), + ); + }); +}); diff --git a/apps/cli/tests/unit/config/normalize.test.ts b/apps/cli/tests/unit/config/normalize.test.ts new file mode 100644 index 000000000..23435feb3 --- /dev/null +++ b/apps/cli/tests/unit/config/normalize.test.ts @@ -0,0 +1,501 @@ +import { describe, expect, it } from "bun:test"; +import { + defaultConfig, + defaultMachineConfig, + defaultRootDriveConfig, + defineConfig, + type DockerDriveConfig, + DuplicateLabelError, + getAccountsDrive, + InvalidAccountsDriveError, + InvalidAddressValueError, + InvalidBooleanValueError, + InvalidBuilderError, + InvalidBytesValueError, + InvalidDriveFormatError, + InvalidEnumValueError, + InvalidNumberValueError, + InvalidNvramSizeError, + InvalidTableError, + InvalidStringValueError, + InvalidWithdrawalConfigError, + MAX_NVRAMS, + MissingNvramSourceError, + MultipleAccountsDrivesError, + normalizeConfig, + NVRAM_ALIGNMENT, + RequiredFieldError, + TooManyNvramsError, +} from "../../../src/config.js"; + +describe("defineConfig", () => { + it("should return the configuration unchanged", () => { + const config = { sdk: "my/sdk:1.0.0" }; + expect(defineConfig(config)).toBe(config); + }); + + it("should return a configuration function unchanged", () => { + const fn = () => ({ sdk: "my/sdk:1.0.0" }); + expect(defineConfig(fn)).toBe(fn); + }); +}); + +// the default root drive is built by docker, narrowed so the expectations +// below can spread it and still be checked against that member of the union +const defaultRootDrive = () => defaultRootDriveConfig() as DockerDriveConfig; + +describe("normalizeConfig", () => { + it("should apply the defaults to an empty configuration", () => { + expect(normalizeConfig(undefined)).toEqual(defaultConfig()); + expect(normalizeConfig({})).toEqual(defaultConfig()); + }); + + it("should be idempotent on an already resolved configuration", () => { + const config = normalizeConfig({ + drives: { data: { builder: "empty", size: "64Mb" } }, + machine: { entrypoint: "dapp", ramLength: "256Mi" }, + run: { epochLength: 10 }, + sdk: "my/sdk:1.0.0", + }); + expect(normalizeConfig(config)).toEqual(config); + }); + + it("should ignore the '$schema' key of a JSON configuration", () => { + expect( + normalizeConfig({ $schema: "https://cartesi.io/schema.json" }), + ).toEqual(defaultConfig()); + }); + + it("should fail for a configuration that is not an object", () => { + expect(() => normalizeConfig(42)).toThrowError( + new InvalidTableError("the configuration", 42), + ); + }); + + describe("drives", () => { + it("should add a default root drive", () => { + const config = normalizeConfig({ + drives: { data: { builder: "empty", size: 128 } }, + }); + expect(config.drives.root).toEqual(defaultRootDrive()); + expect(config.drives.data).toEqual({ + builder: "empty", + format: "ext2", + mount: undefined, + shared: undefined, + size: 128, + user: undefined, + }); + }); + + it("should default the builder to docker", () => { + const config = normalizeConfig({ + drives: { root: { dockerfile: "backend/Dockerfile" } }, + }); + expect(config.drives.root).toEqual({ + ...defaultRootDrive(), + dockerfile: "backend/Dockerfile", + }); + }); + + it("should parse human readable sizes", () => { + const config = normalizeConfig({ + drives: { + data: { builder: "directory", directory: "data" }, + root: { extraSize: "10Mb" }, + }, + }); + expect(config.drives.root).toEqual({ + ...defaultRootDrive(), + extraSize: 10 * 1024 * 1024, + }); + expect(config.drives.data).toMatchObject({ extraSize: 0 }); + }); + + it("should infer the format of an existing drive from its filename", () => { + const config = normalizeConfig({ + drives: { + doom: { builder: "none", filename: "doom.sqfs" }, + }, + }); + expect(config.drives.doom).toMatchObject({ format: "sqfs" }); + }); + + it("should keep the drive options common to every builder", () => { + const config = normalizeConfig({ + drives: { + data: { + builder: "tar", + filename: "data.tar", + mount: "/mnt/data", + shared: true, + user: "dapp", + }, + }, + }); + expect(config.drives.data).toEqual({ + builder: "tar", + extraSize: 0, + filename: "data.tar", + format: "ext2", + mount: "/mnt/data", + shared: true, + user: "dapp", + }); + }); + + it("should fail for an invalid builder", () => { + expect(() => + normalizeConfig({ drives: { root: { builder: "invalid" } } }), + ).toThrowError(new InvalidBuilderError("invalid")); + }); + + it("should fail for an invalid format", () => { + expect(() => + normalizeConfig({ drives: { root: { format: "invalid" } } }), + ).toThrowError(new InvalidDriveFormatError("invalid")); + }); + + it("should fail for an invalid size", () => { + expect(() => + normalizeConfig({ + drives: { root: { extraSize: "abc" } }, + }), + ).toThrowError(new InvalidBytesValueError("abc")); + }); + + it("should fail when a required field is missing", () => { + expect(() => + normalizeConfig({ drives: { data: { builder: "directory" } } }), + ).toThrowError(new RequiredFieldError("directory")); + expect(() => + normalizeConfig({ drives: { data: { builder: "tar" } } }), + ).toThrowError(new RequiredFieldError("filename")); + }); + + it("should fail when drives is not an object", () => { + expect(() => normalizeConfig({ drives: 42 })).toThrowError( + new InvalidTableError("drives", 42), + ); + }); + }); + + describe("machine", () => { + it("should apply the machine defaults", () => { + expect(normalizeConfig({ machine: {} }).machine).toEqual( + defaultMachineConfig(), + ); + }); + + it("should coerce non-string environment variables", () => { + expect( + normalizeConfig({ + machine: { env: { ENABLED: true, PORT: 8080 } }, + }).machine.env, + ).toEqual({ ENABLED: "true", PORT: "8080" }); + }); + + it("should accept a number as the maximum cycle count", () => { + expect( + normalizeConfig({ machine: { maxMCycle: 100 } }).machine + .maxMCycle, + ).toBe(100n); + }); + + it("should fail for an invalid boolean", () => { + expect(() => + normalizeConfig({ machine: { useDockerEnv: 42 } }), + ).toThrowError(new InvalidBooleanValueError(42)); + }); + + it("should fail for an invalid entrypoint", () => { + expect(() => + normalizeConfig({ machine: { entrypoint: 42 } }), + ).toThrowError(new InvalidStringValueError(42)); + }); + }); + + describe("run", () => { + it("should be empty by default", () => { + expect(normalizeConfig({}).run).toEqual({}); + }); + + it("should keep the run options", () => { + expect( + normalizeConfig({ + run: { + blockTime: 1, + defaultBlock: "finalized", + epochLength: 10, + projectName: "my-app", + prt: true, + services: ["explorer", "bundler"], + }, + }).run, + ).toEqual({ + blockTime: 1, + claimStagingPeriod: undefined, + cpus: undefined, + defaultBlock: "finalized", + epochLength: 10, + forkBlockNumber: undefined, + forkUrl: undefined, + memory: undefined, + port: undefined, + projectName: "my-app", + prt: true, + runtimeVersion: undefined, + services: ["explorer", "bundler"], + verbose: undefined, + }); + }); + + it("should fail for an invalid default block", () => { + expect(() => + normalizeConfig({ run: { defaultBlock: "invalid" } }), + ).toThrowError( + new InvalidEnumValueError("defaultBlock", "invalid", [ + "latest", + "safe", + "pending", + "finalized", + ]), + ); + }); + + it("should fail for an invalid number", () => { + expect(() => + normalizeConfig({ run: { epochLength: "ten" } }), + ).toThrowError(new InvalidNumberValueError("ten", "epochLength")); + }); + + it("should fail for invalid services", () => { + expect(() => + normalizeConfig({ run: { services: [42] } }), + ).toThrowError(new InvalidStringValueError(42)); + }); + }); + + describe("nvrams", () => { + it("should be empty by default", () => { + expect(normalizeConfig({}).nvrams).toEqual({}); + }); + + it("should normalize a pristine nvram", () => { + expect( + normalizeConfig({ nvrams: { state: { size: "8Ki" } } }).nvrams + .state, + ).toEqual({ + filename: undefined, + shared: undefined, + size: 8192, + user: undefined, + }); + }); + + it("should normalize an nvram backed by an image", () => { + expect( + normalizeConfig({ + nvrams: { state: { filename: "state.raw", shared: true } }, + }).nvrams.state, + ).toEqual({ + filename: "state.raw", + shared: true, + size: undefined, + user: undefined, + }); + }); + + it("should fail without a size or a filename", () => { + expect(() => + normalizeConfig({ nvrams: { state: {} } }), + ).toThrowError(new MissingNvramSourceError("state")); + }); + + it("should fail for a size that is not a multiple of the alignment", () => { + expect(() => + normalizeConfig({ nvrams: { state: { size: 100 } } }), + ).toThrowError(new InvalidNvramSizeError("state", 100)); + expect(() => + normalizeConfig({ nvrams: { state: { size: 0 } } }), + ).toThrowError(new InvalidNvramSizeError("state", 0)); + }); + + it("should fail for more nvrams than the guest exposes", () => { + const nvrams = Object.fromEntries( + Array.from({ length: MAX_NVRAMS + 1 }, (_, i) => [ + `state${i}`, + { size: NVRAM_ALIGNMENT }, + ]), + ); + expect(() => normalizeConfig({ nvrams })).toThrowError( + new TooManyNvramsError(MAX_NVRAMS + 1), + ); + }); + + it("should fail when a label is used by both a drive and an nvram", () => { + expect(() => + normalizeConfig({ + drives: { state: { builder: "empty", size: "4Ki" } }, + nvrams: { state: { size: "4Ki" } }, + }), + ).toThrowError(new DuplicateLabelError("state")); + }); + }); + + describe("withdrawal", () => { + // the accounts drive is what emergency withdrawal is derived from + const accountsNvram = { + nvrams: { accounts: { accountsDrive: true, size: "4Ki" } }, + }; + + it("should be undefined when not given", () => { + expect(normalizeConfig(accountsNvram).withdrawal).toBeUndefined(); + }); + + it("should keep the overrides of an application with an accounts drive", () => { + expect( + normalizeConfig({ + ...accountsNvram, + withdrawal: { + account_size: "64", + guardian: "0x1111111111111111111111111111111111111111", + withdrawal_output_builder: + "0x2222222222222222222222222222222222222222", + }, + }).withdrawal, + ).toEqual({ + account_size: 64, + guardian: "0x1111111111111111111111111111111111111111", + withdrawal_output_builder: + "0x2222222222222222222222222222222222222222", + }); + }); + + it("should accept an empty table, the whole configuration being derived", () => { + expect( + normalizeConfig({ ...accountsNvram, withdrawal: {} }) + .withdrawal, + ).toEqual({ + account_size: undefined, + guardian: undefined, + withdrawal_output_builder: undefined, + }); + }); + + it("should fail without an accounts drive", () => { + expect(() => normalizeConfig({ withdrawal: {} })).toThrowError( + InvalidWithdrawalConfigError, + ); + }); + + it("should fail for an invalid address", () => { + expect(() => + normalizeConfig({ + ...accountsNvram, + withdrawal: { guardian: "invalid" }, + }), + ).toThrowError(new InvalidAddressValueError("invalid", "guardian")); + }); + + it("should fail for an account size that is not a power of two", () => { + expect(() => + normalizeConfig({ + ...accountsNvram, + withdrawal: { account_size: 48 }, + }), + ).toThrowError(InvalidWithdrawalConfigError); + expect(() => + normalizeConfig({ + ...accountsNvram, + withdrawal: { account_size: 16 }, + }), + ).toThrowError(InvalidWithdrawalConfigError); + }); + }); + + describe("accounts drive", () => { + it("should accept a raw empty drive", () => { + const config = normalizeConfig({ + drives: { + accounts: { + accountsDrive: true, + builder: "empty", + format: "raw", + size: "4Ki", + }, + }, + }); + expect(getAccountsDrive(config)).toMatchObject({ + kind: "flash_drive", + label: "accounts", + }); + }); + + it("should accept an nvram", () => { + const config = normalizeConfig({ + nvrams: { accounts: { accountsDrive: true, size: "4Ki" } }, + }); + expect(getAccountsDrive(config)).toMatchObject({ + kind: "nvram", + label: "accounts", + }); + }); + + it("should be undefined when nothing is marked", () => { + expect(getAccountsDrive(normalizeConfig({}))).toBeUndefined(); + }); + + it("should fail when a drive has a filesystem", () => { + expect(() => + normalizeConfig({ + drives: { + accounts: { + accountsDrive: true, + builder: "empty", + size: "4Ki", + }, + }, + }), + ).toThrowError(InvalidAccountsDriveError); + }); + + it("should fail when more than one is marked", () => { + expect(() => + normalizeConfig({ + drives: { + accounts: { + accountsDrive: true, + builder: "empty", + format: "raw", + size: "4Ki", + }, + }, + nvrams: { other: { accountsDrive: true, size: "4Ki" } }, + }), + ).toThrowError( + new MultipleAccountsDrivesError(["accounts", "other"]), + ); + }); + + it("should fail for a size that is not a power of two", () => { + expect(() => + normalizeConfig({ + nvrams: { + accounts: { accountsDrive: true, size: 3 * 4096 }, + }, + }), + ).toThrowError(InvalidAccountsDriveError); + }); + + it("should fail when the accounts size is set without the marker", () => { + expect(() => + normalizeConfig({ + nvrams: { + state: { accountsDriveSize: "4Ki", size: "4Ki" }, + }, + }), + ).toThrowError(InvalidAccountsDriveError); + }); + }); +}); diff --git a/apps/cli/tests/unit/config/size.test.ts b/apps/cli/tests/unit/config/size.test.ts new file mode 100644 index 000000000..32f1110fe --- /dev/null +++ b/apps/cli/tests/unit/config/size.test.ts @@ -0,0 +1,76 @@ +import { describe, expect, it } from "bun:test"; +import { + InvalidBytesValueError, + parseBytes, + parseSize, +} from "../../../src/config.js"; + +describe("parseSize", () => { + it("should leave a size that was not given undefined", () => { + // the caller decides what the default is, see parseBytes + expect(parseSize(undefined)).toBeUndefined(); + }); + + it("should take a number of bytes as is", () => { + expect(parseSize(0)).toBe(0); + expect(parseSize(128)).toBe(128); + expect(parseSize(128n)).toBe(128); + expect(parseSize("128")).toBe(128); + }); + + it("should parse the decimal looking units as binary multiples", () => { + expect(parseSize("1kb")).toBe(1024); + expect(parseSize("128MB")).toBe(128 * 1024 ** 2); + expect(parseSize("2Gb")).toBe(2 * 1024 ** 3); + }); + + it("should parse the IEC units", () => { + // 'bytes.parse' reads these as a plain number of bytes, which silently + // makes a drive several orders of magnitude too small + expect(parseSize("64Mi")).toBe(64 * 1024 ** 2); + expect(parseSize("64MiB")).toBe(64 * 1024 ** 2); + expect(parseSize("1Ki")).toBe(1024); + }); + + it("should parse a unit with no prefix", () => { + expect(parseSize("512b")).toBe(512); + expect(parseSize("4M")).toBe(4 * 1024 ** 2); + }); + + it("should accept fractions and spacing", () => { + expect(parseSize("1.5Mb")).toBe(Math.floor(1.5 * 1024 ** 2)); + expect(parseSize(" 10 Mb ")).toBe(10 * 1024 ** 2); + }); + + it("should reject anything it does not understand", () => { + expect(() => parseSize("abc")).toThrowError( + new InvalidBytesValueError("abc"), + ); + expect(() => parseSize("64Xy")).toThrowError( + new InvalidBytesValueError("64Xy"), + ); + expect(() => parseSize("")).toThrowError( + new InvalidBytesValueError(""), + ); + expect(() => parseSize({})).toThrowError( + new InvalidBytesValueError({}), + ); + }); +}); + +describe("parseBytes", () => { + it("should fall back to the default when no size is given", () => { + expect(parseBytes(undefined, 0)).toBe(0); + expect(parseBytes(undefined, 4096)).toBe(4096); + }); + + it("should parse a size that is given", () => { + expect(parseBytes("64Mi", 0)).toBe(64 * 1024 ** 2); + }); + + it("should reject a size it does not understand", () => { + expect(() => parseBytes("abc", 0)).toThrowError( + new InvalidBytesValueError("abc"), + ); + }); +});