Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,13 @@ To serve your config, do:

`workerd serve my-config.capnp`

To check a config without serving it, do:

`workerd check my-config.capnp`

This builds every service and compiles every Worker, but binds no socket. It reports every
error rather than only the first, and exits with status 1 if there was any.

For more details about command-line usage, use `workerd --help`.

Prebuilt binaries are distributed via `npm`. Run `npx workerd ...` to use these. If you're running a prebuilt binary, you'll need to make sure your system has the right dependencies installed:
Expand Down
8 changes: 4 additions & 4 deletions src/workerd/server/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

The `workerd` binary and its server. `:workerd` (`workerd.rs`, this package) is a Rust binary over three crates in this directory:

- `cli/` -- the `workerd-cli` crate (declared in this package's `BUILD.bazel`, sources globbed from `cli/`): parses the command line (clap), produces the encoded config (schema files are compiled by this package's `config-compiler.c++`), handles `compile`, `--watch` and re-exec, and hands `serve`, `test` (which `fuzzilli` and `make-pyodide-baseline-snapshot` are built on) and `bench` to the server crate's `entry` module.
- `cli/` -- the `workerd-cli` crate (declared in this package's `BUILD.bazel`, sources globbed from `cli/`): parses the command line (clap), produces the encoded config (schema files are compiled by this package's `config-compiler.c++`), handles `compile`, `--watch` and re-exec, and hands `serve`, `check`, `test` (which `fuzzilli` and `make-pyodide-baseline-snapshot` are built on) and `bench` to the server crate's `entry` module.
- `server/` -- the `workerd-server` crate: the single-tenant server. It interprets the config, builds the service graph, binds the sockets, runs the listeners, actors and dynamic workers, drains on SIGTERM, and runs `workerd test` and `workerd bench`. `Error` is `cxx::KjError`; `bridge.rs` is the one module that allows `unsafe`.
- `factory/` -- C++: `WorkerFactory` (`worker-factory*`), everything that needs the isolate, driven by the server through the cxx bridge in `server/bridge.rs`; and `bootstrap.{h,c++}`, the C++ half of a command. `with_process_context` sets up the process's logging (the JSON logger, when the config asks for structured logging) on the C++ stack around the Rust command; `bootstrap` sets up perfetto, autogates and V8 on the Rust-owned event loop and builds the `WorkerFactory`, which owns them, from the encoded config and the command-line options for `server/entry.rs`, which owns the loop (`kj_rs_tokio::Runtime`) and blocks on `server_serve` / `server_test` / `server_bench`.
- `cluster/` -- the `cluster` crate and, in `cluster/logic/`, `cluster-logic`: cluster mode (`Config.cluster`, experimental), Durable Objects whose shared state is an S3-compatible bucket. Leases, ownership, LTX capture and restore; pure Rust, driven by `server/cluster.rs`. `cluster-logic` holds decisions only, with no dependencies to do I/O with.
Expand Down Expand Up @@ -67,8 +67,8 @@ Bazel labels: `//src/workerd/server:workerd`, `//src/workerd/server:workerd-cli`
| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `lib.rs` | Crate root: `Error` / `Result` |
| `bridge.rs` | The cxx bridge (`workerd::server`) to `factory/worker-factory.h` and `factory/bootstrap.h`. `extern "C++"`: the factory (compile, start request, bench, actors and storage, RPC, tokens, inspector, `factory_spawn`), the bootstrap, the process's logging and its exit; `extern "Rust"`: the channel objects, the channel factory, the pending command. Ownership rules in its `//!` |
| `entry.rs` | `serve` / `test` / `bench`: the process's schedule (`with_process_context`, the `kj_rs_tokio::Runtime`, `bootstrap`, `block_on`, exit), `server_serve` / `server_test` / `server_bench`, the `Watch` future the command line supplies, printing config errors and the exit status they make, SIGTERM drain, `--watch` reload, shutdown order (settle tasks, drop the factory and with it V8 and the perfetto session, then the Runtime) |
| `run.rs` | `run()` / `test()` / `bench()`: bind sockets, start the inspector (before any isolate exists), build the graph, start the listeners (not for a config the `Reporter` refuses), drain, control-fd events, the check that every `--directory-path` / `--external-addr` override named a service |
| `entry.rs` | `serve` / `check` / `test` / `bench`: the process's schedule (`with_process_context`, the `kj_rs_tokio::Runtime`, `bootstrap`, `block_on`, exit), `server_serve` / `server_check` / `server_test` / `server_bench`, the `Watch` future the command line supplies, printing config errors and the exit status they make, SIGTERM drain, `--watch` reload, shutdown order (settle tasks, drop the factory and with it V8 and the perfetto session, then the Runtime) |
| `run.rs` | `run()` / `test()` / `bench()`, and `check()`, which only builds the graph: bind sockets, start the inspector (before any isolate exists), build the graph, start the listeners (not for a config the `Reporter` refuses), drain, control-fd events, the check that every `--directory-path` / `--external-addr` override named a service |
| `bench.rs` | `workerd bench`'s options and report (`io/bench.capnp`): a group for each `bench()` handler `run::bench` runs (through the factory's `worker_bench`), progress logged as `workerd test` logs it, the environment, and the report as a table or as JSON |
| `config.rs` | `Server`: the service graph from the config in three passes (actor configs, construction, link); `Factory`, the Rust owner of the C++ `WorkerFactory`; `Reporter`; resolves channel tokens and debug-port requests |
| `channels.rs` | The handles C++ holds and the runtime reaches into the server through: `SubrequestChannel` (an `Rc<dyn Channel>`), `ActorClassChannel` (an `Rc<dyn ActorClass>`), `ChannelFactory` (a worker's `LinkedChannels`), `WorkerStub`, and the `Weak` back-pointers (`ServerHandle`, `ActorNamespaceHandle`, `ActorHooks`). Single-threaded |
Expand Down Expand Up @@ -99,7 +99,7 @@ Bazel labels: `//src/workerd/server:workerd`, `//src/workerd/server:workerd-cli`
- **Rust unit tests**: a module's tests are in the `<module>-test.rs` beside it (`config-test.rs` for `config.rs`, `listen/mod-test.rs` for `listen/mod.rs`), compiled only into the crates' test targets (`//src/workerd/server/server:workerd-server_test`, `//src/workerd/server:workerd-cli_test`); see `src/rust/AGENTS.md`, UNIT TEST FILES. The server crate's test binary links `//src/workerd/util:setup-async-io` because the C++ it reaches calls `kj::setupAsyncIo()`.
- **C++ unit tests** (`*-test.c++`, `kj_test`): alarm scheduler, channel tokens, container client, facet tree index, fallback service, JSON logger, actor IDs.
- **`server-test.c++`** (`//src/workerd/server:server-test@`): the server end to end, in the test process. Each `TestServer` builds a `WorkerFactory` on the test's V8, timer, network and a temp directory, and hands it to `InProcessServer` (`server/in_process.rs`, through the server crate's bridge), which runs `run::run` / `run::test` / `run::bench` on it and queues the config errors and warnings for the test. No connection but UDP's leaves the process: the config's socket and external addresses become `loopback:` names, the test connects to the server's sockets and accepts the server's outbound connections by name (the `network` services' too), and the factory's own C++ clients (Docker, the debug port) dial the same names through the test's `kj::Network`. The factory's timer is the test's mock `kj::TimerImpl`, which `test.wait(n)` advances; the server's own delays run on it too (`Factory::sleep` / `Factory::now`, `server/config.rs`). An unexpected error log, config error or warning fails the case, as does a factory still referenced once the server is closed. Add a case here for every new config error or warning.
- **`tests/`**: `server-harness.mjs`, a Node.js harness spawning the `workerd` binary for E2E tests; it reads the `listen` events off `--control-fd`. Subdirectories: `compile-tests`, `container-client`, `container-shutdown`, `extensions`, `inspector`, `module-fallback`, `python`, `socket-close`, `structured-logging`, `udp-connect`, `udp-connect-ts-streams`, `udp-datagram-drop`, `unsafe-eval`, `unsafe-module`, `weakref`, `wrapped-binding-serialization`. Every `.wd-test` in the repository also runs through `workerd test` (`wd_test` in `build/wd_test.bzl`, over `:workerd`), and every `wd_bench()` config through `workerd bench --quick`, in its `@smoke` tests (`build/wd_bench.bzl`).
- **`tests/`**: `server-harness.mjs`, a Node.js harness spawning the `workerd` binary for E2E tests; it reads the `listen` events off `--control-fd`. Subdirectories: `check` (`workerd check`, as a shell test), `compile-tests`, `container-client`, `container-shutdown`, `extensions`, `inspector`, `module-fallback`, `python`, `socket-close`, `structured-logging`, `udp-connect`, `udp-connect-ts-streams`, `udp-datagram-drop`, `unsafe-eval`, `unsafe-module`, `weakref`, `wrapped-binding-serialization`. Every `.wd-test` in the repository also runs through `workerd test` (`wd_test` in `build/wd_test.bzl`, over `:workerd`), and every `wd_bench()` config through `workerd bench --quick`, in its `@smoke` tests (`build/wd_bench.bzl`).
- **kj-hyper contract tests** (`src/rust/cxx/kj-hyper/tests`): HTTP and WebSocket semantics against kj's interfaces.
- `:rust-io-link-check` inspects the linked binary's symbols (unix): kj's own `setupAsyncIo()` must not be linked in.

Expand Down
27 changes: 27 additions & 0 deletions src/workerd/server/cli/args-test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,33 @@ fn wd_test_invocation() {
);
}

#[test]
fn check_invocation() {
let Command::Check {
config,
const_name,
serve_or_test,
} = parse(&[
"check",
"config.capnp",
"myConfig",
"--experimental",
"-dstorage=/tmp/x",
])
.command
else {
panic!("expected check")
};
assert_eq!(config.config_file, "config.capnp");
assert_eq!(const_name.const_name.as_deref(), Some("myConfig"));
assert!(serve_or_test.experimental);
assert_eq!(serve_or_test.directory_overrides.len(), 1);
// `check` serves nothing, so it takes none of `serve`'s socket options.
assert!(
Main::try_parse_from(["workerd", "check", "c.capnp", "--socket-addr=http=*:0"]).is_err()
);
}

#[test]
fn serve_options() {
let Command::Serve {
Expand Down
20 changes: 20 additions & 0 deletions src/workerd/server/cli/args.rs
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,26 @@ pub enum Command {
serve: ServeArgs,
},

/// check a config and report every problem, without serving
#[command(
long_about = "Loads a config and reports every problem with it, without serving.",
after_long_help = "Accepts the same <config-file> as `serve`. Every service is built and \
every Worker compiled, so errors in a Worker's top-level code are reported too, but no \
socket is bound, so a check can run next to a server using the same ports. Every \
error is reported, rather than only the first, followed by their count, or by \
\"Config OK\" if there were none. Exits with status 1 if there was any error."
)]
Check {
#[command(flatten)]
config: ConfigFileArgs,

#[command(flatten)]
const_name: ConstNameArg,

#[command(flatten)]
serve_or_test: ServeOrTestArgs,
},

/// create a self-contained binary
#[command(
long_about = "Builds a self-contained binary from a config.",
Expand Down
39 changes: 39 additions & 0 deletions src/workerd/server/cli/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,17 @@ fn run_command(args: args::Main, executable: Option<Executable>) -> workerd_serv
watching(process),
)
}
args::Command::Check {
config,
const_name,
serve_or_test,
} => run_check(
verbose,
config,
const_name.const_name,
&serve_or_test,
executable,
),
args::Command::Compile {
config,
const_name,
Expand Down Expand Up @@ -193,6 +204,34 @@ fn run_command(args: args::Main, executable: Option<Executable>) -> workerd_serv
}
}

/// `workerd check`: every error in the config, without serving it. Options that only make sense
/// for a running server are usage errors rather than silently ignored.
fn run_check(
verbose: bool,
config: args::ConfigFileArgs,
const_name: Option<String>,
serve_or_test: &args::ServeOrTestArgs,
executable: Option<Executable>,
) -> workerd_server::Result<i32> {
type T = args::Main;

check_serve_or_test::<T>(serve_or_test);
if serve_or_test.watch {
usage_error::<T>("--watch: `check` does not serve, so has nothing to reload.");
}
if serve_or_test.inspector_addr.is_some() {
usage_error::<T>("--inspector-addr: `check` does not serve, so starts no inspector.");
}
let process = Process::new(executable, None, Vec::new());
let config = load_config::<T>(config, const_name, &process);
entry::check(
verbose,
config.into_words(),
serve_or_test.into(),
run_options(serve_or_test).overrides,
)
}

/// `workerd compile`: the config, encoded, either on its own (`--config-only`) or appended to a
/// copy of this executable.
fn run_compile(
Expand Down
31 changes: 31 additions & 0 deletions src/workerd/server/server/config-test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,37 @@ fn lookup_of_a_missing_service_is_reported() {
);
}

#[test]
fn lookup_of_a_misspelled_service_suggests_the_closest_name() {
let services = leaf_services();
let (report, errors) = collecting_reporter();
lookup_service(&services, &report, &designator("nett", None));
lookup_actor_class(&services, &report, &designator("NET", None));
assert_eq!(
*errors.borrow(),
[
"Worker \"w\"'s binding \"b\" refers to a service \"nett\", but no such service \
is defined. Did you mean \"net\"?",
"Worker \"w\"'s binding \"b\" refers to a service \"NET\", but no such service \
is defined. Did you mean \"net\"?",
]
);
}

#[test]
fn did_you_mean_suggests_only_close_names() {
let names = ["main", "internet", "storage"];
assert_eq!(did_you_mean("mian", names), " Did you mean \"main\"?");
assert_eq!(did_you_mean("MAIN", names), " Did you mean \"main\"?");
assert_eq!(
did_you_mean("interet", names),
" Did you mean \"internet\"?"
);
assert_eq!(did_you_mean("database", names), "");
assert_eq!(did_you_mean("main", names), "");
assert_eq!(did_you_mean("x", []), "");
}

#[test]
fn lookup_of_a_leaf_with_an_entrypoint_is_reported() {
let services = leaf_services();
Expand Down
51 changes: 45 additions & 6 deletions src/workerd/server/server/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -893,6 +893,49 @@ impl<'a> WorkflowEngine<'a> {
}
}

/// The Levenshtein distance between `a` and `b`, ignoring ASCII case.
fn edit_distance(a: &str, b: &str) -> usize {
let a: Vec<u8> = a.bytes().map(|c| c.to_ascii_lowercase()).collect();
let b: Vec<u8> = b.bytes().map(|c| c.to_ascii_lowercase()).collect();
let mut prev: Vec<usize> = (0..=b.len()).collect();
let mut cur = vec![0; b.len() + 1];
for (i, &ca) in a.iter().enumerate() {
cur[0] = i + 1;
for (j, &cb) in b.iter().enumerate() {
let cost = usize::from(ca != cb);
cur[j + 1] = (prev[j + 1] + 1).min(cur[j] + 1).min(prev[j] + cost);
}
std::mem::swap(&mut prev, &mut cur);
}
prev[b.len()]
}

/// A hint for an error about a name that is not defined: ` Did you mean "x"?`, naming the closest
/// of `candidates`, or empty when none is close enough to be worth suggesting.
pub fn did_you_mean<'a>(name: &str, candidates: impl IntoIterator<Item = &'a str>) -> String {
let threshold = (name.len() / 3).max(2);
let best = candidates
.into_iter()
.filter(|candidate| *candidate != name)
.map(|candidate| (edit_distance(name, candidate), candidate))
.filter(|(distance, _)| *distance <= threshold)
.min_by_key(|(distance, _)| *distance);
best.map_or_else(String::new, |(_, candidate)| {
format!(" Did you mean \"{candidate}\"?")
})
}

/// The error for a designator whose service is not defined, with a hint naming the closest
/// service that is.
fn no_such_service(
context: &str,
target: &str,
services: &LinkedHashMap<String, Service>,
) -> String {
let hint = did_you_mean(target, services.keys().map(String::as_str));
format!("{context} refers to a service \"{target}\", but no such service is defined.{hint}")
}

/// What a `ServiceDesignator` resolves to, or the invalid-config stand-in after the error was
/// reported.
pub fn lookup_service(
Expand All @@ -907,9 +950,7 @@ pub fn lookup_service(
error_context: context,
} = designator;
let Some(service) = services.get(target) else {
report.error(format!(
"{context} refers to a service \"{target}\", but no such service is defined."
));
report.error(no_such_service(context, target, services));
return Rc::new(InvalidConfigChannel);
};
let props = configured_props(props_json.as_deref());
Expand Down Expand Up @@ -966,9 +1007,7 @@ pub fn lookup_actor_class(
error_context: context,
} = designator;
let Some(service) = services.get(target) else {
report.error(format!(
"{context} refers to a service \"{target}\", but no such service is defined."
));
report.error(no_such_service(context, target, services));
return Rc::new(InvalidConfigActorClass);
};
let props = configured_props(props_json.as_deref());
Expand Down
Loading
Loading