Repository navigation
Generate Typescript types for Rust JSG Bindings - #8077
nathanwhit wants to merge 3 commits into
Conversation
|
✅ Approved — No issues found in the changes since the last review. One earlier informational note about the root AGENTS.md wording is still open and has not been repeated. The decision is advisory: this comment never approves, blocks or requests changes on the pull request. ✅ All checks passed🔒 Security: ✅ LGTMThis change only generates types. It adds no paths for untrusted input and does not change authentication or authorization. 📖 AGENTS.md: ✅ LGTMNo new issues. One note from earlier reviews is still open and is not repeated here: the root 📋 Code Quality: ✅ LGTMNo correctness, error-handling or test-coverage problems in
⚡ Performance: ✅ LGTM
📝 Documentation: ✅ LGTMThe README and Reviewed commit: |
|
Since last review: 2 resolved, 0 still open, 0 new. Not re-run: kj-style (no author changes in their files since the last review) Reviewed commit: b8b3d091 · github run |
b8b3d09 to
3b8788e
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #8077 +/- ##
==========================================
- Coverage 69.25% 68.17% -1.08%
==========================================
Files 698 713 +15
Lines 182499 186136 +3637
Branches 29224 29284 +60
==========================================
+ Hits 126383 126907 +524
- Misses 39043 42133 +3090
- Partials 17073 17096 +23 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Resource::members adds each member to a registry, members(registry, flags), instead of returning a Vec<Member>, like the Registry parameter of C++ registerMembers. Generated member factories return `impl jsg::MemberDefinition`, which template creation builds into a Member; a hand-built Member is also a MemberDefinition. This prepares for RTTI: an RTTI registry can run the same registration code and collect type descriptions instead of callbacks, so TypeScript declarations follow the members a resource registers under a given set of compatibility flags.
Add jsg::TypeInfo, which describes a binding type's JavaScript representation in the shared rtti.capnp schema without an isolate. #[jsg_resource], #[jsg_struct] and #[jsg_oneof] generate it; built-in conversion types implement it by hand, as custom conversion types must. A resource is described with the members it registers. Member factories return a definition that can both build the member and describe it, and RTTI runs Resource::members with its own registry under the compatibility flags of the jsg::rtti::Context, as C++ runs registerMembers with the RTTI builder's flags. Template creation never runs the descriptions, and RTTI never builds callbacks. As in C++, a nested type is described with its whole structure, through which the generator reaches it, and inspect properties, wildcard properties and other hand-built members have no description. The Context interns named types by TypeId before describing their members, so shared and recursive types are emitted once. Their type names identify them to the generator, so two types with one type name are rejected. Declaration names follow `name = "..."` separately, and ts_root marks a root of the ambient declarations, like JSG_TS_ROOT. Parameter names are camelCased, lose unused-parameter underscores and escape TypeScript reserved words; parameters whose names collide are rejected. Cover the mappings with an end-to-end test that runs a Rust fixture's RTTI through the types/ generator and type-checks the result. The fixture runs as test data in the target configuration, to avoid building V8 again as an exec-configuration tool. Teach the formatter to cover Rust sources under types/.
The RTTI export could not describe Rust builtin modules: they register an opaque callback, and its EncoderModuleRegistryImpl overload for module callbacks does nothing. Have module callbacks return their exports resource, jsg::Result<Rc<T>>, which registration converts, so the exports type is known statically. Registration passes the C++ registry a second function pointer next to the callback, which describes T in a jsg::rtti::Context. The RustModuleRegistry adapter hands it to registries that define addRustBuiltinModule, which only the workerd:rtti encoder does, and runtime registries ignore it. The encoder thus collects every module's types in one Context, under the export's compatibility flags, without running module code, and merges the encoded groups and modules with the C++ ones. Callbacks still capture nothing, and an error one returns is still thrown to the importer. api-rtti-test checks that the export discovers node-internal:dns, its exports type and the structs that type returns.
3b8788e to
a884f68
Compare
Motivation
@cloudflare/workers-typesis generated from the RTTI export (workerd:rtti), which only knows about C++ bindings. Rust modules register a callback the exporter can't inspect, and its overload for them is aTODOthat does nothing. As more APIs are written in Rust, they need to show up in the generated types like C++ ones do.Design
1. Members are added to a registry
To describe a resource, the RTTI export needs the list of members that the resource's JavaScript class actually gets.
That list comes from
Resource::members, which returns aVec<Member>. That doesn't work for RTTI, for two reasons:A
Memberhas no types. It holds what template creation needs: a name and the callbacks V8 calls. There isnothing in it to describe in TypeScript.
The list is chosen at runtime. With
custom_members,membersis hand-written code that can choose members bycompatibility flag:
A separate, macro-generated description couldn't see these choices. It would have to copy them, and would drift from
the real class when the code changes.
So RTTI has to run
membersitself, andmembershas to produce something RTTI can describe.Resource::membersnowadds each member to a registry instead of returning a list, like the
Registryparameter of C++registerMembers:Template creation and the RTTI export run the same
members, each with its own registry. Template creation keeps thecallbacks and never builds a type description; RTTI keeps the type descriptions and never builds a callback. So the
TypeScript declarations match what each resource registers under the export's compatibility flags, including choices like
those above.
Generated member factories (
Self::foo_member()) now returnimpl jsg::MemberDefinitioninstead ofjsg::Member. A definition can produce the member's callbacks or its type description, whichever the registry asks for. Ajsg::Memberbuilt by hand can still be passed to
registry.add. RTTI describes a hand-built nested type through the type it nests,and omits other hand-built members, which have no description.
2.
TypeInfodescribes Rust typesjsg::TypeInfodescribes how a Rust type looks in JavaScript, using the existingrtti.capnpschema and without an isolate.#[jsg_resource],#[jsg_struct]and#[jsg_oneof]generate the implementation; built-in and custom conversion types implement it by hand. Thetypes/generator is unchanged.Option<T>T | undefined(optional)jsg::Nullable<T>T | null(input also readsundefinedasnull; the parameter is required)Option<jsg::Nullable<T>>,jsg::Lenient<T>T | null | undefined(optional)Vec<T>,&[T]T[](numbers included)jsg::Dict<V>Record<string, V>v8::Local<v8::Uint8Array>and other typed arraysUint8Array, etc.Result<T, E>TSystemTimeDate(arguments also acceptnumber)v8::Local<v8::Date>Datei64,u64,v8::Local<BigInt>number | bigint(as C++ describesint64_t)v8::Local<Promise>Promise<any>(arguments also accept plain values, as in C++)name = "..."sets the TypeScript name of a method, property, struct or resource. References between types use the full Rust type name, so renaming a type doesn't break them; two types with the same full name are rejected. Each type is recorded before its members are described, so shared and self-referencing types appear once.thisparameter and a&mut Lockparameter are left out. Parameter names are converted to camelCase, lose the leading_of unused parameters, and escape TypeScript reserved words. Two parameters that end up with the same name are a compile error.nested_types(...), or a hand-builtMember::NestedType) is described with its whole structure, so it is declared as a class even if nothing else refers to it. Inspect properties, wildcard properties and other hand-built members have no description, also as in C++.ts_rootmatchesJSG_TS_ROOT. A type only appears in the ambient declarations if it can be reached from a root.3. Module types are included in the export
To describe a module, the exporter needs its exports type without running the module. So module callbacks now return the exports resource, and registration converts it to JS:
Callbacks still capture nothing, and an error a callback returns is still thrown to the importer.
Registration passes the C++ registry a second function pointer next to the callback, which describes the exports type.
RustModuleRegistryuses anif constexprto check whether the wrapped registry hasaddRustBuiltinModule; only the exporter does.Commits
MemberRegistry. Runtime only; contains the API change.TypeInfo.Not in this PR
JSG_TS_OVERRIDE/JSG_TS_DEFINEequivalents: a follow-up PR.Testing
rtti-test.rs: shared and recursive types are recorded once, modules share one set of types, custommembersregistration is followed under different compatibility flags, a hand-built nested type is described with its whole structure, and two types with one name are rejected.types/test/generator/rust.spec.ts: a Rust fixture covering the mappings above is run through the generator, and the resulting declarations are type-checked with TypeScript, including that aNullableargument is required and anOption<Nullable>one is optional, and that a renamed nested type reachable only through nesting is declared, along with a struct only it refers to. A flag-dependent resource is exported with two sets of flags. The fixture is built as test data rather than as a build tool, so V8 isn't built a second time for the host platform.api-rtti-test: the real export includesnode-internal:dns, its exports typeDnsUtil, and theCaaRecordstruct it returns.@cloudflare/workers-typesis unchanged, since no Rust type is a TypeScript root yet.