Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,26 @@ jobs:
with:
go-version: ${{ matrix.go-version }}

# The runner image ships older versions of both. Java 25 to match the
# container and to clear the agent's floor of 24, where
# java.lang.classfile became standard; .NET 10 because a port is a single
# .cs file and `dotnet run` only accepts one from 10 onwards.
- name: setup java
uses: actions/setup-java@v5
with:
distribution: temurin
java-version: '25'

- name: setup dotnet
uses: actions/setup-dotnet@v5
with:
dotnet-version: '10.0.x'

# The Java shim is a jar, so unlike every other runner's it has to exist
# before the suite can use it.
- name: make shims
run: make shims

- name: make test
# Every language the shipped runners seed is present on this image, so
# a runner test that skips here is a runner nobody is checking. This
Expand Down
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,12 @@
# CPython writes this next to the seeding shim the moment a runner uses it.
__pycache__/

# The Java shim is built by `make shims` rather than checked in: a jar is a
# binary, and one built from sources in this repository is easier to trust than
# one that arrived with it.
runners/java/classes/
runners/java/prescript-agent.jar

# other
TODO.md
tmp/
Expand Down
19 changes: 17 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ VINTBAS_ASSET := vintbas-$(shell uname -s | tr '[:upper:]' '[:lower:]')-$(shell
VINTBAS_URL := https://github.com/rjnienaber/prescript/releases/download/$(VINTBAS_RELEASE)/$(VINTBAS_ASSET)

.PHONY: dependencies
dependencies: tools vintbas
dependencies: tools vintbas shims

# Build/lint tooling only. Kept separate from `vintbas` so CI can install
# what it needs without depending on an external download.
Expand All @@ -26,6 +26,21 @@ vintbas:
curl -fsSL $(VINTBAS_URL) -o ${HOME}/.local/bin/vintbas
chmod +x ${HOME}/.local/bin/vintbas

# The one runner shim that has to be compiled. Ruby, Python, Node and .NET all
# ship theirs as source the interpreter or the compiler reads on its way past;
# a JVM agent is a jar or it is nothing.
#
# --release 24 rather than whatever JDK is to hand, because the jar is built on
# a developer's machine and run in the container: 24 is where java.lang.classfile
# became standard, and is therefore both the floor the agent needs and the
# oldest JVM the jar should refuse to run on.
.PHONY: shims
shims:
javac --release 24 -Xlint:all -d runners/java/classes \
runners/java/prescript/Tape.java runners/java/prescript/Agent.java
jar --create --file runners/java/prescript-agent.jar \
--manifest runners/java/manifest.txt -C runners/java/classes .

# Re-records the tape of random values from the reference interpreter. Not
# wired into any other target on purpose: the tape is checked in because a
# recorded sequence is only useful when every port replays the same one, and
Expand Down Expand Up @@ -73,7 +88,7 @@ docker_shell:
# environment, and pinning a linter into the image would mean rebuilding it to
# upgrade one.
.PHONY: container_checks
container_checks: test build_dev examples
container_checks: shims test build_dev examples

.PHONY: format
format:
Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,9 @@ testable:
prescript play scripts/33-dice.yaml --runner runners/ruby.yaml -- 33_Dice/ruby/dice.rb
```

Ruby, Python, Node and Go ship and are covered by tests.
Ruby, Python, Node, Java and .NET ship and take a tape; Go ships and can be
seeded but not taped. All are covered by tests. The Java shim is a javaagent
jar, so it has to be built once with `make shims` before its runner will work.

A seed makes one port repeatable, but the same seed gives each language a
different sequence, so one script still cannot serve them all. `--tape` closes
Expand Down
51 changes: 46 additions & 5 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,24 @@ ARG BASE_DIGEST

# Ruby and Python come from Debian stable, which is what pinning means for a
# distribution: a fixed series that takes security fixes and no new features
# for the life of the release. Go, Node and the reference interpreter are not
# in it, so they are fetched from upstream and checked against a published
# checksum before anything is unpacked.
# for the life of the release. Go, Node, Java, .NET and the reference
# interpreter are not in it, so they are fetched from upstream and checked
# against a published checksum before anything is unpacked.
ARG GO_VERSION=1.24.13
ARG GO_SHA256=1fc94b57134d51669c72173ad5d49fd62afb0f1db9bf3f798fd98ee423f8d730
ARG NODE_VERSION=24.20.0
ARG NODE_SHA256=2f2c0da162318f0de47665410c7c8c2ed3d36c8f3105de4bbc61176c70a7cbf2
# Temurin rather than Debian's default-jdk, which is 17: the Java shim rewrites
# the port's call sites with java.lang.classfile, standard since 24. 25 is the
# long-term release above that floor. The tag carries a + that a URL cannot.
ARG JAVA_VERSION=25.0.4.1_1
ARG JAVA_TAG=jdk-25.0.4.1%2B1
ARG JAVA_SHA256=dbb698396d478e7fa2b1e50f4103324b2a99b90569ee27c33f2261f9215cf41e
# The SDK rather than the runtime: a .NET port arrives as source, and `dotnet
# run` compiles it. Checked against the sha512 Microsoft publishes beside the
# tarball, which is the checksum there is to check it against.
ARG DOTNET_VERSION=10.0.400
ARG DOTNET_SHA512=1033977dd837150e0814cf0c5d5b17ceb63925fda7ba2158b47258a4bd7c048cf82eac3bc1166f3146f53124a3f5fba09db1de1260d2ce96399860303b404b48
ARG VINTBAS_RELEASE=vintbas-1.0.3-1
ARG VINTBAS_SHA256=72dda1d16ed21f0d812f6978a32d105dadea49dd8163692b44c462678eb16a17

Expand Down Expand Up @@ -58,6 +69,28 @@ RUN curl -fsSL -o /tmp/node.tar.xz "https://nodejs.org/dist/v${NODE_VERSION}/nod
--exclude CHANGELOG.md --exclude LICENSE --exclude README.md \
&& rm /tmp/node.tar.xz

RUN curl -fsSL -o /tmp/jdk.tar.gz \
"https://github.com/adoptium/temurin25-binaries/releases/download/${JAVA_TAG}/OpenJDK25U-jdk_x64_linux_hotspot_${JAVA_VERSION}.tar.gz" \
&& echo "${JAVA_SHA256} /tmp/jdk.tar.gz" | sha256sum --check --strict - \
&& mkdir -p /usr/local/jdk \
&& tar --directory /usr/local/jdk --extract --file /tmp/jdk.tar.gz --strip-components 1 \
&& rm /tmp/jdk.tar.gz

# libicu is what the .NET runtime uses for anything culture-aware, and without
# it the host refuses to start rather than quietly falling back -- which is the
# right way round, since a program that formats a number differently depending
# on which library it found is exactly the churn this image exists to remove.
RUN apt-get update \
&& apt-get install --no-install-recommends --yes libicu72 \
&& rm -rf /var/lib/apt/lists/*

RUN curl -fsSL -o /tmp/dotnet.tar.gz \
"https://builds.dotnet.microsoft.com/dotnet/Sdk/${DOTNET_VERSION}/dotnet-sdk-${DOTNET_VERSION}-linux-x64.tar.gz" \
&& echo "${DOTNET_SHA512} /tmp/dotnet.tar.gz" | sha512sum --check --strict - \
&& mkdir -p /usr/local/dotnet \
&& tar --directory /usr/local/dotnet --extract --file /tmp/dotnet.tar.gz \
&& rm /tmp/dotnet.tar.gz

# The reference BASIC interpreter, patched to seed its RNG from VINTBAS_SEED.
# The same binary the Makefile installs, so a transcript recorded here and one
# recorded on a developer's machine came from the same interpreter.
Expand All @@ -66,7 +99,8 @@ RUN curl -fsSL -o /usr/local/bin/vintbas \
&& echo "${VINTBAS_SHA256} /usr/local/bin/vintbas" | sha256sum --check --strict - \
&& chmod +x /usr/local/bin/vintbas

ENV PATH=/usr/local/go/bin:/usr/local/bin:$PATH
ENV PATH=/usr/local/go/bin:/usr/local/jdk/bin:/usr/local/dotnet:/usr/local/bin:$PATH
ENV DOTNET_ROOT=/usr/local/dotnet

# The rest of what moves. Each of these has been a transcript diff at some
# point in somebody's life:
Expand All @@ -78,14 +112,19 @@ ENV PATH=/usr/local/go/bin:/usr/local/bin:$PATH
# PYTHONHASHSEED the iteration order of a set or a dict of strings
# GOTOOLCHAIN stops the go command downloading a different Go than the one
# pinned above, which would quietly unpin the image
# DOTNET_* the first `dotnet` command on a fresh machine prints a welcome
# banner and sends a telemetry ping, either of which would
# arrive in the middle of a transcript
ENV LANG=C.UTF-8 \
LC_ALL=C.UTF-8 \
TZ=UTC \
COLUMNS=80 \
LINES=24 \
PYTHONHASHSEED=0 \
PYTHONDONTWRITEBYTECODE=1 \
GOTOOLCHAIN=local
GOTOOLCHAIN=local \
DOTNET_NOLOGO=1 \
DOTNET_CLI_TELEMETRY_OPTOUT=1

# What a bug report has to carry, written from the binaries that were actually
# installed rather than from the arguments above, so it cannot claim a version
Expand All @@ -97,6 +136,8 @@ RUN { \
echo "arch=$(uname -m)"; \
echo "go=$(go version | cut -d' ' -f3)"; \
echo "node=$(node --version)"; \
echo "java=$(java --version | head -n 1 | cut -d' ' -f2)"; \
echo "dotnet=$(dotnet --version)"; \
echo "python=$(python3 --version | cut -d' ' -f2)"; \
echo "ruby=$(ruby --version | cut -d' ' -f2)"; \
echo "vintbas=${VINTBAS_RELEASE}"; \
Expand Down
13 changes: 13 additions & 0 deletions docker/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,17 @@
# tmp/ is not: `make build_dev` in here produces a linux binary, and dropping
# that on top of the one the host just built turns the next `make examples`
# into "cannot execute binary file".
#
# The cache volumes are only about speed. Go, and .NET more so, rebuild the
# world in a fresh container, and `dotnet run` on a file-based app compiles the
# program every time it starts -- which happens once per scripted run rather
# than once per suite. Nothing in them affects what a program prints.
#
# Not ~/.dotnet, which looks like the obvious one and breaks the SDK: it holds
# the first-run sentinel, and the first run is also what creates ~/.local/share.
# Persist the sentinel without persisting the directory it stood for and every
# later run skips the setup, finds no local application data folder, and dies
# with "Unable to determine a temporary directory path".
set -euo pipefail

IMAGE="${PRESCRIPT_IMAGE_NAME:-prescript-env:dev}"
Expand Down Expand Up @@ -62,6 +73,8 @@ exec docker run --rm ${INTERACTIVE[@]+"${INTERACTIVE[@]}"} ${SECCOMP[@]+"${SECCO
--mount type=volume,source=prescript-tmp,target=/workspace/tmp \
--mount type=volume,source=prescript-go-build,target=/root/.cache/go-build \
--mount type=volume,source=prescript-go-mod,target=/root/go/pkg/mod \
--mount type=volume,source=prescript-nuget,target=/root/.nuget/packages \
--mount type=volume,source=prescript-dotnet,target=/root/.local/share/dotnet \
--env PRESCRIPT_IMAGE="$DIGEST" \
--env PRESCRIPT_REQUIRE_RUNNERS=1 \
"$IMAGE" "$@"
5 changes: 4 additions & 1 deletion docs/container.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ make docker_shell # look around
| --- | --- |
| Base | `debian:bookworm-slim` by digest |
| Ruby, Python | Debian stable, a fixed series taking security fixes only |
| Go, Node | upstream tarballs, checked against a published SHA-256 |
| Go, Node, Java | upstream tarballs, checked against a published SHA-256 |
| .NET | the SDK tarball, checked against the SHA-512 Microsoft publishes |
| vintbas | the same release asset the Makefile installs, checked the same way |
| Locale | `LANG=LC_ALL=C.UTF-8` |
| Timezone | `TZ=UTC` |
Expand Down Expand Up @@ -72,6 +73,8 @@ base=debian:bookworm-slim@sha256:8820...
arch=x86_64
go=go1.24.13
node=v24.20.0
java=25.0.4.1
dotnet=10.0.400
python=3.11.2
ruby=3.1.2p20
vintbas=vintbas-1.0.3-1
Expand Down
71 changes: 54 additions & 17 deletions docs/determinism.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,32 @@ prescript play scripts/33-dice.yaml --runner runners/ruby.yaml -- 33_Dice/ruby/d
| Ruby | `RUBYOPT=-r<shim>` | `rand`, `Kernel#rand` | yes |
| Python | `sitecustomize.py` on `PYTHONPATH` | the `random` module's functions | yes |
| Node | `NODE_OPTIONS=--require <shim>` | `Math.random` | yes |
| Java | `-javaagent:<jar>` rewriting call sites | `Math.random`, `Random`, `ThreadLocalRandom` | yes |
| .NET | a `System.Random` compiled in ahead of the real one | everything on `Random`, including `Shared` | yes |
| Go | `GODEBUG=randautoseed=0` | `math/rand`'s global functions | no |

Java and .NET are the two that reach the program by getting in front of the
compiler rather than by setting a variable the runtime reads, because neither
runtime has a variable that would do.

The Java agent rewrites the port's own bytecode as each class loads: a call to
`Random.nextInt()` becomes a call to the shim, which returns the next tape value
and ignores the generator it was called on. It rewrites the port and leaves the
JDK alone, so no platform class is patched and no retransformation happens. The
agent is a jar, and the only shim that has to be built before it can be used:

```
make shims
```

The .NET shim is a source file declaring `namespace System { public class Random }`.
The C# compiler prefers a type it is compiling over the same type from a
reference, so the port's `new Random()` binds to the shim without the port
changing a line. `runners/dotnet.yaml` adds that file to the compilation with an
MSBuild targets file, which works the same for a single `.cs` file as for a
`.csproj`. It is the widest of the shims by accident of the mechanism: replacing
the class replaces `Shuffle`, `GetItems` and `Random.Shared` along with the draws.

Go seeds but cannot be taped. `GODEBUG` turns off the automatic seeding that
Go 1.20 introduced, which is enough to make one build repeatable, but there is
no exported way to put a different source behind `math/rand`'s package-level
Expand All @@ -80,32 +104,39 @@ program constructs for itself, or one that seeds itself deliberately:
- Ruby: `Random.new(...)`, `SecureRandom`
- Python: `random.Random(...)` instances, `secrets`, `numpy`
- Node: `crypto.randomBytes`, `crypto.getRandomValues`
- Java: `SecureRandom`, `SplittableRandom`, generators built by
`RandomGeneratorFactory`, and draws made from inside `java.base` rather than
from the port — `Collections.shuffle(list, rnd)` and `rnd.ints()` call
`nextInt` at a call site the agent does not rewrite
- .NET: `System.Security.Cryptography`, `Guid.NewGuid`, and any draw made
inside a pre-compiled dependency, which was compiled against the real
`Random` before the shim existed
- Go: `math/rand/v2`, which has no global seed and ignores `GODEBUG` — verified,
not assumed

A port using one of those needs either its own runner or a
[redaction](script-format.md#redactions) for the values it prints.

Node is the one replacement rather than a seeding: V8's `Math.random` cannot be
seeded — there is no API, and the engine takes entropy when a context is
created — so the shim substitutes mulberry32, small enough to read in one
sitting and identical on every platform.
Node and .NET are replacements rather than seedings. V8's `Math.random` cannot
be seeded — there is no API, and the engine takes entropy when a context is
created — and the .NET shim has replaced `Random` outright by the time the port
asks for one, so there is nothing left underneath to seed. Both substitute
mulberry32, small enough to read in one sitting and identical on every
platform.

### Not yet shipped

| Language | Mechanism | Why it is not here |
| --- | --- | --- |
| .NET | `DOTNET_STARTUP_HOOKS` | no toolchain to verify it against |
| Java | `-javaagent` instrumenting `Math.random` / `Random` | no toolchain to verify it against |
| Rust | `[patch.crates-io] getrandom` in `.cargo/config.toml` | build-time, not launch-time |
| Haskell | vendored `random` with `initStdGen = pure (mkStdGen n)` | build-time, not launch-time |
| C / C++ | `-include shim.h`, or a `rand.o` linked ahead of libc | build-time, not launch-time |

The first two are runners waiting for a machine to test them on. The last three
are a different problem: a compiled language's randomness is decided when it is
built, and a runner only gets to speak at launch. Those need the build to
cooperate, which is what makes the [pinned image](container.md) the place they
belong.
These are all the same problem: a compiled language's randomness is decided when
it is built, and a runner only gets to speak at launch. Java and .NET are in the
table above rather than in this one because both compile on the way to running,
which gives a runner a compilation to join; a Rust port arrives as a binary, or
as a build a runner would have to drive.

An untested runner is worse than a missing one — it makes a corpus look seeded
when it is not — so these are listed rather than guessed at.
Expand All @@ -122,18 +153,24 @@ It looks like the general answer and is not:

## What a seed does not buy

The same seed does not mean the same numbers. Ruby, Python, Node and Go each
draw a different sequence from `PRESCRIPT_SEED=0`, because each has a different
generator underneath. The test fixtures show it plainly — one program, one
seed, four answers:
The same seed does not mean the same numbers. Each language draws a different
sequence from `PRESCRIPT_SEED=0`, because each has a different generator
underneath. The test fixtures show it plainly — one program, one seed:

| Language | First three draws |
| --- | --- |
| Ruby | `548 715 602` |
| Python | `844 757 420` |
| Node | `358 105 675` |
| Java | `730 240 637` |
| .NET | `358 105 675` |
| Go | `604 940 664` |

.NET agrees with Node because both shims substitute the same generator, for the
reason given above. Java's row is the JDK's own LCG, seeded the ordinary way.
That one agreement is a fact about two shims rather than a property anything
should lean on — a tape is how ports are made to agree on purpose.

So a seed makes each port repeatable, and one expected-output file still cannot
serve them all. That is what a tape is for.

Expand All @@ -152,8 +189,8 @@ prescript play scripts/33-dice.yaml --runner runners/ruby.yaml \
loaded switches from seeding to replaying. Nothing else about the run changes,
and the port is still untouched.

The same tape through the Ruby, Python and Node runners gives the same
transcript — which is the whole point, and is what
The same tape through the Ruby, Python, Node, Java and .NET runners gives the
same transcript — which is the whole point, and is what
`TestOneTapeGivesEveryLanguageTheSameNumbers` checks.

### Where the numbers come from
Expand Down
Loading