Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,11 @@ jobs:
with:
dotnet-version: '10.0.x'

# Perl is already on the image; Lua is not, and 5.4 is what the
# container has.
- name: install lua
run: sudo apt-get install --no-install-recommends --yes lua5.4

# The Java shim is a jar, so unlike every other runner's it has to exist
# before the suite can use it.
- name: make shims
Expand Down
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,9 +201,10 @@ testable:
prescript play scripts/33-dice.yaml --runner runners/ruby.yaml -- 33_Dice/ruby/dice.rb
```

Ruby, Python, Node, Java and .NET ship and take a tape; Go ships and can be
seeded but not taped. All are covered by tests. The Java shim is a javaagent
jar, so it has to be built once with `make shims` before its runner will work.
Ruby, Python, Node, Perl, Lua, Java and .NET ship and take a tape; Go ships and
can be seeded but not taped. All are covered by tests. The Java shim is a
javaagent jar, so it has to be built once with `make shims` before its runner
will work.

A seed makes one port repeatable, but the same seed gives each language a
different sequence, so one script still cannot serve them all. `--tape` closes
Expand Down
17 changes: 12 additions & 5 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,14 @@ ARG BASE_DIGEST=sha256:88200866dfff7ea7f5cbcb6ec7c8a701889efe6fe859fe64d6990e4b0
FROM --platform=linux/amd64 debian:bookworm-slim@${BASE_DIGEST}
ARG BASE_DIGEST

# Ruby and Python come from Debian stable, which is what pinning means for a
# distribution: a fixed series that takes security fixes and no new features
# for the life of the release. Go, Node, Java, .NET and the reference
# interpreter are not in it, so they are fetched from upstream and checked
# against a published checksum before anything is unpacked.
# Ruby, Python, Perl and Lua come from Debian stable, which is what pinning
# means for a distribution: a fixed series that takes security fixes and no new
# features for the life of the release. Lua's package registers /usr/bin/lua
# through alternatives, which is the name runners/lua.yaml starts.
#
# Go, Node, Java, .NET and the reference interpreter are not in it, so they are
# fetched from upstream and checked against a published checksum before
# anything is unpacked.
ARG GO_VERSION=1.24.13
ARG GO_SHA256=1fc94b57134d51669c72173ad5d49fd62afb0f1db9bf3f798fd98ee423f8d730
ARG NODE_VERSION=24.20.0
Expand All @@ -50,7 +53,9 @@ RUN apt-get update \
ca-certificates \
curl \
git \
lua5.4 \
make \
perl \
python3 \
ruby \
tzdata \
Expand Down Expand Up @@ -140,6 +145,8 @@ RUN { \
echo "dotnet=$(dotnet --version)"; \
echo "python=$(python3 --version | cut -d' ' -f2)"; \
echo "ruby=$(ruby --version | cut -d' ' -f2)"; \
echo "perl=$(perl -e 'print $^V')"; \
echo "lua=$(lua -v | cut -d' ' -f2)"; \
echo "vintbas=${VINTBAS_RELEASE}"; \
echo "locale=${LC_ALL}"; \
echo "timezone=${TZ}"; \
Expand Down
4 changes: 3 additions & 1 deletion docs/container.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ make docker_shell # look around
| | How |
| --- | --- |
| Base | `debian:bookworm-slim` by digest |
| Ruby, Python | Debian stable, a fixed series taking security fixes only |
| Ruby, Python, Perl, Lua | Debian stable, a fixed series taking security fixes only |
| Go, Node, Java | upstream tarballs, checked against a published SHA-256 |
| .NET | the SDK tarball, checked against the SHA-512 Microsoft publishes |
| vintbas | the same release asset the Makefile installs, checked the same way |
Expand Down Expand Up @@ -77,6 +77,8 @@ java=25.0.4.1
dotnet=10.0.400
python=3.11.2
ruby=3.1.2p20
perl=v5.36.0
lua=5.4.4
vintbas=vintbas-1.0.3-1
locale=C.UTF-8
timezone=UTC
Expand Down
43 changes: 32 additions & 11 deletions docs/determinism.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ prescript play scripts/33-dice.yaml --runner runners/ruby.yaml -- 33_Dice/ruby/d
| Ruby | `RUBYOPT=-r<shim>` | `rand`, `Kernel#rand` | yes |
| Python | `sitecustomize.py` on `PYTHONPATH` | the `random` module's functions | yes |
| Node | `NODE_OPTIONS=--require <shim>` | `Math.random` | yes |
| Perl | `PERL5OPT=-M<shim>` | `rand`, `srand` | yes |
| Lua | `LUA_INIT=@<shim>` | `math.random`, `math.randomseed` | yes |
| Java | `-javaagent:<jar>` rewriting call sites | `Math.random`, `Random`, `ThreadLocalRandom` | yes |
| .NET | a `System.Random` compiled in ahead of the real one | everything on `Random`, including `Shared` | yes |
| Go | `GODEBUG=randautoseed=0` | `math/rand`'s global functions | no |
Expand Down Expand Up @@ -104,6 +106,11 @@ program constructs for itself, or one that seeds itself deliberately:
- Ruby: `Random.new(...)`, `SecureRandom`
- Python: `random.Random(...)` instances, `secrets`, `numpy`
- Node: `crypto.randomBytes`, `crypto.getRandomValues`
- Perl: `Math::Random::MT` and the other CPAN generators, `/dev/urandom` read
directly, and anything drawing inside an XS module rather than through Perl's
own `rand`
- Lua: a generator a port builds from `math.randomseed`'s return values, and
RNGs from LuaRocks
- Java: `SecureRandom`, `SplittableRandom`, generators built by
`RandomGeneratorFactory`, and draws made from inside `java.base` rather than
from the port — `Collections.shuffle(list, rnd)` and `rnd.ints()` call
Expand All @@ -128,15 +135,25 @@ platform.

| Language | Mechanism | Why it is not here |
| --- | --- | --- |
| Kotlin | the Java agent, plus `kotlin.random` in its owner list | JVM already; not yet verified against a port |
| VB.NET | a Visual Basic twin of the C# shim | `Rnd` and `Randomize` live in a module rather than a class |
| MiniScript | a patched interpreter seeded from the environment | `rnd` is an intrinsic and the interpreter has no launch hook |
| Rust | `[patch.crates-io] getrandom` in `.cargo/config.toml` | build-time, not launch-time |
| Haskell | vendored `random` with `initStdGen = pure (mkStdGen n)` | build-time, not launch-time |
| C / C++ | `-include shim.h`, or a `rand.o` linked ahead of libc | build-time, not launch-time |

These are all the same problem: a compiled language's randomness is decided when
it is built, and a runner only gets to speak at launch. Java and .NET are in the
table above rather than in this one because both compile on the way to running,
which gives a runner a compilation to join; a Rust port arrives as a binary, or
as a build a runner would have to drive.
The bottom three are all the same problem: a compiled language's randomness is
decided when it is built, and a runner only gets to speak at launch. Java and
.NET are in the table above rather than in this one because both compile on the
way to running, which gives a runner a compilation to join; a Rust port arrives
as a binary, or as a build a runner would have to drive.

The top three are near misses of different kinds. Kotlin is JVM bytecode, so the
Java agent already rewrites the ports that reach for `java.util.Random`, and
`kotlin.random` is a line in the agent's owner list away. MiniScript is the
largest single set of ports in the corpus and the only one with no launch hook
at all — the move that fits is the one already made for the reference
interpreter, which is patched and built from `patches/` rather than hooked.

An untested runner is worse than a missing one — it makes a corpus look seeded
when it is not — so these are listed rather than guessed at.
Expand All @@ -162,14 +179,19 @@ underneath. The test fixtures show it plainly — one program, one seed:
| Ruby | `548 715 602` |
| Python | `844 757 420` |
| Node | `358 105 675` |
| Perl | `170 749 96` |
| Lua | `246 234 69` |
| Java | `730 240 637` |
| .NET | `358 105 675` |
| Go | `604 940 664` |

.NET agrees with Node because both shims substitute the same generator, for the
reason given above. Java's row is the JDK's own LCG, seeded the ordinary way.
That one agreement is a fact about two shims rather than a property anything
should lean on — a tape is how ports are made to agree on purpose.
Each row is the language's own generator, not a snapshot of one machine: Java's
is the LCG `java.util.Random` specifies exactly, Perl's is drand48, and Lua's is
the xoshiro256** it has used since 5.4 — checked against 5.4 and 5.5, which
agree. .NET agrees with Node because both shims substitute the same generator,
for the reason given above; that one agreement is a fact about two shims rather
than a property anything should lean on. A tape is how ports are made to agree
on purpose.

So a seed makes each port repeatable, and one expected-output file still cannot
serve them all. That is what a tape is for.
Expand All @@ -189,8 +211,7 @@ prescript play scripts/33-dice.yaml --runner runners/ruby.yaml \
loaded switches from seeding to replaying. Nothing else about the run changes,
and the port is still untouched.

The same tape through the Ruby, Python, Node, Java and .NET runners gives the
same transcript — which is the whole point, and is what
The same tape through every runner but Go's gives the same transcript — which is the whole point, and is what
`TestOneTapeGivesEveryLanguageTheSameNumbers` checks.

### Where the numbers come from
Expand Down
9 changes: 8 additions & 1 deletion internal/play/runner_integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,10 @@ import (
// one machine's: Ruby and Python both promise a reproducible Mersenne Twister,
// Go's is the fixed source it used before it started seeding itself, Java's is
// the linear congruential generator java.util.Random specifies exactly, and
// Node's is the generator in runners/node/random.js.
// Node's is the generator in runners/node/random.js. Perl's is drand48, and
// Lua's the xoshiro256** it has used since 5.4 -- checked against both 5.4 and
// 5.5, which agree, so the row belongs to the language rather than to whichever
// one is installed.
//
// .NET draws what Node draws, and that is not a mistake. Its shim replaces
// System.Random outright, so it has to bring a generator rather than seed one,
Expand All @@ -41,6 +44,8 @@ var seededRunners = []struct {
{runner: "go.yaml", program: "rng.go", drawn: "604 940 664"},
{runner: "java.yaml", program: "rng.java", drawn: "730 240 637"},
{runner: "dotnet.yaml", program: "rng.cs", drawn: "358 105 675"},
{runner: "perl.yaml", program: "rng.pl", drawn: "170 749 96"},
{runner: "lua.yaml", program: "rng.lua", drawn: "246 234 69"},
}

func repositoryPath(t *testing.T, parts ...string) string {
Expand Down Expand Up @@ -222,6 +227,8 @@ var tapedRunners = []struct {
{runner: "node.yaml", program: "rng.js"},
{runner: "java.yaml", program: "rng.java"},
{runner: "dotnet.yaml", program: "rng.cs"},
{runner: "perl.yaml", program: "rng.pl"},
{runner: "lua.yaml", program: "rng.lua"},
}

// referenceDraws is what the reference BASIC interpreter itself prints for
Expand Down
27 changes: 27 additions & 0 deletions runners/lua.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# How to start a Lua port, and how to make its randomness repeatable.
#
# prescript play dice.yaml --runner runners/lua.yaml
version: "0.8"
executable: lua

env:
# The standalone interpreter runs this before the program, so a port does not
# have to be edited to be made deterministic. The leading @ is what makes it a
# file to run rather than a chunk to compile. ${runnerDir} is where this file
# lives, so the runner and its shim travel together whatever directory
# prescript is run from.
#
# Lua looks for LUA_INIT_5_4 (or _5_5, and so on) before falling back to this
# name, so a port run under a version-specific variable set by something else
# would not see the shim -- but nothing sets those by default.
LUA_INIT: "@${runnerDir}/lua/random.lua"

# One variable seeds every language. Two ports of the same program still draw
# different numbers from it -- that is what a tape fixes, not a seed -- but
# each of them draws the same numbers every time.
PRESCRIPT_SEED: "0"

# A lua installed by a version manager, and any C module a port loads, are both
# found through the environment. Pinning the environment is a container's job
# rather than a runner's.
inheritEnv: true
104 changes: 104 additions & 0 deletions runners/lua/random.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
-- Makes a Lua port's randomness repeatable, in one of two ways.
--
-- With PRESCRIPT_TAPE set, math.random returns values recorded from the
-- reference BASIC interpreter, in order, so this port and every other port of
-- the same program draw the same numbers. Without it, Lua's own generator is
-- seeded from PRESCRIPT_SEED, which makes this port repeatable but leaves it
-- drawing a different sequence from every other language. See
-- docs/determinism.md.
--
-- Activated by LUA_INIT=@<this file>, which the standalone interpreter runs
-- before the program, so the program does not have to know it exists and does
-- not have to be edited. Lua looks for a version-specific LUA_INIT_5_4 first
-- and falls back to LUA_INIT, so the plain name works whichever 5.x is
-- installed.
--
-- What this does not reach: a generator a port builds for itself with
-- math.randomseed's return values or an RNG from LuaRocks, and os.time or
-- os.clock used as a value rather than as a seed.

local tape = os.getenv("PRESCRIPT_TAPE")

if not tape or tape == "" then
local seed = tonumber(os.getenv("PRESCRIPT_SEED") or 0) or 0
math.randomseed(math.tointeger(seed) or 0)
return
end

local values = {}
for line in io.lines(tape) do
-- A fresh local rather than the loop variable, which 5.5 makes const.
local trimmed = line:match("^%s*(.-)%s*$")
if trimmed ~= "" and trimmed:sub(1, 1) ~= "#" then
values[#values + 1] = tonumber(trimmed)
end
end

local drawn = 0

-- How much of the tape a port used is the number that separates a real logic
-- bug from a port that restructured its draws, so it is written down rather
-- than left to be inferred from the transcript.
--
-- The count is rewritten after every draw rather than written once at exit,
-- because the number that matters is how much had been drawn when the run
-- stopped agreeing with the transcript -- and a run that diverged by hanging
-- is killed, which runs no exit handler at all. Rewriting in place is safe
-- without truncating: the count only ever grows, so its decimal form only ever
-- gets longer, and each write covers the last.
local usage_path = os.getenv("PRESCRIPT_TAPE_USAGE")
local usage = nil
if usage_path and usage_path ~= "" then
usage = io.open(usage_path, "w")
if usage then
usage:setvbuf("no")
end
end

local function write_usage()
if not usage then
return
end
usage:seek("set", 0)
usage:write(drawn, "\n")
end

-- Written from the start, so that a port which drew nothing at all says so
-- rather than saying nothing.
write_usage()

-- Running off the end is an error rather than a wrap. A port that draws more
-- than the reference did has restructured how it consumes randomness, which is
-- a finding; silently starting the tape again would turn that finding into a
-- transcript that merely looks wrong somewhere further on.
local function next_value()
if drawn >= #values then
io.stderr:write(("prescript: the tape ran out after %d values; this port " ..
"draws more randomness than the reference did\n"):format(#values))
os.exit(1)
end

drawn = drawn + 1
local value = values[drawn]
write_usage()
return value
end

-- math.random's three shapes, in terms of one uniform value: no argument for a
-- float in [0,1), one for an integer in [1,m], two for an integer in [m,n].
math.random = function(m, n)
local value = next_value()

if m == nil then
return value
end

if n == nil then
m, n = 1, m
end

return m + math.floor(value * (n - m + 1))
end

-- A port that seeds itself is asking for a sequence it has already been given.
math.randomseed = function() end
25 changes: 25 additions & 0 deletions runners/perl.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# How to start a Perl port, and how to make its randomness repeatable.
#
# prescript play dice.yaml --runner runners/perl.yaml
version: "0.8"
executable: perl

env:
# Perl loads this before it compiles the program, so a port does not have to
# be edited to be made deterministic -- and the ordering is not incidental:
# an override of rand has to be in place before the program's call sites are
# compiled, or they bind to the built-in. -I puts the shim's directory on the
# module path so -M can name it. ${runnerDir} is where this file lives, so
# the runner and its shim travel together whatever directory prescript is run
# from.
PERL5OPT: "-I${runnerDir}/perl -Mprescript"

# One variable seeds every language. Two ports of the same program still draw
# different numbers from it -- that is what a tape fixes, not a seed -- but
# each of them draws the same numbers every time.
PRESCRIPT_SEED: "0"

# A perl installed by perlbrew or plenv finds its own modules through the
# environment, and a port with dependencies resolves them through it too.
# Pinning the environment is a container's job rather than a runner's.
inheritEnv: true
Loading