diff --git a/README.md b/README.md index e924d01..6623025 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,82 @@ # terrain-diffusion -A reproduction of the InfiniteDiffusion paper: generating realistic landscape terrain that continues forever in every direction, comes out the same every time from the same seed, and uses a fixed amount of memory no matter how far you explore. +A reproduction of the [InfiniteDiffusion paper](https://arxiv.org/abs/2512.08309). +A game world like Minecraft lets a player walk in any direction forever, but the terrain looks blocky and artificial. +An AI image generator can paint a realistic landscape, but only one fixed size image, not an endless world. + +This project produces a realistic, endless landscape from a seed. +A user chooses a seed, which is a number that fully determines the world, and explores the terrain in a 3D view. +The land has sharp ridges, river valleys, and coastlines, and it continues in every direction without end. + +Two properties define it: + +- Seamless and infinite. However far a user travels, the land lines up and never repeats. +- Constant memory. Exploring a large area uses no more memory than exploring a small one, because terrain is created only where the user is looking. + +The same seed always recreates the same world. + +The models are already trained by the paper's authors. Training terrain models of this kind costs thousands of dollars and takes months, so this project does not train them. It uses the released weights and rebuilds the algorithm that runs on top of them. Running a model needs about 2.2 GB of GPU memory, so a free cloud GPU or a gaming laptop is enough. + +## What we are building first + +The MVP: + +- As a user, I can give a seed and a region size and receive a height map image. +- As a user, I can give a seed and a region size of more than one tile and get terrain whose tiles meet with no visible seams. +- As a user, I can give the same seed twice and get exactly the same terrain both times. +- As a user, I can give a seed and a region and view the result as a 3D surface. + +## How the system fits together + +The system answers one question: what does the world look like at a given seed and coordinate. +It turns the answer into images and an interactive 3D view. + +```mermaid +graph TD + CLI["Output and CLI"] --> ORCH["Generation Orchestration"] + VIZ["3D Visualizer"] --> ORCH + ORCH --> SAMP["Windowed Blending Sampler"] + SAMP --> PIPE["Model Pipeline"] + SAMP --> STORE["Terrain Store"] + PIPE --> MODEL["Model Inference"] + PIPE --> ENC["Elevation Encoding"] + BENCH["Benchmarking and Evaluation"] --> ORCH + MODEL -. loads .-> WEIGHTS["Pretrained model weights"] + MODEL -. runs on .-> GPU["GPU compute node"] + STORE -. optional .-> DISK["Local disk cache"] +``` + +Which features live where: + +- Model Inference runs a single neural model on a patch and holds the GPU and weight concerns. +- The Model Pipeline composes the core model, the decoder, and the elevation transforms into one finished full resolution patch. +- Elevation Encoding holds the reversible transforms the real models require. +- The Windowed Blending Sampler holds the core generation math. +- The Terrain Store holds memory behaviour, from bounded to lazy to infinite. +- Generation Orchestration drives a region, and in the build-out coordinates the multi scale hierarchy. +- Output and CLI and the 3D Visualizer hold user facing output. +- Benchmarking and Evaluation holds paper comparison. + +## Where the code lives + +Each component has one module. None of them are implemented yet, they hold the plan for that component and nothing else. Open the module to see what it is responsible for, who it talks to, and what has to be built. + +| Component | Where it lives | +| --- | --- | +| Model Inference | `src/terrain_diffusion/inference.py` | +| Elevation Encoding | `src/terrain_diffusion/encoding.py` | +| Model Pipeline | `src/terrain_diffusion/pipeline.py` | +| Windowed Blending Sampler | `src/terrain_diffusion/sampler.py` | +| Terrain Store | `src/terrain_diffusion/store.py` | +| Generation Orchestration | `src/terrain_diffusion/orchestration.py` | +| Output and CLI | `src/terrain_diffusion/cli.py` | +| Benchmarking and Evaluation | `src/terrain_diffusion/benchmark.py` | +| 3D Visualizer | `web/` | + +Python tests go in `tests/` and web tests go in `web/tests/`. The scripts that run the checks are in `scripts/`. + +A module can grow into a folder later, by turning `sampler.py` into `sampler/__init__.py`, without changing how anything imports it. ## Getting set up @@ -19,6 +94,38 @@ scripts/install-dependencies.sh > You do not need to install python, uv handles and maintains the correct python version. +Check it worked: + +```bash +uv run terrain-diffusion +``` + +That runs `main()` in `src/terrain_diffusion/cli.py`, which for now only prints that there is nothing to run yet. + +## Skills you will pick up + +A developer who builds this project uses: + +- Overlapping window diffusion sampling +- Few step image generation +- Lazy and constant memory data structures +- 3D terrain rendering +- Reproducing and benchmarking a research paper + +Helpful familiarity before starting: + +- Python and numeric arrays, used throughout the project +- PyTorch and the idea of a diffusion cleaning step, for the model facing parts +- Reading a dense research paper + +## Reference material + +- The paper InfiniteDiffusion, arXiv:2512.08309, at https://arxiv.org/abs/2512.08309 +- The authors' project site +- The authors' open source library for the lazy generation idea + +The reference code may be read to understand the method. The implementation here is built from scratch. The goal is to understand and rebuild the method, not to fork it. + ## Contributing See [CONTRIBUTING.md](CONTRIBUTING.md) for how to run the checks, how tests are organised, and how changes get reviewed and merged. @@ -26,4 +133,4 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for how to run the checks, how tests are ## TODO - Add end to end tests with [Playwright](https://playwright.dev/) once the visualizer draws something. The current tests run in jsdom, which has no canvas and no WebGL, thus cannot check what appears on screen -- Playwright simulates a real browser and can compare screenshots \ No newline at end of file +- Playwright simulates a real browser and can compare screenshots diff --git a/pyproject.toml b/pyproject.toml index 73d430f..fcc0f72 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -10,16 +10,28 @@ requires-python = ">=3.14" # then run `uv sync` to install and update uv.lock dependencies = [] +# The commands the project installs. `uv run terrain-diffusion` runs main() in +# src/terrain_diffusion/cli.py. +[project.scripts] +terrain-diffusion = "terrain_diffusion.cli:main" + [dependency-groups] dev = [ "pytest>=8.3", "ruff>=0.9", ] -[tool.uv] -# TODO: When the first package is created under src/, delete this line and add a -# [build-system] section so the package can be installed. -package = false + +# --------------------------------------------------------------------------- +# packaging: how the code under src/ becomes an installable package +# --------------------------------------------------------------------------- + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.hatch.build.targets.wheel] +packages = ["src/terrain_diffusion"] # --------------------------------------------------------------------------- diff --git a/src/terrain_diffusion/__init__.py b/src/terrain_diffusion/__init__.py new file mode 100644 index 0000000..e77e82d --- /dev/null +++ b/src/terrain_diffusion/__init__.py @@ -0,0 +1,13 @@ +"""A reproduction of the InfiniteDiffusion paper. + +One module per component: + + inference Model Inference + encoding Elevation Encoding + pipeline Model Pipeline + sampler Windowed Blending Sampler + store Terrain Store + orchestration Generation Orchestration + cli Output and CLI + benchmark Benchmarking and Evaluation +""" diff --git a/src/terrain_diffusion/benchmark.py b/src/terrain_diffusion/benchmark.py new file mode 100644 index 0000000..762e92a --- /dev/null +++ b/src/terrain_diffusion/benchmark.py @@ -0,0 +1,11 @@ +"""Benchmarking and Evaluation + +Overview + +Measures the reproduction against the paper's reported results. + +Neighbours and communication + +- Drives Generation Orchestration and Output to generate sample terrain. +- Measures quality, speed, and memory. +""" diff --git a/src/terrain_diffusion/cli.py b/src/terrain_diffusion/cli.py new file mode 100644 index 0000000..cb77ece --- /dev/null +++ b/src/terrain_diffusion/cli.py @@ -0,0 +1,27 @@ +"""Output and CLI + +Overview + +The command line entry point and the code that turns a finished height grid into files. + +Neighbours and communication + +- Takes a user request. +- Asks Generation Orchestration for the region. +- Writes the result to disk. + +Run this file's main() with `uv run terrain-diffusion`. The command is wired up in +pyproject.toml under [project.scripts]. +""" + +import sys + + +def main() -> int: + """Placeholder, so the command runs before any of the project is built.""" + print("terrain-diffusion is not built yet. See README.md for what goes here.") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/src/terrain_diffusion/encoding.py b/src/terrain_diffusion/encoding.py new file mode 100644 index 0000000..9cb2749 --- /dev/null +++ b/src/terrain_diffusion/encoding.py @@ -0,0 +1,21 @@ +"""Elevation Encoding + +Overview + +A set of reversible transforms between real elevations and the form the models work in. +Real elevations span a very large range, from deep ocean trenches to high mountains, and the +models were trained on compressed and split elevations rather than raw ones. +The forward direction compresses the range and splits a height grid into a base layer and a +detail layer. The reverse direction reassembles the base and detail and undoes the compression +to recover real elevations. +Generation uses the reverse direction: the Model Pipeline turns the models' base and detail +outputs back into real elevations. The forward direction is its exact inverse, used to define +the transforms and to confirm they round trip. + +Neighbours and communication + +- The Model Pipeline calls the reverse direction to turn the models' base and detail outputs + into real elevations. +- The transforms are reversible, so applying the forward direction and then the reverse returns + the original within a small tolerance. +""" diff --git a/src/terrain_diffusion/inference.py b/src/terrain_diffusion/inference.py new file mode 100644 index 0000000..948f8d1 --- /dev/null +++ b/src/terrain_diffusion/inference.py @@ -0,0 +1,18 @@ +"""Model Inference + +Overview + +Runs a single neural model on a patch. It is the only component that loads model weights +and uses the GPU. +Given a patch, a choice of which model to run, and any conditioning, it runs that one model +and returns its raw output. +Different models return different things. The core model returns a low resolution elevation +summary and a latent map. The decoder returns a full resolution grid. Composing these into a +finished patch is the Model Pipeline's job, not this component's. + +Neighbours and communication + +- The Model Pipeline sends a patch and a choice of model and receives that model's raw output. +- It loads weights from the external model weights download. +- It runs on the GPU compute node. +""" diff --git a/src/terrain_diffusion/orchestration.py b/src/terrain_diffusion/orchestration.py new file mode 100644 index 0000000..ab2bdc1 --- /dev/null +++ b/src/terrain_diffusion/orchestration.py @@ -0,0 +1,14 @@ +"""Generation Orchestration + +Overview + +Produces a finished region for a given seed and coordinate by driving the other core components. +In the build-out it also coordinates several models at different zoom levels. + +Neighbours and communication + +- Receives requests from Output and CLI and from the 3D Visualizer. +- Drives the Windowed Blending Sampler. +- Reads finished height grids from the Terrain Store. +- In the build-out it also runs the coarse model to produce conditioning for a region. +""" diff --git a/src/terrain_diffusion/pipeline.py b/src/terrain_diffusion/pipeline.py new file mode 100644 index 0000000..b098397 --- /dev/null +++ b/src/terrain_diffusion/pipeline.py @@ -0,0 +1,21 @@ +"""Model Pipeline + +Overview + +Turns an uncleaned patch into a finished full resolution elevation patch, and hides how that is +done from the rest of the system. +It exposes one operation: given an uncleaned patch and any conditioning, return a cleaned patch +of the same size. Behind that operation it runs whatever models are needed and applies the +elevation transforms, so callers never see latent maps or model internals. +For the first working version it can be a stand-in that just smooths the patch. For real terrain +it runs the core model to get a latent map and a low resolution summary, then the decoder to +produce the detail layer, then reassembles the base and detail into real elevations with +Elevation Encoding. + +Neighbours and communication + +- The Windowed Blending Sampler sends an uncleaned patch and any conditioning and receives a + cleaned patch of the same size. +- It asks Model Inference to run a named model on a patch. +- It asks Elevation Encoding to turn the models' base and detail outputs into real elevations. +""" diff --git a/src/terrain_diffusion/sampler.py b/src/terrain_diffusion/sampler.py new file mode 100644 index 0000000..e85f0e5 --- /dev/null +++ b/src/terrain_diffusion/sampler.py @@ -0,0 +1,12 @@ +"""Windowed Blending Sampler + +Overview + +The algorithm that turns a fixed size model into a generator of any size with no seams. + +Neighbours and communication + +- Receives a request for a region from Generation Orchestration. +- Asks the Model Pipeline to clean patches. +- Writes results into the Terrain Store and reads them back. +""" diff --git a/src/terrain_diffusion/store.py b/src/terrain_diffusion/store.py new file mode 100644 index 0000000..0db068c --- /dev/null +++ b/src/terrain_diffusion/store.py @@ -0,0 +1,16 @@ +"""Terrain Store + +Overview + +Holds the height grids for tiles being generated and finished. +Its behaviour is what takes the project from a bounded picture to an infinite world in constant +memory. +For a bounded region the sum and weight buffers cover the whole region. For infinite exploration +they are held per tile, so each tile can be finished, cached, evicted, and recomputed on its own. + +Neighbours and communication + +- The Windowed Blending Sampler writes window contributions into it and reads them back. +- Generation Orchestration reads finished height grids from it. +- It can persist tiles to local disk. +""" diff --git a/tests/test_package.py b/tests/test_package.py new file mode 100644 index 0000000..2058838 --- /dev/null +++ b/tests/test_package.py @@ -0,0 +1,38 @@ +"""Checks the package skeleton is installed and importable. + +These tests do not check any terrain behaviour, there is none yet. They check +that `src/terrain_diffusion` is actually installed into the environment, so a +broken packaging setup is caught here rather than in someone's first import. +""" + +import importlib + +import pytest + +COMPONENT_MODULES = [ + "terrain_diffusion.inference", + "terrain_diffusion.encoding", + "terrain_diffusion.pipeline", + "terrain_diffusion.sampler", + "terrain_diffusion.store", + "terrain_diffusion.orchestration", + "terrain_diffusion.cli", + "terrain_diffusion.benchmark", +] + + +@pytest.mark.parametrize("name", COMPONENT_MODULES) +def test_component_module_imports(name: str) -> None: + """Every component from the project plan has a module and it imports. + + If this fails, run `uv sync`. It installs the project into the environment. + """ + assert importlib.import_module(name) is not None + + +def test_the_command_line_entry_point_runs(capsys) -> None: + """`uv run terrain-diffusion` calls this, so it should return 0 and say something.""" + from terrain_diffusion.cli import main + + assert main() == 0 + assert capsys.readouterr().out.strip() != "" diff --git a/uv.lock b/uv.lock index bdf91b4..fe691f7 100644 --- a/uv.lock +++ b/uv.lock @@ -91,7 +91,7 @@ wheels = [ [[package]] name = "terrain-diffusion" version = "0.1.0" -source = { virtual = "." } +source = { editable = "." } [package.dev-dependencies] dev = [