Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/reuse-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,14 +73,14 @@ jobs:
echo "OK: $f"
done

- name: CONTRIBUTING.md references the DCO / sign-off
- name: Contribution guide retains DCO policy
run: |
set -euo pipefail
if ! grep -qE "Developer.{1,40}Certificate.{1,10}of.{1,10}Origin|Signed-off-by|sign-off" CONTRIBUTING.md; then
echo "::error::CONTRIBUTING.md does not reference DCO / sign-off"
exit 1
fi
echo "OK: CONTRIBUTING.md references DCO / sign-off"
echo "OK: CONTRIBUTING.md retains DCO policy"

# Inline SPDX-License-Identifier on first-party source files.
# REUSE.toml provides aggregate coverage so 'reuse lint' is green even
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ Before performance work, collect an nsys report with `uv run flashdreams-profile

## Testing Guidance

Every pytest test must carry exactly one of `ci_cpu`, `ci_gpu`, or `manual`; `CONTRIBUTING.md` has the exact rules, including how pytest discovers files and functions (`test_*.py`, `test_*`). Use module-level `pytestmark = pytest.mark.ci_cpu` for pure Python/metadata tests. Keep GPU, `libGL`/`cv2`, large-checkpoint, credential, and download-heavy checks out of `ci_cpu`.
Every pytest test must carry exactly one of `ci_cpu`, `ci_gpu`, or `manual`; [Testing](https://nvidia.github.io/flashdreams/main/documentation/tools/testing/) has the exact discovery and marker rules. Use module-level `pytestmark = pytest.mark.ci_cpu` for pure Python/metadata tests. Keep GPU, `libGL`/`cv2`, large-checkpoint, credential, and download-heavy checks out of `ci_cpu`.

**v2 test ownership** — put a new test next to the thing it validates:

Expand Down
171 changes: 65 additions & 106 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ We have intentionally kept the process light. If something below is
unclear or feels heavier than it should be, that's a bug; please file an
issue and we'll fix it.

Setup and build instructions:

- [install and run FlashDreams](https://nvidia.github.io/flashdreams/main/quickstart/);
- [work on one integration package](https://nvidia.github.io/flashdreams/main/quickstart/#install-one-integration);
- [build and preview the website](https://nvidia.github.io/flashdreams/main/documentation/tools/build_website/).

## Table of contents

1. [Ways to contribute](#ways-to-contribute)
Expand All @@ -19,13 +25,14 @@ issue and we'll fix it.
4. [Submitting a pull request](#submitting-a-pull-request)
5. [Code review and merge](#code-review-and-merge)
6. [File Tree Of FlashDreams](#file-tree-of-flashdreams)
7. [Coding conventions](#coding-conventions)
8. [Testing](#testing)
9. [Dependency version bounds](#dependency-version-bounds)
10. [Working with a single integration package](#working-with-a-single-integration-package)
11. [Licensing of contributions](#licensing-of-contributions)
12. [Reporting issues](#reporting-issues)
13. [Code of Conduct](#code-of-conduct)
7. [Adding documentation](#adding-documentation)
8. [Coding conventions](#coding-conventions)
9. [Testing](#testing)
10. [Dependency version bounds](#dependency-version-bounds)
11. [Working with a single integration package](#working-with-a-single-integration-package)
12. [Licensing of contributions](#licensing-of-contributions)
13. [Reporting issues](#reporting-issues)
14. [Code of Conduct](#code-of-conduct)

## Ways to contribute

Expand All @@ -35,9 +42,9 @@ overhead" to "high overhead":
- **Try FlashDreams and tell us what broke.** A clear bug report — what
you ran, what you expected, what you saw — is one of the most valuable
contributions a project of this kind can receive.
- **Improve documentation.** README clarifications, integration walkthroughs,
performance notes, and FAQ entries all land easily and benefit every
future user.
- **Improve documentation.** Website articles, integration walkthroughs,
performance notes, and troubleshooting entries all land easily and benefit
every future user.
- **Fix bugs.** Issues labelled `good first issue` are a friendly
starting point. Larger fixes are welcome too — please leave a comment
on the relevant issue first so we can avoid duplicate work.
Expand Down Expand Up @@ -242,11 +249,13 @@ tracking issue where available.

## File Tree Of FlashDreams

Where a new app or model goes. Each documents its own layout:
Where a new app or model goes:

```text
apps/ # reusable apps; layout in apps/README.md
integrations_v2/ # model packages; layout in integrations_v2/README.md
apps/ # reusable, model-independent applications
integrations_v2/ # model packages and application adapters
docs/source/demos/ # canonical demo articles
docs/source/models/ # canonical model articles
```

The framework package, and the test and doc trees:
Expand Down Expand Up @@ -274,6 +283,46 @@ tests/ # repo-wide test-runner scripts + meta checks, n
docs/source/ # Zensical Markdown sources
```

## Adding documentation

Write documentation for users and contributors as a website article under
`docs/source/`, not as a package `README.md`. Update an existing article when
it already owns the topic. Otherwise, add a Markdown file in the section that
owns it:

| Article | Website source | Example |
| --- | --- | --- |
| Install or first run | `docs/source/quickstart/` | `docs/source/quickstart/index.md` |
| Model requirements, setup, commands, or limitations | `docs/source/models/` | `docs/source/models/my_model.md` |
| Demo controls, options, or behavior | `docs/source/demos/` | `docs/source/demos/my_demo.md` |
| API concept or development guide | `docs/source/documentation/<area>/guides/` | `docs/source/documentation/demo_api/guides/my_topic.md` |
| API reference | `docs/source/documentation/<area>/api_reference/` | `docs/source/documentation/inferencing_api/api_reference/my_api.md` |
| Project or community process | `docs/source/community/` | `docs/source/community/my_process.md` |
| Common user failure | `docs/source/troubleshooting.md` | Add a focused section to the existing page |

For example, a new demo article starts at `docs/source/demos/my_demo.md`:

```markdown
---
title: 'My Demo'
---

<!-- SPDX-FileCopyrightText: Copyright (c) 2026 YOUR NAME. All rights reserved. -->
<!-- SPDX-License-Identifier: Apache-2.0 -->

Explain what the demo does, how to launch it, its controls, and its options.
```

Register a new page in the `.nav.yml` beside it; for this example, add
`My Demo: my_demo.md` to `docs/source/demos/.nav.yml`. Use a lowercase
snake-case filename that matches the model or demo package when applicable.

Files outside `docs/source/` should link to the published website article
instead of copying its instructions. `README.md` outside website, should try to point to the canonical website as much as possible.

Build or preview the site with the commands in
[Build Website](https://nvidia.github.io/flashdreams/main/documentation/tools/build_website/#build-the-website).

## Coding conventions

- Python 3.10+. Type-annotate new code; the project type-checks with
Expand Down Expand Up @@ -303,65 +352,8 @@ docs/source/ # Zensical Markdown sources

## Testing

Pytest collection is configured in the root `pyproject.toml`. There is
no `testpaths` setting, so `pytest` from the repo root walks the tree:

- Files named `test_*.py`. Pytest's default also collects `*_test.py`;
don't use that name here.
- Classes named `Test*` (pytest default). Some modules use classes,
some only module-level functions.
- Functions and methods named `test_*` (pytest default), including
`async def test_*`.

Same test filename in different folders is fine (several
`test_application.py` files exist). Root `pyproject.toml` already sets
`--import-mode=importlib` so pytest keeps them separate. Do not remove
that flag.

A `test_*.py` next to the code it validates is collected with no CI
change. The root `pyproject.toml` skips `parity_check`,
`parity_check_v2`, `baseline_fastvideo`, and `baseline_lightx2v` via
`norecursedirs`.

Every test function must be marked with exactly one **CI tier marker**.
A pytest plugin (`flashdreams._pytest_plugins.marker_enforcement`)
enforces this at collection time -- tests without a marker are
rejected, and tests with both `ci_cpu` and `ci_gpu` are rejected.

| Marker | When to use | CI runner |
|--------|-------------|-----------|
| `@pytest.mark.ci_cpu` | Pure CPU logic, no GPU or `libGL` needed | CPU runner |
| `@pytest.mark.ci_gpu` | Needs CUDA, `libGL`, or transitive `cv2` | GPU runner (RTX Pro 6000) |
| `@pytest.mark.manual` | Heavy (OOM risk), flaky, needs credentials or large downloads | Not run in CI |

Use a module-level `pytestmark` when every test in a file shares the
same marker:

```python
import pytest

pytestmark = pytest.mark.ci_cpu
```

Use per-function markers when tests in the same file have different
tiers:

```python
@pytest.mark.ci_cpu
def test_basic_math(): ...

@pytest.mark.ci_gpu
def test_cudagraph_path(): ...
```

Running tests locally:

```bash
uv run pytest -m ci_cpu # CPU-safe tests only
uv run pytest -m ci_gpu # GPU tests only (needs CUDA)
uv run pytest -m "not manual" # everything that runs in CI
uv run pytest # all tests including manual
```
See [Testing](https://nvidia.github.io/flashdreams/main/documentation/tools/testing/) for test discovery,
CI tier markers, and commands.

## Dependency version bounds

Expand All @@ -382,40 +374,7 @@ the declared minimums. This means:

## Working with a single integration package

The workspace contains many integration packages under `integrations_v2/`.
A full `uv sync` installs dependencies for *all* of them. If you only
need one (e.g. you're working on `omnidreams`), use the distribution
package name with `--package` to sync only that package's dependencies:

```bash
# Only install omnidreams + its deps (skips unrelated heavy packages)
uv sync --package flashdreams-omnidreams --extra dev

# Run a script/test from that integration only
uv run --package flashdreams-omnidreams pytest integrations_v2/omnidreams/tests/ -m ci_gpu
```

This avoids pulling in (and compiling) dependencies that other
integrations require but yours does not, further reducing setup time.

Available integration packages:

| Path | `uv --package` name |
|------|---------------------|
| `integrations_v2/causal_forcing` | `flashdreams-causal-forcing` |
| `integrations_v2/cosmos_predict2` | `flashdreams-cosmos-predict2` |
| `integrations_v2/fastvideo_causal_wan22` | `flashdreams-fastvideo-causal-wan22` |
| `integrations_v2/flashvsr` | `flashdreams-flashvsr` |
| `integrations_v2/lingbot` | `flashdreams-lingbot` |
| `integrations_v2/omnidreams` | `flashdreams-omnidreams` |
| `integrations_v2/self_forcing` | `flashdreams-self-forcing` |
| `integrations_v2/swiftvr` | `flashdreams-swiftvr` |
| `integrations_v2/wan21` | `flashdreams-wan21` |
| `integrations_v2/wan22` | `flashdreams-wan22` |

The nested `integrations_v2/omnidreams/impl/ludus-renderer` workspace package is
named `ludus-renderer` and is installed as part of Omnidreams workflows
that need it.
See our [model gallery](https://nvidia.github.io/flashdreams/main/models/) page.

## Licensing of contributions

Expand Down
122 changes: 13 additions & 109 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,129 +22,31 @@ and has grown into a general platform for real-time world-model applications
across gaming, autonomous vehicles, robotics, simulated or virtual
environments, and more.

[omnidreams-blog]: https://research.nvidia.com/labs/sil/projects/omnidreams-blog/

https://github.com/user-attachments/assets/2b000ce9-effe-4cc9-a227-5b4619413e4d

## System Requirements

- NVIDIA GPU with **80 GB VRAM or more** (e.g. H100 80GB), see notes below.
- NVIDIA driver from the **R580 series or newer** (default compatible with CUDA 13.x)
- **CUDA 13.x by default** (PyTorch `2.11.0+cu130` and the `nvidia-*-cu13` libraries are
resolved by `uv sync`. A system CUDA toolkit is needed only for the
developer extras and is included in `nvidia/cuda:13.2.1-cudnn-devel-ubuntu24.04`)
- **Python >= 3.10**
- **PyTorch >= 2.11.0+cu130** (`>= 2.9` for bare PyPI library install)
- Linux x86-64 or arm64
- **100 GB+ free storage space** recommended for environment and model checkpoints.
- Docker with the
[NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
(optional, only for the container workflow)

> Development and testing were performed on GPUs with **80 GB of VRAM or more**.
> Inference can fail (out-of-memory) on consumer and even enthusiast GPUs.
> Per-model GPU and VRAM requirements are listed on each model page in
> [the model gallery](https://nvidia.github.io/flashdreams/main/models/index.html).

## Quickstart

The complete setup is in
[the installation guide](https://nvidia.github.io/flashdreams/main/quickstart/index.html#install).
Assuming `uv` is [installed](https://docs.astral.sh/uv/getting-started/installation), the shortest viable path is:

```bash
git clone https://github.com/NVIDIA/flashdreams.git
cd flashdreams
uv sync --extra runners
export HF_TOKEN=<your-hf-token>
uv run flashdreams-run --help
```
The complete setup is in our
[quick-start guide](https://nvidia.github.io/flashdreams/main/quickstart/index.html#install).

Note for developers/maintainers you would want to run `uv sync --extra dev --extra runners` instead.
## Checkout Our Demos! 🎉

### Select CUDA Version
Quick-start to our interactive driving demo is [here](https://nvidia.github.io/flashdreams/main/quickstart/index.html#run-your-first-model).

FlashDreams defaults to the CUDA 13 PyTorch stack (the `cuda13` dependency
group is activated automatically via `default-groups`):
![](docs/source/_static/demo_clips/interactive-drive.avif)

- **Linux:** `uv sync` pulls `torch>=2.9` from PyPI, whose Linux wheels are
the CUDA 13 build (the `+cu130` local-version tag is just stripped on
PyPI).
- **Windows:** `uv sync` pulls torch from NVIDIA's `cu130` index
(`https://download.pytorch.org/whl/cu130`) — PyPI's Windows torch wheel
is CPU-only.
Checkout our game-like crazy-robotaxi demo [here](https://nvidia.github.io/flashdreams/main/demos/crazy_robotaxi/).

CUDA 12.8 is available as an opt-in side profile (Linux only) via the
`cuda12` dependency group:

```bash
uv sync --group cuda12 --extra runners
```

The two groups are declared mutually exclusive in `pyproject.toml`, so uv
deactivates the default `cuda13` group automatically when `--group cuda12`
is passed.

Then launch your first model by following [the Get Started guide](https://nvidia.github.io/flashdreams/main/quickstart/index.html#run-your-first-model).
For example, the Self-Forcing T2V v2 application is:

```bash
uv run --project integrations_v2/self_forcing \
flashdreams-run-v2 t2v-self-forcing-wan2.1-t2v-1.3b \
--output-path artifacts/t2v-self-forcing-wan2.1-t2v-1.3b.mp4 --timeout unbound -- \
--prompt "A cat surfing." --total-blocks 7
```

You can also install FlashDreams as a library from PyPI:

```bash
pip install flashdreams
```

### Try the interactive driving demo

Drive a world model in real time with the unified OmniDreams `local-window` or
`webrtc` launch mode. See the
**[interactive demo guide](https://nvidia.github.io/flashdreams/main/models/omnidreams.html#launch-the-interactive-demo)**.

## Supported models

FlashDreams ships first-party integrations under
[`integrations_v2/`](integrations_v2/). Each model has a dedicated docs page with
runner slugs, multi-GPU commands, and (where available) profiling benchmarks.

| Model | Family |
| --- | --- |
| [Self-Forcing](https://nvidia.github.io/flashdreams/main/models/self_forcing.html) | Streaming Wan2.1 T2V |
| [OmniDreams](https://nvidia.github.io/flashdreams/main/models/omnidreams.html) | HDMap-conditioned driving world model |
| [LingBot-World](https://nvidia.github.io/flashdreams/main/models/lingbot_world.html) | Camera-controllable I2V world model |
| [Waypoint 1.5](https://nvidia.github.io/flashdreams/main/models/waypoint.html) | Interactive image-established keyboard/mouse-controlled world model |
| [Wan2.1](https://nvidia.github.io/flashdreams/main/models/wan21.html) | Bidirectional T2V / I2V |
| [Causal-Forcing](https://nvidia.github.io/flashdreams/main/models/causal_forcing.html) | Streaming Wan2.1 T2V / I2V |
| [Causal Wan2.2](https://nvidia.github.io/flashdreams/main/models/causal_wan22.html) | FastVideo Causal Wan 2.2 14B MoE T2V |
| [FlashVSR](https://nvidia.github.io/flashdreams/main/models/flashvsr.html) | Streaming video super-resolution |
| [Cosmos-Predict2.5](https://nvidia.github.io/flashdreams/main/models/cosmos_predict2.html) | Bidirectional T2V / I2V |

See [the model gallery](https://nvidia.github.io/flashdreams/main/models/index.html) and
[the new method guide](https://nvidia.github.io/flashdreams/main/developer_guides/new_integration.html)
to add your own.
![](docs/source/_static/demo_clips/crazy-robotaxi.avif)

## Developer guides

- [Architecture](ARCHITECTURE.md)
- [Inference pipeline overview](https://nvidia.github.io/flashdreams/main/developer_guides/inference_pipeline_overview.html)
- [Config system](https://nvidia.github.io/flashdreams/main/developer_guides/config_system.html)
- [Add a new method](https://nvidia.github.io/flashdreams/main/developer_guides/new_integration.html)
### For Demo Developers

For day-to-day development:
Make demos using world-models with our [Demo API](https://nvidia.github.io/flashdreams/main/documentation/demo_api/).

```bash
uv sync --extra dev --extra runners
uv run --group lint pre-commit run -a
uv run pytest -m "not manual"
```
### For Model Developers

See [`DEV.md`](DEV.md) for repository-specific workflow notes.
Implement world-models with our [Inferencing API](https://nvidia.github.io/flashdreams/main/documentation/inferencing_api/).

## Contributing

Expand Down Expand Up @@ -197,3 +99,5 @@ If FlashDreams is useful in your research or product, please cite the project:
url = {https://arxiv.org/abs/2606.03159},
}
```

[omnidreams-blog]: https://research.nvidia.com/labs/sil/projects/omnidreams-blog/
Loading
Loading