Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions .github/workflows/clock.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
name: Clock gates

# agent-env's libfaketime (src/agent_env/computers/clock), the library every computer's session programs load to read
# universe time, built with stock 0.9.12 beside it in an Ubuntu 24.04 image and run through tst/clock's gates on x86_64
# and arm64: every wait API in C, Python, Java and Node at several rates, jumps, cancelled waits, timerfd duplicates,
# condvar and semaphore stress, the waits again with futex_waitv refused, a stalled writer and a dead one, torn page
# reads, the marker, the environment race and Chromium launches. It runs on pull requests that touch the library, its
# tier or Chrome's pin, weekly, and on demand; it isn't a required check.

on:
pull_request:
paths:
- "src/agent_env/computers/clock/**"
- "src/agent_env/browsers/envs/chromium/chromium.py"
- "tst/clock/**"
- ".github/workflows/clock.yml"
schedule:
- cron: "41 6 * * 1"
workflow_dispatch:

permissions:
contents: read

concurrency:
group: clock-${{ github.event_name == 'pull_request' && github.ref || github.sha }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

env:
PIP_DISABLE_PIP_VERSION_CHECK: "1"
AGENTENV_TEST_CLOCK_CACHE: ${{ github.workspace }}/.clock-downloads

jobs:
gates:
name: gates (${{ matrix.arch }})
runs-on: ${{ matrix.arch == 'arm64' && 'ubuntu-24.04-arm' || 'ubuntu-24.04' }}
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
arch: [amd64, arm64]
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
- name: Install from uv.lock
run: |
python -m pip install --quiet -r .github/ci-requirements.txt
uv sync --locked --extra dev --python "$(command -v python)"
# Chrome's archive and upstream's tarball, checked by sha256 when the tier uses them; a new pin is a new key.
- name: Downloads
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ env.AGENTENV_TEST_CLOCK_CACHE }}
key: clock-downloads-${{ matrix.arch }}-${{ hashFiles('src/agent_env/browsers/envs/chromium/chromium.py', 'src/agent_env/computers/clock/build.sh') }}
- name: Clock gates
run: .venv/bin/python -m pytest tst/clock -rA --timeout=2400 --junitxml=junit-clock-${{ matrix.arch }}.xml
- name: No skips
if: always()
run: .venv/bin/python .github/scripts/check_skip_policy.py junit-clock-${{ matrix.arch }}.xml
5 changes: 3 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ or one of them only where it is used.
| `make int-test-fast` | `tst/integration` minus `int_test_slow`, in parallel; needs Docker and a local OCI registry (`docker run -d -p 5000:5000 public.ecr.aws/docker/library/registry:2`) |
| `make int-test-slow` | the `int_test_slow` tests, serially (they build the same Docker tags from module fixtures and race under xdist) |
| `make installer-test` | `tst/installer`: `plugin add` / `remove` through the real pip, uv and pipx, offline against wheels built from the checkout, and the container user journey (`python:3.12-slim`, `--network none`) with the PEP 668 refusal; needs uv, pipx and Docker |
| `make clock-test` | `tst/clock`: agent-env's libfaketime (`src/agent_env/computers/clock`) and stock 0.9.12 built in an Ubuntu 24.04 image for this machine's Docker, and its gates run there: every wait API in C, Python, Java and Node at several rates, jumps, cancelled waits, timerfd duplicates, condvar and semaphore stress, the waits again with `futex_waitv` refused (as before Linux 5.16 and under gVisor), a stalled writer, a dead one, torn reads, the marker, the environment race, and Chromium launched 60 times with it and 10 with stock; needs Docker, and fetches Chrome's archive and upstream's tarball once into `AGENTENV_TEST_CLOCK_CACHE` |
| `make trl-test` | `tst/trl`: one `GRPOTrainer` step on CPU through `agent_env.training.trl`, with a tiny model from the Hub; needs trl and CPU torch, which the `dev` extra leaves out |
| `make nemo-gym-test` | `tst/nemo_gym`: rollouts through `agent_env.training.nemo_gym`'s resources server in-process; needs the `nemo-gym` extra, on Python 3.13.14 or later, which the `dev` extra leaves out (`uv sync --extra dev --extra nemo-gym`) |
| `make verl-test` | `tst/verl`: rows played through `agent_env.training.verl`'s agent loop on verl itself, with Qwen3's tokenizer from the Hub and a stand-in rollout server; needs verl and CPU torch, which the `dev` extra leaves out |
Expand Down Expand Up @@ -137,7 +138,7 @@ and deserialized through a registry.
| `explorer/` | Optional local web UI: `agent-env up`, binds loopback `:8234`. |
| `cli/` | Click CLI with the groups `a2a-agent`, `artifact`, `bundle`, `config`, `env`, `eval`, `plugin`, `run`, `task`, `up`. `task run`, `task run-batch` and `eval run` hand their runs to the configured `[runner]` and follow them. |
| `agents/` | The `agents` bundle: the agents agent-env ships, one folder each under `agents/agents/`, with a smoke task each under `agents/tasks/`. Its Claude Code, `@local/agentenv-framework/agents/claude-code`, is the default agent. An agent's Python packages are its folder's `requirements.txt`, installed in its image, never in agent-env's own environment. A run that pulls one of its agents reads the whole bundle, so `tst/unit/bundle/shipped_bundles_test.py` parses and checks every shipped bundle. |
| `computers/` | The `computers` bundle: the base computers agent-env ships as `computer` artifacts, `ubuntu-24.04` (XFCE on Xvfb, supervisord as the session user, settings as `AGENTENV_SETTING_<NAME>`), and `omarchy-4.5`, a disk alone built on the composing machine from Omarchy's own VM image (its `[forms.disk]` `source`) by `disk/omarchy/`'s recipe, its services systemd user units of the graphical session that start through `with-seed`, booted by the launcher as a `linux-desktop` guest (`env/envs/computer/utils/omarchy_disk.py`, on `thin_disk.py`, which Windows' disks use too), aliased through `[artifacts]`. It has no tasks either. `disk/linux/` is the Linux disk form's recipe: `boot/`, whose Dockerfile makes a disk form (a GPT disk of an EFI system partition holding one UKI, Ubuntu's kernel with a busybox initramfs whose init reads a NoCloud seed and `switch_root`s into the image's entrypoint, and an ext4 root filesystem that is a container image's, plus `/etc/agentenv/vm/image.env` from its OCI config), `tools/`, the disk tools that make it unprivileged on the builder's own platform (`mkfs.ext4 -d`), and `launcher/`, the QEMU launcher that boots one under KVM. A base's `[forms.disk]` with no image is made from its container form; the composer makes each arch's disk from its composed container, keeping the base disk's EFI system partition, keyed by the container's key, the base disk and the recipe; gate 3 probes it where a provider boots disks natively (`disk_image_platforms()`), and a deploy's first boot with KVM probes one left `unverified`; a deploy naming `form = "disk"` boots it through `create_vm(image=..., env=...)`, its settings in the seed. |
| `computers/` | The `computers` bundle: the base computers agent-env ships as `computer` artifacts, `ubuntu-24.04` (XFCE on Xvfb, supervisord as the session user, settings as `AGENTENV_SETTING_<NAME>`), and `omarchy-4.5`, a disk alone built on the composing machine from Omarchy's own VM image (its `[forms.disk]` `source`) by `disk/omarchy/`'s recipe, its services systemd user units of the graphical session that start through `with-seed`, booted by the launcher as a `linux-desktop` guest (`env/envs/computer/utils/omarchy_disk.py`, on `thin_disk.py`, which Windows' disks use too), aliased through `[artifacts]`. It has no tasks either. `disk/linux/` is the Linux disk form's recipe: `boot/`, whose Dockerfile makes a disk form (a GPT disk of an EFI system partition holding one UKI, Ubuntu's kernel with a busybox initramfs whose init reads a NoCloud seed and `switch_root`s into the image's entrypoint, and an ext4 root filesystem that is a container image's, plus `/etc/agentenv/vm/image.env` from its OCI config), `tools/`, the disk tools that make it unprivileged on the builder's own platform (`mkfs.ext4 -d`), and `launcher/`, the QEMU launcher that boots one under KVM. A base's `[forms.disk]` with no image is made from its container form; the composer makes each arch's disk from its composed container, keeping the base disk's EFI system partition, keyed by the container's key, the base disk and the recipe; gate 3 probes it where a provider boots disks natively (`disk_image_platforms()`), and a deploy's first boot with KVM probes one left `unverified`; a deploy naming `form = "disk"` boots it through `create_vm(image=..., env=...)`, its settings in the seed. `clock/` is the computer's clock: agent-env's build of libfaketime 0.9.12 (`libfaketime.patch` and the two `agentenv_clock_*.c` files it includes, GPL-2.0-only as libfaketime is, built by `build.sh` from upstream's sha256-pinned tarball), which session programs load to read universe time and wait for a set clock time from a page its follower writes (`clockpage.py`), monotonic and relative waits staying real, with its default settings (`clock.conf`); `tst/clock` gates it. |
| `browsers/` | The `browsers` bundle: the website browsers agent-env ships, today `chromium`, a `computer_app` env (Chrome for Testing at the build Playwright pins, for amd64 and arm64, set up for unattended use by managed policies in `/etc/opt/chrome_for_testing/policies/managed`, opened at boot on its `start_urls`, its profile its data). All of it, its pin, install step, policies and command, is in `envs/chromium/chromium.py`; `tst/integration/computer/chromium_app_test.py` runs it. Like `drivers`, it has no tasks. |
| `drivers/` | The `drivers` bundle: the computer drivers agent-env ships as `computer_driver` envs, the reference driver (X11 on Linux, recording through `x11grab`) aliased `reference`, and trycua's cua-driver aliased `cua-driver` (its daemon from the pinned PyPI wheel, its own instructions, tools and skill resources served to agents through `@server_instructions`, `forwarded_tools` and `@resource`, its tools beside `computer_*`, `computer/v1` on its desktop-scope calls, its skill pack leading the computer skill; `tst/integration/computer/cua_driver_test.py` runs it). It has no tasks: a bundle with neither tasks nor evals is refused unless its `aliases.toml` names something for other bundles to use. The shipped `desktop` example composes it with the `ubuntu-24.04` base into a computer env, which `tst/integration/computer/` deploys, probes, records and has an agent use. |
| `examples/` | The example bundles agent-env ships (`hello`, and `echo-mcp`, which needs Docker, and `desktop`, a computer env of `ubuntu-24.04` and `reference` with a probe task and an agent task). Tests run them, but they are examples for users, not fixtures. Every shipped bundle is registered under `agent_env.bundles` in `pyproject.toml`, `<name> = "<package>"` shipping `src/<package as a path>/<name>/`. |
Expand Down Expand Up @@ -438,7 +439,7 @@ skips; a `changes` job decides, and a skipped `installer` counts as passed.
`.github/workflows/trl.yml` runs `tst/trl` against TRL 1.16 (1.15 can't train on CPU) and the latest, and on
transformers 5.17, the newest Unsloth's `GRPOTrainer` allows, on pull
requests that touch `agent_env/training`, `agent_env/openenv` or the tier, weekly, and on demand; it allows no skips
and isn't a required check. `.github/workflows/nemo-gym.yml` runs `tst/nemo_gym` on Python 3.13 with the `nemo-gym`
and isn't a required check. `.github/workflows/clock.yml` runs `tst/clock` on x86_64 and arm64 runners on pull requests that touch the library, the tier or Chrome's pin, weekly, and on demand, keeping Chrome's archive and upstream's tarball between runs; it allows no skips and isn't a required check. `.github/workflows/nemo-gym.yml` runs `tst/nemo_gym` on Python 3.13 with the `nemo-gym`
extra from `uv.lock` and with the latest nemo-gym, on the same triggers; it allows no skips and isn't a required check.
`.github/workflows/verifiers.yml` runs `tst/verifiers`, `vf-eval` through verifiers' NeMo Gym taskset (prime-rl's)
against `serve`, on verifiers 0.4.0 and its main branch, installed in a venv of its own since verifiers pins mcp 2, on
Expand Down
8 changes: 7 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: help install lint unit-test int-test int-test-fast int-test-slow installer-test trl-test nemo-gym-test verl-test skyrl-test miles-test slime-test clean-install-test test clean
.PHONY: help install lint unit-test int-test int-test-fast int-test-slow installer-test clock-test trl-test nemo-gym-test verl-test skyrl-test miles-test slime-test clean-install-test test clean

VENV := .venv
VENV_BIN := $(abspath $(VENV))/bin
Expand Down Expand Up @@ -66,6 +66,12 @@ installer-test: ## Run plugin add/remove through the real pip, uv and pipx, and

# The TRL contract tier trains one GRPOTrainer step on CPU through agent_env.training.trl, with a tiny model from the
# Hub. trl and torch aren't in the dev extra: install them first, as .github/workflows/trl.yml does.
# The clock tier builds agent-env's libfaketime (src/agent_env/computers/clock) and stock 0.9.12 in an Ubuntu 24.04 image
# for this machine's Docker and runs its gates there, Chromium launches included; .github/workflows/clock.yml runs it on
# x86_64 and arm64.
clock-test: ## Build agent-env's libfaketime and run its gates in Docker (tst/clock)
$(PYTHON) -m pytest tst/clock --timeout=2400 $(PYTEST_ARGS)

trl-test: ## Run one GRPOTrainer step on CPU through the TRL helper (needs trl and CPU torch, and the Hub)
$(PYTHON) -m pytest tst/trl --timeout=900 $(PYTEST_ARGS)

Expand Down
16 changes: 14 additions & 2 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

This project is licensed under the Apache License, Version 2.0, but depends on third-party
software identified below. These dependencies are installed separately from the Python Package
Index and imported at runtime. Third-party source code is not incorporated into or statically
linked with the source repository.
Index and imported at runtime. Apart from agent-env's changes to libfaketime, described below, third-party
source code is not incorporated into or statically linked with the source repository.

Each third-party component remains subject to its applicable license. Copyright notices, license
files, source-code offers, and other materials accompanying each component form part of its
Expand Down Expand Up @@ -2544,6 +2544,18 @@ Platform-specific native binaries that npm installs for the machine running the
| @next/swc-win32-arm64-msvc | 15.5.25 | MIT |
| @next/swc-win32-x64-msvc | 15.5.25 | MIT |

## libfaketime

`src/agent_env/computers/clock/` holds agent-env's changes to [libfaketime](https://github.com/wolfcw/libfaketime)
0.9.12: `libfaketime.patch`, `agentenv_clock_page.c` and `agentenv_clock_waits.c`. Being derived from libfaketime,
those three files are licensed as it is, under the GNU General Public License version 2 (`GPL-2.0-only`); the folder's
`COPYING` is that license's text. libfaketime's own source is not in the repository: the folder's `build.sh` downloads
upstream's v0.9.12 tarball, checks its sha256 and applies the changes, and the library it builds runs inside computers.
The folder's other files, like the rest of the repository, are licensed under the Apache License, Version 2.0.

- Source: <https://github.com/wolfcw/libfaketime/tree/v0.9.12>
- License: GPL-2.0-only

## License texts

License and notice files as shipped by each package, with copyright lines removed: they are listed with each package above. Packages whose files differ only in those lines share a text.
Expand Down
Loading
Loading