Repository navigation
feat(computers): agent-env's libfaketime for the computer's clock, gated on x86_64 and arm64 (FD-5474) - #619
Conversation
…ted on x86_64 and arm64 (FD-5474) src/agent_env/computers/clock is the library every computer's session programs will load to read universe time: upstream libfaketime 0.9.12 from its sha256-pinned tarball with agent-env's changes, as a small libfaketime.patch and two engine files it includes (agentenv_clock_page.c, agentenv_clock_waits.c; GPL-2.0-only, as libfaketime is), built by build.sh, with the page writer the follower uses (clockpage.py) and the library's default settings (clock.conf). Clock reads and waits for a set realtime clock time follow the universe; monotonic and boot clocks and every relative wait stay real; only the session user is faked; settings are read once at load. A clock read now copies the page before reading the real time and keeps the pair only if the page is still the one copied: a reader that waited out a write paired a time from before it with the page it wrote, which stepped CLOCK_REALTIME back by up to ~280 ns at a rate change. tst/clock (make clock-test, .github/workflows/clock.yml on x86_64 and arm64) builds it and stock 0.9.12 in an Ubuntu 24.04 image and gates it: every wait API in C, Python, Java and Node at several rates, a rate change, a set_time back and 30 jumps, condvar and semaphore stress, a stalled writer, torn reads, who is faked, the marker, the environment race and Chromium launched directly 60 times (stock 0.9.12 as the positive control), in under 30 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| r = real_pthread_cond_clockwait(cond, mutex, CLOCK_REALTIME, &end); | ||
| ft_tab_lock(&old); | ||
| woken = w->woken; | ||
| w->used = 0; |
There was a problem hiding this comment.
Canceled waits leave stale pointers
ft_u_cond_wait clears its table entry only after real_pthread_cond_clockwait returns. If the thread is canceled during that call, the entry stays active. The caller can then destroy or free the condition variable, but the helper still broadcasts to its saved pointer on clock changes. Add a cancellation cleanup handler that removes the entry before the caller's cleanup runs.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agent_env/computers/clock/agentenv_clock_waits.c
Line: 667-670
Comment:
**Canceled waits leave stale pointers**
`ft_u_cond_wait` clears its table entry only after `real_pthread_cond_clockwait` returns. If the thread is canceled during that call, the entry stays active. The caller can then destroy or free the condition variable, but the helper still broadcasts to its saved pointer on clock changes. Add a cancellation cleanup handler that removes the entry before the caller's cleanup runs.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| k = ft_wait2((uint32_t *) &sem->data, 0, sem->private_flag == 0, (uint32_t) s.seq, ft_real_of(&s, due)); | ||
| if (k == FT_W_INTR) | ||
| { | ||
| err = EINTR; | ||
| break; |
There was a problem hiding this comment.
Semaphore waiters ignore cancellation
ft_u_sem_wait replaces the cancelable semaphore wait with raw futex calls, without checking for pending thread cancellation. With futex_waitv available, canceling a thread blocked in sem_timedwait or sem_clockwait no longer reliably stops it. A frozen clock can leave shutdown waiting forever in pthread_join. Preserve cancellation behavior and remove the semaphore's waiter count when cancellation runs.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agent_env/computers/clock/agentenv_clock_waits.c
Line: 602-606
Comment:
**Semaphore waiters ignore cancellation**
`ft_u_sem_wait` replaces the cancelable semaphore wait with raw futex calls, without checking for pending thread cancellation. With `futex_waitv` available, canceling a thread blocked in `sem_timedwait` or `sem_clockwait` no longer reliably stops it. A frozen clock can leave shutdown waiting forever in `pthread_join`. Preserve cancellation behavior and remove the semaphore's waiter count when cancellation runs.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| struct ft_clock_page s; | ||
| int64_t due = ft_ns(abs); | ||
| *handled = false; | ||
| if (ft_waitv_missing || !ft_snap(&s)) return 0; |
There was a problem hiding this comment.
Fallback waits use wrong deadlines
If the kernel lacks futex_waitv or blocks it, ft_waitv_missing makes later calls leave ft_u_futex_rt without translating their deadlines. The first fallback translates once but also stops watching clock changes. With universe time 182 days behind real time, later waits time out immediately; with it ahead, they wait far too long. Keep translating deadlines and checking page changes in the fallback.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agent_env/computers/clock/agentenv_clock_waits.c
Line: 539
Comment:
**Fallback waits use wrong deadlines**
If the kernel lacks `futex_waitv` or blocks it, `ft_waitv_missing` makes later calls leave `ft_u_futex_rt` without translating their deadlines. The first fallback translates once but also stops watching clock changes. With universe time 182 days behind real time, later waits time out immediately; with it ahead, they wait far too long. Keep translating deadlines and checking page changes in the fallback.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| static struct ft_timer *ft_timer_find_fd(int fd) | ||
| { | ||
| for (int i = 0; i < ft_timers_top; i++) | ||
| if (ft_timers[i].kind == 1 && ft_timers[i].fd == fd) return &ft_timers[i]; |
There was a problem hiding this comment.
Timer records outlive their descriptors
Timer tracking follows a descriptor number, not the open timer. dup does not copy ft_tfd_clock, and close does not remove the tracked entry. Duplicating an armed timerfd and closing the original leaves the helper updating the closed descriptor while the surviving timer misses clock changes. Reusing that number can also make the helper update another timer. Track duplication and closure along with timer creation.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agent_env/computers/clock/agentenv_clock_waits.c
Line: 252-255
Comment:
**Timer records outlive their descriptors**
Timer tracking follows a descriptor number, not the open timer. `dup` does not copy `ft_tfd_clock`, and `close` does not remove the tracked entry. Duplicating an armed timerfd and closing the original leaves the helper updating the closed descriptor while the surviving timer misses clock changes. Reusing that number can also make the helper update another timer. Track duplication and closure along with timer creation.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| body = LAYOUT.pack(MAGIC, VERSION, seq + 2, universe_ns, anchor_ns, rate, armed, 0, generation, | ||
| check(seq + 2, universe_ns, anchor_ns, rate, armed, generation, now_mono, now_mono), | ||
| now_mono, now_mono) | ||
| m[16:] = body[16:] | ||
| m[8:16] = struct.pack("<Q", seq + 2) |
There was a problem hiding this comment.
Restarted writers cannot recover
write assumes the existing seq is even. If the writer dies after publishing an odd value, the next writer finishes with seq + 2, which is still odd. Every later write stays odd too, so readers never accept the recovered writer's updates. Recover an interrupted page before writing, then publish an even sequence with a matching check without replacing the mapped file.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agent_env/computers/clock/clockpage.py
Line: 101-105
Comment:
**Restarted writers cannot recover**
`write` assumes the existing `seq` is even. If the writer dies after publishing an odd value, the next writer finishes with `seq + 2`, which is still odd. Every later write stays odd too, so readers never accept the recovered writer's updates. Recover an interrupted page before writing, then publish an even sequence with a matching check without replacing the mapped file.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| return; | ||
| } | ||
| for (char **e = environ; e != NULL && *e != NULL; e++) |
There was a problem hiding this comment.
Documented overrides do nothing
clock.conf says the environment can override settings, but ft_conf_load returns immediately after reading the file. Once that file is installed, environment settings such as AGENTENV_CLOCK_PAGE and AGENTENV_WAIT_SLICE_MS are silently ignored. Either merge environment overrides into the startup snapshot or change the documentation to describe file-only settings.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agent_env/computers/clock/agentenv_clock_page.c
Line: 76-78
Comment:
**Documented overrides do nothing**
`clock.conf` says the environment can override settings, but `ft_conf_load` returns immediately after reading the file. Once that file is installed, environment settings such as `AGENTENV_CLOCK_PAGE` and `AGENTENV_WAIT_SLICE_MS` are silently ignored. Either merge environment overrides into the startup snapshot or change the documentation to describe file-only settings.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| elapsed = {r["api"]: r["real"] for r in results if r["class"] == "elapsed"} | ||
| clock = {r["api"]: r["real"] for r in results if r["class"] == "clock_time"} | ||
| assert len(clock) == 19 | ||
| assert {api: s for api, s in elapsed.items() if not 0.99 <= s <= 1.3} == {} | ||
| assert {api: s for api, s in clock.items() if not 1 / rate - 0.01 <= s <= 1 / rate + 0.15} == {} |
There was a problem hiding this comment.
The wait gates collect return codes but mostly check only elapsed time. The jump gate never checks the reported return codes. A wait that ends at the expected time with an error can therefore pass. Assert the expected result for each API, including timerfd reads and semaphore waits, alongside the timing checks.
Prompt To Fix With AI
This is a comment left during a code review.
Path: tst/clock/waits_test.py
Line: 19-23
Comment:
**Wait errors pass the gates**
The wait gates collect return codes but mostly check only elapsed time. The jump gate never checks the reported return codes. A wait that ends at the expected time with an error can therefore pass. Assert the expected result for each API, including timerfd reads and semaphore waits, alongside the timing checks.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
Part of FD-5420, the computer's own clock following the gateway's universe clock. This is FD-5474: the library every computer's session programs will load to read universe time. Every program on a computer reads its own clock, so it can't ask the gateway's
get_timethe way an MCP server does. Instead, agent-env's build of libfaketime answers those programs' clock reads and clock-time waits from a page a follower process keeps in step with the gateway. Wiring the library into the base computer (/etc/ld.so.preload, the follower, supervisord) is FD-5476.What it is
src/agent_env/computers/clock/:libfaketime.patchsrc/, applied withpatch -p1agentenv_clock_page.clibfaketime.cagentenv_clock_waits.cCOPYINGbuild.shlibfaketimeMT.so.1from upstream's sha256-pinned tarball and the files aboveclockpage.pyclock.confWhat the library does:
CLOCK_REALTIME,_COARSE,_ALARM, TAI,gettimeofday,time) and waits for a set realtime clock time (clock_nanosleep TIMER_ABSTIME, realtime futex waits,sem_timedwaitandsem_clockwait, realtime condvars, absolute realtime timerfds and POSIX timers). Each wait is planned in real time and planned again whenever the page changes.AGENTENV_REAL_TIME=1stays real, and its children are faked.getenvraced programs' ownsetenv; that crashed Chromium on about one launch in five and hangs it outright in headless mode.The patch and the engine files are the prototype's library, moved out of the two Python scripts that applied it as string replacements (about 900 lines of C inside a Python string). Their preprocessed source was byte-identical to the prototype's before the fix below.
A bug the gates found, fixed here
The gates found it on arm64: with the follower stalled 5 ms in a write,
CLOCK_REALTIMEstepped back up to 280 ns at a rate change, 3 to 7 times per 2,000 writes.ft_snap_at). The real time is ordered before that check: a dependent load anddmb ishldon aarch64,lfenceon x86_64.Where it lives
It's at
computers/clock/, beside the bundle's other shared recipes (disk/,vm_bundle/), not atartifacts/ubuntu-24.04/clock/as the issue said. Inside the base computer's folder it would change the base's build context, and with it the image's tag, though nothing loads the library yet. FD-5476 brings it into the base.Licensing
The patch and the two engine files are derived from libfaketime, so they carry
GPL-2.0-onlyand upstream'sCOPYINGsits beside them. Everything else in the folder and the repo stays Apache-2.0, and the package's license metadata is unchanged.THIRD_PARTY_NOTICES.mdgets a libfaketime section. I also changed its opening sentence, which said no third-party source is incorporated in the repository; that's no longer true for these three files. That sentence is legal wording, so please check the change. Whether the wheel's license metadata should also list GPL-2.0 for these files is a question for legal.Tests
tst/clock(make clock-test,.github/workflows/clock.ymlonubuntu-24.04andubuntu-24.04-arm, not a required check):browsersbundle's pin, read fromchromium.py) and upstream's tarball are fetched once into a cache, checked by sha256, and kept between CI runs.The gates:
sem_timedwaitposts, with the page changing under them, lose nothing; every handoff is under 2 ms.CLOCK_REALTIME: 0 torn reads in about 319 million on arm64.setenvracing clock reads and timed waits, 3 runs of 10 s each: no crash.Results:
Docs
The docs site should cover the computer's clock: what follows universe time and what stays real, who is faked,
AGENTENV_REAL_TIME, andclock.conf. That page belongs with FD-5476, when computers load the library.🤖 Generated with Claude Code
This PR needs fixes before merge because cancellation, timer tracking, fallback waits, and interrupted writes can break programs using the library.
Fix with agent prompt
Summary
Adds a patched libfaketime build and shared clock page for computer programs.
Acknowledged behavior:
earakely-scaleexplicitly describes continuing small backward clock changes as intentional.earakely-scaleexplicitly accepts spurious returns from single-call condition-variable waits when clock changes move their deadlines.Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR Writer["Clock page writer"] --> Page["Shared clock page"] Page --> Reads["Program clock reads"] Page --> Helper["Per-process helper"] Helper --> Timers["Rearm absolute timers"] Helper --> Conditions["Wake condition-variable waits"] Page --> Waits["Recheck absolute waits"] Kernel["Real kernel clocks"] --> Relative["Monotonic reads and relative waits"]Reviews (1) · Last reviewed commit: "feat(computers): agent-env's libfaketime..." · Reviewed by Greptile