Skip to content

daemon: replace exit(1) callbacks with recoverable state machine - #62

Closed
silentone12725 wants to merge 14 commits into
WorldObservationLog:mainfrom
silentone12725:main
Closed

silentone12725 wants to merge 14 commits into
WorldObservationLog:mainfrom
silentone12725:main

Conversation

@silentone12725

@silentone12725 silentone12725 commented Aug 6, 2026 •

Copy link
Copy Markdown

Summary

This PR changes the wrapper from a process that terminates on Apple playback lease errors into a long-running, self-recovering daemon.

Previously, both SVPlaybackLeaseManager callbacks (endLeaseCb and pbErrCb) called exit(1) unconditionally. A lease termination therefore killed the entire wrapper, requiring an external supervisor to restart the process.

The new implementation keeps the wrapper and its HTTP services alive, moves FairPlay context reacquisition into a dedicated recovery worker, makes access to the shared decryption context thread-safe, exposes lifecycle state to external consumers, and brings the rootless wrapper in line with the same recovery model.

It also adds build/drift tooling to help keep the C and rootless wrapper implementations synchronized.


What changed

Recovery state machine

main.cpp now maintains an explicit recovery lifecycle:

  • Running
  • Scheduled
  • Refreshing
  • Failed

get_recovery_state() is exported as an extern "C" API so external consumers can observe the daemon state:

Value State
0 Running
1 Scheduled
2 Refreshing
3 Failed

is_recovery_active() provides a simple recovery gate for request handlers.

Non-blocking lease callbacks

endLeaseCb and pbErrCb no longer perform recovery work or terminate the process directly.

Instead, callbacks enqueue a recovery event and return immediately.

This keeps FairPlay/library calls out of the lease-manager callback context and reduces the risk of callback reentrancy or deadlocks.

Dedicated recovery worker

Recovery is handled by a persistent worker thread that owns context reacquisition.

The worker:

  • drains and coalesces queued lease events before each recovery attempt;
  • performs one refresh for a burst of related errors instead of repeatedly refreshing for every callback;
  • calls refresh_decrypt_ctx() as the single owner of reacquisition;
  • verifies recovery using is_preshare_ctx_ready();
  • resets the consecutive-failure counter only after a usable preshareCtx exists; and
  • schedules its own retry after a failed refresh, so recovery continues even when Apple sends no additional lease event.

Retry delay uses bounded exponential backoff:

1s -> 2s -> 5s -> 10s -> 30s

The 30-second delay is clamped rather than terminating recovery, allowing the daemon to survive prolonged transient failures.

Thread-safe FairPlay context access

main.c now protects preshareCtx with a PTHREAD_MUTEX_INITIALIZER-backed mutex.

All relevant reads and writes are synchronized between the decrypt path and the recovery worker.

The mutex is deliberately released before FairPlay/network reacquisition work so long-running recovery operations do not hold the context lock.

is_preshare_ctx_ready() also checks the context while holding the mutex.

Request gating during recovery

Decrypt and M3U8 request handling now checks the recovery state before using the FairPlay context.

During recovery:

  • the decrypt endpoint returns immediately/EOF;
  • the M3U8 endpoint returns an empty response and continues serving subsequent requests; and
  • key-delivery work is prevented from racing context reacquisition.

The decrypt, M3U8, and account servers themselves remain running throughout lease recovery.

External supervision is therefore reserved for actual process failures such as crashes, OOM termination, or other unrecoverable failures rather than normal lease lifecycle events.


DRM lifecycle state

The wrapper now publishes its lifecycle through:

<base-dir>/drm-state

The state file allows the surrounding Go/Electron layer to observe wrapper lifecycle changes without inferring state solely from process existence.

Lifecycle states include:

  • STARTING
  • LOGIN
  • WAITING_2FA
  • INITIALIZING_FAIRPLAY
  • RUNNING
  • RECOVERY
  • FAILED
  • STOPPED

This provides a lightweight interface for status reporting and supervisor integration while keeping recovery ownership inside the wrapper.


Rootless wrapper

wrapper-rootless.c has been updated to follow the persistent-daemon recovery model rather than treating lease termination as an unconditional process-exit condition.

This keeps rootless operation aligned with the recovery behavior introduced in the main wrapper implementation.


Build and drift protection

This PR also adds tooling around the wrapper build:

  • adds Dockerfile.build for a reproducible wrapper build environment;
  • adds scripts/check-drift.py;
  • runs the wrapper drift check from the x86_64 GitHub Actions workflow.

The drift check is intended to catch unintended divergence between related wrapper implementations as recovery/lifecycle behavior evolves.


Result

Before

Apple lease error
       |
       v
 endLeaseCb / pbErrCb
       |
       v
     exit(1)
       |
       v
 wrapper process dies
       |
       v
 external supervisor must restart it

After

Apple lease error
       |
       v
 endLeaseCb / pbErrCb
       |
       v
 enqueue recovery event
       |
       v
 callback returns immediately
       |
       v
 recovery worker
       |
       +--> coalesce events
       |
       +--> refresh FairPlay context
       |
       +--> success --> RUNNING
       |
       `--> failure --> backoff --> retry

The wrapper therefore behaves as a persistent service: lease expiration and transient FairPlay recovery failures are handled internally, while the process and HTTP servers remain alive.

Update: rebased onto current origin/main

This PR now also contains the host-native build, the in-process library, and follow-ups. It was rebased onto the latest origin/main. Upstream moved from the old key-server code, and conflicts were resolved in cmdline.*, main.c, wrapper.ggo and README.md.

Rebase notes

  • --key-port/-K (from origin/main) and --mv-port/-G (from this PR) are both kept.
  • cmdline.c / cmdline.h are hand-merged in gengetopt 2.23 style. Regenerating with 2.23.1 differs only in version-related output (help/version ordering, whitespace), so the 2.23-style files were kept.
  • FHinstance, preshareCtx and getKdContext are non-static again, because drm_lib.c links against them.

Host-native build via libhybris

  • build-native.sh builds drm-native and libdrm-native.so with host gcc/g++ and libhybris; build-and-deploy.sh builds and deploys into apple-music-linux/drm/.
  • Fixes the Bionic std::function ABI layout mismatch (__f_-first).
  • hybris_stubs.c, hybris_ctor.c, hybris_types.h and import.h provide the host-side glue.

In-process C / CGO library (drm_lib)

  • drm_lib.h / drm_lib.c expose drm_lib_init, drm_lib_decrypt and drm_lib_shutdown, with auth (2FA) and state callbacks, so Go can link the DRM engine without socket IPC.
  • Library mode builds with -DDRM_LIB_BUILD, which compiles out main().
  • GUID is now copied into a null-terminated buffer (Data::bytes() is not null-terminated).
  • get_music_user_token and get_dev_token use heap buffers, and the request body is no longer freed before run().
  • Android lib load failures return an error instead of calling exit().
  • drm_lib_decrypt uses the key context, not its slot.

Behavior changes to review

  • --mv-port default is now 50020 (itun decrypt listens on mv-port + 10000 = 60020). It previously shared 40020 with --key-port. Because every listener sets SO_REUSEPORT, both binds could succeed and traffic on that port could reach either service. Passing --mv-port explicitly is unaffected. Scripts relying on the old 40020 default for the MV server will need updating. Docker users need -p 50020:50020 -p 60020:60020 (the README example is updated).
  • build-native.sh now links Dobby. The R1 key-server hook from origin/main is no longer gated by MyRelease, so native builds need dobby.h and a built libdobby.a (DOBBY_SRC / DOBBY_BUILD).

Docs

README covers the three execution modes, the CGO API, the recovery state machine, both port options, and the extra Docker ports. The Protocol column for the 50020 and 60020 rows is a placeholder.

Testing

  • Both drm-native and libdrm-native.so build from this branch against Dobby and the deployed libhybris.
  • drm-native --help lists -K (default 40020) and -G (default 50020), and --version prints wrapper 1.2.0.
  • scripts/check-drift.py passes locally.
  • Not yet run: [fill in: runtime decrypt test / key and MV servers against a real rootfs].
  • CI runs for this fork PR are waiting for maintainer approval ("Approve and run").

Commits

  1. daemon: replace exit(1) callbacks with recoverable state machine
  2. Add DRM state tracking and wrapper drift check
  3. Add host-native DRM wrapper build via libhybris
  4. Fix Bionic std::function ABI mismatch: use old __f_-first layout
  5. Add build-and-deploy.sh: compile drm-native and deploy to drm/
  6. Fix GUID null-termination, add in-process CGO DRM bridge (drm_lib)
  7. Return an error instead of exiting when Android libs fail to load
  8. Decrypt with the key context, not its slot, in drm_lib_decrypt
  9. docs: update README with libhybris host-native mode, CGO API, and recovery state machine
  10. Give --mv-port its own default (50020) so it no longer shares 40020 with --key-port
  11. docs: document mv/itun ports and expose them in the Docker example
  12. docs: expose mv/itun ports in the Docker run example
  13. build-native: link Dobby (R1 key-server hook is no longer MyRelease-gated)

Previously, both SVPlaybackLeaseManager callbacks (endLeaseCb, pbErrCb)
called exit(1) unconditionally, making every Apple lease termination a
fatal process death. wrapper-rootless was behaving like a short-lived
utility rather than a persistent background service.

This commit introduces a proper recovery lifecycle:

Recovery state machine (main.cpp)
- RecoveryState enum: Running / Scheduled / Refreshing / Failed
- get_recovery_state() exported as extern C int for status endpoints
  and Electron IPC (0=Running, 1=Scheduled, 2=Refreshing, 3=Failed)
- is_recovery_active() derived from state, used to gate HTTP requests

Non-blocking callbacks
- endLeaseCb / pbErrCb now push a code onto a queue and return
  immediately — no library calls from within the lease manager thread
- Eliminates reentrancy and deadlock risk from callback context

Dedicated recovery worker thread
- Drains the entire queue on each wake (coalescing): a burst of
  3084+3084+PLAYBACK_ERR produces one refresh cycle, not three
- Exponential backoff: 1s -> 2s -> 5s -> 10s -> 30s (clamped, never stops)
- Calls refresh_decrypt_ctx() as the sole owner of reacquisition logic
- Verifies success via is_preshare_ctx_ready() — resets consec_fails
  only when preshareCtx is non-null after the refresh
- Self-schedules retry on failure (kRetryInternal) so the daemon keeps
  retrying even when Apple sends no further lease-end event (e.g.
  transient network failure during refresh)

Thread safety (main.c)
- g_ctx_mutex (PTHREAD_MUTEX_INITIALIZER) protects all preshareCtx
  reads and writes against concurrent access from the decrypt thread
  and the recovery worker
- Lock is released before FairPlay network calls to avoid blocking
  decryption during reacquisition
- is_preshare_ctx_ready() reads preshareCtx under g_ctx_mutex

Client-visible state gating (main.c)
- handle() and handle_m3u8() check is_recovery_active() per request
- Decrypt server: returns immediately (EOF) during recovery
- M3U8 server: writes empty line, continues loop — no hanging requests
- Prevents FairPlay key-delivery calls from racing the recovery worker

HTTP servers (decrypt, m3u8, account) remain alive across all lease
events. The Electron supervisor continues to provide the outer safety
net for true process deaths (segfault, OOM, etc.).
Proves in-process loading of Android x86-64 Bionic-linked .so files
into a glibc Linux process without proot or Docker.

New files:
- hybris_stubs.c: lazy-dispatch stubs for all 111 symbols in import.h,
  resolved at first call via android_dlsym against libstoreservicescore.so
  and libandroidappmusic.so. Special cases: curl → system libcurl,
  __android_log_* → stderr, _resolv_set_nameservers_for_net → no-op.
- hybris_types.h: minimal struct definitions (shared_ptr, std_string,
  std_vector, FairPlaySinf) for hybris_stubs.c without pulling in the
  android_id/fairplayCert const data from import.h (avoids duplicate symbol).
- hybris_ctor.c: __attribute__((constructor)) auto-init; reads
  HYBRIS_ANDROID_LIB64 env var and calls hybris_init_libs() before main().
- build-native.sh: one-shot glibc build (no NDK, no Docker); fetches
  cJSON, compiles main.c + main.cpp + cmdline.c + hybris_stubs.c +
  hybris_ctor.c, links against libhybris-core.so and system libcurl.

Runtime env vars required:
  HYBRIS_LINKER_DIR=/tmp/hybris-linker
  HYBRIS_LD_LIBRARY_PATH=<rootfs>/system/lib64
  HYBRIS_ANDROID_LIB64=<rootfs>/system/lib64
Bionic NDK r21 libc++ std::__ndk1::function<F> uses an older field
ordering than modern LLVM libc++:

  OLD (NDK r21): [0..7] __f_  [8..31] __buf_
  NEW (modern):  [0..23] __buf_  [24..31] __f_

The previous implementation assumed the new layout, so the vptr we
stored in el[0] was being read by Bionic as __f_ (a pointer to the
__func object). Since &vtab_endlease[2] != &endLeaseCallback[8],
Bionic took the heap path and tried to load a vtable from the value
stored at el[0] (our function address). This caused an immediate
SIGSEGV.

Fix: lay out the 32-byte buffers with __f_ at [0] pointing to
el+8 (__buf_[0]), vptr at [1], fn ptr at [2], allocator at [3].

With this fix drm-native fully initializes:
  SVPlaybackLeaseManagerC2 → refreshLeaseAutomatically →
  requestLease → SVFootHillSessionCtrl::instance →
  recovery thread started → offline_available (listening)
Compiles via build-native.sh then copies:
  drm/drm-native             binary (rpath patched to \$ORIGIN)
  drm/libhybris-core.so      hybris runtime (co-located for rpath)
  drm/hybris-linker/q.so     Android linker plugin

Engine already auto-selects drm-native over drm-rootless when present
(apiserver.go:503) and sets the required HYBRIS_* env vars automatically.
- Fix get_guid(): Data::bytes() is not null-terminated; use Data::length()
  to copy a clean null-terminated string. Corrupted GUID was causing
  createMusicToken POST to receive garbage bytes, returning empty body.
- Add _ZNK13mediaplatform4Data6lengthEv stub to hybris_stubs.c and
  declare in import.h so Data::length() resolves at runtime via hybris.
- Do not free(body) before URLRequest::run() — new hybris-core.so stores
  a pointer, not a copy, so premature free corrupted the POST body.
- Add drm_lib.c / drm_lib.h: C API exported for CGO linking into the Go
  engine (drm_lib_init, drm_lib_get_m3u8, drm_lib_get_mv,
  drm_lib_open_kd_ctx, drm_lib_decrypt, drm_lib_decrypt_itun, etc.).
  Compiles with -DDRM_LIB_BUILD to exclude int main().
- build-native.sh: compile drm_lib.c and link libdrm-native.so alongside
  the standalone drm-native binary.
- Remove debug-only fprintf logs from get_dev_token and createMusicToken.
hybris_init_libs() called exit(1) if libstoreservicescore.so or
libandroidappmusic.so could not be loaded. Inside the Go engine
(libdrm-native.so, in-process) that killed the whole engine — Electron then
restarted it in a loop — instead of leaving it running without DRM.
It now returns -1; drm_lib_init() propagates it, and only the standalone
drm-native constructor still exits.

build-native.sh: HYBRIS_BUILD can be overridden to link against an existing
libhybris-core.so (e.g. the copy deployed in apple-music-linux/drm).
getKdContext() returns the key-context slot; handle() in main.c (the proven
TCP path) calls the decryptor with *kdContext. drm_lib_decrypt passed the
slot pointer itself, so every in-process (libdrm-native.so) ALAC sample was
"decrypted" with the wrong context and VLC's ALAC decoder saw garbage from
the first frame. Dereference it like handle() does.
@silentone12725
silentone12725 marked this pull request as draft October 1, 2026 07:41
@silentone12725
silentone12725 marked this pull request as ready for review October 1, 2026 07:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant