Skip to content

test: add an automated Android end to end check for the example - #8

Merged
janicduplessis merged 3 commits into
mainfrom
@janic/e2e-android
Sep 21, 2026
Merged

janicduplessis merged 3 commits into
mainfrom
@janic/e2e-android

Conversation

@janicduplessis

Copy link
Copy Markdown
Contributor

Description

The example has no automated device coverage. Every hinge check is a manual pass
on an emulator, so nothing catches a regression in the native angle or posture
reaching the JavaScript readouts.

Solution

pnpm e2e:android builds and installs the example, drives the emulator hinge
sensor, and asserts what the app actually renders.

The run must not depend on a Metro server, so it installs the release variant:
the React Native template signs it with the checked-in debug.keystore and
embeds the JavaScript bundle. That exposed a pre-existing break.
assembleRelease failed with Couldn't determine Hermesc location, because the
React Native Gradle plugin resolves hermes-compiler relative to its root
setting, which defaults to example/, while the hoisted pnpm workspace keeps it
at the repository root. example/android/app/build.gradle already redirects
reactNativeDir, codegenDir and cliFile for the same reason, so this adds
hermesCommand alongside them.

What the check asserts, in order:

  • Field Notes is up after a cold launch.
  • The Sensor lab mode pill reads NATIVE HINGE.
  • After adb -s <serial> emu sensor set hinge-angle0 45, the NATIVE ANGLE metric
    reads 45.0° within 5 seconds, and the status detail reads partiallyOpen.
  • After setting 180, the metric reads 180.0° and the status reads fullyOpen.
  • Back on Field Notes, the readout reads NATIVE 180.0°.

Each asserted state is screenshotted into e2e/artifacts/, already covered by
the existing artifacts/ ignore rule. A failed assertion exits non-zero and
dumps the accessibility snapshot.

Three behaviours look arbitrary in the diff and are worth naming:

  • The check runs cmd device_state state reset before driving the sensor. A
    posture override left behind by an earlier session pins the committed device
    state, so the hinge angle stops driving posture and the status assertions read
    whatever that session left. This bit a verification run here: the angle read
    180.0° while the status stayed partiallyOpen, with device_state showing
    Base state: OPENED and Override state: HALF_OPENED.
  • Posture lands a few seconds after the angle, so the status assertions allow 15
    seconds while the angle assertions keep the 5 second bound.
  • The return to Field Notes is a relaunch, not a press on the Sensor lab's own
    Back to Field Notes control. That control occupies the bottom 44dp of an
    edge-to-edge window and the system taskbar covers it on a foldable inner
    display, so it is not hittable. That looks like a real example layout bug and
    is worth a separate look.

Every adb call is bounded by a 60 second timeout. An unbounded
adb shell getprop hung a run here against a flapping emulator.

How to run

export ANDROID_HOME="$HOME/Library/Android/sdk"
E2E_ANDROID_SERIAL=emulator-5590 pnpm e2e:android

E2E_ANDROID_SERIAL is required when more than one device is attached. Set
E2E_SKIP_BUILD=1 to reuse the installed APK. The target needs a hinge sensor
and real device states; the pixel_fold and pixel_9_pro_fold AVD profiles give
both.

Local run evidence

Device: emulator-5590, AVD stim-react-native-hinges-fold3-camera, Pixel 9 Pro
Fold profile, API 34 google_apis arm64-v8a.

Building the example release APK for emulator-5590 (arm64-v8a)
hinge-angle0 set to 180 degrees on emulator-5590
LAUNCH open field notes: hinges.example on emulator-5590 (stim-react-native-hinges-fold3-camera)
PASS field notes ready: "Sensor lab ↗"
PASS sensor lab reads the native hinge: "NATIVE HINGE"
SHOT e2e/artifacts/01-sensor-lab-native-hinge.png
hinge-angle0 set to 45 degrees on emulator-5590
PASS native angle at 45 degrees: "45.0°"
PASS status at 45 degrees: "partiallyOpen"
SHOT e2e/artifacts/02-sensor-lab-hinge-45.png
hinge-angle0 set to 180 degrees on emulator-5590
PASS native angle at 180 degrees: "180.0°"
PASS status at 180 degrees: "fullyOpen"
SHOT e2e/artifacts/03-sensor-lab-hinge-180.png
LAUNCH return to field notes: hinges.example on emulator-5590 (stim-react-native-hinges-fold3-camera)
PASS field notes native readout: "NATIVE 180.0°"
SHOT e2e/artifacts/04-field-notes-native-180.png
e2e:android passed on emulator-5590

An earlier run on the same emulator, before the device-state reset was added,
reached the same seven assertions green. A third attempt could not start because
another local agent-device session held the emulator lease, which is a shared
host constraint rather than a check failure.

CI workflow is unverified

.github/workflows/e2e-android.yml has never executed. It cannot run locally, so
it is written from the action documentation:
reactivecircus/android-emulator-runner pinned to the v2.38.0 commit,
profile: pixel_fold, API 34 google_apis x86_64, KVM enabled exactly as the
action README describes, and the serial passed as emulator-$EMULATOR_PORT. I
did verify locally that an AVD created with --device pixel_fold gets the same
hinge configuration as the emulator used here: one hinge, 0 to 180 range, posture
bands 0-30, 30-150 and 150-180. Everything else about the runner is unproven,
including whether the emulator publishes device states under -no-window, which
the status assertions depend on. Expect to iterate on the first real run.

The e2e-android label does not exist in this repository yet, so the label
trigger cannot fire until someone creates it.

Adds e2e/android.mjs plus a pnpm e2e:android script. The check builds and
installs the example release variant, which embeds the JavaScript bundle and is
signed with the checked-in debug keystore, cold launches it, opens the Sensor
lab, drives the emulator hinge angle and asserts the readouts with a screenshot
per asserted state.

Sets hermesCommand in the example Android build so the release variant finds
hermesc in the hoisted pnpm workspace at the repository root. Without it
assembleRelease fails, because the React Native Gradle plugin resolves
hermes-compiler under example/ rather than the workspace root.

Adds .github/workflows/e2e-android.yml, which runs the same script on
workflow_dispatch and on pull requests labeled e2e-android. That workflow has
not been executed yet.
@janicduplessis
janicduplessis marked this pull request as ready for review September 21, 2026 09:20
@janicduplessis
janicduplessis merged commit 4229d11 into main Sep 21, 2026
4 checks passed
@janicduplessis
janicduplessis deleted the @janic/e2e-android branch September 21, 2026 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant