Repository navigation
Ship beta to main: the app, server and experiments as submitted #179
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| name: Evals | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - "experiments/evals/**" | ||
| - ".github/workflows/evals.yml" | ||
| pull_request: | ||
| paths: | ||
| - "experiments/evals/**" | ||
| - ".github/workflows/evals.yml" | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: evals-${{ github.ref }} | ||
| cancel-in-progress: ${{ github.event_name == 'pull_request' }} | ||
|
|
||
| jobs: | ||
| evals-checks: | ||
| name: Evals checks | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 10 | ||
| defaults: | ||
| run: | ||
| working-directory: experiments/evals | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
| # Reads tool.uv.required-version from experiments/evals/pyproject.toml. | ||
| - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0 | ||
| with: | ||
| working-directory: experiments/evals | ||
| # The datasets are non-commercial and never in git, so CI runs the metric code's unit tests | ||
| # (hand-computed cases), lint and format; the result tables are regenerated locally. | ||
| - run: uv sync --locked | ||
| - run: uv run ruff check . | ||
| - run: uv run ruff format --check . | ||
| - run: uv run pytest -q |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| name: Measure lab | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - "experiments/measure-lab/**" | ||
| - ".github/workflows/measure-lab.yml" | ||
| pull_request: | ||
| paths: | ||
| - "experiments/measure-lab/**" | ||
| - ".github/workflows/measure-lab.yml" | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: measure-lab-${{ github.ref }} | ||
| cancel-in-progress: ${{ github.event_name == 'pull_request' }} | ||
|
|
||
| jobs: | ||
| geometry-linux: | ||
| name: Measure lab geometry (Linux) | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 15 | ||
| container: | ||
| # Swift 6.2 is the oldest toolchain the tests compile with (their raw-identifier names need | ||
| # it); the app job's Xcode 26.6 ships Swift 6.3. Pinned by digest so a retag can't change | ||
| # the compiler under the tests. | ||
| image: swift:6.2-noble@sha256:9bea530093ffff8cf6c259991715ee843fe4d0f932e612f7f4b79cca6e00db87 | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - name: Show toolchain | ||
| run: swift --version | ||
|
|
||
| - name: Test | ||
| working-directory: experiments/measure-lab/Geometry | ||
| run: swift test | ||
|
|
||
| app-build: | ||
| name: Measure lab app build | ||
| # Same image and Xcode as .github/workflows/ios.yml. | ||
| runs-on: macos-26 | ||
| timeout-minutes: 30 | ||
| env: | ||
| DEVELOPER_DIR: /Applications/Xcode_26.6.app/Contents/Developer | ||
| XCODEGEN_VERSION: 2.46.0 | ||
| XCODEGEN_SHA256: 4d9e34b62172d645eed6457cac13fc222569974098ef4ee9c3368bedf0196806 | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - name: Show toolchain | ||
| run: | | ||
| if [ ! -d "$DEVELOPER_DIR" ]; then | ||
| echo "::error::$DEVELOPER_DIR is not on this runner image. Pick an installed Xcode from the image readme." | ||
| exit 1 | ||
| fi | ||
| xcodebuild -version | ||
| swift --version | ||
|
|
||
| - name: Install XcodeGen | ||
| run: | | ||
| curl -fsSL -o "$RUNNER_TEMP/xcodegen.zip" \ | ||
| "https://github.com/yonaskolb/XcodeGen/releases/download/${XCODEGEN_VERSION}/xcodegen.zip" | ||
| echo "${XCODEGEN_SHA256} $RUNNER_TEMP/xcodegen.zip" | shasum -a 256 -c - | ||
| unzip -q "$RUNNER_TEMP/xcodegen.zip" -d "$RUNNER_TEMP" | ||
| echo "$RUNNER_TEMP/xcodegen/bin" >> "$GITHUB_PATH" | ||
|
|
||
| - name: Generate project | ||
| run: xcodegen generate --spec experiments/measure-lab/project.yml | ||
|
|
||
| - name: Check committed project matches project.yml | ||
| run: | | ||
| if ! git diff --exit-code -- experiments/measure-lab; then | ||
| echo "::error::MeasureLab.xcodeproj differs from project.yml. Run 'make project' in experiments/measure-lab (XcodeGen ${XCODEGEN_VERSION}) and commit the result." | ||
| exit 1 | ||
| fi | ||
| untracked="$(git ls-files --others --exclude-standard -- experiments/measure-lab)" | ||
| if [ -n "$untracked" ]; then | ||
| echo "::error::xcodegen generate wrote files that are not committed. Run 'make project' and commit them." | ||
| echo "$untracked" | ||
| exit 1 | ||
| fi | ||
|
|
||
| - name: Test geometry on macOS | ||
| working-directory: experiments/measure-lab/Geometry | ||
| run: swift test | ||
|
|
||
| - name: Build | ||
| run: >- | ||
| xcodebuild -project experiments/measure-lab/MeasureLab.xcodeproj -scheme MeasureLab | ||
| -configuration Debug -destination "generic/platform=iOS" CODE_SIGNING_ALLOWED=NO build |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| name: Meter close-up experiment | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - "experiments/meter-closeup/**" | ||
| - ".github/workflows/meter-closeup.yml" | ||
| pull_request: | ||
| paths: | ||
| - "experiments/meter-closeup/**" | ||
| - ".github/workflows/meter-closeup.yml" | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: meter-closeup-${{ github.ref }} | ||
| cancel-in-progress: ${{ github.event_name == 'pull_request' }} | ||
|
|
||
| jobs: | ||
| checks: | ||
| name: Meter close-up checks | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 10 | ||
| defaults: | ||
| run: | ||
| working-directory: experiments/meter-closeup | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
| - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0 | ||
| with: | ||
| working-directory: experiments/meter-closeup | ||
| # The unit tests cover the metric code only; the Vision reader needs macOS and the photos. | ||
| - run: uv sync --locked | ||
| - run: uv run ruff check . | ||
| - run: uv run ruff format --check . | ||
| - run: uv run pytest -q | ||
| # Looks for any transcribed meter identifier in the tracked files. The key stays in a | ||
| # repository secret; without it the step fails with an explanation. | ||
| - run: uv run python -m meter_eval.leakcheck | ||
| env: | ||
| METER_HMAC_KEY: ${{ secrets.METER_HMAC_KEY }} | ||
|
Comment on lines
+43
to
+45
There was a problem hiding this comment.
When a fork pull request changes the meter-closeup experiment, this job runs without access to the repository's Prompt To Fix With AIThis is a comment left during a code review.
Path: .github/workflows/meter-closeup.yml
Line: 43-45
Comment:
**Fork pull requests fail checks**
When a fork pull request changes the meter-closeup experiment, this job runs without access to the repository's `METER_HMAC_KEY` secret. The leak check then looks for a local key that is not provisioned on the runner and raises `FileNotFoundError`, so the check fails regardless of the contributor's changes.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly. |
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| name: Panel label experiment | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - "experiments/panel-label/**" | ||
| - ".github/workflows/panel-label.yml" | ||
| pull_request: | ||
| paths: | ||
| - "experiments/panel-label/**" | ||
| - ".github/workflows/panel-label.yml" | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: panel-label-${{ github.ref }} | ||
| cancel-in-progress: ${{ github.event_name == 'pull_request' }} | ||
|
|
||
| jobs: | ||
| checks: | ||
| name: Panel label checks | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 10 | ||
| defaults: | ||
| run: | ||
| working-directory: experiments/panel-label | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
| - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0 | ||
| with: | ||
| working-directory: experiments/panel-label | ||
| - run: uv sync --locked | ||
| - run: uv run ruff check . | ||
| - run: uv run ruff format --check . | ||
| - run: uv run pytest -q | ||
| # Photos of real panels can show addresses and inspection stickers; they stay out of git. | ||
| - name: No photos committed | ||
| run: | | ||
| if git ls-files . | grep -iE '\.(jpe?g|png|heic|webp|tiff?)$'; then | ||
| echo "Photos must stay in ~/house-scanning-data/panel, not in the repository." >&2 | ||
| exit 1 | ||
| fi |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The job runs all of
HouseScanUITestsand skips only the accessibility audit. That includestestPracticeMeterFromReplay, which the PR description identifies as red on beta. As submitted, an iOS change cannot get a green CI run until that test is fixed or excluded.Prompt To Fix With AI