Skip to content

Close challenge shortcuts and make each challenge test its skill - #129

Closed
madebygps wants to merge 5 commits into
mainfrom
challenge-quality-review
Closed

madebygps wants to merge 5 commits into
mainfrom
challenge-quality-review

Conversation

@madebygps

Copy link
Copy Markdown
Collaborator

Why

A review of all 18 challenges on a live Azure VM found that every challenge was solvable, but many could be shortcut without the skill they teach:

  • All flags shared one random suffix, so solving challenge 1 revealed the other 17.
  • One grep -r 'CTF{' without sudo found 13 of 18 flags.
  • Several challenges (8, 9, 11, 13) didn't actually check what the README described.

What changed

# Change
All Each flag gets its own random suffix
5 Hint adds -type f (the old command printed ~9,000 lines)
6 README no longer names the port
8 Flag is awarded only after a real SSH key login and shown in the login banner
9 A private DNS zone replaces the flag-in-a-comment design
10 Only files uploaded via scp/sftp count (checked with auditd); flag no longer written to /tmp
11 Port 8083 serves a decoy; the flag appears only once nginx is moved to port 80. 8083 closed in all firewall rules
12 Flag no longer present in the ping script or its log
13 Cron job publishes its output briefly; learners trace the job and its script
14 Flag loaded via root-only EnvironmentFile=, not visible via systemctl
16 Flag is in the final target's path, so the link chain must be resolved
17 History file needs sudo; content is a realistic leaked credential
18 Disk image is root-only so it must be mounted; README wording fixed

New VM packages: auditd, dnsmasq-base, dnsutils. Challenge 8 is renamed "SSH Key Authentication" in verify list and the certificate.

The test suite now includes shortcut regression checks, and the reboot phase checks the new services.

Testing

  • deploy_and_test.sh azure --with-reboot: 41/41 before reboot, 9/9 after, resources cleaned up
  • ShellCheck (CI severity) passes
  • Not yet tested on AWS or GCP. Challenges 8–10 depend on sshd, auditd and systemd-resolved behavior, so those runs should happen before release.

madebygps and others added 5 commits September 25, 2026 11:42
- Give every flag its own random suffix so one flag can't predict the rest
- Rework ch8 (real SSH key login), ch9 (private DNS zone), ch10 (only
  scp/sftp uploads count), ch11 (flag only on port 80), ch13 (trace a
  cron job's short-lived output)
- Close leaks in ch12, ch14, ch16, ch17, ch18
- Close port 8083 in AWS/Azure/GCP firewall rules
- Update README descriptions and verify hints
- Add shortcut regression checks and new services to the test suite

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93529270-84bf-485c-99ca-0109bc533d3d
- Move the login-banner reward plumbing (challenges 8 and 10) into
  setup/external_rewards.py, including the ExposeAuthInfo sshd drop-in,
  and have ch08/ch10 use grant_command() instead of hardcoding the path.
- Let challenges declare their own apt packages via PACKAGES; system.py
  now only lists learner tools.
- Add CHALLENGE_DIR and DONE_MARKER to helpers.py and use them instead of
  hardcoded paths (ch10 previously duplicated the done-marker path).
- Keep a single CHALLENGES registry in setup/challenges/__init__.py.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: eb6ef7ff-cd36-49b9-8051-47b519236889
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: eb6ef7ff-cd36-49b9-8051-47b519236889
Nothing depends on it alone: every readiness check also accepts
/var/lib/linux-ctfs/setup.done or the cloud-init instance marker, both of
which ctf_setup.sh still writes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: eb6ef7ff-cd36-49b9-8051-47b519236889
systemd-resolved starts before sysinit.target and network.target, while
ctf-dns is a regular service after both, so Before=systemd-resolved made two
ordering cycles. systemd broke them by dropping jobs at boot (seen: resolved
and network.target), leaving DNS down after reboot and, depending on which job
was dropped, SSH unreachable. resolved doesn't need dnsmasq up to start; it
only forwards ctf.internal queries on demand.

Add post-reboot checks for ordering cycles and a running systemd-resolved,
which the service is-active loop didn't catch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: eb6ef7ff-cd36-49b9-8051-47b519236889
@madebygps

Copy link
Copy Markdown
Collaborator Author

Follow-up changes from review

1. Split challenge-specific concerns out of setup/system.py (03683ac)

  • New setup/external_rewards.py holds the login-banner reward plumbing used only by ch08 and ch10 (rewards dir, profile.d banner, and the ExposeAuthInfo sshd drop-in). ch08/ch10 call grant_command(n) instead of hardcoding /var/lib/ctf-rewards.
  • Challenges now declare their own apt packages via a module-level PACKAGES list (ch06 netcat-openbsd, ch09 dnsmasq-base, ch10 auditd/inotify-tools, ch11 nginx). system.py only lists learner tools. The installed package set is unchanged.
  • CHALLENGE_DIR and DONE_MARKER live in helpers.py, replacing hardcoded paths in 8 files (ch10 previously duplicated the done-marker path).
  • Single CHALLENGES registry in setup/challenges/__init__.py; contributor docs updated.

2. Drop unused PROGRESS_FILE import and a placeholder-free f-string in verify (42f5622)

3. Remove the legacy /var/log/setup_complete marker (ef1e769)
Nothing depended on it alone; every readiness check still accepts /var/lib/linux-ctfs/setup.done or the cloud-init instance marker.

4. Fix a boot ordering cycle from ch09's ctf-dns.service (88b8eab)
Before=systemd-resolved.service created two systemd ordering cycles (resolved starts before sysinit.target/network.target; ctf-dns starts after both). systemd broke them by dropping jobs at boot, which left DNS down after reboot and, in one run, SSH unreachable. Removed the Before= (resolved only forwards ctf.internal queries on demand) and added post-reboot checks for ordering cycles and a running systemd-resolved.

Testing

  • Rendered ch08/ch10 scripts and the banner are byte-identical to the previous commit, aside from the new ExposeAuthInfo drop-in file.
  • shellcheck, ruff, bash -n, and terraform fmt pass.
  • Azure basic test: 41/41 passed.
  • Azure full test with reboot: 41/41 pre-reboot, 11/11 post-reboot. The run before the ch09 fix failed with SSH unreachable after reboot.

@madebygps madebygps closed this Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant