sync with main branch

.agents/AGENTS.md

@@ -1,4 +1,4 @@
-# Search-SWE Contributor Workflow
+# Search-SWE Task Workflow
 
 This is the contributor workflow entry point. Detailed task-authoring knowledge
 lives inside the skill, not here. Paths below are relative to the repository
@@ -11,6 +11,10 @@ root unless expressed as Markdown links.
   workflow, references, and validation gates. If no native skill loader is
   available, open that `SKILL.md` directly. It can also be invoked independently
   as `$create-searchswe-task`; it does not need this file.
+- **Review, promote or finalize a task PR as maintainer:** load
+  [maintain-searchswe-task](skills/maintain-searchswe-task/SKILL.md), independently
+  invocable as `$maintain-searchswe-task`. Default to read-only gh evidence review;
+  use its bundled offline promotion/HF staging helpers only at approved stages.
 - **Run an existing task:** use `docs/quickstart.md`; do not scaffold a task.
 - **Restore fixed inputs:** use `docs/assets.md` and the existing download tools.
 - **Edit project documentation:** keep `README.md` and `README_zh.md` aligned.
@@ -23,15 +27,33 @@ root unless expressed as Markdown links.
 1. Inspect the worktree and agree on the requested deliverable. Preserve other
    contributors' changes. Ask about uncertainties that affect task design or
    authorization, not information already supplied by the user.
-2. For task contributions, let the skill drive design → authoring → validation.
-   Track which stage is complete and which inputs or approvals are missing.
+2. For task contributions, let the skill drive design → structured source →
+   scenario rewrite → grader checks → configured model evaluation. Preserve `raw-instruction.md` and audit all its conditions
+   in the agent-facing `instruction.md`; the skill defines both gates.
+   Before scaffolding, inspect one or two closest reviewed packages under
+   `tasks/` as read-only structural precedents, selected by mode, grading shape,
+   resources and hardware. Current repository contracts and validators override
+   older examples; never inherit task-specific data, thresholds, licenses or
+   access merely because another task uses them.
+   A PR may add multiple tasks, all under exactly one first-name namespace, at
+   `task-submissions/<first-name-slug>/<1|2>-x-<positive-ordinal>`; do not use
+   usernames, reuse a promoted temporary ordinal, or guess formal IDs. Follow
+   `CONTRIBUTING.md` and `docs/contributing.md`: require actual authors and
+   explicit `--task-path` for local trials, then promote every task with its own
+   same-PR pure rename and finalization commits before a merge commit (no
+   squash/rebase). Track which stage is complete and which inputs or approvals
+   are missing.
    Do not duplicate the skill's specifications in this file.
 3. Keep changes within the selected task plus directly necessary integration
    changes. Updating a website, publishing a dataset, pushing images, and opening
    or merging a PR are separate actions, not implied by task creation.
 4. Review the resulting diff and validation evidence. Hand off changed paths,
    task mode/hardware, commands actually run, observed results, and blockers.
    Static checks alone are not evidence of a working end-to-end benchmark.
+   Include the exact instruction revision, model/effort, trajectory, score and
+   judge health from the configured-model validation. Reuse session-authorized
+   runs and credentials; do not repeat successful runs just because results
+   were moved or a monitoring pane still looks active.
 
 Agents are not guaranteed to auto-load an `AGENTS.md` located under `.agents/`
 when editing `tasks/`. Open this entry point explicitly or invoke the skill;

.agents/skills/create-searchswe-task/SKILL.md

@@ -5,9 +5,10 @@ description: Create or substantially revise Search-SWE task packages with CPU/GP
 
 # Create a Search-SWE Task
 
-Deliver a reproducible task whose instructions, environment, submission
-interface, and grading agree. This skill contains its authoring guidance and
-helpers; no outer AGENTS.md, other skill, or historical Harbor workspace is
+Deliver a reproducible task whose structured source, scenario instruction,
+environment, submission interface, and grading agree. Validate the setting
+with an actual configured coding-model run as well as grader checks. This skill
+contains its authoring guidance and helpers; no outer AGENTS.md, other skill, or historical Harbor workspace is
 required. It still operates on a **target Search-SWE checkout**: its tasks,
 Docker definitions, and release/download/launch tools are project inputs, not
 bundled copies of this skill. Missing tools or incompatible versions are
@@ -19,17 +20,29 @@ Read [references/task-authoring.md](references/task-authoring.md). Inspect the
 target checkout's status and preserve unrelated changes. Resolve its root
 explicitly; do not infer it from where this skill was installed.
 
+Before scaffolding, inspect one or two closest **formal** packages under the
+target checkout's `tasks/`. Choose them by engineering mode, evaluation/judge
+shape, resources/APIs, hardware and input/artifact layout—not merely by task
+number. Record their exact paths and why they are relevant. Treat them as
+read-only structural precedents: the current skill, repository validators,
+schema and shared-image documentation take priority. Do not copy task-specific
+IDs, authors, datasets, thresholds, pins, licenses or access policy without
+independently establishing them for the new task. If no close precedent exists,
+say so rather than forcing an analogy. See the bounded-reference procedure in
+`references/task-authoring.md`.
+
 Record a short design summary before writing the package:
 
 | Decision | What must be known |
 | --- | --- |
-| Identity | Unique task ID and name, author, version |
+| Identity | Temporary ID (`task-<1|2>-x-<positive-ordinal>`), one PR-wide first-name slug, actual authors, version |
 | Mode | Implementation → `metadata.task_type = "create"`; Optimization → `"optimize"` |
 | Goal | What working capability or quality/efficiency improvement is measured |
 | Interface | Container input paths, commands, output paths/formats, artifact transfer |
 | Evaluation | Metric, public gates, baseline if needed, failure/timeout behavior |
 | Inputs | Public/hidden split, actual files, provenance, redistribution rights |
 | Resources | CPU/GPU, memory/storage/time, network, permitted APIs/models |
+| Precedent | Formal task path(s), matching dimensions, structural patterns reused, intentional differences |
 
 Only these two engineering modes exist. CPU is the default; choose GPU for
 actual GPU execution, not merely because model files are involved. Ask for
@@ -45,14 +58,46 @@ For a new task, run the bundled helper using its **actual installed path**:
 # Set these to real absolute paths; neither depends on the current directory.
 SKILL_DIR=/path/to/create-searchswe-task
 REPO=/path/to/Search-SWE
-python "$SKILL_DIR/scripts/scaffold_task.py" task-1-new \
-  --repo-root "$REPO" --mode implementation --hardware cpu
+python "$SKILL_DIR/scripts/scaffold_task.py" task-1-x-1 \
+  --repo-root "$REPO" --submission-first-name Alice --author 'Alice Example' \
+  --mode implementation --hardware cpu
 ```
 
-Use `--mode optimization` for optimization and `--hardware gpu` for GPU work.
-The flags are independent. The helper refuses existing paths. Edit an existing
+Category 1 is Implementation (`create`); category 2 is Optimization (`optimize`).
+The temporary ID and mode must agree. For optimization:
+
+```bash
+python "$SKILL_DIR/scripts/scaffold_task.py" task-2-x-1 \
+  --repo-root "$REPO" --submission-first-name Alice --author 'Alice Example' \
+  --mode optimization --hardware cpu
+```
+
+Hardware is independent of category/mode; use `--hardware gpu` only for GPU work.
+New tasks belong in
+`task-submissions/<first-name-slug>/<1|2>-x-<positive-ordinal>`, not formal `tasks/` directories.
+Use a supplied ASCII first name, not a username; non-Latin names need an explicit
+transliteration. Reuse that contributor directory for every task in the PR.
+Ordinals are temporary, category-local, positive, and unique in the PR/checkout;
+they are not final IDs; never reuse one after promoting its earlier task in the
+same PR. If a different contributor with the same first name needs a namespace,
+explicitly supply `Alice-2` (then `Alice-3`), including after
+checking active PRs; the scaffolder never derives name suffixes from task
+collisions. Repeat `--author` for coauthors.
+The helper refuses overwrites and symlink roots. Regular mode without
+`--submission-first-name` remains compatible for maintainers. Edit an existing
 task in place instead; do not delete it to make the helper succeed.
 
+One PR may add multiple tasks, but every task must use exactly one contributor
+first-name namespace. Read the bundled
+[submission and PR workflow](references/submission-and-pr.md) for branch,
+review gates, maintainer handoff and author identity. A maintainer assigns each final ID close to
+merge and performs a separate pure rename then finalization for each task in
+**the same PR**, using `scripts/promote_task.py`. All tasks must be promoted
+before merge. Preserve every promotion commit and original author commit:
+**merge commit only**, never squash/rebase. No submission task package enters
+final main. Missing workflow tools in an older checkout are a prerequisite to
+report, not a reason to invent a formal ID.
+
 The scaffold is **not a finished task**: its verifier always fails, its asset
 manifest is empty, data/model mounts are absent, and instructions are writing
 prompts. Replace them and review the generated budgets, author, artifact list,
@@ -64,8 +109,13 @@ the matching CPU/GPU image, consistent with both Compose files and `gpus`.
 
 Follow the detailed rules in `references/task-authoring.md`:
 
-1. Define the submission interface and grader behavior, then write an instruction
-   containing every graded requirement. Do not reveal hidden answers.
+1. Define the submission interface and grader behavior in `raw-instruction.md`.
+   Preserve an existing structured instruction there before rewriting. Follow
+   [references/instruction-rewrite.md](references/instruction-rewrite.md) to write
+   the agent-facing `instruction.md`: a named person in a concrete physical
+   setting asking their coding agent for help in long, natural prose. Preserve
+   every condition and exact interface; do not reveal hidden answers. Audit the
+   two documents for semantic equivalence before running a model.
 2. When adding fixed data/models or external services, read
    [references/assets-and-resources.md](references/assets-and-resources.md).
    Build exact manifests and read-only phase-specific mounts. Keep hidden inputs,
@@ -78,19 +128,33 @@ Follow the detailed rules in `references/task-authoring.md`:
 5. Add author-facing README context and task-local tests, including known-good
    and invalid submissions. Do not change unrelated tasks or scoring policies.
 
-## 4. Validate and hand off
+## 4. Validate the task and run a configured model
 
 Read [references/validation.md](references/validation.md) and follow its staged
 checks. Stop advancing to expensive runtime tests when inputs, permissions,
 hardware, or required configuration are missing. Record the failed command and
 diagnosis; retry after a relevant fix, not in an unbounded loop.
 
-Task authoring does not itself authorize publishing data/images, modifying host
+After local package, asset and grader checks, follow
+[references/model-evaluation.md](references/model-evaluation.md). Run the final
+scenario instruction with the configured coding model and any required judges;
+inspect its trajectory, artifacts, grader result and judge health to validate
+the task setting. An infrastructure failure does not measure difficulty, and
+one low-scoring model run does not establish that a task is impossible.
+
+Local task authoring/validation is the default authorization boundary. It does
+not itself authorize commits, pushes, PR creation/comments, publishing data/images, modifying host
 network/proxy configuration, or spending on model APIs/GPU jobs. Obtain missing
-authorization for those actions; ordinary local checks may proceed.
+authorization for those actions only when it is absent from the current
+request or prior session. An explicit request to evaluate with a named model
+authorizes that scoped run; reuse already-authorized local credentials without
+asking again. Ordinary local checks may proceed.
 
 Report the task path, mode/hardware, commands actually executed and their
-results, known-good/negative-case evidence, and every unrun validation layer.
+results, formal reference tasks consulted (or that none was close), intentional
+reuse/deviations, the raw-to-scenario equivalence review, known-good/negative-case
+evidence, the exact evaluated revision/model/effort, observed scores and judge
+health, and every unrun validation layer.
 Never equate a scaffold, a dry-run, or a static release check with a solvable
 end-to-end task.
 

.agents/skills/create-searchswe-task/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
-  display_name: "Create Search-SWE Task"
-  short_description: "Create and validate a new Search-SWE task package"
-  default_prompt: "Use $create-searchswe-task to add a validated Search-SWE task package to this repository."
+  display_name: "Contribute Search-SWE Task"
+  short_description: "Author and validate a temporary task submission"
+  default_prompt: "Use $create-searchswe-task to build and locally validate a first-name task submission, then prepare an authorized PR handoff."

.agents/skills/create-searchswe-task/references/assets-and-resources.md

@@ -7,7 +7,9 @@ task. Nothing in this reference grants permission to publish or use paid APIs.
 ## Manifest construction
 
 Keep downloadable fixed files outside the build contexts, under
-`tasks/<task-id>/data/` or `models/`. Ignore both directories in Git. Hidden
+`data/` or `models/` in the selected package (initially
+`task-submissions/<first-name-slug>/<1|2>-x-<positive-ordinal>`, later `tasks/<final-id>`).
+Ignore both directories in Git. Hidden
 queries/labels and grading code go in `tests/`, not the public asset bundle.
 For a verifier-only corpus stored in `data/verifier/`, mount only public
 subdirectories into the agent; never mount all of `data/` in that case.
@@ -23,10 +25,10 @@ valid checksums/revisions to submit):
     "size_bytes": 1234,
     "sha256": "REPLACE_WITH_COMPUTED_SHA256",
     "source": {
-      "repo_id": "search-swe/Search-SWE",
+      "repo_id": "YOUR_ACCOUNT/search-swe-development",
       "repo_type": "dataset",
       "revision": "REPLACE_WITH_PUBLISHED_40_HEX_COMMIT",
-      "filename": "tasks/task-1-new/corpus.jsonl"
+      "filename": "development/corpus.jsonl"
     }
   }]
 }
@@ -44,11 +46,43 @@ file modes are `"0644"` and `"0600"`; private model directories can be declared
 as `"directory_modes": {"models/private-model": "0700"}` at manifest root.
 Do not attempt `chmod` on read-only mounted files during verification.
 
+## Development and incremental publication
+
+For pre-promotion development, use an authorized **personal public temporary HF
+dataset** with manifest, provenance and license; pin its immutable 40-hex SHA in
+submission `assets.json`. The downloader accepts arbitrary HF repositories:
+
+```bash
+python scripts/download_assets.py --task-path task-submissions/alice/1-x-1
+python scripts/download_assets.py --task-path task-submissions/alice/1-x-1 --verify-only
+```
+
+After the maintainer assigns the final ID in the same PR, prepare new official
+files at `tasks/<final-id>/...`. The target checkout's
+`scripts/prepare_hf_upload.py` accepts a current official manifest snapshot,
+the final task package and **only new data**. It verifies hashes, preserves old
+entries, refuses collisions and stages new files plus a merged manifest offline.
+Read the bundled [publication workflow](publication.md) for exact commands
+and approval boundaries. Append/update
+HF `SOURCES.md`, dataset card/license and LFS metadata. Never delete old assets.
+Publish through a **HF community PR (`create_pr=True`)** with your personal
+account, or ask a maintainer to mirror reviewed public files. Never share or
+request an official token. Refresh a stale manifest snapshot before merging.
+
+The official HF change must merge **before** pinning the resulting official SHA
+in final `assets.json` and merging the GitHub task PR. Do not pin the community
+PR SHA or use a mutable branch. Keep original model pins unchanged. Official
+publication is separately authorized; helpers do not upload. Do not publish
+permanent official paths containing temporary task IDs.
+
+## Full dataset audit (maintainers)
+
 For unpublished data, prepare an authorized local HF staging directory with
 `README.md`, `SOURCES.md`, `.gitattributes`, `manifest.json`, and `tasks/` files.
 Its manifest has `schema_version: 1` and `files` entries containing dataset-relative
 `path`, `size_bytes`, and `sha256`. It must match the checkout's complete dataset
-inventory, not just new files. Local checks support:
+inventory, not just new files. Contributors normally use the incremental
+helper above instead; it does not need all old assets. Full local checks support:
 
 ```bash
 python scripts/check_release.py --hf-data /path/to/hf-data \