Skip to content

Adopt descriptive task names and update task authoring workflows - #8

Merged
hanhainebula merged 5 commits into
mainfrom
jianlv-dev
Oct 9, 2026
Merged

hanhainebula merged 5 commits into
mainfrom
jianlv-dev

Conversation

@hanhainebula

Copy link
Copy Markdown
Collaborator

Summary

Use stable, descriptive task names across task packages, evaluation commands, and Hugging Face data paths. Update contribution tooling and agent skills to guide authors through building, testing, and refining reproducible search-system tasks.

Changes

  • Rename all nine task packages and synchronize their titles, configuration, asset manifests, and references. Pin migrated dataset assets to revision 6713b1eb7474e0768e906a0151c76af92f0cfcdd.
  • Update launchers, downloaders, validators, and regression tests to use task names. Describe tasks through their objectives, interfaces, metrics, and resource requirements.
  • Organize submissions as task-submissions/<task-name>, preserving the name through promotion to tasks/<task-name>. Update scaffolding, promotion, asset publication, and CI checks.
  • Rewrite agent instructions as direct engineering requests while preserving structured specifications, submission contracts, scoring rules, and resource limits.
  • Extend task-authoring guidance with configured coding-model trials, trajectory and score analysis, feedback-driven refinement, and validation of the final task revision.
  • Require authoring agents to explain which API keys are missing and how contributors should configure them locally.
  • Synchronize English and Chinese overviews, contribution guides, task documentation, and skill references with the current workflow.

Migration notes

  • Commands now accept descriptive names, for example --task reasoning-query-rewriting. Numbered task identifiers are no longer accepted.
  • New submissions use --task-path task-submissions/<task-name> for downloads and evaluation.
  • Rename the local environment variable TASK_1_1_OPENROUTER_API_KEY to REASONING_QUERY_REWRITING_OPENROUTER_API_KEY, keeping its value unchanged.

Validation evidence

  • Main regression suite: 177 passed, 2 skipped because the host lacks the required RewardKit version.
  • Skill tests: 5 authoring and 10 maintenance tests passed.
  • Task-local tests: 23 passed.
  • Validated 9 Harbor configurations, 18 Compose overlays, 27 launcher previews, and 9 download previews.
  • Verified all 335 migrated dataset asset references.
  • Static package checks, merge-readiness and submission-history checks against the main base, skill validation, and whitespace checks passed.

Full container builds and new paid-model/GPU evaluations were not run for this migration.

Rename nine task packages and pin assets to the migrated Hugging Face paths. Select tasks by name, flatten submission paths, and update evaluation, scaffolding, promotion, and validation tools with regression coverage.
Document same-name submission promotion and asset publication. Require configured model trials, feedback-driven task refinement, and local configuration prompts for missing API keys. Describe the current authoring contract and preserve structured instruction sources during revisions.
Synchronize the English and Chinese overviews, task commands, asset paths, credential examples, and network guidance. Describe the current benchmark and contribution process directly.
@hanhainebula
hanhainebula merged commit c013f45 into main Oct 9, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant