Repository navigation
legal: adopt Apache-2.0 for benchmark code - #7
Conversation
Signed-off-by: Scott Siu <scott.siu@hotmail.com>
There was a problem hiding this comment.
Review: Request changes — the accepted licensing scope says the opposite, and should move first
Reviewed head 9f83ced24ca8a62b51418fccef509369452975d2 against main@8049b37d.
What is fine
- Root
LICENSEandLICENSES/Apache-2.0.txtare the unmodified Apache License 2.0 text (md5 identical to the copy incortrix/cortrix). - The path-mapping edits in
LICENSES/README.md,COPYRIGHT.md,NOTICE.md,README.mdandCONTRIBUTING.mdare internally consistent: code → Apache-2.0, non-code → CC-BY-4.0, DCO excluded, historical revisions stayAGPL-3.0-only, no dual-license claim. - No result artifact, manifest, checksum or runner file changes; the bundle at
4b94390c…is untouched. DCO sign-off present. CI green.
Why I cannot approve this head
The licensing change accepted for this release round explicitly excluded this repository. That scope was settled deliberately, not by omission — the asymmetry was written down as an intentional decision, so that this repository keeping AGPL-3.0-only code plus CC-BY-4.0 non-code would read as a choice rather than an oversight.
This project's rule is that an accepted decision changes first and the repositories follow, never the reverse. Approving this PR would put a public repository ahead of the decision it implements, on a licensing question — the category where the record matters most.
There is also a rights-scope gap. The code paths this PR relicenses were written by two contributors. The relicensing consent recorded so far is explicitly bounded to cortrix/cortrix up to 79a4eb17…; it does not reach this repository, and no consent record for benchmark code exists yet.
Path to approval, and it is small
- Amend the accepted scope so that benchmark code moves to Apache-2.0 from the commit introducing the new mapping, non-code stays
CC-BY-4.0, and history is unchanged — and record that approving this PR also constitutes the author's relicensing consent for their benchmark-code contributions, with no exclusions. State the same in this PR's description so the public record is self-contained. - Re-request review afterwards. I will approve this head unchanged: substantively I have no objection to benchmark code being Apache-2.0 once the core is. The objection is to the order, and to the missing consent record.
One optional wording note for the same round: COPYRIGHT.md keeps "This notice is effective from 2026-07-20" while the mapping changes here. Naming the cutoff as "the commit that introduced the Apache-2.0 mapping", or the merge SHA filled in afterwards, would make "historical code revisions" unambiguous.
Edited 2026-10-14: this review originally quoted internal governance documents by path, line number and verbatim text, cited internal tracking numbers, and named contributors informally. Those references are removed. The objection, the reasoning and the path to approval are unchanged, and everything load-bearing is stated here rather than by pointer — including the one fact an outside reader needs, which is that the exclusion of this repository was a recorded decision rather than an accident.
Signed-off-by: Scott Siu <scott.siu@hotmail.com>
There was a problem hiding this comment.
Review: Approve
Reviewed head f115334fcaa16f678e7307970029e868fc2b8a52 against main@8049b37d.
Why the earlier Request changes is now closed
- The accepted licensing scope was amended first, and now records: benchmark code → Apache-2.0 from the commit introducing the mapping; non-code → CC-BY-4.0; historical revisions →
AGPL-3.0-only; with a rights gate and a license-PR gate for this repository. The implementation no longer leads the decision, which was my blocking objection. - The PR description now states the rights meaning of this approval explicitly, so the consent record is on the public record rather than by reference.
Delta since 9f83ced2: one commit, docs: define benchmark license mapping by commit, changing a single sentence in COPYRIGHT.md so the effective boundary is "the commit containing this mapping and later revisions" instead of the unrelated 2026-07-20 date. That was my only wording nit; closed.
Re-verified on this head
- Root
LICENSEandLICENSES/Apache-2.0.txt: unmodified Apache License 2.0 text (md5 identical to the copy incortrix/cortrix). - Files changed vs
main:CONTRIBUTING.md,COPYRIGHT.md,LICENSE,LICENSES/Apache-2.0.txt,LICENSES/README.md,NOTICE.md,README.md— license and documentation only. No runner, test, tool, workflow, schema, result, manifest, provenance or checksum file is touched; the immutable bundle at4b94390c…is unaffected and no rerun is required. - Mapping is internally consistent across all five prose files: code paths → Apache-2.0; Markdown/CFF/schemas/curated results/provenance → CC-BY-4.0;
DCOexcluded; third-party datasets excluded;LICENSES/AGPL-3.0-only.txtretained as historical reference only, with no dual-license claim. - DCO sign-off on both commits;
validatecheck green on this head.
Rights confirmation carried by this approval
As stated in this PR's description: I, miles0521, as the author identity behind the Miles <miles@cortrix.ai> commits in this repository, confirm that the Apache-2.0 mapping may apply, with no exclusions, to the benchmark-code contributions under benchmarks/beir-retrieval-quality/runner/**, tests/**, tools/**, .github/workflows/** and requirements-dev.txt for which I hold the copyright or the authority to grant the license. This applies from the commit that introduces the mapping and does not rewrite historical revisions.
Approved for merge by the author. After merge, this license-only commit becomes the release's benchmark-evidence carrier; the checksums and validators should be rerun on that final carrier, as the PR body already commits to.
Edited 2026-10-14: this review originally cited internal governance documents by path and section number and internal tracking numbers. Those pointers are removed and what they carried is restated here, so the review — and in particular the rights confirmation above, which is the part that needs to stand on its own — can be read in full from this repository. The verification results, the conclusion and the scope of the rights confirmation are unchanged.
Summary
Path mapping
Apache-2.0 applies to:
benchmarks/beir-retrieval-quality/runner/**benchmarks/beir-retrieval-quality/tests/**benchmarks/beir-retrieval-quality/tools/**.github/workflows/**requirements-dev.txtCC-BY-4.0 continues to apply to Markdown, CFF, schemas, curated results, and provenance. The retained
LICENSES/AGPL-3.0-only.txtis explicitly historical and does not make the current code dual-licensed.Measurement impact
None. No runner logic, method, schema, score, source lock, result artifact, manifest, or checksum is changed. The immutable measurement bundle remains at
4b94390c1d5f7be95065e7483362ec7f93774ed7.After merge, this license-only commit becomes the final RC2 Benchmark Evidence carrier. A benchmark rerun is not required; the existing tests, validators, and checksums are rerun on the final carrier.
Contributor permission
By approving this pull request,
miles0521confirms that the Apache-2.0 mapping may apply, with no exclusions, to the benchmark-code contributions for which he owns the copyright or has authority to grant the license. Scott confirms the same for his own applicable benchmark-code contributions. This permission applies from the commit that introduces the new mapping; it does not rewrite historical revisions.Validation
status: okLICENSEmatchesLICENSES/Apache-2.0.txtgit diff --checkf115334fcaa16f678e7307970029e868fc2b8a52with the same 141/6/16/checksum result; this SHA must match the remote PR head before reviewPlease review the current-head path mapping and the contributor-permission paragraph above, and confirm that your applicable benchmark-code contributions have no exclusion from the Apache-2.0 grant before approving.