Skip to content

fix(wiley-nexus): keep NexusWriter citation numbers aligned with evidence_passages.md - #248

Merged
ryansok merged 2 commits into
microsoft:mainfrom
wiley:fix/nexus-writer-citation-numbering
Oct 27, 2026
Merged

ryansok merged 2 commits into
microsoft:mainfrom
wiley:fix/nexus-writer-citation-numbering

Conversation

@Sahasra98

Copy link
Copy Markdown
Contributor

Summary

In the Wiley Nexus Discovery Engine pipeline, NexusWriter numbered citations by order of first appearance in its own answer instead of reusing the numbers NexusSearch assigns in evidence_passages.md. A paper listed as ### [7] in the evidence could appear as [1] in research_response.md, so the answer could not be checked against its evidence by number.

This PR makes NexusWriter reuse the evidence numbers. Testing on a full pipeline run then showed a second problem: on a long evidence file (20 papers), NexusWriter only saw roughly the first 13 entries, so papers further down got wrong numbers. To fix that, NexusSearch now writes a short ## Index at the top of evidence_passages.md, and NexusWriter takes its numbers from it.

Type of change

  • New agent (under agents/<name>/)
  • Update to an existing agent
  • New starter kit (under starter-kits/<name>/)
  • Update to an existing starter kit
  • Schema change (docs/schemas/**)
  • Workflow / script change (.github/**)
  • Documentation only
  • Other (describe below)

Related issue / tracking

Follow-up to #111. Internal tracking: AISVC-3007.

Schema impact

  • No schema changes
  • Schema changes are backward compatible
  • Schema changes are breaking — add the breaking-change label and link the rollout plan below

Validation checklist

  • Documentation (README.md, schemas) is updated to match the change.
  • No hand-edits to .auto-registry/** (it is generated post-merge).
  • No OS / editor artefacts committed (.DS_Store, .env, *.swp, .idea/, .vs/, etc.).
  • If model-weight files are added, they are Git-LFS tracked, ≤5 GB each, and in an allowed format. — n/a
  • No secrets, customer data, or internal-only URLs are committed.
  • All Markdown links resolve.

What changed

All changes are in agents/wiley-nexus/tasks/. NexusResearcher (agent.yaml) is unchanged: it is a standalone chat agent with no evidence file to match.

File Change
nexus-writer-agent.md Takes each paper's number from the ## Index (falls back to reading every ### [n] entry if there is no Index), matches analysis files by DOI, never renumbers by first appearance, keeps gaps when papers are not cited, orders References by evidence number, and only assigns a new number above the highest existing one, never one already in use.
nexus-search-agent.md evidence_passages.md now opens with ## Index, one line per publication: [n] DOI — Title, matching the ### entries.
nexus-research-task.md Task 1 asks for the Index and adds a validation check for it. Task 3 no longer asks for "numbering sources in order of first appearance", which contradicted the agent.
tasks/README.md The Outputs table mentions the Index.

Testing

All runs in Discovery Studio with the updated agent instructions.

Test Result
Chat test, sample evidence (8 papers; a full-text paper [6] cited first) Pass. [6] stayed [6]; the old behaviour would make it [1].
Chat test, sample evidence, answer scoped to some papers Pass. References [1], [3], [5], [6], [8], gaps kept.
Full pipeline run, 20-paper evidence file, writer change only 13 of 14 citations correct. The full-text paper at ### [19] was cited as [20], then [14] on a rerun. NexusWriter never cited entries [14]–[20], even on-topic ones, consistent with the long file being only partly read. This led to the Index change.
Full pipeline run with the Index (12 papers, 5 read in full) Pass. All 12 citations match the Index, including all 5 full-text papers ([1], [2], [4], [9], [12]). The answer first cites [1], [4], [2], [12], [9] and none were renumbered.

Not yet covered: the last run's evidence file had 12 papers, below the length at which the earlier run lost entries, so the Index was not exercised against a truncated read. It is designed for that case, since every number is in the file's first lines.

Reviewer notes

  • Side finding: before this change, later papers in a long evidence_passages.md were being silently left out of the final answer. The old renumbering hid this, because the answer never pointed back to evidence numbers. The Index makes those papers visible to the writer again.
  • Setup note, unrelated to this change: when copying the pipeline agents into a new project, the Nexus MCP tool must be set to "Always auto-approve all tools" on each agent. Otherwise tasks finish with empty results and no error.

@Sahasra98
Sahasra (Sahasra98) requested a review from a team as a code owner October 20, 2026 14:13
@github-actions github-actions Bot added 3p-contribution Authored by a third-party partner (party='3p' in metadata) markdown-only Every file changed by this PR is Markdown pr-validation-failed The pr-review workflow's validator failed; see inline comments. labels Oct 20, 2026
@github-actions

This comment was marked as resolved.

…mbers

NexusWriter renumbered citations by first appearance in its own answer,
so [n] in research_response.md did not match [n] in evidence_passages.md.

- NexusWriter reuses each publication's evidence number, matches analysis
  files by DOI, keeps gaps, orders References by evidence number, and
  never reuses a number that belongs to another paper.
- NexusSearch writes a "## Index" ([n] DOI - Title) at the top of
  evidence_passages.md so the writer sees every number even when a long
  file is only partly read; NexusWriter takes its numbers from it.
- Task 1 asks for and validates the Index; Task 3 no longer asks for
  first-appearance numbering; the pipeline README lists the Index.
@Sahasra98
Sahasra (Sahasra98) force-pushed the fix/nexus-writer-citation-numbering branch from 3d0fad1 to 13e4502 Compare October 20, 2026 14:48
@github-actions github-actions Bot added 3p-contribution Authored by a third-party partner (party='3p' in metadata) markdown-only Every file changed by this PR is Markdown pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed 3p-contribution Authored by a third-party partner (party='3p' in metadata) pr-validation-failed The pr-review workflow's validator failed; see inline comments. markdown-only Every file changed by this PR is Markdown labels Oct 20, 2026
@Sahasra98

Copy link
Copy Markdown
Contributor Author

@microsoft-github-policy-service agree company="Wiley"

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The fallback numbering instruction can assign the same citation number to multiple publications absent from the evidence file.

1 open issue
What changed in this PR

Aligns Wiley Nexus pipeline citations with evidence-file numbering so answers can be checked against their sources.

Changes:

  • Adds a compact publication index to search output.
  • Updates writer instructions to preserve evidence numbers and citation gaps.
  • Aligns task instructions, validation criteria, and output documentation.
File Description
agents/​wiley-nexus/​tasks/​README.md Documents the evidence index.
agents/​wiley-nexus/​tasks/​nexus-writer-agent.md Preserves evidence numbering in citations and references.
agents/​wiley-nexus/​tasks/​nexus-search-agent.md Adds the index before detailed evidence.
agents/​wiley-nexus/​tasks/​nexus-research-task.md Updates task requirements and index validation.

Review effort: Balanced

Comment thread agents/wiley-nexus/tasks/nexus-writer-agent.md Outdated
…umber

Addresses review feedback: two cited publications absent from
evidence_passages.md could both receive the same new number. Each now
gets the next unused number above the evidence maximum, and keeps it
for every citation.
@github-actions github-actions Bot added 3p-contribution Authored by a third-party partner (party='3p' in metadata) markdown-only Every file changed by this PR is Markdown pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed 3p-contribution Authored by a third-party partner (party='3p' in metadata) pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. markdown-only Every file changed by this PR is Markdown labels Oct 21, 2026

@faranm-stack faranm-stack left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to approve

Comment thread agents/wiley-nexus/tasks/nexus-writer-agent.md Outdated

@ryansok ryansok left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve

@ryansok
ryansok merged commit 4f9879b into microsoft:main Oct 27, 2026
16 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3p-contribution Authored by a third-party partner (party='3p' in metadata) markdown-only Every file changed by this PR is Markdown pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants