Skip to content

[feat] OAI-PMH provider, harvest framework, and persistent identifiers - #454

Merged
St4NNi merged 102 commits into
mainfrom
feat/metadata-interop
Aug 11, 2026
Merged

[feat] OAI-PMH provider, harvest framework, and persistent identifiers#454
St4NNi merged 102 commits into
mainfrom
feat/metadata-interop

Conversation

@St4NNi

@St4NNi St4NNi commented Aug 9, 2026

Copy link
Copy Markdown
Member

OAI-PMH data provider and harvest framework over the metadata registry, w3id persistent identifier minting and landing resolution, a registry timestamp index, and durable quarantine for permanently invalid sync events.

Changes

  • OAI-PMH 2.0 data provider: all six verbs over the local realm's metadata registry, unauthenticated by protocol and limited to public documents, with opaque resumption cursors.
  • Harvest framework: repository connector and harvest source models, a fenced idempotent harvest job on the existing job runtime, provenance keyed by namespace and canonical record id with no silent remap, and OAI-PMH as the first harvester kind.
  • w3id persistent identifiers: idempotent minting job keyed by document id, mapping state, public landing resolution redirecting into the guarded RO-Crate read, and 410 tombstones for withdrawn ids.
  • Mutable updated-at timestamp index over the metadata registry riding the same write batch as the record, with lazy cleanup of stale keys, backing incremental OAI listing.
  • Durable quarantine for permanently invalid sync events instead of endless redelivery.
  • Screened egress GET on the blob handle through the egress guard.

References

Closes #338
Relates to #442

St4NNi added 29 commits August 10, 2026 15:49
@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.07048% with 648 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.29%. Comparing base (134a850) to head (051e4ef).

Files with missing lines Patch % Lines
api/src/routes/pid.rs 47.09% 82 Missing ⚠️
operations/src/metadata/forward.rs 84.19% 70 Missing ⚠️
operations/src/metadata/visibility_index.rs 93.59% 67 Missing ⚠️
operations/src/jobs/harvest.rs 91.39% 52 Missing ⚠️
operations/src/sync_quarantine.rs 90.87% 51 Missing ⚠️
api/src/routes/oai.rs 96.39% 46 Missing ⚠️
operations/src/persistent_id.rs 91.48% 40 Missing ⚠️
operations/src/harvest/create_harvest_source.rs 87.73% 32 Missing ⚠️
operations/src/delete_metadata_document.rs 83.85% 26 Missing ⚠️
operations/src/jobs/persistent_id.rs 0.00% 24 Missing ⚠️
... and 20 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #454      +/-   ##
==========================================
+ Coverage   87.11%   87.29%   +0.18%     
==========================================
  Files         380      400      +20     
  Lines      247586   257468    +9882     
==========================================
+ Hits       215679   224766    +9087     
- Misses      31907    32702     +795     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@St4NNi
St4NNi merged commit ed33d68 into main Aug 11, 2026
10 checks passed
@St4NNi
St4NNi deleted the feat/metadata-interop branch August 11, 2026 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[fix] Quarantine permanently invalid sync events

1 participant