Use 572 digital collections class

[ehanson8]

Purpose and background context

Adding DigitalCollections transformer class to extend DspaceDim class based on this metadata mapping. Feedback from stakeholders has been requested so I may delay merging until they have time to weigh-in.

How can a reviewer manually see the effects of these changes?

Review Digital Collections records in our Dev1 UI. Note: Reload the page if you get a 500 error since the Dev1 goes to sleep if unused.

Sample search for "medicine" in the Digital Collections tab

Includes new or updated dependencies?

YES

Changes expectations for external applications?

YES - timdex-pipeline-lambdas and timdex-index-manager will require small updates, PRs coming after these changes are merged

What are the relevant tickets?

• https://mitlibraries.atlassian.net/browse/USE-572

Code review

• Code review best practices are documented here and you are encouraged to have a constructive dialogue with your reviewers about their preferences and expectations.

[Copilot]

Pull request overview

Adds a dedicated DigitalCollections XML transformer to support the Digital Collections source (USE-572), wiring it into the source configuration and validating expected field mappings with new unit tests.

Changes:

• Introduces DigitalCollections transformer subclassing DspaceDim with Digital Collections-specific field overrides.
• Registers the new source in SOURCES config and adds a comprehensive test suite for the transformer.
• Minor refactor in exclusion-list loading and updates Ruff lint ignore list.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
transmogrifier/sources/xml/digital_collections.py	New DigitalCollections transformer extending DspaceDim with source-specific mappings.
transmogrifier/sources/transformer.py	Small refactor to cast exclusion_list_path to str for smart_open.
transmogrifier/config.py	Adds new digitalcollections source entry pointing to the new transformer and base URL.
tests/sources/xml/test_digital_collections.py	Adds unit tests covering the new transformer’s field extraction behavior.
pyproject.toml	Expands Ruff ignore list for additional lint rules.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ghukill]

Left a couple of comments, but overall looks good to me!

I'd be remiss if I didn't ask: we're pretty sure we want to go with DIM? I can appreciate the metadata side of things, but I'm a little bummed that we'll have the more minimal DIM files vs METS which do seem to carry more information about the object (surprisingly). Either way, that's likely a question/discussion out of scope for this PR. I can live with DIM for sure.

While I did leave some comments, I'd like to go ahead with an approval review seeing as I'm out Thursday/Friday of this week and don't want to hold things up. Can always take more passes or revisit things, but everything here looks good in situ.

[ehanson8]

@ghukill Valid question about dim but given the amount of vra metadata in Digital Collections (see Appendix A of the mapping), stakeholders did not want to lose that when I presented them with the pros and cons of mets and dim, including the near-term potential for fulltext with mets. Annoying that we have to make this choice between file info in mets and richer descriptive metadata in dim but I think there is a future discussion of how to address this when we want to get fulltext from Digital Collections (which stakeholders expressed interest in even if they wanted the richer metadata for now)