[costero-e]

Summary

• Add new node-specific SOP template GDI-SOP0012_withdraw_dataset_from_node_allele_frequency_beacon.md.
• Convert the reviewed draft into repository-compliant Markdown and align it with the SOP template, style guide, and glossary rules.
• Add the allele frequency beacon specific withdrawal of the broader dataset-withdrawal workflow and link it from SOP0009.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New content (non-breaking change which adds new content)
• Modified content (non-breaking change which modifies existing content)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Motivation and Context

Issue #65 requested a node-specific SOP describing how a node allele frequency beacon should handle dataset withdrawal. This PR adds the allele frequency beacon of that workflow as a reusable node-specific template.

References

• Addresses [SOP Request] Dataset Withdrawal from Node Allele Frequency Beacon #65

Changes Introduced

• Added sops/node-specific/GDI-SOP0012_withdraw_dataset_from_node_allele_frequency_beacon.md
• Updated sops/european-level/GDI-SOP0009_dataset-withdrawal.md to link to the new nodes' allele frequency beacon SOP from step 8.7
• Updated sops/README.md with the new SOP index entry
• Updated CHANGELOG.md under Unreleased
• Updated docs/GDI-SOP_charter.md with the new Glossary entries

Review

The SOP content is based on the reviewed draft prepared for issue #65 and converted into the repository template. In this PR, the SOP is limited to deletion of the dataset from the allele frequency beacon. Full or partial withdrawal don't apply as frequencies are calculated per dataset and statically, meaning that the removal of a dataset doesn't affect other datasets and global counts are to be made on the client side and on the fly.

Additional Notes

• The SOP is added as a generic node-specific template, not as a node instance.
• Auditing is to be done manually per node, until beacon provides this feature in a future release.
• There is no way in the Beacon specification that allows a beacon instance to currently show a metadata response with the history of datasets that existed and were removed. We need a clearer discussion before this can be retrieved via a simple beacon query.
• This PR will have conflicts with the ongoing Add SOP0011 for soft withdrawal from node FAIR Data Point #71

Checklist:

General Compliance:

• My changes follow the code style of this project (GDI SOP Style Guide) and the file naming conventions of the file accessioning proposal.
• I have verified that all new updated content is accessible, including checking that all external references are readable (i.e., no broken links). These may include references to external resources that should be resolvable, and internal references among SOPs.
• I have properly added this PR's changes to the repository CHANGELOG.md.

Only applicable if the PR includes new, or changes to, GDI SOPs (i.e., documents at sops/):

• My SOP-related changes adhere to the Generic SOP Template, including format and required fields.
• I have consulted the Charter, ISM, and ORR documents to ensure compliance.
• I am complying with the established procedure for SOP creations and modifications, including respecting review phases and notifying needed contributors for reviews.

[costero-e]

I have added @AlesCUMSI to the review as he is an active beacon user and I think he can provide some help. Thank you, @AlesCUMSI. Let me know if you don't have the time now to do it and I will choose someone else.

[AlesCUMSI]

Looks good overall 👍 I’ve left a few minor suggestions regarding wording, clarity, and consistency.

[AlesCUMSI]

You have redinex in both hard deletions, but not here. Is this correct?

[costero-e]

Yes, because in the soft one, the variants are never removed 👍

[M-casado]

If variants are never removed, then can the effects of the presence of that data propagate to other datasets?

In other words, the effective deletion of the dataset would not be real, as it would not change the AFs?

[costero-e]

The AFs are statically calculated, not on the fly. There's confusion about this, because we are not allowing progressive updates. Everything works as a single package. This is what was agreed in GDI, as for now. So, no, no propagation possible.

[costero-e]

I have now finished adding @AlesCUMSI suggestions. Thanks for the useful feedback @AlesCUMSI , it helped a lot!

[M-casado]

Great work @costero-e . Also thanks to @AlesCUMSI , as many of my comments went in line with his.

I have some comments that I left in line, and some general comments here:

• If the soft deletion is defined as data being retained, isn't it inconsistent to use db.datasets.deleteMany to delete the dataset from mongoDB? I'm not familiar with AF Beacon, so maybe it's not, but the command sounds like we're deleting data.
• The dataset_config.yml referenced as per the documentation (in main) here contradicts the isDeprecated in this SOP. Either isTest and isSynthetic are not the only values, or... something is off
• Verification in step 4 doesn't distinguish deletion type: it says to verify "datasets and variants" queries for both types, but if I went through the soft-deletion path, I only removed dataset records (not variants).
• Not sure if this is meant in the Beacon spec, but isDeprecated flag applies to both types of deletion without explanation. For hard-deletion everything is already deleted, so would we be adding a flag to a dataset that is gone? Is this the way to audit changes in the Beacon?

[M-casado]

Missing "AF" for Array Frequency

[M-casado]

In both the charter and the SOP0012

[AlesCUMSI]

If i might make a suggestion based on @M-casado comment (If the soft deletion is defined as data being retained, isn't it inconsistent to use db.datasets.deleteMany to delete the dataset from mongoDB? I'm not familiar with AF Beacon, so maybe it's not, but the command sounds like we're deleting data.): What if we nothing is deleted soft-delete, mainteiner just needs to remove dataset id from beacon/permissions/datasets/datasets_permissions.yml file? If my memory serves me correct, this prevents beacon from returning dataset (and g-variants connected to it)?

[costero-e]

Regarding the soft-deletion, if we only modify the datasets_permissions.yml file, the dataset will continue appearing in the /datasets endpoint, which for beacon means: we have this dataset and you will be able to retrieve it depending on your permissions. I think this is not the situation we want, exactly. Deleting just the datasets entry will block any beacon response to return anything from this dataset. This is a very small record, that only contains metadata for the dataset, and I think is just what a soft-deletion pursues, deletion of metadata regarding a dataset (at the end is soft-deletion, something, very minimal, has to be deleted). What hard-deletion does is deleting the much bigger amount of variants, which can't be rolled back with a simple insert. For me, the approach I did is correct.

[omllobet]

Hi Oriol,

Thanks for the corrections.

In the section about "Remove dataset permission", I think it would be useful to remind the operator to save the relevant information for auditing purposes before removing the dataset from datasets_permissions.yml, rather than only describing the removal step, specially for controlled access.

Also, in the diagram, nodes K and L appear to have the same label, which results in "Remove dataset permissions" being shown twice.

[costero-e]

Hi @omllobet,

Thanks a lot for your useful feedback. I have addressed now the identified issues.

Let me know if you find anything else that is not making sense.

Best!

[omllobet]

I approve this SOP as a member of the OC

[omllobet]

I approve this SOP as a member of the SDPC

[M-casado]

Status summary:

• We are still pending another approver (besides @omllobet). @GabiRinck is listed as an approver in the RR section of the SOP0012, which could account for it. Have you officially approved this SOP as a member of both committees, Gabi?
• @costero-e please add @omllobet as an approver in the table within SOP0012.
• The conflicts are minimal and can easily be resolved. Essentially it's just merging with what the SOP0011 added to the shared files.

[GabiRinck]

Thanks, Marcos,
Oscar approved on behalf of the OC & SDPC
Kjell (@kjellp) volunteered as 2nd approver for the OC & Nejc (@NNadizar) as 2nd approver for the SDPC - apologies for not tagging you as reviewers before. Many thanks!

[M-casado]

@costero-e - Please add the approvers with their roles to the table in the SOP before merging

[costero-e]

Done @M-casado

[kjellp]

I've commented a few things, most notably the test after hard-deletion to check for success.

If these are handled satisfactory I'm happy to approve the SOP on behalf of the OC.

[kjellp]

I'm happy to approve the SOP for the OC now.

[NNadizar]

Reviewed. I have no blocking comments. Approved as a member of SDPC.

Just a minor style remark, in text (not code) sometimes DB (in mongoDB) is in all caps and sometimes in small caps.

[M-casado]

Notification sent to the Management Board. By October 7th 2026, if no issues are raised, this SOP will be formally authorised and the PR can be merged.