analysis: consider ga4gh slides

[bwalsh]

See ga4gh connect slides

---

Issue Title

Add genomic fingerprint index for privacy-preserving duplicate detection

---

Summary

Add an optional genomic fingerprint index to vrs-matcher for privacy-preserving duplicate detection and fast candidate prefiltering.

This should complement the existing VRS allele-based matcher rather than replace it.

---

Background

vrs-matcher already supports:

• VRS-annotated VCF ingestion
• SQLite-backed sample allele indexing
• Jaccard and genotype concordance scoring
• Pluggable matcher algorithms

The current model answers:

How similar are two samples based on shared VRS allele identifiers?

A genomic fingerprint layer would answer:

Have we likely seen this individual before, without exposing the full genotype?

---

Motivation

For cohort deduplication, data release QC, and federated discovery, it is useful to identify likely duplicate samples before performing full VRS-set comparison.

A fixed SNP fingerprint panel could provide:

• Fast duplicate detection
• Reduced comparison space for one-vs-all matching
• Privacy-preserving sample identity confirmation
• Future support for error-tolerant matching

---

Proposed Design

Add a new optional fingerprint subsystem:

VCF/BCF
  |
  +-- existing VRS sample_allele index
  |
  +-- fingerprint panel extraction
          |
          +-- canonical SNP genotype vector
          |
          +-- fingerprint hash
          |
          +-- sample_fingerprint table

Suggested schema:

CREATE TABLE IF NOT EXISTS sample_fingerprint (
    sample_id TEXT,
    panel_name TEXT,
    panel_version TEXT,
    algorithm TEXT,
    fingerprint TEXT,
    missing_sites INTEGER,
    total_sites INTEGER,
    PRIMARY KEY (sample_id, panel_name, panel_version, algorithm)
);

CREATE INDEX IF NOT EXISTS idx_fingerprint
ON sample_fingerprint(fingerprint);

---

Suggested CLI

vrs-matcher build-fingerprints cohort.vcf.gz \
  --db matches.db \
  --panel panels/ncpi-default-v1.tsv \
  --algorithm sha256

vrs-matcher fingerprint SAMPLE_A \
  --db matches.db

vrs-matcher duplicate-search SAMPLE_A \
  --db matches.db \
  --panel ncpi-default-v1

---

Initial Scope

Phase 1 should implement exact fingerprints only:

• Versioned SNP panel
• Canonical SNP ordering
• Genotype extraction from VCF/BCF
• Deterministic serialization
• SHA-256 hash
• Exact duplicate lookup

Approximate or property-preserving hashing should remain future work.

---

Future Work

• Locality-sensitive hashing
• Property-preserving hashing
• Error-tolerant duplicate detection
• First-degree / second-degree relative detection
• Plugin integration for fingerprint-aware ranking

---

Acceptance Criteria

• Add a versioned SNP fingerprint panel format
• Add sample_fingerprint table
• Add fingerprint builder from VCF/BCF
• Add CLI commands for building and querying fingerprints
• Add tests using examples/example-cohort.vcf
• Document privacy limitations
• Document relationship to the existing VRS allele matcher

---

Design Note

The fingerprint layer should remain orthogonal to the existing VRS matcher.

• VRS allele index: exact variant-level similarity
• Genomic fingerprint index: sample identity / duplicate prefiltering

This separation preserves the current matcher architecture while adding a practical privacy-preserving sample identity layer.

Potential implementations:

• SHA256Fingerprinter
• PPHFingerprinter
• LSHFingerprinter
• MinHashFingerprinter

---

flowchart TD
    A[VCF / BCF input] --> B[VRS-annotated sample ingestion]
    A --> C[Fingerprint panel extraction]

    B --> D[(sample_allele SQLite table)]
    D --> E[VRS allele matcher]
    E --> F[Jaccard similarity]
    E --> G[Weighted genotype concordance]

    C --> H[Canonical SNP genotype vector]
    H --> I[Fingerprint algorithm]
    I --> J[(sample_fingerprint SQLite table)]

    J --> K[Duplicate search]
    J --> L[Candidate prefiltering]

    K --> M[Likely duplicate samples]
    L --> E

    E --> N[Final ranked sample matches]

    subgraph Existing vrs-matcher
        B
        D
        E
        F
        G
    end

    subgraph Proposed fingerprint extension
        C
        H
        I
        J
        K
        L
    end

Loading