|
| 1 | +# Ancestral-origin locality icicle on the public Y tree |
| 2 | + |
| 3 | +**Status:** proposed (2026-08-06). **AppView half only** — the Navigator publisher is deferred by |
| 4 | +the project owner, so this ships *dark*: schema, ingest, aggregate and view are built and tested |
| 5 | +against seeded records, and the surface fills when a PDS starts publishing. |
| 6 | +**Scope:** a new `com.decodingus.atmosphere.ancestralOrigin` lexicon, `fed.ancestral_origin` |
| 7 | +(migration 0074), a jetstream arm, `du_db::origins`, and a server-rendered icicle at |
| 8 | +`/ytree/node/:name/origins`. No change to `core.biosample`, no change to placement. |
| 9 | +**Privacy posture:** unchanged from `biosample-identifier-dedup.md` — the corpus carries no |
| 10 | +living-donor PII, and MDKA (surname / origin / birth-year of the earliest paternal-line ancestor) is |
| 11 | +**genealogical context, not PII**. This design does not relax that; it *enforces* it (§2). |
| 12 | + |
| 13 | +## 1. Why |
| 14 | + |
| 15 | +The public Y tree renders as a cladogram (`du-web/src/tree_layout.rs`) with a per-clade Leaflet map |
| 16 | +in the "Geography & Time" panel. Neither answers the question a surname project asks: **as this |
| 17 | +branch splits, where do the lines go?** ytree.net answers it by putting geography onto the phylogeny |
| 18 | +itself — depth down the page, a block's height its elapsed time, each block spanning its |
| 19 | +descendants. This adds that view with locality as the fill. |
| 20 | + |
| 21 | +It only means anything in the genealogical era. Deeper than ~1,500 ybp every block aggregates to |
| 22 | +"Europe" and the counts explode, so the view is age-gated rather than offered tree-wide (§5). |
| 23 | + |
| 24 | +**The data gap, measured on `decodingus_cutover` (2026-08-06).** The AppView's only locality datum |
| 25 | +is `core.specimen_donor.geocoord`; there is no country and no place text, and legacy had none |
| 26 | +either, so the ETL did not drop one. Of **9,642 placed Y samples, 1,380 carry a coordinate**, and |
| 27 | +that coverage is entirely ancient/academic: |
| 28 | + |
| 29 | +| source | placed | with geocoord | |
| 30 | +|---|---:|---:| |
| 31 | +| `EXTERNAL` (ancient + academic) | 1,760 | 1,377 | |
| 32 | +| `STANDARD` (`cohort=bigy` D2C tips) | 7,882 | **3** | |
| 33 | + |
| 34 | +The genealogical era — the only era this view is for — has effectively no locality data. That is |
| 35 | +what the lexicon exists to supply. |
| 36 | + |
| 37 | +## 2. What may cross the wire, and how it is enforced |
| 38 | + |
| 39 | +The posture is already the project's (`biosample-identifier-dedup.md` §Privacy posture). What is new |
| 40 | +is that this design **enforces** it at ingest rather than asserting it in prose. A record failing any |
| 41 | +gate is **rejected**, not merely un-rendered: |
| 42 | + |
| 43 | +1. **Surname only.** No given name, ever. The publisher derives it; the AppView independently |
| 44 | + rejects a `surname` containing whitespace or more than one name token, so a buggy or hostile |
| 45 | + client cannot leak a given name through a field labelled `surname`. |
| 46 | +2. **Date ceiling — `birthYear <= 1900`.** A person born in 1900 is 126 today. This is the check |
| 47 | + that makes "not PII" verifiable rather than asserted. |
| 48 | +3. **Precision ladder when the birth year is absent.** With a birth year: place text + coarsened |
| 49 | + coordinate. Without one: **country only** — place text and coordinate are dropped at ingest. |
| 50 | +4. **Coordinates coarsened to 2 decimal places (~1 km).** Applied at publish *and* re-applied at |
| 51 | + ingest, because the client cannot be trusted to have done it. A county-scale view cannot use more |
| 52 | + precision; full precision plus a surname narrows to one family. |
| 53 | +5. **The join key is never rendered.** Resolution runs through an FTDNA kit id (§4), and every vendor |
| 54 | + namespace is `is_public = false` (`du_db::identifier::is_public_namespace`). The icicle shows |
| 55 | + `Kane · Co. Clare`; the kit number must not reach any public projection. |
| 56 | + |
| 57 | +**The bulk-load exclusion stands.** `du-jobs/src/import_kit_identifiers.rs:17` records the decision |
| 58 | +that MDKA "enters only when a PDS publishes the sample, never from this bulk load." This design does |
| 59 | +not create a manifest or curator path. It is the reason the view ships dark, and that is accepted. |
| 60 | + |
| 61 | +**Two migration headers say the opposite and are deliberately not edited.** Both repos use |
| 62 | +`sqlx::migrate!`, which checksums applied migrations — editing a comment in |
| 63 | +`0030_mdka.up.sql` (Navigator) or `0012_fed_reporting.sql` (AppView) would fail every existing |
| 64 | +database with `VersionMismatch`. **This document is the amendment of record**, and migration 0074's |
| 65 | +header points back to it. |
| 66 | + |
| 67 | +The D4 assertion store's PII rail (`research.assertion` rejecting `MDKA_IS`) **stands unchanged**. |
| 68 | +It governs assertions made *about a living research subject* within a project, which is a different |
| 69 | +question from publishing a deceased ancestor's parish — and the rail is what keeps the two apart. |
| 70 | + |
| 71 | +## 3. The record |
| 72 | + |
| 73 | +`com.decodingus.atmosphere.ancestralOrigin`, one per `(biosample, lineage)`: |
| 74 | + |
| 75 | +```jsonc |
| 76 | +{ |
| 77 | + "biosampleRef": "at://did:plc:…/com.decodingus.atmosphere.biosample/…", // when federated |
| 78 | + "externalIds": [{ "namespace": "FTDNA", "value": "B5163" }], // the join that fires |
| 79 | + "lineage": "Y_DNA", // Y_DNA | MT_DNA |
| 80 | + "surname": "Kane", // single token; never a given name |
| 81 | + "originPlace": "Creegh South, Co. Clare, Ireland", // as recorded; normalized server-side |
| 82 | + "originCountry": "Ireland", |
| 83 | + "birthYear": 1830, |
| 84 | + "deathYear": 1908, |
| 85 | + "lat": 52.75, // 2dp |
| 86 | + "lon": -9.43, |
| 87 | + "createdAt": "2026-08-06T…Z" |
| 88 | +} |
| 89 | +``` |
| 90 | + |
| 91 | +**Place text is published as recorded and normalized in the AppView**, not at the edge. One |
| 92 | +implementation, fixable without a client release, and re-runnable over records already ingested. The |
| 93 | +normalizer is a pure function in `du_db::place` with a country/admin synonym table — the corpus needs |
| 94 | +it: `Ireland` / `Republic of Ireland` / `ireland`; `UK` / `United Kingdom` / `Scotland`; `Co. Cork` |
| 95 | +vs `Cork`; `VA` vs `Virginia`; UK postcodes embedded mid-string (`Moulin, Pitlochry PH16 5EP, UK`). |
| 96 | +705 distinct raw admin strings across the reference corpus. |
| 97 | + |
| 98 | +## 4. Resolving a record to a placed sample |
| 99 | + |
| 100 | +The obvious join — `core.biosample.atproto->>'uri' = biosample_ref`, as `discovery.rs:185` does — |
| 101 | +**matches nothing**: zero placed samples carry an at-uri, because the tips were bulk-loaded rather |
| 102 | +than federated. |
| 103 | + |
| 104 | +The working key already exists. **All 7,548 placed bigy tips carry an `FTDNA` row in |
| 105 | +`core.biosample_identifier`** (migrations 0059/0060, built precisely to "match a re-published donor |
| 106 | +to its existing biosample"). So resolution is `(namespace, value)` against that table, with the |
| 107 | +at-uri as a fallback for genuinely federated samples. No re-federation, no new identity work. |
| 108 | + |
| 109 | +## 5. The view |
| 110 | + |
| 111 | +`/ytree/node/:name/origins` (+ the mt sibling), a server-rendered inline SVG — no client layout |
| 112 | +library, matching `tree_layout.rs`. |
| 113 | + |
| 114 | +- **Geometry**: depth → y; each node a rect spanning its subtree's horizontal extent; children flush |
| 115 | + against the parent's underside, so containment carries descent and no connector is drawn. |
| 116 | +- **A block shows its equivalent SNPs, and its height is their count** — one line each, nothing |
| 117 | + elided, exactly as the Big Tree draws it. The mutations on a branch are unordered, so the list |
| 118 | + *is* the block. Vertical position is therefore cumulative: how far down a block sits is the |
| 119 | + mutations accrued along the path to it, and the left gutter rules that off in SNPs. |
| 120 | + |
| 121 | + **Do not size blocks by the age model.** Both obvious forms were tried against real data and both |
| 122 | + fail: |
| 123 | + |
| 124 | + 1. *A node's own `formed_ybp` → its own `tmrca_ybp`.* These are independent point estimates under |
| 125 | + no monotonicity constraint; they agree with the parent's TMRCA on only **898 of 10,252 edges**, |
| 126 | + and **4,243 (41%)** have the child forming *earlier* than its parent's split — so children |
| 127 | + draw on top of their parents. `R-A13318` (formed 1622) landed at exactly its parent |
| 128 | + `R-S764`'s y. |
| 129 | + 2. *Parent TMRCA → own TMRCA.* Monotone, so containment holds — but degenerate: |
| 130 | + `formed_ybp == tmrca_ybp` on **41% of terminal branches and 26.5% of internal ones**, collapsing |
| 131 | + the branch to a point. On R-DF85 at depth 4 that left **30 of 75 blocks unable to show a single |
| 132 | + one of their SNPs** — `R-BY18328` got 3px of span for 9 mutations, `R-BY170664` 16 SNPs in an |
| 133 | + 18px sliver. |
| 134 | + |
| 135 | + SNP count never degenerates, and it is still a time axis: measured on this tree, branch length |
| 136 | + tracks SNP count at **r = 0.975, ≈69 years per mutation**. Ages keep their two real jobs — gating |
| 137 | + the view to the genealogical era, and labelling each block — they simply do not drive geometry, |
| 138 | + because a per-branch estimate is precisely what is missing or degenerate when a block most needs |
| 139 | + a height. |
| 140 | + |
| 141 | +- **Colour belongs to the men, not the branches.** An early cut tinted each block by the composition |
| 142 | + of its descendants' origins. That asserts something the data does not support: a branch has no |
| 143 | + locality, only the men standing on it do, and a modal-origin tint reads as a claim about the whole |
| 144 | + lineage. Colour lives on each man's box, keyed to his own MDKA; the legend and table carry the |
| 145 | + composition that explains those colours. |
| 146 | +- **Tips**: one leaf box per placed sample carrying an origin — `Kane · Co. Clare`, coloured by his |
| 147 | + own locality. Never the kit id. A man too narrow to label is counted rather than drawn as an |
| 148 | + unreadable sliver, and the count is stated. |
| 149 | +- **"No locality recorded" is a visible category**, in the legend and the table, never bare |
| 150 | + background. A view of who published is not a view of where a branch is from, and the difference |
| 151 | + must be on screen. |
| 152 | +- **Colours**: categorical, colourblind-safe, legible in both themes; assigned by frequency rank |
| 153 | + *within the rendered subtree* (deterministic, tie-broken on name), top N distinct + a neutral |
| 154 | + "other". |
| 155 | +- **Era gate**: serves nodes with `tmrca_ybp <= 1500` (adjustable within bounds). Above the cutoff it |
| 156 | + renders the breadcrumb, one line of explanation, and links down to eligible children rather than |
| 157 | + drawing a block that means nothing. |
| 158 | +- **Every branch in the window is drawn, origins or not.** Origins are an overlay on the tree, not |
| 159 | + a filter of it: a branch with no locality data is still part of the clade's shape, and hiding it |
| 160 | + would misrepresent the phylogeny to make a sparse overlay look dense. Legibility is bounded by |
| 161 | + the depth selector instead (default 4 levels), which folds rather than drops — folded branches |
| 162 | + are marked, and their men still count in their nearest drawn ancestor. |
| 163 | +- **A man gets a box only where his own branch is drawn.** Attributed upward from a folded branch |
| 164 | + he still counts in the composition, but is not given a box under a branch that is not his; on |
| 165 | + R-DF85 one boundary block had otherwise absorbed 179 men and stacked them 90 rows deep. Drilling |
| 166 | + in draws him where he belongs. |
| 167 | +- **De-novo nodes stay hidden but their men still count.** A sample placed on an auto-named node is |
| 168 | + attributed to the nearest named ancestor, as the public tree already does for sample tips. |
| 169 | + Dropping it instead made every band above it understate its own composition. |
| 170 | +- **No silent caps.** Pruned branches, samples with no published origin, and the placed total are |
| 171 | + all stated on the page. |
| 172 | + |
| 173 | +## 6. Phasing |
| 174 | + |
| 175 | +1. `du_db::place` normalizer — pure, unit-tested. *(No wire format, nothing published.)* |
| 176 | +2. Migration 0074 + `fed::ancestral_origin` + the jetstream arm + every gate in §2. |
| 177 | +3. `du_db::origins` aggregate + `origins_layout` + route + template + i18n. |
| 178 | +4. **Deferred, Navigator:** the lexicon's client half — `AncestralOriginRecord`, the surname |
| 179 | + splitter, and the publish predicate (the workspace holds primary data for the subject **and** |
| 180 | + `ftdna_member.publicly_shares = 1` or there is no roster row). Measured on the reference |
| 181 | + workspace: 583 Y MDKA rows sit on subjects with primary data, 558 of them publicly-sharing — |
| 182 | + **the 25 that are not must never publish.** |
| 183 | + |
| 184 | +## 7. Open items |
| 185 | + |
| 186 | +- **The precision ladder (§2.3)** is a proposed default, not a derived rule. It withholds place-level |
| 187 | + detail for the ~57% of MDKA rows with no birth year. |
| 188 | +- **The 1900 ceiling** is a round number, not a legal standard. Cheap to set now, expensive to lower |
| 189 | + once records exist. |
| 190 | +- **Retraction.** A withdrawn consent needs the PDS record deleted *and* the mirror tombstoned. The |
| 191 | + jetstream `delete` path (`jetstream.rs:163`) is the mechanism; the workflow is unspecified. |
| 192 | +- **The view ships empty** until the Navigator half lands. That is a consequence of §2's bulk-load |
| 193 | + exclusion, not a defect. |
| 194 | +- **mtDNA** costs almost nothing extra (the lexicon is lineage-keyed). Ship Y first and validate there. |
0 commit comments