build-stats under-reports settlements reaching the country-named region

[navjottomer]

What happens

When containment cannot be resolved for a settlement, the pipeline attaches it to the country node, and that country node is then published as one of the country's own region records. The result is a division named after the country, sitting inside it:

{"type":"region","id":668,"name":"India","slug":"india","iso_3166_2":null,
 "place_type":"Q6256","latitude":"22.800000","longitude":"83.000000",
 "population":1326093247}

id 668 is the country record's own id. place_type is Q6256 (country) or, more often, Q3624078 (sovereign state) — never an admin1 type. iso_3166_2 is always null.

Scale

Measured across 2026-08-15T0426Z, every country's data/<CC>.ndjson:

• 249 of 255 countries publish a region whose id equals the country's id
• 57,250 settlements are attached to one — 3.4% of the dataset

Worst affected:

settlements	cc	region name	place_type
23,193	LT	Lithuania	Q3624078
4,220	FR	France	Q3624078
3,338	RU	Russia	Q3624078
3,149	DE	Germany	Q3624078
1,010	MA	Morocco	Q41614
971	UA	Ukraine	Q3624078
966	IN	India	Q6256

Lithuania is effectively unusable for a picker. 23,193 of its 23,404 settlements (99%) are in the bucket; its ten real counties hold 211 between them:

23193  —        Lithuania      <-- the country posing as a region
  114  LT-VL    Vilnius County
   27  LT-KU    Kaunas County
   22  LT-SA    Siauliai County
   ...
    0  LT-AL    Alytus County

Why it is a correctness problem, not just a tidiness one

The affected settlements are real and their coordinates are fine — they are simply filed under the wrong parent. For India's 966, plotting them against state bounding boxes puts them all over the country: ~125 in Maharashtra, ~59 in Rajasthan, ~57 in Uttar Pradesh, ~31 in West Bengal, and roughly 490 in southern states. Agra, for instance, appears in the bucket at 27.1601,77.9922 while a separate Agra record sits correctly under IN-UP.

So a consumer asking "which division is this settlement in?" gets an answer that is wrong but well-formed — the failure mode LIMITATIONS.md is otherwise careful to avoid.

It also weakens a stated guarantee. README.md says no region contains the same name twice; it does not say a region may be its country, and a consumer reasonably reads type: region as meaning an administrative division.

Reproducing

python3 - <<'PY'
import json, collections
cid=None; regions={}; counts=collections.Counter()
for line in open('data/LT.ndjson', encoding='utf-8'):
    r=json.loads(line); t=r.get('type')
    if t=='country': cid=r['id']
    elif t=='region': regions[r['id']]=r
    elif t=='settlement': counts[r.get('admin1_id')]+=1
print('country id', cid, '| settlements attached to it:', counts[cid])
print('is it also published as a region?', cid in regions)
PY

Suggested directions

Not prescriptive — you know the containment code. Options that seem open:

1. Do not emit a region whose id is the country's id. Settlements attached to it become genuinely unparented, which at least surfaces the gap instead of hiding it.
2. Emit it, but not as type: region — a distinct type, or an unassigned: true flag, so consumers can tell a real division from a fallback.
3. Improve the fallback itself — for the affected rows the coordinates are present and correct, so a point-in-polygon or nearest-admin1 pass would place most of them. Lithuania suggests the admin1 relation is simply not being picked up for that country at all, which may be a per-country mapping gap rather than a general algorithm problem.

Happy to help with whichever direction you prefer.

Interim handling downstream

placedb.org, which serves these releases, currently relabels the bucket as "name": "Division not recorded" with "unassigned": true, strips the country's population and centroid off it, sorts it last, and excludes it from autocomplete suggestions while keeping its settlements searchable. That is presentation only — it does not attempt to reassign anything — and it will be removed once a corrected release lands.

[navjottomer]

Correction to the framing above

The mechanism in this report is already documented and deliberate. docs/LIMITATIONS.md states it plainly:

Each country also has one region named after the country itself, which catches settlements whose containment reaches no division. It appears only if something lands in it, which is why counts here are typically ISO + 1.

So "the country node is published as one of its own regions" is by design, not a bug, and the original title overstated it. Apologies for that.

What survives, and is the reason this is still worth an issue, is narrower and more specific:

build-stats.json under-reports how many settlements land there.

settlements in a country-named region (shipped data)	57,250
sum(no_division) in build-stats.json	52,734
countries where the region holds more than no_division reports	154

no_division counts settlements whose containment reached no division. A settlement also lands in the fallback when its division was resolved and then removed by the hand-maintained exclusions — ISO groupings, abolished types, Lithuania's 60 municipalities. Containment succeeded, so nothing increments, but the settlement still has no parent.

Lithuania is the case worth looking at: no_division: 43, against 23,193 settlements actually in the region — 99% of the country. The ten counties hold 211 between them. That is the documented municipality exclusion working exactly as described, but its cost is not visible anywhere in the stats.

Faroe Islands and Singapore both report no_division: 0 while every settlement they ship is in the fallback.

Full per-country measurement, reproduction script and suggested directions are now in docs/UNPLACED-SETTLEMENTS.md (6e432e2), linked from LIMITATIONS.md and the README.

Suggested retitle: build-stats under-reports settlements reaching the country-named region.

[navjottomer]

#2 fixes the counting and takes most of the region with it.

The counting

build-stats.json now carries country_region per country, counted on the
rows that actually ship rather than on containment failures, plus
in_country_region over the whole build. It can be checked against the
published file, which is what the original report asked for. no_division is
untouched and still means what it always meant.

The gap you identified is exactly what it looked like: Lithuania reported 43
against 23,193 shipped.

The region itself

Rather than only measure it, three things now place what lands there.

P150. All 60 Lithuanian municipalities point P131 straight at the country —
the county administrations were abolished in 2010, so the county edge carries
an end date and is not truthy — while the ten counties still list their
municipalities under P150. Reading containment from both ends places 22,070 of
Lithuania's 23,193 and 553 of Greece's 708.

Coordinates, against Natural Earth, which is public domain and so clears
the licence bar this project holds itself to. It answers with an ISO 3166-2
code and the code must name a division already shipped for that country. Where
it names another level or an older ISO edition — Natural Earth gives France its
départements while this ships régions — what the code means here is read off
the settlements already placed inside it.

Tier 4. The Faroe Islands ship one region called "Faroe Islands" while
Wikidata records 147 of their 149 settlements under one of 33 municipalities,
every municipality inside a sýsla. Tier 2 missed it because the sýslur state no
P131 at all. Divisions now come from what a country's own settlements say
contains them.

Measured over 2026-08-15T0426Z, all 57,196 rows in a country-named region:

placed by a code this dataset ships	44,558
placed by reading what a code means here	6,685
left	5,953

France 4,220 → 85. India 966 → 64. Lithuania 23,193 → 293.

A correction to the framing, again

"Regionless" was the wrong word, mine included. Almost none of the 5,953 are
places with no division — they are places whose division exists and that no
evidence available here reaches. Kiribati settles it: all 105 of its unplaced
rows fall inside the longitude band of one of the three island groups the
dataset already ships, which lie a thousand kilometres apart. They are atolls
too small to be drawn at 1:10m, not places nobody knows the location of.

Genuinely division-free is Sint Maarten, the Vatican, Gibraltar, Christmas
Island, the Cocos Islands and Pitcairn. On the order of a hundred rows.

One thing found on the way

Q12685199, labelled "Cochin", English description "Synonym for Kochi, India",
one sitelink, no P131 — shipping as a settlement in India's country-named
region while Q1800 Kochi ships properly under Kerala. It carries P460,
said to be the same as, pointing at Kochi.

493 P460 statements — 241 unique pairs — link two rows that both ship here.
Where one side has a division and the other has none, the one without is not a
place: "Warszawa" beside Warsaw, "Łódź" beside Łódź, an item labelled "do not
use" 200 m from Stuttgart. 93 such rows, none with a population, 89 with no
P131 at all. Distance is not consulted, because P460 links confusable places as
readily as identical ones — Hoya in Lower Saxony to La Hoya in Salamanca, 1,778
km apart — and both sides of those pairs have a division, so the rule cannot
reach them.

Not taken

A vote of the nearest already-placed settlements would answer for 10,962 of the
12,638 left before the code mapping ran, and reproduce the known division 96% of
the time on held-out data. 96% over six thousand rows is 240 settlements
confidently filed in the wrong division, which is the failure this issue
objects to. It is documented and refused.

Still open

Marking the region, which was direction 2 of the original report. After all of
the above roughly six thousand rows still sit in a region whose id is the
country's, detectable only by inference. That is an additive schema change and
has not been made here.

None of this is released. The graphs it needs exist in no scan on disk; the
numbers above are measured against the published release and become real at the
next refresh.