Add SPARQL skills: write-ltlod-sparql (mechanics) + investigate-ltlod (recipes)

.claude/skills/investigate-ltlod/SKILL.md

@@ -0,0 +1,82 @@
+---
+name: investigate-ltlod
+description: Answer an investigative or journalistic question from the LTLOD (Lithuanian Linked Open Data) datasets — who controls or funds what, how much public money a company or person receives, and which hidden connections the linked registers reveal. Use when asked to follow a company's public money, check a politician's declared interests against contracts or funds, find suppliers below the procurement radar, trace campaign donors, or otherwise connect the Lithuanian registers; it names the domains and join keys to use and the caveats that decide whether a finding is solid.
+---
+
+# Investigate LTLOD
+
+Turn an investigative **question** into the right query. For the mechanics of writing and
+running the query (named-graph model, `run.sh` vs. the throttled endpoint, the query-shape
+rules), load **`write-ltlod-sparql`** — every recipe here produces a query that must obey it.
+49 worked queries with explanations are in **`queries/EXAMPLES.md`**; the URI map is in
+**`etl/URI-SCHEME.md`**. Start from the closest example, don't write blank.
+
+## The join keys (this is the whole point)
+
+The datasets are separate registers; their investigative power is the keys that connect them.
+**The same key is the same IRI in every domain, so joins need no lookup.**
+
+| Key | IRI | Connects (predicate → the shared IRI) |
+|---|---|---|
+| **JAR code** (9-digit company code) | `legal-entities/{code}/#this` | procurement buyer/contractor (`epo:playedBy`), taxpayer (`ltlod:taxpayer`), Sodra employer (`ltlod:employer`), campaign donor (`ltlod:donor`), EU-fund beneficiary/partner (`ltlod:beneficiary`/`ltlod:fundingPartner`), insolvent company (`ltlod:insolvent`), invoice buyer/supplier (`ltlod:spendingBuyer`/`ltlod:spendingSupplier`), land applicant (`ltlod:applicant`), VTEK interest org (`ltlod:interestOrganization`), lobbyist/agent (`ltlod:agent`) |
+| **`asmens_id`** (Seimas person id) | `persons/{asmens_id}/#this` | Seimas members, officials, VMI/PINREG declarants, VRK candidates, lobbied contacts, membership/vote history |
+| **AR code** | `admin-units/{code}/#this` | the municipality/county of an observation or location (`sdmx-dimension:refArea`, `schema:location`) |
+| **calendar day/year** | `calendar/{yyyy}[/{mm}/{dd}]/#this` | every dated event/period (`dct:temporal`) — "what happened on/in …" |
+| **Wikidata QID** | via `owl:sameAs` | aligned persons and places |
+
+So *"which company links procurement and EU funds?"* is just: the same `legal-entities/{code}/#this`
+is the object of `epo:playedBy` in one graph and `ltlod:beneficiary` in another.
+
+## Where to look (container → what it holds)
+
+`legal-entities/` JAR register · `persons/` Seimas members & officials · `procurement-contracts/`
+VPT contracts · `taxes/` VMI arrears + taxes paid · `employment/` Sodra employer stats ·
+`eu-funds/` 2014-2020 EU structural funds · `insolvency-cases/` bankruptcies/restructurings ·
+`public-spending/` public-sector invoice aggregates · `land-acquisitions/` NŽT land applications ·
+`campaign-finance/` + `election-results/` VRK · `interest-declarations/` PINREG private interests ·
+`lobbying-declarations/` + `lobbyists/` VTEK SKAIDRIS · `declarations/` VMI asset declarations ·
+`seimas` votes/bills/sittings · `admin-units/` + `calendar/` + `facilities/`.
+
+## Recipes (question → approach → start from)
+
+- **Follow a company's public money.** Join `legal-entities/{code}/#this` across procurement
+  (`epo:playedBy`), taxes (`ltlod:taxpayer` — arrears/paid), eu-funds (`ltlod:beneficiary`),
+  public-spending (`ltlod:spendingSupplier`), insolvency (`ltlod:insolvent`). → `eu-funds-and-procurement.rq`,
+  `taxes-contractors-with-arrears.rq`, `contractors-in-insolvency.rq`.
+- **A politician's declared interests vs. public money.** VTEK `ltlod:interestOrganization` →
+  the same JAR IRI as a procurement contractor / eu-fund beneficiary / invoice supplier. Filter the
+  tie `?role skos:notation` to ownership/control (`akcininkas, dalininkas, savininkas, vadovas,
+  valdybos-narys, …`). → `vtek-interest-procurement.rq`, `pinreg-ties-eu-funds.rq`,
+  `pinreg-ties-public-spending.rq`.
+- **A supplier below the procurement radar.** public-spending suppliers billed a lot via many
+  small invoices (group per supplier, avg invoice under the threshold) who hold **no** procurement
+  contract. → `suppliers-under-threshold.rq`.
+- **Campaign donors and what they got.** campaign-finance `ltlod:donor` → procurement contractor /
+  later insolvent. → `finance-donors-procurement.rq`, `donors-later-insolvent.rq`.
+- **New companies / land.** land-acquisitions `ltlod:applicant` × legal-entities registration date
+  × Sodra headcount. → `land-new-companies.rq`.
+- **How a member acted vs. lobbying / their wealth.** seimas votes × VTEK lobbying / VMI
+  declarations, by `asmens_id`. → `lobbying-then-vote.rq`, and the story `app/stories/seimo-turtas-2024.ttl`.
+
+## Caveats that change the conclusion (state these in any finding)
+
+- **Name-based person links are marked.** Where a source has no shared code with `persons/`
+  (VMI/PINREG declarants, VRK candidates, lobbying contacts, officials), the link carries
+  `ltlod:matchMethod` → `name-and-office` (name + an office held then — strong) or `exact-name`
+  (name only — weaker; audited precision in CLAUDE.md). **Filter to `name-and-office`** for a solid
+  person claim; treat `exact-name` as a lead. **Code joins (JAR, `asmens_id`, AR, aligned QID) are
+  exact by construction** and carry no method.
+- **Natural persons are counts, never named.** Land (and NMA farm subsidies) publish natural-person
+  figures only as per-cell aggregates, with **cells under 3 suppressed**. Don't infer individuals.
+- **Links dangle until the full JAR is loaded.** A `legal-entities/{code}/#this` link is correct
+  even when the name doesn't resolve — the full register (`entities.trig`) must be loaded
+  (`make -C etl/legal-entities entities`) for names and `rov:*` details. "No name" ≠ "no match".
+- **Snapshots and windows.** Tax **arrears** are the *current* snapshot (not the state at a past
+  contract date); **procurement** is the last 12 months ≥ 10 000 EUR; so "owes tax now" / "has no
+  contract" are as-of the data date, not history.
+- **Known gaps — don't over-claim.** EU-fund **amounts** exist for only ~3 % of records (the
+  ~685 coded projects; the rest are code-less instrument payouts). EU-fund **repayments and
+  irregularities**, and **NMA farm-subsidy** applicant identities, are keyed by salted/unpublished
+  **hashes** that don't join to any public code — they are *not in the data*. The **2021-2027** EU
+  programme is not published. Public-sector invoice flows at the top by value are **state financial
+  transfers**, not purchases — filter to small-average-invoice suppliers for real procurement-like spend.

.claude/skills/write-ltlod-sparql/SKILL.md

@@ -0,0 +1,109 @@
+---
+name: write-ltlod-sparql
+description: Write, debug or speed up a SPARQL query against the LTLOD datasets — the named-graph-per-document data shape, how to run a query (the throttled live endpoint vs. a local TDB2 store), and the query-shape rules that decide whether a query returns the right rows or hangs. Use whenever a query over the Lithuanian Linked Open Data returns empty or wrong results, is slow, needs a cross-document join, or is being authored for an app view (app/*.ttl) or an example (queries/*.rq).
+---
+
+# Write LTLOD SPARQL
+
+How to write a **correct, fast** query against the LTLOD datasets, independent of what
+question you are asking. For *which* domains and keys answer a given investigative
+question, use the `investigate-ltlod` skill — it builds on this one.
+
+## The data shape (read this first)
+
+**Entity per named graph.** Every document is one named graph whose URI is the document
+URI `{base}{container}/{slug}/`; the entity is `{graph}#this`, and secondary entities are
+`#fragment`s **in the same graph** (a person's memberships, their intervals and instants;
+a contract's value and role fragments; a proceeding's `#period`/`#start`/`#end`). So:
+
+- **One `GRAPH ?g { … }` block per document.** Put `#this` and all its `#fragment`s in
+  one block; join *across* documents by sharing a variable between blocks.
+- There is **no default graph** of instance data — everything is in named graphs. A query
+  with no `GRAPH` wrapper matches nothing.
+- `alignments.trig` / `photos.trig` / `*-geo.trig` reuse the entity graph names, so they
+  merge with the main data on load — query them as if they were one graph.
+
+The full URI map (containers, slugs, `#fragment` types, which source each comes from) is in
+**`etl/URI-SCHEME.md`**. 49 worked, explained queries are in **`queries/EXAMPLES.md`** — copy
+the closest one rather than starting blank.
+
+## Running a query
+
+### Local, over the TriG files — preferred for development and anything bulk
+`queries/run.sh` bulk-loads **all** `datasets/current/*/*.trig` into a throwaway TDB2 store
+and runs the query against it (TDB2 answers an unbound `GRAPH ?g` from its quad indexes;
+in-memory `arq` walks every one of ~190k named graphs and is unusable for GRAPH queries):
+
+```shell
+queries/run.sh queries/some-query.rq                 # one query, CSV/table to stdout
+queries/run.sh queries/some-query.rq --results=csv   # pass extra tdb2.tdbquery args
+BASE=https://linkeddata.lt/ queries/run.sh q.rq       # override base (default: prod)
+```
+
+It rebuilds the store **every call** (~20 s for the committed data, minutes with the bulk
+datasets). **For many queries, build the store once and reuse it:**
+
+```shell
+J=/Users/martynas/WebRoot/apache-jena-5.6.0        # 5.6.0 or 6.x both query fine
+"$J/bin/riot" --base=https://linkeddata.lt/ --syntax=trig --output=nquads \
+    datasets/current/*/*.trig > /tmp/all.nq
+"$J/bin/tdb2.tdbloader" --loc /tmp/tdb /tmp/all.nq
+"$J/bin/tdb2.tdbquery" --loc /tmp/tdb --query q.rq --results=csv   # repeat, fast
+```
+
+Load only the domains a query touches to save time (e.g. `datasets/current/procurement/*.trig
+datasets/current/insolvency/*.trig datasets/current/legal-entities/*.trig` for a
+procurement×insolvency join — plus the taxonomies if the query resolves a `skos:prefLabel`).
+
+**Datasets must be present.** Committed `.trig` are in the repo; the bulk ones (the full JAR
+`legal-entities/entities.trig`, `procurement`, `seimas/votes`, `eu-funds`, `public-spending`,
+`taxes/taxpayers`, …) are **gitignored** and built with `make -C etl <domain>` (see CLAUDE.md
+Commands). A query that joins a missing bulk domain returns empty, not an error.
+
+**Always pass `--base`.** Committed TriG is base-**relative** with no `@base`; without a base,
+relative IRIs resolve to `file://…` garbage and joins silently fail.
+
+### Live endpoint — `https://linkeddata.lt/sparql` — throttled, for a few interactive queries only
+```shell
+curl -sG https://linkeddata.lt/sparql --data-urlencode "query=$(cat q.rq)" -H "Accept: text/csv"
+```
+It is **rate-limited by nginx** (default `2r/s`, burst 10; 2 connections per client, 8 total)
+and answers **`429`** beyond that, keyed on the client address. Fine for a handful of queries;
+**do not** loop or bulk-query it — use `run.sh` / a local TDB2 for that. (On the local stack,
+RFC-1918/loopback clients are exempt, so you won't see 429 there.)
+
+## Query-shape rules (these decide empty vs. right, fast vs. hung)
+
+- **One `GRAPH ?g` block per document; join across blocks on shared variables.** A block is
+  evaluated on its own, so a `BIND`/`FILTER EXISTS` inside it **cannot see** a variable bound
+  outside it (it silently matches everything) — put such a test *after* the block, with the
+  `GRAPH` inside the `EXISTS`.
+- **A property path that crosses documents can't sit in one block** — split it into hops
+  (e.g. settlement → county is a UNION of the 2- and 3-hop `dct:isPartOf+` chains).
+- **No alternative path `(a|b)` with an unbound subject inside a `GRAPH` block** — ARQ won't
+  resolve it through the quad indexes. Use `?p` + `FILTER(?p IN (…))`, or a `UNION` of the two
+  plain triples.
+- **Never `FROM <urn:x-arq:UnionGraph>`** (Fuseki/ARQ-specific) — use standard `GRAPH` patterns.
+- **Read a value from a triple, never by string-parsing an IRI.** No `REPLACE(STR(?x), …)` /
+  `STRAFTER`/`SUBSTR(STR(?x), …)` to pull a code or label out of a resource URI: a concept's
+  code is `skos:notation`, its label `skos:prefLabel` (filter `lang`), a legal entity's code
+  `dct:identifier`. String ops on genuine *literals* are fine. Matching a *known* resource
+  base-agnostically with `FILTER(STRENDS(STR(?x), "/…/#this"))` is OK (the TriG is base-relative,
+  so a bare absolute IRI wouldn't match) — that selects a resource, it doesn't extract a value.
+- **Avoid per-row `FILTER NOT EXISTS` over the whole store** — it is quadratic and hangs.
+  Group/aggregate first so the check runs over a few thousand rows, or dump both URI sets with
+  two SELECTs and `comm -23`.
+- **Dates link to the calendar.** Events carry `dct:temporal` → `calendar/{yyyy}/{mm}/{dd}/#this`
+  (or the year doc) plus a literal; periods are a `time:Interval` with `time:Instant`s that each
+  carry `dct:temporal`. Join on the day/year doc for "what happened on/in …".
+- **Labels / language.** Lithuanian labels are `@lt`; filter `FILTER(LANG(?l) = "lt")`. A legal
+  entity's name is `skos:prefLabel`/`rov:legalName`; a person's is `foaf:name`.
+
+## When a query returns empty — checklist
+1. Is every pattern inside a `GRAPH ?g { … }`? (No default graph.)
+2. Is the joined **bulk** domain actually loaded? (`entities.trig`, `procurement`, the new
+   domains, and the **taxonomies** a `skos:prefLabel` join needs — e.g. `insolvency-types`.)
+3. Did you pass `--base` and the same base everywhere?
+4. Does a cross-document property path need splitting into hops?
+5. Is a value being read by IRI string-parsing (returns nothing if the slug scheme differs)
+   instead of from `skos:notation`/`dct:identifier`?