Language negotiation for service and collection metadata

CHANGELOG.md

@@ -2,6 +2,56 @@
 
 ## Unreleased
 
+### Internationalization / Localization
+
+- **Language negotiation for descriptive elements.** The landing page,
+  `/collections` and `/collections/{collectionId}` honour a `lang` query
+  parameter and the `Accept-Language` header, and report `Content-Language`.
+  `lang` wins over `Accept-Language`; an unsupported value falls back to the
+  first declared language. Titles and descriptions come from optional
+  per-language catalogs declared with `locale=`:
+
+  ```properties
+  locale=en,fi
+  locale.fi.path=messages_fi.properties
+  ```
+
+  A catalog is a sparse UTF-8 overlay on the main config, resolved per key, so
+  partial translations are fine and a declared language needs no catalog at all.
+  Only `api.title`, `api.description`, `collections.<id>.title` and
+  `collections.<id>.description` may be translated; any other key fails at
+  startup naming the key. Link titles are not localizable — declare one link per
+  language instead, each with its own `api.links.<name>.hreflang`, which is now
+  supported. See `docker/cfg/addresses.properties` for a worked example.
+
+  Multi-language resources emit `rel=alternate` links with `hreflang`, and
+  `lang` is propagated into generated links. `Link.hreflang` is now populated
+  (previously never set). `/collections` advertises the union of its
+  collections' languages.
+
+  Configuring one language or none is unchanged behaviour. Not yet localizable:
+  hakunapi's own English strings in JSON and HTML chrome, and the OpenAPI
+  document at `/api`.
+
+- Behaviour change to `/collections/{collectionId}/schema`: an unmatched `lang`
+  parameter previously fell through to the non-localized defaults. It now falls
+  back to the first configured language, as an unmatched `Accept-Language`
+  already did.
+
+- When `locale=` is configured, every `schema=` entry must resolve to a declared
+  language or startup fails naming the schema. The language is
+  `schema.<name>.lang`, defaulting to the schema's name, so `schema=en,fi`
+  alongside `locale=en,fi` needs no change while a schema named for anything
+  else must now declare its language:
+
+  ```properties
+  schema=my_schema
+  schema.my_schema.path=my_schema.json
+  schema.my_schema.lang=fi
+  ```
+
+  Services with no `locale=` are unaffected.
+
 ### Functional changes
 
 - An unknown `crs`, `bbox-crs` or `filter-crs` SRID returns 400 instead of 500, and a collection `srid` missing from the global `srid` fails startup with a message naming both.
@@ -23,6 +73,10 @@
 - New community module `hakunapi-tiles-source-gpkg`: a `MapTileSource` and a `VectorTileSource` serving the tiles stored in a GeoPackage tile pyramid, read through `sqlite-jdbc`. Not part of the reference webapp.
 - New community module `hakunapi-tiles-source-pmtiles`: a `MapTileSource` and a `VectorTileSource` serving the tiles of a PMTiles v3 archive, read in place from a local file or over HTTP range requests. Not part of the reference webapp.
 - New community module `hakunapi-tiles-generator`: the seed tool `TilesGenerator` writes a vector tile pyramid into a GeoPackage (`hakunapi-tiles-source-gpkg` can serve it), and `PMTilesConvert` converts such a pyramid into a PMTiles v3 archive. Not part of the reference webapp.
+- `collectionInfo.ftl` shows the collection id alongside its title and
+  description, and links to the collection's schema. `collections.ftl` lists
+  entries as `collection_id (Title)`.
+
 
 ## 2.0.0
 

docker/cfg/addresses.properties

@@ -63,6 +63,43 @@ schema=en,fi
 schema.en.path=addresses_schema_en.json
 schema.fi.path=addresses_schema_fi.json
 
+# Language negotiation for descriptive elements on the landing page,
+# /collections and /collections/{collectionId}. Selected with the lang query
+# parameter or the Accept-Language header; lang wins.
+#
+# The first declared language is the default and the fallback. A language
+# needs no catalog: without one it is served entirely from the keys in this
+# file, which is how the base representation's language is declared so that
+# Content-Language and hreflang can be stated rather than guessed. Here "en"
+# is that base language and only "fi" has a catalog.
+#
+# A catalog is a sparse overlay: it holds only the keys that differ, and
+# resolution is per key, so a partial translation is fine. Catalogs are read
+# as UTF-8. Only these keys may appear in one, and anything else fails at
+# startup naming the offending key:
+#
+#   api.title
+#   api.description
+#   collections.<id>.title
+#   collections.<id>.description
+#
+# Link titles are not among them. An additional link points at a resource this
+# service does not control, so a translated title would promise a language the
+# target may not have. Declare one link per language instead, each stating the
+# language of what it points at:
+#
+#   api.links=metadata-en,metadata-fi
+#   api.links.metadata-fi.href=https://example.org/metatiedot
+#   api.links.metadata-fi.hreflang=fi
+#   api.links.metadata-fi.title=Aineiston metatiedot
+#
+# locale= and schema= are independent: a collection can be localized through
+# its schema files without the landing page being localized, and each
+# resource advertises only the languages it can actually serve.
+locale=en,fi
+locale.fi.path=messages_fi.properties
+#locale.fi.lang=fi-FI
+
 # Collections and their configurations
 
 ## Required, comma-separated list of all collections

docker/cfg/messages_fi.properties

@@ -0,0 +1,15 @@
+# Finnish localization catalog for addresses.properties
+#
+# A sparse overlay: only keys that differ from the base configuration need to
+# be present, and resolution is per key. Read as UTF-8.
+#
+# For collection titles and descriptions a schema file entry
+# (addresses_schema_fi.json) is consulted when this catalog has no value, so a
+# key here is only needed to override the schema, or where no schema exists.
+
+api.title=INSPIRE-osoitteet, Suomi
+api.description=Esimerkkitoteutus OGC API -rajapinnoista hakunapilla. INSPIRE Simple Addresses -aineisto Suomesta.
+
+collections.addresses.title=Osoitteet
+collections.addresses.description=Osoitepisteet koko Suomesta
+collections.addresses_view.title=Osoitteet (näkymä)

src/hakunapi-core/src/main/java/fi/nls/hakunapi/core/FeatureServiceConfig.java

@@ -6,13 +6,16 @@
 import java.util.List;
 import java.util.Map;
 import java.util.Optional;
+import java.util.concurrent.ConcurrentHashMap;
 import java.util.function.Function;
 import java.util.stream.Collectors;
 
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
 import fi.nls.hakunapi.core.extension.ApiExtension;
+import fi.nls.hakunapi.core.i18n.LocalizedFeatureServiceConfig;
+import fi.nls.hakunapi.core.i18n.Localization;
 import fi.nls.hakunapi.core.schemas.ConformanceClasses;
 import fi.nls.hakunapi.core.schemas.FunctionsContent;
 import fi.nls.hakunapi.core.schemas.Link;
@@ -42,6 +45,9 @@ public abstract class FeatureServiceConfig {
     protected List<SRIDCode> knownSrids;
     protected ServiceTelemetry telemetry = ServiceTelemetry.NOP;
     protected List<Link> additionalLinks;
+    protected Localization localization;
+
+    private final Map<String, FeatureServiceConfig> localizedCache = new ConcurrentHashMap<>();
 
     public int getLimitDefault() {
         return limitDefault;
@@ -238,4 +244,36 @@ public void setAdditionalLinks(List<Link> additionalLinks) {
         this.additionalLinks = additionalLinks;
     }
 
+    public Localization getLocalization() {
+        return localization == null ? Localization.EMPTY : localization;
+    }
+
+    public void setLocalization(Localization localization) {
+        this.localization = localization;
+        // Wrappers close over the old Localization, so drop them
+        this.localizedCache.clear();
+    }
+
+    /**
+     * @return declared languages in declared order, first is the default; empty
+     *         if this service is not localized
+     */
+    public List<String> getLanguages() {
+        return getLocalization().getLanguages();
+    }
+
+    /**
+     * Returns a localized view of this service.
+     *
+     * @param lang language to localize in
+     * @return a localized view, or this if nothing here can serve lang
+     * @see LocalizedFeatureServiceConfig
+     */
+    public FeatureServiceConfig localized(String lang) {
+        if (lang == null || !getLanguages().contains(lang)) {
+            return this;
+        }
+        return localizedCache.computeIfAbsent(lang, l -> new LocalizedFeatureServiceConfig(this, l));
+    }
+
 }

src/hakunapi-core/src/main/java/fi/nls/hakunapi/core/FeatureType.java

@@ -11,6 +11,7 @@
 import fi.nls.hakunapi.core.property.HakunaProperty;
 import fi.nls.hakunapi.core.property.simple.HakunaPropertyGeometry;
 import fi.nls.hakunapi.core.schemas.Link;
+import io.swagger.v3.oas.models.media.Schema;
 
 public interface FeatureType {
 
@@ -62,4 +63,8 @@ public default List<Link> getAdditionalLinks() {
         return Collections.emptyList();
     }
 
+    public default Map<String, Schema<?>> getLangToSchema() {
+        return Collections.emptyMap();
+    }
+
 }

src/hakunapi-core/src/main/java/fi/nls/hakunapi/core/SimpleFeatureType.java

@@ -215,8 +215,9 @@ public void setAdditionalLinks(List<Link> additionalLinks) {
         this.additionalLinks = additionalLinks;
     }
 
+    @Override
     public Map<String, Schema<?>> getLangToSchema() {
-        return langToSchema;
+        return langToSchema == null ? Collections.emptyMap() : langToSchema;
     }
 
     public void setLangToSchema(Map<String, Schema<?>> langToSchema) {

src/hakunapi-core/src/main/java/fi/nls/hakunapi/core/config/HakunaConfigParser.java

@@ -1,6 +1,10 @@
 package fi.nls.hakunapi.core.config;
 
 import java.io.File;
+import java.io.IOException;
+import java.io.Reader;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
 import java.nio.file.Path;
 import java.time.Duration;
 import java.time.Instant;
@@ -39,6 +43,8 @@
 import fi.nls.hakunapi.core.SimpleFeatureType;
 import fi.nls.hakunapi.core.SimpleSource;
 import fi.nls.hakunapi.core.filter.Filter;
+import fi.nls.hakunapi.core.i18n.LocalizableKeys;
+import fi.nls.hakunapi.core.i18n.Localization;
 import fi.nls.hakunapi.core.param.GetFeatureParam;
 import fi.nls.hakunapi.core.projection.ProjectionTransformerFactory;
 import fi.nls.hakunapi.core.property.HakunaProperty;
@@ -159,7 +165,8 @@ private List<Link> readLinksWithPrefix(String keyPrefix) {
             String rel = getRequired(prefix + "rel");
             String type = getRequired(prefix + "type");
             String title = getRequired(prefix + "title");
-            links.add(new Link(href, rel, type, title));
+            String hreflang = get(prefix + "hreflang");
+            links.add(new Link(href, rel, type, title, hreflang));
         }
         return links;
     }
@@ -729,7 +736,7 @@ public HakunaProperty getProperty(FeatureType sft, String prop) {
      * schema.fi.path=path/to/file_fi.json
      * schema.sv.path=path/to/file_sv.json
      *
-     * Returns a map from normalized lang to root Schema.
+     * @return root schemas keyed by normalized lang
      */
     public Map<String, Schema<?>> readSchemas(Path configPath) {
         String[] schemaNames = getMultiple("schema");
@@ -757,6 +764,98 @@ public Map<String, Schema<?>> readSchemas(Path configPath) {
         return result;
     }
 
+    /**
+     * Reads localization catalogs from config:
+     * locale=fi,sv
+     * locale.fi.path=messages_fi.properties
+     * locale.sv.path=messages_sv.properties
+     * locale.sv.lang=sv-FI  (optional, defaults to the name)
+     *
+     * Catalogs are sparse overlays on the base config; a declared language with
+     * no catalog path is served entirely from it. The first declared language is
+     * the default, ordered by the locale= list.
+     *
+     * @return Localization with catalogs keyed by normalized lang, never null
+     */
+    public Localization readLocalization(Path configPath) {
+        String[] localeNames = getMultiple("locale");
+        if (localeNames.length == 0) {
+            return Localization.EMPTY;
+        }
+
+
+        List<String> languages = new ArrayList<>(localeNames.length);
+        Map<String, Map<String, String>> catalogs = new LinkedHashMap<>();
+        for (String name : localeNames) {
+            String lang = get("locale." + name + ".lang", name).toLowerCase(Locale.ROOT);
+            if (languages.contains(lang)) {
+                throw new IllegalArgumentException(
+                        "Duplicate language '" + lang + "' declared in locale=");
+            }
+            languages.add(lang);
+
+            String filePath = get("locale." + name + ".path");
+            if (filePath == null) {
+                // A language with no catalog is served from the base config
+                LOG.info("Locale '{}' has no catalog (locale.{}.path=...), serving it from the base config",
+                        name, name);
+                continue;
+            }
+
+            File file = new File(filePath);
+            Path path = file.isAbsolute() ? file.toPath() : configPath.getParent().resolve(filePath);
+            catalogs.put(lang, readCatalog(path));
+        }
+
+        checkSchemaLanguages(languages);
+
+        return new Localization(languages, catalogs, this::get);
+    }
+
+    /**
+     * A schema's language must also be declared in locale=, otherwise it is
+     * never negotiated and the schema file sits unread.
+     */
+    private void checkSchemaLanguages(List<String> languages) {
+        for (String name : getMultiple("schema")) {
+            String lang = get("schema." + name + ".lang", name).toLowerCase(Locale.ROOT);
+            if (!languages.contains(lang)) {
+                throw new IllegalArgumentException(String.format(
+                        "Schema '%s' resolves to language '%s', which is not declared in locale=%s. "
+                                + "Set schema.%s.lang to one of them.",
+                        name, lang, String.join(",", languages), name));
+            }
+        }
+    }
+
+    /**
+     * Read through an explicit UTF-8 Reader: Properties.load(InputStream) is
+     * Latin-1 and would mangle Finnish and Swedish.
+     */
+    private Map<String, String> readCatalog(Path path) {
+        Properties props = new Properties();
+        try (Reader reader = Files.newBufferedReader(path, StandardCharsets.UTF_8)) {
+            props.load(reader);
+        } catch (IOException e) {
+            throw new IllegalArgumentException("Could not read localization catalog " + path, e);
+        }
+
+        Map<String, String> catalog = new LinkedHashMap<>();
+        for (String key : props.stringPropertyNames()) {
+            if (!LocalizableKeys.isLocalizable(key)) {
+                throw new IllegalArgumentException(String.format(
+                        "Localization catalog %s contains key '%s', which is not localizable. "
+                                + "Localizable keys are: %s",
+                        path, key, LocalizableKeys.describe()));
+            }
+            String value = props.getProperty(key);
+            if (value != null && !value.isBlank()) {
+                catalog.put(key, value.trim());
+            }
+        }
+        return catalog;
+    }
+
     public FeatureType applyJsonSchema(FeatureType ft, Map<String, Schema<?>> schemas) {
         if (schemas == null || schemas.isEmpty() || !(ft instanceof SimpleFeatureType)) {
             return ft;