Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,9 @@
- The HTML collection page links to the collection schema and shows the spatial and temporal extent when available (#185).
- HTML templates: Bootstrap 5.0.0-beta3 -> 5.3.8, Leaflet 1.7.1 -> 1.9.4, proj4 2.7.5 -> 2.22.0 and swagger-ui 3.45.1 -> 5.29.1 (still loaded from CDNs, with updated SRI hashes). The Leaflet and proj4 imports now live in `mapLibs.ftl`, shared by `featureCollection.ftl` and `feature.ftl`; a custom template directory that overrides these two templates keeps its own versions (#199).
- New config key `api.collections.links.*` adds links to the top-level `links` array of the `/collections` response, in both JSON and HTML. Parallel to the existing `api.links.*` (landing page) and `default.collections.links.*` / `collections.<id>.links.*` (per-collection entries).
- New [configuration reference](docs/overview/configuration.md): every configuration key of the core modules and the reference webapp, with its default, on one page. Pull requests that change a key are expected to update it.
- `servers` is now trimmed like the other list keys: `servers=dev, prod` works instead of failing startup with a `NullPointerException`.
- PostGIS source: the keys `collections.<id>.id.alias`, `geometry.alias` and `properties.<p>.alias` are removed. They were a not-working and unfinished idea (`properties.<p>.alias` made startup fail with a `NullPointerException`); they are now ignored.

## 2.0.0

Expand Down
5 changes: 5 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,11 @@ ask you to make changes before merging it.
of the content, that you have the necessary rights to the content and that the
content you contribute may be provided under the project license.

> ### Configuration reference
> A Pull Request that adds, renames or removes a configuration key, or changes its
default, is expected to update [docs/overview/configuration.md](docs/overview/configuration.md)
in the same Pull Request.

## Community Modules

Hakunapi has a second tier of modules for contributions that are useful but not part of
Expand Down
4 changes: 1 addition & 3 deletions docker/cfg/addresses.properties
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,6 @@ collections=addresses,addresses_view
collections.addresses.casei=lower

## Should properties with null values be omitted from the response
collections.addresses.writeNulls=true

## PostgreSQL/PostGIS configuration
### This collection is backed by 'pg' (PostgreSQL/PostGIS) source/driver
Expand Down Expand Up @@ -205,7 +204,6 @@ collections.addresses_view.title=Simple Addresses (EPSG:3903 view)
collections.addresses_view.description=Addresses of Finlands Buildings using a database view transformed to EPSG:3903. Demonstrates database-level view usage and storage in a latLon-ordered projected CRS (north,east axis order).

collections.addresses_view.casei=lower
collections.addresses_view.writeNulls=true

collections.addresses_view.type=pg
collections.addresses_view.db=db
Expand All @@ -218,7 +216,7 @@ collections.addresses_view.srid=3903,10774,3067,4258

collections.addresses_view.id.mapping=id
collections.addresses_view.geometry.mapping=geom
collections.addresses_view.geometry.storageSrid=3903
collections.addresses_view.srid.storage=3903

collections.addresses_view.properties=inspireId_localId,inspireId_namespace,component_ThoroughfareName,component_PostalDescriptor,component_AdminUnitName_1,component_AdminUnitName_4,locator_designator_addressNumber,locator_level,locator_level_href,position_specification,position_specification_href,position_method,position_method_href,position_default,building,parcel

Expand Down
1 change: 0 additions & 1 deletion docker/cfg/addresses_gpkg.properties
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,6 @@ collections=simple_addresses
collections.simple_addresses.casei=lower

## Should properties with null values be omitted from the response
collections.simple_addresses.writeNulls=true

## GeoPackage configuration
### This collection is backed by 'gpkg' (GeoPackage) source/driver
Expand Down
374 changes: 374 additions & 0 deletions docs/overview/configuration.md

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,10 @@ To use one, add it as an explicit dependency of your own webapp. See

See [CONTRIBUTING.md](CONTRIBUTING.md)

## Configuration

All configuration keys and their defaults: [docs/overview/configuration.md](docs/overview/configuration.md)

## Example

Check out [Finnish addresses](examples/finnish_addresses)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ public List<Server> readServers() {
String[] serverNames = serverList.split(",");
List<Server> servers = new ArrayList<>();
for (String serverName : serverNames) {
String prefix = "servers." + serverName;
String prefix = "servers." + serverName.trim();
String url = cfg.getProperty(prefix + ".url");
while (url.endsWith("/")) {
url = url.substring(0, url.length() - 1);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,20 @@ public void testGetSRIDs() {
assertArrayEquals(expecteds, actuals);
}

@Test
public void testServersAreTrimmed() {
Properties cfg = new Properties();
cfg.setProperty("servers", "dev, prod");
cfg.setProperty("servers.dev.url", "https://dev.example.org/");
cfg.setProperty("servers.prod.url", "https://example.org");

List<io.swagger.v3.oas.models.servers.Server> servers = new HakunaConfigParser(cfg).readServers();

assertEquals(2, servers.size());
assertEquals("https://dev.example.org", servers.get(0).getUrl());
assertEquals("https://example.org", servers.get(1).getUrl());
}

@Test
public void testExternalizedProperties() {
String property = "hakunapi.test";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -198,30 +198,27 @@ public SimpleFeatureType parse(HakunaConfigParser cfg, Path path, String collect
ft.setDbSchema(dbSchema);
ft.setJoins(joins);

Map<String, Set<ColumnAlias>> columnsByTable = new HashMap<>();
Map<String, Set<String>> columnsByTable = new HashMap<>();
String[] idTables = cfg.getMultiple(p + "id.table");
String[] idMappings = cfg.getMultiple(p + "id.mapping");
String[] idAliases = cfg.getMultiple(p + "id.alias");
if (idMappings.length == 0) {
throw new IllegalArgumentException("Missing required property " + (p + "id.mapping"));
}
for (int i = 0; i < idMappings.length; i++) {
String table = idTables.length > i ? idTables[i] : primaryTable;
String mapping = idMappings[i];
String alias = idAliases.length > i ? idAliases[i] : null;
if (!HakunaConfigParser.isStaticMapping(mapping)) {
columnsByTable.computeIfAbsent(table, __ -> new HashSet<>()).add(new ColumnAlias(mapping, alias));
columnsByTable.computeIfAbsent(table, __ -> new HashSet<>()).add(mapping);
}
}

String geometryTable = cfg.get(p + "geometry.table", primaryTable);
String geometryMapping = cfg.get(p + "geometry.mapping");
String geometryAlias = cfg.get(p + "geometry.alias");
if (geometryMapping != null) {
geometryMapping = geometryMapping.replace("\"", "");
}
if (geometryMapping != null) {
columnsByTable.computeIfAbsent(geometryTable, __ -> new HashSet<>()).add(new ColumnAlias(geometryMapping, geometryAlias));
columnsByTable.computeIfAbsent(geometryTable, __ -> new HashSet<>()).add(geometryMapping);
}

String[] properties = cfg.getMultiple(p + "properties");
Expand All @@ -244,19 +241,17 @@ public SimpleFeatureType parse(HakunaConfigParser cfg, Path path, String collect
for (String property : properties) {
String table = cfg.get(p + "properties." + property + ".table", primaryTable);
String mapping = cfg.get(p + "properties." + property + ".mapping", property);
String alias = cfg.get(p + "properties." + property + ".alias", null);
mapping = mapping.replace("\"", "");
ColumnAlias ca = new ColumnAlias(mapping, alias);
if (!HakunaConfigParser.isStaticMapping(mapping)) {
columnsByTable.computeIfAbsent(table, __ -> new HashSet<>()).add(ca);
columnsByTable.computeIfAbsent(table, __ -> new HashSet<>()).add(mapping);
}
}

Map<String, Map<String, List<HakunaPropertyType>>> propertyTypes = new HashMap<>();
Map<String, Map<String, Boolean>> propertyNullability = new HashMap<>();
for (Map.Entry<String, Set<ColumnAlias>> entry : columnsByTable.entrySet()) {
for (Map.Entry<String, Set<String>> entry : columnsByTable.entrySet()) {
String table = entry.getKey();
Set<ColumnAlias> columns = entry.getValue();
Set<String> columns = entry.getValue();
String select = getSelectSchema(dbSchema, table, columns);
Map<String, List<HakunaPropertyType>> tablePropertyTypes = getPropertyTypes(ds, select);
propertyTypes.put(table, tablePropertyTypes);
Expand All @@ -269,9 +264,8 @@ public SimpleFeatureType parse(HakunaConfigParser cfg, Path path, String collect
// Simple
String table = idTables.length > 0 ? idTables[0] : primaryTable;
String mapping = idMappings[0];
String alias = idAliases.length > 0 ? idAliases[0] : mapping;
HakunaPropertyType idColumnType = propertyTypes.get(table).get(alias).get(0);
boolean nullable = propertyNullability.get(table).get(alias);
HakunaPropertyType idColumnType = propertyTypes.get(table).get(mapping).get(0);
boolean nullable = propertyNullability.get(table).get(mapping);
HakunaPropertyWriter propWriter = HakunaPropertyWriters.getIdPropertyWriter(ft, ft.getName(), "id", idColumnType);
switch (idColumnType) {
case INT:
Expand All @@ -295,12 +289,11 @@ public SimpleFeatureType parse(HakunaConfigParser cfg, Path path, String collect
for (int i = 0; i < idMappings.length; i++) {
String table = idTables.length > i ? idTables[i] : primaryTable;
String mapping = idMappings[i];
String alias = idAliases.length > 0 ? idAliases[0] : mapping;
boolean nullable = propertyNullability.get(table).get(alias);
boolean nullable = propertyNullability.get(table).get(mapping);
if (HakunaConfigParser.isStaticMapping(mapping)) {
parts.add(HakunaPropertyStatic.create("id", "table", mapping.substring(1, mapping.length() - 1)));
} else {
List<HakunaPropertyType> typeChain = propertyTypes.get(table).get(alias);
List<HakunaPropertyType> typeChain = propertyTypes.get(table).get(mapping);
boolean unique = false;
boolean hidden = true;
parts.add(getDynamicProperty(cfg, HakunaConfigParser.REF_ID_PROP, table, mapping, typeChain, nullable, unique, hidden));
Expand Down Expand Up @@ -335,7 +328,6 @@ public SimpleFeatureType parse(HakunaConfigParser cfg, Path path, String collect
boolean unique = Boolean.parseBoolean(cfg.get(propertyPrefix + "unique", "false"));
boolean hidden = Boolean.parseBoolean(cfg.get(propertyPrefix + "hidden", "false"));
String table = cfg.get(p + "properties." + property + ".table", primaryTable);
String alias = cfg.get(p + "properties." + property + ".alias", mapping);

if (HakunaConfigParser.isStaticMapping(mapping)) {
String value = mapping.substring(1, mapping.length() - 1);
Expand All @@ -344,7 +336,7 @@ public SimpleFeatureType parse(HakunaConfigParser cfg, Path path, String collect
}

Map<String, List<HakunaPropertyType>> tableTypes = propertyTypes.get(table);
List<HakunaPropertyType> type = tableTypes.get(alias);
List<HakunaPropertyType> type = tableTypes.get(mapping);
if (type == null) {
throw new IllegalArgumentException(
"Property " + property + " with mapping " + mapping + " can't find type from table " + table);
Expand Down Expand Up @@ -637,59 +629,6 @@ private HakunaProperty getDynamicProperty(HakunaConfigParser cfg, String name, S
return cfg.getDynamicProperty(name, table, column, typeChain, nullable, unique, hidden);
}


private final class ColumnAlias {

private final String column;
private final String alias;
private final boolean escape;

public ColumnAlias(String column, String alias) {
this(column, alias, true);
}

public ColumnAlias(String column, String alias, boolean escape) {
this.column = column;
this.alias = alias;
this.escape = escape;
}

@Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result + ((alias == null) ? 0 : alias.hashCode());
result = prime * result + ((column == null) ? 0 : column.hashCode());
result = prime * result + (escape ? 1231 : 1237);
return result;
}

@Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
ColumnAlias other = (ColumnAlias) obj;
if (alias == null) {
if (other.alias != null)
return false;
} else if (!alias.equals(other.alias))
return false;
if (column == null) {
if (other.column != null)
return false;
} else if (!column.equals(other.column))
return false;
if (escape != other.escape)
return false;
return true;
}

}

private Join parseJoin(HakunaConfigParser cfg, String join) {
String propertyPrefix = "joins." + join + ".";
String type = cfg.get(propertyPrefix + "type");
Expand All @@ -715,20 +654,11 @@ private Join parseJoin(HakunaConfigParser cfg, String join) {
}
}

private String getSelectSchema(String schema, String table, Set<ColumnAlias> columns) {
private String getSelectSchema(String schema, String table, Set<String> columns) {
StringBuilder sb = new StringBuilder();
sb.append("SELECT ");
for (ColumnAlias col : columns) {
if (col.escape) {
sb.append('"').append(col.column).append('"');
} else {
sb.append(col.column);
}
if (col.alias != null) {
sb.append(" AS ");
sb.append('"').append(col.alias).append('"');
}
sb.append(',');
for (String column : columns) {
sb.append('"').append(column).append('"').append(',');
}
sb.setLength(sb.length() - 1);
sb.append(" FROM ");
Expand Down
Loading