Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,10 +199,12 @@ Download the self-contained fat JAR for your Elasticsearch version:

| Elasticsearch Version | Artifact |
|-----------------------|----------------------------------------|
| ES 6.x | `softclient4es6-jdbc-driver-0.2.0.jar` |
| ES 7.x | `softclient4es7-jdbc-driver-0.2.0.jar` |
| ES 8.x | `softclient4es8-jdbc-driver-0.2.0.jar` |
| ES 9.x | `softclient4es9-jdbc-driver-0.2.0.jar` |
| ES 6.x | `softclient4es6-jdbc-driver-0.2.1.jar` |
| ES 7.x | `softclient4es7-jdbc-driver-0.2.1.jar` |
| ES 8.x | `softclient4es8-jdbc-driver-0.2.1.jar` |
| ES 9.x | `softclient4es9-jdbc-driver-0.2.1.jar` |

> **Java 11+ recommended** (17+ for ES 9.x): **cross-index JOINs require Java 11+** — the embedded JOIN engine is built on Apache Arrow 18.x, which ships Java-11 bytecode.

```text
JDBC URL: jdbc:elastic://localhost:9200
Expand All @@ -217,20 +219,20 @@ Driver class: app.softnetwork.elastic.jdbc.ElasticDriver
<dependency>
<groupId>app.softnetwork.elastic</groupId>
<artifactId>softclient4es8-jdbc-driver</artifactId>
<version>0.2.0</version>
<version>0.2.1</version>
</dependency>
```

**Gradle:**

```groovy
implementation 'app.softnetwork.elastic:softclient4es8-jdbc-driver:0.2.0'
implementation 'app.softnetwork.elastic:softclient4es8-jdbc-driver:0.2.1'
```

**sbt:**

```scala
libraryDependencies += "app.softnetwork.elastic" % "softclient4es8-jdbc-driver" % "0.2.0"
libraryDependencies += "app.softnetwork.elastic" % "softclient4es8-jdbc-driver" % "0.2.1"
```

The JDBC driver JARs are Scala-version-independent (no `_2.12` or `_2.13` suffix) and include all required dependencies.
Expand Down Expand Up @@ -309,6 +311,8 @@ For programmatic access, add SoftClient4ES to your project.
| 8.x | `softclient4es8-java-client` | 2.12, 2.13 | 8+ |
| 9.x | `softclient4es9-java-client` | 2.13 only | 17+ |

> JDK versions above apply to the client libraries themselves. **Cross-index JOINs (via the arrow extensions, the JDBC/ADBC drivers) require Java 11+** — see [Cross-Index JOIN](documentation/sql/joins.md).

### sbt Setup

```scala
Expand All @@ -320,7 +324,7 @@ libraryDependencies += "app.softnetwork.elastic" %% "softclient4es8-java-client"
// Add the community extensions for materialized views (optional)
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es-community-extensions" % "0.2.0"
// Add the JDBC driver if you want to use it from Scala (optional)
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es-jdbc-driver" % "0.2.0"
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es-jdbc-driver" % "0.2.1"
```

```scala
Expand Down
14 changes: 7 additions & 7 deletions documentation/client/adbc_driver.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@ Download the self-contained fat JAR for your Elasticsearch version:

| Elasticsearch | Artifact |
|----------------|-----------------------------------------------------|
| ES 6.x | `softclient4es6-adbc-driver-0.2.0.jar` |
| ES 7.x | `softclient4es7-adbc-driver-0.2.0.jar` |
| ES 8.x | `softclient4es8-adbc-driver-0.2.0.jar` |
| ES 9.x | `softclient4es9-adbc-driver-0.2.0.jar` |
| ES 6.x | `softclient4es6-adbc-driver-0.2.1.jar` |
| ES 7.x | `softclient4es7-adbc-driver-0.2.1.jar` |
| ES 8.x | `softclient4es8-adbc-driver-0.2.1.jar` |
| ES 9.x | `softclient4es9-adbc-driver-0.2.1.jar` |

### Maven / Gradle / sbt

Expand All @@ -36,20 +36,20 @@ Download the self-contained fat JAR for your Elasticsearch version:
<dependency>
<groupId>app.softnetwork.elastic</groupId>
<artifactId>softclient4es8-adbc-driver</artifactId>
<version>0.2.0</version>
<version>0.2.1</version>
</dependency>
```

**Gradle:**

```groovy
implementation 'app.softnetwork.elastic:softclient4es8-adbc-driver:0.2.0'
implementation 'app.softnetwork.elastic:softclient4es8-adbc-driver:0.2.1'
```

**sbt:**

```scala
libraryDependencies += "app.softnetwork.elastic" % "softclient4es8-adbc-driver" % "0.2.0"
libraryDependencies += "app.softnetwork.elastic" % "softclient4es8-adbc-driver" % "0.2.1"
```

---
Expand Down
10 changes: 5 additions & 5 deletions documentation/client/arrow_flight_sql.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,15 +49,15 @@ Available images per ES version:
### Fat JAR

```bash
java -jar softclient4es8-arrow-flight-sql-0.2.0.jar
java -jar softclient4es8-arrow-flight-sql-0.2.1.jar
```

| Elasticsearch | Artifact |
|---------------|----------|
| ES 6.x | `softclient4es6-arrow-flight-sql-0.2.0.jar` |
| ES 7.x | `softclient4es7-arrow-flight-sql-0.2.0.jar` |
| ES 8.x | `softclient4es8-arrow-flight-sql-0.2.0.jar` |
| ES 9.x | `softclient4es9-arrow-flight-sql-0.2.0.jar` |
| ES 6.x | `softclient4es6-arrow-flight-sql-0.2.1.jar` |
| ES 7.x | `softclient4es7-arrow-flight-sql-0.2.1.jar` |
| ES 8.x | `softclient4es8-arrow-flight-sql-0.2.1.jar` |
| ES 9.x | `softclient4es9-arrow-flight-sql-0.2.1.jar` |

---

Expand Down
2 changes: 1 addition & 1 deletion documentation/client/download_analytics.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ beacon to a public endpoint with exactly these fields:
|---------------|----------|-----------------------------------------------|
| `source` | `portal` | Where the count came from (the docs button) |
| `driver` | `jdbc` | Which driver family (`jdbc` or `adbc`) |
| `version` | `0.2.0` | The published artifact version |
| `version` | `0.2.1` | The published artifact version |
| `count_delta` | `1` | One download |

A timestamp is added on the server. That is the **entire** record.
Expand Down
14 changes: 7 additions & 7 deletions documentation/client/jdbc.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ Download the self-contained fat JAR for your Elasticsearch version. The JARs are

| Elasticsearch | Artifact |
|---------------|----------|
| ES 6.x | `softclient4es6-jdbc-driver-0.2.0.jar` |
| ES 7.x | `softclient4es7-jdbc-driver-0.2.0.jar` |
| ES 8.x | `softclient4es8-jdbc-driver-0.2.0.jar` |
| ES 9.x | `softclient4es9-jdbc-driver-0.2.0.jar` |
| ES 6.x | `softclient4es6-jdbc-driver-0.2.1.jar` |
| ES 7.x | `softclient4es7-jdbc-driver-0.2.1.jar` |
| ES 8.x | `softclient4es8-jdbc-driver-0.2.1.jar` |
| ES 9.x | `softclient4es9-jdbc-driver-0.2.1.jar` |

### Build Tool Integration

Expand All @@ -33,20 +33,20 @@ Download the self-contained fat JAR for your Elasticsearch version. The JARs are
<dependency>
<groupId>app.softnetwork.elastic</groupId>
<artifactId>softclient4es8-jdbc-driver</artifactId>
<version>0.2.0</version>
<version>0.2.1</version>
</dependency>
```

**Gradle:**

```groovy
implementation 'app.softnetwork.elastic:softclient4es8-jdbc-driver:0.2.0'
implementation 'app.softnetwork.elastic:softclient4es8-jdbc-driver:0.2.1'
```

**sbt:**

```scala
libraryDependencies += "app.softnetwork.elastic" % "softclient4es8-jdbc-driver" % "0.2.0"
libraryDependencies += "app.softnetwork.elastic" % "softclient4es8-jdbc-driver" % "0.2.1"
```

---
Expand Down
66 changes: 58 additions & 8 deletions documentation/client/repl.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,15 @@ It provides:

| Elasticsearch Version | Minimum Java Version |
|-----------------------|----------------------|
| ES 6 | Java 8+ |
| ES 7 | Java 8+ |
| ES 8 | Java 8+ |
| ES 6 | Java 11+ |
| ES 7 | Java 11+ |
| ES 8 | Java 11+ |
| ES 9 | Java 17+ |

> The 0.20+ REPL requires **Java 11+** on every ES version: the CLI bundles logback
> 1.5.x and the JOIN engine is built on Apache Arrow 18.x — both ship Java-11
> bytecode. See [Extensions](#extensions-cross-index-joins-materialized-views).

### Network Requirements

- Network access to JFrog repository (`softnetwork.jfrog.io`)
Expand All @@ -60,27 +64,27 @@ It provides:
#### Linux / macOS

```bash
curl -fsSL https://raw.githubusercontent.com/SOFTNETWORK-APP/softclient4es/main/install.sh | bash
curl -fsSL https://raw.githubusercontent.com/SOFTNETWORK-APP/SoftClient4ES/main/install.sh | bash
```

Or download and run manually:

```bash
curl -O https://raw.githubusercontent.com/SOFTNETWORK-APP/softclient4es/main/install.sh
curl -O https://raw.githubusercontent.com/SOFTNETWORK-APP/SoftClient4ES/main/install.sh
chmod +x install.sh
./install.sh
```

#### Windows (PowerShell)

```powershell
irm https://raw.githubusercontent.com/SOFTNETWORK-APP/softclient4es/main/install.ps1 | iex
irm https://raw.githubusercontent.com/SOFTNETWORK-APP/SoftClient4ES/main/install.ps1 | iex
```

Or download and run manually:

```powershell
Invoke-WebRequest -Uri https://raw.githubusercontent.com/SOFTNETWORK-APP/softclient4es/main/install.ps1 -OutFile install.ps1
Invoke-WebRequest -Uri https://raw.githubusercontent.com/SOFTNETWORK-APP/SoftClient4ES/main/install.ps1 -OutFile install.ps1
.\install.ps1
```

Expand Down Expand Up @@ -193,7 +197,8 @@ softclient4es/
│ ├── application.conf # Application configuration
│ └── logback.xml # Logging configuration
├── lib/
│ └── softclient4es8-cli_2.13-x.y.z-assembly.jar
│ ├── softclient4es8-cli_2.13-x.y.z-assembly.jar
│ └── ... # extension jars + their dependencies (see Extensions)
├── logs/ # Log files directory
│ └── softclient4es.log # (created at runtime)
├── LICENSE
Expand All @@ -202,6 +207,51 @@ softclient4es/
└── uninstall.sh
```

### Extensions (cross-index JOINs, materialized views)

Cross-index `JOIN`s and materialized views are delivered as **extensions** — separate
jars discovered on the classpath through the `ExtensionSpi` ServiceLoader mechanism:

- `softclient4es-arrow-extensions` — the cross-index JOIN engine (**required for
`SELECT … JOIN` across indices** — the same engine the JDBC driver embeds)
- `softclient4es-community-extensions` — materialized views and quota enforcement

**The installer sets these up by default** — no manual step:

- **community extensions**: always installed — any engine version (matching 0.1.x line
for engines < 0.20)
- **arrow extensions**: installed for engines ≥ 0.20 (requires Java 11+, like the
0.20+ CLI itself — Apache Arrow 18.x ships Java-11 bytecode)

Each extension is resolved **with its full dependency closure** (Apache Arrow, DuckDB,
…; ~250 jars) via a bundled [coursier](https://get-coursier.io) resolver into `lib/`.
The launcher runs the REPL with `java -cp "<assembly>:lib/*"`, so every jar in `lib/`
is visible.

To skip them:

```bash
./install.sh --no-extensions # minimal install — JOINs and MVs unavailable
```

**Manual installation on an existing install** (or air-gapped preparation):

```bash
# Resolve each extension with all its dependencies into lib/
cs fetch --repository https://softnetwork.jfrog.io/artifactory/releases \
"app.softnetwork.elastic:softclient4es-arrow-extensions_2.13:<version>" \
"app.softnetwork.elastic:softclient4es-community-extensions_2.13:<version>" \
| xargs -I{} cp {} ~/softclient4es/lib/
```

> ⚠️ **Copying only the two extension jars into `lib/` is NOT enough** — they are thin
> jars whose engine (Arrow memory, DuckDB, Flight) arrives transitively. Always resolve
> the full dependency closure as above. Installations made with an installer older than
> the 0.20 line launched the REPL with `java -jar`, which ignores `lib/` entirely —
> re-run the installer to get the classpath-based launcher.

The launcher adds the required Arrow `--add-opens` flags automatically on Java 9+.

### Add to PATH

#### Linux / macOS
Expand Down
2 changes: 2 additions & 0 deletions documentation/sql/joins.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ Elasticsearch has no native cross-index JOIN — and a stock JDBC driver on top
1. **Query-time cross-index JOIN** — `INNER` / `LEFT` / `RIGHT` / `FULL OUTER` joins across indices (and across clusters), at query time, on **every** surface: the REPL, the JDBC driver, the ADBC driver, the Arrow Flight SQL sidecar, and Federation. The drivers are the **free delivery channel**; the JOIN *depth* is metered.
2. **Persisted [Materialized Views](materialized_views.md)** — the other superpower: denormalize cross-index data once into a queryable view. (That has its own page; this page is about query-time JOINs.)

> **Runtime requirement — Java 11+.** The JOIN engine is built on Apache Arrow 18.x, which ships Java-11 bytecode, and the 0.20+ REPL/driver line also bundles logback 1.5.x (Java-11 as well). Every JVM-side surface (the REPL, the JDBC/ADBC drivers) requires **Java 11 or newer** at runtime (17+ for ES 9); the Flight SQL and Federation Docker images bundle their own Java 17. In the REPL, the installer sets the extensions up for you (see the [REPL Extensions section](../client/repl.md#extensions-cross-index-joins-materialized-views)).

## The JOIN ladder

Three meters gate how far a JOIN can reach. Think of them as rungs:
Expand Down
2 changes: 1 addition & 1 deletion install.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Examples:
.\install.ps1
.\install.ps1 -ListVersions -EsVersion 8
.\install.ps1 -Target "C:\tools\softclient4es" -EsVersion 8 -Version 1.0.0
.\install.ps1 -EsVersion 7 -Version 0.2.0
.\install.ps1 -EsVersion 7 -Version 0.20.0

"@
exit 0
Expand Down
Loading
Loading