Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 42 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,22 @@
# ORAYLIS DataM8 CLI

DataM8's Python backend opens `.dm8s` solutions, imports and refreshes source metadata,
validates models, generates output, and serves the desktop/web editor over HTTP.
DataM8's Python CLI maintains local metadata models and generates target artifacts
from templates. Use it to inspect and edit `.dm8s` solutions, validate changes,
import source metadata or automate generation. The same package provides the
HTTP backend used by the visual editor.

## Start with your task

| I want to… | Begin here | First result |
| --- | --- | --- |
| Use the CLI on a local solution | [Install the CLI](#install-the-cli), then [first steps](#first-steps) | Initialize or validate a solution without cloning the backend sources |
| Script model edits or agent workflows | [Model-maintenance contract](docs/cli-agent-contract.md) | Discover schemas, inspect entities and apply explicit local changes with JSON results |
| Validate and generate target files | [CLI usage](docs/cli.md) | Run a named target and inspect its new output; generation does not deploy it |
| Develop a source connector | [Offline connector example](docs/plugin-example.md) and [plugin reference](docs/connectors.md) | Import static metadata into a disposable solution without a cloud account |
| Change the CLI or HTTP backend | [Local development](#local-development) and [backend contract](docs/backend-contract.md) | Run the matching checkout and a focused regression test |

For visual modeling, use the
[DataM8 app and handbook](https://github.com/oraylis/datam8/blob/main/docs/installation.md).

> [!IMPORTANT]
> The main branch may contain active development, which could contain a broken solution.
Expand Down Expand Up @@ -42,6 +57,31 @@ datam8 --help

See the [uv tools guide](https://docs.astral.sh/uv/guides/tools/) for details.

### First steps

For an existing solution:

```sh
datam8 validate --solution "path/to/MySolution.dm8s"
datam8 generate --help
```

With a version that supports blank solution initialization (`datam8 init --help` lists
`--type empty`), create a blank solution in a new directory:

```sh
mkdir MySolution
cd MySolution
datam8 init MySolution --type empty
```

The directory must be empty. The command creates `MySolution.dm8s` and default
Base definitions. Run `datam8 validate --solution ./MySolution.dm8s` to check the
new local solution. Add your own targets or use the
[public sample](https://github.com/oraylis/datam8-sample-solution) before trying
generation; blank initialization does not supply platform templates.
Use `datam8 <command> --help` to see the options for each command.

### Alternative: pipx

If you already use [pipx](https://pipx.pypa.io/stable/installation/) and have Python 3.12 or newer
Expand Down Expand Up @@ -73,27 +113,6 @@ uv tool install --force --python 3.12 "./datam8-<version>-py3-none-any.whl[api]"
Combine extras as `[api,azure,sql]` when needed. With pipx, use the same wheel path with extras
in `pipx install --force`.

### First steps

For an existing solution:

```sh
datam8 validate --solution "path/to/MySolution.dm8s"
datam8 generate --help
```

With a version that supports blank solution initialization (`datam8 init --help` lists
`--type empty`), create a blank solution in a new directory:

```sh
mkdir MySolution
cd MySolution
datam8 init MySolution --type empty
```

The directory must be empty. The command creates `MySolution.dm8s` and the default base entities.
Use `datam8 <command> --help` to see the options for each command.

### Update or uninstall

To update an installation made with uv, download the wheel from the desired release and install it
Expand Down
2 changes: 1 addition & 1 deletion docs/builtin-sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ datam8 sources import Contracts "sales/customers.yaml#customers" modelEntities/S

## Databricks Unity Catalog

The builtin was adapted from `datam8-plugins/plugins/databricks_uc` version 1.1.0.
The builtin supplies Unity Catalog metadata and SQL preview as part of the backend.
Metadata uses the Unity Catalog REST API; preview uses the SQL Statement Execution
API. No workspace runtime extension is installed.

Expand Down
7 changes: 3 additions & 4 deletions docs/cli-agent-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,9 @@ The CLI operates directly on a local solution using the same model operations as
the [HTTP API](backend-contract.md). It does not require a running server. Each
command reads persisted state; editor drafts are not part of that state.

For a complete Windows PowerShell example, follow the
[connected sample workflow](https://github.com/oraylis/datam8/blob/main/docs/user-guide/cli-sample-workflow.md)
in the central handbook. It covers solution selection, plugin setup and a new
Address pipeline through all sample layers.
For a complete model and a first local validation/generation exercise, follow
[the public sample guide](https://github.com/oraylis/datam8-sample-solution/blob/main/docs/getting-started.md).
Use the commands below for model maintenance and automation on a disposable copy.

Use `--solution PATH` (`-s`) before or after the command. `DATAM8_SOLUTION_PATH`
and the current directory remain supported. Use `--json` on the selected command.
Expand Down
12 changes: 6 additions & 6 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,10 @@ Read the readiness JSON for the actual port. Desktop passes a nonempty token;
browser development requires compatible frontend URL/auth configuration. See
[server](server.md) and [the HTTP contract](backend-contract.md).

## Evidence and boundaries
## Verification boundaries

On 2026-10-01, CLI validate and docs generation ran successfully against a disposable
archive of sample main `68001a2dcaa0b2931ee341f3527b96266a6e46da` with backend
`85683fcaeaf13c4573bee3de9c4668b1ba3ab3b9`. This is CLI evidence, not packaged desktop
or live-source acceptance. See [central compatibility](https://github.com/oraylis/datam8/blob/main/docs/compatibility.md)
and [handoff](documentation-handoff.md).
Run validation and generation on a disposable solution and inspect newly written
output. These local checks do not establish installed-desktop, live-source or
deployment acceptance. See [test setup](../tests/README.md) for fixtures and
[release compatibility](https://github.com/oraylis/datam8/blob/main/docs/compatibility.md)
for supported evidence boundaries.
18 changes: 0 additions & 18 deletions docs/documentation-handoff.md

This file was deleted.

File renamed without changes.
3 changes: 1 addition & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,8 @@ releases; exact versions remain in release notes and recorded evidence.
- [Python connector development](connectors.md)
- [Runnable offline connector example](plugin-example.md)
- [ODCS and Unity Catalog builtin configuration](builtin-sources.md)
- [Frontend integration](neon-cli-mapping.md)
- [Frontend integration](frontend-cli-mapping.md)
- [Test fixtures and commands](../tests/README.md)
- [CI and package releases](ci.md)
- [Local SQL fixture](../tests/db/README.md)
- [Central user handbook](https://github.com/oraylis/datam8/blob/main/docs/user-guide/README.md)
- [Documentation handoff](documentation-handoff.md)
2 changes: 1 addition & 1 deletion src/datam8/plugins/builtins/databricks_uc.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""Builtin Unity Catalog metadata and SQL preview connector.

Adapted from ORAYLIS datam8-plugins/plugins/databricks_uc (1.1.0).
Adapted from the ORAYLIS Unity Catalog connector (1.1.0).
"""

from __future__ import annotations
Expand Down
Loading