From 0daa9e7c88f83502d0a5b82bfd9abc9671369ede Mon Sep 17 00:00:00 2001 From: Fabio Kayser Date: Thu, 5 Nov 2026 08:41:58 +0100 Subject: [PATCH 1/2] docs: add task-oriented README entrypoints --- README.md | 65 ++++++++++++------- docs/builtin-sources.md | 2 +- docs/cli-agent-contract.md | 7 +- ...cli-mapping.md => frontend-cli-mapping.md} | 0 docs/index.md | 2 +- src/datam8/plugins/builtins/databricks_uc.py | 2 +- 6 files changed, 48 insertions(+), 30 deletions(-) rename docs/{neon-cli-mapping.md => frontend-cli-mapping.md} (100%) diff --git a/README.md b/README.md index e717b91..496c27a 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,22 @@ # ORAYLIS DataM8 CLI -DataM8's Python backend opens `.dm8s` solutions, imports and refreshes source metadata, -validates models, generates output, and serves the desktop/web editor over HTTP. +DataM8's Python CLI maintains local metadata models and generates target artifacts +from templates. Use it to inspect and edit `.dm8s` solutions, validate changes, +import source metadata or automate generation. The same package provides the +HTTP backend used by the visual editor. + +## Start with your task + +| I want to… | Begin here | First result | +| --- | --- | --- | +| Use the CLI on a local solution | [Install the CLI](#install-the-cli), then [first steps](#first-steps) | Initialize or validate a solution without cloning the backend sources | +| Script model edits or agent workflows | [Model-maintenance contract](docs/cli-agent-contract.md) | Discover schemas, inspect entities and apply explicit local changes with JSON results | +| Validate and generate target files | [CLI usage](docs/cli.md) | Run a named target and inspect its new output; generation does not deploy it | +| Develop a source connector | [Offline connector example](docs/plugin-example.md) and [plugin reference](docs/connectors.md) | Import static metadata into a disposable solution without a cloud account | +| Change the CLI or HTTP backend | [Local development](#local-development) and [backend contract](docs/backend-contract.md) | Run the matching checkout and a focused regression test | + +For visual modeling, use the +[DataM8 app and handbook](https://github.com/oraylis/datam8/blob/main/docs/installation.md). > [!IMPORTANT] > The main branch may contain active development, which could contain a broken solution. @@ -42,6 +57,31 @@ datam8 --help See the [uv tools guide](https://docs.astral.sh/uv/guides/tools/) for details. +### First steps + +For an existing solution: + +```sh +datam8 validate --solution "path/to/MySolution.dm8s" +datam8 generate --help +``` + +With a version that supports blank solution initialization (`datam8 init --help` lists +`--type empty`), create a blank solution in a new directory: + +```sh +mkdir MySolution +cd MySolution +datam8 init MySolution --type empty +``` + +The directory must be empty. The command creates `MySolution.dm8s` and default +Base definitions. Run `datam8 validate --solution ./MySolution.dm8s` to check the +new local solution. Add your own targets or use the +[public sample](https://github.com/oraylis/datam8-sample-solution) before trying +generation; blank initialization does not supply platform templates. +Use `datam8 --help` to see the options for each command. + ### Alternative: pipx If you already use [pipx](https://pipx.pypa.io/stable/installation/) and have Python 3.12 or newer @@ -73,27 +113,6 @@ uv tool install --force --python 3.12 "./datam8--py3-none-any.whl[api]" Combine extras as `[api,azure,sql]` when needed. With pipx, use the same wheel path with extras in `pipx install --force`. -### First steps - -For an existing solution: - -```sh -datam8 validate --solution "path/to/MySolution.dm8s" -datam8 generate --help -``` - -With a version that supports blank solution initialization (`datam8 init --help` lists -`--type empty`), create a blank solution in a new directory: - -```sh -mkdir MySolution -cd MySolution -datam8 init MySolution --type empty -``` - -The directory must be empty. The command creates `MySolution.dm8s` and the default base entities. -Use `datam8 --help` to see the options for each command. - ### Update or uninstall To update an installation made with uv, download the wheel from the desired release and install it diff --git a/docs/builtin-sources.md b/docs/builtin-sources.md index a99e324..079d038 100644 --- a/docs/builtin-sources.md +++ b/docs/builtin-sources.md @@ -143,7 +143,7 @@ datam8 sources import Contracts "sales/customers.yaml#customers" modelEntities/S ## Databricks Unity Catalog -The builtin was adapted from `datam8-plugins/plugins/databricks_uc` version 1.1.0. +The builtin supplies Unity Catalog metadata and SQL preview as part of the backend. Metadata uses the Unity Catalog REST API; preview uses the SQL Statement Execution API. No workspace runtime extension is installed. diff --git a/docs/cli-agent-contract.md b/docs/cli-agent-contract.md index 08f2be4..c2dcb28 100644 --- a/docs/cli-agent-contract.md +++ b/docs/cli-agent-contract.md @@ -4,10 +4,9 @@ The CLI operates directly on a local solution using the same model operations as the [HTTP API](backend-contract.md). It does not require a running server. Each command reads persisted state; editor drafts are not part of that state. -For a complete Windows PowerShell example, follow the -[connected sample workflow](https://github.com/oraylis/datam8/blob/main/docs/user-guide/cli-sample-workflow.md) -in the central handbook. It covers solution selection, plugin setup and a new -Address pipeline through all sample layers. +For a complete model and a first local validation/generation exercise, follow +[the public sample guide](https://github.com/oraylis/datam8-sample-solution/blob/main/docs/getting-started.md). +Use the commands below for model maintenance and automation on a disposable copy. Use `--solution PATH` (`-s`) before or after the command. `DATAM8_SOLUTION_PATH` and the current directory remain supported. Use `--json` on the selected command. diff --git a/docs/neon-cli-mapping.md b/docs/frontend-cli-mapping.md similarity index 100% rename from docs/neon-cli-mapping.md rename to docs/frontend-cli-mapping.md diff --git a/docs/index.md b/docs/index.md index f00b3b1..d872efe 100644 --- a/docs/index.md +++ b/docs/index.md @@ -12,7 +12,7 @@ releases; exact versions remain in release notes and recorded evidence. - [Python connector development](connectors.md) - [Runnable offline connector example](plugin-example.md) - [ODCS and Unity Catalog builtin configuration](builtin-sources.md) -- [Frontend integration](neon-cli-mapping.md) +- [Frontend integration](frontend-cli-mapping.md) - [Test fixtures and commands](../tests/README.md) - [CI and package releases](ci.md) - [Local SQL fixture](../tests/db/README.md) diff --git a/src/datam8/plugins/builtins/databricks_uc.py b/src/datam8/plugins/builtins/databricks_uc.py index 7b4e400..a0ae2de 100644 --- a/src/datam8/plugins/builtins/databricks_uc.py +++ b/src/datam8/plugins/builtins/databricks_uc.py @@ -1,6 +1,6 @@ """Builtin Unity Catalog metadata and SQL preview connector. -Adapted from ORAYLIS datam8-plugins/plugins/databricks_uc (1.1.0). +Adapted from the ORAYLIS Unity Catalog connector (1.1.0). """ from __future__ import annotations From 976aa5762d1592476b0b267216e679b1945a9627 Mon Sep 17 00:00:00 2001 From: Fabio Kayser Date: Thu, 5 Nov 2026 10:17:41 +0100 Subject: [PATCH 2/2] docs: replace obsolete task reports with current guidance --- docs/cli.md | 12 ++++++------ docs/documentation-handoff.md | 18 ------------------ docs/index.md | 1 - 3 files changed, 6 insertions(+), 25 deletions(-) delete mode 100644 docs/documentation-handoff.md diff --git a/docs/cli.md b/docs/cli.md index fd3e4bd..77a9b7d 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -50,10 +50,10 @@ Read the readiness JSON for the actual port. Desktop passes a nonempty token; browser development requires compatible frontend URL/auth configuration. See [server](server.md) and [the HTTP contract](backend-contract.md). -## Evidence and boundaries +## Verification boundaries -On 2026-10-01, CLI validate and docs generation ran successfully against a disposable -archive of sample main `68001a2dcaa0b2931ee341f3527b96266a6e46da` with backend -`85683fcaeaf13c4573bee3de9c4668b1ba3ab3b9`. This is CLI evidence, not packaged desktop -or live-source acceptance. See [central compatibility](https://github.com/oraylis/datam8/blob/main/docs/compatibility.md) -and [handoff](documentation-handoff.md). +Run validation and generation on a disposable solution and inspect newly written +output. These local checks do not establish installed-desktop, live-source or +deployment acceptance. See [test setup](../tests/README.md) for fixtures and +[release compatibility](https://github.com/oraylis/datam8/blob/main/docs/compatibility.md) +for supported evidence boundaries. diff --git a/docs/documentation-handoff.md b/docs/documentation-handoff.md deleted file mode 100644 index 28c87fb..0000000 --- a/docs/documentation-handoff.md +++ /dev/null @@ -1,18 +0,0 @@ -# Historical Backend Documentation Handoff - -This page preserves the original documentation handoff. References to local pages, -feature branches and pending publication below describe that task, not the current -repository status. Maintained handbook links now use `main`; tutorial evidence -keeps its recorded commits. Current installation starts at -[Install the CLI](../README.md#install-the-cli). - -This is a documentation-only package on the agreed feature branch. No product API, -schema JSON, dependency pin or target implementation is changed here. - -Run `node --test scripts/check-docs.test.mjs` and `node scripts/check-docs.mjs` -with Node 24. CI runs those checks without a frontend or sample checkout. - -Read the [central handoff](https://github.com/oraylis/datam8/blob/main/docs/handoff.md) for branch/workspace mapping, -results and manual merge order. The new central and component pages are local until -their branches are published. After manual merges, update development links to main; -keep tutorial evidence bound to commits. Public user workflows belong in the handbook. diff --git a/docs/index.md b/docs/index.md index d872efe..8cdf3f0 100644 --- a/docs/index.md +++ b/docs/index.md @@ -17,4 +17,3 @@ releases; exact versions remain in release notes and recorded evidence. - [CI and package releases](ci.md) - [Local SQL fixture](../tests/db/README.md) - [Central user handbook](https://github.com/oraylis/datam8/blob/main/docs/user-guide/README.md) -- [Documentation handoff](documentation-handoff.md)