feat(ui): security access control, zero-knowledge guidance, FAQ, and interactive user guide

CHANGELOG.md

@@ -10,6 +10,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 > See `BENCHMARK-AUDIT.md` and `test/BENCHMARKS.md` for the audited methodology.
 > No latency-histogram p99 is claimed.
 
+## [Unreleased] - Rekuiper Manager & Visual Stream Studio (Phases 1–4)
+
+### Added
+- **Zero-Copy Record Metadata (`RecordMeta`)**: Added an immutable, `Arc`-shared `RecordMeta` structure (`topic`, `source`, `custom`) to `StreamRecord`. Metadata passes through SQL projection pipelines and multi-port graph stages with zero map allocations and constant time ($O(1)$) overhead. Added zero-copy `meta()` SQL function with backward-compatible `__meta__` and JSON envelope fallback.
+- **Multi-Port DAG Execution Engine**: Added native Directed Acyclic Graph (DAG) rule routing with port-indexed selective edge dispatch (`EdgeTargets::Multi`). Nodes selectively route records across output ports `0..N` based on ordered predicates without broadcast flattening.
+- **Native Stream Control Operators**: Added `SwitchNode` (multi-port predicate routing), `RateLimitNode` (token-bucket drop and buffer rate limiter), and `WatchdogNode` (inactivity timeout dead-man alert generator).
+- **CommsHub and Wire-Tap Observability**: Added an asynchronous event bus (`CommsHub`) in `AppState`. Added rate-limited wire-tap probes (capped at 20 events per second with drop counters) toggleable via `POST /rules/:id/tap/:node_id/enable|disable`. Added real-time `/comms` WebSocket broadcast stream.
+- **Visual Stream Studio**: Embedded a visual flow canvas directly in `/ui`. Users can design stream pipelines with drag-and-drop source, operator, and sink nodes, inspect live node badges (events per second, queue depth, error count), and sample live tuple data in an interactive wire-tap drawer.
+- **Outbound Edge Agent and Yamux Tunneling**: Added `--control-plane-url` CLI option and `REKUIPER_CONTROL_PLANE_URL` environment configuration to `kuiperd`. The edge agent establishes an outbound mTLS WebSocket connection with exponential backoff. Yamux multiplexing handles control telemetry on stream 0 and tunnels remote HTTP and WebSocket requests into the local Axum router on streams 1..N.
+- **Transactional Deployment Watchdog and Auto-Rollback**: Added automatic health monitoring for 30 seconds following declarative configuration deployment. The watchdog automatically reverts the edge gateway to the previous configuration snapshot and raises an alert if error counts or panics exceed thresholds.
+- **Credential Masking and Secret Protection**: Added automatic credential redaction (`***REDACTED***`) for sensitive fields (passwords, bearer tokens, private keys, client secrets) across all configuration export and inspection APIs. Preserves original credentials bidirectionally during declarative reloads.
+- **Fleet Control Plane ("Rekuiper Manager")**: Added device fleet catalog (`/api/v1/fleet/devices`) and group management (`/api/v1/fleet/groups`). Added staged canary rollout engine (`/api/v1/fleet/rollouts`) supporting progressive rollout stages (5% -> 25% -> 100%) and automatic canary rollback. Added remote Yamux proxy (`/fleet/proxy/:device_id/*path`), unified audit logging (`/api/v1/fleet/audit`), and fleet control dashboard in `/ui`.
+- **Fleet Provisioning and Bearer Authentication**: Added token validation (`BasicConfig.fleet_token`, `REKUIPER_FLEET_TOKEN`) for `/fleet/ws` and `/api/v1/fleet/*` endpoints. Rejects unauthorized registrations, proxy requests, and rollout commands with HTTP 401 Unauthorized.
+- **Model Context Protocol (`rekuiper-mcp`) Expansion**: Expanded the native MCP server to 57 tools with dedicated categories for Fleet Management (`list_fleet_devices`, `get_fleet_rollouts`, `trigger_fleet_rollout`, `list_fleet_snapshots`, `proxy_fleet_device_api`) and Visual Stream Studio & Wire-Tap (`get_studio_palette`, `validate_graph_rule`, `toggle_wire_tap`, `get_wire_tap_status`). Added 3 new MCP live resources (`rekuiper://fleet/devices`, `rekuiper://fleet/rollouts`, `rekuiper://studio/palette`) and 2 interactive orchestration prompts (`staged_fleet_rollout`, `design_visual_dag_pipeline`).
+
 ## [1.0.0-rc4] - 2026-10-16
 
 ### Added

Cargo.toml

@@ -16,17 +16,18 @@ version = "1.0.0-rc4"
 edition = "2021"
 authors = ["I-Dacs Labs <measure@i-dacs.com>", "rekuiper contributors"]
 license = "MIT OR Apache-2.0"
+description = "Single-binary Rust MQTT rule engine and streaming SQL processor for edge gateways."
 repository = "https://github.com/Indus-Stack/rekuiper"
-homepage = "https://i-dacs.com"
-documentation = "https://github.com/Indus-Stack/rekuiper#documentation"
+homepage = "https://rekuiper.indramqtt.com"
+documentation = "https://rekuiper.indramqtt.com"
 readme = "README.md"
-keywords = ["iot", "stream-processing", "edge-computing", "sql", "mqtt"]
-categories = ["embedded", "network-programming", "science"]
+keywords = ["mqtt", "rule-engine", "sql", "stream-processing", "iot"]
+categories = ["network-programming", "database", "embedded", "asynchronous"]
 
 
 [workspace.dependencies]
 tokio = { version = "1.38", features = ["full"] }
-axum = { version = "0.7", features = ["macros"] }
+axum = { version = "0.7", features = ["macros", "ws"] }
 serde = { version = "1.0", features = ["derive"] }
 serde_json = { version = "1.0", features = ["preserve_order"] }
 indexmap = { version = "2.6", features = ["serde"] }
@@ -61,6 +62,7 @@ sha1 = "0.10"
 uuid = { version = "1", features = ["v4"] }
 rand = "0.8"
 boa_engine = "0.20"
+yamux = "0.13"
 
 # Internal crate dependencies
 rekuiper-core = { path = "crates/rekuiper-core" }

README.md

@@ -1,23 +1,23 @@
-# rekuiper
+# rekuiper — Rust MQTT Rule Engine & Streaming SQL Processor
 
 **Process device data where it is produced.**
 
-rekuiper is a small SQL engine that sits between your devices and your cloud. It filters, aggregates, and routes live data on the gateway itself, so you send upstream only what matters.
+rekuiper is a single-binary **MQTT rule engine** and **streaming SQL processor** written in **Rust**. It sits between your devices and your cloud—filtering, aggregating, and routing live telemetry over MQTT, Kafka, HTTP, Modbus, and OPC-UA on the gateway itself, so you send upstream only what matters.
 
 [![Release](https://img.shields.io/badge/release-v1.0.0--rc4-blue.svg)](https://github.com/Indus-Stack/rekuiper/releases)
 [![Rust CI](https://github.com/Indus-Stack/rekuiper/actions/workflows/ci.yml/badge.svg)](https://github.com/Indus-Stack/rekuiper/actions/workflows/ci.yml)
 [![License: MIT or Apache-2.0](https://img.shields.io/badge/license-MIT%20%2F%20Apache--2.0-yellow.svg)](LICENSE)
 [![Docker Pulls](https://img.shields.io/docker/pulls/indusstack/rekuiper?color=blue&logo=docker)](https://hub.docker.com/r/indusstack/rekuiper)
 
-[Documentation](https://rekuiper.indramqtt.com) · [Run the demo](#try-it-in-60-seconds) · [Is it a fit?](#is-it-a-fit) · [Coming from eKuiper?](#coming-from-ekuiper)
+[Documentation](https://rekuiper.indramqtt.com) · [Run the demo](#try-it-in-60-seconds) · [Benchmarks](#measured-on-one-cpu-core) · [Is it a fit?](#is-it-a-fit) · [Coming from eKuiper?](#coming-from-ekuiper)
 
 ---
 
-## Where it fits
+## Where the rule engine fits
 
 ![Where rekuiper fits: devices and feeds on the left, rekuiper on the gateway in the middle, destinations on the right](docs/en_US/public/diagrams/where-it-fits.svg)
 
-You write a rule in SQL. rekuiper runs it on every message as the message arrives. You add, change, and remove rules through a REST API while the engine runs.
+You write a rule in SQL. The Rust rule engine runs it on every MQTT message or sensor reading as it arrives. You add, change, and remove rules through a REST API or declarative GitOps files while the engine runs.
 
 What changes when you put it in the path:
 
@@ -61,7 +61,7 @@ curl -X POST http://localhost:9081/streams \
   -d '{"sql": "CREATE STREAM demo () WITH (TYPE=\"simulator\", CONF_KEY=\"demo\")"}'
 ```
 
-**4. Add a rule that turns every five seconds of readings into one row.**
+**4. Add a SQL rule that turns every five seconds of readings into one row.**
 
 ```bash
 curl -X POST http://localhost:9081/rules \
@@ -83,13 +83,13 @@ docker logs -f rekuiper
 [RULE five_second_summary] Matched record: {"readings": Number(49), "avg_temp": Number(23.03)}
 ```
 
-About fifty readings go in. One row comes out. To send that row to MQTT, Kafka, or a database, replace `log` with another action.
+About fifty readings go in. One row comes out. To route that row to an MQTT broker, Kafka topic, or time-series database, replace `log` with another sink action.
 
 Next: [Getting Started Guide](docs/en_US/getting_started/getting_started.md)
 
 ---
 
-## Common patterns
+## Common MQTT & edge rule patterns
 
 Each pattern is one or two SQL rules.
 
@@ -116,9 +116,9 @@ GROUP BY deviceId, TUMBLINGWINDOW(ss, 60)
 | Use rekuiper when | Look elsewhere when |
 | :--- | :--- |
 | Your data arrives via MQTT, HTTP, WebSocket, Kafka, AMQP, OTLP, Modbus TCP/RTU, or OPC-UA. | You need legacy proprietary fieldbus drivers (such as BACnet or Profibus). Put a protocol gateway in front and publish to MQTT. |
-| You want the logic in SQL, and you want to change it at runtime. | You need a visual flow builder for people who do not write SQL. |
+| You want the rule logic in SQL, and you want to change it at runtime. | You need a visual flow builder for people who do not write SQL. |
 | The machine has one or two CPU cores and limited memory. | You need a distributed cluster that scales across many nodes. rekuiper runs on one node. |
-| You need time windows, joins, and rules that keep state. | You only need to collect metrics and forward them without processing. |
+| You need time windows, stream joins, and rules that keep state in a native Rust binary. | You only need to collect metrics and forward them without processing. |
 
 ---
 
@@ -131,9 +131,9 @@ All tests run in a container limited to one CPU core and 1 GiB of RAM.
 | End-to-end streaming SQL throughput | 1,085,885 events per second (460 ms for 500k records) | 1.0.0-rc4 |
 | Zero-parse in-process SQL pipeline | 2,207,367 events per second (226 ms for 500k records) | 1.0.0-rc4 |
 | Parallel rules on one stream | 1,000 rules, 500,000 evaluations per second | 0.505 |
-| MQTT throughput, single rule | 150,000 to 200,000 messages per second | 0.500 |
+| MQTT rule throughput, single rule | 150,000 to 200,000 messages per second | 0.500 |
 | Heap memory under load | 4.4 to 10.2 MiB | 0.500 |
-| Footprint | One static binary, about 30 MB | 1.0.0-rc4 |
+| Footprint | One static Rust binary, about 30 MB | 1.0.0-rc4 |
 
 Full method, hardware limits, and comparisons with eKuiper, Telegraf, and Redpanda Connect: [Benchmark Overview](docs/en_US/benchmarks/overview.md).
 
@@ -155,18 +155,18 @@ Details: [Sources](docs/en_US/guide/sources/overview.md) · [Sinks](docs/en_US/g
 
 ---
 
-## What else is in the box
+## Architecture & Engine Capabilities
 
-- **Stream SQL.** Tumbling, hopping, sliding, count, and session windows. Stream-to-stream, stream-to-table, and temporal `ASOF JOIN` / `LEFT ASOF JOIN`. A rich library of **221 built-in functions** (including DFA CEP `sequence_match`/`sequence_count`, CompactDdSketch percentiles, Welford variance, and vector math). See the [SQL Reference](docs/en_US/sqls/overview.md).
-- **High-Throughput Zero-Clone Engine.** Zero-clone borrowed WHERE comparisons, register-level arithmetic evaluation (`try_eval_numeric_fast`), and reference-counted `SharedMap` CoW across rules.
+- **Stream SQL.** Tumbling, hopping, sliding, count, and session windows. Stream-to-stream, stream-to-table, and temporal `ASOF JOIN` / `LEFT ASOF JOIN`. A library of **221 built-in functions** (including DFA CEP `sequence_match`/`sequence_count`, CompactDdSketch percentiles, Welford variance, and vector math). See the [SQL Reference](docs/en_US/sqls/overview.md).
+- **High-Throughput Zero-Clone Engine.** Zero-clone borrowed WHERE comparisons, register-level arithmetic evaluation (`try_eval_numeric_fast`), and reference-counted `SharedMap` copy-on-write across rules.
 - **Rete-Lite StreamBus Routing.** Single-subscriber lock-free channel fast-paths and multi-rule indexable predicate routing.
 - **Parquet Columnar Buffering.** Automatic batch buffering with Arrow RecordBatch output and internal metadata isolation.
 - **Bounded memory.** Queues between sources, rules, and sinks have fixed sizes. A slow destination slows the source; it does not fill the memory.
 - **Declarative GitOps & Dead-Letter Queues (DLQ).** Run in read-only GitOps mode (`--declarative`) with automatic file watching, and isolate poison-pill records across 5 pipeline stages (`GET/DELETE /rules/:id/errors`).
-- **Extensions.** Write SQL functions as sandboxed WebAssembly modules.
+- **Extensions.** Write custom SQL functions as sandboxed WebAssembly modules.
 - **Secrets.** Read credentials from HashiCorp Vault or environment variables.
-- **Operations.** Prometheus metrics, a self-contained Web UI (`GET /ui`), an interactive terminal dashboard (`rekuiper top`), a Helm chart, and a `rekuiper` / `kuiper` CLI.
-- **AI tooling.** A Model Context Protocol (MCP) server (`rekuiper-mcp`) exposing **48 tools** (including `explain_pipeline_profile`), 15 live resources, and prompt workflows. See [MCP](docs/en_US/mcp/overview.md).
+- **Operations & Fleet Management.** Prometheus metrics, a self-contained Web UI (`GET /ui`), an interactive terminal dashboard (`rekuiper top`), a Helm chart, a `rekuiper` / `kuiper` CLI, a centralized Fleet Control Plane with staged canary rollouts, and the Visual Stream DAG Studio with real-time wire-tap telemetry.
+- **AI tooling.** A Model Context Protocol (MCP) server (`rekuiper-mcp`) exposing **57 tools** (including `explain_pipeline_profile`, fleet rollouts, and visual DAG inspection), 25 live resources, and 12 prompt workflows. See [MCP](docs/en_US/mcp/overview.md).
 
 ---
 

crates/kuiperd/Cargo.toml

@@ -7,6 +7,7 @@ description = "High-performance edge stream processing engine daemon (100% eKuip
 authors.workspace = true
 repository.workspace = true
 homepage.workspace = true
+documentation.workspace = true
 keywords.workspace = true
 categories.workspace = true