Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 37 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ srxsync check --inventory inv.yaml [--verbose] [--transport {pyez,rustez}]

| Flag | Meaning |
|---|---|
| `--replace` / `--merge` | Mutually exclusive. `--replace` annotates each category root with `replace="replace"` so Junos wipes slave-specific siblings within synced subtrees. `--merge` only adds/updates. |
| `--replace` / `--merge` | Mutually exclusive. `--replace` annotates each category root with `replace="replace"` so Junos wipes slave-specific siblings within synced subtrees. `--merge` only adds/updates. Security zones are always merged, never replaced. |
| `--commit-confirmed N` | Commit with an N-minute auto-rollback timer; srxsync calls `commit` again to seal. Default: 5. |
| `--max-parallel N` | Concurrent target workers. Default: 5. |
| `--on-error continue\|abort` | On a per-target failure, continue (exit non-zero at end) or cancel remaining work. Default: continue. |
Expand Down Expand Up @@ -133,6 +133,42 @@ ntp:

Then add `ntp` to the `include:` list of whichever targets should receive it.

## Zone translation

A target can use different zone names than the master. Add a `zone_map`
(master zone → slave zone) to the target; unlisted zones keep their name.

```yaml
targets:
- host: srx-site-a.example.net
auth: { provider: env }
include: [objects, policies, nat, zones]
zone_map:
APP1: APPS
APP2: APPS
APP3: APPS
```

- Zone names are rewritten in policies, global policies, NAT rule-sets and
address-book `attach`.
- Policy contexts that collapse into one (`APP1→UNTRUST`, `APP2→UNTRUST` →
`APPS→UNTRUST`) are concatenated in master order. A duplicate policy name in
a merged context fails that target. `APP1→APP2` becomes `APPS→APPS`.
- **Zones are never deleted.** `security zones` is always merged, even with
`--replace`. A zone fed by several master zones (`APPS`) is pushed as an
empty zone: created if missing, otherwise left exactly as the slave has it.
- srxsync warns (stderr, never fails) when it creates a zone, or a zone exists,
with no interfaces — bind one on the slave.
- Translation is single-pass: `{A: B, B: C}` maps A to B and B to C (no chaining).
- A warning is also printed when a translated policy references a zone that is
missing on the slave and `zones` is not in `include` (the slave commit will fail).
- A `zone_map` key that isn't a zone on the master prints a warning (likely a
typo) and is ignored.
- `check` only reports zones srxsync manages: zones that exist only on the slave
are not drift, and N:1 zones are checked for existence only.
- Entries in a master zone's *own* address-book are not carried to a merged
zone; keep shared objects in a global address-book.

## Secrets

Credentials never live in the inventory file. `auth.provider` selects a
Expand Down
164 changes: 164 additions & 0 deletions docs/superpowers/specs/2026-10-16-zone-translation-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
# Zone translation — Design Spec

**Date:** 2026-10-16
**Status:** Approved, ready for implementation planning
**Issue:** #11 (add zone translation)
**Branches:** `master` (rustperformance was merged into master in #7, so no separate port)

## Purpose

Master and slave SRXs do not always share a zone layout. The master may have
`APP1`, `APP2`, `APP3` and `UNTRUST` while a slave has only `APPS` and
`UNTRUST`. Each target gets a configurable master→slave zone mapping; srxsync
rewrites every zone reference in the synced config before pushing it.

srxsync runs after every change on the master, so the translation must be
idempotent and must never delete anything on the slave that the operator
created there.

## Inventory

New optional per-target key `zone_map`, master zone → slave zone:

```yaml
targets:
- host: srx-site-a.example.net
auth: { provider: env }
include: [objects, policies, nat, zones]
zone_map:
APP1: APPS
APP2: APPS
APP3: APPS
```

- Zones not listed keep their name (identity). `UNTRUST` above stays `UNTRUST`.
- `zone_map` must be a mapping of non-empty strings to non-empty strings;
anything else is an `InventoryError` at load time.
- Translation is single-pass, never chained: with `{A: B, B: C}`, master `A`
becomes `B` and master `B` becomes `C`.
- A key that does not exist as a zone on the master is logged as a warning, not
an error.
- `Target` gains `zone_map: dict[str, str]` (empty when absent). No `zone_map`
means behaviour is byte-for-byte unchanged.

### Terms

For a target, the **preimage** of a slave zone is the set of master zones that
translate to it (mapped keys plus an identically named, unmapped master zone).

- **1:1 zone** — preimage has exactly one master zone (identity or rename).
- **N:1 zone** — preimage has two or more master zones (e.g. `APPS`).

## Translation — `srxsync/zonemap.py`

`ZoneTranslator(zone_map).apply(config) -> etree._Element` returns a translated
deep copy of the fetched master config; the input is never mutated. The
orchestrator calls it once per target, before `DiffBuilder`. Errors raise
`ZoneTranslationError`, which fails only that target.

Zone references rewritten:

| Category | Element | Rule |
|---|---|---|
| `policies` | `policy` context `from-zone-name` / `to-zone-name` | Rename, then merge contexts (below) |
| `policies` | `global/policy/match/from-zone`, `to-zone` | Rename each entry, de-duplicate, keep first-seen order |
| `nat` | `source`/`destination`/`static` `rule-set/from/zone`, `rule-set/to/zone` | Rename, de-duplicate |
| `objects` | `address-book/attach/zone` | Rename, de-duplicate |
| `zones` | `security-zone/name` | 1:1 renamed and kept; N:1 handled below |

### Merging policy contexts

After renaming, two or more contexts can share a `(from, to)` pair — e.g.
`APP1→UNTRUST`, `APP2→UNTRUST`, `APP3→UNTRUST` all become `APPS→UNTRUST`.

- Contexts are merged into one, its policies concatenated in master document
order: all of the first context's policies (internal order kept), then the
second's, and so on. Rule order on the slave therefore matches master order.
- A duplicate policy name within a merged context raises
`ZoneTranslationError` naming the policy and both source contexts. No
auto-renaming.
- Contexts between zones that merge (`APP1→APP2`) become intra-zone
(`APPS→APPS`) and are kept, merged by the same rule.

## Zones are additive only

srxsync never deletes a zone on a slave.

- The `zones` category is always loaded as merge. `DiffBuilder._mark_replace`
never sets `replace="replace"` on `security/zones`, even under `--replace`.
Other categories keep their `--replace` behaviour. This also stops
`--replace` from wiping zone interface bindings, a side effect of the
interface prune that exists today.
- **1:1 zones** — the translated zone (interfaces pruned as today) is merged
onto the slave. A zone the slave lacks is created.
- **N:1 zones** — only an empty shell, `<security-zone><name>APPS</name></security-zone>`,
is pushed. The master zones' settings (host-inbound-traffic, screen,
zone address-book) are not carried over. In merge mode the shell creates the
zone if missing and is a no-op if present, so the slave's own settings for
`APPS` are never touched.

Verified 2026-10-16 with `commit check` on lab `vsrx-ci`: zones with no
interfaces, plus policies and source-NAT rule-sets referencing them, commit
cleanly. An empty zone does not block a push; it only carries no traffic.

## Empty-zone warning

Before loading, the push fetches the target's `/configuration/security/zones`
(one extra RPC). For every zone the translated payload references on the slave:

- missing on the slave → `WARNING <host>: creating zone APPS with no interfaces — bind an interface on the slave (set security zones security-zone APPS interfaces <ifl>)`
- present with no `interfaces` → same message, "zone APPS exists with no interfaces".

Warnings never fail a target or change the exit code. They are emitted for
`push`, `push --dry-run` and `check` (check already fetches target zones when
`zones` is included; otherwise it fetches them for this purpose).

## Results and CLI

- `TargetResult` and `DriftLine` gain `warnings: list[str]` (default empty).
- The CLI prints each warning on its own line after the summary table, prefixed
with `WARNING`, to stderr.
- `ZoneTranslationError` surfaces as a normal per-target failure (`FAIL` row,
`--on-error` honoured).

## Drift detection

`check` translates the source with the same `ZoneTranslator` before comparing.

- Policies, NAT and address-book compare the translated source against the target.
- 1:1 zones compare content (interfaces pruned, as today).
- N:1 zones are dropped from the content comparison on both sides; drift is
reported only when the zone is missing on the slave.

## Known limitation

Address entries defined in a master zone's own address-book (`security-zone
APP1 address-book`) are not carried to an N:1 slave zone. A pushed policy that
references one fails the slave's commit, and commit-confirmed rolls it back.
The fix is to put shared objects in a global address-book.

## Testing

Unit tests with XML fixtures (no devices):

- `zone_map` inventory validation: absent, valid, wrong types, empty strings
- identity when `zone_map` is empty — payload unchanged
- 1:1 rename across policies, global policies, NAT, address-book attach, zones
- N:1 policy contexts concatenated in master order
- duplicate policy name in a merged context → `ZoneTranslationError`
- `APP1→APP2` becomes `APPS→APPS`
- de-duplication of global-policy, NAT and attach zone lists
- N:1 zone pushed as an empty shell; master zone settings absent
- `security/zones` never carries `replace` under `--replace`; other roots do
- warning generated for missing zone and for an existing zone with no interfaces
- drift: translated source in sync with a correctly translated target; N:1 zone
existence-only

Integration (lab, optional): push a mapped inventory to a lab vSRX and confirm
the commit and the warning.

## Out of scope

- Merging or templating N:1 zone settings
- Translating names other than zones (routing instances, interfaces)
- Deleting slave zones that no longer have a master counterpart
9 changes: 9 additions & 0 deletions srxsync/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import argparse
import asyncio
import sys
from collections.abc import Iterable
from pathlib import Path

from srxsync.categories import CategoryModel
Expand Down Expand Up @@ -99,6 +100,7 @@ def _print_push_summary(summary: PushSummary) -> None:
status = "OK" if r.ok else "FAIL"
err = r.error or ""
print(f"{r.host:40s} {status:10s} {r.duration_s:>8.2f}s {err}")
_print_warnings(warning for result in summary.results for warning in result.warnings)


def _print_drift_summary(summary: DriftSummary, *, verbose: bool) -> None:
Expand All @@ -116,6 +118,13 @@ def _print_drift_summary(summary: DriftSummary, *, verbose: bool) -> None:
if verbose:
for p in line.differing_paths:
print(f" - {p}")
_print_warnings(warning for line in summary.reports for warning in line.warnings)


def _print_warnings(warnings: Iterable[str]) -> None:
"""Print each warning to stderr prefixed with ``WARNING ``."""
for warning in warnings:
print(f"WARNING {warning}", file=sys.stderr)


if __name__ == "__main__":
Expand Down
7 changes: 7 additions & 0 deletions srxsync/diff.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

from lxml import etree

from srxsync.zonemap import ZONES_PATH


@dataclass(frozen=True)
class DiffBuilder:
Expand Down Expand Up @@ -38,8 +40,13 @@ def _mark_replace(self, out: etree._Element) -> None:
the element at that path in the payload and set replace="replace".
Junos then replaces that subtree wholesale on the target device — the
semantic users expect from --replace.

security/zones is always merged: srxsync never deletes a slave zone, and
replacing it would also wipe the (pruned) interface bindings.
"""
for abs_path in self.paths:
if abs_path == ZONES_PATH:
continue
rel = abs_path.removeprefix("/configuration/")
for node in out.xpath(rel):
node.set("replace", "replace")
Expand Down
38 changes: 37 additions & 1 deletion srxsync/drift.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from lxml import etree

from srxsync.diff import DiffBuilder
from srxsync.zonemap import ZONES_PATH


@dataclass(frozen=True)
Expand All @@ -23,11 +24,17 @@ def in_sync(self) -> bool:
class DriftDetector:
paths: list[str]
prune: list[str]
merged_zones: frozenset[str] = frozenset()

def diff(self, source: etree._Element, target: etree._Element, host: str = "") -> DriftReport:
builder = DiffBuilder(paths=self.paths, prune=self.prune)
src_scoped = builder.build(source)
tgt_scoped = builder.build(target)
missing_merged = self._missing_merged_zones(tgt_scoped)
if ZONES_PATH in self.paths:
self._drop_slave_only_zones(src_scoped, tgt_scoped)
self._drop_merged_zones(src_scoped)
self._drop_merged_zones(tgt_scoped)

differing: list[str] = []
for abs_path in self.paths:
Expand All @@ -40,10 +47,39 @@ def diff(self, source: etree._Element, target: etree._Element, host: str = "") -
# not be sorted: a reordered-but-identical-set list is drift.
src_nodes = [_canonicalize(n) for n in src_scoped.findall(rel)]
tgt_nodes = [_canonicalize(n) for n in tgt_scoped.findall(rel)]
if src_nodes != tgt_nodes:
zone_missing = abs_path == ZONES_PATH and bool(missing_merged)
if zone_missing or src_nodes != tgt_nodes:
differing.append(abs_path)
return DriftReport(host=host, differing_paths=differing)

def _missing_merged_zones(self, target_scoped: etree._Element) -> set[str]:
"""N:1 zones are slave-owned: only their existence on the target is checked."""
present = {
name.text for name in target_scoped.xpath("security/zones/security-zone/name")
}
return set(self.merged_zones) - present

def _drop_slave_only_zones(
self, source_scoped: etree._Element, target_scoped: etree._Element
) -> None:
"""Ignore zones only the target has: push never deletes zones, so they are unfixable."""
source_names = {
name.text for name in source_scoped.xpath("security/zones/security-zone/name")
}
for zone in target_scoped.xpath("security/zones/security-zone"):
if zone.findtext("name") not in source_names:
zone.getparent().remove(zone)
if source_scoped.xpath("security/zones"):
return
for zones in target_scoped.xpath("security/zones[not(*)]"):
zones.getparent().remove(zones)

def _drop_merged_zones(self, scoped: etree._Element) -> None:
"""Remove merged zones' security-zone elements so their content is not compared."""
for zone in scoped.xpath("security/zones/security-zone"):
if zone.findtext("name") in self.merged_zones:
zone.getparent().remove(zone)


def _canonicalize(node: etree._Element) -> bytes:
result = etree.tostring(node, method="c14n2")
Expand Down
25 changes: 24 additions & 1 deletion srxsync/inventory.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

from __future__ import annotations

from dataclasses import dataclass
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any

Expand Down Expand Up @@ -31,6 +31,7 @@ class Target:
host: str
auth: Auth
include: list[str]
zone_map: dict[str, str] = field(default_factory=dict)


@dataclass(frozen=True)
Expand Down Expand Up @@ -90,4 +91,26 @@ def _parse_target(raw: dict[str, Any], *, known_categories: set[str]) -> Target:
host=raw["host"],
auth=_parse_auth(raw["auth"]),
include=include,
zone_map=_parse_zone_map(raw["host"], raw.get("zone_map")),
)


def _parse_zone_map(host: str, raw: Any) -> dict[str, str]:
"""Validate a target's master→slave zone_map; None or absent means no translation."""
if raw is None:
return {}
if not isinstance(raw, dict):
raise InventoryError(f"target {host} zone_map must be a mapping")
for master_zone, slave_zone in raw.items():
valid = (
isinstance(master_zone, str)
and isinstance(slave_zone, str)
and master_zone
and slave_zone
)
if not valid:
raise InventoryError(
f"target {host} zone_map entries must be non-empty strings: "
f"{master_zone!r}: {slave_zone!r}"
)
return dict(raw)
Loading
Loading