Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/wiki/Bundles.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ A bundle remove deletes a plugin only when the bundle was the last one in its `i
- **Not recorded.** The ledger at `<plugin dir>/.nself-state/bundles.json` records a bundle only when all its plugins are installed. For a bundle it does not record, `bundle remove` fails with E134 (exit 1): nothing it installed is removable as a unit. Remove the individual plugins with `nself plugin remove <plugin>`.
- **`--dry-run`** prints the plan and changes nothing; it never creates the ledger. `nself remove <plugin> --dry-run` prints the plugin, the data rule, the referencing bundles and whether E133 would refuse, and changes nothing.
- **A failure stops the run.** The ledger then keeps the bundle recorded and drops only the plugins already removed, so running the same command again continues from there.
- **Resume.** The remove is recorded in `.nself-state/remove-intent.json` in the plugin directory before the first plugin goes. If it is interrupted, run the same command again: it prints `resuming an interrupted remove` and removes only what is left. `nself bundle install <bundle>` clears that bundle's intent, so a later remove starts fresh.
- **Kept for a dependent.** A member that an installed plugin outside the plan lists in its `plugin.json` `dependencies` is kept and recorded `explicit` (`kept: px (needed by outsider; recorded explicit)`); dependents are matched by plugin directory name. If an outside plugin's `plugin.json` cannot be read, the remove refuses before removing anything, because that plugin might depend on a member: fix or remove it, then rerun. `nself plugin remove` refuses the same way unless `--force`.
- `--force` keeps its meaning (overwrite hand-edited generated files). `--keep-data` is a hidden, deprecated alias; from v1.5 it does nothing and prints a warning. It is removed in v1.6.0.

`nself remove <plugin>` (and `nself plugin remove`) refuses a plugin an installed bundle still references: **E133**, exit 1, naming the bundle. Remove the bundle, or pass `--force` to remove only that plugin; the ledger then drops the plugin's record.
Expand Down
8 changes: 5 additions & 3 deletions .github/wiki/Upgrade-From-v0.9.md
Original file line number Diff line number Diff line change
Expand Up @@ -302,7 +302,7 @@ If `nself migrate run` fails partway through:
nself migrate rollback --list

# Restore the last backup
nself migrate rollback
nself migrate rollback --yes
```

The rollback restores all files that were backed up before the failure.
Expand All @@ -329,12 +329,14 @@ If you need to revert to v0.9 after migrating:
nself migrate rollback --list

# Restore the most recent backup
nself migrate rollback
nself migrate rollback --yes

# Or restore a specific backup by timestamp
nself migrate rollback --backup 20260420-143022
nself migrate rollback --backup 20260420-143022 --yes
```

`--yes` confirms the restore without a prompt. From v1.5 a rollback without `--yes` asks on a terminal and is refused with E403 anywhere else (scripts, CI); in v1.4 it still runs but prints a deprecation warning (under `--json`, in `meta.deprecations`).

After rollback, reinstall the v0.9 CLI (use the Homebrew formula version pin or download from GitHub releases).

---
Expand Down
6 changes: 4 additions & 2 deletions .github/wiki/Upgrading-from-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,15 +108,17 @@ Then restore your data: `nself db restore /path/to/backup.sql`
If migration fails, roll back to v1 state:

```bash
nself migrate rollback
nself migrate rollback --yes
```

Or restore a specific backup:

```bash
nself migrate rollback --backup 20260328-143022
nself migrate rollback --backup 20260328-143022 --yes
```

`--yes` confirms the restore without a prompt. From v1.5 a rollback without `--yes` asks on a terminal and is refused with E403 anywhere else (scripts, CI); in v1.4 it still runs but prints a deprecation warning (under `--json`, in `meta.deprecations`).

---

See [[Guide-Migration-from-v1]] for a detailed walkthrough with troubleshooting steps.
4 changes: 3 additions & 1 deletion .github/wiki/cmd-bundle.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,9 @@ An interrupted remove is resumed by running the same command again (the
progress is kept in `.nself-state/remove-intent.json` in the plugin directory).
A remove interrupted after the ledger file was deleted cannot resume: the
leftover plugins are kept as explicit, and `nself plugin remove` removes them.
A member that a kept plugin depends on is kept and recorded as explicit.
A member that a plugin outside the plan depends on is kept and recorded as explicit.
If such a plugin's `plugin.json` cannot be read, the remove refuses before removing
anything. `nself bundle install` clears the bundle's old remove intent.

```
nself bundle remove <name> [--dry-run] [--keep-data]
Expand Down
2 changes: 1 addition & 1 deletion .github/wiki/cmd-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ With `--json` in v1.5 mode an applied change prints the envelope with `applied:

**Rotating a live secret.** Put the new value in `.env.secrets` and let the regeneration carry it: `nself config set HASURA_GRAPHQL_ADMIN_SECRET <new> --env secrets --yes` on a prod-class env. The generated env and compose files are updated; the running containers keep the old value until they are recreated (`nself start`). `nself secrets rotate` is a different tool: it rotates the encrypted secret store (`.secrets/<env>.age`), which the build does not read, so it does not regenerate anything. Its schedule tracks `HASURA_GRAPHQL_ADMIN_SECRET`, the name the stack reads.

The `validate` subcommand runs all registered configuration validators and reports pass/fail per rule. It is automatically run during `nself build`, but you can run it independently to check config without rebuilding.
The `validate` subcommand runs all registered configuration validators and reports pass/fail per rule. It is automatically run during `nself build`, but you can run it independently to check config without rebuilding. It also warns, on stderr, about each env-file value whose unquoted or double-quoted `$` the loader would expand (`.env.secrets:12: KEY holds an unquoted '$' ...`): the warning gives the file, line and key only, never the value; a `${NAME}` reference is not flagged; and warnings do not change the exit code or the `--json` document. Single-quote the value if the `$` is literal.

### JSON output

Expand Down
6 changes: 4 additions & 2 deletions .github/wiki/cmd-migrate.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,15 +93,17 @@ nself migrate detect
nself migrate run

# Restore from the most recent backup
nself migrate rollback
nself migrate rollback --yes

# List available backups
nself migrate rollback --list

# Restore from a specific backup
nself migrate rollback --backup 20260417-143022
nself migrate rollback --backup 20260417-143022 --yes
```

`--yes` confirms the restore without a prompt. From v1.5 a rollback without `--yes` asks on a terminal and is refused with E403 anywhere else (scripts, CI); in v1.4 it still runs but prints a deprecation warning (under `--json`, in `meta.deprecations`).

See [[Upgrade-From-v0.9]] for the full step-by-step migration guide.
<!-- END PROSE:examples -->

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/proxy-swap-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,10 @@ jobs:
if: ${{ github.event_name == 'pull_request' || inputs.leg == 'add' }}
run: bash scripts/e2e/proxy-swap.sh --add
- name: Round-trip leg
if: ${{ github.event_name == 'workflow_dispatch' && inputs.leg == 'round-trip' }}
if: ${{ github.event_name == 'pull_request' || (github.event_name == 'workflow_dispatch' && inputs.leg == 'round-trip') }}
run: bash scripts/e2e/proxy-swap.sh --round-trip --assert-identical
- name: Kill-mid-remove leg
if: ${{ github.event_name == 'workflow_dispatch' && inputs.leg == 'round-trip' }}
if: ${{ github.event_name == 'pull_request' || (github.event_name == 'workflow_dispatch' && inputs.leg == 'round-trip') }}
run: bash scripts/e2e/proxy-swap.sh --kill-mid-remove
- name: Renew leg
if: ${{ github.event_name == 'pull_request' || (github.event_name == 'workflow_dispatch' && inputs.leg == 'renew') }}
Expand Down
14 changes: 12 additions & 2 deletions cmd/commands/data_consent.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ package commands
//
// Behaviour (ADR 0021 compat switch):
// - --yes set: runs, silent.
// - v1.4 (default), no --yes: runs, with one stderr deprecation line.
// - v1.4 (default), no --yes: runs, with one stderr deprecation line (and,
// under --json, a meta.deprecations entry).
// - v1.5 (NSELF_V15=1), no --yes: asks on a terminal, otherwise refuses
// with E403 before any side effect.
// - a read-only or preview flag (--dry-run, --list) skips the gate entirely.
Expand All @@ -22,9 +23,13 @@ import (

"github.com/nself-org/cli/internal/compat"
"github.com/nself-org/cli/internal/errs"
"github.com/nself-org/cli/internal/output"
"github.com/spf13/cobra"
)

// consentRefusedFrom is the release from which a bare data-loss command is refused.
const consentRefusedFrom = "v1.5.0"

// consentFlagUsage is the help text of the --yes flag on the gated commands.
const consentFlagUsage = "Confirm the data loss without a prompt (required from v1.5 unless you answer a terminal prompt)"

Expand Down Expand Up @@ -90,8 +95,13 @@ func requireDataLossConsent(cmd *cobra.Command, what string) error {
return errs.New("E403", fmt.Sprintf("%s %s and was not confirmed", cmd.CommandPath(), what)).
WithFix("pass --yes to confirm, or run it on a terminal and answer y")
}
path := strings.TrimPrefix(cmd.CommandPath(), "nself ")
// A --json caller gets the notice in meta.deprecations like a canon move
// notice (D-0762), even when the stderr line is silenced.
if output.JSONRequestedFromArgs(os.Args) {
output.AddDeprecation("nself "+path+" without --yes", "nself "+path+" --yes", consentRefusedFrom)
}
if !warningsSilenced(os.Args) {
path := strings.TrimPrefix(cmd.CommandPath(), "nself ")
_, _ = fmt.Fprintf(cmd.ErrOrStderr(), "warning: nself %s without --yes is deprecated (%s); from v1.5 it needs --yes or a terminal confirmation. Pass --yes now.\n", path, what)
}
return nil
Expand Down
52 changes: 52 additions & 0 deletions cmd/commands/data_consent_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,13 @@ package commands
import (
"bytes"
"errors"
"io"
"os"
"strings"
"testing"

"github.com/nself-org/cli/internal/errs"
"github.com/nself-org/cli/internal/output"
"github.com/spf13/cobra"
"golang.org/x/term"
)
Expand Down Expand Up @@ -167,3 +169,53 @@ func TestDataLossGateSkipFlagsExist(t *testing.T) {
t.Fatal("rollback lost its --list flag")
}
}

// TestDataLossGateJSONDeprecation: a v1.4 --json caller sees the consent
// warning in meta.deprecations, even when the stderr line is silenced, and a
// run with --yes or without --json records nothing (D-0762).
func TestDataLossGateJSONDeprecation(t *testing.T) {
cases := []struct {
name string
argv []string
yes bool
want bool
}{
{"json bare", []string{"nself", "backup", "prune", "--json"}, false, true},
{"json bare silenced", []string{"nself", "backup", "prune", "--json", "--quiet"}, false, true},
{"json yes", []string{"nself", "backup", "prune", "--json", "--yes"}, true, false},
{"text bare", []string{"nself", "backup", "prune"}, false, false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Setenv("NSELF_V15", "0")
output.ResetState()
t.Cleanup(output.ResetState)
withArgs(t, tc.argv...)
var buf bytes.Buffer
c := gateCmd(&buf)
if tc.yes {
if err := c.Flags().Set("yes", "true"); err != nil {
t.Fatal(err)
}
}
if err := requireDataLossConsent(c, "deletes old backups"); err != nil {
t.Fatal(err)
}
var doc bytes.Buffer
if err := output.EmitData(output.Writer{Out: &doc, Err: io.Discard}, "backup prune", map[string]string{"ok": "1"}); err != nil {
t.Fatal(err)
}
got := strings.Contains(doc.String(), `"deprecations"`)
if got != tc.want {
t.Fatalf("meta.deprecations present = %v, want %v:\n%s", got, tc.want, doc.String())
}
if tc.want {
for _, w := range []string{`"old": "nself backup prune without --yes"`, `"new": "nself backup prune --yes"`, `"removal_at": "v1.5.0"`} {
if !strings.Contains(doc.String(), w) {
t.Errorf("meta lacks %s:\n%s", w, doc.String())
}
}
}
})
}
}
13 changes: 3 additions & 10 deletions cmd/commands/deprecation_wiring_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -246,16 +246,9 @@ func TestInstalledBinaryEmitsDeprecationWarning(t *testing.T) {
t.Skip("binary staging in this test is POSIX-only")
}

root := findRepoRootForTest(t)
installDir := t.TempDir()
binPath := filepath.Join(installDir, "nself")

build := exec.Command("go", "build", "-mod=vendor", "-o", binPath, "./cmd/nself/")
build.Dir = root
build.Env = append(os.Environ(), "CGO_ENABLED=0")
if out, err := build.CombinedOutput(); err != nil {
t.Fatalf("build failed: %v\n%s", err, out)
}
// pilotBinary is one shared build of ./cmd/nself in a directory of its own,
// with no source tree next to it (D-0735: no second compile and link).
binPath := pilotBinary(t)

// Run from a *different* empty directory so neither the binary's own
// directory nor the working directory contains a source tree.
Expand Down
26 changes: 9 additions & 17 deletions cmd/commands/mcp_integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,20 +32,14 @@ import (
"github.com/mark3labs/mcp-go/mcp"
)

// buildNSelfBinary compiles ./cmd/nself into dir and returns its path. Used
// so mcpExecSelf's re-exec has a real binary to invoke — os.Executable()
// inside `go test` resolves to the test harness, which doesn't understand
// "build"/"start" as subcommands, hence the NSELF_MCP_EXEC_OVERRIDE hook.
func buildNSelfBinary(t *testing.T, dir string) string {
// buildNSelfBinary returns the nself binary the suite builds once per test
// process (pilotBinary), so mcpExecSelf's re-exec has a real binary to invoke:
// os.Executable() inside `go test` resolves to the test harness, which doesn't
// understand "build"/"start" as subcommands, hence the NSELF_MCP_EXEC_OVERRIDE
// hook. Reusing it spares a second ./cmd/nself compile and link (D-0735).
func buildNSelfBinary(t *testing.T) string {
t.Helper()
bin := filepath.Join(dir, "nself")
cmd := exec.Command("go", "build", "-mod=vendor", "-o", bin, "../../cmd/nself")
cmd.Dir = mustGetwd(t)
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("building nself for integration test: %v\n%s", err, out)
}
return bin
return pilotBinary(t)
}

func mustGetwd(t *testing.T) string {
Expand All @@ -64,8 +58,7 @@ func TestMCPIntegration_InitBuildVerify(t *testing.T) {
t.Skip("skipping binary-build integration test in -short mode")
}

binDir := t.TempDir()
bin := buildNSelfBinary(t, binDir)
bin := buildNSelfBinary(t)

projectDir := t.TempDir()
t.Setenv("NSELF_ALLOW_SOURCE_DIR", "1")
Expand Down Expand Up @@ -138,8 +131,7 @@ func TestMCPIntegration_StartStatusStop(t *testing.T) {
t.Skip("skipping in -short mode")
}

binDir := t.TempDir()
bin := buildNSelfBinary(t, binDir)
bin := buildNSelfBinary(t)

projectDir := t.TempDir()
t.Setenv(mcpSelfExecOverrideEnv, bin)
Expand Down
15 changes: 15 additions & 0 deletions cmd/commands/pilot_json_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,18 @@ func runPilot(t *testing.T, proj string, v15 bool, extra []string, args ...strin
return pilotRun{code: code, stdout: o.String(), stderr: e.String()}
}

// skipLiveUnderV15Guard skips a subprocess ("live") subtest when it runs inside
// TestPilotJSONUnderV15Guard. runPilot rebuilds the child environment without
// any NSELF_ variable and sets NSELF_V15 itself, so the guard's t.Setenv changes
// nothing for a live run: the standalone test already covers it, and running it
// twice doubled the slowest tests of the package (D-0735).
func skipLiveUnderV15Guard(t *testing.T) {
t.Helper()
if strings.HasPrefix(t.Name(), "TestPilotJSONUnderV15Guard/") {
t.Skip("live runs rebuild the child environment; the standalone test covers them")
}
}

// ---- status ----------------------------------------------------------------

func statusReport(results ...health.HealthResult) *health.HealthReport {
Expand Down Expand Up @@ -281,6 +293,7 @@ func TestPilotJSONStatus(t *testing.T) {
})

t.Run("live", func(t *testing.T) {
skipLiveUnderV15Guard(t)
proj := pilotProject(t)
r := runPilot(t, proj, true, nil, "status", "--json")
data := validateEnvelope(t, r.stdout, "status")
Expand Down Expand Up @@ -408,6 +421,7 @@ func TestPilotJSONDoctor(t *testing.T) {
})

t.Run("live", func(t *testing.T) {
skipLiveUnderV15Guard(t)
proj := pilotProject(t)
for _, args := range [][]string{{"doctor", "--json"}, {"doctor", "--format", "json"}, {"doctor", "--json", "--deep"}} {
r := runPilot(t, proj, true, nil, args...)
Expand Down Expand Up @@ -561,6 +575,7 @@ func TestPilotJSONConfig(t *testing.T) {
})

t.Run("live", func(t *testing.T) {
skipLiveUnderV15Guard(t)
p := pilotProject(t)
// v1.5: schema-valid envelopes, secrets masked, --reveal honoured.
for _, tc := range []struct {
Expand Down
13 changes: 11 additions & 2 deletions internal/bundle/installer_appready.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ func finishInstall(ctx context.Context, rn *appready.Runner, store *ledger.Store
if len(members) == 0 {
return "no installed members to record", nil
}
if err := store.Update(func(l *ledger.Ledger) error {
if err := store.Locked(func(tx *ledger.Tx, l ledger.Ledger) error {
// A member that was on disk before this install and has no ledger
// record was installed on its own: record it explicit so a later
// bundle remove keeps it. (Errs toward keeping the plugin.)
Expand All @@ -95,7 +95,16 @@ func finishInstall(ctx context.Context, rn *appready.Runner, store *ledger.Store
}
}
l.AddBundle(result.Bundle, members, time.Now())
addCatalogHolders(l, result.Bundle, members, time.Now())
addCatalogHolders(&l, result.Bundle, members, time.Now())
if err := tx.Write(l); err != nil {
return err
}
// A remove that was interrupted before this install is superseded: the
// bundle is installed again, so a later remove must not "resume" it
// (D-0662). A failed clear is a warning, never a failed install.
if err := tx.ClearRemoveIntent(result.Bundle); err != nil {
_, _ = fmt.Fprintf(out, "warning: could not clear the old remove intent of bundle %s: %v\n", result.Bundle, err)
}
return nil
}); err != nil {
return "", fmt.Errorf("the plugins are installed (not rolled back) but recording them in %s failed: %w", store.Path(), err)
Expand Down
40 changes: 40 additions & 0 deletions internal/bundle/installer_intent_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
package bundle

// installer_intent_test.go — D-0662: installing a bundle supersedes an old
// remove intent of that bundle, so a later remove does not print "resuming".

import (
"bytes"
"testing"
"time"

"github.com/nself-org/cli/internal/bundle/ledger"
)

func TestBundleInstallClearsStaleRemoveIntent(t *testing.T) {
pluginDir := isolate(t)
setupOfflineRegistry(t, mockNotificationsOnly())
store := newLedgerStore(pluginDir)
for _, slug := range []string{"task", "other"} {
err := store.Locked(func(tx *ledger.Tx, _ ledger.Ledger) error {
return tx.SetRemoveIntent(slug, ledger.RemoveIntent{StartedAt: time.Now(), Remaining: []string{"notifications"}})
})
if err != nil {
t.Fatal(err)
}
}
var out bytes.Buffer
opts := freeOpts(t, pluginDir, &out, func(name string) error {
writeMember(t, pluginDir, name, 1, nil)
return nil
})
if _, err := Install(opCtx(t), "task", opts); err != nil {
t.Fatalf("install: %v\n%s", err, out.String())
}
if _, ok := intentOf(t, pluginDir, "task"); ok {
t.Error("the install left task's stale remove intent")
}
if _, ok := intentOf(t, pluginDir, "other"); !ok {
t.Error("the install cleared another bundle's remove intent")
}
}
Loading
Loading