diff --git a/README-install.md b/README-install.md index 3319a65a..c3ee8195 100644 --- a/README-install.md +++ b/README-install.md @@ -1,22 +1,66 @@ # Install Debuglet -Debuglet releases support Linux amd64. For the full bundle, download the archive, `install.sh`, and `SHA256SUMS` from the [latest release](https://github.com/netsec-ethz/debuglet/releases/latest), then install to your user directory: +Use Linux amd64, a POSIX shell, GNU tar, and coreutils. Choose the operator's +exact **full bundle** for executor enrollment, managed services, or the local +demo. No source checkout or compiler is needed. The published unsigned v0.2.0 +package predates `dbl executor join`; the current enrollment flow requires an +operator-supplied candidate until a compatible signed release is published. + +## Install a private development candidate + +Obtain a trusted standalone `bootstrap.sh`, the exact version and source +revision, and the matching `debuglet-VERSION-linux-amd64.tar.gz`, `install.sh`, +and `SHA256SUMS` through your operator's trusted channel. Keep those three +package files together with their original names. Replace the version +placeholder below with the supplied `v0.0.0-dev.` and use your +actual local paths. Candidates require a local directory and explicit unsigned +opt-in; they are not public releases. Checksums are verified before the installer +runs, but do not authenticate an unsigned publisher. ```sh -archive=$(printf '%s\n' ./debuglet-v*-linux-amd64.tar.gz) -version=${archive#./debuglet-} -version=${version%-linux-amd64.tar.gz} -sh ./install.sh --archive "$archive" --checksums ./SHA256SUMS \ - --version "$version" --prefix "$HOME/.local" +DEBUGLET_VERSION=VERSION_PROVIDED_BY_OPERATOR \ +DEBUGLET_RELEASE_DIR="$HOME/Downloads/debuglet-candidate" \ +DEBUGLET_ALLOW_UNSIGNED=1 \ +DEBUGLET_PREFIX="$HOME/.local" \ +sh "$HOME/Downloads/bootstrap.sh" export PATH="$HOME/.local/bin:$PATH" +dbl --output json version dbl demo ``` -The installer verifies the release files before writing them. It requires a POSIX shell, GNU tar, and coreutils. The demo needs no Go compiler, source checkout, service, account, wallet, or root access. +Leave component and signature/trust settings unset for this unsigned full +bundle; unsigned mode cannot override configured signature requirements. +Confirm the reported version and source revision. The demo needs no service, +account, wallet, or root access. Installation does not enroll a node, create a +service account, or restart an existing service; follow [executor +setup](docs/operations/executor-onboarding.md) or [managed +services](docs/operations/services.md) next. -For signed releases, provision trust and verify the installer before execution -using [Signed releases and offline verification](docs/operations/releases.md). -That guide also describes the explicit mode for existing unsigned packages. +## Install a signed release + +Once your operator identifies an exact published version with signed assets, +obtain the trusted standalone bootstrap and provision the signer's trust file +independently using [Signed releases and offline +verification](docs/operations/releases.md). Replace the version placeholder +below and leave unsigned, component and local-directory settings unset. This +route also requires curl, Python 3.11 or later, and OpenSSH with `ssh-keygen -Y`. + +```sh +DEBUGLET_VERSION=VERSION_PROVIDED_BY_OPERATOR \ +DEBUGLET_REQUIRE_SIGNATURE=1 \ +DEBUGLET_RELEASE_TRUST=/etc/debuglet/release-allowed-signers \ +DEBUGLET_RELEASE_SIGNER=releases@debuglet.netsec.ethz.ch \ +DEBUGLET_PREFIX="$HOME/.local" \ +sh /path/to/trusted/bootstrap.sh +``` + +Signature verification precedes installer execution and never falls back to +unsigned mode. For offline use, set `DEBUGLET_RELEASE_DIR` to the package +directory including `release.json` and `release.json.sig`. + +For the older public v0.2.0 demo only, use `DEBUGLET_VERSION=v0.2.0 +DEBUGLET_ALLOW_UNSIGNED=1 sh /path/to/trusted/bootstrap.sh` with component, +local-directory and signature/trust settings unset. ## Supported platforms @@ -54,7 +98,13 @@ make ci-build make ci-package ``` -The full bundle is under `.cache/ci/packages/`. Use it with the same installer command above. The build also produces the component packages below. +The full bundle is under `.cache/ci/packages/`. The build prints its exact +archive name, including the generated `v0.0.0-dev.` version. Run +the development-candidate command above with that version, +`DEBUGLET_RELEASE_DIR="$PWD/.cache/ci/packages"`, and `sh scripts/bootstrap.sh`. +For another machine, copy the trusted bootstrap and the three package files; +the source checkout stays on the build machine. The build also produces the +component packages below. The package output includes `compatibility.json` and `SHA256SUMS-compatibility`. This separate record identifies the core package, @@ -73,20 +123,16 @@ Component packages are available from current source builds and are intended for | Executor operators | `executor` | `.cache/ci/packages/executor/` | `debuglet-executor` | | Dispatcher operators | `dispatcher` | `.cache/ci/packages/dispatcher/` | `debuglet-dispatcher` | -Choose a component, then run this from the source checkout after building: - -```sh -component=cli # or executor or dispatcher -cd ".cache/ci/packages/$component" -archive=$(printf '%s\n' ./debuglet-"$component"-v*-linux-amd64.tar.gz) -version=${archive#./debuglet-$component-} -version=${version%-linux-amd64.tar.gz} -sh "./install-$component.sh" --archive "$archive" \ - --checksums "./SHA256SUMS-$component" --version "$version" --prefix "$HOME/.local" -export PATH="$HOME/.local/bin:$PATH" -``` - -Each directory contains only its archive, installer and checksums. These filenames are also unique release asset names. Role installations share a prefix safely: upgrading one changes only its own command link. The daemon packages include their SQLite support and embedded migrations; they do not require `goose` or `sqlc` at runtime. Daemons still need configuration and a prepared database; see the operator guide below. +Use either bootstrap invocation above with `DEBUGLET_COMPONENT=cli`, +`executor`, or `dispatcher`. For local packages, point `DEBUGLET_RELEASE_DIR` +to that component's directory containing its archive, `install-COMPONENT.sh`, +and `SHA256SUMS-COMPONENT`. Select the same exact version as the full bundle. +Check the installed command with `--help`; the component packages omit the +bundled demo. Role installations share a prefix safely: upgrading one changes +only its own command link. The daemon packages include their SQLite support +and embedded migrations; they do not require +`goose` or `sqlc` at runtime. Daemons still need configuration and a prepared +database; see the operator guide below. Use the full bundle for `dbl demo`, `dbl up`, `dbl dispatcher up`, `dbl executor up`, and bundled samples. The CLI-only package is for connecting to an existing dispatcher and submitting your own WASM files. diff --git a/cmd/dbl/executor_join.go b/cmd/dbl/executor_join.go index c1460e8f..f31d3b67 100644 --- a/cmd/dbl/executor_join.go +++ b/cmd/dbl/executor_join.go @@ -4,6 +4,7 @@ import ( "bufio" "bytes" "context" + "crypto" "crypto/ecdsa" "crypto/elliptic" "crypto/rand" @@ -13,6 +14,7 @@ import ( "encoding/json" "encoding/pem" "errors" + "flag" "fmt" "io" "net" @@ -35,6 +37,7 @@ import ( const executorJoinUsage = `Usage: dbl executor join --dispatcher URL --executor ID --state-dir DIR [--ca-file FILE] [--token-file FILE] [--capacity BITS_PER_SECOND] [--host-tag TAG ...] + [--renew] Enroll this machine using the one-time token from Console > My nodes. The token is read without echo from the terminal, or from standard input or @@ -43,13 +46,19 @@ listing: home, office, datacentre, academic, cloud, dsl, cable, fibre, wifi, mobile, satellite or nat; it is written to [metadata] host_tags. HTTPS verifies the dispatcher using the system trust store or --ca-file; plain HTTP is accepted only on a literal loopback IP. -DIR must not exist. The command creates private identity, configuration and +For a new enrollment DIR must not exist. The command creates private identity, configuration and storage files, then prints the command to start the executor. It does not start a service or configure payments. Keep DIR for subsequent restarts. --capacity sets the bandwidth offered to the dispatcher (default 10000000 bit/s). Choose a sustainable limit for this host and its uplink. For a persistent Linux system service, follow docs/operations/executor-onboarding.md and enroll directly into /var/lib/debuglet/executors/NAME before service install. + +--renew replaces only the certificate of an existing managed executor, using a +fresh Set up again token. First drain its service and confirm joined shutdown +and disabled boot startup. Its key, configuration, database and native trust +stay unchanged. Resume the service explicitly after success. See the renewal +procedure in docs/operations/executor-onboarding.md; renewal is not automatic. ` type executorJoinResponse struct { @@ -61,10 +70,15 @@ type executorJoinResponse struct { } func executorJoinCommand(ctx context.Context, args []string, options globalOptions, stdout, stderr io.Writer) int { + return executorJoinCommandWith(ctx, args, options, stdout, stderr, productionServiceDependencies()) +} + +func executorJoinCommandWith(ctx context.Context, args []string, options globalOptions, stdout, stderr io.Writer, deps serviceDependencies) int { fs := newCommandFlagSet("executor join") endpoint := fs.String("dispatcher", "", "dispatcher HTTPS URL, including any API prefix") id := fs.String("executor", "", "executor ID shown in the console") - state := fs.String("state-dir", "", "new directory for this executor's identity and storage") + state := fs.String("state-dir", "", "directory for this executor's identity and storage") + renew := fs.Bool("renew", false, "renew a drained managed executor certificate using a fresh setup token") caFile := fs.String("ca-file", "", "trusted dispatcher CA certificate") tokenFile := fs.String("token-file", "", "file containing the enrollment token") capacity := fs.Int64("capacity", 10_000_000, "bandwidth offered to the dispatcher in bits per second") @@ -77,7 +91,14 @@ func executorJoinCommand(ctx context.Context, args []string, options globalOptio return code } if fs.NArg() != 0 || options.EndpointSet || options.Dispatcher != "" || strings.TrimSpace(*state) == "" { - return usageError("dbl executor join", executorJoinUsage, stderr, "supply --dispatcher, --executor and a new --state-dir after executor join") + return usageError("dbl executor join", executorJoinUsage, stderr, "supply --dispatcher, --executor and --state-dir after executor join") + } + changedSettings := false + fs.Visit(func(f *flag.Flag) { + changedSettings = changedSettings || f.Name == "capacity" || f.Name == "host-tag" + }) + if *renew && changedSettings { + return usageError("dbl executor join", executorJoinUsage, stderr, "--renew preserves capacity and host tags; omit their flags") } if *capacity <= 0 { return usageError("dbl executor join", executorJoinUsage, stderr, "--capacity must be positive bits per second") @@ -108,7 +129,20 @@ func executorJoinCommand(ctx context.Context, args []string, options globalOptio transport.TLSClientConfig.RootCAs = roots } statePath, err := filepath.Abs(*state) - if err == nil { + var installer *service.Installer + var retained service.Profile + if err == nil && *renew { + var code int + var ok bool + installer, code, ok = newInstaller("dbl executor join", serviceOptions{}, deps, stderr) + if !ok { + return code + } + retained, err = installer.CheckCertificateRenewal(ctx, filepath.Base(statePath)) + if err == nil && (retained.StateDir != statePath || retained.ExecutorID != *id) { + err = errors.New("--renew must name the installed executor ID and its canonical managed state directory") + } + } else if err == nil { _, err = os.Lstat(statePath) if err == nil { err = fmt.Errorf("state directory already exists: %s; keep existing identities and choose a new directory", statePath) @@ -138,6 +172,17 @@ func executorJoinCommand(ctx context.Context, args []string, options globalOptio client := &http.Client{Transport: transport, CheckRedirect: func(*http.Request, []*http.Request) error { return errors.New("enrollment redirects are not followed; use the dispatcher's HTTPS API URL") }} + if *renew { + if err := renewExecutorCertificate(ctx, installer, retained, client, joinURL, token); err != nil { + return reportFailure(ctx, "dbl executor join", stderr, err) + } + return emitReported(ctx, "dbl executor join", options.Output, stdout, stderr, map[string]string{ + "executor_id": *id, "state_dir": statePath, "operation": "renewed", + }, func(w io.Writer) error { + _, err := fmt.Fprintf(w, "Executor %s certificate renewed. Key, configuration and stored work retained.\nThe service remains stopped and disabled. Resume explicitly:\n sudo %s drain --role executor --name %s --resume\n", *id, roleShellWord(executable), roleShellWord(retained.Name)) + return err + }) + } if err := joinExecutor(ctx, client, joinURL, *id, statePath, token, hostTags, *capacity); err != nil { return reportFailure(ctx, "dbl executor join", stderr, err) } @@ -302,38 +347,51 @@ func joinExecutor(ctx context.Context, client *http.Client, endpoint, id, state, if err := storagecheck.BootstrapFresh(ctx, storagecheck.Executor, filepath.Join(state, "executor.sqlite")); err != nil { return err } - csr, err := x509.CreateCertificateRequest(rand.Reader, &x509.CertificateRequest{Subject: pkix.Name{CommonName: id}}, key) + result, err := requestExecutorCertificate(ctx, client, endpoint, id, token, key, keyPEM) if err != nil { return err } + for _, file := range []struct{ name, data string }{{"executor.crt", result.CertificatePEM}, {"ca.crt", result.CAPEM}, {"service.toml", executorJoinConfig(id, state, result, hostTags, capacity)}} { + if err := writeJoinFile(filepath.Join(state, file.name), []byte(file.data)); err != nil { + return err + } + } + return nil +} + +func requestExecutorCertificate(ctx context.Context, client *http.Client, endpoint, id, token string, key crypto.Signer, keyPEM []byte) (executorJoinResponse, error) { + var result executorJoinResponse + csr, err := x509.CreateCertificateRequest(rand.Reader, &x509.CertificateRequest{Subject: pkix.Name{CommonName: id}}, key) + if err != nil { + return result, err + } body, err := json.Marshal(map[string]string{"executor_id": id, "token": token, "csr": string(pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE REQUEST", Bytes: csr}))}) if err != nil { - return err + return result, err } req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(body)) if err != nil { - return err + return result, err } req.Header.Set("Content-Type", "application/json") req.Header.Set("Debuglet-API-Version", "1.10") resp, err := client.Do(req) if err != nil { - return fmt.Errorf("request executor enrollment: %w", err) + return result, fmt.Errorf("request executor enrollment: %w", err) } defer resp.Body.Close() if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusCreated { - return fmt.Errorf("dispatcher refused enrollment (HTTP %d); check the dispatcher URL and replace expired or used tokens in My nodes", resp.StatusCode) + return result, fmt.Errorf("dispatcher refused enrollment (HTTP %d); check the dispatcher URL and replace expired or used tokens in My nodes", resp.StatusCode) } - var result executorJoinResponse decoder := json.NewDecoder(io.LimitReader(resp.Body, 64*1024)) if err := decoder.Decode(&result); err != nil { - return fmt.Errorf("decode enrollment response: %w", err) + return result, fmt.Errorf("decode enrollment response: %w", err) } if decoder.Decode(new(any)) != io.EOF { - return errors.New("enrollment response contains unexpected data") + return result, errors.New("enrollment response contains unexpected data") } if err := validateEnrollmentCertificate(result, id, keyPEM); err != nil { - return err + return result, err } // The server can use a public certificate while issuing client identities // from its own CA. Keep both authenticated trust roots for control TLS. @@ -344,12 +402,7 @@ func joinExecutor(ctx context.Context, client *http.Client, endpoint, id, state, } } } - for _, file := range []struct{ name, data string }{{"executor.crt", result.CertificatePEM}, {"ca.crt", result.CAPEM}, {"service.toml", executorJoinConfig(id, state, result, hostTags, capacity)}} { - if err := writeJoinFile(filepath.Join(state, file.name), []byte(file.data)); err != nil { - return err - } - } - return nil + return result, nil } func validateEnrollmentCertificate(result executorJoinResponse, id string, keyPEM []byte) error { diff --git a/cmd/dbl/executor_join_renew.go b/cmd/dbl/executor_join_renew.go new file mode 100644 index 00000000..1c61568e --- /dev/null +++ b/cmd/dbl/executor_join_renew.go @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: Apache-2.0 +// Copyright 2026 ETH Zurich + +package main + +import ( + "context" + "crypto" + "crypto/tls" + "errors" + "fmt" + "net/http" + "os" + "path/filepath" + + "github.com/netsec-ethz/debuglet/internal/demo/service" +) + +func renewExecutorCertificate(ctx context.Context, installer *service.Installer, p service.Profile, client *http.Client, endpoint, token string) (err error) { + keyPEM, err := os.ReadFile(filepath.Join(p.StateDir, "executor.key")) + if err != nil { + return err + } + pair, err := tls.LoadX509KeyPair(filepath.Join(p.StateDir, "executor.crt"), filepath.Join(p.StateDir, "executor.key")) + if err != nil { + return err + } + key, ok := pair.PrivateKey.(crypto.Signer) + if !ok { + return errors.New("retained executor key cannot sign a certificate request") + } + defer func() { + if err != nil { + err = fmt.Errorf("%w; keep the service stopped. If enrollment was sent, obtain a fresh Set up again token and repeat the same --renew command; do not delete or re-enroll the state directory", err) + } + }() + result, err := requestExecutorCertificate(ctx, client, endpoint, p.ExecutorID, token, key, keyPEM) + if err != nil { + return err + } + if result.GRPCAddress != p.DispatcherGRPC || result.YamuxAddress != p.DispatcherHTTP { + return errors.New("renewal returned different native endpoints; ask the service operator for recovery instructions") + } + if err := installer.RenewCertificate(ctx, p.Name, []byte(result.CertificatePEM)); err != nil { + return fmt.Errorf("publish renewed certificate under retained native trust: %w", err) + } + return nil +} diff --git a/cmd/dbl/executor_join_renew_test.go b/cmd/dbl/executor_join_renew_test.go new file mode 100644 index 00000000..eb0c3073 --- /dev/null +++ b/cmd/dbl/executor_join_renew_test.go @@ -0,0 +1,180 @@ +// SPDX-License-Identifier: Apache-2.0 +// Copyright 2026 ETH Zurich + +package main + +import ( + "bytes" + "context" + "crypto" + "crypto/rand" + "crypto/tls" + "crypto/x509" + "crypto/x509/pkix" + "encoding/pem" + "errors" + "math/big" + "os" + "path/filepath" + "strings" + "syscall" + "testing" + "time" + + "github.com/netsec-ethz/debuglet/internal/demo/service" + "github.com/netsec-ethz/debuglet/internal/storagecheck" +) + +type renewalManager struct { + *recordingManager + failed bool +} + +func (m *renewalManager) State(ctx context.Context, unit string) (service.UnitState, error) { + state, err := m.recordingManager.State(ctx, unit) + state.Result = "success" + if m.failed { + state.Active, state.Result, state.ExecMainStatus = "failed", "exit-code", 1 + } + return state, err +} + +func TestExecutorJoinRenewManagedCertificate(t *testing.T) { + for _, scenario := range []string{"success", "expired certificate", "write failure retry", "changed endpoints", "changed issuer", "invalid response", "enabled unit", "running unit", "failed shutdown", "changed identity", "changed settings"} { + t.Run(scenario, func(t *testing.T) { + alterResponse := false + fixture := newJoinFixture(t, func(result *executorJoinResponse) { + if alterResponse && scenario == "changed endpoints" { + result.GRPCAddress = "another.example:9001" + } + if alterResponse && scenario == "invalid response" { + result.CertificatePEM = "not a certificate" + } + }) + root := t.TempDir() + state := service.StateDirectory(root, storagecheck.Executor, "worker") + var stdout, stderr bytes.Buffer + if code := run(t.Context(), fixture.args(state), &stdout, &stderr); code != exitOK { + t.Fatalf("join: %d %s", code, stderr.String()) + } + manager := newRecordingManager() + deps := serviceTestDependencies(t, manager, root) + observed := &renewalManager{recordingManager: manager} + deps.manager = func(string) (service.Manager, error) { return observed, nil } + deps.lookup = func(string, string) (service.Account, error) { return service.Account{UID: 4242, GID: 4343}, nil } + if code := serviceCommandWith(t.Context(), []string{"install", "--role", "executor", "--enrolled-state", state, "--start=false", "--enable=false"}, globalOptions{}, &stdout, &stderr, deps); code != exitOK { + t.Fatalf("install: %d %s", code, stderr.String()) + } + certPath := filepath.Join(state, "executor.crt") + if err := os.Chmod(certPath, 0640); err != nil { + t.Fatal(err) + } + if scenario == "expired certificate" { + pair, err := tls.LoadX509KeyPair(certPath, filepath.Join(state, "executor.key")) + if err != nil { + t.Fatal(err) + } + key := pair.PrivateKey.(crypto.Signer) + template := &x509.Certificate{SerialNumber: big.NewInt(99), Subject: pkix.Name{CommonName: joinExecutorID}, NotBefore: time.Now().Add(-2 * time.Hour), NotAfter: time.Now().Add(-time.Hour)} + der, err := x509.CreateCertificate(rand.Reader, template, template, key.Public(), key) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(certPath, pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}), 0640); err != nil { + t.Fatal(err) + } + } + if scenario == "changed issuer" { + // The API remains trusted, but its issuer is absent from the saved + // native roots. Renewal may not silently replace those roots. + data, _ := os.ReadFile(fixture.caFile) + if err := os.WriteFile(filepath.Join(state, "ca.crt"), data, 0600); err != nil { + t.Fatal(err) + } + } + original := map[string][]byte{} + for _, name := range []string{"executor.key", "executor.crt", "ca.crt", "executor.sqlite", "service.toml"} { + data, err := os.ReadFile(filepath.Join(state, name)) + if err != nil { + t.Fatal(err) + } + original[name] = data + } + info, _ := os.Stat(certPath) + owner := info.Sys().(*syscall.Stat_t) + chowned := false + failWrite := scenario == "write failure retry" + deps.chown = func(_ *os.File, uid, gid int) error { + if failWrite { + return errors.New("fixture refused certificate ownership") + } + if uid != int(owner.Uid) || gid != int(owner.Gid) { + t.Errorf("certificate owner changed: %d:%d", uid, gid) + } + chowned = true + return nil + } + args := append(fixture.args(state)[2:], "--renew") + unit := "debuglet-executor-worker.service" + switch scenario { + case "enabled unit": + manager.enabled[unit] = true + case "running unit": + manager.mainPID = 12345 + case "failed shutdown": + observed.failed = true + case "changed identity": + for n, arg := range args { + if arg == joinExecutorID { + args[n] = "00000000-1111-2222-3333-444444444444" + } + } + case "changed settings": + args = append(args, "--capacity", "1000") + } + alterResponse = true + renew := func() int { + stdout.Reset() + stderr.Reset() + return executorJoinCommandWith(context.Background(), args, globalOptions{}, &stdout, &stderr, deps) + } + code := renew() + wantSuccess := scenario == "success" || scenario == "expired certificate" + if !wantSuccess { + if code == exitOK || strings.Contains(stdout.String(), "certificate renewed") { + t.Fatalf("renewal unexpectedly succeeded: %s %s", stdout.String(), stderr.String()) + } + for name, want := range original { + got, _ := os.ReadFile(filepath.Join(state, name)) + if !bytes.Equal(got, want) { + t.Fatalf("failed renewal changed %s", name) + } + } + if scenario != "write failure retry" { + return + } + if !strings.Contains(stderr.String(), "fresh Set up again token") { + t.Fatalf("missing recovery instructions: %s", stderr.String()) + } + failWrite = false + code = renew() + } + if code != exitOK || !strings.Contains(stdout.String(), "certificate renewed") || !chowned { + t.Fatalf("renewal: %d %s %s", code, stdout.String(), stderr.String()) + } + for name, want := range original { + got, _ := os.ReadFile(filepath.Join(state, name)) + if bytes.Equal(got, want) == (name == "executor.crt") { + t.Errorf("unexpected contents after renewal: %s", name) + } + } + info, _ = os.Stat(certPath) + if info.Mode().Perm() != 0640 || manager.enabled[unit] || manager.mainPID != 0 { + t.Fatal("renewal changed certificate mode or enabled/started unit") + } + if strings.Contains(stdout.String()+stderr.String(), fixSecret) { + t.Fatal("renewal disclosed token") + } + }) + } +} diff --git a/configs/release-compatibility.json b/configs/release-compatibility.json index 43c2d4ae..cfc1343e 100644 --- a/configs/release-compatibility.json +++ b/configs/release-compatibility.json @@ -1,4 +1,4 @@ { "console_repository": "https://gitlab.inf.ethz.ch/OU-PERRIG/yimin/debuglet/debuglet-dashboard", - "console_revision": "49f6ff6248000081287418d9d2c1d3f4a5cdc2bd" + "console_revision": "5312f70a1ebd9e78630ac6ffc4ae67383dcb4e14" } diff --git a/deploy/README.md b/deploy/README.md index 59ce51bd..d993242c 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -224,11 +224,11 @@ lets the candidate's own installer place the payload under its own activation links, so updating dev cannot switch prod's executable. Redeploying the same release installs nothing again. -`make deploy-seed-db` writes the schema-only SQLite databases the roles -install alongside them. It runs the already-built payload's daemon commands -with `-init-database PATH`, after checking its installed manifest matches the -release in `deploy/dist`. Run `make deploy-build` first; an image from another -release is refused. Neither `deploy/dist/` nor `deploy/certs/` is committed. +For a new host, each role initializes its database with the installed release's +`-init-database PATH` command as the service account. No seed files need to be +added to a signed release bundle. Neither `deploy/dist/` nor `deploy/certs/` +is committed. `make deploy-seed-db` remains available for tools that need +separate schema-only files; the Ansible roles do not use them. ## The provisioner @@ -415,11 +415,10 @@ Executors on a shared host use separate `debuglet-prod`/`debuglet-dev` users, `debuglet-executor-.service` units. Package staging and deployment records are also separate. Prod and dev dispatchers remain on separate hosts. -A daemon never creates its database. When a host has none yet, the roles -install the schema-only seed that `make deploy-seed-db` wrote, which is the -step a daemon's absent-database refusal names. -The roles never overwrite a nonempty database with a seed, and they never -upgrade one. A zero-byte executor database can be replaced with the seed. If a +Normal daemon startup never creates its database. When a host has none yet, +the roles explicitly initialize it using the installed daemon's embedded +migrations. They never overwrite or upgrade a nonempty database. A zero-byte +executor database can be reinitialized. If a previous installation has nonempty state and the selected environment has no database yet, deployment stops and preserves that state and its legacy unit. Back the database up, place it in the selected state directory, deploy again diff --git a/deploy/ansible/roles/dispatcher/tasks/main.yml b/deploy/ansible/roles/dispatcher/tasks/main.yml index 78c9d3be..a036a252 100644 --- a/deploy/ansible/roles/dispatcher/tasks/main.yml +++ b/deploy/ansible/roles/dispatcher/tasks/main.yml @@ -36,14 +36,15 @@ upgrade-database.yml. quiet: true -- name: Provision dispatcher database (schema only, preserved on redeploy) - ansible.builtin.copy: - src: "{{ dist_dir }}/dispatcher-seed.db" # built by `make deploy-seed-db`, not committed - dest: "{{ state_dir }}/dispatcher/dispatcher.db" - owner: "{{ debuglet_user }}" - group: "{{ debuglet_group }}" - mode: "0600" - force: false # never overwrite an existing database: it holds recorded runs +- name: Initialize a missing dispatcher database with the installed release + become: true + become_user: "{{ debuglet_user }}" + ansible.builtin.command: + argv: + - "{{ payload_prefix }}/lib/debuglet/{{ debuglet_release.version }}/bin/debuglet-dispatcher" + - -init-database + - "{{ state_dir }}/dispatcher/dispatcher.db" + creates: "{{ state_dir }}/dispatcher/dispatcher.db" notify: Restart dispatcher # The RIS ASN database refresh. The dispatcher refuses to start with a diff --git a/deploy/ansible/roles/executor/tasks/main.yml b/deploy/ansible/roles/executor/tasks/main.yml index ecd6dd1c..48a4d342 100644 --- a/deploy/ansible/roles/executor/tasks/main.yml +++ b/deploy/ansible/roles/executor/tasks/main.yml @@ -56,7 +56,7 @@ - executor_legacy_unit.stat.exists # An interrupted initial start can leave a zero-byte database. It contains no -# state; unlike a nonempty database it must not block installation of the seed. +# state; unlike a nonempty database it must not block initialization. - name: Discard an empty executor database ansible.builtin.file: path: "{{ executor_state_dir }}/executor.db" @@ -65,14 +65,15 @@ - executor_db.stat.exists - executor_db.stat.size == 0 -- name: Provision executor database (schema only, preserved on redeploy) - ansible.builtin.copy: - src: "{{ dist_dir }}/executor-seed.db" # built by `make deploy-seed-db`, not committed - dest: "{{ executor_state_dir }}/executor.db" - owner: "{{ executor_user }}" - group: "{{ executor_group }}" - mode: "0600" - force: false # never overwrite an existing database: it holds recorded runs +- name: Initialize a missing executor database with the installed release + become: true + become_user: "{{ executor_user }}" + ansible.builtin.command: + argv: + - "{{ payload_prefix }}/lib/debuglet/{{ debuglet_release.version }}/bin/debuglet-executor" + - -init-database + - "{{ executor_state_dir }}/executor.db" + creates: "{{ executor_state_dir }}/executor.db" notify: Restart executor # The rendered configuration carries the TESLA seed, so its content stays out diff --git a/deploy/debuglet-deploy b/deploy/debuglet-deploy index 8faa8dd2..944018e5 100755 --- a/deploy/debuglet-deploy +++ b/deploy/debuglet-deploy @@ -92,7 +92,7 @@ if [ "$action" = certs ]; then exit fi -make -C "$root" deploy-build deploy-seed-db +make -C "$root" deploy-build deploy_version=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["version"])' \ "$root/deploy/dist/release.json") extra+=(-e "deploy_version=$deploy_version") diff --git a/deploy/test/ansible-render.sh b/deploy/test/ansible-render.sh index 9e88cbca..201f5ddd 100755 --- a/deploy/test/ansible-render.sh +++ b/deploy/test/ansible-render.sh @@ -12,7 +12,7 @@ # origin and a non-UUID executor identity, and accepts a complete set, # 3. the playbooks apply in the documented order — deploy-certs.yml, then # site.yml — and the roles create the directory skeleton, install the -# verified release package with its own installer, install the seed +# verified release package with its own installer, initialize missing # databases and render both daemon configurations and both systemd units, # 4. the rendered configurations name the configured listener addresses and # ports, carry explicit wallet-free payment settings, and put each @@ -106,17 +106,13 @@ certs=$work/certs # fixture provides it the same way, so the roles install into it rather than # create it. mkdir -p "$host" "$dist" "$host/etc/systemd/system" -# The real release package, plus stand-ins for the schema-only databases -# `make deploy-seed-db` writes. The package is installed by its own installer, -# which verifies it; the two database files are only copied into place. +# The real release package, without separately generated seed databases. +# Each installed daemon initializes its own schema on the fixture host. cp "$root/deploy/dist/release.json" "$root/deploy/dist/SHA256SUMS" \ "$root/deploy/dist/install.sh" "$dist/" release_version=$(tr -d ' \n' <"$dist/release.json" | grep -o '"version":"[^"]*"' | cut -d'"' -f4) release_archive=$(tr -d ' \n' <"$dist/release.json" | grep -o '"archive":"[^"]*"' | cut -d'"' -f4) cp "$root/deploy/dist/$release_archive" "$dist/" -for artifact in dispatcher-seed.db executor-seed.db; do - printf 'fixture %s\n' "$artifact" >"$dist/$artifact" -done # The preflight requires pinned host identities before anything connects. The # fixture host is reached over the local connection and never over SSH, but @@ -498,6 +494,34 @@ fi dispatcher_toml=$host/etc/debuglet/dispatcher/dispatcher.toml executor_toml=$host/etc/debuglet/executor-prod/executor.toml +databases=("$host/var/lib/debuglet/dispatcher/dispatcher.db" "$host/var/lib/debuglet/executor-prod/executor.db") +if python3 - "${databases[@]}" <<'PY' +import sqlite3, sys +for path, table in zip(sys.argv[1:], ('transactions', 'debuglets')): + with sqlite3.connect('file:' + path + '?mode=rw', uri=True) as db: + assert db.execute('PRAGMA quick_check').fetchone() == ('ok',) + assert db.execute('SELECT name FROM sqlite_master WHERE name=?', (table,)).fetchone() + db.execute('CREATE TABLE deployment_retention_fixture(value TEXT)') + db.execute("INSERT INTO deployment_retention_fixture VALUES ('preserved')") +PY +then + check 'installed daemons initialize valid databases without seed files' pass +else + check 'installed daemons initialize valid databases without seed files' fail +fi +sha256sum "${databases[@]}" >"$work/databases-before" +if run "$work/state-redeploy.log" site.yml && + sha256sum --check "$work/databases-before" >/dev/null; then + check 'redeployment preserves populated databases byte for byte' pass +else + check 'redeployment preserves populated databases byte for byte' fail + tail -20 "$work/state-redeploy.log" >&2 +fi +# The configuration/TLS checks below deliberately reach an invalid database, +# so a successful parse cannot accidentally start a long-running daemon. +for database in "${databases[@]}"; do + printf 'fixture database\n' >"$database" +done for file in "$dispatcher_toml" "$executor_toml" \ "$host/etc/systemd/system/debuglet-dispatcher.service" \ "$host/etc/systemd/system/debuglet-executor-prod.service" \ @@ -978,7 +1002,8 @@ rm -f "$host/opt/debuglet/dev/bin/debuglet-executor" if run "$work/dev-repair.log" deploy-executors.yml --limit executors \ -e @vars/dev.yml -e "executor_id=$fixture_dev_executor" && [ -x "$host/opt/debuglet/dev/bin/debuglet-executor" ] && - cmp -s "$dist/executor-seed.db" "$host/var/lib/debuglet/executor-dev/executor.db"; then + "$host/opt/debuglet/dev/bin/debuglet-executor" -check-database \ + -config "$host/etc/debuglet/executor-dev/executor.toml" >"$work/dev-db-check.log" 2>&1; then check 'dev repairs its own activation link and empty database' pass else check 'dev repairs its own activation link and empty database' fail @@ -1153,13 +1178,9 @@ refute 'no enrollment token is rendered by default' "$executor_toml" 'enrollment bind_host=$work/bind-host bind_certs=$work/bind-certs bind_dist=$work/bind-dist -mkdir -p "$bind_host/etc/systemd/system" "$bind_dist" "$work/bind-seed" -chmod 0700 "$work/bind-seed" +mkdir -p "$bind_host/etc/systemd/system" "$bind_dist" cp -R "$certs" "$bind_certs" cp "$dist"/* "$bind_dist/" -rm -f "$bind_dist/dispatcher-seed.db" -"$host/opt/debuglet/prod/bin/debuglet-dispatcher" -init-database "$work/bind-seed/dispatcher.db" \ - >"$work/bind-seed.log" 2>&1 && cp "$work/bind-seed/dispatcher.db" "$bind_dist/dispatcher-seed.db" cat >"$work/bind.yml" <