Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 23 additions & 14 deletions docs/operations/systemd.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,13 @@ None of these need `sudo`, as long as you are in the `hispec-ops` group. If you
are being asked for a password, see
[systemctl keeps asking for a password](#systemctl-keeps-asking-for-a-password).

The `hispec` command itself comes from
[instrumentctl](https://github.com/CaltechOpticalObservatories/instrumentctl),
a dependency shared with other instruments. What makes it speak HISPEC is
`src/hispec/cli/instrument.toml`, which names the unit, the ops group and the
`HISPEC_` variable prefix. So a bug in `deploy` or `doctor` is fixed there,
while anything naming HISPEC is fixed here.

```bash
hispec status # every deployed daemon: running? at boot? since when?
hispec status -v hsfei_adc # in detail: its files, PID, exit code, last log lines
Expand Down Expand Up @@ -98,18 +105,18 @@ still work if you prefer them: `systemctl start hispec@hsfei_adc`,

### Starting and stopping a whole subsystem

Wherever `hispec` takes an instance name it also takes a subsystem, written
either in full or without the `hs`, or `all`:
Wherever `hispec` takes an instance name it also takes the prefix a subsystem's
instances share, or `all`:

```bash
hispec start fei # every deployed hsfei_* daemon
hispec stop fei # all of them, in reverse order
hispec restart cal # every hscal_* daemon
hispec start power fei # PDUs first, then the FEI
hispec stop power fei # FEI first, then the PDUs
hispec start hsfei # every deployed hsfei_* daemon
hispec stop hsfei # all of them, in reverse order
hispec restart hscal # every hscal_* daemon
hispec start hspower hsfei # PDUs first, then the FEI
hispec stop hspower hsfei # FEI first, then the PDUs
hispec start hsfei_adc hsfei_ms # or name the ones you want
hispec start --dry-run fei # print what it would do, do nothing
hispec status fei # just the FEI
hispec start --dry-run hsfei # print what it would do, do nothing
hispec status hsfei # just the FEI
```

Names are matched against what is deployed in `/etc/hispec/instances/`, so
Expand Down Expand Up @@ -183,7 +190,7 @@ subsystem, or everything the repo defines that this host does not have yet:

```bash
hispec deploy hsfei_adc hsfei_ms
hispec deploy fei # every hsfei_* the repo defines
hispec deploy hsfei # every hsfei_* the repo defines
hispec deploy all --no-start --no-enable # refresh the files, change nothing else
hispec deploy --new # e.g. after a git pull added instances
hispec deploy --new --dry-run # see what that would do first
Expand Down Expand Up @@ -517,17 +524,19 @@ You do **not** need it to add a daemon or to pick up code changes;
- installs `hispec@.service` (then runs `daemon-reload`), the polkit rule, the
sudoers drop-in, `/usr/local/bin/hispec` and `/usr/local/sbin/hispec-enable`;
- removes the retired `hispec-fei-start`, `hispec-fei-stop` and `hispec-doctor`
scripts (now `hispec start fei`, `hispec stop fei` and `hispec doctor`);
scripts (now `hispec start hsfei`, `hispec stop hsfei` and `hispec doctor`);
- migrates any instance still running under the old `hispec-daemon@` name,
preserving whether it was enabled and whether it was up;
- lists instances that are deployed but not set to start at boot.

`HISPEC_REPO_DIR` and `HISPEC_VENV_DIR` override the paths.

`/usr/local/bin/hispec` is a tiny wrapper that runs the CLI from the venv. The
CLI lives in `src/hispec/cli/`, uses only the standard library so that
`hispec doctor` works on a half-broken venv, and, being an editable install,
follows `git pull`.
commands come from `instrumentctl`; what stays here is
`src/hispec/cli/instrument.toml` and a few lines that pass it. Both are an
editable install, so `git pull` changes HISPEC's own configuration without a
reinstall, while updating the commands themselves means reinstalling the
dependency.

### What the unit does

Expand Down
7 changes: 6 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
where = ["src"]
include = ["hispec", "hispec.*"]

[tool.setuptools.package-data]
"hispec.cli" = ["instrument.toml"]

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
Expand All @@ -16,7 +19,7 @@ authors = [
{ name="Prakriti Gupta", email="pgupta@astro.caltech.edu" }
]
readme = "README.md"
requires-python = ">=3.10"
requires-python = ">=3.11"
dependencies = [
"keyring",
"pipython",
Expand All @@ -30,6 +33,7 @@ dependencies = [
# influxdb extra: needed by daemons/generic/keygrabber
"libby[influxdb]@git+https://github.com/CaltechOpticalObservatories/libby.git",
"hardware_device_base@git+https://github.com/COO-Utilities/hardware_device_base",
"instrumentctl@git+https://github.com/CaltechOpticalObservatories/instrumentctl.git",
"camera_daemon@git+https://github.com/COO-Utilities/camera-daemon.git"
]

Expand All @@ -45,6 +49,7 @@ ignored-modules = [
# Third-party dependencies, not installed in the lint environment
"astropy",
"camera_daemon",
"instrumentctl",
"libby",
"matplotlib",
"numpy",
Expand Down
121 changes: 12 additions & 109 deletions src/hispec/cli/__init__.py
Original file line number Diff line number Diff line change
@@ -1,118 +1,21 @@
"""``hispec``: deploy, run and inspect the HISPEC daemons on this host.
"""The ``hispec`` command: instrumentctl, driven by this instrument's config.

One command for operators and developers, replacing the separate
hispec-fei-start / hispec-fei-stop / hispec-doctor scripts. Host setup
(users, groups, unit file, polkit, venv) is still systemd/install.sh, run by
an admin; nothing here needs root.
Everything the command does lives in instrumentctl, which knows nothing about
HISPEC. What stays here is the configuration describing this instrument, and
where to find the checkout.
"""
from __future__ import annotations

import argparse
from importlib import resources
from pathlib import Path
from typing import List, Optional

from . import commands
from .doctor import doctor
from .instances import Paths

TARGETS_HELP = ("instance names (hsfei_adc), subsystems (hsfei, or just fei), "
"or 'all'")

EPILOG = """\
examples:
hispec status what is deployed here, and is it running?
hispec status -v hsfei_adc why did it stop? PID, exit code, last log lines
hispec deploy hsfei_newthing copy its files from the repo, enable, start
hispec deploy --new the same for every instance not deployed yet
hispec start fei start every deployed hsfei_* daemon
hispec stop fei stop them, in reverse order
hispec restart hsfei_adc e.g. after editing /etc/hispec/hsfei_adc.yaml
hispec logs hsfei_adc -f follow its log
hispec doctor why isn't it working?

Every subcommand takes --help. Full guide:
https://caltechopticalobservatories.github.io/hispec/operations/systemd.html
"""


def _parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="hispec",
description="Deploy, run and inspect the HISPEC daemons on this host.",
epilog=EPILOG,
formatter_class=argparse.RawDescriptionHelpFormatter,
)
sub = parser.add_subparsers(dest="command", metavar="<command>")

def add(name, func, help_text, description=None):
p = sub.add_parser(name, help=help_text, description=description or help_text,
formatter_class=argparse.RawDescriptionHelpFormatter)
p.set_defaults(func=func)
return p

p = add("deploy", commands.deploy,
"copy instances from the repo to this host, enable and start them",
"Copy each instance's .env and config from the repo into /etc/hispec, add\n"
"it to the boot set, and start it. No daemon-reload or install.sh needed.\n\n"
"The instance file always follows the repo. A deployed config that differs\n"
"from the repo is kept, since it holds this host's real hardware values;\n"
"pass --force to replace it.")
p.add_argument("names", nargs="*", metavar="name", help="instances to deploy")
p.add_argument("--new", action="store_true",
help="deploy every repo instance not yet deployed here")
p.add_argument("--force", action="store_true",
help="replace deployed configs that differ from the repo")
p.add_argument("--any-host", action="store_true",
help="deploy even if the instance names a different host")
p.add_argument("--no-start", action="store_true", help="do not start them now")
p.add_argument("--no-enable", action="store_true", help="do not start them at boot")
p.add_argument("-n", "--dry-run", action="store_true", help="print what would happen")

for verb, text in (("start", "start daemons (already-running ones are left alone)"),
("stop", "stop daemons, in reverse order (they still start at boot)"),
("restart", "restart daemons, e.g. after a config edit or git pull")):
p = add(verb, commands.act, text)
p.set_defaults(verb=verb)
p.add_argument("targets", nargs="+", metavar="target", help=TARGETS_HELP)
p.add_argument("-n", "--dry-run", action="store_true", help="print what would happen")

for verb, text in (("enable", "start daemons at every boot (--now: and start now)"),
("disable", "stop starting daemons at boot (--now: and stop now)")):
p = add(verb, commands.enable, text)
p.set_defaults(verb=verb)
p.add_argument("targets", nargs="+", metavar="target", help=TARGETS_HELP)
now = "start" if verb == "enable" else "stop"
p.add_argument("--now", action="store_true", help=f"also {now} them now")
p.add_argument("-n", "--dry-run", action="store_true", help="print what would happen")

p = add("status", commands.status, "show each daemon: running? enabled at boot? since when?")
p.add_argument("targets", nargs="*", metavar="target", help=TARGETS_HELP)
p.add_argument("-a", "--all", action="store_true",
help="include instances defined in the repo but not deployed here")
p.add_argument("-v", "--verbose", action="store_true",
help="full detail per daemon: its files, then systemctl status "
"(PID, memory, exit code, recent log lines)")
p.add_argument("-n", "--lines", type=int, default=10,
help="with -v, how many log lines to show (default 10)")

p = add("logs", commands.logs, "show daemon logs (journalctl)")
p.add_argument("targets", nargs="+", metavar="target", help=TARGETS_HELP)
p.add_argument("-f", "--follow", action="store_true", help="keep printing new lines")
p.add_argument("-n", "--lines", type=int, help="how many lines (default 100)")
p.add_argument("--since", help='e.g. "1 hour ago", today, "2026-09-21 09:00"')
p.add_argument("-p", "--priority", help="e.g. err, warning")
from instrumentctl import run

add("doctor", doctor, "check this host and your account, and say what to fix")
return parser
CONFIG = "instrument.toml"


def main(argv: Optional[List[str]] = None) -> int:
"""Entry point for the ``hispec`` console script."""
parser = _parser()
args = parser.parse_args(argv)
if not getattr(args, "func", None):
parser.print_help()
return 2
try:
return args.func(Paths.from_env(), args)
except KeyboardInterrupt:
return 130
# parents[3] is the checkout under an editable install, which is what lets
# a `git pull` change the CLI without reinstalling
return run(resources.files(__package__).joinpath(CONFIG),
repo_hint=Path(__file__).resolve().parents[3], argv=argv)
Loading
Loading