Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ One binary, no config to write, no browser.
- Lists jobs, allocations, tasks, task groups, deployments, evaluations,
services, namespaces, variables, clients, node pools, servers, volumes,
CSI plugins and the ACL objects.
- Watches the deploy of a job with `w`: the newest evaluation, the deployment
and its allocations, until it ends.
- Updates the screen when the cluster changes, using the Nomad event stream,
and shows the stream itself on `:events`.
- Counts on one screen what needs attention: dead jobs, failed allocations,
Expand Down
29 changes: 29 additions & 0 deletions docs/jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,12 @@ urga submits exactly what was planned. If someone changed the job after the
plan, the cluster refuses the submit, and urga asks you to plan again with
`r`.

After a submit or a revert, urga goes back to where the edit or the revert
started (the job list or the versions screen) and opens the watch of the deploy
of the job on top of it. `esc` leaves the watch; the deploy goes
on. A periodic or parameterized job has no deploy of its own, so after its
submit urga only goes back.

## Versions and reverting

`v` lists the versions the cluster keeps for a job. `enter` shows what a
Expand Down Expand Up @@ -232,3 +238,26 @@ nothing left to promote, pause or fail.

System jobs have deployments since Nomad 1.11. They have no canaries, so
their Canaries and Promoted columns show `-`.

## Watching a deploy

`w` on a job opens `Deploy (Job: web, Version: 7)`: the screen of the
deployment of the version the job runs, which updates as the deploy goes. At
the top, urga shows the newest evaluation of the job and, when it blocked or
failed to place something, why, in the words of the evaluation screen. Under
it, one line says how the deploy stands:

- `Waiting for the deployment of version 7.` until the job has a deployment
of that version. The table lists the allocations of the job at that
version, with the keys of an allocation.
- The status and description of the deployment while it runs, then
`Deployment successful.`, `Deployment failed: ...` or
`Deployment cancelled: ...` when it ends.
- `web has no deployment for version 7: its allocations are below.` for a
batch or sysbatch job, and for a version that made none: the newest
evaluation of the job saw the version, completed, and no deployment came of
it (a stopped job, an update with `max_parallel = 0`).

Below are the groups of the deployment and its allocations, with the keys of
the deployment screen. `esc` leaves the screen; the deploy goes on. `w` is not
shown on a periodic or parameterized job, and it works in read-only mode.
10 changes: 9 additions & 1 deletion docs/keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,7 @@ start, stop, restart or drain then applies to every marked row.
| `h` | Job file the job was submitted with | |
| `v` | Versions | |
| `l` | Logs of a task in every allocation of the job. Not shown for a periodic or parameterized job. | |
| `w` | Watch the deploy of the job at the version it runs. Not shown for a periodic or parameterized job. | |
| `e` | Edit the job in your editor | yes |
| `ctrl-s` | Start a stopped job, or stop a running one | yes |
| `ctrl-p` | Purge a stopped job: it leaves the cluster with its versions. Shown on a stopped job. | yes |
Expand Down Expand Up @@ -259,7 +260,7 @@ On what was done to the count of a group, `d` describes its policy.
| --- | --- | --- |
| `left`, `right`, `tab` | Choose Cancel or Submit | |
| `enter` | Press the chosen button | |
| `y` | Submit, or revert, at once | yes |
| `y` | Submit, or revert, at once, and watch the deploy of a job that runs itself | yes |
| `r` | Plan again | |
| `esc` | Go back without submitting | |
| `w` | Wrap long lines | |
Expand Down Expand Up @@ -393,6 +394,13 @@ keys of [Allocations](#allocations), and these:
These keys are shown only while the deployment is active. `p` is shown only
when the group under the cursor has canaries waiting to be promoted.

### A deploy

The deploy screen, opened with `w` on a job, has the keys of
[A deployment](#a-deployment) once the job has a deployment of the version it
watches. Before that it has the keys of [Allocations](#allocations). `esc`
leaves it; the deploy goes on.

### Namespaces and services

| Key | What it does | Changes |
Expand Down
27 changes: 26 additions & 1 deletion internal/nomad/deployments.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ import (
"context"
"sort"
"time"

"github.com/hashicorp/nomad/api"
)

// Active says the deployment is not over: it can still be promoted, paused
Expand Down Expand Up @@ -56,6 +58,29 @@ func (c *Client) Deployment(ctx context.Context, namespace, deploymentID string)
return DeploymentDetail{}, err
}

return newDeploymentDetail(d), nil
}

// JobDeployments are the deployments of a job, the newest first: one for
// each version that made one. A batch job has none.
func (c *Client) JobDeployments(ctx context.Context, namespace, jobID string) ([]DeploymentDetail, error) {
list, _, err := c.api.Jobs().Deployments(jobID, false, c.query(ctx, namespace))
if err != nil {
return nil, err
}

out := make([]DeploymentDetail, 0, len(list))

for _, d := range list {
if d != nil {
out = append(out, newDeploymentDetail(d))
}
}

return out, nil
}

func newDeploymentDetail(d *api.Deployment) DeploymentDetail {
detail := DeploymentDetail{Deployment: Deployment{
ID: d.ID,
JobID: d.JobID,
Expand Down Expand Up @@ -87,7 +112,7 @@ func (c *Client) Deployment(ctx context.Context, namespace, deploymentID string)

sort.Slice(detail.Groups, func(i, j int) bool { return detail.Groups[i].Name < detail.Groups[j].Name })

return detail, nil
return detail
}

// DeploymentAllocations are the allocations a deployment placed, with how
Expand Down
39 changes: 39 additions & 0 deletions internal/nomad/deployments_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -122,3 +122,42 @@ func TestDeployment_Active(t *testing.T) {
r.False(nomad.Deployment{Status: status}.Active(), status)
}
}

func TestJobDeployments_NewestFirst(t *testing.T) {
r := require.New(t)

client, asked := recorder(t, `[
{"ID": "dep-7", "JobID": "web", "JobVersion": 7, "Status": "cancelled", "CreateIndex": 20,
"StatusDescription": "Cancelled due to newer version of job"},
{"ID": "dep-8", "JobID": "web", "JobVersion": 8, "Status": "running", "CreateIndex": 30,
"TaskGroups": {"web": {"DesiredTotal": 2}, "api": {"DesiredTotal": 1}}},
{"ID": "dep-6", "JobID": "web", "JobVersion": 6, "Status": "successful", "CreateIndex": 10}
]`)

deployments, err := client.JobDeployments(context.Background(), "production", "web")
r.NoError(err)

r.Equal("/v1/job/web/deployments", asked.URL.Path)
r.Equal("production", asked.URL.Query().Get("namespace"))
r.Equal("false", asked.URL.Query().Get("all"))

ids := []string{}
for _, d := range deployments {
ids = append(ids, d.ID)
}

r.Equal([]string{"dep-8", "dep-7", "dep-6"}, ids)
r.Equal(uint64(7), deployments[1].JobVersion)
r.Equal("Cancelled due to newer version of job", deployments[1].StatusDescription)
r.Equal([]string{"api", "web"}, []string{deployments[0].Groups[0].Name, deployments[0].Groups[1].Name})
}

func TestJobDeployments_WhenTheJobHasNone(t *testing.T) {
r := require.New(t)

client, _ := recorder(t, `[]`)

deployments, err := client.JobDeployments(context.Background(), "production", "batch-job")
r.NoError(err)
r.Empty(deployments)
}
41 changes: 35 additions & 6 deletions internal/nomad/evaluation.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ type EvaluationDetail struct {

StatusDescription string

// CreateIndex is the index of the cluster when the evaluation was made.
CreateIndex uint64

// BlockedEval waits for room to place what this one could not;
// PreviousEval and NextEval are the decisions before and after it.
BlockedEval string
Expand Down Expand Up @@ -68,6 +71,7 @@ func newEvaluationDetail(eval *api.Evaluation) EvaluationDetail {
detail := EvaluationDetail{
Evaluation: newEvaluation(eval),
StatusDescription: eval.StatusDescription,
CreateIndex: eval.CreateIndex,
BlockedEval: eval.BlockedEval,
PreviousEval: eval.PreviousEval,
NextEval: eval.NextEval,
Expand Down Expand Up @@ -137,10 +141,39 @@ func (c *Client) FailedPlacement(ctx context.Context, namespace, jobID string) (
return EvaluationDetail{}, err
}

newest := newestOf(evals, func(eval *api.Evaluation) bool { return len(eval.FailedTGAllocs) > 0 })

if newest == nil {
return EvaluationDetail{}, ErrNoFailures
}

return newEvaluationDetail(newest), nil
}

// NewestEvaluation is the newest evaluation of the job, nil when it has none.
func (c *Client) NewestEvaluation(ctx context.Context, namespace, jobID string) (*EvaluationDetail, error) {
evals, _, err := c.api.Jobs().Evaluations(jobID, c.query(ctx, namespace))
if err != nil {
return nil, err
}

newest := newestOf(evals, func(*api.Evaluation) bool { return true })
if newest == nil {
return nil, nil
}

detail := newEvaluationDetail(newest)

return &detail, nil
}

// newestOf is the evaluation with the highest create index among those that
// pass wanted, nil when none does.
func newestOf(evals []*api.Evaluation, wanted func(*api.Evaluation) bool) *api.Evaluation {
var newest *api.Evaluation

for _, eval := range evals {
if eval == nil || len(eval.FailedTGAllocs) == 0 {
if eval == nil || !wanted(eval) {
continue
}

Expand All @@ -149,11 +182,7 @@ func (c *Client) FailedPlacement(ctx context.Context, namespace, jobID string) (
}
}

if newest == nil {
return EvaluationDetail{}, ErrNoFailures
}

return newEvaluationDetail(newest), nil
return newest
}

// EvaluationAllocations are the allocations an evaluation placed or changed.
Expand Down
34 changes: 34 additions & 0 deletions internal/nomad/evaluation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,37 @@ func TestEvaluationAllocations(t *testing.T) {
r.Equal("web", allocs[0].TaskGroup)
r.Equal("failed", allocs[1].Status)
}

func TestNewestEvaluation(t *testing.T) {
r := require.New(t)

client, asked := recorder(t, `[
{"ID": "old", "JobID": "web", "Status": "complete", "CreateIndex": 5},
{"ID": "blocked", "JobID": "web", "Status": "blocked", "CreateIndex": 30, "JobModifyIndex": 25,
"FailedTGAllocs": {"web": {"NodesEvaluated": 3, "NodesExhausted": 3}}},
{"ID": "middle", "JobID": "web", "Status": "complete", "CreateIndex": 20}
]`)

eval, err := client.NewestEvaluation(context.Background(), "production", "web")
r.NoError(err)

r.Equal("/v1/job/web/evaluations", asked.URL.Path)
r.Equal("production", asked.URL.Query().Get("namespace"))

r.NotNil(eval)
r.Equal("blocked", eval.ID)
// When it was made says it is of a version or not, even for a submit that
// changed nothing, which gives the evaluation no job index.
r.Equal(uint64(30), eval.CreateIndex)
r.Equal(3, eval.Failures[0].NodesExhausted)
}

func TestNewestEvaluation_WhenTheJobHasNone(t *testing.T) {
r := require.New(t)

client, _ := recorder(t, `[]`)

eval, err := client.NewestEvaluation(context.Background(), "production", "web")
r.NoError(err)
r.Nil(eval)
}
6 changes: 6 additions & 0 deletions internal/nomad/version.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ type JobVersion struct {

Submitted time.Time

// JobModifyIndex is the index of the cluster when the version was made:
// an evaluation of the job at or above it has seen this version.
JobModifyIndex uint64

// Changes is how many fields differ from the version before this one.
Changes int
}
Expand All @@ -53,6 +57,8 @@ func (c *Client) JobVersions(ctx context.Context, namespace, jobID string) ([]Jo
Version: valueOf(job.Version),
Stable: valueOf(job.Stable),
Current: i == 0,

JobModifyIndex: valueOf(job.JobModifyIndex),
}

if job.VersionTag != nil {
Expand Down
3 changes: 2 additions & 1 deletion internal/nomad/version_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ const jobVersions = `{
"Versions": [
{
"ID": "web", "Name": "web", "Namespace": "production",
"Version": 3, "Stable": true, "SubmitTime": 1758499200000000000,
"Version": 3, "Stable": true, "SubmitTime": 1758499200000000000, "JobModifyIndex": 40,
"VersionTag": {"Name": "golden", "Description": "the one that works"}
},
{"ID": "web", "Name": "web", "Version": 2, "Stable": false, "SubmitTime": 1758412800000000000},
Expand Down Expand Up @@ -54,6 +54,7 @@ func TestJobVersions_Read(t *testing.T) {
r.True(versions[0].Current)
r.True(versions[0].Stable)
r.Equal("golden", versions[0].Tag)
r.Equal(uint64(40), versions[0].JobModifyIndex)
r.False(versions[0].Submitted.IsZero())

r.False(versions[1].Current)
Expand Down
2 changes: 1 addition & 1 deletion internal/ui/actions.go
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ func revertJob(p jobsPage, e env) (jobsPage, outcome) {
return p, outcome{}
}

return p, outcome{cmd: planFor(e.client, planState{revert: true, namespace: job.Namespace, jobID: job.ID})}
return p, outcome{cmd: planFor(e.client, planState{revert: true, namespace: job.Namespace, jobID: job.ID, jobType: job.Type, launches: launchesOthers(job)})}
}

// allocAction is what a key does to each allocation, and how the question
Expand Down
19 changes: 19 additions & 0 deletions internal/ui/app_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,12 @@ type fakeClient struct {
deploymentNamespace string
deploymentID string

// jobDeployments are the deployments of a job, the newest first.
jobDeployments []nomad.DeploymentDetail

// newest is the newest evaluation of a job; nil is none.
newest *nomad.EvaluationDetail

// promotedGroups are the groups promoted on their own, paused whether
// each call asked to pause or to resume.
promotedGroups []string
Expand Down Expand Up @@ -406,6 +412,7 @@ func (f *fakeClient) NodeAllocations(_ context.Context, nodeID string) ([]nomad.

func (f *fakeClient) JobVersions(_ context.Context, namespace, jobID string) ([]nomad.JobVersion, error) {
f.askedNamespace, f.askedJobID = namespace, jobID
f.listed("versions", namespace)

return f.versions, f.err
}
Expand Down Expand Up @@ -994,6 +1001,18 @@ func (f *fakeClient) DeploymentAllocations(_ context.Context, namespace, id stri
return f.deploymentAllocs, f.err
}

func (f *fakeClient) JobDeployments(_ context.Context, namespace, jobID string) ([]nomad.DeploymentDetail, error) {
f.listed("job deployments", namespace+"/"+jobID)

return f.jobDeployments, f.err
}

func (f *fakeClient) NewestEvaluation(_ context.Context, namespace, jobID string) (*nomad.EvaluationDetail, error) {
f.listed("newest evaluation", namespace+"/"+jobID)

return f.newest, f.err
}

func (f *fakeClient) Deployments(_ context.Context, namespace string) ([]nomad.Deployment, error) {
f.askedNamespace = namespace
f.listed("deployments", namespace)
Expand Down
7 changes: 6 additions & 1 deletion internal/ui/deployment.go
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,11 @@ type deploymentPage struct {
read bool

allocs []nomad.Alloc

// version narrows the logs to the allocations of that version, when
// versioned: the watch of a deploy has rows of one version only.
version uint64
versioned bool
}

// deploymentOf is the page of a deployment in its namespace, which is where
Expand Down Expand Up @@ -329,7 +334,7 @@ func (deploymentPage) reading(ctx context.Context, client Client, ref rowRef) (n

// logsOf: the logs of the allocations the deployment placed for its job.
func (p deploymentPage) logsOf(env) logScope {
return logScope{namespace: p.namespace, jobID: p.jobID, deploymentID: p.deploymentID}
return logScope{namespace: p.namespace, jobID: p.jobID, deploymentID: p.deploymentID, version: p.version, versioned: p.versioned}
}

// inView is the deployment the page read.
Expand Down
2 changes: 2 additions & 0 deletions internal/ui/edit.go
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,8 @@ func jobFile(client jobsClient, job nomad.Job) load {
return planFor(client, planState{
namespace: job.Namespace,
jobID: job.ID,
jobType: job.Type,
launches: launchesOthers(job),
source: source,
vars: spec.Variables,
read: spec.Index,
Expand Down
Loading
Loading