Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
apiVersion: runwhen.com/v1
kind: GenerationRules
spec:
generationRules:
- resourceTypes:
- deployment
- statefulset
matchRules:
- type: pattern
pattern: "openobserve"
properties: [name]
mode: substring
slxs:
- baseName: k8s-openobserve-hc
shortenedBaseName: openobs-hc
qualifiers: ["resource", "namespace", "cluster"]
baseTemplateName: k8s-openobserve-healthcheck
levelOfDetail: basic
outputItems:
- type: slx
- type: sli
- type: runbook
templateName: k8s-openobserve-healthcheck-taskset.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
apiVersion: runwhen.com/v1
kind: ServiceLevelIndicator
metadata:
name: {{slx_name}}
labels:
{% include "common-labels.yaml" %}
annotations:
{% include "common-annotations.yaml" %}
runwhen.com/sli: "true"
spec:
displayUnitsLong: OK
displayUnitsShort: ok
locations:
- {{default_location}}
description: Aggregates OpenObserve API reachability, workload replica readiness, and storage health for {{match_resource.resource.metadata.name}}.
codeBundle:
{% if repo_url %}
repoUrl: {{repo_url}}
{% else %}
repoUrl: https://github.com/runwhen-contrib/rw-cli-codecollection.git
{% endif %}
{% if ref %}
ref: {{ref}}
{% else %}
ref: main
{% endif %}
pathToRobot: codebundles/k8s-openobserve-healthcheck/sli.robot
intervalStrategy: intermezzo
intervalSeconds: 300
configProvided:
- name: CONTEXT
value: "{{context}}"
- name: NAMESPACE
value: "{{match_resource.resource.metadata.namespace}}"
- name: OBJECT_NAME
value: "{{match_resource.resource.metadata.name}}"
- name: KUBERNETES_DISTRIBUTION_BINARY
value: "{{custom.kubernetes_distribution_binary | default('kubectl')}}"
- name: OPENOBSERVE_SERVICE_NAME
value: ""
- name: OPENOBSERVE_HTTP_PORT
value: "5080"
- name: OPENOBSERVE_LOCAL_PORT
value: "5080"
- name: OPENOBSERVE_ORGANIZATION
value: "default"
- name: OPENOBSERVE_BASE_URL
value: ""
- name: OPENOBSERVE_API_TIMEOUT
value: "25"
- name: STORAGE_S3_FAILURE_THRESHOLD
value: "10"
secretsProvided:
{% if wb_version %}
{% include "kubernetes-auth.yaml" ignore missing %}
{% else %}
- name: kubeconfig
workspaceKey: {{custom.kubeconfig_secret_name | default("kubeconfig")}}
{% endif %}
alertConfig:
tasks:
persona: eager-edgar
sessionTTL: 10m
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
apiVersion: runwhen.com/v1
kind: ServiceLevelX
metadata:
name: {{slx_name}}
labels:
{% include "common-labels.yaml" %}
annotations:
{% include "common-annotations.yaml" %}
spec:
imageURL: https://storage.googleapis.com/runwhen-nonprod-shared-images/icons/kubernetes/resources/labeled/ns.svg
alias: OpenObserve Health Check for {{match_resource.resource.metadata.name}}
asMeasuredBy: OpenObserve workload / Pod / API health and storage metrics in Kubernetes.
configProvided:
- name: OBJECT_NAME
value: {{match_resource.resource.metadata.name}}
owners:
- {{workspace.owner_email}}
statement: The OpenObserve deployment in the namespace should have ready replicas, stable Pods, a reachable HTTP API, and healthy storage/ingestion/cluster state.
additionalContext:
{% include "kubernetes-hierarchy.yaml" ignore missing %}
qualified_name: "{{ match_resource.qualified_name }}"
tags:
{% include "kubernetes-tags.yaml" ignore missing %}
- name: cloud
value: kubernetes
- name: service
value: openobserve
- name: scope
value: project
- name: access
value: read-only
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
apiVersion: runwhen.com/v1
kind: Runbook
metadata:
name: {{slx_name}}
labels:
{% include "common-labels.yaml" %}
annotations:
{% include "common-annotations.yaml" %}
spec:
location: {{default_location}}
description: OpenObserve workload, Pod, API, storage, cluster, ingestion queue, and PVC health checks for {{match_resource.resource.metadata.name}} in namespace {{match_resource.resource.metadata.namespace}}.
codeBundle:
{% if repo_url %}
repoUrl: {{repo_url}}
{% else %}
repoUrl: https://github.com/runwhen-contrib/rw-cli-codecollection.git
{% endif %}
{% if ref %}
ref: {{ref}}
{% else %}
ref: main
{% endif %}
pathToRobot: codebundles/k8s-openobserve-healthcheck/runbook.robot
configProvided:
- name: CONTEXT
value: "{{context}}"
- name: NAMESPACE
value: "{{match_resource.resource.metadata.namespace}}"
- name: OBJECT_NAME
value: "{{match_resource.resource.metadata.name}}"
- name: KUBERNETES_DISTRIBUTION_BINARY
value: "{{custom.kubernetes_distribution_binary | default('kubectl')}}"
- name: OPENOBSERVE_SERVICE_NAME
value: ""
- name: OPENOBSERVE_HTTP_PORT
value: "5080"
- name: OPENOBSERVE_LOCAL_PORT
value: "5080"
- name: OPENOBSERVE_ORGANIZATION
value: "default"
- name: OPENOBSERVE_BASE_URL
value: ""
- name: OPENOBSERVE_API_TIMEOUT
value: "25"
- name: OPENOBSERVE_PF_WAIT_SECS
value: "15"
- name: OPENOBSERVE_CREDENTIAL_SECRET_NAME
value: "{{custom.openobserve_credential_secret_name | default('')}}"
- name: OPENOBSERVE_CREDENTIAL_USER_KEY
value: ""
- name: OPENOBSERVE_CREDENTIAL_PASSWORD_KEY
value: ""
- name: OPENOBSERVE_ALLOW_EXEC
value: "false"
- name: OPENOBSERVE_POD_RESTART_THRESHOLD
value: "5"
- name: STORAGE_S3_FAILURE_THRESHOLD
value: "10"
- name: INGESTION_QUEUE_KEY_THRESHOLD
value: "100000"
- name: CLUSTER_PENDING_JOBS_THRESHOLD
value: "100"
- name: PVC_USAGE_THRESHOLD_PERCENT
value: "80"
secretsProvided:
{% if wb_version %}
{% include "kubernetes-auth.yaml" ignore missing %}
{% else %}
- name: kubeconfig
workspaceKey: {{custom.kubeconfig_secret_name | default("kubeconfig")}}
- name: openobserve_credentials
workspaceKey: {{custom.openobserve_credentials_secret_name | default("openobserve_credentials")}}
{% endif %}
48 changes: 48 additions & 0 deletions codebundles/k8s-openobserve-healthcheck/.test/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# k8s-openobserve-healthcheck - Tests

## Overview

This `.test/` directory provides infrastructure and Taskfile automation used to
validate the `k8s-openobserve-healthcheck` CodeBundle against a Kubernetes
cluster with RunWhen Local.

## Prerequisites

- A running Kubernetes cluster (local `kind`/`minikube`/k3s or a real cluster).
- `kubectl` configured with a kubeconfig file.
- Docker (to run the `runwhen-local` container).
- `task`, `yq`, `curl`, `ajv` on PATH.

## Quick start

1. Point `kubectl` at your test cluster.
2. Drop a kubeconfig file named `kubeconfig.secret` in this directory
(used by RunWhen Local discovery).
3. `task build-infra` - creates the test namespace and OpenObserve resources.
4. `task default` - runs discovery and generates SLX/SLI/runbook configs under
`output/workspaces/`.
5. `task clean` - removes the test objects and discovery output.

## Test resources

The `kubernetes/manifest.yaml` creates a namespace `test-openobserve-health`
containing:

- a Secret `openobserve-credentials` with `ROOT_USER_EMAIL` /
`ROOT_USER_PASSWORD` (matching Helm defaults)
- a Service `openobserve` exposing port `5080`
- a Deployment `openobserve` (name matches the `openobserve` substring used by
the generation rule)
- a PVC `openobserve-data`

The generation rule matches the Deployment by name, so discovery should produce
one SLX named after the workload `openobserve` in `test-openobserve-health`.

## Scenarios

The deployment declares `ZO_LOCAL_MODE=true` and a bound PVC. For failure
scenarios you can edit the manifest before applying:

- `replicas: 2` with a broken image -> replica/pod health issues.
- Unbind the PVC (`persistentVolumeReclaimPolicy` / delete the PV) -> PVC issues.
- Point the Service at a bogus port -> API health issues.
Loading
Loading