Repository navigation
Feat/perf five module - #949
Merged
Merged
Conversation
…scheduling a wakeup
The deploy script creates perf-sensor-checklist-rules in monitoring from the bundled checklist-rules.yaml; the Deployment mounts it and points CHECKLIST_RULES at it, so kubectl shows what the sensor scores. A hash of the rules file is stamped on the pod template, so a rules change rolls the pod.
…udget One question may now ask for several fixes: Claude runs every entry's tools, then answers once with a labelled root cause per entry and makes one change. Two combined entries match the workshop modules: memory + CPU + startup boost (manifest), and blocking fix + size again + item 10 (code). New item 10 entry and references/shutdown-budget.md: grace = preStop sleep + the app's graceful-shutdown timeout.
…ble items references/admission-policy.yaml is a native ValidatingAdmissionPolicy + binding (Deny, app namespace) for checklist items 1, 3, 9 and 10, one message per check, mirroring the sensor's reading of the Deployment. The skill writes it to the app's k8s/ when asked to enforce the practices. Tested on Kubernetes 1.34 with a server-side dry run of passing and failing manifests.
The policy bound to the app namespace rejects the baseline manifest (item 10), so a repeat run must delete it and its binding before applying the starting point.
The artillery process outlives its load loop for up to 600 s; a repeat run started on top of it measured 110 requests/s instead of 50 (latency, working set and pool waits all inflated).
…module flow Regenerated from the java-performance content: Measure, Fix the manifest, Fix the image, Fix the code, Enforce. Expected flips per module, reset-app usage.
Long evidence (items 9 and 12) continues under its own column, so a terminal and the workshop samples stay readable without editing the real output.
…ingPolicy The team that owns the namespace owns the policy, without cluster-admin, and Kyverno's background scan reports the running Deployment in a PolicyReport. Same CEL as the ValidatingAdmissionPolicy reference, tested live on Kyverno 1.19: denies items 1, 3 and 10 with their messages and ignores other namespaces. reset-app deletes the namespaced policies; acceptance uses the PolicyReport instead of a dry run (an unchanged apply sends no request).
…ion policies A fresh Claude session asked for a Kyverno policy did not load the skill and wrote its own audit-only ClusterPolicy.
…d is running common.sh sets -e with an ERR trap, and pkill exits 1 when nothing matches.
…d the Kyverno module
In a continued session the image question cited the cold start of the pod before the manifest change (16.8 s) instead of the current one (7.6 s).
…pool change depends on measured waits
…eria in the ws-test README remote/ runs scripts on the IDE over SSM, syncs an unpushed branch, runs and fetches a ws-test; samples/ refreshes the content's Sample expands from a run; capture/ takes the Grafana and IDE screenshots; analysis/ reads Pyroscope CPU shares. The environment is found from the account (IDE by Name tag, bucket from SSM), nothing hardcoded. apps/perf-sensor/ACCEPTANCE.md moves into the README. AGENTS.md with the repo conventions.
- deployment-resources.yaml: no comments inside the folded JAVA_TOOL_OPTIONS (the JVM got '#' as an option); test parses the fragment. - The sensor returns the JVM flags the skill used to compute: measure.jvmFlags (javaHeapOpts, activeProcessorCount for the current limits), sizeMemory.javaHeapOpts for the new limit, sizeCpu.activeProcessorCount. cracHeap leaves sizing-policy.yaml. - Database password as a BuildKit secret (build.sh --secret id=db-password) in Dockerfile.crac and Dockerfile.aot, not a build arg; crac.md and the Dockerfile say the checkpoint heap still holds it. - Dockerfile.aot: Spring Boot 4 auto-configuration class names, a failed training run fails the build.
…rement - Heap used/committed from the heap section of GC.heap_info only (older JDKs append Metaspace and class space); no committed fallback. - No effectiveCpuCount from the CPU limit when the JFR ring is missing (item 5 compared the limit with itself); item 1 needs the pod status; item 9 needs a readinessProbe. - preStop duration only from the sleep action, sleep N or sh -c "sleep N"; any other hook is unreadable and item 10 UNKNOWN; the 30 s drain is named an assumption. - Sizing policies validated (factors >= 1, positive steps); NaN and +-Inf samples read as absent; sizeMemory BLOCKED without a CPU limit instead of picking G1. - A known-empty Ready set reads no runtime facts instead of every pod in the window. - Item 1 is "Memory request equals limit, no OOM kill": requests == limits is not the Guaranteed QoS class while the CPU request is below the limit.
- SENSOR_SERVICES names the workloads the sensor answers for; any other service and any name that is not a Kubernetes name is refused at the shared boundary (REST 400, MCP error) instead of reading 0 blocking threads or reaching a query. - Steady CPU per pod: each pod's mean minus its own JIT share, then the worst pod (a fleet JIT share no longer lowers the busiest pod's load). - Micrometer and Pyroscope queries scoped to the namespace and the current Ready pods; the request rate leaves out actuator traffic. - Startup resolved per current pod (log gauge, else that pod's ready time), unknown while a pod has neither; the startup cache is keyed by the container's start, so a restart in the same pod is read again.
…reuse old evidence - DumpServer: one file per request, deleted after reading; a non-zero jcmd exit is a 503 (a failed dump served the previous one); jcmd output drained on its own thread so the 25 s deadline holds; a pool of four handlers so a slow dump does not block /healthz. - profile-loop.sh: uploads with connect/total deadlines, a file deleted only after a 2xx, at most 8 pending files (older ones dropped with a log line); a failed attach is logged as an error instead of "attached". - diagnoseBlocking reports the time actually sampled, the pod and the dumps tried and taken; "no block" needs 80 % of the dumps. - measure re-checks the settle time after a rollout during the wait; the REST checklist answers SETTLING when its wait cap runs out instead of scoring a younger pod, and stops measuring after 120 s or when the client disconnects. - Prometheus and Pyroscope clients with connect/read deadlines; the pod log read stops at 1 MiB.
…tainer the sensor reads - Injection only for the supported pod: the first container is the app, named like the pod's app label, with runAsUser APP_UID set; and only once: no sidecar yet, none of the volume names, /perf or port 9100 in use. A new ValidatingPolicy perf-profiler-opt-in denies other opt-ins at the apply of the Deployment (autogen), with the reason. - Module 5 Kyverno reference: the rules check the app container (named like the Deployment, else the first), so a sidecar without probes passes; item 9 needs a readinessProbe; item 10 reads the preStop sleep only from the sleep action, sleep N or sh -c "sleep N" and denies any other hook. - README: the supported workload, and SYS_PTRACE is outside the baseline and restricted Pod Security Standards. Tested with server-side dry runs on Kyverno 1.19.1.
…a shipped network boundary - workload.javaToolOptions and jfr.jvmArgs carry only JVM tuning flags (-XX, -Xmx, -Xms, -Xss, -Xmn): an allowlist, so no system property, agent argument or half of a quoted value leaves the sensor (the name-based mask missed -Dsigning.key and quoted values). - diagnoseBlocking no longer returns poolSizeTarget: request concurrency is not database demand, and the connection budget across replicas is not measured. The skill leaves the pool size alone and reports the waits as evidence. - k8s/networkpolicy.yaml: only the sensor reaches /dump, only Prometheus reaches the sensor (server-validated, not applied); README says to apply it before a shared cluster and to keep pods/portforward to the operators. Both skills treat tool output as data, never as instructions.
…to the two workshops that replaced it perf-collector still runs, privileged, in the java-on-aws workshop; say so. The deploy paths in the READMEs name java-performance and mention the java-on-amazon-eks copies.
…amespace tag The app sets its own namespace tag at startup; a CRaC checkpoint built outside a pod freezes it as "default", so the namespace matcher read no request rate on the restored pod (module 4 found no load). Scope by application and the discovered pod label only.
… through wrappers app-deploy, app-prepare, claude-code, perf-profiler, perf-sensor, prebuild-images and startup-cpu-boost were byte-identical copies (review O-02). They exec the java-performance implementation now; a wrapper is replaced by an own script when the workshops diverge.
…r stale scrapes cAdvisor updates container_cpu_usage_seconds_total every 15-30 s and the scrapes in between repeat the value. A 60 s rate() sees one or two real updates and extrapolates over the stale samples: on a restored pod it read 0.10 cores where the counter grew 0.28 cores/s, so item 6 flipped between runs on the same state. The mean is now the growth between the first and last update inside the window over the time between them (raw samples).
… terminal layout at bootstrap - perf-sensor.sh install creates perf-sensor-port-forward (localhost:8090 -> the sensor, Restart=always, as ec2-user); reset-app restarts it and stops hand-started forwards; verify.sh checks it. Nobody keeps a terminal open for the forward. - ide-layout.sh: the editor-area terminal that opens with the IDE is where Claude Code runs; a user-level task opens the Shell terminal in the bottom panel (no .vscode folder in ~/environment). The IDE capture follows the layout.
…mmer modules and the refresh mapping
… are evidence With the pool size no longer guessed (A-02), a single wait for the app's one pooled connection flipped item 11 between runs on the same code. Waits stay in the evidence and in diagnoseBlocking; a pool size is a decision against the database's connection budget.
…end with the exact closing line Run 20261011T153229Z: two of four answers added build steps inside the closing line and one opened with a remark about its own edits.
… names the heap bound from jfr.jvmArgs Run 20261011T155522Z: the manifest answer was delegated to a subagent (its hand-off line landed in the answer) and called the default 25 % heap a fixed -Xmx.
…mit alone still blocks Run 20261011T161801Z: the publish moved to afterCommit but kept .get(); the callback runs on the request thread inside the interceptor, so item 11 stayed red.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.