Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 37 additions & 9 deletions docs/info/checks.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,11 @@ See the [symbol table](https://clowdhaus.github.io/eksup/#symbol-table) for furt
| `K8S009` | Pod security policies present (removed in 1.25) | Retired | Up to 1.24 |
| `K8S010` | EBS CSI driver not installed | Retired | Up to 1.24 |
| `K8S011` | kube-proxy version skew with kubelet | Active | All versions |
| `K8S012` | kube-proxy IPVS mode deprecated (1.35+, removed 1.36) | Active | 1.35+ |
| `K8S012` | kube-proxy IPVS mode deprecated (1.35+, removal planned 1.43) | Active | 1.35+ |
| `K8S013` | Ingress NGINX controller retirement (1.35+) | Active | 1.35+ |
| `K8S014` | Nodes running containerd 1.x (unsupported from 1.36) | Active | 1.36+ |
| `K8S015` | gitRepo volumes in use (disabled in 1.36) | Active | 1.36+ |
| `K8S016` | Service externalIPs deprecated (1.36+, removal planned 1.43) | Active | 1.36+ |

<!-- END GENERATED CHECKS TABLE -->

Expand Down Expand Up @@ -201,6 +204,9 @@ Table below shows the checks that are applicable, or not, to the respective Kube
| `K8S011` | ➖ | ➖ | ➖ | ➖ | ➖ | ➖ | ➖ |
| `K8S012` | ➖ | ➖ | ➖ | ➖ | ➖ | ➖ | ➖ |
| `K8S013` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
| `K8S014` | ➖ | ➖ | ➖ | ➖ | ➖ | ➖ | ➖ |
| `K8S015` | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |
| `K8S016` | ➖ | ➖ | ➖ | ➖ | ➖ | ➖ | ➖ |

#### K8S001

Expand Down Expand Up @@ -346,17 +352,11 @@ If you recently updated your cluster to a new Kubernetes minor version, then upd

#### K8S012

`kube-proxy` IPVS proxy mode is deprecated starting in Kubernetes `v1.35` and will be removed in `v1.36`. Clusters using IPVS mode MUST migrate to `iptables` or `nftables` proxy mode.

**❌ Remediation required**

For clusters upgrading to Kubernetes `v1.36` or later — IPVS proxy mode MUST NOT be used as it is removed.

**⚠️ Remediation recommended**

For clusters upgrading to Kubernetes `v1.35` — IPVS proxy mode is deprecated and SHOULD be migrated.
`kube-proxy` IPVS proxy mode is deprecated starting in Kubernetes `v1.35`. It is expected to be disabled by default in `v1.40` and removed in `v1.43`. Clusters using IPVS mode SHOULD migrate to `nftables` or `iptables` proxy mode.

[Kubernetes kube-proxy documentation](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/)
[Kubernetes kube-proxy documentation](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/), [KEP-5495](https://github.com/kubernetes/enhancements/issues/5495)

#### K8S013

Expand All @@ -368,6 +368,34 @@ This check scans Deployments and DaemonSets for container images referencing the

[Ingress NGINX Controller](https://kubernetes.github.io/ingress-nginx/)

#### K8S014

**❌ Remediation required**

Kubernetes `v1.35` is the last release that supports containerd 1.x. Nodes MUST run containerd 2.0 or later before upgrading to Kubernetes `v1.36` or later. The check reads the container runtime version each node reports.

EKS optimized AMIs ship containerd 2.x from Kubernetes `v1.34`, so findings usually point at nodes from older or custom AMIs. The replacement nodes MUST also come from an AMI that ships containerd 2.0 or later. Fargate and EKS Auto Mode nodes are skipped because AWS manages their container runtime.

[EKS Kubernetes 1.35 release notes](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions-standard.html)

#### K8S015

**❌ Remediation required**

The `gitRepo` volume type is permanently disabled in Kubernetes `v1.36`. The API server still accepts pods that use it, but the kubelet refuses to run them. Workloads using `gitRepo` volumes MUST migrate to an init container or a git-sync sidecar before upgrading to Kubernetes `v1.36` or later.

Like the other workload checks, this check reads the pod templates of the workload controllers in the table above. Pods created directly, or by controllers outside that table, are not inspected.

[KEP-5040](https://github.com/kubernetes/enhancements/issues/5040)

#### K8S016

**⚠️ Remediation recommended**

The `spec.externalIPs` field of a Service is deprecated starting in Kubernetes `v1.36`, and its removal is planned for `v1.43`. Services using `externalIPs` SHOULD migrate to a `LoadBalancer` or `NodePort` Service, or to the Gateway API.

[KEP-5707](https://github.com/kubernetes/enhancements/issues/5707)

---

## Retired Checks
Expand Down
5 changes: 4 additions & 1 deletion docs/info/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ checks:
K8S003: { ignore: [{ name: "*-batch", namespace: "*" }] }
K8S004: { ignore: [{ name: "singleton-*", namespace: "default" }] }
K8S005: { ignore: [{ name: "*-logger*", namespace: "*monitoring*" }] }
# ... K8S006, K8S007, K8S008, K8S013 all use the same `ignore:` shape
# ... K8S006, K8S007, K8S008, K8S013, K8S015, K8S016 all use the same `ignore:` shape
```

- `K8S002.min_replicas`: Global minimum replica threshold (default: 2). Must be >= 1.
Expand Down Expand Up @@ -210,6 +210,8 @@ The following workload checks accept the `ignore:` shape shown above (each rule
- `K8S007` — termination grace period
- `K8S008` — Docker socket mount
- `K8S013` — `ingress-nginx` controller retirement
- `K8S015` — `gitRepo` volumes
- `K8S016` — Service `externalIPs` (matched against the Service `name` + `namespace`)

#### Cluster-level checks (no `ignore` support)

Expand All @@ -222,6 +224,7 @@ Cluster-level checks do not support `ignore` because their findings have no reso
- `K8S010` — EBS CSI driver
- `K8S011` — kube-proxy / kubelet version skew
- `K8S012` — kube-proxy IPVS mode (singleton DaemonSet; finding has no per-resource fields)
- `K8S014` — containerd version (node-level finding)

The right control for a cluster-level check is "disable the check entirely" — a separate, future feature.

Expand Down
17 changes: 17 additions & 0 deletions eksup/src/analysis.rs
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,18 @@ impl Results {
.kubernetes
.pod_disruption_budgets
.retain(|f| !f.finding.remediation.is_recommended());
self
.kubernetes
.containerd_version
.retain(|f| !f.finding.remediation.is_recommended());
self
.kubernetes
.git_repo_volumes
.retain(|f| !f.finding.remediation.is_recommended());
self
.kubernetes
.service_external_ips
.retain(|f| !f.finding.remediation.is_recommended());
self
.service_limits
.ec2_limits
Expand Down Expand Up @@ -153,6 +165,9 @@ impl Results {
output.push_str(&self.kubernetes.kube_proxy_ipvs_mode.to_stdout_table()?);
output.push_str(&self.kubernetes.ingress_nginx_retirement.to_stdout_table()?);
output.push_str(&self.kubernetes.pod_disruption_budgets.to_stdout_table()?);
output.push_str(&self.kubernetes.containerd_version.to_stdout_table()?);
output.push_str(&self.kubernetes.git_repo_volumes.to_stdout_table()?);
output.push_str(&self.kubernetes.service_external_ips.to_stdout_table()?);
output.push_str(&self.service_limits.ec2_limits.to_stdout_table()?);
output.push_str(&self.service_limits.ebs_gp2_limits.to_stdout_table()?);
output.push_str(&self.service_limits.ebs_gp3_limits.to_stdout_table()?);
Expand All @@ -171,6 +186,8 @@ impl Results {
output.push_str(&self.kubernetes_suppressed.docker_socket.to_stdout_table()?);
output.push_str(&self.kubernetes_suppressed.ingress_nginx_retirement.to_stdout_table()?);
output.push_str(&self.kubernetes_suppressed.pod_disruption_budgets.to_stdout_table()?);
output.push_str(&self.kubernetes_suppressed.git_repo_volumes.to_stdout_table()?);
output.push_str(&self.kubernetes_suppressed.service_external_ips.to_stdout_table()?);
} else {
writeln!(
output,
Expand Down
7 changes: 6 additions & 1 deletion eksup/src/clients.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use anyhow::Result;
use aws_sdk_autoscaling::types::AutoScalingGroup;
use aws_sdk_eks::types::{Addon, Cluster, FargateProfile, Nodegroup};
use k8s_openapi::api::core::v1::ConfigMap;
use k8s_openapi::api::core::v1::{ConfigMap, Service};

use crate::{
eks::resources::{self as eks_resources, AddonVersion, ClusterInsight, LaunchTemplate, VpcSubnet},
Expand Down Expand Up @@ -56,6 +56,7 @@ pub trait K8sClients {
fn get_eniconfigs(&self) -> impl std::future::Future<Output = Result<Vec<ENIConfig>>> + Send;
fn get_resources(&self) -> impl std::future::Future<Output = Result<Vec<StdResource>>> + Send;
fn get_pod_disruption_budgets(&self) -> impl std::future::Future<Output = Result<Vec<StdPdb>>> + Send;
fn get_services(&self) -> impl std::future::Future<Output = Result<Vec<Service>>> + Send;
}

/// Real AWS client implementation wrapping the SDK clients
Expand Down Expand Up @@ -167,4 +168,8 @@ impl K8sClients for RealK8sClients {
async fn get_pod_disruption_budgets(&self) -> Result<Vec<StdPdb>> {
k8s_resources::get_pod_disruption_budgets(&self.client).await
}

async fn get_services(&self) -> Result<Vec<Service>> {
k8s_resources::get_services(&self.client).await
}
}
23 changes: 20 additions & 3 deletions eksup/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,15 @@ pub struct ChecksConfig {
#[serde(default, rename = "K8S013")]
pub k8s013: WorkloadCheckConfig,

// K8S012 (KubeProxyIpvsMode) is intentionally absent — its finding type has
// no `resource` field, so name+namespace ignore doesn't apply.
#[serde(default, rename = "K8S015")]
pub k8s015: WorkloadCheckConfig,

#[serde(default, rename = "K8S016")]
pub k8s016: WorkloadCheckConfig,

// K8S012 (KubeProxyIpvsMode) and K8S014 (ContainerdVersion) are intentionally
// absent: their findings are cluster or node level, so name+namespace ignore
// doesn't apply.
#[serde(skip)]
compiled: OnceLock<CompiledChecks>,
}
Expand All @@ -69,6 +76,8 @@ impl Clone for ChecksConfig {
k8s007: self.k8s007.clone(),
k8s008: self.k8s008.clone(),
k8s013: self.k8s013.clone(),
k8s015: self.k8s015.clone(),
k8s016: self.k8s016.clone(),
compiled: OnceLock::new(),
}
}
Expand Down Expand Up @@ -181,6 +190,8 @@ impl ChecksConfig {
(Code::K8S007, &self.k8s007.ignore),
(Code::K8S008, &self.k8s008.ignore),
(Code::K8S013, &self.k8s013.ignore),
(Code::K8S015, &self.k8s015.ignore),
(Code::K8S016, &self.k8s016.ignore),
] {
let compiled = selectors.iter().map(|s| s.compile()).collect::<Result<Vec<_>>>()?;
if !compiled.is_empty() {
Expand Down Expand Up @@ -639,6 +650,8 @@ checks:
assert!(cfg.k8s007.ignore.is_empty());
assert!(cfg.k8s008.ignore.is_empty());
assert!(cfg.k8s013.ignore.is_empty());
assert!(cfg.k8s015.ignore.is_empty());
assert!(cfg.k8s016.ignore.is_empty());
}

#[test]
Expand All @@ -657,7 +670,7 @@ checks:
}

#[test]
fn deserialize_k8s003_through_k8s013() {
fn deserialize_k8s003_through_k8s016() {
let yaml = r#"
checks:
K8S003: { ignore: [{ name: "a", namespace: "x" }] }
Expand All @@ -666,6 +679,8 @@ checks:
K8S007: { ignore: [{ name: "d", namespace: "w" }] }
K8S008: { ignore: [{ name: "e", namespace: "v" }] }
K8S013: { ignore: [{ name: "g", namespace: "t" }] }
K8S015: { ignore: [{ name: "h", namespace: "s" }] }
K8S016: { ignore: [{ name: "i", namespace: "r" }] }
"#;
let cfg: Config = serde_yaml::from_str(yaml).unwrap();
assert_eq!(cfg.checks.k8s003.ignore[0].name, "a");
Expand All @@ -674,6 +689,8 @@ checks:
assert_eq!(cfg.checks.k8s007.ignore[0].name, "d");
assert_eq!(cfg.checks.k8s008.ignore[0].name, "e");
assert_eq!(cfg.checks.k8s013.ignore[0].name, "g");
assert_eq!(cfg.checks.k8s015.ignore[0].name, "h");
assert_eq!(cfg.checks.k8s016.ignore[0].name, "i");
}

#[test]
Expand Down
13 changes: 12 additions & 1 deletion eksup/src/finding.rs
Original file line number Diff line number Diff line change
Expand Up @@ -174,8 +174,11 @@ define_codes! {
K8S009 => { desc: "Pod security policies present (removed in 1.25)", from: None, until: Some(24) },
K8S010 => { desc: "EBS CSI driver not installed", from: None, until: Some(24) },
K8S011 => { desc: "kube-proxy version skew with kubelet", from: None, until: None },
K8S012 => { desc: "kube-proxy IPVS mode deprecated (1.35+, removed 1.36)", from: Some(35), until: None },
K8S012 => { desc: "kube-proxy IPVS mode deprecated (1.35+, removal planned 1.43)",from: Some(35), until: None },
K8S013 => { desc: "Ingress NGINX controller retirement (1.35+)", from: Some(35), until: None },
K8S014 => { desc: "Nodes running containerd 1.x (unsupported from 1.36)", from: Some(36), until: None },
K8S015 => { desc: "gitRepo volumes in use (disabled in 1.36)", from: Some(36), until: None },
K8S016 => { desc: "Service externalIPs deprecated (1.36+, removal planned 1.43)", from: Some(36), until: None },
}

#[allow(dead_code)]
Expand Down Expand Up @@ -276,6 +279,14 @@ mod tests {
assert!(Code::K8S013.is_applicable(35));
}

#[test]
fn k8s014_through_k8s016_applicable_from_1_36() {
for code in [Code::K8S014, Code::K8S015, Code::K8S016] {
assert!(!code.is_applicable(35), "{code} should not be applicable at 1.35");
assert!(code.is_applicable(36), "{code} should be applicable at 1.36");
}
}

#[test]
fn code_display() {
assert_eq!(Code::AWS001.to_string(), "AWS001");
Expand Down
Loading
Loading