API Server Request Timeout Should Be Set As Appropriate

#!/usr/bin/env bash
#
# Report kube-apiserver --request-timeout settings on all control plane nodes.
# Requirements:
#   - Run on any machine with kubectl access and cluster-admin privileges.
#   - kubectl must be configured to talk to the target cluster.
#
# This is READ-ONLY: it does not change anything.

set -euo pipefail

echo "Collecting kube-apiserver --request-timeout settings from all nodes..."
echo

# 1) Find all nodes that are labeled as control plane / master
CONTROL_PLANE_NODES=$(
  kubectl get nodes \
    -l 'node-role.kubernetes.io/master=,node-role.kubernetes.io/control-plane=' \
    -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' 2>/dev/null || true
)

if [ -z "${CONTROL_PLANE_NODES}" ]; then
  echo "No nodes labeled with node-role.kubernetes.io/master or node-role.kubernetes.io/control-plane found."
  echo "If this is a self-managed control plane, you may need to adjust the label selector."
  exit 1
fi

# 2) For each control plane node, try to read the static pod manifest and inspect the flag
for NODE in ${CONTROL_PLANE_NODES}; do
  echo "=== Node: ${NODE} ==="

  # Path is from the finding; we assume standard kubeadm/static pod layout.
  MANIFEST_PATH="/etc/kubernetes/manifests/kube-apiserver.yaml"

  # Use kubectl debug (where available) or privileged pod to read the manifest.
  # This is implemented as: create a temporary pod on the node and cat the file.
  # It is read-only and safe, but does require permission to schedule pods.

  # Create a short-lived pod that mounts /etc/kubernetes as hostPath
  POD_NAME="apiserver-timeout-inspect-${NODE//./-}-$$"
  kubectl run "${POD_NAME}" \
    --restart=Never \
    --image=busybox:1.36 \
    --overrides="$(
      cat <<EOF
{
  "apiVersion": "v1",
  "kind": "Pod",
  "metadata": {
    "name": "${POD_NAME}"
  },
  "spec": {
    "nodeName": "${NODE}",
    "hostPID": true,
    "tolerations": [
      {
        "operator": "Exists"
      }
    ],
    "containers": [
      {
        "name": "inspect",
        "image": "busybox:1.36",
        "command": ["sleep", "300"],
        "volumeMounts": [
          {
            "name": "k8s-etc",
            "mountPath": "/host-etc-kubernetes",
            "readOnly": true
          }
        ]
      }
    ],
    "volumes": [
      {
        "name": "k8s-etc",
        "hostPath": {
          "path": "/etc/kubernetes",
          "type": "Directory"
        }
      }
    ]
  }
}
EOF
    " >/dev/null 2>&1

  # Wait briefly for the pod to be running or failed
  kubectl wait pod "${POD_NAME}" --for=condition=Ready --timeout=60s >/dev/null 2>&1 || true

  echo "- Manifest: ${MANIFEST_PATH}"

  # Try to read the manifest
  if kubectl exec "${POD_NAME}" -- sh -c "test -f '${MANIFEST_PATH}'" >/dev/null 2>&1; then
    # Extract the request-timeout flag from the manifest
    TIMEOUT_LINE=$(kubectl exec "${POD_NAME}" -- sh -c \
      "grep -E -- '--request-timeout=' '${MANIFEST_PATH}' || true")

    if [ -z "${TIMEOUT_LINE}" ]; then
      echo "  request-timeout: NOT SET (kube-apiserver will use its compiled default, often 1m0s)"
    else
      # Try to isolate the value
      TIMEOUT_VALUE=$(printf '%s\n' "${TIMEOUT_LINE}" | \
        sed -n 's/.*--request-timeout=\([^[:space:]]*\).*/\1/p' | head -n1)

      if [ -z "${TIMEOUT_VALUE}" ]; then
        echo "  request-timeout: PRESENT but could not parse line:"
        echo "    ${TIMEOUT_LINE}"
      else
        echo "  request-timeout: ${TIMEOUT_VALUE}"
      fi
    fi
  else
    echo "  ERROR: ${MANIFEST_PATH} not found or not readable on this node."
  fi

  # Clean up the temporary pod
  kubectl delete pod "${POD_NAME}" --ignore-not-found --grace-period=0 --force >/dev/null 2>&1

  echo
done

cat <<'EOF'
Interpretation guidance:
- Problematic / requires review when:
  * "request-timeout: NOT SET" -> timeout is relying on the default and may not match your risk appetite.
  * "request-timeout: <very large value>" (e.g., many minutes or '0') -> long-running or hung requests
    can tie up apiserver worker threads and expose you to slowloris-style attacks.
  * "ERROR: ... not found" -> kube-apiserver is not using the expected static pod manifest path and must be investigated.

- Acceptable (subject to your environment and SLAs) when:
  * "request-timeout: <reasonable value>" such as 300s as in the benchmark example, or another value
    that is explicitly chosen and documented for your environment.

This script does NOT change the configuration; it only reports what is currently set so you can decide
whether a manual adjustment to /etc/kubernetes/manifests/kube-apiserver.yaml is warranted on each
control plane node.
EOF