Sensitive Values Should Not Be Passed As Literal Env Vars

Manual Steps

Identify offending Pods and env vars (any machine with kubectl access)

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
    | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
    | .metadata as $m
    | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
    | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
    | ((.spec.containers // []) + (.spec.initContainers // []))[]
    | .name as $c
    | (.env // [])[]
    | select((.value != null) and (.name | test("PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY"; "i")))
    | "ns=\($m.namespace) pod=\($m.name) container=\($c) env=\(.name)"
  ][]'

For each offending Pod, determine the owning controller (any machine with kubectl access)

# Example: inspect one violating Pod
kubectl -n <namespace> get pod <pod-name> -o json | jq '
  {metadata:{name,namespace,ownerReferences},spec:{nodeName,containers:[.spec.containers[].name]}}'

If ownerReferences is set (e.g., Deployment, StatefulSet, Job), you must edit that controller, not the live Pod.

Create a Secret with the sensitive value (any machine with kubectl access)

For a single key:

kubectl -n <namespace> create secret generic <secret-name> \
  --from-literal=<env-var-name>=<sensitive-value>

Or from a file:

kubectl -n <namespace> create secret generic <secret-name> \
  --from-file=<env-var-name>=/absolute/path/to/file

Update the controller manifest to use valueFrom.secretKeyRef (any machine with kubectl access)

Get the current manifest:

kubectl -n <namespace> get deployment <deployment-name> -o yaml > /tmp/deployment.yaml

In /tmp/deployment.yaml, under the relevant container’s env: section, replace:

- name: MY_SECRET_PASSWORD
  value: "super-secret-value"

with:

- name: MY_SECRET_PASSWORD
  valueFrom:
    secretKeyRef:
      name: <secret-name>
      key: MY_SECRET_PASSWORD

Apply the change:
kubectl apply -f /tmp/deployment.yaml

This will roll the Pods for that controller and recreate them with the Secret reference.

For standalone Pods (no ownerReferences), recreate them using a manifest (any machine with kubectl access)
# Export current Pod spec to a file kubectl -n <namespace> get pod <pod-name> -o yaml > /tmp/pod.yaml
Edit /tmp/pod.yaml:
- Remove fields under metadata such as uid, resourceVersion, creationTimestamp, managedFields, selfLink, generation.
- Remove status: entirely.
- In each offending container’s env: entry, change value: to valueFrom.secretKeyRef as in step 4. Then delete and recreate:
kubectl -n <namespace> delete pod <pod-name> kubectl apply -f /tmp/pod.yaml

Verify the cluster is compliant (any machine with kubectl access)

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
    | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
    | .metadata as $m
    | (.spec.nodeName // "") as $node
    | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
    | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
    | ((.spec.containers // []) + (.spec.initContainers // []))[]
    | .name as $c
    | (.env // [])[]
    | select((.value != null) and (.name | test("PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY"; "i")))
    | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
      + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
      + (if $node   == ""   then "" else " node=\($node)" end)
      + (if $labels == ""   then "" else " labels=\($labels)" end)
      + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
      + " container=\($c) env=\(.name) is_compliant=false"
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'

Confirm the output is is_compliant=true or that no offending env vars remain.

Using kubectl

On any machine with kubectl access:

Inspect the offending Pod and identify the literal sensitive env var(s):

kubectl get pod POD_NAME -n NAMESPACE -o yaml

Look under spec.containers[].env (and initContainers[].env) for entries like:

env:
  - name: DB_PASSWORD
    value: super-secret-password        # <-- needs to be moved to a Secret

Create (or update) a Secret containing the sensitive value(s). Example:

kubectl create secret generic app-secrets \
  -n NAMESPACE \
  --from-literal=DB_PASSWORD='super-secret-password'

If the Secret must be changed later, use:

kubectl delete secret app-secrets -n NAMESPACE
kubectl create secret generic app-secrets \
  -n NAMESPACE \
  --from-literal=DB_PASSWORD='new-secret-password'

Patch the Pod’s controller (Deployment/StatefulSet/Job/etc.) manifest so the Pod uses valueFrom.secretKeyRef instead of a literal value. First, export the owning controller manifest:

kubectl get deploy DEPLOYMENT_NAME -n NAMESPACE -o yaml > deployment.yaml

Edit deployment.yaml and change:

env:
  - name: DB_PASSWORD
    value: super-secret-password

to:

env:
  - name: DB_PASSWORD
    valueFrom:
      secretKeyRef:
        name: app-secrets
        key: DB_PASSWORD

Repeat for every sensitive variable flagged by the check.

Apply the updated manifest:

kubectl apply -f deployment.yaml

This will roll out new Pods that reference the Secret.

If the Pod is not managed by a higher-level controller (a naked Pod), you must delete and recreate it from a corrected manifest:

kubectl get pod POD_NAME -n NAMESPACE -o yaml > pod.yaml
# edit pod.yaml as in step 3 to use valueFrom.secretKeyRef
kubectl delete pod POD_NAME -n NAMESPACE
kubectl apply -f pod.yaml

Verify that no literal sensitive env vars remain and that the check passes:

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
    | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
    | .metadata as $m
    | (.spec.nodeName // "") as $node
    | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
    | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
    | ((.spec.containers // []) + (.spec.initContainers // []))[]
    | .name as $c
    | (.env // [])[]
    | select((.value != null) and (.name | test("PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY"; "i")))
    | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
      + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
      + (if $node   == ""   then "" else " node=\($node)" end)
      + (if $labels == ""   then "" else " labels=\($labels)" end)
      + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
      + " container=\($c) env=\(.name) is_compliant=false"
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'

Automation

#!/usr/bin/env bash
#
# Remediates: Sensitive values should not be passed as literal env vars
# Scope: Any machine with kubectl access to the OKE cluster
#
# Strategy:
# - Discover all Pod specs that set secret-like env vars via literal `value:`
# - Export their owning controller manifests (Deployment/StatefulSet/DaemonSet/Job/CronJob)
# - For each offending env var, replace:
#       env:
#         - name: FOO_PASSWORD
#           value: literal
#   with:
#       env:
#         - name: FOO_PASSWORD
#           valueFrom:
#             secretKeyRef:
#               name: <auto-created Secret>
#               key: FOO_PASSWORD
#   where <auto-created Secret> is `<workload-name>-env-secrets`
# - Create/patch Secrets per-namespace with those keys and literal values
# - Apply the updated controller manifests
# - Verify via the audit query
#
# WARNING:
# - This script *changes* how your workloads receive sensitive values.
# - It must know the actual literal values; it reads them from the current
#   Pod specs, which still contain them.
# - If an env var already uses valueFrom, it is left untouched.
# - Controllers will roll Pods as they are updated.

set -euo pipefail

#-----------------------------
# Configuration / helpers
#-----------------------------

SENSITIVE_PATTERN='PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY'

# Only process these controller kinds; Pods directly created (no controller)
# will be skipped and must be handled manually.
CONTROLLER_KINDS='Deployment|StatefulSet|DaemonSet|Job|CronJob'

timestamp() { date -u +"%Y%m%dT%H%M%SZ"; }

workdir_base="${PWD}/env-secret-remediation-$(timestamp)"
mkdir -p "${workdir_base}"

echo "Working directory: ${workdir_base}"

# Require jq, yq (v4), kubectl
for bin in kubectl jq yq; do
  if ! command -v "${bin}" >/dev/null 2>&1; then
    echo "ERROR: ${bin} is required on this machine. Install it and re-run." >&2
    exit 1
  fi
done

#-----------------------------
# Step 1: Discover offending Pods
#-----------------------------

echo "Discovering Pods with secret-like env vars set via literal value..."

offenders_json="${workdir_base}/offending-pods.json"
kubectl get pods --all-namespaces -o json > "${offenders_json}"

# Build a compact list of offending containers + env vars + owning controller
offenders_list="${workdir_base}/offenders-list.json"
jq --arg re "${SENSITIVE_PATTERN}" '
  [
    .items[]
    | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
    | .metadata as $m
    | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
    | select($own != null and ($own.kind | test("'"${CONTROLLER_KINDS}"'")))
    | (.spec.containers // [] + .spec.initContainers // [])
      as $containers
    | reduce ($containers[]) as $c ([]; . + (
        ($c.env // [])
        | map(select((.value != null)
                     and (.name | test($re; "i"))))
        | map({
            namespace: $m.namespace,
            podName: $m.name,
            controllerKind: $own.kind,
            controllerName: $own.name,
            containerName: $c.name,
            envName: .name,
            envValue: .value
          })
      ))
  ] | add // []' "${offenders_json}" > "${offenders_list}"

if [[ ! -s "${offenders_list}" ]]; then
  echo "No offending literal secret-like env vars found. Cluster is compliant."
  exit 0
fi

echo "Found $(jq 'length' "${offenders_list}") offending env vars."

#-----------------------------
# Step 2: Prepare per-namespace Secret specs
#-----------------------------

# Map: namespace -> controller (kind/name) -> secretName -> keys
# We choose secret name: <controller-name>-env-secrets (dns-1123 trimmed)
echo "Preparing Secret manifests..."

secrets_dir="${workdir_base}/secrets"
mkdir -p "${secrets_dir}"

jq -c '.[]' "${offenders_list}" | while read -r row; do
  ns=$(jq -r '.namespace' <<<"${row}")
  ctrl_kind=$(jq -r '.controllerKind' <<<"${row}")
  ctrl_name=$(jq -r '.controllerName' <<<"${row}")
  env_name=$(jq -r '.envName' <<<"${row}")
  env_value=$(jq -r '.envValue' <<<"${row}")

  # Normalized secret name: <controller-name>-env-secrets
  base_name="${ctrl_name}-env-secrets"
  # Truncate to 253 chars to satisfy DNS-1123 label length, strip invalid chars
  secret_name=$(printf '%s' "${base_name}" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9-.' '-' | cut -c1-253)

  ns_dir="${secrets_dir}/${ns}"
  mkdir -p "${ns_dir}"
  secret_file="${ns_dir}/${secret_name}.yaml"

  if [[ ! -f "${secret_file}" ]]; then
    cat > "${secret_file}" <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: ${secret_name}
  namespace: ${ns}
type: Opaque
data: {}
EOF
  fi

  # Add/overwrite the key with base64-encoded value (idempotent)
  b64=$(printf '%s' "${env_value}" | base64 | tr -d '\n')
  yq -i ".data.\"${env_name}\" = \"${b64}\"" "${secret_file}"
done

echo "Generated Secret manifests under ${secrets_dir}"

#-----------------------------
# Step 3: Export and patch controllers
#-----------------------------

controllers_dir="${workdir_base}/controllers"
mkdir -p "${controllers_dir}"

echo "Exporting and patching controller manifests..."

# Unique list of controller refs
controllers_list="${workdir_base}/controllers-unique.json"
jq '[ .[] | {namespace, controllerKind, controllerName} ] | unique' \
  "${offenders_list}" > "${controllers_list}"

jq -c '.[]' "${controllers_list}" | while read -r row; do
  ns=$(jq -r '.namespace' <<<"${row}")
  kind=$(jq -r '.controllerKind' <<<"${row}")
  name=$(jq -r '.controllerName' <<<"${row}")

  ctrl_file="${controllers_dir}/${ns}-${kind}-${name}.yaml"

  echo "  Processing ${kind}/${ns}/${name}"

  # Export current controller spec
  if ! kubectl get "${kind}" "${name}" -n "${ns}" -o yaml > "${ctrl_file}.orig" 2>/dev/null; then
    echo "    WARN: Unable to fetch ${kind}/${ns}/${name}, skipping."
    continue
  fi

  cp "${ctrl_file}.orig" "${ctrl_file}"

  # Determine corresponding Secret name
  base_name="${name}-env-secrets"
  secret_name=$(printf '%s' "${base_name}" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9-.' '-' | cut -c1-253)

  # Patch env entries for both containers and initContainers
  for field in spec.template.spec.containers spec.template.spec.initContainers; do
    # If field exists
    if ! yq e ".${field} == null" "${ctrl_file}" | grep -qi 'true'; then
      # For each container, rewrite offending env entries
      # Logic:
      #   env[].value != null and name matches sensitive pattern
      #   => delete .value, set .valueFrom.secretKeyRef
      yq -i "
        .${field}[] |= (
          .env |= (
            map(
              if (.value != null and (.name | test(\"${SENSITIVE_PATTERN}\"; \"i\"))) then
                .value = null
                | .valueFrom = {
                    secretKeyRef: {
                      name: \"${secret_name}\",
                      key: .name
                    }
                  }
              else .
              end
            )
          )
        )
      " "${ctrl_file}"
    fi
  done

  # Apply patched controller
  kubectl apply -f "${ctrl_file}"
done

#-----------------------------
# Step 4: Apply Secrets (create or update)
#-----------------------------

echo "Applying Secret manifests..."

find "${secrets_dir}" -type f -name '*.yaml' -print0 | while IFS= read -r -d '' f; do
  kubectl apply -f "${f}"
done

#-----------------------------
# Step 5: Verification
#-----------------------------

echo "Waiting for controller rollouts (Deployments/StatefulSets/DaemonSets)..."

# Best-effort rollout status for Deployments and StatefulSets
jq -c '.[]' "${controllers_list}" | while read -r row; do
  ns=$(jq -r '.namespace' <<<"${row}")
  kind=$(jq -r '.controllerKind' <<<"${row}")
  name=$(jq -r '.controllerName' <<<"${row}")

  case "${kind}" in
    Deployment|StatefulSet)
      echo "  Checking rollout for ${kind}/${ns}/${name}..."
      kubectl rollout status "${kind}/${name}" -n "${ns}" --timeout=180s || \
        echo "    WARN: rollout status timed out for ${kind}/${ns}/${name}"
      ;;
    DaemonSet)
      echo "  Checking DaemonSet availability for ${kind}/${ns}/${name}..."
      kubectl get ds "${name}" -n "${ns}"
      ;;
    Job|CronJob)
      echo "  Skipping rollout wait for ${kind}/${ns}/${name} (job-type)."
      ;;
  esac
done

echo "Re-running compliance query..."

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
    | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
    | .metadata as $m
    | (.spec.containers // [] + .spec.initContainers // [])[]
    | (.env // [])[]
    | select((.value != null) and (.name | test("'"${SENSITIVE_PATTERN}"'"; "i")))
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else "is_compliant=false (remaining violations: \($rows | length))" end'

echo "Done. Review ${workdir_base} for backup manifests and applied changes."

OCI Misconfigurations

Sensitive Values Should Not Be Passed As Literal Env Vars

More Info:

Risk Level

Address

Compliance Standards

Triage and Remediation

Remediation

​More Info:

​Risk Level

​Address

​Compliance Standards

​Triage and Remediation

​Remediation