Skip to main content

More Info:

Consider the use of an external secrets storage and management system, instead of using Kubernetes Secrets directly, if you have more complex secret management needs. Ensure the solution requires authentication to access secrets, has auditing of access to and use of secrets, and encrypts secrets. Some solutions also make it easier to rotate secrets.

Risk Level

Medium

Address

Security

Compliance Standards

  • CIS Kubernetes

Triage and Remediation

Remediation

Manual Steps

  1. Inventory current Secret usage
    Run on any machine with kubectl access:
    Identify which applications store sensitive data (passwords, tokens, keys) directly in Kubernetes Secrets.
  2. Identify how applications consume Secrets
    Run on any machine with kubectl access:
    For workloads using Secrets, note whether they mount them as environment variables or volumes and how many distinct secrets each app depends on.
  3. Assess complexity and compliance requirements
    For a representative set of applications (e.g., by namespace or criticality), review:
    • Frequency of secret changes/rotations (ask app owners or check change history in Git/IaC).
    • Regulatory or internal requirements for:
      • Strong auditing of secret access
      • Centralized key management
      • Automated or frequent rotation
        Decide whether built‑in Kubernetes Secrets (with at-rest encryption enabled) are sufficient or whether centralized external management is required.
  4. Evaluate existing external secret solutions in your environment
    Check if an external manager is already in use (examples: AWS Secrets Manager, AWS KMS + Parameter Store, GCP Secret Manager, Azure Key Vault, HashiCorp Vault):
    If such components exist, review their configuration and confirm they provide: authentication, encryption, and auditable access logs.
  5. Decide and design the target pattern
    Based on steps 1–4, choose per application (or namespace):
    • Continue using Kubernetes Secrets (if complexity/compliance needs are low), ensuring:
      • Encryption at rest is enabled at the cluster/etcd level.
      • Access is restricted via RBAC.
    • Or adopt an external secret manager, typically by:
      • Configuring a SecretStore/ClusterSecretStore or similar CRD for your provider.
      • Mapping external secret entries to Kubernetes Secrets consumed by workloads.
        Document the chosen approach and migration plan (which apps, in which order).
  6. Implement and verify the chosen solution
    For one pilot application, implement your chosen pattern (for external manager, deploy the integration operator/agent and define the mapping manifests). Then verify:
    Separately, in your cloud provider console or external secrets system, confirm: authentication is required, access is logged/audited, and secrets are encrypted and can be rotated.
Review which namespaces are in use for production or sensitive workloads.
Problem indication: Many Opaque or kubernetes.io/dockerconfigjson secrets in app namespaces may mean application credentials are stored only in Kubernetes Secrets.
Review:
  • type: (commonly Opaque for app secrets)
  • metadata.annotations and labels (look for external-secrets operators, e.g. external-secrets.io/*, vault.hashicorp.com/*, secrets-store.csi.x-k8s.io/*)
Problem indication: Sensitive app secrets of type Opaque with no annotations/labels tying them to an external manager likely come only from in-cluster storage.
Common indicators:
  • externalsecrets.external-secrets.io
  • secretproviderclasses.secrets-store.csi.x-k8s.io
  • vaultsecrets.*, etc.
Problem indication: No CRDs or resources related to external secrets providers may indicate the cluster relies entirely on native Kubernetes Secrets.
Problem indication: Application namespaces with Secrets but no ExternalSecret or SecretProviderClass resources suggest no external system is backing those secrets.
Problem indication: Large numbers of such secrets in app namespaces, without any external-secret mechanism in the cluster, indicate a stronger case to consider external secret storage.
Problem indication: Many Pods directly consuming Opaque Secrets that are not clearly sourced from an external manager points to a need to evaluate external secret storage.Use these outputs to decide:
  • Whether secrets are currently managed only as native Kubernetes Secrets.
  • Whether that is acceptable for your risk profile, or whether you should adopt an external secrets system as the benchmark recommends.