Skip to main content

Event Information

  • Event meaning & impact
    • google.iam.admin.v1.CreateServiceAccountKey is emitted when a new key (JSON or P12) is created for a service account using IAM API/Console/CLI.
    • This produces a new long‑lived credential that can be used to authenticate as that service account from outside GCP, with all roles/permissions it has.
  • Security & compliance considerations (PCI, ISO 27001, SOC 2, HIPAA)
    • Treat this as a high‑risk event: it can enable non-interactive access and potential data exfiltration.
    • Must be logged, monitored and reviewed as part of key management and least‑privilege controls; ensure keys are rotated, stored securely (e.g., secret manager), and not embedded in code or CI logs.
  • Practical monitoring & controls
    • Create log-based alerts in Cloud Logging for this event and require justification/approval for any key creation (e.g., via ticketing system).
    • Restrict serviceAccountKeys.create permission to a small admin group; prefer Workload Identity / short-lived tokens over service account keys wherever possible.

Examples

  • Untracked long‑lived keys for privileged service accounts
    • A key is created for a service account with roles like roles/owner, roles/editor, or broad org‑level roles and then stored in code repos, CI/CD variables, or shared drives.
    • Violates least‑privilege and key‑management expectations in ISO 27001, SOC 2, and CIS GCP Benchmarks (e.g., unmanaged long‑lived credentials).
  • Bypassing workload identity / short‑lived credentials
    • An engineer or attacker creates a new service account key to run workloads locally or from unmanaged hosts instead of using Workload Identity Federation or GCE default credentials.
    • Increases risk of credential theft and lateral movement, conflicting with NIST 800‑63 and zero‑trust guidance around strong identity and session management.
  • Stealthy access after off‑boarding or role change
    • A user who is about to be off‑boarded (or whose permissions are being reduced) creates a service account key and exports it, retaining persistent API access even after their own IAM permissions are revoked.
    • Undermines access revocation and traceability controls required by PCI‑DSS, HIPAA, and general IAM governance (no human‑to‑machine mapping once the key is exfiltrated).

Remediation

Using Console

  • Immediately find and revoke risky keys (console + detective controls)
    • In GCP Console go to IAM & Admin → Service Accounts → [project] → for each high‑privilege SA (Owner/Editor/org‑level roles): click the SA → Keys tab → under User‑managed keys, click Delete next to any unknown/long‑lived keys; confirm.
    • In Logs Explorer, run a query like:
      • resource.type="service_account" protoPayload.methodName="google.iam.admin.v1.IAMCredentials.GenerateServiceAccountKey"
      • and protoPayload.methodName="google.iam.admin.v1.IAM.CreateServiceAccountKey"
        to identify who created keys, from where, and for which SAs; document incidents and update your CMDB / key inventory for ISO 27001 & SOC 2 evidence.
    • In Security Command Center (if enabled) or Cloud Logging → Log‑based metrics → Alerts, create alerts on new key creation for privileged SAs so any future keys are immediately detected and reviewed.
  • Enforce short‑lived, federated, and least‑privilege access (stop local long‑lived key usage)
    • Replace local/CI usage of keys with Workload Identity Federation or GCE/GKE default identities:
      • For developers: use gcloud auth application-default login with user identity or set up Workload Identity Federation (IAM & Admin → Workload Identity Federation) to map IdP identities (OIDC/SAML) to GCP service accounts; update tooling (Terraform, CI, scripts) to use these providers instead of JSON keys.
      • For GKE: enable Workload Identity by mapping Kubernetes service accounts to GCP SAs, then remove JSON keys from pods and CI; for GCE/Cloud Run/Cloud Functions, attach minimal‑scope SAs and rely on metadata server credentials.
    • In IAM & Admin → IAM, review and tighten roles for SAs with roles/owner, roles/editor, or broad org‑level roles: replace with specific roles (e.g., roles/storage.objectAdmin, roles/pubsub.publisher) and document justification per CIS GCP and least‑privilege requirements; where possible, move org‑level roles to project‑level granular roles.
  • Prevent stealth keys and lock down key lifecycle (org‑wide controls)
    • In IAM & Admin → Organization policies (at org or folder level), search and set these to ENFORCED where feasible:
      • constraints/iam.disableServiceAccountKeyCreation for high‑risk projects or privileged SAs (or use Selective with allowlist if needed).
      • constraints/iam.allowedPolicyMemberDomains to restrict who can bind to SAs; avoid external accounts where not necessary.
    • For off‑boarding: before disabling a user, in Cloud Logging → Logs Explorer filter by their principal:
      • protoPayload.authenticationInfo.principalEmail="user@domain.com" AND protoPayload.methodName=("google.iam.admin.v1.IAM.CreateServiceAccountKey" OR "google.iam.admin.v1.IAMCredentials.GenerateServiceAccountKey")
        identify any SAs/keys they created; then in Service Accounts → [SA] → Keys, delete those keys and, if not needed, disable or delete the SA.
    • Implement a periodic control (e.g., monthly) using IAM → Service Accounts list: export CSV of SAs and keys (via Download keys list or using gcloud in a controlled environment) to review for: orphaned keys, keys older than X days, keys on privileged SAs, and keys without documented owners; revoke and document remediation for PCI‑DSS / HIPAA / SOC 2 evidence.

Using CLI

  • Discovery, immediate containment, and key lifecycle enforcement
    • List all service accounts and keys; flag privileged roles and long‑lived keys:
      • gcloud iam service-accounts list --format="table(email,uniqueId,disabled)"
      • gcloud iam service-accounts keys list --iam-account=SA_EMAIL --format="table(name, keyType, validAfterTime, validBeforeTime)"
      • To script org‑wide, combine with gcloud projects list and loop per project.
    • For keys that are untracked, over‑privileged, or associated with off‑boarded users, disable then delete after impact assessment:
      • Disable (preferred first step):
        gcloud iam service-accounts keys disable KEY_ID --iam-account=SA_EMAIL
      • Delete:
        gcloud iam service-accounts keys delete KEY_ID --iam-account=SA_EMAIL
    • Enforce key policies and logging in line with ISO 27001 / SOC 2 / CIS:
      • Disallow key creation except for break‑glass SAs using org policy:
        gcloud org-policies set-policy org-policy.yaml --organization=ORG_ID
        Example constraints/iam.disableServiceAccountKeyCreation: enforced: true at org/folder level.
      • Ensure Admin Activity & Data Access logs for google.iam.admin.v1.* and serviceAccount.keys.* are enabled for all projects.
  • Prevent bypass of workload identity / enforce short‑lived access
    • Mandate Workload Identity Federation / GCE metadata instead of keys:
      • For GKE:
        gcloud container clusters update CLUSTER_NAME --workload-pool=PROJECT_ID.svc.id.goog --location=LOCATION
      • For external ID providers: set up WIF pools/providers (one‑time):
        gcloud iam workload-identity-pools create POOL_ID --location=global
        gcloud iam workload-identity-pools providers create-oidc PROVIDER_ID --workload-identity-pool=POOL_ID --issuer-uri="https://issuer.example.com" --location=global
    • Block or tightly gate key creation to stop “local key” patterns:
      • Use org policy constraints/iam.disableServiceAccountKeyCreation (above) and create a narrow exception folder/project for audited break‑glass SAs.
      • Monitor and alert on key creation: log‑based metric on AddServiceAccountKey and CreateServiceAccountKey with notifications to security.
    • Provide compliant, auditable alternatives for engineers:
      • Use gcloud auth application-default login with user identities and fine‑grained IAM, or WIF for local/dev.
      • Document that direct key usage is prohibited under NIST 800‑63 / zero‑trust and codify in CI: fail pipelines when a key JSON is checked in (e.g., using secret scanners).
  • Access revocation, privileged SA hardening, and governance
    • During off‑boarding / role change, explicitly remove and rotate SA keys:
      • For a user‑linked or sensitive SA, list and remove keys:
        gcloud iam service-accounts keys list --iam-account=SA_EMAIL
        gcloud iam service-accounts keys delete KEY_ID --iam-account=SA_EMAIL
      • Rotate keys for shared/critical SAs periodically and always upon admin departure, reflecting PCI‑DSS/HIPAA expectations for timely revocation.
    • Reduce excessive roles and enforce least privilege:
      • Replace roles/owner / roles/editor and broad org roles with custom or predefined least‑privilege roles:
        gcloud projects remove-iam-policy-binding PROJECT_ID --member=serviceAccount:SA_EMAIL --role=roles/editor
        gcloud projects add-iam-policy-binding PROJECT_ID --member=serviceAccount:SA_EMAIL --role=roles/storage.objectAdmin (example; tailor to need)
      • For org‑level bindings:
        gcloud organizations remove-iam-policy-binding ORG_ID --member=serviceAccount:SA_EMAIL --role=ROLE
    • Strengthen traceability and policy around human ↔ machine mapping:
      • For human‑operated actions, prefer user identities + gcloud or just‑in‑time elevation (no generic SAs) to maintain an audit trail.
      • Periodically audit for privileged SAs and keys (CIS GCP 1.x controls) via scripted gcloud + Org Policy; reconcile against CMDB / IAM registry to ensure every SA and key has a documented owner, purpose, and expiration.

Using Python

  • Detect and inventory risky keys (privileged SAs, long‑lived, or owned by soon‑to‑be‑offboarded users)
    • Enumerate all service accounts and keys, flagging: (a) keys on SAs with roles/owner, roles/editor, or org‑/folder‑level roles; (b) user‑managed keys older than X days; (c) keys created by specific users (e.g., about‑to‑be‑offboarded employees) using Cloud Audit Logs.
    • Example Python (list all service accounts and keys; you can extend with IAM/Logging filters to label risky ones):
  • Enforce workload identity / short‑lived credentials and remove legacy keys
    • For GCE/GKE/Cloud Run/Cloud Functions, migrate to IAM‑based identities (Workload Identity / Workload Identity Federation / default service account with no user‑managed keys). Enforce org policies:
      • constraints/iam.disableServiceAccountKeyCreation = TRUE (or restricted)
      • constraints/iam.allowedPolicyMemberDomains (limit external principals)
      • CI/CD: use OIDC‑based Workload Identity Federation instead of JSON keys.
    • After migration, rotate and delete existing keys, prioritising privileged SAs, keys older than threshold, and keys not seen in logs for N days:
  • Prevent stealthy persistence and ensure compliant off‑boarding
    • Embed in HR/IT off‑boarding runbooks: before disabling a user, query Cloud Audit Logs for recent google.iam.admin.v1.CreateServiceAccountKey events by that principal, enumerate keys on any SAs they manage, then revoke/delete keys; ensure logs are retained for ≥1 year (PCI‑DSS, SOC 2, ISO 27001).
    • Example: basic Python using Logging API to find keys created by a user in last N days and then delete them (you can wire this into an off‑boarding automation):