K8s sandbox running untrusted remediation

Using Console

To remediate the misconfiguration “Consider GKE Sandbox For Running Untrusted Workloads” for GCP using the GCP console, you can follow these steps:

Open the Google Cloud Console and navigate to the GKE cluster that you want to remediate.
Click on the “Workloads” tab and select the workload that you want to run in the GKE Sandbox.
Click on the “Edit” button to open the workload configuration.
In the “Security” section of the configuration, enable the “Sandbox” option.
Click on the “Save” button to apply the changes.
Verify that the workload is now running in the GKE Sandbox by checking the “Status” column in the “Workloads” tab. The status should be “Running” with a green checkmark.
Repeat steps 2-6 for any other workloads that need to be remediated.

By enabling the GKE Sandbox for running untrusted workloads, you are providing an additional layer of security to your GKE cluster by running potentially malicious workloads in a secure, isolated environment.

Using CLI

The misconfiguration of using GKE Sandbox for running untrusted workloads can be remediated in GCP using the following steps through GCP CLI:

Firstly, check if there are any workloads running on GKE Sandbox that are untrusted. This can be done by running the following command in GCP CLI:
gcloud alpha container node-pools describe [NODE_POOL_NAME] --cluster=[CLUSTER_NAME] --zone=[ZONE]
Replace [NODE_POOL_NAME], [CLUSTER_NAME] and [ZONE] with the corresponding values for your GKE Sandbox.
If there are any workloads running on GKE Sandbox that are untrusted, create a new node pool with the appropriate security settings. This can be done by running the following command in GCP CLI:
gcloud beta container node-pools create [NEW_NODE_POOL_NAME] --cluster=[CLUSTER_NAME] --zone=[ZONE] --sandbox=unconfined
Replace [NEW_NODE_POOL_NAME], [CLUSTER_NAME] and [ZONE] with the corresponding values for your GKE Sandbox. The --sandbox=unconfined flag will create a new node pool with unrestricted sandboxing, which is suitable for running untrusted workloads.
Migrate the workloads from the old node pool to the new node pool. This can be done by running the following command in GCP CLI:
kubectl drain [NODE_NAME] --ignore-daemonsets --delete-local-data
Replace [NODE_NAME] with the name of the node that is running the untrusted workload. This command will safely evict all the pods running on the node.
Delete the old node pool. This can be done by running the following command in GCP CLI:
gcloud beta container node-pools delete [NODE_POOL_NAME] --cluster=[CLUSTER_NAME] --zone=[ZONE]
Replace [NODE_POOL_NAME], [CLUSTER_NAME] and [ZONE] with the corresponding values for your GKE Sandbox.

By following these steps, you can remediate the misconfiguration of using GKE Sandbox for running untrusted workloads in GCP using GCP CLI.

Using Python

To remediate the misconfiguration “Consider GKE Sandbox For Running Untrusted Workloads” for GCP using Python, follow the below steps:

Install the necessary libraries:

pip install google-auth google-auth-oauthlib google-auth-httplib2 google-cloud-container

Set the project ID and cluster name for which you want to enable GKE Sandbox.
project_id = "your-project-id" cluster_name = "your-cluster-name"

Authenticate using your Google Cloud credentials:

from google.oauth2 import service_account

credentials = service_account.Credentials.from_service_account_file('path/to/your/credentials.json')

Enable GKE Sandbox for the cluster:

from google.cloud import container_v1

client = container_v1.ClusterManagerClient(credentials=credentials)

cluster = client.get_cluster(project_id, "us-central1", cluster_name)

if not cluster.sandbox_config:
    sandbox_config = container_v1.SandboxConfig(pod_sandbox_spec=container_v1.PodSandboxConfig())
    cluster.sandbox_config = sandbox_config
    update_request = container_v1.UpdateClusterRequest(cluster=cluster, update_mask={"paths": ["sandbox_config"]})
    operation = client.update_cluster(update_request)
    operation.result()

Verify that GKE Sandbox is enabled:

cluster = client.get_cluster(project_id, "us-central1", cluster_name)

if cluster.sandbox_config:
    print("GKE Sandbox is enabled for the cluster.")
else:
    print("GKE Sandbox is not enabled for the cluster.")

By following these steps, you can remediate the misconfiguration “Consider GKE Sandbox For Running Untrusted Workloads” for GCP using Python.

Using Terraform

resource "google_container_cluster" "GKE_CLUSTER" {
  name     = "GKE_CLUSTER_NAME"       # replace with your cluster name
  location = "GCP_REGION_OR_ZONE"     # e.g. "us-central1" or "us-central1-a"
  project  = "GCP_PROJECT_ID"         # replace with your GCP project ID

  # Other required cluster arguments here (networking, IP allocation, etc.)
}

# Node pool dedicated to untrusted workloads, with GKE Sandbox (gVisor) enabled
resource "google_container_node_pool" "UNTRUSTED_WORKLOAD_POOL" {
  name     = "untrusted-workloads-pool"
  project  = "GCP_PROJECT_ID"         # replace with your GCP project ID
  location = "GCP_REGION_OR_ZONE"     # must match the cluster's location
  cluster  = google_container_cluster.GKE_CLUSTER.name

  node_config {
    machine_type = "e2-medium"        # adjust as needed

    sandbox_config {
      # Enable GKE Sandbox using gVisor for this node pool
      sandbox_type = "gvisor"
    }

    # Other node_config options as needed (service_account, tags, etc.)
  }

  initial_node_count = 3              # adjust as needed
}

Enabling or changing sandbox_config.sandbox_type on an existing node pool will force recreation of the nodes in that pool (pods will be rescheduled; plan for a rolling replacement or drain as needed).After you add this configuration and assign untrusted workloads to this node pool (e.g., via node selectors/taints and a RuntimeClass using gVisor), terraform plan should show creation or update of google_container_node_pool.UNTRUSTED_WORKLOAD_POOL with a new sandbox_config block setting sandbox_type = "gvisor" and no other unexpected changes.

GCP Introduction

GCP Pricing

GCP Threats

GCP Misconfigurations

Resources

GCP Threats

K8s sandbox running untrusted remediation

Triage and Remediation

Remediation

​Triage and Remediation

​Remediation

​

Triage and Remediation

Remediation