AWS Introduction
AWS Pricing
AWS Threats
AWS Misconfigurations
- Getting Started with AWS Audit
- Permissions required for Misconfigurations Detection
- API Gateway Audit
- Cloudformation Audit
- CloudFront Audit
- CloudTrail Audit
- Cloudwatch Audit
- DynamoDB Audit
- EC2 Audit
- Elastic Search Audit
- ELB Audit
- IAM Audit
- KMS Audit
- Kubernetes Audit
- Lambda Audit
- RDS Audit
- Redshift Audit
- Route53 Audit
- S3 Audit
- Security Groups Audit
- SES Audit
- SNS Audit
- IAM Deep Dive
- App Sync Audit
- Code Build Audit
- Open Search Audit
- Shield Audit
- SQS Audit
EMR Clusters Should Be In VPC
More Info:
Ensure that your Amazon Elastic MapReduce (EMR) clusters are provisioned using the AWS EC2-VPC platform instead of EC2-Classic platform (outdated from 2013.12.04) for better flexibility and control over security, better traffic routing and availability.
Risk Level
Medium
Address
Cost Optimisation, Security
Compliance Standards
HIPAA, ISO27001
Triage and Remediation
Remediation
To remediate the misconfiguration of EMR clusters not being in a VPC for AWS Redshift using the AWS console, follow these steps:
-
Navigate to the Amazon EMR console:
- Open the AWS Management Console.
- Go to the Amazon EMR console by searching for “EMR” in the services search bar.
-
Select the EMR cluster:
- Select the EMR cluster that is not in a VPC.
-
Update the cluster settings:
- Click on the “Edit” button or navigate to the “Hardware” tab in the cluster details.
- Scroll down to the “Network and Security” section.
-
Select VPC and Subnet:
- Under the “Network and Security” section, choose the VPC where you want to place the EMR cluster.
- Select a subnet within the chosen VPC.
-
Configure security groups:
- Review and update the security groups associated with the EMR cluster to ensure proper network access controls.
-
Save the changes:
- Click on the “Save” or “Update” button to apply the changes to the EMR cluster.
-
Verify the VPC configuration:
- After saving the changes, verify that the EMR cluster is now running within the specified VPC by checking the cluster details.
By following these steps, you can remediate the misconfiguration of having EMR clusters not in a VPC for AWS Redshift using the AWS console.
To remediate the misconfiguration of EMR Clusters not being in a VPC for AWS Redshift using AWS CLI, follow these steps:
-
Create a VPC (if not already created):
Use the following AWS CLI command to create a VPC:
aws ec2 create-vpc --cidr-block 10.0.0.0/16
-
Create Subnets within the VPC:
Create at least two subnets in different Availability Zones within the VPC using the following command:
aws ec2 create-subnet --vpc-id <your-vpc-id> --cidr-block 10.0.1.0/24 --availability-zone <az1> aws ec2 create-subnet --vpc-id <your-vpc-id> --cidr-block 10.0.2.0/24 --availability-zone <az2>
-
Create an Internet Gateway (IGW) and attach it to the VPC:
aws ec2 create-internet-gateway aws ec2 attach-internet-gateway --vpc-id <your-vpc-id> --internet-gateway-id <your-igw-id>
-
Create Route Tables and associate them with the subnets:
aws ec2 create-route-table --vpc-id <your-vpc-id> aws ec2 associate-route-table --route-table-id <your-route-table-id> --subnet-id <your-subnet-id>
-
Create Security Groups for EMR Clusters:
Create a security group for EMR clusters that allows necessary inbound and outbound traffic using the following command:
aws ec2 create-security-group --group-name EMR-SG --description "Security Group for EMR Clusters" --vpc-id <your-vpc-id>
-
Launch EMR Cluster within the VPC:
Launch the EMR cluster within the VPC and specify the security group and subnets created earlier using the AWS Management Console or AWS CLI.
-
Verify the Configuration:
Use the following command to describe the EMR cluster and verify that it is running within the VPC:
aws emr describe-cluster --cluster-id <your-cluster-id>
By following these steps, you can remediate the misconfiguration of EMR Clusters not being in a VPC for AWS Redshift using AWS CLI.
To remediate the misconfiguration of EMR clusters not being in a VPC for AWS Redshift using Python, you can follow these steps:
-
Create a VPC:
- Use the AWS SDK for Python (Boto3) to create a new Virtual Private Cloud (VPC) in your AWS account.
- You can use the
create_vpc
method from theec2
client in Boto3 to create a VPC.
-
Create Subnets:
- Create one or more subnets within the VPC. Make sure the subnets are in different availability zones for high availability.
- Use the
create_subnet
method from theec2
client in Boto3 to create subnets.
-
Create an Internet Gateway (IGW):
- Attach an Internet Gateway to the VPC to allow internet access for resources within the VPC.
- Use the
create_internet_gateway
method from theec2
client in Boto3 to create an Internet Gateway.
-
Create Route Tables:
- Create route tables for the public and private subnets within the VPC.
- Use the
create_route_table
method from theec2
client in Boto3 to create route tables.
-
Associate Subnets with Route Tables:
- Associate the subnets with their respective route tables to control the routing of network traffic.
- Use the
associate_route_table
method from theec2
client in Boto3 to associate subnets with route tables.
-
Launch EMR Cluster in VPC:
- When launching an EMR cluster using Boto3, specify the
Ec2SubnetId
parameter to launch the cluster within the VPC. - Set the
Ec2SubnetId
parameter to the ID of the subnet where you want to launch the EMR cluster.
- When launching an EMR cluster using Boto3, specify the
Here is a sample Python code snippet to launch an EMR cluster within a VPC using Boto3:
import boto3
# Create a VPC
ec2 = boto3.client('ec2')
vpc = ec2.create_vpc(CidrBlock='10.0.0.0/16')
# Create a subnet
subnet = ec2.create_subnet(VpcId=vpc['Vpc']['VpcId'], CidrBlock='10.0.0.0/24', AvailabilityZone='us-east-1a')
# Create an Internet Gateway
igw = ec2.create_internet_gateway()
ec2.attach_internet_gateway(InternetGatewayId=igw['InternetGateway']['InternetGatewayId'], VpcId=vpc['Vpc']['VpcId'])
# Create a route table
route_table = ec2.create_route_table(VpcId=vpc['Vpc']['VpcId'])
# Associate subnet with route table
ec2.associate_route_table(RouteTableId=route_table['RouteTable']['RouteTableId'], SubnetId=subnet['Subnet']['SubnetId'])
# Launch EMR cluster within the VPC
emr = boto3.client('emr')
response = emr.run_job_flow(
Name='MyEMRCluster',
ReleaseLabel='emr-6.2.0',
Instances={
'Ec2SubnetId': subnet['Subnet']['SubnetId'],
'EmrManagedMasterSecurityGroup': 'sg-12345678',
'EmrManagedSlaveSecurityGroup': 'sg-12345678',
'InstanceGroups': [
{
'InstanceRole': 'MASTER',
'InstanceType': 'm5.xlarge',
'InstanceCount': 1
}
]
},
VisibleToAllUsers=True
)
Make sure to replace placeholder values like sg-12345678
with actual security group IDs and customize the code as per your specific requirements.