I am trying to launch a jupyterlab instance using cloudformation (its something I do a lot and sagemaker does not have a 1y free tier) so the beginning looks like this which does not work. Specifically the password parameter
# AWSTemplateFormatVersion: "2010-09-09"
Description: Creates a Jupyter Lab Instance with an Elastic Load Balancer
Parameters:
KeyName:
Description: >-
Name of an existing EC2 KeyPair to enable SSH access to the instance
Type: AWS::EC2::KeyPair::KeyName
ConstraintDescription: Must be the name of an existing EC2 KeyPair.
Default: eduinstance
VPC:
Description: VPC ID of the VPC in which to deploy this stack.
Type: AWS::EC2::VPC::Id
ConstraintDescription: Must be the name of a valid VPC
Default: vpc-10a7ac6a
Subnets:
Type: List<AWS::EC2::Subnet::Id>
Default: subnet-8cde25d3,subnet-531fda72,subnet-4bbe3006
Description: >-
Subnets for the Elastic Load Balancer.
Please include at least two subnets
Password:
Type: String
NoEcho: false
MinLength: 4
Default: '{{resolve:ssm:JLabPassword:1}}'
Description: Password to set for Jupyter Lab
EBSVolumeSize:
Type: Number
Description: EBS Volume Size (in GiB) for the instance
Default: 8
MinValue: 8
MaxValue: 64000
ConstraintDescription: Please enter a value between 8 GB and 64 TB
EC2InstanceType:
Type: String
Default: t2.micro
AllowedValues:
- t2.micro
- c5.large
- m5.large
Description: Enter t2.micro, c5.large or m5.large. Default is t2.micro.
Conditions:
JupyterPasswordDefault: !Equals
- !Ref Password
- DEFAULT
Resources:
ALB:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
IpAddressType: ipv4
Scheme: internet-facing
SecurityGroups:
- !GetAtt [ALBSG, GroupId]
Subnets: !Ref Subnets
Type: application
ALBListener:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
DefaultActions:
- Type: forward
TargetGroupArn: !Ref ALBTargetGroup
LoadBalancerArn: !Ref ALB
Port: 80
Protocol: HTTP
ALBTargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Port: 8888
Protocol: HTTP
Targets:
- Id: !Ref ComputeInstance
TargetType: instance
VpcId: !Ref VPC
ComputeInstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Roles:
- !Ref ComputeIAMRole
ComputeInstance:
Type: AWS::EC2::Instance
Properties:
InstanceType: t2.micro
SubnetId: !Select [0, !Ref Subnets]
KeyName: !Ref KeyName
ImageId: '{{resolve:ssm:/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2:33}}'
SecurityGroupIds:
- !GetAtt [ComputeSG, GroupId]
IamInstanceProfile: !Ref ComputeInstanceProfile
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
VolumeType: gp2
VolumeSize: !Ref EBSVolumeSize
DeleteOnTermination: true
UserData:
Fn::Base64: !Sub
- |
#!/bin/bash
yum update -y
yum install python3-pip -y
yum install java-1.8.0-openjdk -y
cd /home/ec2-user/
wget https://repo.anaconda.com/archive/Anaconda3-2020.11-Linux-x86_64.sh
sudo -u ec2-user bash Anaconda3-2020.11-Linux-x86_64.sh -b -p /home/ec2-user/anaconda
echo "PATH=/home/ec2-user/anaconda/bin:$PATH" >> /etc/environment
source /etc/environment
jupyter notebook --generate-config
mkdir .jupyter
cp /root/.jupyter/jupyter_notebook_config.py /home/ec2-user/.jupyter/
echo "c = get_config()" >> .jupyter/jupyter_notebook_config.py
echo "c.NotebookApp.ip = '*'" >> .jupyter/jupyter_notebook_config.py
NB_PASSWORD=$(python3 -c "from notebook.auth import passwd; print(passwd('${password}'))")
echo "c.NotebookApp.password = u'$NB_PASSWORD'" >> .jupyter/jupyter_notebook_config.py
rm Anaconda3-2020.11-Linux-x86_64.sh
mkdir Notebooks
chmod 777 -R Notebooks .jupyter
su -c "jupyter lab" -s /bin/sh ec2-user
- password: !Ref Password #!If [JupyterPasswordDefault, '{{resolve:ssm:JupyterLabPassword:1}}', !Ref Password]
ALBSG:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Security Group for JupyterLab ALB. Created Automatically.
SecurityGroupIngress:
- CidrIp: 0.0.0.0/0
Description: Allows HTTP Traffic from anywhere
FromPort: 80
ToPort: 80
IpProtocol: tcp
ComputeSG:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Security Group for JupyterLab EC2 Instance. Created Automatically.
SecurityGroupIngress:
- Description: Allows JupyterLab Server Traffic from ALB.
FromPort: 8888
IpProtocol: tcp
SourceSecurityGroupId: !GetAtt [ALBSG, GroupId]
ToPort: 8890
- CidrIp: 0.0.0.0/0
Description: Allows SSH Access from Anywhere
FromPort: 22
ToPort: 22
IpProtocol: tcp
ComputeIAMRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- 'sts:AssumeRole'
Description: Allows EC2 Access to S3. Created Automatically.
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonS3FullAccess
- arn:aws:iam::aws:policy/AmazonSageMakerFullAccess
Outputs:
URL:
Description: URL of the ALB
Value: !Join
- ''
- - 'http://'
- !GetAtt
- ALB
- DNSName
ConnectionString:
Description: Connection String For SSH On EC2
Value: !Join
- ''
- - 'ssh -i "'
- !Ref KeyName
- '.pem" ec2-user#'
- !GetAtt
- ComputeInstance
- PublicDnsName
It however interprets the string literally so I don't actually get my password but the resolve... itself.
Based on the comments and new, updated template by OP, and to expand on #DennisTraub answer.
SSM parameters resolve in almost all cases in the template, with the exception of UserData (btw, Init will also not work). This means, that dynamic reference will not resolve when used in the context of UserData. This is due to security issues.
UserData can be read in plain text by anyone who can view basic attributes of the instance. This means, that your JLabPassword would be in plain text available in UserData for everyone to see, if such resolution would be possible.
To rectify the issue, the SSM parameters should be used in UserData as follows:
Attach IAM permission ssm:GetParameter to the instance role/profile which allows instance to access the SSM Parameter Store.
Instead on {{resolve:ssm:JLabPassword:1}} in your Parameter, you can just pass JLabPassword so that the name of the SSM paramtter gets passed into the UserData, not the actual value of it.
In the UserData, please use AWS CLI get-parameter to get the actual value of your JLabPassword.
The above ensures that the value of JLabPassword is kept private and not visible in plain text in UserData.
Your passwort parameter's default value is missing the service name (ssm) as well as single quotes.
// What you have:
Password:
Default: {{resolve:JupyterPassword:1}}
...
// What it should be:
Password:
Default: '{{resolve:ssm:JupyterPassword:1}}'
...
Update: You've fixed the code in your question. Did my answer and the comments below solve your question? If not, I'm not sure what else you need.
Related
I have a cloudformation template to create an ec2-instance. That template also starts an httpd along with some content that is served.
I'm using the Parameter section to allow a key to be specified or selected - see snippet below:
Parameters:
paramKeyPair:
Description: KeyPairName
Type: AWS::EC2::KeyPair::KeyName
I'm calling the ec2-instance through the AWS CLI like this :
aws cloudformation create-stack --stack-name stack-ec2instance --template-body file://demo-ec2instance --parameters ParameterKey=paramKeyPair,ParameterValue=peterKeyPair
So the instance can be created and the keypair can be passed through as an argument - BUT - frankly I don't actually care that much if the instance can be access. It's just a web server that can be spun up or down. SSH access is nice but no big deal.
In fact, if I removed the keypair Parameter from the cloudformation template - and removed the associated reference in the AWS CLI call - Cloudformation will happily spin up the instance without a keypair. Great !
What I would really like is for cloudformation to deal with the keypair being present or not.
I thought the best way to do this would be to update the code so that the parameter has a default value of "None" (for example) and then the ec2-instance could be run from the AWS CLI and if the keypair parameter is not specified then AWS would know not to bother with the keypair at all.
The problem is that by specifying the Type as AWS::EC2::KeyPair::KeyName, the AWS CLI expects an actual value.
I'm out of ideas - if anyone else has figured this out - I would really appreciate it.
Thankyou
Peter.
If I understand you correctly you want to be able to keep the parameter in your Cloudformation template, but only "allocate" a key pair to an instance if you specify a value, otherwise don't allocate a key pair to the ec2 instance resource. You can do this with AWS::NoValue pseudo parameter.
Here is a sample template:
Description: My EC2 instance
Parameters:
SSHKeyName:
Type: String
Conditions:
Has-EC2-Key:
!Not [ !Equals [ !Ref SSHKeyName, '' ] ]
Resources:
Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: <InstanceImageID>
InstanceType: t2.micro
KeyName: !Ref SSHKeyName
KeyName:
Fn::If:
- Has-EC2-Key
- Ref: SSHKeyName
- Ref: AWS::NoValue
<other properties as required
So what this does is the condition checks if a SSHKeyName value is blank, if it's blank then the KeyName property will be ignored, if it isn't blank then it will use the value of SSHKeyName.
Thankyou WarrenG, your solution worked with one small exception which was to change the parameter type from AWS::EC2::KeyPair::KeyName to String.
Without your help I am certain I would have burned many more hours on this.
So in conclusion, the fix was
1: Change the Parameter type to String.
Parameters:
SSHKeyName:
Type: String
2: Add a function that determines if the key is present.
Conditions:
Has-EC2-Key:
!Not [ !Equals [ !Ref SSHKeyName, '' ] ]
Use the function within the resources section.
KeyName:
Fn::If:
- Has-EC2-Key
- Ref: SSHKeyName
- Ref: AWS::NoValue
Within my question I kept the code snippets to a minimum for readability but now I have marked this as solved I'm adding two blocks of code just for documentation and incase this helps anyone else.
One example of calling the template through the AWS CLI.
aws cloudformation create-stack --stack-name stack-ec2instance --template-body file://demo-ec2instance --parameters ParameterKey=paramSubnetId,ParameterValue=$SubnetId ParameterKey=paramKeyPair,ParameterValue=peterKeyPair ParameterKey=paramSecurityGroupIds,ParameterValue=$SecurityGroupId
The template to create a EC2 instance.
AWSTemplateFormatVersion: 2010-09-09
Parameters:
SSHKeyName:
Description: EC2 KeyPair for SSH access.
Type: String
Conditions:
Has-EC2-Key:
!Not [ !Equals [ !Ref SSHKeyName, '' ] ]
Mappings:
RegionMap:
eu-west-1:
AMI: ami-3bfab942
eu-west-2:
AMI: ami-098828924dc89ea4a
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Metadata:
AWS::CloudFormation::Init:
config:
packages:
yum:
httpd: []
php: []
files:
/var/www/html/index.php:
content: !Sub |
<?php print "Hello Peter !"; ?>
services:
sysvinit:
httpd:
enabled: true
ensureRunning: true
Properties:
InstanceType: t2.micro
ImageId:
Fn::FindInMap:
- RegionMap
- !Ref AWS::Region
- AMI
SecurityGroupIds:
- !Ref MySecurityGroup
KeyName:
Fn::If:
- Has-EC2-Key
- Ref: SSHKeyName
- Ref: AWS::NoValue
UserData:
'Fn::Base64':
!Sub |
#!/bin/bash -xe
# Ensure AWS CFN Bootstrap is the latest
yum install -y aws-cfn-bootstrap
# Install the files and packages from the metadata
/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource EC2Instance --region ${AWS::Region}
MySecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Open Ports 22 and 80
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: '80'
ToPort: '80'
CidrIp: 0.0.0.0/0
Outputs:
Website:
Description: The Public DNS for the EC2 Instance
Value: !Sub 'http://${EC2Instance.PublicDnsName}'
I am trying to create an AWS infrastructure consisting of 2 security groups and 1 EC2 instance. The instance creation fails with an error:
Security group sg-0ca713960ef97b70b and subnet
subnet-0fb1a03979897974d belong to different networks. (Service:
AmazonEC2; Status Code: 400; Error Code: InvalidParameter; Request ID:
5f03e0f1-fc1b-4ab4-8bef-0d71a1756212; Proxy: null)
I am using VPC value as input in the template. Not sure what I am missing here. Can someone help me solve this issue. Below is my complete template:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Template for immediately isolation and forensic investigation of compromised instances
Parameters:
VpcId:
Type: AWS::EC2::VPC::Id
Ec2KeyName:
Type: AWS::EC2::KeyPair::KeyName
PurposeTag:
Type: String
Default: forensics
SSHLocation:
Description: >-
Enter desired Network CIDR to access EC2 instance. Default is set to
access from anywhere and it is not recommended. Please change to appropriate
CIDR.
AllowedPattern: '(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2})'
MinLength: '9'
MaxLength: '18'
Default: 0.0.0.0/0
Type: String
ConstraintDescription: >-
Must be a valid Network CIDR of the form x.x.x.x/y. Default is set to
0.0.0.0/0, in production do not set default to 0.0.0.0/0
Mappings:
ImageId:
us-east-1:
AmazonLinux2: ami-00dc79254d0461090
UbuntuCanonical: ami-04b9e92b5572fa0d1
us-east-2:
AmazonLinux2: ami-00bf61217e296b409
UbuntuCanonical: ami-0d5d9d301c853a04a
us-west-1:
AmazonLinux2: ami-024c80694b5b3e51a
UbuntuCanonical: ami-0dd655843c87b6930
us-west-2:
AmazonLinux2: ami-0a85857bfc5345c38
UbuntuCanonical: ami-06d51e91cea0dac8d
eu-west-1:
AmazonLinux2: ami-040ba9174949f6de4
UbuntuCanonical: ami-02df9ea15c1778c9c
Resources:
ForensicSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Security group for forensic EC2 instances
SecurityGroupIngress:
- Description: Allow SSH from company ip address
CidrIp: !Ref SSHLocation
IpProtocol: tcp
FromPort: 22
ToPort: 22
Tags:
- Key: Purpose
Value: !Ref PurposeTag
VpcId: !Ref VpcId
ForensicInstance:
Type: AWS::EC2::Instance
Metadata:
AWS::CloudFormation::Init:
config:
commands:
1_sift_download:
command: "wget https://github.com/teamdfir/sift-cli/releases/download/v1.7.1/sift-cli-linux -P /tmp/"
2_rename:
command: "mv /tmp/sift-cli-linux /usr/local/bin/sift"
3_sift_permissions:
command: "chmod 755 /usr/local/bin/sift"
4_sift_install:
command: "/usr/local/bin/sift install"
Properties:
IamInstanceProfile: !Ref ForensicInstanceProfile
ImageId: !FindInMap
- ImageId
- !Ref 'AWS::Region'
- UbuntuCanonical
InstanceType: t2.micro
KeyName: !Ref Ec2KeyName
SecurityGroupIds:
- !GetAtt ForensicSecurityGroup.GroupId
UserData:
Fn::Base64: !Sub |
#!/bin/bash
sudo su
apt update
apt upgrade
apt -y install python-pip pcre-tools gcc autoconf automake libtool nc git kernel-devel libdwarf-tools python unzip
pip install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz
cfn-init -s ${AWS::StackName} --region ${AWS::Region} -r ForensicInstance
pip install distorm3 pycrypto pillow openpyxl ujson pytz IPython netaddr yara-python pylzma psutil colorama
cd /home/ubuntu
wget http://downloads.volatilityfoundation.org/releases/2.6/volatility-2.6.zip
unzip volatility-2.6.zip
mv volatility-master volatility
chown -R ubuntu.ubuntu volatility
# Install LiME
git clone https://github.com/504ensicsLabs/LiME.git
chown -R ubuntu.ubuntu LiME
# Install Loki
wget https://github.com/Neo23x0/Loki/archive/v0.30.5.tar.gz
tar -xzvf v0.30.5.tar.gz
cd Loki-0.30.5/
pip install -r requirements.txt
# Install aws_ir
pip install aws_ir
Tags:
- Key: Purpose
Value: !Ref PurposeTag
ForensicInstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Roles:
- !Ref ForensicInstanceRole
ForensicInstanceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
Service: ec2.amazonaws.com
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonEC2FullAccess
IsolatedSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Security group that allows only SSH from the forensics group
SecurityGroupIngress:
- Description: Allow SSH
SourceSecurityGroupId: !GetAtt ForensicSecurityGroup.GroupId
IpProtocol: tcp
FromPort: 22
ToPort: 22
SecurityGroupEgress:
- Description: Limit outbound traffic to only localhost, removes the default quad-zero outbound rule
CidrIp: 127.0.0.1/32
IpProtocol: '-1'
VpcId: !Ref VpcId
Tags:
- Key: Purpose
Value: !Ref PurposeTag
The AWS::EC2::Instance does not have its Subnet property specified or it does not a network interface explicitly attached to it. Because of this the EC2 instances is most likely provisioned in the default VPC in a random subnet, meanwhile the security group attached to it is created in another VPC.
I have 2 EC2 instances in 2 public subnets. The EC2 instances are fronted my an Application Load Balancer in the same public subnet as the EC2 instances. The Security groups for the EC2's is set to only accept tcp traffic from the security group the load balancer is in.
I am getting a 502 when I hit the Application Load Balancers endpoint.
I am deploying using CloudFormation. Here is the relevant bit of code.
AWSTemplateFormatVersion: "2010-09-09"
Description: Deploy a 3-tier wordpress system. (Plublic and Private subnets and DB on RDS)
Parameters:
VpcId:
Description: VPC id
Type: String
Default: vpc-0b6a616f830dd7d5a
PublicSubnetA:
Description: Subnet Id where instance will create
Type: String
Default: subnet-0616a6183bee2b276
PrivateSubnetA:
Description: Subnet Id where instance will create
Type: String
Default: subnet-06784a19612a64444
PublicSubnetB:
Description: Subnet Id where instance will create
Type: String
Default: subnet-04f7e39ac1431f22a
PrivateSubnetB:
Description: Subnet Id where instance will create
Type: String
Default: subnet-0fa6aa79eaee582bf
EC2KeyName:
Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
Type: AWS::EC2::KeyPair::KeyName
Default: test
ConstraintDescription: must be the name of an existing EC2 KeyPair.
EC2InstanceType:
Description: EC2 instance type
Type: String
Default: t2.micro
ConstraintDescription: must be a valid EC2 instance type.
WebServerInstanceAMI:
Description: EC2 instance type
Type: AWS::EC2::Image::Id
Default: ami-0210560cedcb09f07
ConstraintDescription: must be an existing AMI ID.
SSHLocation:
Description: The IP address range that can be used to SSH to the EC2 instances
Type: String
MinLength: 9
MaxLength: 18
Default: 0.0.0.0/0
AllowedPattern: (\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2})
ConstraintDescription: must be a valid IP CIDR range of the form x.x.x.x/x.
Application:
Description: Application Name
Type: String
AllowedPattern: "[A-Za-z0-9-]+"
Default: test
Environment:
AllowedValues: [preprod,prod]
Default: preprod
Description: The name of the Environment
Type: String
Resources:
LoadBalancerSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: !Ref VpcId
GroupDescription: ELB Security Group
SecurityGroupIngress:
- FromPort: 80
IpProtocol: tcp
CidrIp: 0.0.0.0/0
ToPort: 80
Description: Allow from internet
Tags:
- Key: Name
Value: !Sub '${Application}-loadbalancer-sg'
- Key: Project
Value: !Ref Application
WebServerSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: 'SSH and Port 80'
VpcId:
Ref: VpcId
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: !Ref 'SSHLocation'
- IpProtocol: tcp
FromPort: 80
ToPort: 80
SourceSecurityGroupId:
Ref: LoadBalancerSecurityGroup
Tags:
- Key: Name
Value: !Sub '${Application}-webserver-sg'
- Key: Project
Value: !Ref Application
LoadBalancer:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
Name: ApplicationLoadBalancer
Scheme: internet-facing
Subnets:
- !Ref PublicSubnetA
- !Ref PublicSubnetB
SecurityGroups:
- !Ref LoadBalancerSecurityGroup
LoadBalancerListener:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
LoadBalancerArn: !Ref LoadBalancer
Port: 80
Protocol: HTTP
DefaultActions:
- Type: forward
TargetGroupArn: !Ref ApplicationTargetGroup
ApplicationTargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
HealthCheckIntervalSeconds: 30
HealthCheckProtocol: HTTP
HealthCheckTimeoutSeconds: 15
HealthyThresholdCount: 3
UnhealthyThresholdCount: 3
HealthCheckPath: /index.html
Matcher:
HttpCode: '200'
Name: ApplicationTargetGroup
VpcId: !Ref VpcId
Port: 80
Protocol: HTTP
TargetGroupAttributes:
- Key: deregistration_delay.timeout_seconds
Value: '20'
Targets:
- Id: !Ref WebServerInstance1
Port: 80
- Id: !Ref WebServerInstance2
Port: 80
WebServerInstance1:
Type: AWS::EC2::Instance
Properties:
InstanceType: !Ref EC2InstanceType
KeyName: !Ref EC2KeyName
SubnetId: !Ref PublicSubnetA
SecurityGroupIds:
- !Ref WebServerSecurityGroup
ImageId: !Ref WebServerInstanceAMI
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
cd /tmp
sudo yum update -y
sudo yum install -y httpd
echo "Welcome from the instance 1" > /var/www/html/index.html
sudo -u root service httpd start
WebServerInstance2:
Type: AWS::EC2::Instance
Properties:
InstanceType: !Ref EC2InstanceType
KeyName: !Ref EC2KeyName
SubnetId: !Ref PublicSubnetB
SecurityGroupIds:
- !Ref WebServerSecurityGroup
ImageId: !Ref WebServerInstanceAMI
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
cd /tmp
sudo yum update -y
sudo yum install -y httpd
echo "Welcome from the instance 2" > /var/www/html/index.html
sudo -u root service httpd start
Outputs:
LoadBalancerDnsName:
Description: Load Balancer public facing DNS
Export:
Name: !Sub ${AWS::StackName}-LoadBaancer
Value: !GetAtt LoadBalancer.DNSName
I have a look at the resources deployed in the console UI and I can see the correct security group rules on the EC2 instances. I don't see when the ALB would have trouble sending messages to the EC2 instances.
Question: Why am I getting a 502 error when I hit the ALBs endpoint?
I deployed your template in my VPC. The template is perfectly fine and works without any issues, including your load balancer and the website.
Thus whatever is causing your issues, is outside of this template. Probably VPC definition is incorrect, but it is not showed. You can make new question with details of your VPC setup if you want.
I am currently doing the Cloud Architecting course from Amazon. I am stuck on Module 10 Challenge Lab - Automating Infrastructure Deployment.
The challenges are:
Create a static website for the cafe by using AWS CloudFormation.
Store templates in a version control system.
Use a continuous delivery service, create the network and application layers for the cafe.
Define an EC2 instance resource and creating the application stack.
I am up to challenge 4 and not sure how to create a new EC2 with the following resource:
Set the Logical ID to CafeInstance (see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resources-section-structure.html for reference, if needed)
Include an ImageId that references the LatestAmiId parameter
For InstanceType, reference the instance type parameter that you defined in the previous step.
For KeyName, use the following line of code, which references the RegionMap mapping that is already defined in the template:
KeyName: !FindInMap [RegionMap, !Ref "AWS::Region", keypair]
For the InstanceProfile (the AWS Identity and Access Management, or IAM, role that is attached to the instance), specify CafeRole
Note: The CafeRole IAM role already exists in your account. Attaching it grants your EC2 instance the permissions to retrieve Parameter Store values from AWS Systems Manager.
In the Properties section, include the following lines of code:
NetworkInterfaces:
- DeviceIndex: '0'
AssociatePublicIpAddress: 'true'
SubnetId: !ImportValue
'Fn::Sub': '${CafeNetworkParameter}-SubnetID'
GroupSet:
- !Ref CafeSG
Analysis: The previous lines help ensure that your instance deploys to the Public Subnet that you created when you ran the café network stack. Recall that at the beginning of this task, you updated the network stack to define outputs with export names. In the preceding code, you import the value for the SubnetId. The preceding code also helps ensure that the instance you create will be in the CafeSG security group that is already defined for you in this template.
Set a tag with a key of Name and a value of Cafe Web Server
Tip: Observe how a Name tag was applied to the security group resource that is already defined in the template.
In the Properties section, include the following additional UserData code:
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
yum -y update
yum install -y httpd mariadb-server wget
amazon-linux-extras install -y lamp-mariadb10.2-php7.2 php7.2
systemctl enable httpd
systemctl start httpd
systemctl enable mariadb
systemctl start mariadb
wget https://aws-tc-largeobjects.s3-us-west-2.amazonaws.com/ILT-TF-200-ACACAD-20-EN/mod10-challenge/cafe-app.sh
chmod +x cafe-app.sh
./cafe-app.sh
Analysis: The previous code runs on the instance at the end of the boot process. It installs an Apache HTTP web server, a MariaDB database, and PHP on the Amazon Linux instance. Next, it starts the web server and the database. Then, it downloads a script named cafe-app.sh and runs it. The cafe-app script configures the database and installs the PHP code that makes the café website function.
After you are satisfied with your template updates, save the changes. To validate the template format in the Bash terminal, run the following command:
aws cloudformation validate-template --template-body file:///home/ec2-user/environment/CFTemplatesRepo/templates/cafe-app.yaml
If you receive a JSON-formatted response that includes the three parameters that were defined at the top of your template, then your template passed the validation. However, if you received a ValidationError response (or some other error response), you must correct the issue. Then, save the changes and run the validate-template command again.
If your template passed the validation check, add the file to "CodeCommit". In the Bash terminal, run git commands to add the file, commit it, and push it to the repository.
I have tried to do it, but keep getting an error "An error occurred (ValidationError) when calling the ValidateTemplate operation: Template format error: YAML not well-formed. (line 52, column 9)". Here is what I have written so far for cafe-app.yaml:
AWSTemplateFormatVersion: 2010-09-09
Description: Cafe application
Parameters:
InstanceTypeParameter:
Type: String
Default: t2.small
AllowedValues:
- t2.micro
- t2.small
- t3.micro
- t3.small
Description: Enter t2.micro, t2.small, t3.micro, or t3.small. Default is t2.small.
LatestAmiId:
Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
CafeNetworkParameter:
Type: String
Default: update-cafe-network
Mappings:
RegionMap:
us-east-1:
"keypair": "vockey"
us-west-2:
"keypair": "cafe-oregon"
Resources:
CafeSG:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupDescription: Enable SSH, HTTP access
VpcId: !ImportValue
'Fn::Sub': '${CafeNetworkParameter}-VpcID'
Tags:
- Key: Name
Value: CafeSG
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '80'
ToPort: '80'
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: 0.0.0.0/0
Logical ID: CafeInstance
Type: 'AWS::E2::Instance'
Properties:
ImageId: !Ref LatestAmiId
InstanceType: !Ref InstanceTypeParameter
KeyName: !FindInMap [RegionMap, !Ref "AWS::Region", keypair]
IamInstanceProfile: CafeRole
NetworkInterfaces:
- DeviceIndex: '0'
AssociatePublicIpAddress:
'true'
SubnetId: !ImportValue
'Fn::Sub': '${CafeNetworkParameter}-SubnetID'
GroupSet:
- !Ref CafeSG
Tags:
- Key: Name
- Value: Cafe Web Server
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
yum -y update
yum install -y httpd mariadb-server wget
amazon-linux-extras install -y lamp-mariadb10.2-php7.2 php7.2
systemctl enable httpd
systemctl start httpd
systemctl enable mariadb
systemctl start mariadb
wget https://aws-tc-largeobjects.s3-us-west-2.amazonaws.com/ILT-TF-200-ACACAD-20-EN/mod10-challenge/cafe-app.sh
chmod +x cafe-app.sh
./cafe-app.sh
Outputs:
WebServerPublicIP:
Value: !GetAtt 'CafeInstance.PublicIp'
I really appreciate all the help that I can get. Thank you :)
Your template has many issues, such as
incorrect syntax
missing instance profile
I fixed them, and it deploys now. I don't know if UserData works or not, but at least the syntax is correct now:
AWSTemplateFormatVersion: 2010-09-09
Description: Cafe application
Parameters:
InstanceTypeParameter:
Type: String
Default: t2.small
AllowedValues:
- t2.micro
- t2.small
- t3.micro
- t3.small
Description: Enter t2.micro, t2.small, t3.micro, or t3.small. Default is t2.small.
LatestAmiId:
Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
CafeNetworkParameter:
Type: String
Default: update-cafe-network
Mappings:
RegionMap:
us-east-1:
"keypair": "vockey"
us-west-2:
"keypair": "cafe-oregon"
Resources:
CafeSG:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupDescription: Enable SSH, HTTP access
VpcId: !ImportValue
'Fn::Sub': '${CafeNetworkParameter}-VpcID'
Tags:
- Key: Name
Value: CafeSG
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '80'
ToPort: '80'
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: 0.0.0.0/0
CafeInstance:
Type: 'AWS::EC2::Instance'
Properties:
ImageId: !Ref LatestAmiId
InstanceType: !Ref InstanceTypeParameter
KeyName: !FindInMap [RegionMap, !Ref "AWS::Region", keypair]
#IamInstanceProfile: CafeRole
NetworkInterfaces:
- DeviceIndex: '0'
AssociatePublicIpAddress: true
SubnetId: !ImportValue
'Fn::Sub': '${CafeNetworkParameter}-SubnetID'
GroupSet:
- !Ref CafeSG
Tags:
- Key: Name
Value: Cafe Web Server
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
yum -y update
yum install -y httpd mariadb-server wget
amazon-linux-extras install -y lamp-mariadb10.2-php7.2 php7.2
systemctl enable httpd
systemctl start httpd
systemctl enable mariadb
systemctl start mariadb
wget https://aws-tc-largeobjects.s3-us-west-2.amazonaws.com/ILT-TF-200-ACACAD-20-EN/mod10-challenge/cafe-app.sh
chmod +x cafe-app.sh
./cafe-app.sh
Outputs:
WebServerPublicIP:
Value: !GetAtt 'CafeInstance.PublicIp'
I have been attempting to set up a cloud formation script to create a VPC hosting fragate containers and a aurora DB. When attempting to deploy my aurora script I receive the following.
The DB instance and EC2 security group are in different VPCs. The DB instance is in vpc-f0ec9d98 and the EC2 security group is in vpc-01c5e9bcdb87dc39c (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterCombination; Request ID: 7aa14530-d73c-4b27-a6d6-fcc8aea61d93)
I do not understand why this is the case as I am using the same security group created by my VPC script, my aurora script is as follows
Aurora
Description: Set up a serverles PostgreSQL cluster with a bastion host (using Aurora)
Parameters:
DatabaseName:
Type: String
EngineVersion:
Type: String
Default: 11.4
MasterUsername:
Type: String
Default: root
MasterUserPassword:
Type: String
Default: root
NoEcho: true
VpcId:
Type: AWS::EC2::VPC::Id
VpcSecurityGroupId:
Type: AWS::EC2::SecurityGroup::Id
BastionImageId:
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
Default: /aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-ebs
BastionKeyName:
Type: AWS::EC2::KeyPair::KeyName
Description: EC2 key used to connect to the bastion host
DeletionProtection:
Type: String
Default: false
AllowedValues:
- true
- false
Resources:
Cluster:
Type: AWS::RDS::DBCluster
Properties:
Engine: aurora-postgresql
EngineVersion: !Ref EngineVersion
DatabaseName: !Ref DatabaseName
MasterUsername: !Ref MasterUsername
MasterUserPassword: !Ref MasterUserPassword
DBClusterIdentifier: !Ref AWS::StackName
BackupRetentionPeriod: 35
DeletionProtection: !Ref DeletionProtection
VpcSecurityGroupIds:
- !Ref VpcSecurityGroupId
BastionSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: !Sub Bastion for ${AWS::StackName}
SecurityGroupEgress:
- CidrIp: 0.0.0.0/0
FromPort: -1
ToPort: -1
IpProtocol: -1
- DestinationSecurityGroupId: !Ref VpcSecurityGroupId
IpProtocol: tcp
FromPort: 3306
ToPort: 3306
SecurityGroupIngress: []
VpcId: !Ref VpcId
Bastion:
Type: AWS::EC2::Instance
Properties:
DisableApiTermination: true
ImageId: !Ref BastionImageId
InstanceType: t2.nano
KeyName: !Ref BastionKeyName
Monitoring: false
SecurityGroupIds:
- !Ref VpcSecurityGroupId
- !Ref BastionSecurityGroup
UserData: !Base64 'yum install postgresql --assumeyes' # if this script does not work this line broke it
Outputs:
Host:
Value: !GetAtt Cluster.Endpoint.Address
Export:
Name: !Sub ${AWS::StackName}Host
Name:
Value: !Ref DatabaseName
Export:
Name: !Sub ${AWS::StackName}Name
BastionHost:
Value: !GetAtt Bastion.PublicDnsName
Export:
Name: !Sub ${AWS::StackName}BastionHost
BastionIp:
Value: !GetAtt Bastion.PublicIp
Export:
Name: !Sub ${AWS::StackName}BastionIp
BastionSecurityGroupId:
Value: !GetAtt BastionSecurityGroup.GroupId
Export:
Name: !Sub ${AWS::StackName}BastionSecurityGroupId
Without the inclusion of the DBSubnetGroupName property in the AWS::RDS::DBCluster resource, it looks like CloudFormation is attempting to launch the cluster in the default VPC. A DB subnet group allows you to specify a particular VPC when you create DB instances.
Try adding this property and referencing an associated subnet parameter/resource and the issue should be resolved.
Information about creating RDS instances within a VPC can be found in the RDS User Guide.