I am trying to combine the Cloud formation template multi-tier-web-app-in-vpc.template with the cloudformation template used by viusal studio to create Load Balanced instances. The goal is to create 2 application servers within a private subnet of a VPC. The template works fine but when I start plugging in windows instances they just fail.
Error Message
CREATE_FAILED WaitCondition timed out. Received 0 conditions when expecting 1
The following resource(s) failed to create: [FrontendWaitCondition]. . Rollback requested by user.
Template used to create the cloud formation
https://s3.amazonaws.com/hg-optimise/Windows-Multi-Tier.template
I am trying to use the following Amazon templates as guides.
Amazon Visual Studio Template
https://s3.amazonaws.com/hg-optimise/Visual-Studio.template
Amazon Multi-tier Web Sample - http://aws.amazon.com/cloudformation/aws-cloudformation-templates/
https://s3.amazonaws.com/cloudformation-templates-us-east-1/multi-tier-web-app-in-vpc.template
It looks like you are taking on too much, trying to get everything working all at once. I would try to take it one step at a time. Create a template that gets one instance up, then add auto scaling, then load balancer, then subnet, routing, etc. The problem that presents itself now is likely because you have not signaled success for the wait condition.
Below is the Properties section of an Instance resource. This snipet was taken from an AWS documentation page. Note that the "UserData" section has a call to cfn-init.exe in order to perform the actions specified in the Instance's Cloud Formation section, and has a call to cfn-signal.exe to signal to the WaitCondition that the instance is up.
"Properties": {
"InstanceType" : { "Ref" : "InstanceType" },
"ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" },
{ "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" }, "Arch" ] } ] },
"SecurityGroups" : [ {"Ref" : "SharePointFoundationSecurityGroup"} ],
"KeyName" : { "Ref" : "KeyPairName" },
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"<script>\n",
"cfn-init.exe -v -s ", { "Ref" : "AWS::StackName" },
" -r SharePointFoundation",
" --region ", { "Ref" : "AWS::Region" }, "\n",
"cfn-signal.exe -e %ERRORLEVEL% ", { "Fn::Base64" : { "Ref" : "SharePointFoundationWaitHandle" }}, "\n",
"</script>"
]]}}
}
You have set the front end wait condition to basically wait until your FrontendFleet is up and running.
You should set a desired capacity for your front end fleet.
When you get this error, what is the state of your autoscaling group FrontendFleet? If this is still bringing up instances, then your timeout is simply to short.
I honestly wouldn't worry about the waitcondtions unless you really need them.
Related
I'm trying to use an existing role (present in the AWS account) in a cloudformation template to setup a lambda function, i plan to be use this across multiple AWS accounts.
In the CF template, I'm using Parameters to set the name of the Role and then using Ref in the Role property for the Lambda function. This is what my template looks like,
"Parameters" : {
"ExistingRoleName" : {
"Type" : "String",
"Default" : "MyCustomRole"
}
"Resources" : {
"CustomLambdaFunction" : {
"Type" : "AWS::Lambda::Function",
"Properties" : {
"MemorySize" : "128",
"Role" : { "Ref" : "ExistingRoleName" },
}
},
...
However, the CF template fails with the following error :
Properties validation failed for resource CustomLambdaFunction with message: #/Role: failed validation constraint for keyword [pattern]
Is this because Lambda resource in Cloudformation needs the role arn instead of RoleName as i seen in this docaws-resource-lambda-function
Based on which i updated the CF like so,
"Resources" : {
"CustomLambdaFunction" : {
"Type" : "AWS::Lambda::Function",
"Properties" : {
"MemorySize" : "128",
"Role" : "arn:aws:iam::AccountID:role/MyCustomRole",
}
},
However, i still see the same error.
Properties validation failed for resource CustomLambdaFunction with message: #/Role: failed validation constraint for keyword [pattern]
I was wondering if i'm missing something here ?
The Ref of an IAM Role “returns the resource name”, not its ARN. But you can use GetAtt on the Arn attribute of the role instead.
In JSON:
{"Fn::GetAtt": ["MyRole", "Arn"]}
In YAML:
!GetAtt MyRole.Arn
Format to reference the iam role arn
"Role" : { "Fn::Sub" : "arn:aws:iam::${AWS::AccountId}:role/MyCustomRole" }
In yaml if you are pointing to an already existing role the syntax is:
function:
...
role: !Sub arn:aws:iam::${AWS::AccountId}:role/MyRoleName
Somehow I have forgotten the !Sub in the beginning
This is what worked for me,
"Role": { "Fn::Join" : [ "", [ "arn:aws:iam::", { "Ref" : "AWS::AccountId" }, ":role/MyCustomRole" ] ] }
I was getting the same problem with below syntax -
"Resources" : {
"CustomLambdaFunction" : {
"Type" : "AWS::Lambda::Function",
"Properties" : {
"Role" : "arn:aws:iam::<account-id>:role/MyCustomRole",
}
},
I solved it like this -
The issue was that when inserting my AWS account ID in place of "account-id", I was keeping it in the same format as is given on the AWS console i.e. xxxx-xxxx-xxxx. However, the "account-id" space expects "\d{12}" format, i.e. 12 digits only. Removing the '-' in between digits solved the problem for me.
In the CloudFormation template I am deploying, I am running a long running operation in the UserData block.
It looks as follows:
"UserData": {
"Fn::Base64" : {
"Fn::Join" : [
"",
[
"/usr/local/bin/mylongrunningscript.sh"
]
]
}
}
The contents of the script are:
echo "UserData starting!" > /var/log/mycustomlog.log
sleep 300
echo "UserData finished!" >> /var/log/mycustomlog.log
The issue I am seeing is that I believe the CloudFormation template is completing it's deployment before the UserData script finishes running. I believe this is the case because if I am quick enough and ssh into the instance, I will see something as follows:
$ cat /var/log/mycustomlog.log
UserData starting
which suggests that the UserData didn't finish running
How can I make sure that the UserData code execution is completed before the stack is in the "CreateComplete" status?
To ensure the CloudFormation template waits for the completion of the UserData script, you must do two things:
Add a CreationPolicy to the resource you are targeting (virtual machine in my case).
Add logic in the script to signal its completion. This custom logic uses the cfn-signal utility, which you might have to install in your instance.
Here's how the template looks now:
"UserData": {
"Fn::Base64" : {
"Fn::Join" : [
"",
[
"/usr/local/bin/mylongrunningscript.sh"
]
]
}
},
"CreationPolicy": {
"ResourceSignal" : {
"Count": "1",
"Timeout": "PT10M"
}
}
The cfn-signal utility is used to signal the termination of the script:
"/home/ubuntu/aws-cfn-bootstrap-*/bin/cfn-signal -e $? ",
" --stack ", { "Ref": "AWS::StackName" },
" --resource MyInstance" ,
" --region ", { "Ref" : "AWS::Region" }, "\n"
See here for a Windows example.
I'm trying to create a CloudFormation template that'll deploy a Lambda function, And I need the security options to be optional parameters.
I was able to partially accomplish this using the question here:
How to make a whole object in CloudFormation templates optional?
Interestingly, that method worked great to make the VpcConfig property optional in the AWS GUI Console, but it did NOT work to make it optional for the CLI. And unfortunately, I need it to work in the CLI, since I'll be using CodeBuild to call and deploy this template's resources.
Here are the relevant parameters:
"SecurityGroupIds" : {
"Type" : "CommaDelimitedList",
"Description" : "A list of one or more security groups IDs in the VPC that includes the resources to which your Lambda function requires access."
},
"SubnetIds" : {
"Type" : "CommaDelimitedList",
"Description" : "A list of one or more subnet IDs in the VPC that includes the resources to which your Lambda function requires access."
}
And conditions:
"HasVPC": {"Fn::And": [{"Fn::Not": [{"Fn::Equals": [{"Fn::Join": ["", {"Ref": "SubnetIds"}]}, ""]}]}, {"Fn::Not": [{"Fn::Equals": [{"Fn::Join": ["", {"Ref": "SecurityGroupIds"}]}, ""]}]}]}
And here's where that condition is used in the Lambda resource being defined in the Resources section of the template:
"VpcConfig": {
"Fn::If": [
"HasVPC",
{
"SecurityGroupIds" : {"Ref": "SecurityGroupIds"},
"SubnetIds" : {"Ref": "SubnetIds"}
},
{ "Ref":"AWS::NoValue" }
]
},
When I issue the command to deploy this stack in the CLI, I get the following error:
An error occurred (ValidationError) when calling the CreateChangeSet
operation: Parameters: [SecurityGroupIds, SubnetIds] must have values
Here's the AWS CLI command I'm issuing, from the same directory in which the template is located. Note: the ARN values have all been heavily modified to not be real values from my account, but I kept them in the right format so you can see the real format of the command:
aws cloudformation deploy --template-file lambda-template.json --stack-name "CLI-lambda-stack" --parameter-overrides S3BucketName="myBucket" S3FileLocation="lambda_function.zip" S3ObjectVersion="ZuB0iueEghOyh5q00.DiykLNudujdsc5" DeadLetterArn="arn:aws:sns:us-west-2:577898337216:CloudFormationTests" EnvironmentVariable="testing" KmsKeyArn="arn:aws:kms:us-west-2:504398934246:key/b24e7b72-a94d-6a3e-b848-165115c86212" HandlerFunctionName="lambda_function.lambda_handler" MemorySize="128" Role="arn:aws:iam::102893937243:role/serverless-test-default-us-east-1-lambdaRole" FuncName="myCLILambda"
You are not providing SecurityGroupIds neither SubnetIds default values and your are not providing them on your --parameter-overrides. Therefore, CloudFormation doesn't know how to process them if no values are provided.
Adding the Default statement should do the trick:
{
"Parameters" : {
"SecurityGroupIds" : {
"Type" : "CommaDelimitedList",
"Description" : "A list of one or more security groups IDs in the VPC that includes the resources to which your Lambda function requires access.",
"Default" : ""
},
"SubnetIds" : {
"Type" : "CommaDelimitedList",
"Description" : "A list of one or more subnet IDs in the VPC that includes the resources to which your Lambda function requires access.",
"Default" : ""
}
}
My goal is to have a Cloud Formation template which not only automatically creates a VPC with a NAT host and bastion host, but which deploys a .NET app pulled from S3 into an Elastic Beanstalk which is load balanced and more importantly only allows access to the app from my office, NOT the whole internet. It seems that even though the app might be in a VPC in a Private subnet and the ELB is in the Public subnet, that the Network ACL on the Public subnet is irrelevant. If I lock down the Public subnet to only my office, connections from outside the office can still come into the ELB and hit the application.
What seems to work is applying a Security group to the ELB, but I do not see any way to force the creation of a specific ELB with a specific SG inside a "AWS::ElasticBeanstalk::Environment" object. The ELB and ELB SG are created automatically by beanstalk and must be manually altered after CF runs. I don't want that. I want a way to create everything in CF in an automated way with no manual steps after the fact. yes, I've tried Cloud Former on a manually created stakc. No, it doesn't give me what I want.
Here's an excerpt from my CF Template:
"MyWebApp": {
"Type": "AWS::ElasticBeanstalk::Application",
"Properties": {
"ApplicationName" : "AlmDemoWebApp",
"Description": "MyWebapp"
}
},
"MyWebAppVersion": {
"Type": "AWS::ElasticBeanstalk::ApplicationVersion",
"Properties": {
"ApplicationName": {"Ref": "MyWebApp"},
"SourceBundle": {
"S3Bucket": "mywebapp",
"S3Key": {"Fn::Join" : ["", ["MyWebApp.", {"Ref":"Version"}, ".zip"]]}
}
}
},
"MyWebAppEnvironment" : {
"DependsOn" : ["MyWebApp", "MyWebAppVersion", "BastionSecurityGroup", "BeanstalkSecurityGroup", "VPC", "EBLoadBalancer", "EBLoadBalancerSecurityGroup", "PrivateSubnet", "PublicSubnet"],
"Type" : "AWS::ElasticBeanstalk::Environment",
"Properties" : {
"ApplicationName" : { "Ref" : "MyWebApp" },
"Description" : "MyWebApp Target Environment",
"SolutionStackName": "64bit Windows Server 2012 R2 running IIS 8.5",
"OptionSettings" : [
{"Namespace" : "aws:autoscaling:launchconfiguration", "OptionName" : "SecurityGroups", "Value" : { "Ref" : "BeanstalkSecurityGroup" }},
{"Namespace" : "aws:autoscaling:launchconfiguration", "OptionName" : "EC2KeyName", "Value" : { "Ref" : "InstanceKeyName" }},
{"Namespace" : "aws:ec2:vpc", "OptionName" : "VPCId", "Value" : { "Ref" : "VPC" }},
{"Namespace" : "aws:ec2:vpc", "OptionName" : "Subnets", "Value" : { "Ref" : "PrivateSubnet" }},
{"Namespace" : "aws:ec2:vpc", "OptionName" : "ELBSubnets", "Value" : { "Ref" : "PublicSubnet" }}],
"VersionLabel": {"Ref": "MyWebAppVersion"}
}
}
Is there some mysterious and poorly documented option that I can put in the AWS::ElasticBeanstalk::Environment -> Properties -> OptionSettings that will force the Elastic Beanstalk to use a specific ELB configured previously in the CF template ("EBLoadBalancer") rather than automatically create one with a random name? Applying ingress rules to the "BeanstalkSecurityGroup" doesn't seem to help. The SG rules have to be on the ELB to actually work, apparently.
Sounds like you need to create an internal ELB for your Elastic Beanstalk stack. For this, create a property as below.
"ELBScheme":{
"Type":"String",
"AllowedValues":[
"internal"
],
"Default":"internal",
"Description":"Internal load balancer in VPC so that your Elastic Beanstalk application cannot be accessed from outside your VPC."
}
And refer it as a namespace in your AWS::ElasticBeanstalk::Environment "MyWebAppEnvironment"
{
"Namespace":"aws:ec2:vpc",
"OptionName":"ELBScheme",
"Value":"internal"
}
I tried running a data pipeline job but the EmrActivity step reached a FAILED status but there is no error code or error message:
Name:
#EMR cluster to perform the work_2013-09-03T16:15:00
View instance fields
Description:
Latest attempt count: 3, Tries left: 0
Select attempt for this instance:
Status:
FAILED
Error code:
Error message:
any idea why? Where can I find out more info about the underlying problem?
The job is simple: fire up EMR cluster and run a pig script (where xxx is my bucket name):
{
"objects": [
{
"id":"Default",
"failureAndRerunMode":"cascade"
},
{
"id" : "MyScheduleID",
"type" : "Schedule",
"period" : "1 hour",
"startDateTime" : "2013-09-03T19:00:00",
"endDateTime" : "2013-09-03T20:00:00"
},
{
"id" : "MyEmrCluster",
"name" : "EMR cluster to perform the work",
"type" : "EmrCluster",
"hadoopVersion" : "0.20",
"masterInstanceType" : "m1.small",
"coreInstanceType" : "m1.medium",
"coreInstanceCount" : "2",
"terminateAfter": "1 Hours",
"schedule": {
"ref": "MyScheduleID"
},
"logUri":"s3://xxx/amazonlogs",
"emrLogUri":"s3://xxx/amazonlogs"
},
{
"id" : "MyEmrActivity",
"name" : "Work to perform on my data",
"type" : "EmrActivity",
"runsOn" : {"ref" : "MyEmrCluster"},
"schedule": {
"ref": "MyScheduleID"
},
"step": "s3://us-east-1.elasticmapreduce/libs/script-runner/script-runner.jar,s3://us-east-1.elasticmapreduce/libs/pig/pig-script,--base-path,s3://us-east-1.elasticmapreduce/libs/pig/,--install-pig,--pig-versions,latest",
"step": "s3://us-east-1.elasticmapreduce/libs/script-runner/script-runner.jar,s3://us-east-1.elasticmapreduce/libs/pig/pig-script,--base-path,s3://us-east-1.elasticmapreduce/libs/pig/,--pig-versions,latest,--run-pig-script,--args,-f,s3://xxx/carls_minimal_script.pig"
}
]
}
Does this config look OK?
I don't see anything in s3://xxx/amazonlogs
Here are couple of things you could try
Go to "https://console.aws.amazon.com/elasticmapreduce/home", find the corresponding cluster that got started (based on timestamp), click on "Debug", you should find logs about each step.
Or start an EMR cluster from AWS Console, login into the Master node, run the Pig script to check if its working.