Cloudformation Elasticbeanstalk specify target group for shared load balancer - amazon-web-services
I have two Cloudformation templates
one which creates a VPC, ALB and any other shared resources etc.
one which creates an elastic beanstalk environment and relevant listener rules to direct traffic to this environment using the imported shared load balancer (call this template Environment)
The problem I'm facing is the Environment template creates a AWS::ElasticBeanstalk::Environment which subsequently creates a new CFN stack which contains things such as the ASG, and Target Group (or process as it is known to elastic beanstalk). These resources are not outputs of the AWS owned CFN template used to create the environment.
When setting
- Namespace: aws:elasticbeanstalk:environment
OptionName: LoadBalancerIsShared
Value: true
In the optionsettings for my elastic beanstalk environment, a load balancer is not created which is fine. I then try to attach a listener rule to my load balancer listener.
ListenerRule:
Type: AWS::ElasticLoadBalancingV2::ListenerRule
Properties:
Priority: 1
ListenerArn:
Fn::ImportValue: !Sub '${NetworkStackName}-HTTPS-Listener'
Actions:
- Type: forward
TargetGroupArn: WHAT_GOES_HERE
Conditions:
- Field: host-header
HostHeaderConfig:
Values:
- mywebsite.com
DependsOn:
- Environment
The problem here is that I don't have access as far as I can tell to the ARN of the target group created by the elastic beanstalk environment resource. If I create a target group then it's not linked to elastic beanstalk and no instances are present.
I found the this page which states
The resources that Elastic Beanstalk creates for your environment have names. You can use these names to get information about the resources with a function, or modify properties on the resources to customize their behavior.
But because they're in a different stack (of which i don't know the name in advance), not ouputs of the template, I have no idea how to get hold of them.
--
Edit:
Marcin pointed me in the direction of a custom resource in their answer. I have expanded on it slightly and got it working. The implementation is slightly different in a couple of ways
it's in Node instead of Python
the api call describe_environment_resources in the example provided returns a list of resources, but seemingly not all of them. In my implementation I grab the auto scaling group, and use the Physical Resource ID to look up the other resources in the stack to which it belongs using the Cloudformation API.
const AWS = require('aws-sdk');
const cfnResponse = require('cfn-response');
const eb = new AWS.ElasticBeanstalk();
const cfn = new AWS.CloudFormation();
exports.handler = (event, context) => {
if (event['RequestType'] !== 'Create') {
console.log(event[RequestType], 'is not Create');
return cfnResponse.send(event, context, cfnResponse.SUCCESS, {
Message: `${event['RequestType']} completed.`,
});
}
eb.describeEnvironmentResources(
{ EnvironmentName: event['ResourceProperties']['EBEnvName'] },
function (err, { EnvironmentResources }) {
if (err) {
console.log('Exception', e);
return cfnResponse.send(event, context, cfnResponse.FAILED, {});
}
const PhysicalResourceId = EnvironmentResources['AutoScalingGroups'].find(
(group) => group.Name
)['Name'];
const { StackResources } = cfn.describeStackResources(
{ PhysicalResourceId },
function (err, { StackResources }) {
if (err) {
console.log('Exception', e);
return cfnResponse.send(event, context, cfnResponse.FAILED, {});
}
const TargetGroup = StackResources.find(
(resource) =>
resource.LogicalResourceId === 'AWSEBV2LoadBalancerTargetGroup'
);
cfnResponse.send(event, context, cfnResponse.SUCCESS, {
TargetGroupArn: TargetGroup.PhysicalResourceId,
});
}
);
}
);
};
The Cloudformation templates
LambdaBasicExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
Path: /
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AWSCloudFormationReadOnlyAccess
- arn:aws:iam::aws:policy/AWSElasticBeanstalkReadOnly
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
GetEBLBTargetGroupLambda:
Type: AWS::Lambda::Function
Properties:
Handler: index.handler
Description: 'Get ARN of EB Load balancer'
Timeout: 30
Role: !GetAtt 'LambdaBasicExecutionRole.Arn'
Runtime: nodejs12.x
Code:
ZipFile: |
... code ...
ListenerRule:
Type: AWS::ElasticLoadBalancingV2::ListenerRule
Properties:
Priority: 1
ListenerArn:
Fn::ImportValue: !Sub '${NetworkStackName}-HTTPS-Listener'
Actions:
- Type: forward
TargetGroupArn:
Fn::GetAtt: ['GetEBLBTargetGroupResource', 'TargetGroupArn']
Conditions:
- Field: host-header
HostHeaderConfig:
Values:
- mydomain.com
Things I learned while doing this which hopefully help others
using async handlers in Node is difficult with the default cfn-response library which is not async and results in the Cloudformation creation (and deletion) process hanging for many hours before rolling back.
the cfn-response library is included automatically by cloudformation if you use ZipFile. The code is available on the AWS Docs if you were so inclined to include it manually (you could also wrap it in a promise then and use async lambda handlers). There are also packages on npm to achieve the same effect.
Node 14.x couldn't run, Cloudformation threw up an error. I didn't make note of what it was, unfortunately.
The policy AWSElasticBeanstalkFullAccess used in the example provided no longer exists and has been replaced with AdministratorAccess-AWSElasticBeanstalk.
My example above needs less permissive policies attached but I've not yet addressed that in my testing. It'd be better if it could only read the specific elastic beanstalk environment etc.
I don't have access as far as I can tell to the ARN of the target group created by the elastic beanstalk environment resource
That's true. The way to overcome this is through custom resource. In fact I developed fully working, very similar resource for one of my previous answers, thus you can have a look at it and adopt to your templates. The resource returns ARN of the EB load balancer, but you could modify it to get the ARN of EB's target group instead.
Related
AWS API Gateway: Api Event must have a String specified for 'Path'
I'm trying to deploy an AWS API Gateway and a Lambda function using Gitlab CICD pipeline using sam template. Since I want individual stacks for each environment, the Path property under Events of the Lambda function needs to change with each environment. Below is my code for implementing this: Mappings: StackEnv: stack-dev: envm: "dev" stack-qa: envm: "qa" Resources: MyFunction: Type: AWS::Serverless::Function Properties: Events: MyAPI: Type: Api Properties: Path: Fn::Sub: - "/${path}" - path: !FindInMap [StackEnv, !Ref AWS::StackName, envm] However, while running the pipeline, an error is returned: [InvalidResourceException('MyFunction', "Event with id [MyAPI] is invalid. Api Event must have a String specified for 'Path'.")] Is this behaviour expected while passing in values using !FindInMap? Is there any other way of dynamically passing in the value for Path?
Expose SNSTopic TopicArn in AWS CloudFormation Template: How might I expose my TopicArn in my CloudFormation script for my SNS Topic?
I'd like to expose the TopicArn Value (referenced in the outputs section at the bottom of my code snippet) of my SNStopic via Cloudformation template in the outputs tab of my stack in a similar manner to the way it's exposed in the resources when I create an SNStopic through the service catalog. I tried to access it by referencing it in the outputs section of my yaml script using dot notation but have been unsuccessful thus far. How might I be able to do so? I'm looking to do this so others using my script in the future won't have to go searching for the TopicArn in another place in order to subscribe to it. Another important thing to note is that the provisioned product id below, under the properties section of the resources code block generates an SNSTopic. Resources: LabTrainingSnsTopic: Type: "AWS::ServiceCatalog::CloudFormationProvisionedProduct" Properties: ProductId: prod-4iafsjovqrsrm # Sns Topic ProvisioningArtifactName: "v1.1" # Must be an actual version number. ProvisionedProductName: !Ref ProvisionedProductName ... Outputs: AccountID: Description: The account in which this was built. Value: !Ref 'AWS::AccountId' TopicArn: Description: Arn of the topic we created Value: !GetAtt LabTrainingHigSnsTopic.ProvisionedProductName.Resources.SNSTopic service catalog screenshot cloudformation screenshot
Cannot connect RDS Database to a Lambda Function (Using serverless framework)
I want to connect my RDS Database table with my lambda function, for this, I have created a lambda function and used knex.js and postgres database in rds, I got the knex object, but I cannot work with any query. To give some more information about the services, RDS database server security group can be access from anywhere I have given the vpc in the serverless.yml file in the function. Region of both lambda and rds are different, but not sure whether it is the problem. My serverless function note: this knex code is working when I tried this separately. module.exports.storeTransaction = async (event) => { ... knex('Transactions') .select('*') .then(response => { console.log('response is '); console.log(response); }) ... }; Serverless.yml file service: <service-name> provider: name: aws runtime: nodejs8.10 stage: dev region: us-east-1 package: exclude: - node_modules/** plugins: - serverless-plugin-include-dependencies functions: storeEmail: handler: handler.storeTransaction vpc: securityGroupIds: - <security-group-id-of-rds> subnetIds: - <subnet-id-of-rds> - <subnet-id-of-rds> ... region: - us-east-1a events: - http: path: email/store method: post cors: true So can you identify my issue on why I can't connect my rds db with lambda function, and let me know what I did wrong or what is missing.
I think the problem is that RDS and Lambda are in different regions, which means they are also in different VPCs, as a VPC cannot span across multiple regions. Although you can enable Inter VPC Peering (https://aws.amazon.com/vpc/faqs/#Peering_Connections). Consider that when you deploy a lambda function in a VPC, it won't have internet access as long as you don't attach a NAT Gateway to that VPC/subnet. If the RDS is open to the world (and does it really need to be??), you can try to deploy in the same region (without a VPC) and verify if that works.
Do AWS support SES in CloudFormation?
I'm trying to figure out how to automate the creation of several cloud resources in AWS, using CloudFormation. Now I need to include the creation of SES (Simple Email Service) domain, but couldn't find the documentation, but I've already checked: Simple Email Service Documentation CloudFormation Resource Types Documentation Do AWS support SES in CloudFormation?
CloudFormation provides several built-in Amazon SES resource types, but as of 2018 2020 2022 is still missing the ones many people need: domain and email verification. Fortunately, CloudFormation has the ability to define your own custom resource types. I've built Custom::SES_Domain and Custom::SES_EmailIdentity resources that are designed to play well with other CloudFormation resources. Get them here: https://github.com/medmunds/aws-cfn-ses-domain. Once you've pulled the custom CfnSESResources into your template, you can verify an SES domain like this: Resources: # Provision a domain with Amazon SES: MySESDomain: Type: Custom::SES_Domain Properties: ServiceToken: !GetAtt CfnSESResources.Outputs.CustomDomainIdentityArn Domain: "example.com" EnableSend: true EnableReceive: false # Then add all required DNS records for SES verification and usage: MyRoute53RecordsForSES: Type: AWS::Route53::RecordSetGroup Properties: HostedZoneName: "example.com." RecordSets: !GetAtt MySESDomain.Route53RecordSets Full instructions are in the repository. Custom::SES_Domain has properties for controlling several common SES domain options, and exposes attributes that feed into your CloudFormation DNS resources: either a standard AWS::Route53::RecordSetGroup resource as shown above, or other (external) DNS providers via zone file entries.
Unfortunately this is currently not supported, but who knows Re:Invent 2017 is around the corner ,,, Question asked on AWS Developer Forum It is possible by creating a custom function, some blog about SES and cloudformation.
Though AWS Cloudformation is not currently supported use the AWS SDKs ( e.g Node SDK) to provision the SES resources required. Its a common practice to use custom code with AWS SDKs and AWS CLI commands in combination with CloudFormation to provision resources AWS since each approach can be advantages, based on the parameters, number of resources, repetitions and etc.
Here is the current list of SES Resource Types supported by CloudFormation: AWS::SES::ConfigurationSet AWS::SES::ConfigurationSetEventDestination AWS::SES::ReceiptFilter AWS::SES::ReceiptRule AWS::SES::ReceiptRuleSet AWS::SES::Template
Update October 2022 CloudFormation now supports the AWS::SES::EmailIdentity resource, which allows us to define both domains and email addresses through infrastructure as code. According to the CloudFormation release history this resource was added on June 30, 2022.
CloudFormation provides a nativ AWS::SES::EmailIdentity resource now. (since 30.07.2022) Here is an example with automated Route53 DEKIM setup/verification: EmailIdentity: Type: AWS::SES::EmailIdentity Properties: EmailIdentity: {your.domain.com} Route53DEKIM: Type: AWS::Route53::RecordSetGroup Properties: HostedZoneId: {ZoneId} RecordSets: - Name: !GetAtt EmailIdentity.DkimDNSTokenName1 Type: CNAME TTL: '3600' ResourceRecords: - !GetAtt EmailIdentity.DkimDNSTokenValue1 - Name: !GetAtt EmailIdentity.DkimDNSTokenName2 Type: CNAME TTL: '3600' ResourceRecords: - !GetAtt EmailIdentity.DkimDNSTokenValue2 - Name: !GetAtt EmailIdentity.DkimDNSTokenName3 Type: CNAME TTL: '3600' ResourceRecords: - !GetAtt EmailIdentity.DkimDNSTokenValue3 {your.domain.com} and {ZoneId} must be adapted.
Not supported. But, you can make it handled by lambda. AWSTemplateFormatVersion: '2010-09-09' Transform: 'AWS::Serverless-2016-10-31' Description: >- A simple email example Resources: FunctionEmailHandler: Type: 'AWS::Serverless::Function' Properties: Handler: email.handler Runtime: nodejs6.10 CodeUri: .. Description: >- ... Tags: App: your app MemorySize: 128 Timeout: 10 Policies: - Version: '2012-10-17' Statement: - Effect: Allow Action: - 's3:GetObject' Resource: '*' LambdaInvokePermission: Type: "AWS::Lambda::Permission" Properties: Action: 'lambda:InvokeFunction' FunctionName: !GetAtt FunctionEmailHandler.Arn Principal: ses.amazonaws.com SESEmailRecievedRule: Type: "AWS::SES::ReceiptRule" Properties: RuleSetName: your default rule set name After: store-email-to-s3 Rule: Name: email-recieved-rule Enabled: true Actions: - LambdaAction: FunctionArn: !GetAtt FunctionEmailHandler.Arn InvocationType: Event
How to attach an existing role to serverless.yml?
I want to attach an existing role to my serverless.yml file, I have created a role in aws console, my code works fine when I test it in aws console, but when I try to test it with the http endpoint it gives me the following: {"message": "Internal server error"} I think is because I did not specify any role in the serverless.yml file for the simple reason that I don't know how to do it. Here is my serverless.yml file : Resources: ec2-dev-instance-status: Properties: Path: "arn:aws:iam::119906431229:role/lambda-ec2-describe-status" RoleName: lambda-ec2-describe-status Type: "AWS::IAM::Role" functions: instance-status: description: "Status ec2 instances" events: - http: method: get path: users/create handler: handler.instance_status role: "arn:aws:iam::119906431229:role/lambda-ec2-describe-status" provider: name: aws region: us-east-1 runtime: python2.7 stage: dev resources: ~ service: ec2 Please help. Thank you.
According to the documentation, there's a few ways to attach existing roles to a function (or entire stack) Role defined as a Serverless resource resources: Resources: myCustRole0: Type: AWS::IAM::Role # etc etc functions: func0: role: myCustRole0 Role defined outside of the Serverless stack functions: func0: role: arn:aws:iam::0123456789:role//my/default/path/roleInMyAccount Note that the role you use must have additional permissions to log to cloudwatch etc, otherwise you won't get logging.