Create SimpleDB domain using serverless.yaml - amazon-web-services

So I am creating a server less application on Amazon AWS using the Serverless Framework.
For our stack, we create a number of Lambda functions, DynamoDB table, API Gateway and now we want to add a simpleDB domain as well.
I cannot seem to find any information online on what code snippet to add to serverless.yaml to create a SimpleDB domain.
I wrote the following code, which creates the domain but the name of the domain is not as expected
resources:
Resources:
LogSimpleDBTable:
Type: "AWS::SDB::Domain"
Properties:
DomainName : ${self:provider.environment.SIMPLEDB}
Description: "SDB Domain to store data log"
And the variable SimpleDB is defined as
SIMPLEDB: git-pushr-processing-${opt:stage, self:provider.stage}
So when I deploy using the command
serverless deploy --stage staging --awsaccountid XXXXX
I expect the name of the SimpleDB table to be
git-pushr-processing-staging
instead I get a domain with the following name
git-pushr-api-staging-LogSimpleDBTable-1P7CQH4SGAWGI
Where the last bit of sequence (1P7CQH4SGAWGI) varies every time.
We are using the exact same pattern to name our DynamoDB tables and they seem to get created with correct name
DYNAMODB_TABLE: git-pushr-processing-${opt:stage, self:provider.stage}
resources:
Resources:
TodosDynamoDbTable:
Type: 'AWS::DynamoDB::Table'
DeletionPolicy: Retain
Properties:
AttributeDefinitions:
-
AttributeName: id
AttributeType: S
KeySchema:
-
AttributeName: id
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
TableName: ${self:provider.environment.DYNAMODB_TABLE}
StreamSpecification:
StreamViewType: NEW_AND_OLD_IMAGES
We get a DynamoDB table with the following name
git-pushr-processing-staging
So what am I doing wrong here ?

I don't know how to make serverless use the domain name of your choice.
However, it is possible to reference the domain generated using Ref: LogSimpleDBTable syntax
E.g. to pass the domain name to lambda (making it available as process.env.SDB_DOMAIN_NAME variable):
functions:
queueRequests:
handler: src/consumer.handler
name: consumer
environment:
SDB_DOMAIN_NAME:
Ref: LogSimpleDBTable
Or reference it in IAM role statements
provider:
...
iamRoleStatements:
- Effect: Allow
Action:
- sdb:GetAttributes
- sdb:PutAttributes
Resource:
Fn::Join:
- ""
- - "arn:aws:sdb:*:*:domain/"
- Ref: LogSimpleDBTable

Related

How to trigger an any aws resource based on creation of another aws resource?

I am using cloudformation template to build a dynamo db ( see below), once the stack builds successfully, I would like to alert/notify/trigger another aws resource, like a lambda function or a step function or a aws data pipeline to start such that I can start populating the dynamo. what is the best way to trigger another process or aws resource once cloudformation stack creation is successful ?
AWSTemplateFormatVersion: 2010-09-09
Description: AWS CloudFormation Template To Create a DynamoDB
Parameters:
HashKeyElementName:
Type: String
Default: Id
Description: Hash Key Name
HashKeyElementType:
Type: String
Default: S
Description: Hash Key Type
Resources:
Table:
Type: AWS::DynamoDB::Table
Properties:
TableName: MyTable
AttributeDefinitions:
-
AttributeName: !Ref HashKeyElementName
AttributeType: !Ref HashKeyElementType
KeySchema:
-
AttributeName: !Ref HashKeyElementName
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: 5
WriteCapacityUnits: 5
Outputs:
MyTable:
Description: Table Created using this template.
Value: !Ref Table
You can setup SNS topic and a lambda function to get events from stack creation. An example of how to do it is given in AWS docs:
How can I receive an email alert when my AWS CloudFormation stack enters ROLLBACK_IN_PROGRESS status?
You would have to adapt the example and the lambda to suit your needs.

Adding Tags in DynamoDB GlobalTable CloudFormation Template

I need to add some tags for DynamoDB Global table CFT ,trying to write one sample cft below .
Here I have creating resource for dynamoDB Global Table with KMS key and adding tags inside SSESpecification:
AWSTemplateFormatVersion: 2010-09-09
Description: AWS CloudFormation Template to create global tables
Resources:
globalTableExample:
Type: 'AWS::DynamoDB::GlobalTable'
Properties:
TableName: sample1
AttributeDefinitions:
- AttributeName: PK
AttributeType: S
KeySchema:
- AttributeName: PK
KeyType: HASH
BillingMode: PAY_PER_REQUEST
StreamSpecification:
StreamViewType: NEW_AND_OLD_IMAGES
SSESpecification:
SSEEnabled: true
SSEType: "KMS"
Replicas:
- Region: us-east-1
Tags:
- Key: Name
Value: mytable
- Key: Region
Value: east
getting error **Properties validation failed for resource globalTableExample with message: #: extraneous key [Tags] is not permitted**
Could someone please suggest me how to use tag ?
You need to define the tags under the Replica resource, as the tags belong to the replica and not the global table.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-dynamodb-globaltable-replicaspecification.html

Trying to create a stack in cloud formation using SAM and getting an error message

When I am trying to deploy a stack using SAM I am getting the following error message.
"Failed to create changeset for the stack: test, ex: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state: For expression "Status" we matched expected path: "FAILED" Status: FAILED. Reason: Invalid template resource property 'KeySchema'"
I am trying to create an API gateway that is connected to a Lambda function that would interact with a DynamoDB table. my YAML template file is valid and KeySchema seems to be a valid resource property according to documentation https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/quickref-dynamodb.html
My Yaml file looks like this
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: "Stack for DynamoDB, Lambda and APIGateway"
Resources:
CounterFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: Backend/
Handler: lambda.lambda_handler
Runtime: python3.8
Events:
VisitorData:
Type: Api
Properties:
Path: /Counter
Method: get
Visitors:
Type: AWS::DyanmoDB::Table
Properties:
AttributeDefinitions:
-
AttributeName: "ID"
AttributeType: "N"
-
AttributeName: "Counter"
AttributeType: "N"
KeySchema:
- AttributeName: "ID"
KeyType: "HASH"
ProvisionedThroughput:
ReadCapacityUnits: 5
WriteCapacityUnits: 5
Outputs:
VisitorDataApi:
Description: "API Gateway endpoint URL for Prod stage for Counter Function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
CounterFunction:
Description: "Counter Function lambda ARN"
Value: !GetAtt CounterFunction.Arn
HelloWorldFunctionIamRole:
Description: "Implicit IAM Role created for Counter Function"
Value: !GetAtt Lambda-Get-Function-role-40isam1j
Visitors Table:
Description: "Visitor table ARN"
Value: !GetAtt "Visitors.ARN"
I would appreciate any feedback and some insight to what the error message is trying to tell me thanks.
It should be:
Type: AWS::DynamoDB::Table
not:
Type: AWS::DyanmoDB::Table
I understand my mistakes in my code. #Marcin made some good points, but why nothing happened is due to the fact that it was making a new file called template.yml every time i built it. my original file name was Infrastructure.yml but the updates were created just in a new file thanks for your help

[serverless-framework]Deploying app to a new stage tries to update old stage

We have a simple serverless application that contains a dynamo DB table, a few lambdas and an API endpoint. We currently have the app deployed in the dev stage.
We are having some trouble deploying to the prod stage.
Here is the serverless.yaml file.
service: lookups
# app and org for use with dashboard.# serverless.com
app: lookups
org: xxxxxx
provider:
name: aws
runtime: python3.8
environment:
DYNAMO_DB_LOOKUP_TABLE_NAME: lookup_${self:provider.stage}
S3_BUCKET: com.yyyyy.lookups.${self:provider.stage}
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:PutItem
- dynamodb:UpdateItem
- dynamodb:GetItem
Resource: "arn:aws:dynamodb:${self:provider.region}:*:table/${self:provider.environment.DYNAMO_DB_LOOKUP_TABLE_NAME}"
functions:
createOrUpdateLookups:
handler: createOrUpdateLookups.createOrUpdateLookups
description: create or update lookup entry in dynamodb
environment:
lookupTable: ${self:provider.environment.DYNAMO_DB_LOOKUP_TABLE_NAME}
events:
- s3:
bucket: ${self:provider.environment.S3_BUCKET}
event: s3:ObjectCreated:*
rules:
- suffix: .json
getLookup:
handler: getLookup.getLookup
description: get persigned url for a lookup by location and lookup type
environment:
lookupTable: ${self:provider.environment.DYNAMO_DB_LOOKUP_TABLE_NAME}
lookupBucket: ${self:provider.environment.S3_BUCKET}
events:
- http:
path: v1/lookup
method: get
request:
parameters:
querystrings:
location: true
lookupType: true
resources:
Resources:
lookupTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: ${self:provider.environment.DYNAMO_DB_LOOKUP_TABLE_NAME}
AttributeDefinitions:
- AttributeName: location
AttributeType: S
- AttributeName: lookup
AttributeType: S
KeySchema:
- AttributeName: location
KeyType: "HASH"
- AttributeName: lookup
KeyType: "RANGE"
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
We deployed this to the dev stage using the following cli command:
serverless deploy
This created a stack in CloudFormtion called lookups-dev, a DymanoDB table DB table called lookup-dev and lambdas called lookups-dev-createOrUpdateLookups and lookups-dev-getLookup .
Now when try to deploy to a new stage called prod using this cli command
serverless deploy --stage prod
We get an error saying the table lookups-dev already exists in the stack with the stack id of the lookups-dev stack.
This is the full error:
An error occurred: lookupTable - lookup_dev already exists in stack arn:aws:cloudformation:us-east-1:aaaaabbbbbbbccccccdddddd:stack/lookups-dev/wwwwwww-wwwwwww-wwwwwwaws.
Question:
How do we deploy to a new stage when we have already deployed out app the dev stage.
What has happened is that you need to make sure that the name of the table changes on different stages. I see you use ${self:provider.stage} to try and do this but all that does is use the value for stage under the provider section and because you haven't set one, it uses the default of dev always. I would suggest adding the following line under providers so that you have something like this:
provider:
stage: ${opt:stage, 'dev'}
What this means is that if you pass the stage on the CLI using --stage it will set the provider.stage to that value or to the default of dev.

AWS Exporting DynamoDB Name to Python

I’m trying to make a DynamoDB table, without having a name property in the .yml file so that it’s name by cloud formation, and export it’s name to python for access can I do that if so how?
My current idea is to to export the name as a ssm parameter but I’m not sure how.
You can tag it in the cloudformation template and get resources by tag in boto.
import boto3
client = boto3.client('rds')
custom_filter = [{
'Name':'tag:Owner',
'Values': ['user#example.com']}]
response = client.describe_instances(Filters=custom_filter)
(This code is mostly copied from https://stackoverflow.com/a/48073016/10553976)
And this would correspond to tagging instances with the following:
Tags:
-
Key: "Owner"
Value: "user#example.com"
Important: if you want to apply the tags to something other than the instances you would need to use a different method from the describe methods in the rds client.
I assume your question is related to AWS CloudFormation, because you mention .yml file.
You can report the name in the output section of your cloudformation template, if your python function is declared in another template. You can then use the describe-stack API or CLI to fetch the value from the output.
See :
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html and https://docs.aws.amazon.com/cli/latest/reference/cloudformation/describe-stacks.html
If your python function is declared in the same template, you can just refer to your logical resource to get the name (as per https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-dynamodb-table.html)
For example,
The section that creates the table
MyTable:
Type: AWS::DynamoDB::Table
Description: your decsriptions
Properties:
... your properties ...
The section that refers to it (here is an example with AWS::AppSync::DataSource but it applies to any type of resources
MyTableDataSource:
Type: AWS::AppSync::DataSource
Properties:
...
DynamoDBConfig:
TableName:
Ref: MyTable
AwsRegion:
Fn::Sub: ${AWS::Region}
or to get the table ARN in a IAM Policy
Policies:
- PolicyName: mypolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:DeleteItem
- dynamodb:UpdateItem
- dynamodb:Query
- dynamodb:Scan
Resource:
- Fn::Join:
- ''
- - Fn::GetAtt:
- MyTable
- Arn
- '*'
According to your reply to my comment, you wish to add the DynamoDB table into the yml file. If the table and the lambda are in the same .yml, then you can simply do !Ref YourTable, inside the Lambda Environment variables.
Something like this:
YourLambda:
Type: AWS::Serverless::Function
Properties:
Environment:
Variables:
YourTableName: !Ref YourTable
You will also need to add a policy attached to the Lambda, under Properties -> Policies, and you can reference the table name there the same way.
However, if you wish to reference the name without moving the DynamoDB instance inside the .yml file, then you have to make it a static reference by making an entry in Parameter Store, and then referencing it like so (making sure your CFN has access to ssm:getParameter):
YourLambda:
Type: AWS::Serverless::Function
Properties:
Environment:
Variables:
YourTableName: '{{resolve:ssm:/PATH/TO/TABLENAME:1}}'
Here’s how I ended up doing it:
Python:
ssm = boto3.client('ssm')
resp = dict(ssm.get_parameter(Name='TableName', WithDecryption=False))
tableName = str(json.loads(json.dumps(resp['Parameter'],default=str))['Value'])
This is my cloudformation yaml file:
Resources:
DynamoTable:
Type: "AWS::DynamoDB::Table"
Properties:
AttributeDefinitions:
- AttributeName: A_Key
AttributeType: "S"
- AttributeName: Serial
AttributeType: "S"
KeySchema:
- AttributeName: A_Key
KeyType: HASH
- AttributeName: Serial
KeyType: RANGE
DynamoTableParameter:
Type: "AWS::SSM::Parameter”
Properties:
Name: "TableName”
Type: String
Value: !Ref DynamoTable