I am using AWS SAM to invoke lambda function locally.
I have below folder structure of my node.js lambda.
+-rootfolder
|
+--utils
| |
| +--deps -- (This folder is common for all lambda functions)
| | +--mymodule.js
| | +--common.js
| +--foo
| +--some.js
|
+--lambdaOne
| +--index.js
| +--prokect.json
| +--test-event.json
| +--deps -- (linked folder ../utils/deps)
|
+--lambdaTwo
| +--index.js
| +--prokect.json
| +--test-event.json
| +--deps -- (linked folder ../utils/deps)
When I run the sam cli command as below:
$ sam local invoke -e ./lambdaOne/test-event.json LambdaOne
I am getting below error:
Runtime.ImportModuleError: Error: Cannot find module './deps/mymodule.js'
I think SAM is not loading the linked folder 'deps'.
I did try to use 'Layers' in my 'template.yaml' but still same error.
//template.yaml
AWSTemplateFormatVersion: 'xxxx-xx-xx'
Transform: AWS::Serverless-xxxx-xx-xx
Description: >
Run Lambda locally
Resources:
LambdaOne:
Type: AWS::Serverless::Function
Properties:
CodeUri: lambdaOne/
Handler: index.handler
Layers:
- !Ref LambdaDepLayer
Runtime: nodejs14.x
LambdaDepLayer:
Type: AWS::Serverless::LayerVersion
Properties:
LayerName: lambda-deps
Description: dependencies
ContentUri: ./utils/deps/
CompatibleRuntimes:
- nodejs14.x
LicenseInfo: 'MIT'
RetentionPolicy: Retain
Related
I've debugged my application, and identified a problem. I have 2 REST API Gateway, and it seems like since they both bind on the same endpoint, the first one will recieve the call that the second one should handle.
Here's my template.yaml
Resources:
mysampleapi1:
Type: 'AWS::Serverless::Function'
Properties:
Handler: packages/mysampleapi1/dist/index.handler
Runtime: nodejs14.x
CodeUri: .
Description: ''
MemorySize: 1024
Timeout: 30
Role: >-
arn:aws:iam:: [PRIVATE]
Events:
Api1:
Type: Api
Properties:
Path: /users
Method: ANY
Environment:
Variables:
NODE_ENV: local
Tags:
STAGE: local
mysampleapi2:
Type: 'AWS::Serverless::Function'
Properties:
Handler: packages/mysampleapi2/dist/index.handler
Runtime: nodejs14.x
CodeUri: .
Description: ''
MemorySize: 1024
Timeout: 30
Role: >-
arn:aws:iam:: [PRIVATE]
Events:
Api1:
Type: Api
Properties:
Path: /wallet
Method: ANY
Environment:
Variables:
NODE_ENV: local
Tags:
STAGE: local
When I send a HTTP request for mysampleapi2
Here's what's happening in the logs using the startup command
sam local start-api --port 3001 --log-file /tmp/server-output.log --profile personal --debug
2022-04-27 18:2:34,953 | Mounting /home/mathieu_auclair/Documents/Project/repositories/server as /var/task:ro,delegated inside runtime container
2022-04-27 18:20:35,481 | Starting a timer for 30 seconds for function 'mysampleapi1'
2022-04-27 18:21:05,484 | Function 'mysampleapi1' timed out after 30 seconds
2022-04-27 18:21:46,732 | Container was not created. Skipping deletion
2022-04-27 18:21:46,732 | Cleaning all decompressed code dirs
2022-04-27 18:21:46,733 | No response from invoke container for mysampleapi1
2022-04-27 18:21:46,733 | Invalid lambda response received: Lambda response must be valid json
Why is my mysampleapi2 not picking the HTTP call? If I run them in separate template files using different ports, then it works... why is that?
After launching my lambda in separate processes, I discovered that there's an issue in my configuration for the second service.
The issue still occured after this launcher
echo "" > /tmp/server-output-1.log
sam local start-api --port 3001 --log-file /tmp/server-output-1.log --template .template.1.yaml --debug &
tail -f /tmp/server-output-1.log &
echo "" > /tmp/server-output-2.log
sam local start-api --port 3002 --log-file /tmp/server-output-2.log --template .template.2.yaml --debug &
tail -f /tmp/server-output-2.log &
I noticed when I exported my configuration, for one of the services, there's the following in the template.yaml
Path: '/{proxy+}'
without the proxy line, the lambda handler just never gets called for some reason
I'm trying to dynamically pass in options to resolve when deploying my functions with serverless but they're always null or hit the fallback.
custom:
send_grid_api: ${opt:sendgridapi, 'missing'}
SubscribedUsersTable:
name: !Ref UsersSubscriptionTable
arn: !GetAtt UsersSubscriptionTable.Arn
bundle:
linting: false
provider:
name: aws
lambdaHashingVersion: 20201221
runtime: nodejs12.x
memorySize: 256
stage: ${opt:stage, 'dev'}
region: us-west-2
environment:
STAGE: ${self:provider.stage}
SEND_GRID_API_KEY: ${self:custom.send_grid_api}
I've also tried:
environment:
STAGE: ${self:provider.stage}
SEND_GRID_API_KEY: ${opt:sendgridapi, 'missing'}
both yield 'missing', but why?
sls deploy --stage=prod --sendgridapi=xxx
also fails if I try with space instead of =.
Edit: Working Solution
In my github action template, I defined the following:
- name: create env file
run: |
touch .env
echo SEND_GRID_API_KEY=${{ secrets.SEND_GRID_KEY }} >> .env
ls -la
pwd
In addition, I explicitly set the working directory for this stage like so:
working-directory: /home/runner/work/myDir/myDir/
In my serverless.yml I added the following:
environment:
SEND_GRID_API_KEY: ${env:SEND_GRID_API_KEY}
sls will read the contents from the file and load them properly
opt is for serverless' CLI options. These are part of serverless, not your own code.
You can instead use...
provider:
...
environment:
...
SEND_GRID_API_KEY: ${env:SEND_GRID_API_KEY}
And pass the value as an environment variable in your deploy step.
- name: Deploy
run: sls deploy --stage=prod
env:
SEND_GRID_API_KEY: "insert api key here"
I'm new to AWS and attempting to build an API for a basic course scheduling app. I am currently able to get the API running locally and able to invoke two functions. Function 1 is executing properly, but function 2 seems to be executing the code from function 1. Here is how I have my SAM app structured:
- sam-app
| - events
| - tests
| - src
| - api
| - course
| - AddCourse Lambda
| app.js (Index Lambda, the default hello world sample function, mostly just using to check that API is up)
The Index Lambda at app.js does a GET / and returns status code 200 and body with message "Hello World!" so long as the API is reachable.
The AddCourse Lambda is supposed to do the following via POST /courses:
try {
console.log("Adding a new item...");
await docClient.put(params).promise();
response = {
'statusCode': 200,
'headers': {
'Content-Type': "application/json"
},
'body': JSON.stringify({
message: 'Successfully created item!'
})
}
} catch (err) {
console.error(err);
response = {
'statusCode': 400,
'headers': {
'Content-Type': "application/json"
},
'body': JSON.stringify(err)
}
}
Instead, it is returning status code 200 and body with message "Hello World!".
My template.yml seems to have the correct routes specified too:
Resources:
Index:
Type: AWS::Serverless::Function
Properties:
CodeUri: src
Handler: app.handler
Runtime: nodejs14.x
Policies: AmazonDynamoDBReadOnlyAccess
PackageType: Image
Events:
GetEvent:
Type: Api
Properties:
Path: /
Method: get
Metadata:
DockerTag: nodejs14.x-v1
DockerContext: ./src
Dockerfile: Dockerfile
AddCourse:
Type: AWS::Serverless::Function
Properties:
CodeUri: src/api/course/Course-POST-CreateNewCourse
Handler: index.lambdaHandler
Runtime: nodejs14.x
Policies: AmazonDynamoDBFullAccess
PackageType: Image
Events:
GetEvent:
Type: Api
Properties:
Path: /courses
Method: post
Metadata:
DockerTag: nodejs14.x-v1
DockerContext: ./src
Dockerfile: Dockerfile
What could possibly be going on here? Is there something inherently wrong with how I structured my app?
I started writing something for AWS Systems Manager to:
Create custom Windows and Linux images and
How to apply Windows and Linux Updates to the AMIs that would be useful...
I'm following this example, but am not able to get results when I run it.
This is the error I receive:
Template format error: At least one Resources member must be defined.
Please let me know what I'm doing wrong here, I have gone through the code but not able to find what I am doing wrong.
#description: Create a Golden AMI with Linux distribution packages(ClamAV) and Amazon
#software(SSM & Inspector). For details,see https://github.com/miztiik/AWS-Demos/tree/master/How-To/setup-ami-lifecycle-management-using-ssm
Resources:
Ec2Instance:
Type: 'AWS::EC2::Instance'
assumeRole: "{{AutomationAssumeRole}}"
parameters:
SourceAmiId:
type: String
description: "(Required) The source Amazon Machine Image ID."
default: ami-0d2692b6acea72ee6
InstanceIamRole:
type: String
description: "(Required) The name of the role that enables Systems Manager (SSM)
to manage the instance."
default: ManagedInstanceRole
AutomationAssumeRole:
type: String
description: "(Required) The ARN of the role that allows Automation to perform
the actions on your behalf."
default: arn:aws:iam::{{global:ACCOUNT_ID}}:role/AutomationServiceRole
SubnetId:
type: String
description: "(Required) The subnet that the created instance will be placed into."
default: subnet-0a72519be8028a56c
TargetAmiName:
type: String
description: "(Optional) The name of the new AMI that will be created. Default
is a system-generated string including the source AMI id, and the creation time
and date."
default: GoldenAMI-RH-7_on_{{global:DATE_TIME}}
InstanceType:
type: String
description: "(Optional) Type of instance to launch as the workspace host. Instance
types vary by region. Default is t2.micro."
default: t2.micro
PreUpdateScript:
type: String
description: (Optional) URL of a script to run before updates are applied. Default
("none") is to not run a script.
default: none
PostUpdateScript:
type: String
description: (Optional) URL of a script to run after package updates are applied.
Default ("none") is to not run a script.
default: none
IncludePackages:
type: String
description: (Optional) Only update these named packages. By default ("all"),
all available updates are applied.
default: all
ExcludePackages:
type: String
description: (Optional) Names of packages to hold back from updates, under all
conditions. By default ("none"), no package is excluded.
default: none
lambdaFunctionName:
type: String
description: "(Required) The name of the lambda function. Default ('none') is
to not run a script."
default: Automation-UpdateSsmParam
mainSteps:
- name: launchInstance
action: aws:runInstances
maxAttempts: 5
timeoutSeconds: 1200
onFailure: Abort
inputs:
ImageId: "{{SourceAmiId}}"
InstanceType: "{{InstanceType}}"
SubnetId: "{{ SubnetId }}"
UserData: IyEvYmluL2Jhc2gNCg0KZnVuY3Rpb24gZ2V0X2NvbnRlbnRzKCkgew0KICAgIGlmIFsgLXggIiQod2hpY2ggY3VybCkiIF07IHRoZW4NCiAgICAgICAgY3VybCAtcyAtZiAiJDEiDQogICAgZWxpZiBbIC14ICIkKHdoaWNoIHdnZXQpIiBdOyB0aGVuDQogICAgICAgIHdnZXQgIiQxIiAtTyAtDQogICAgZWxzZQ0KICAgICAgICBkaWUgIk5vIGRvd25sb2FkIHV0aWxpdHkgKGN1cmwsIHdnZXQpIg0KICAgIGZpDQp9DQoNCnJlYWRvbmx5IElERU5USVRZX1VSTD0iaHR0cDovLzE2OS4yNTQuMTY5LjI1NC8yMDE2LTA2LTMwL2R5bmFtaWMvaW5zdGFuY2UtaWRlbnRpdHkvZG9jdW1lbnQvIg0KcmVhZG9ubHkgVFJVRV9SRUdJT049JChnZXRfY29udGVudHMgIiRJREVOVElUWV9VUkwiIHwgYXdrIC1GXCIgJy9yZWdpb24vIHsgcHJpbnQgJDQgfScpDQpyZWFkb25seSBERUZBVUxUX1JFR0lPTj0idXMtZWFzdC0xIg0KcmVhZG9ubHkgUkVHSU9OPSIke1RSVUVfUkVHSU9OOi0kREVGQVVMVF9SRUdJT059Ig0KDQpyZWFkb25seSBTQ1JJUFRfTkFNRT0iYXdzLWluc3RhbGwtc3NtLWFnZW50Ig0KIFNDUklQVF9VUkw9Imh0dHBzOi8vYXdzLXNzbS1kb3dubG9hZHMtJFJFR0lPTi5zMy5hbWF6b25hd3MuY29tL3NjcmlwdHMvJFNDUklQVF9OQU1FIg0KDQppZiBbICIkUkVHSU9OIiA9ICJjbi1ub3J0aC0xIiBdOyB0aGVuDQogIFNDUklQVF9VUkw9Imh0dHBzOi8vYXdzLXNzbS1kb3dubG9hZHMtJFJFR0lPTi5zMy5jbi1ub3J0aC0xLmFtYXpvbmF3cy5jb20uY24vc2NyaXB0cy8kU0NSSVBUX05BTUUiDQpmaQ0KDQppZiBbICIkUkVHSU9OIiA9ICJ1cy1nb3Ytd2VzdC0xIiBdOyB0aGVuDQogIFNDUklQVF9VUkw9Imh0dHBzOi8vYXdzLXNzbS1kb3dubG9hZHMtJFJFR0lPTi5zMy11cy1nb3Ytd2VzdC0xLmFtYXpvbmF3cy5jb20vc2NyaXB0cy8kU0NSSVBUX05BTUUiDQpmaQ0KDQpjZCAvdG1wDQpGSUxFX1NJWkU9MA0KTUFYX1JFVFJZX0NPVU5UPTMNClJFVFJZX0NPVU5UPTANCg0Kd2hpbGUgWyAkUkVUUllfQ09VTlQgLWx0ICRNQVhfUkVUUllfQ09VTlQgXSA7IGRvDQogIGVjaG8gQVdTLVVwZGF0ZUxpbnV4QW1pOiBEb3dubG9hZGluZyBzY3JpcHQgZnJvbSAkU0NSSVBUX1VSTA0KICBnZXRfY29udGVudHMgIiRTQ1JJUFRfVVJMIiA+ICIkU0NSSVBUX05BTUUiDQogIEZJTEVfU0laRT0kKGR1IC1rIC90bXAvJFNDUklQVF9OQU1FIHwgY3V0IC1mMSkNCiAgZWNobyBBV1MtVXBkYXRlTGludXhBbWk6IEZpbmlzaGVkIGRvd25sb2FkaW5nIHNjcmlwdCwgc2l6ZTogJEZJTEVfU0laRQ0KICBpZiBbICRGSUxFX1NJWkUgLWd0IDAgXTsgdGhlbg0KICAgIGJyZWFrDQogIGVsc2UNCiAgICBpZiBbWyAkUkVUUllfQ09VTlQgLWx0IE1BWF9SRVRSWV9DT1VOVCBdXTsgdGhlbg0KICAgICAgUkVUUllfQ09VTlQ9JCgoUkVUUllfQ09VTlQrMSkpOw0KICAgICAgZWNobyBBV1MtVXBkYXRlTGludXhBbWk6IEZpbGVTaXplIGlzIDAsIHJldHJ5Q291bnQ6ICRSRVRSWV9DT1VOVA0KICAgIGZpDQogIGZpIA0KZG9uZQ0KDQppZiBbICRGSUxFX1NJWkUgLWd0IDAgXTsgdGhlbg0KICBjaG1vZCAreCAiJFNDUklQVF9OQU1FIg0KICBlY2hvIEFXUy1VcGRhdGVMaW51eEFtaTogUnVubmluZyBVcGRhdGVTU01BZ2VudCBzY3JpcHQgbm93IC4uLi4NCiAgLi8iJFNDUklQVF9OQU1FIiAtLXJlZ2lvbiAiJFJFR0lPTiINCmVsc2UNCiAgZWNobyBBV1MtVXBkYXRlTGludXhBbWk6IFVuYWJsZSB0byBkb3dubG9hZCBzY3JpcHQsIHF1aXR0aW5nIC4uLi4NCmZp
MinInstanceCount: 1
MaxInstanceCount: 3
IamInstanceProfileName: "{{InstanceIamRole}}"
- name: updateOSSoftware
action: aws:runCommand
maxAttempts: 3
timeoutSeconds: 3600
onFailure: Abort
inputs:
DocumentName: AWS-RunShellScript
InstanceIds:
- "{{launchInstance.InstanceIds}}"
Parameters:
commands:
- set -e
- '[ -x "$(which wget)" ] && get_contents=''wget $1 -O -'''
- '[ -x "$(which curl)" ] && get_contents=''curl -s -f $1'''
- eval $get_contents https://aws-ssm-downloads-{{global:REGION}}.s3.amazonaws.com/scripts/aws-update-linux-instance
> /tmp/aws-update-linux-instance
- chmod +x /tmp/aws-update-linux-instance
- "/tmp/aws-update-linux-instance --pre-update-script '{{PreUpdateScript}}'
--post-update-script '{{PostUpdateScript}}' --include-packages '{{IncludePackages}}'
--exclude-packages '{{ExcludePackages}}' 2>&1 | tee /tmp/aws-update-linux-instance.log"
- name: installCustomizations
action: aws:runCommand
maxAttempts: 3
timeoutSeconds: 600
onFailure: Abort
inputs:
DocumentName: AWS-RunShellScript
InstanceIds:
- "{{launchInstance.InstanceIds}}"
Parameters:
commands:
- curl -O http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
- rpm -ivh epel-release-latest-7.noarch.rpm
- yum -y install httpd
- systemctl enable httpd
- systemctl restart httpd
- sudo yum --enablerepo=epel install -y clamav
- yum-config-manager --disable epel
- cat /etc/motd >> /var/www/html/index.html
- echo 'Welcome' >> /var/www/html/index.html
- cat > /etc/motd <<- EOF
- " __ __ _ _ _ _ "
- " /\\ | \\/ (_) /\\ | | | | (_) "
- " / \\ | \\ / |_ / \\ _ _| |_ ___ _ __ ___ __ _| |_ _ ___
\ _ __ "
- " / /\\ \\ | |\\/| | | / /\\ \\| | | | __/ _ \\| '_ ` _ \\ / _` | __|
|/ _ \\| '_ \\ "
- " / ____ \\| | | | | / ____ \\ |_| | || (_) | | | | | | (_| | |_| | (_)
| | | |"
- " /_/ \\_\\_| |_|_| /_/ \\_\\__,_|\\__\\___/|_| |_| |_|\\__,_|\\__|_|\\___/|_|
|_|"
- " "
- " "
- EOF
- name: installInspectorAgent
action: aws:runCommand
maxAttempts: 3
timeoutSeconds: 600
onFailure: Abort
inputs:
DocumentName: AmazonInspector-ManageAWSAgent
InstanceIds:
- "{{launchInstance.InstanceIds}}"
Parameters:
Operation: Install
- name: installUnifiedCloudWatchAgent
action: aws:runCommand
maxAttempts: 3
timeoutSeconds: 600
onFailure: Abort
inputs:
DocumentName: AWS-ConfigureAWSPackage
InstanceIds:
- "{{launchInstance.InstanceIds}}"
Parameters:
name: AmazonCloudWatchAgent
action: Install
- name: stopInstance
action: aws:changeInstanceState
maxAttempts: 3
timeoutSeconds: 1200
onFailure: Abort
inputs:
InstanceIds:
- "{{launchInstance.InstanceIds}}"
DesiredState: stopped
- name: createImage
action: aws:createImage
maxAttempts: 3
onFailure: Abort
inputs:
InstanceId: "{{launchInstance.InstanceIds}}"
ImageName: "{{TargetAmiName}}"
NoReboot: true
ImageDescription: AMI Generated by EC2 Automation on {{global:DATE_TIME}} from
{{SourceAmiId}}
- name: createEncryptedCopy
action: aws:copyImage
maxAttempts: 3
onFailure: Abort
inputs:
SourceImageId: "{{createImage.ImageId}}"
SourceRegion: "{{global:REGION}}"
ImageName: Encrypted-{{TargetAmiName}}
ImageDescription: Encrypted GoldenAMI by SSM Automation on {{global:DATE_TIME}}
from source AMI {{createImage.ImageId}}
Encrypted: true
- name: createTagsForEncryptedImage
action: aws:createTags
maxAttempts: 1
onFailure: Continue
inputs:
ResourceType: EC2
ResourceIds:
- "{{createEncryptedCopy.ImageId}}"
Tags:
- Key: Automation-Id
Value: "{{automation:EXECUTION_ID}}"
- Key: Owner
Value: Mystique
- Key: SourceAMI
Value: "{{SourceAmiId}}"
- Key: Amazon-Inspector
Value: 'true'
- Key: Amazon-SSM
Value: 'true'
- Key: Encrypted
Value: 'true'
- name: updateSsmParam
action: aws:invokeLambdaFunction
timeoutSeconds: 1200
maxAttempts: 1
onFailure: Abort
inputs:
FunctionName: Automation-UpdateSsmParam
Payload: '{"parameterName":"/GoldenAMI/Linux/RedHat-7/latest", "parameterValue":"{{createEncryptedCopy.ImageId}}"}'
- name: terminateInstance
action: aws:changeInstanceState
maxAttempts: 3
onFailure: Continue
inputs:
InstanceIds:
- "{{launchInstance.InstanceIds}}"
DesiredState: terminated
- name: deleteUnEcryptedImage
action: aws:deleteImage
maxAttempts: 3
timeoutSeconds: 180
onFailure: Abort
inputs:
ImageId: "{{createImage.ImageId}}"
outputs:
- createImage.ImageId
I expect it to run and create a parameter store and a document inside AWS SSM.
The code you pasted above is a Systems Manager (SSM) document rather than a CloudFormation template. That's why CF complains Template format error.
Similar to CloudFormation template, the SSM Document supports both JSON and YAML formats.
As you can see from the README description in that same GitHub repository, the JSON file is used to create a SSM Document.
I want to get the content of a file in the UserData without providing inline content as following, but I'm getting the path of the file in the content when the ec2 boots instead of the content of the file.
Here's a snippet of my template:
ServiceInstance:
Type: "AWS::EC2::Instance"
Properties:
. . .
UserData:
'Fn::Base64': !Sub |
#cloud-config
write_files:
- path: /etc/sysconfig/cloudformation
permissions: 0644
owner: root
content: |
STACK_NAME=${AWS::StackName}
AWS_REGION=${AWS::Region}
- path: /etc/path-to-file/conf.yaml
permissions: 0644
owner: root
content: "#file://./config/conf-${Env}.yaml"
runcmd:
## run some commands
when I ssh to ec2 and check the file content I get this:
[ec2-user#ec2ip ~]$ cat /etc/path-to-file/conf.yaml
#file://./config/conf-dev.yaml
I checked this cloud init docs, but can't find something related.
any idea what did I do wrong in here ?
Encode the file content in base64 and pass it as argument. Cloud Init will decode the string b64.
Be careful of the cloudformation size limits on template and variables.
Parameters:
ConfContent:
Type: String
Description: "Conf content in base64 format."
Resources:
ServiceInstance:
Type: "AWS::EC2::Instance"
Properties:
. . .
UserData:
'Fn::Base64': !Sub
- |
#cloud-config
write_files:
- path: /etc/sysconfig/cloudformation
permissions: 0644
owner: root
content: |
STACK_NAME=${AWS::StackName}
AWS_REGION=${AWS::Region}
- path: /etc/path-to-file/conf.yaml
permissions: 0644
owner: root
content: ${CONF_CONTENT}
encoding: b64
runcmd:
## run some commands
- CONF_CONTENT: !Ref ConfContent
Then expose the file content as property :
aws cloudformation create-stack \
--stack-name "mystack" \
--template-body "template.yaml" \
--parameters \
ParameterKey=ConfContent,ParameterValue=\"$(base64 -w0 ./config/conf-dev.yaml)\"