I am using aws sam cli in version 1.40.1 and I notice that the way in which the layers are defined changed and that is why I can no longer import my code that is in that layer and I get the following error:
START RequestId: 35e6d7dc-99e8-4966-b51d-02cb8c39adf8 Version: $LATEST
Traceback (most recent call last): Unable to import module 'app': No module named 'Citas'
{"errorMessage": "Unable to import module 'app': No module named 'Citas'", "errorType": "Runtime.ImportModuleError", "stackTrace": []}END RequestId: 35e6d7dc-99e8-4966-b51d-02cb8c39adf8
REPORT RequestId: 35e6d7dc-99e8-4966-b51d-02cb8c39adf8 Init Duration: 0.22 ms Duration: 201.65 ms Billed Duration: 202 ms Memory Size: 128 MB Max Memory Used: 128 MB
I did not have this problem in version 0.53.0 of this tool and that was given that when building if you mount the layers without problems and could access the classes easily.
I share here the lambda configuration.
template.yml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
citas
Sample SAM Template for citas
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 3
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: hello_world/
Handler: app.lambda_handler
Runtime: python3.7
Layers:
- !Ref CitasLayer
CitasLayer:
Type: AWS::Serverless::LayerVersion
Properties:
Description: Capa que se encarga de la gestion de citas
ContentUri: layer
RetentionPolicy: Retain
CompatibleRuntimes:
- python3.7
tree
app.py
from Citas import say_hellow
# import requests
import glob
def lambda_handler(event, context):
return say_hellow()
If anyone has managed to work with this version and knows how things are done nowadays I would appreciate it, because the change was important.
Related
Given
root#kun4:/home/ubuntu/Igloo_Backend# sam --version
SAM CLI, version 1.62.0
root#kun4:/home/ubuntu/Igloo_Backend# nohup sam local start-api --log-file logfile.txt --host 0.0.0.0 --warm-containers EAGER >>output.log 2>&1 &
[1] 196068
The time execution is still long (13s, bot local request and remote request).
Here is template:
AWSTemplateFormatVersion: 2010-09-09
Description: BACKEND
Transform: AWS::Serverless-2016-10-31
Resources:
postingFunction:
Type: AWS::Serverless::Function
Properties:
Handler: src.posting
Runtime: nodejs16.x
Environment:
Variables:
NODE_ENV: lc
# Architectures:
# - x86_64
# MemorySize: 128
Timeout: 100
Events:
Api:
Type: Api
Properties:
Path: /credentials
Method: POST
Log:
START RequestId: 62315c0e-f4f9-47c7-bb72-2302f15ac6c4 Version: $LATEST
2022-11-11T08:36:42.012Z 62315c0e-f4f9-47c7-bb72-2302f15ac6c4 INFO ENTER
2022-11-11T08:36:42.053Z 62315c0e-f4f9-47c7-bb72-2302f15ac6c4 INFO Executing (default): SELECT "id", "username", "password", "encrypted", "createdAt", "updatedAt" FROM "users" AS "User" WHERE "User"."username" = 'a' LIMIT 1;2022-11-11T08:36:42.056Z 62315c0e-f4f9-47c7-bb72-2302f15ac6c4 INFO ERROR
2022-11-11T08:36:42.057Z 62315c0e-f4f9-47c7-bb72-2302f15ac6c4 INFO UserExistedError: a - a already existed^M at Object.module.exports.persistCredentials (/var/task/src/service/user.service.js:29:15)^M at processTicksA>END RequestId: 62315c0e-f4f9-47c7-bb72-2302f15ac6c4
REPORT RequestId: 62315c0e-f4f9-47c7-bb72-2302f15ac6c4 Init Duration: 0.12 ms Duration: 10362.78 ms Billed Duration: 10363 ms Memory Size: 128 MB Max Memory Used: 128 MB
Please suggest your idea, thank you
I've debugged my application, and identified a problem. I have 2 REST API Gateway, and it seems like since they both bind on the same endpoint, the first one will recieve the call that the second one should handle.
Here's my template.yaml
Resources:
mysampleapi1:
Type: 'AWS::Serverless::Function'
Properties:
Handler: packages/mysampleapi1/dist/index.handler
Runtime: nodejs14.x
CodeUri: .
Description: ''
MemorySize: 1024
Timeout: 30
Role: >-
arn:aws:iam:: [PRIVATE]
Events:
Api1:
Type: Api
Properties:
Path: /users
Method: ANY
Environment:
Variables:
NODE_ENV: local
Tags:
STAGE: local
mysampleapi2:
Type: 'AWS::Serverless::Function'
Properties:
Handler: packages/mysampleapi2/dist/index.handler
Runtime: nodejs14.x
CodeUri: .
Description: ''
MemorySize: 1024
Timeout: 30
Role: >-
arn:aws:iam:: [PRIVATE]
Events:
Api1:
Type: Api
Properties:
Path: /wallet
Method: ANY
Environment:
Variables:
NODE_ENV: local
Tags:
STAGE: local
When I send a HTTP request for mysampleapi2
Here's what's happening in the logs using the startup command
sam local start-api --port 3001 --log-file /tmp/server-output.log --profile personal --debug
2022-04-27 18:2:34,953 | Mounting /home/mathieu_auclair/Documents/Project/repositories/server as /var/task:ro,delegated inside runtime container
2022-04-27 18:20:35,481 | Starting a timer for 30 seconds for function 'mysampleapi1'
2022-04-27 18:21:05,484 | Function 'mysampleapi1' timed out after 30 seconds
2022-04-27 18:21:46,732 | Container was not created. Skipping deletion
2022-04-27 18:21:46,732 | Cleaning all decompressed code dirs
2022-04-27 18:21:46,733 | No response from invoke container for mysampleapi1
2022-04-27 18:21:46,733 | Invalid lambda response received: Lambda response must be valid json
Why is my mysampleapi2 not picking the HTTP call? If I run them in separate template files using different ports, then it works... why is that?
After launching my lambda in separate processes, I discovered that there's an issue in my configuration for the second service.
The issue still occured after this launcher
echo "" > /tmp/server-output-1.log
sam local start-api --port 3001 --log-file /tmp/server-output-1.log --template .template.1.yaml --debug &
tail -f /tmp/server-output-1.log &
echo "" > /tmp/server-output-2.log
sam local start-api --port 3002 --log-file /tmp/server-output-2.log --template .template.2.yaml --debug &
tail -f /tmp/server-output-2.log &
I noticed when I exported my configuration, for one of the services, there's the following in the template.yaml
Path: '/{proxy+}'
without the proxy line, the lambda handler just never gets called for some reason
I am trying to invoke a lambda locally with sam local invoke. The function invokes fine but my environment variables for my secrets are not resolving. The secrets resolve as expected when you deploy the function. But I want to avoid my local code and my deployed code being any different. So is there a way to resolve those secrets to the actual secret value at the time of invoking locally? Currently I am getting just the string value from the environment variable. Code below.
template.yaml
# This is the SAM template that represents the architecture of your serverless application
# https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-template-basics.html
# The AWSTemplateFormatVersion identifies the capabilities of the template
# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/format-version-structure.html
AWSTemplateFormatVersion: 2010-09-09
Description: >-
onConnect
# Transform section specifies one or more macros that AWS CloudFormation uses to process your template
# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/transform-section-structure.html
Transform:
- AWS::Serverless-2016-10-31
# Resources declares the AWS resources that you want to include in the stack
# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resources-section-structure.html
Resources:
# Each Lambda function is defined by properties:
# https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
# This is a Lambda function config associated with the source code: hello-from-lambda.js
helloFromLambdaFunction:
Type: AWS::Serverless::Function
Properties:
Handler: src/handlers/onConnect.onConnect
Runtime: nodejs14.x
MemorySize: 128
Timeout: 100
Environment:
Variables:
WSS_ENDPOINT: '{{resolve:secretsmanager:prod/wss/api:SecretString:endpoint}}'
onConnect.js
/**
* A Lambda function that returns a static string
*/
exports.onConnect = async () => {
const endpoint = process.env.WSS_ENDPOINT;
console.log(endpoint);
// If you change this message, you will need to change hello-from-lambda.test.js
const message = 'Hellddfdsfo from Lambda!';
// All log statements are written to CloudWatch
console.info(`${message}`);
return message;
}
I came up with a work around that will allow me to have one code base and "resolve" secrets/parameters locally.
I created a very basic lambda layer who's only job is fetching secrets if the environment is set to LOCAL.
import boto3
def get_secret(env, type, secret):
client = boto3.client('ssm')
if env == 'LOCAL':
if type == 'parameter':
return client.get_parameter(
Name=secret,
)['Parameter']['Value']
else:
return secret
I set the environment with a parameter in the lambda that will be calling this layer. BTW this layer will resolve more than one secret eventually so that's why the nested if might look a little strange. This is how I set the environment:
Resources:
...
GetWSSToken:
Type: AWS::Serverless::Function
Properties:
FunctionName: get_wss_token
CodeUri: get_wss_token/
Handler: app.lambda_handler
Runtime: python3.7
Timeout: 30
Layers:
- arn:aws:lambda:********:layer:SecretResolver:8
Environment:
Variables:
ENVIRONMENT: !Ref Env
JWT_SECRET: !FindInMap [ Map, !Ref Env, jwtsecret ]
...
Mappings:
Map:
LOCAL:
jwtsecret: jwt_secret
PROD:
jwtsecret: '{{resolve:ssm:jwt_secret}}'
STAGING:
jwtsecret: '{{resolve:ssm:jwt_secret}}'
Parameters:
...
Env:
Type: String
Description: Environment this lambda is being run in.
Default: LOCAL
AllowedValues:
- LOCAL
- PROD
- STAGING
Now I can simply call the get_secret method in my lambda and depending on what I set Env to the secret will either be fetched at runtime or returned from the environment variables.
import json
import jwt
import os
from datetime import datetime, timedelta
from secret_resolver import get_secret
def lambda_handler(event, context):
secret = get_secret(os.environ['ENVIRONMENT'], 'parameter', os.environ['JWT_SECRET'])
two_hours_from_now = datetime.now() + timedelta(hours=2)
encoded_jwt = jwt.encode({"expire": two_hours_from_now.timestamp()}, secret, algorithm="HS256")
return {
"statusCode": 200,
"body": json.dumps({
"token": encoded_jwt
}),
}
I hope this helps someone out there trying to figure this out. The main issue here is keeping the secrets out of the code base and be able to test locally with the same code that's going into production.
I'm trying to execute aws sam local invoke command in order to test my function locally. The problem occurs when I run this command passing profile parameter. The execution container sets account id as a dummy account like 123456789012
Does anybody knows If there is any thing cached here? When I deploy my project using sam deploy --guided everything runs perfect and code is uploaded to AWS
my ~/.aws/config and ~/.aws/credentials files are filled
credentials
[adm]
aws_access_key_id=XXXXXXXXXXXXXx
aws_secret_access_key=XXXXXXXXXXXXXXXXXX
config
[adm]
region = eu-west-1
output = json
➜ POC sam local invoke "ScheduleBusinessRuleFunction" --profile adm
Invoking scheduleBusinessRule (go1.x)
Skip pulling image and use local one: amazon/aws-sam-cli-emulation-image-go1.x:rapid-1.18.1.
Mounting /run/media/urkob/projects/Projects/POC/.aws-sam/build/ScheduleBusinessRuleFunction as /var/task:ro,delegated inside runtime container
START RequestId: bc99de19-290b-46fa-908a-744d54547cbe Version: $LATEST
operation error CloudWatch Events: PutTargets, https response error StatusCode: 400, RequestID: 8e335486-de38-49bb-a06e-8090e26e9628, api error AccessDeniedException: Access to the resource arn:aws:lambda:us-east-1:123456789012:function:POC-executeBusinessRule is denied. Reason: Adding cross-account target is not permitted.: OperationError
null
END RequestId: bc99de19-290b-46fa-908a-744d54547cbe
REPORT RequestId: bc99de19-290b-46fa-908a-744d54547cbe Init Duration: 0.14 ms Duration: 1415.60 ms Billed Duration: 1500 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"errorMessage":"operation error CloudWatch Events: PutTargets, https response error StatusCode: 400, RequestID: 8e335486-de38-49bb-a06e-8090e26e9628, api error AccessDeniedException: Access to the resource arn:aws:lambda:us-east-1:123456789012:function:POC-executeBusinessRule is denied. Reason: Adding cross-account target is not permitted.","errorType":"OperationError"}%
➜ POC
UPDATED: 21/02/16
I've been searching what could be the problem. I'm using aws-sdk-go-v2 the example they exposed in ther official github documentation to get an instance of configuration:
I tryied two ways of getting config instance as you can see in this piece of code, the line commented and the line that comes after that is not commented.
import (
"context"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/cloudwatchevents"
)
//NewCloudWatchService scheduler service
func NewCloudWatchService() CWEPutEventsAPI {
cfg, err := config.LoadDefaultConfig(context.TODO())
//Happens the same in local with this line
// cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithSharedConfigFiles(config.DefaultSharedCredentialsFiles))
if err != nil {
panic("configuration error, " + err.Error())
}
return cloudwatchevents.NewFromConfig(cfg)
}
This is my template.yaml file. I'm developing usin AWS SAM yaml templates. As you can see I reference ExecuteLambda in ScheduleLambda using GettAtt method but when the code runs in local the region and accountId are always the same: default values.
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Globals:
Function:
Timeout: 5
Resources:
ScheduleBusinessRuleFunction:
Type: AWS::Serverless::Function
Properties:
FunctionName: POC-scheduleBusinessRule
CodeUri: functions/scheduleBusinessRule/
Handler: scheduleBusinessRule
Runtime: go1.x
MemorySize: 128
Timeout: 10
Tracing: Active
Role: !GetAtt SchedulerRole.Arn
Environment:
Variables:
LAMBDA_ARN: !GetAtt ExecuteBusinessRuleFunction.Arn
EVENT_BUS: "default"
ExecuteBusinessRuleFunction:
Type: AWS::Serverless::Function
Properties:
FunctionName: POC-executeBusinessRule
CodeUri: functions/executeBusinessRule/
Handler: executeBusinessRule
Runtime: go1.x
MemorySize: 128
Tracing: Active
I am writing a cloudformation template for creating an aws step function and statemachine. Following is the part of my template which is causing the error
AWSTemplateFormatVersion: 2010-09-09
Transform:
- StepFunctionsYamlTransform
StepFunctionsStateMachine:
Type: AWS::StepFunctions::StateMachine
Properties:
StateMachineName: MyStack
RoleArn: !GetAtt StateMachineRole.Arn
DefinitionStringYaml: !Sub
- |
Comment: My-Stack-workflow
StartAt: LambdaToStart
TimeoutSeconds: 43200
States:
LambdaToStart:
Type: Task
Resource: "${LambdaToStartArn}"
Next: WaitToWriteInS3
WaitToWriteInS3:
Type: Wait
Seconds: 5
Next: Batch_Job_1
Batch_Job_1:
Type: Task
Next: LambdaForTriggerEmrJob
Resource: arn:aws:states:::batch:submitJob.sync
Parameters:
JobName: "${BatchJob1}"
JobDefinition: "${BatchJob1DefinitionArn}"
JobQueue: arn:aws:batch:${AWS::Region}:${AWS::AccountId}:job-queue/${QueueName}
LambdaForTriggerEmrJob:
Type: Task
Resource: "${LambdaForEmrArn}"
Next: WaitFoEmrState
WaitFoEmrState:
Type: Wait
Seconds: 90
Next: CheckEmrState
CheckEmrState:
Type: Task
Resource: "${ClusterStateCheckArn}"
InputPath: "$.input.cluster" # Values coming from lambda
ResultPath: "$.input.cluster" # Values coming from lambda
Retry: *LambdaRetryConfig
Next: IsClusterRunning
IsClusterRunning:
Type: Choice
Default: WaitFoEmrState
Choices:
- Variable: "$.input.cluster.state"
StringEquals: FAILED
Next: StateMachineFailure
- Variable: "$.input.cluster.state" # Values coming from lambda
StringEquals: SUCCEEDED
Next: FinalBatchJob
StateMachineFailure:
Type: Fail
FinalBatchJob:
Type: Task
Resource: arn:aws:states:::batch:submitJob.sync
Parameters:
JobName: "${FinalBatch}"
JobDefinition: "${FinalBatchDefinitionArn}"
JobQueue: arn:aws:batch:${AWS::Region}:${AWS::AccountId}:job-queue/${QueueName
End: true
- LambdaToStartArn: !GetAtt LambdaToStart.Arn
LambdaForEmrArn: !GetAtt LambdaForEmr.Arn
BatchJob1DefinitionArn: !Ref BatchJob1Definition
FinalBatchDefinitionArn: !Ref FinalBatchDefinition
BatchJob1: !Sub ${AWS::StackName}-batch-1
FinalBatch: !Sub ${AWS::StackName}-final-batch
ClusterStateCheckArn: !Sub arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:cluster-state
It returns the following error
Failed to create the changeset: Waiter ChangeSetCreateComplete failed:
Waiter encountered a terminal failure state Status: FAILED. Reason:
Transform {AWS::AccountId}::StepFunctionsYamlTransform failed without an
error message.
Can anyone help in recognising the solution to this? I can't debug a lot since it fails without an error message. TIA
AWS cloudformation errors are sometimes quite wierd and its difficult to debug them. But I found the error. It was 9th line JobQueue: arn:aws:batch:${AWS::Region}:${AWS::AccountId}:job-queue/${QueueName and one can easily see that I missed } at the end. So it was a syntax error