I'm attempting to create an S3 bucket with serverless, which works, however in order to manipulate files in it I need a bucket policy. I'm having a hard time understanding where and how to add a policy that uses the generated S3bucket name created when serverless deploys for the first time
##serverless.yml##
service: vcc-nametags-api
# Use the serverless-webpack plugin to transpile ES6
plugins:
- serverless-webpack
- serverless-offline
- serverless-ding
# serverless-webpack configuration
# Enable auto-packing of external modules
custom:
# Our stage is based on what is passed in when running serverless
# commands. Or fallsback to what we have set in the provider section.
stage: ${opt:stage, self:provider.stage}
# Set our DynamoDB throughput for prod and all other non-prod stages.
# Load our webpack config
webpack:
webpackConfig: ./webpack.config.js
includeModules: true
environment: ${file(env.yml):${self:custom.stage}, file(env.yml):default}
provider:
name: aws
runtime: nodejs8.10
stage: dev
region: us-east-1
# These environment variables are made available to our functions
# under process.env.
environment:
S3DBBucketName:
Ref: NametagsDatabaseBucket
functions:
# Defines an HTTP API endpoint that calls the main function in create.js
# - path: url path is /tags
# - method: POST request
# - cors: enabled CORS (Cross-Origin Resource Sharing) for browser cross
# domain api call
# - authorizer: authenticate using the AWS IAM role
create:
handler: create.main
events:
- http:
path: tags
method: post
cors: true
get:
# Defines an HTTP API endpoint that calls the main function in get.js
# - path: url path is /tags/{id}
# - method: GET request
handler: get.main
events:
- http:
path: tags/{id}
method: get
cors: true
list:
# Defines an HTTP API endpoint that calls the main function in list.js
# - path: url path is /tags
# - method: GET request
handler: list.main
events:
- http:
path: tags
method: get
cors: true
update:
# Defines an HTTP API endpoint that calls the main function in update.js
# - path: url path is /tags/{id}
# - method: PUT request
handler: update.main
events:
- http:
path: tags/{id}
method: put
cors: true
delete:
# Defines an HTTP API endpoint that calls the main function in delete.js
# - path: url path is /tags/{id}
# - method: DELETE request
handler: delete.main
events:
- http:
path: tags/{id}
method: delete
cors: true
# Create our resources with separate CloudFormation templates
resources:
# S3DB
- ${file(resources/s3-database.yml)}
##s3-database.yml##
Resources:
NametagsDatabaseBucket:
Type: AWS::S3::Bucket
Properties:
# Set the CORS policy
CorsConfiguration:
CorsRules:
-
AllowedOrigins:
- '*'
AllowedHeaders:
- '*'
AllowedMethods:
- GET
- PUT
- POST
- DELETE
- HEAD
MaxAge: 3000
NametagsDatabaseBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket:
Ref: NametagsDatabaseBucket
PolicyDocument:
Statement:
- Sid: PublicReadGetObject
Effect: Allow
Principal: "*"
Action:
- "s3:DeleteObject"
- "s3:GetObject"
- "s3:ListBucket"
- "s3:PutObject"
Resource:
Fn::Join: [
"", [
"arn:aws:s3:::",
{
"Ref": "NametagsDatabaseBucket"
},
"/*"
]
]
# Print out the name of the bucket that is created
Outputs:
NametagsDatabaseBucketName:
Value:
Ref: NametagsDatabaseBucket
I've tried various combinations I've found on the internet as well as adding it to an iamroles property in the serverless.yml file but I can't seem to get anything to work
The Resource Reference Name seems to matter, I have always had to use the name of the bucket in the resource name. For example, a bucket with www.example.com needs a reference name of S3BucketWwwexamplecom.
However I also notice that the BucketName element is missing from your example.
This is from working example for a static website with a Bucket Policy:
resources:
Resources:
S3BucketWwwexamplecom:
Type: AWS::S3::Bucket
DeletionPolicy: Delete
Properties:
BucketName: ${self:custom.s3WwwBucket}
CorsConfiguration:
CorsRules:
- AllowedMethods:
- PUT
- GET
- POST
- HEAD
AllowedOrigins:
- "https://${self:custom.myDomain}"
AllowedHeaders:
- "*"
AccessControl: PublicRead
WebsiteConfiguration:
IndexDocument: index.html
BucketPolicyWwwexamplecom:
Type: 'AWS::S3::BucketPolicy'
Properties:
PolicyDocument:
Statement:
- Sid: PublicReadForGetBucketObjects
Effect: Allow
Principal: '*'
Action:
- 's3:GetObject'
Resource: arn:aws:s3:::${self:custom.s3WwwBucket}/*
Bucket:
Ref: S3BucketWwwexamplecom
Since you are using a lambda to upload you should create an IAM Role for your Lambda and an IAM Policy with only the permissions required for operation. You might accomplish this by using the following excerpt in your cloud formation:
AWSTemplateFormatVersion: '2010-09-09'
Description: My Template
Resources:
LambdaRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
RoleName: !Sub ${AWS::StackName}-LambdaRole
S3Policy:
Type: AWS::IAM::Policy
Properties:
PolicyName: S3_Writer
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- s3:*
Resource: !Sub
- arn:aws:s3:::${BucketName}/*
- BucketName: !Ref NametagsDatabaseBucket
Roles:
- !Ref TaskRole
Outputs:
LambdaRole:
Value: !Sub "${LambdaRole.Arn}"
Export:
Name: !Sub ${AWS::StackName}-LambdaRole
Then in your serverless.yml just refer to the task role created using something like this to reference the execution role:
service: vcc-nametags-api
provider:
role: ${cf:${env:YOUR_STACK_ENV, 'YOUR_STACK_NAME'}.LambdaRole}
We have a setup like this working in several projects, I hope it works for you.
Related
I'm trying to create a admin API using api gateway, lambda and dynamoDB. I want only the logged in users in certain group be able to access lambda.
I see that the lambda would not assume role of the cognito user and access the dynamoDB. API only works if I attach the dynamoDB policy to the LambdaRole.
Is there a way for this to work dynamically based on user groups. Do I need to add this assumeRole logic in the lambda handler itself ? Please help
This is the ERROR I get when I git my API from postman with Authorization tokens
{
"message": "User: arn:aws:sts::09723XXXXX357:assumed-role/tango-admin-lambda-role/tango-admin-service-dev-getActiveOrders is not authorized to perform: dynamodb:Query on resource: arn:aws:dynamodb:us-east-1:09723XXXXX357:table/dev-tango-service-tango-order-db",
"code": "AccessDeniedException",
"time": "2022-05-23T18:42:39.241Z",
"requestId": "DDK7AS6SBO8D0U3KXXXXXXXX",
"statusCode": 400,
"retryable": false,
"retryDelay": 46.2351059972586
}
Here is my serverless.yml file
service: tango-admin-service
provider:
name: aws
runtime: nodejs14.x
region: us-east-1
environment:
DYNAMODB_TABLE: ${opt:stage, self:provider.stage}-tango-service-tango-order-db
# functions
functions:
hello:
handler: src/handlers/hello.handler
events:
- http:
path: hello
method: get
getActiveOrders:
handler: src/handlers/getActiveOrders.handler
role: TangoAdminLambdaRole # How can this role be assume from Congito ? It only works if I directly pass the Role with DynamoDB Policy
events: # The events that trigger this function
- http:
path: get-active-orders
method: post
cors: true
authorizer:
type: COGNITO_USER_POOLS
authorizerId:
Ref: TangoAdminApiAuthorizer
# Serverless plugins
plugins:
- serverless-plugin-typescript
- serverless-offline
resources:
Resources:
TangoAdminCognitoUserPool:
Type: AWS::Cognito::UserPool
Properties:
UserPoolName: TangoAdmin
TangoAdminCognitoUserPoolClient:
Type: AWS::Cognito::UserPoolClient
Properties:
ClientName: TangoAdminWebApp
GenerateSecret: false
UserPoolId:
Ref: "TangoAdminCognitoUserPool"
TangoAdminGroup:
Type: AWS::Cognito::UserPoolGroup
Properties:
GroupName: "TangoAdmin"
Description: "Admin group for Tango"
Precedence: 0
RoleArn: !GetAtt CognitoAdminIAMRole.Arn
UserPoolId:
Ref: "TangoAdminCognitoUserPool"
CognitoAdminIAMRole:
Type: AWS::IAM::Role
Properties:
RoleName: "tango-admin-role"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Federated:
- "cognito-identity.amazonaws.com"
Action:
- "sts:AssumeRoleWithWebIdentity"
Policies:
- PolicyName: "tango-admin-group-policy"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- "dynamodb:*"
Resource:
- "arn:aws:dynamodb:${opt:region, self:provider.region}:*:table/${self:provider.environment.DYNAMODB_TABLE}"
TangoAdminApiAuthorizer:
Type: AWS::ApiGateway::Authorizer
Properties:
Name: TangoAdmin
RestApiId:
Ref: ApiGatewayRestApi
Type: COGNITO_USER_POOLS
ProviderARNs:
- Fn::GetAtt: [ TangoAdminCognitoUserPool, Arn ]
IdentitySource: method.request.header.Authorization
TangoAdminLambdaRole:
Type: AWS::IAM::Role
Properties:
RoleName: "tango-admin-lambda-role"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action:
- "sts:AssumeRole"
Outputs:
TangoAdminCognitoUserPoolId:
Description: "Tango Admin Cognito User Pool ID"
Value:
Ref: "TangoAdminCognitoUserPool"
TangoAdminCognitoUserPoolClientId:
Description: "Tango Admin Cognito User Pool Client ID"
Value:
Ref: "TangoAdminCognitoUserPoolClient"
I have a CloudFormation template set up to track a CloudFront distribution among other things. Getting this set up, I created an AWS::CertificateManager::Certificate and an AWS::CloudFront::Distribution resource, where the CDN just serves from a non-website S3 origin.
When I run the change set, I get this incredibly vague failure.
"Access denied for operation 'AWS::CloudFront::Distribution'." kind of loses me here. For one thing, it's not clear to me what operation this is supposed to be. On top of that, the stack rollback after this is incomplete. The CloudFormation events don't even show an attempt to remove the CDN or the cert, and when I try to hit the CloudFront URL from my browser, it works flawlessly, so I am not even sure what my template was trying to do here that failed. In fact, the only reason this is an issue for me is because the incomplete rollback tries to revert my lambdas in the stack to nodejs8.10, which causes larger failures. If that weren't an issue, I don't know that I would feel the effects of this vague error.
Template, based on the static site sample from a couple of years ago:
AWSTemplateFormatVersion: 2010-09-09
Transform:
- AWS::Serverless-2016-10-31
- AWS::CodeStar
Parameters:
ProjectId:
Type: String
Description: AWS CodeStar projectID used to associate new resources to team members
CodeDeployRole:
Type: String
Description: IAM role to allow AWS CodeDeploy to manage deployment of AWS Lambda functions
Stage:
Type: String
Description: The name for a project pipeline stage, such as Staging or Prod, for which resources are provisioned and deployed.
Default: ''
Globals:
Api:
BinaryMediaTypes:
- image~1png
Function:
Runtime: nodejs14.x
AutoPublishAlias: live
DeploymentPreference:
Enabled: true
Type: Canary10Percent5Minutes
Role: !Ref CodeDeployRole
Resources:
MahCert:
Type: AWS::CertificateManager::Certificate
Properties:
DomainName: domain.com
DomainValidationOptions:
- DomainName: domain.com
HostedZoneId: Z2GZX5ZQI1HO5L
SubjectAlternativeNames:
- '*.domain.com'
CertificateTransparencyLoggingPreference: ENABLED
ValidationMethod: DNS
CloudFrontCDN:
Type: AWS::CloudFront::Distribution
Properties:
DistributionConfig:
Comment: Source for all static resources
PriceClass: PriceClass_100
Aliases:
- domain.com
ViewerCertificate:
AcmCertificateArn: !Ref MahCert
MinimumProtocolVersion: TLSv1.2_2021
SslSupportMethod: sni-only
DefaultRootObject: index.html
DefaultCacheBehavior:
ViewerProtocolPolicy: redirect-to-https
CachePolicyId: b2884449-e4de-46a7-ac36-70bc7f1ddd6d
TargetOriginId: SiteBucket
Enabled: True
Origins:
- DomainName: <my_bucket>.s3.amazonaws.com
Id: SiteBucket
S3OriginConfig:
OriginAccessIdentity: ''
ServerlessRestApi:
Type: AWS::Serverless::Api
Properties:
StageName: Prod
DefinitionBody:
swagger: 2.0
info:
title: Static resource proxy
paths:
/static/{proxy+}:
get:
x-amazon-apigateway-integration:
httpMethod: ANY
type: http_proxy
uri: <my_bucket>.s3.amazonaws.com/static/{proxy}
responses: {}
GetHelloWorld:
Type: AWS::Serverless::Function
Properties:
Handler: index.get
Role:
Fn::GetAtt:
- LambdaExecutionRole
- Arn
Events:
GetEvent:
Type: Api
Properties:
Path: /
Method: get
ProxyEvent:
Type: Api
Properties:
Path: /{proxy+}
Method: any
GetStaticContent:
Type: AWS::Serverless::Function
Properties:
Handler: index.getResource
Role:
Fn::GetAtt:
- LambdaExecutionRole
- Arn
Events:
GetResourceEvent:
Type: Api
Properties:
Path: /static/{folder}/{file}
Method: get
GetQuote:
Type: AWS::Serverless::Function
Properties:
Handler: index.getQuote
Role:
Fn::GetAtt:
- LambdaDynamoDBReadRole
- Arn
Events:
GetRandomQuoteEvent:
Type: Api
Properties:
Path: /getquote
Method: get
GetQuoteEvent:
Type: Api
Properties:
Path: /getquote/{id}
Method: get
LambdaExecutionRole:
Description: Creating service role in IAM for AWS Lambda
Type: AWS::IAM::Role
Properties:
RoleName: !Sub 'CodeStar-${ProjectId}-Execution${Stage}'
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [lambda.amazonaws.com]
Action: sts:AssumeRole
Path: /
ManagedPolicyArns:
- !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'
PermissionsBoundary: !Sub 'arn:${AWS::Partition}:iam::${AWS::AccountId}:policy/CodeStar_${ProjectId}_PermissionsBoundary'
LambdaDynamoDBReadRole:
Description: Creating service role in IAM for AWS Lambda
Type: AWS::IAM::Role
Properties:
RoleName: !Sub '${ProjectId}-DynamoDB-Read'
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [lambda.amazonaws.com]
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: "dynamodb-read-quotes"
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action:
- "dynamodb:GetItem"
- "dynamodb:DescribeTable"
Resource: "<dynamo_arn>"
ManagedPolicyArns:
- !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'
Note - domain.com is not the actual domain I'm using here.
Updates:
I deleted the stack completely and recreated it from this template, thinking something was wrong with the history of the stack. However, I got the same error.
The IAM role that the stack uses has these permissions
Indeed, the problem persisted even after granting this role full write access to CloudFront resources.
Based on the chat discussion.
The cause of the issue was found to be missing IAM permissions for the IAM role that is used to deploy the stack. Specifically, the permission that was missing was:
cloudfront:GetDistribution - Grants permission to get the information about a web distribution
Adding that permission to the role, solved the problem.
To find the missing permission, CloudTrial's Event History was used.
I am using a serverless framework to deploy a serverless stack to AWS. My stack consists of some lambda functions, DynamoDB tables and API Gateway.
I am protected The API Gateway using what's called lambda authorizer. Also, I have a custom standalone self-hosted Auth service that can generate tokens.
So the scenario is that the user can request a token from this service (It's IdentityServer4 hosted on Azure) then the user can send a request to the API Gateway with the bearer token so the API gateway will ask the lambda authorizer to generate iam roles if the token is correct. All of that is valid and works as expected.
Here is an example of the lambda authorizer definition in my serverless.yml and how I use it to protect other API gateway endpoints: (You can see the addUserInfo function has API that protected using the custom authorizer )
functions:
# =================================================================
# API Gateway event handlers
# ================================================================
auth:
handler: api/auth/mda-auth-server.handler
addUserInfo:
handler: api/user/create-replace-user-info.handler
description: Create Or Replace user section
events:
- http:
path: user
method: post
authorizer:
name: auth
resultTtlInSeconds: ${self:custom.resultTtlInSeconds}
identitySource: method.request.header.Authorization
type: token
cors:
origin: '*'
headers: ${self:custom.allowedHeaders}
Now I wanted to extend my APIs so I will allow the user to add images, so I followed this approach. So in this approach, the user will initiate what's called a signed S3 URL and I can put an image to my bucket using this S3 signed URL.
Also, the S3 bucket is not publicly accessible but instead, it's connected to CloudFront distribution. Now I missed the things here, I can't understand how I can protect my images. Is it anyway so I can protect the Images in the CloudFront CDN with my custom Authentication service so the user that has a valid token can just access those resources? How can I protect my CDN (CloudFront) using my Custom Authentication service and configure that using the serverless framework?
This is a bit tricky and it takes from me around a day to get all set.
First we have options here:
Instead of authentication we can sign the URL and return a signed CloudFront URL or signed S3 URL and it's pretty straight forward but obviously that not what I was looking for.
The second option is to use Lambda#Edge to authorize the requests of the CloudFront and that what I followed.
So I ended up create a separate stack to handle all the S3, CloudFront, and Lambda#Edge stuff cause they are all deployed on edges which means that the region doesn't matter but for lambda edge we need to deploy it to the main AWS region ((N. Virginia), us-east-1) So i ended up creating one stack for all of them.
First I have the below code in my auth-service.js (It's just some helpers to allow me to verify my custom jwt):
import * as jwtDecode from 'jwt-decode';
import * as util from 'util';
import * as jwt from 'jsonwebtoken';
import * as jwksClient from 'jwks-rsa';
export function getToken(bearerToken) {
if(bearerToken && bearerToken.startsWith("Bearer "))
{
return bearerToken.replace(/^Bearer\s/, '');
}
throw new Error("Invalid Bearer Token.");
};
export function getDecodedHeader(token) {
return jwtDecode(token, { header: true });
};
export async function getSigningKey(decodedJwtTokenHeader, jwksclient){
const key = await util.promisify(jwksclient.getSigningKey)(decodedJwtTokenHeader.kid);
const signingKey = key.publicKey || key.rsaPublicKey;
if (!signingKey) {
throw new Error('could not get signing key');
}
return signingKey;
};
export async function verifyToken(token,signingKey){
return await jwt.verify(token, signingKey);
};
export function getJwksClient(jwksEndpoint){
return jwksClient({
cache: true,
rateLimit: true,
jwksRequestsPerMinute: 10,
jwksUri: jwksEndpoint
});
};
Then inside the serverless.yml here is my file:
service: mda-app-uploads
plugins:
- serverless-offline
- serverless-pseudo-parameters
- serverless-iam-roles-per-function
- serverless-bundle
custom:
stage: ${opt:stage, self:provider.stage}
resourcesBucketName: ${self:custom.stage}-mda-resources-bucket
resourcesStages:
prod: prod
dev: dev
resourcesStage: ${self:custom.resourcesStages.${self:custom.stage}, self:custom.resourcesStages.dev}
provider:
name: aws
runtime: nodejs12.x
stage: ${opt:stage, 'dev'}
region: us-east-1
versionFunctions: true
functions:
oauthEdge:
handler: src/mda-edge-auth.handler
role: LambdaEdgeFunctionRole
memorySize: 128
timeout: 5
resources:
- ${file(resources/s3-cloudfront.yml)}
Quick points here:
The us-east-1 important here.
It's a bit tricky and not practical to create any lambda edge using the serverless framework so I used it to just configure the function and then inside this cloud formation template resources/s3-cloudfront.yml I added all the needed bits.
Then here is the content of resources/s3-cloudfront.yml:
Resources:
AuthEdgeLambdaVersion:
Type: Custom::LatestLambdaVersion
Properties:
ServiceToken: !GetAtt PublishLambdaVersion.Arn
FunctionName: !Ref OauthEdgeLambdaFunction
Nonce: "Test"
PublishLambdaVersion:
Type: AWS::Lambda::Function
Properties:
Handler: index.handler
Runtime: nodejs12.x
Role: !GetAtt PublishLambdaVersionRole.Arn
Code:
ZipFile: |
const {Lambda} = require('aws-sdk')
const {send, SUCCESS, FAILED} = require('cfn-response')
const lambda = new Lambda()
exports.handler = (event, context) => {
const {RequestType, ResourceProperties: {FunctionName}} = event
if (RequestType == 'Delete') return send(event, context, SUCCESS)
lambda.publishVersion({FunctionName}, (err, {FunctionArn}) => {
err
? send(event, context, FAILED, err)
: send(event, context, SUCCESS, {FunctionArn})
})
}
PublishLambdaVersionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Policies:
- PolicyName: PublishVersion
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action: lambda:PublishVersion
Resource: '*'
LambdaEdgeFunctionRole:
Type: "AWS::IAM::Role"
Properties:
Path: "/"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Sid: "AllowLambdaServiceToAssumeRole"
Effect: "Allow"
Action:
- "sts:AssumeRole"
Principal:
Service:
- "lambda.amazonaws.com"
- "edgelambda.amazonaws.com"
LambdaEdgeFunctionPolicy:
Type: "AWS::IAM::Policy"
Properties:
PolicyName: MainEdgePolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
Effect: "Allow"
Action:
- "lambda:GetFunction"
- "lambda:GetFunctionConfiguration"
Resource: !GetAtt AuthEdgeLambdaVersion.FunctionArn
Roles:
- !Ref LambdaEdgeFunctionRole
ResourcesBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: ${self:custom.resourcesBucketName}
AccessControl: Private
CorsConfiguration:
CorsRules:
- AllowedHeaders: ['*']
AllowedMethods: ['PUT']
AllowedOrigins: ['*']
ResourcesBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket:
Ref: ResourcesBucket
PolicyDocument:
Statement:
# Read permission for CloudFront
- Action: s3:GetObject
Effect: "Allow"
Resource:
Fn::Join:
- ""
-
- "arn:aws:s3:::"
-
Ref: "ResourcesBucket"
- "/*"
Principal:
CanonicalUser: !GetAtt CloudFrontOriginAccessIdentity.S3CanonicalUserId
- Action: s3:PutObject
Effect: "Allow"
Resource:
Fn::Join:
- ""
-
- "arn:aws:s3:::"
-
Ref: "ResourcesBucket"
- "/*"
Principal:
AWS: !GetAtt LambdaEdgeFunctionRole.Arn
- Action: s3:GetObject
Effect: "Allow"
Resource:
Fn::Join:
- ""
-
- "arn:aws:s3:::"
-
Ref: "ResourcesBucket"
- "/*"
Principal:
AWS: !GetAtt LambdaEdgeFunctionRole.Arn
CloudFrontOriginAccessIdentity:
Type: AWS::CloudFront::CloudFrontOriginAccessIdentity
Properties:
CloudFrontOriginAccessIdentityConfig:
Comment:
Fn::Join:
- ""
-
- "Identity for accessing CloudFront from S3 within stack "
-
Ref: "AWS::StackName"
- ""
# Cloudfront distro backed by ResourcesBucket
ResourcesCdnDistribution:
Type: AWS::CloudFront::Distribution
Properties:
DistributionConfig:
Origins:
# S3 origin for private resources
- DomainName: !Sub '${self:custom.resourcesBucketName}.s3.amazonaws.com'
Id: S3OriginPrivate
S3OriginConfig:
OriginAccessIdentity: !Sub 'origin-access-identity/cloudfront/#{CloudFrontOriginAccessIdentity}'
# S3 origin for public resources
- DomainName: !Sub '${self:custom.resourcesBucketName}.s3.amazonaws.com'
Id: S3OriginPublic
S3OriginConfig:
OriginAccessIdentity: !Sub 'origin-access-identity/cloudfront/#{CloudFrontOriginAccessIdentity}'
Enabled: true
Comment: CDN for public and provate static content.
DefaultRootObject: index.html
HttpVersion: http2
DefaultCacheBehavior:
AllowedMethods:
- DELETE
- GET
- HEAD
- OPTIONS
- PATCH
- POST
- PUT
Compress: true
TargetOriginId: S3OriginPublic
ForwardedValues:
QueryString: false
Headers:
- Origin
Cookies:
Forward: none
ViewerProtocolPolicy: redirect-to-https
CacheBehaviors:
-
PathPattern: 'private/*'
TargetOriginId: S3OriginPrivate
AllowedMethods:
- DELETE
- GET
- HEAD
- OPTIONS
- PATCH
- POST
- PUT
Compress: true
LambdaFunctionAssociations:
-
EventType: viewer-request
LambdaFunctionARN: !GetAtt AuthEdgeLambdaVersion.FunctionArn
ForwardedValues:
QueryString: false
Headers:
- Origin
Cookies:
Forward: none
ViewerProtocolPolicy: redirect-to-https
-
PathPattern: 'public/*'
TargetOriginId: S3OriginPublic
AllowedMethods:
- DELETE
- GET
- HEAD
- OPTIONS
- PATCH
- POST
- PUT
Compress: true
ForwardedValues:
QueryString: false
Headers:
- Origin
Cookies:
Forward: none
ViewerProtocolPolicy: redirect-to-https
PriceClass: PriceClass_200
Some quick points related to this file:
Here I created the S3 bucket that will contain all my private and public resources.
This bucket is private and not accessible and you will find a role who just give the CDN and the lambda edge access to it.
I decided to create a CloudFront (CDN) with two origins public to be pointed to the S3's public folder and private to point it to the S3's private folder and configure the behavior of the CloudFront private origin to use my lambda edge function for the authentication through the viewer-request event type.
You will find also a code to create the function version and another function called PublishLambdaVersion with its role and it helps to give the lambda edge the correct permissions while deploying.
Finally here it the actually code for the lambda edge function used for CDN auth:
import {getJwksClient, getToken, getDecodedHeader, getSigningKey, verifyToken} from '../../../../libs/services/auth-service';
import config from '../../../../config';
const response401 = {
status: '401',
statusDescription: 'Unauthorized'
};
exports.handler = async (event) => {
try{
const cfrequest = event.Records[0].cf.request;
const headers = cfrequest.headers;
if(!headers.authorization) {
console.log("no auth header");
return response401;
}
const jwtValue = getToken(headers.authorization);
const client = getJwksClient(`https://${config.authDomain}/.well-known/openid-configuration/jwks`);
const decodedJwtHeader = getDecodedHeader(jwtValue);
if(decodedJwtHeader)
{
const signingKey = await getSigningKey(decodedJwtHeader, client);
const verifiedToken = await verifyToken(jwtValue, signingKey);
if(verifiedToken)
{
return cfrequest;
}
}else{
throw Error("Unauthorized");
}
}catch(err){
console.log(err);
return response401;
}
};
In case you are interested, I am using IdentityServer4 and hosted it as a docker image in Azure and using it as a custom authorizer.
So the full scenario now that we have an S3 bucket that totally private. It's only accessible through the CloudFront origins. If the request served through the public origin so no authentication needed but if it's served through the private origin so is I am triggering what's called lambda edge to authenticate it and validate the bearer token.
I was totally new to AWS stack before going deep into all of those but AWS is quite easy so I end up configured everything in a perfect way. Please let me know in case there is something not clear or if there are any questions.
I need to provide the name of the S3 bucket, which serverless create for me, to my application. Here is a simplified version of my serverless.yml file.
service: dummy-service
app: dummy-service
custom:
bucket: "I have no idea what to write here!"
provider:
name: aws
runtime: nodejs10.x
region: eu-central-1
iamRoleStatements:
- Effect: Allow
Action:
- s3:PutObject
- s3:GetObject
Resource:
- "Fn::Join":
- ""
- - "arn:aws:s3:::"
- Ref: DummyBucket
- "*"
environment:
BUCKET: ${self:custom.bucket}
resources:
Resources:
DummyBucket:
Type: AWS::S3::Bucket
functions:
createOrUpdate:
handler: handler.dummy
events:
- http:
path: dummy
method: POST
I have figured out how to make a reference in the iamRoleStatements section. But can't understand how to get it as a string for the environment variable.
Any help is welcome. Thanks.
You can use Ref to get the bucket name
service: dummy-service
app: dummy-service
custom:
bucket:
Ref: DummyBucket
provider:
name: aws
runtime: nodejs10.x
region: eu-central-1
iamRoleStatements:
- Effect: Allow
Action:
- s3:PutObject
- s3:GetObject
Resource:
- "Fn::Join":
- ""
- - "arn:aws:s3:::"
- Ref: DummyBucket
- "*"
environment:
BUCKET: ${self:custom.bucket}
resources:
Resources:
DummyBucket:
Type: AWS::S3::Bucket
functions:
createOrUpdate:
handler: handler.dummy
events:
- http:
path: dummy
method: POST
I have a lambda function that will handle PUT and GET requests using Amazon API Gateway {proxy+}.
It is working correctly when all the settings are set manually by the Amazon Console. but I want to automate it using AWS Cloudformation.
To inform you, I will write steps to set {proxy+}:
1) create a simple Lambda function and paste this lines of code inside it:
import boto3
def lambda_handler(event, context):
return {
"statusCode": 200,
"headers": {
"Content-Type": 'text/html',
"Access-Control-Allow-Origin": "*"
},
"body": "Hello Reza Amya, Your Lambda is working..!"
}
2) goto Amazon API Gateway and click on Create API.
3) choose New API, fill API name, select Edge optimized from the list for Endpoint Type then click on Create API
4) then your API is created and you should be on it's Resources page, if you are not, go to the Resources page for the created API.
5) from Actions select Create Resource
6) Select Configure as proxy resource (then it should change other fields automatically, if it doesn't, type proxy for Resource Name and {proxy+} for Resource Path) then click on Create Resource
7) Select Lambda Function Proxy for Integration type and select your lambda function from Lambda Function and click on Save
8) on the Add Permission to Lambda Function popup, click on Ok
9) from Actions click on Deploy API
10) Select New Stage from the list for Deployment stage then type a name for Stage name (for me, I have typed 'api') and click on Deploy
11) on the stage on the root page for your deployed API, you can see Invoke URL. click on it, and it will open new tab linked to somewhere like this: https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/api/
12) add a simple segment to end of your URL like this:
https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/api/test
now you should see bellow message in your browser page:
Hello Reza Amya, Your Lambda is working..!
Now the problem is I have written all these steps inside a Yaml file:
AWSTemplateFormatVersion: 2010-09-09
Description: My Lambda Function
Parameters:
S3Bucket:
Description: S3 Bucket where the Lambda code is
Type: String
S3Key:
Description: S3 Key where the Lambda code is
Type: String
S3ObjectVersion:
Description: Version of the S3 Key to use
Type: String
Resources:
apiGateway:
Type: "AWS::ApiGateway::RestApi"
Properties:
Name: "my-api"
Description: "My API"
EndpointConfiguration:
Types:
- EDGE
Resource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId:
Ref: "apiGateway"
ParentId:
Fn::GetAtt:
- "apiGateway"
- "RootResourceId"
PathPart: "{proxy+}"
ProxyMethod:
Type: 'AWS::ApiGateway::Method'
Properties:
HttpMethod: ANY
ResourceId: !Ref Resource
RestApiId: !Ref apiGateway
AuthorizationType: NONE
RequestParameters:
method.request.path.proxy: true
Integration:
CacheKeyParameters:
- 'method.request.path.proxy'
RequestParameters:
integration.request.path.proxy: 'method.request.path.proxy'
Type: AWS_PROXY
IntegrationHttpMethod: ANY
Uri: !Sub
- arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${Arn}/invocations
- Arn:
Fn::GetAtt:
- LambdaFunction
- Arn
PassthroughBehavior: WHEN_NO_MATCH
IntegrationResponses:
- StatusCode: 200
apiGatewayDeployment:
Type: "AWS::ApiGateway::Deployment"
DependsOn:
- "ProxyMethod"
Properties:
RestApiId: !Ref "apiGateway"
StageName: "dev"
IAMRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: 'sts:AssumeRole'
Policies:
- PolicyName: Logging
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
Resource: 'arn:aws:logs:*:*:*'
- PolicyName: AccessToDynamoDB
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- 'dynamodb:CreateTable'
- 'dynamodb:DeleteItem'
- 'dynamodb:DeleteTable'
- 'dynamodb:GetItem'
- 'dynamodb:GetRecords'
- 'dynamodb:UpdateItem'
- 'dynamodb:UpdateTable'
- 'dynamodb:PutItem'
- 'dynamodb:UpdateTable'
Resource: 'arn:aws:dynamodb:*:*:*'
LambdaFunction:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket: {Ref: S3Bucket}
S3Key: {Ref: S3Key}
S3ObjectVersion: {Ref: S3ObjectVersion}
Handler: main.lambda_handler
MemorySize: 128
Role: {'Fn::GetAtt': [IAMRole, Arn]}
Runtime: python3.6
Timeout: 300
LambdaInvokePermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !GetAtt
- LambdaFunction
- Arn
Action: 'lambda:InvokeFunction'
Principal: apigateway.amazonaws.com
SourceArn: !Sub arn:${AWS::Partition}:execute-api:${AWS::Region}:${AWS::AccountId}:${apiGateway}/*/*
Outputs:
apiGatewayInvokeURL:
Value: !Sub "https://${apiGateway}.execute-api.${AWS::Region}.amazonaws.com/${apiGateway}"
lambdaArn:
Value: !GetAtt "LambdaFunction.Arn"
The above Yaml file will create the Lambda function and will deploy the API, but it will show bellow error when I am trying to test the API:
{"message": "Internal server error"}
Can you please guide me what is wrong and how I can solve the problem?
The issue is related to you IntegrationHttpMethod setting. Although your APIGateway method is ANY, the IntegrationHttpMethod must always be POST for AWS Lambda.
This would lead to the following method declaration.
ProxyMethod:
Type: 'AWS::ApiGateway::Method'
Properties:
HttpMethod: ANY
ResourceId: !Ref Resource
RestApiId: !Ref apiGateway
AuthorizationType: NONE
RequestParameters:
method.request.path.proxy: true
Integration:
CacheKeyParameters:
- 'method.request.path.proxy'
RequestParameters:
integration.request.path.proxy: 'method.request.path.proxy'
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub
- arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${Arn}/invocations
- Arn:
Fn::GetAtt:
- LambdaFunction
- Arn
PassthroughBehavior: WHEN_NO_MATCH
IntegrationResponses:
- StatusCode: 200