AWS Lambda not working when deployed with CDK - amazon-web-services

I am trying to deploy a lambda using AWS CDK and it seems not to be working/deployed properly.
The "box" in the pipeline is green, so no errors are returned.
Everything appears to be fine, but when I ran it manually to test, I receive the next message:
{
"errorType": "LambdaException",
"errorMessage": "Could not find the required 'QuickSight.Lambdas.SpiceRefresh.deps.json'. This file should be present at the root of the deployment package."
}
The issue is that if I download the artefact manually to my machine, and upload it with the Function package upload button, it is working properly.
I have one Stack which contains CfnParametersCode which is the stack I use to create the lambda.
public class LambdaStack : Stack
{
public CfnParametersCode LambdaCode { get; set; }
//code
private Function BuildSpiceRefreshLambda()
{
LambdaCode = Code.FromCfnParameters();
var func = new Function(this, Constants.Lambda.LambdaName, new FunctionProps
{
Code = LambdaCode,
Handler = Constants.Lambda.LambdaHandler,
FunctionName = Constants.Lambda.LambdaName,
MemorySize = 1024,
Tracing = Tracing.ACTIVE,
Timeout = Duration.Seconds(480),
Runtime = Runtime.DOTNET_CORE_2_1,
Environment = new Dictionary<string, string>()
{
{"ENVIRONMENT", Fn.Ref(Constants.EnvironmentVariables.Environment)},
{"APPLICATION_NAME", Constants.Lambda.ApplicationName},
{"AWS_ACCOUNT_ID", Fn.Ref("AWS::AccountId")},
{"LOG_GROUP_NAME", Constants.Lambda.LogGroupName}
},
ReservedConcurrentExecutions = 1,
Role = SpiceRefreshLambdaRole,
Vpc = this.GetProjectVpc(),
SecurityGroups = new ISecurityGroup[]
{
securityGroup
}
});
return func;
}
}
and then I have the pipeline which one of the steps is build the lambda:
var lambdaBuild = new PipelineProject(this, "appLambda", new PipelineProjectProps
{
BuildSpec = BuildSpec.FromObject(new Dictionary<string, object>
{
["version"] = "0.2",
["phases"] = new Dictionary<string, object>
{
["install"] = new Dictionary<string, object>
{
["commands"] = new string[]
{
"echo \"Installing lambda tools for dotnet\"",
"dotnet tool install -g Amazon.Lambda.Tools",
}
},
["build"] = new Dictionary<string, object>
{
["commands"] = new string[]
{
"echo \"Packaging app lambda\"",
"(cd app/src/Lambdas/app.Lambdas.Action; dotnet lambda package)"
}
}
},
["artifacts"] = new Dictionary<string, object>
{
["files"] = new[]
{
"app/src/Lambdas/app.Lambdas.Action/bin/Release/netcoreapp2.1/app.Lambdas.Action.zip",
}
}
}),
Environment = new BuildEnvironment
{
BuildImage = LinuxBuildImage.STANDARD_2_0
}
});
var lambdaBuildOutput = new Artifact_("LambdaBuildOutput");
new Amazon.CDK.AWS.CodePipeline.Pipeline(this, "appPipeline", new PipelineProps
{
ArtifactBucket = Bucket.FromBucketAttributes(this, "artifact-bucket", new BucketAttributes
{
BucketArn = "bucket",
EncryptionKey = "key"
}),
Role = "role",
Stages = new[]
{
new StageProps
{
StageName = "Source",
Actions = new[]
{
new CodeCommitSourceAction(new CodeCommitSourceActionProps
{
ActionName = "Source",
Repository = code,
Output = sourceOutput,
})
}
},
new StageProps
{
StageName = "Build",
Actions = new[]
{
new CodeBuildAction(new CodeBuildActionProps
{
ActionName = "Lambda_Build",
Project = lambdaBuild,
Input = sourceOutput,
Outputs = new[] {lambdaBuildOutput},
}),
}
},
new StageProps
{
StageName = "Deploy",
Actions = new[]
{
new CloudFormationCreateUpdateStackAction(new CloudFormationCreateUpdateStackActionProps
{
ActionName = "DeployLambdaapp",
TemplatePath = props.appLambdaStack.StackTemplate,
StackName = "appLambdaDeploymentStack",
AdminPermissions = true,
ParameterOverrides = props.appLambdaStack.LambdaCode.Assign(lambdaBuildOutput.S3Location),
ExtraInputs = new[] {lambdaBuildOutput},
Role = "role",
DeploymentRole = "deployRole"
}),
}
}
}
});
there is more steps but they are not relevant.
so as you can see I am applying the ParameterOverrides to props.appLambdaStack.LambdaCode.Assign(lambdaBuildOutput.S3Location) which seems to be fine, because When the lambda gets created, and it specifies the size, which is the same size as the lambda was supposed to be, but when I execute it I receive that "errorMessage": "Could not find the required 'QuickSight.Lambdas.SpiceRefresh.deps.json'. This file should be present at the root of the deployment package."
On the result Cloudformation file seems to be fine too:
"appLambdaF0BB8286": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": {
"Ref": "appLambdaSourceBucketNameParameter"
},
"S3Key": {
"Ref": "appLambdaSourceObjectKeyParameter"
}
},
"Handler": "Constants.Lambda.LambdaHandler", //same as the constant in c#
//Rest of the properties
}
}
I checked before creating the post and most of the people had a problem with the handler. Un fortunately if I download manually the object in appLambdaSourceBucketNameParameter, appLambdaSourceObjectKeyParameter and upload it to the lambda, it works perfectly. I think that will exclude my issue.
Any idea what can be wrong?

Found the solution.
The issue is that in the artifact I am returning the lambda .zip
["files"] = new[]
{
"app/src/Lambdas/app.Lambdas.Action/bin/Release/netcoreapp2.1/app.Lambdas.Action.zip",
}
But what I really need is to return the binaries of the lambda. (the publish folder)
["artifacts"] = new Dictionary<string, object>
{
["base-directory"] = "app/src/Lambdas/app.Lambdas.Action/bin/Release/netcoreapp2.1/publish",
["files"] = new[] { "**.*" }
}
nothing else changed, and it worked.
Cloudformation translation:
before I was exporting artifact::app.Lambdas.Action.zip
then AWS was trying to find the binaries.
Now it is exporting artifact::**.*, so all the files.

Related

AWS Cloudfront for S3 backed website + Rest API: (Error - MethodNotAllowed / The specified method is not allowed against this resource)

I have an AWS S3 backed static website and a RestApi. I am configuring a single Cloudfront Distribution for the static website and the RestApi. I have OriginConfigs done for the S3 origins and the RestApi origin. I am using AWS CDK to define the infrastructure in code.
The approach has been adopted from this article: https://dev.to/evnz/single-cloudfront-distribution-for-s3-web-app-and-api-gateway-15c3]
The API are defined under the relative path /r/<resourcename> or /r/api/<methodname>. Examples would be /r/Account referring to the Account resource and /r/api/Validate referring to an rpc-style method called Validate (in this case a HTTP POST method). The Lambda methods that implement the resource methods are configured with the proper PREFLIGHT OPTIONS with the static website's url listed in the allowed origins for that resource. For eg: the /r/api/Validate method lambda has
exports.main = async function(event, context) {
try {
var method = event.httpMethod;
if(method === "OPTIONS") {
const response = {
statusCode: 200,
headers: {
"Access-Control-Allow-Headers" : "*",
"Access-Control-Allow-Credentials": true,
"Access-Control-Allow-Origin": website_url,
"Vary": "Origin",
"Access-Control-Allow-Methods": "OPTIONS,POST,GET,DELETE"
}
};
return response;
} else if(method === "POST") {
...
}
...
}
The API and website are deployed fine. Here's the CDK deployment code fragment.
const string api_domain = "myrestapi.execute-api.ap-south-1.amazonaws.com";
const string api_stage = "prod";
internal WebAppStaticWebsiteStack(Construct scope, string id, IStackProps props = null) : base(scope, id, props)
{
// The S3 bucket to hold the static website contents
var bucket = new Bucket(this, "WebAppStaticWebsiteBucket", new BucketProps {
PublicReadAccess = false,
BlockPublicAccess = BlockPublicAccess.BLOCK_ALL,
RemovalPolicy = RemovalPolicy.DESTROY,
WebsiteIndexDocument = "index.html",
Cors = new ICorsRule[] {
new CorsRule() {
AllowedHeaders = new string[] { "*" },
AllowedMethods = new HttpMethods[] { HttpMethods.GET, HttpMethods.POST, HttpMethods.PUT, HttpMethods.DELETE, HttpMethods.HEAD },
AllowedOrigins = new string[] { "*" }
}
}
});
var cloudfrontOAI = new OriginAccessIdentity(this, "CloudfrontOAI", new OriginAccessIdentityProps() {
Comment = "Allows cloudfront access to S3"
});
bucket.AddToResourcePolicy(new PolicyStatement(new PolicyStatementProps() {
Sid = "Grant cloudfront origin access identity access to s3 bucket",
Actions = new [] { "s3:GetObject" },
Resources = new [] { bucket.BucketArn + "/*" },
Principals = new [] { cloudfrontOAI.GrantPrincipal }
}));
// The cloudfront distribution for the website
var distribution = new CloudFrontWebDistribution(this, "WebAppStaticWebsiteDistribution", new CloudFrontWebDistributionProps() {
ViewerProtocolPolicy = ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
DefaultRootObject = "index.html",
PriceClass = PriceClass.PRICE_CLASS_ALL,
GeoRestriction = GeoRestriction.Whitelist(new [] {
"IN"
}),
OriginConfigs = new [] {
new SourceConfiguration() {
CustomOriginSource = new CustomOriginConfig() {
OriginProtocolPolicy = OriginProtocolPolicy.HTTPS_ONLY,
DomainName = api_domain,
AllowedOriginSSLVersions = new OriginSslPolicy[] { OriginSslPolicy.TLS_V1_2 },
},
Behaviors = new IBehavior[] {
new Behavior() {
IsDefaultBehavior = false,
PathPattern = $"/{api_stage}/r/*",
AllowedMethods = CloudFrontAllowedMethods.ALL,
CachedMethods = CloudFrontAllowedCachedMethods.GET_HEAD_OPTIONS,
DefaultTtl = Duration.Seconds(0),
ForwardedValues = new CfnDistribution.ForwardedValuesProperty() {
QueryString = true,
Headers = new string[] { "Authorization" }
}
}
}
},
new SourceConfiguration() {
S3OriginSource = new S3OriginConfig() {
S3BucketSource = bucket,
OriginAccessIdentity = cloudfrontOAI
},
Behaviors = new [] {
new Behavior() {
IsDefaultBehavior = true,
//PathPattern = "/*",
DefaultTtl = Duration.Seconds(0),
Compress = false,
AllowedMethods = CloudFrontAllowedMethods.ALL,
CachedMethods = CloudFrontAllowedCachedMethods.GET_HEAD_OPTIONS
}
},
}
}
});
// The distribution domain name - output
var domainNameOutput = new CfnOutput(this, "WebAppStaticWebsiteDistributionDomainName", new CfnOutputProps() {
Value = distribution.DistributionDomainName
});
// The S3 bucket deployment for the website
var deployment = new BucketDeployment(this, "WebAppStaticWebsiteDeployment", new BucketDeploymentProps(){
Sources = new [] {Source.Asset("./website/dist")},
DestinationBucket = bucket,
Distribution = distribution
});
}
I am encountering the following error (extracted from Browser console error log):
bundle.js:67 POST https://mywebapp.cloudfront.net/r/api/Validate 405
bundle.js:67
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>MethodNotAllowed</Code>
<Message>The specified method is not allowed against this resource.</Message>
<Method>POST</Method>
<ResourceType>OBJECT</ResourceType>
<RequestId>xxxxx</RequestId>
<HostId>xxxxxxxxxxxxxxx</HostId>
</Error>
The intended flow is that the POST call (made using fetch() api) to https://mywebapp.cloudfront.net/r/api/Validate is forwarded to the RestApi backend by cloudfront. It appears like Cloudfront is doing it, but the backend is returning an error (based on the error message).
What am I missing? How do I make this work?
This was fixed by doing the following:
Moved to the Distribution construct (which as per AWS documentation is the one to use as it is receiving latest updates).
Adding a CachePolicy and OriginRequestPolicy to control Cookie forwarding and Header forwarding

FederatedPrincipal with multiples Actions

I'm trying to create a FederatedPrincipal in aws-cdk with multiple Action as shown below:
Currently, I'm doing this (as shown below) in c#
new FederatedPrincipal("cognito-identity.amazonaws.com", new Dictionary<string, object>
{
{ "ForAnyValue:StringLike", new Dictionary<string,string> { ["cognito-identity.amazonaws.com:amr"] = "authenticated" } },
{ "StringEquals", new Dictionary<string,string> { ["cognito-identity.amazonaws.com:aud"] = cfn_identitypool.Ref } }
}, "sts:AssumeRoleWithWebIdentity");
How do I add the 2nd action - sts:TagSession?
This is currently not possible using high-level constructs. See this still open issue: https://github.com/aws/aws-cdk/issues/6699
TL;DR
The IPrincipal requires assumeRoleAction to be a string. But what you need is an array. It looks like it's been put on-hold because it means a BC-breaking change that the team does not want to introduce.
What I ended up with, is to use a low-level construct CfnRole. I use TypeScript but it should be straightforward to port it to C#.
const authenticatedRole = new iam.CfnRole(this, 'AuthenticatedRole', {
assumeRolePolicyDocument: {
'Statement': [{
'Effect': iam.Effect.ALLOW,
'Action': ['sts:AssumeRoleWithWebIdentity', 'sts:TagSession'],
'Condition': {
'StringEquals': {
'cognito-identity.amazonaws.com:aud': identityPool.getAtt('Ref')
},
'ForAnyValue:StringLike': {
'cognito-identity.amazonaws.com:amr': 'authenticated'
}
},
'Principal': {
'Federated': 'cognito-identity.amazonaws.com'
}
}]
}
});
const roleAttachment = new cognito.CfnIdentityPoolRoleAttachment(this, 'RoleAttachment', {
identityPoolId: identityPool.getAtt('Ref').toString(),
roles: {
'authenticated': authenticatedRole.getAtt('Arn'),
}
});
You can use the The withSessionTags method of the PrincipalBase class to address this issue, as described here and documented here

Update asset files using Tokens with aws-cdk

I have created this stack:
export class InfrastructureStack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const bucket = new s3.Bucket(this, "My Hello Website", {
websiteIndexDocument: 'index.html',
websiteErrorDocument: 'error.html',
publicReadAccess: true,
removalPolicy: cdk.RemovalPolicy.DESTROY
});
const api = new apigateway.RestApi(this, "My Endpoint", {
restApiName: "My rest API name",
description: "Some cool description"
});
const myLambda = new lambda.Function(this, 'My Backend', {
runtime: lambda.Runtime.NODEJS_8_10,
handler: 'index.handler',
code: lambda.Code.fromAsset(path.join(__dirname, 'code'))
});
const apiToLambda = new apigateway.LambdaIntegration(myLambda)
api.root.addMethod('GET', apiToLambda);
updateWebsiteUrl.newUrl(api.url);
}
}
Last line of code is my function to update asset that will be deployed on S3 as a website with a API url that will be created during deployment. This is just a plain Node.js script that replaces files PLACEHOLDER with api.url.
Of course during compile time the CDK does not know what will be the final adress of REST endpoint because this is happening during deploy time and it updates my url with somethis like:
'https://${Token[TOKEN.26]}.execute-api.${Token[AWS::Region.4]}.${Token[AWS::URLSuffix.1]}/${Token[TOKEN.32]}/;'
Is there any way that I can update this after integrating lambda with API endpooint after deploying those?
I would like to use #aws-cdk/aws-s3-deployment module to deploy code to newly created bucket. All in the same Stack, so one cdk deploy will update everything I need.
To avoid confusion. My updateWebsiteUrl is:
export function newUrl(newUrl: string): void {
const scriptPath = path.join(__dirname, '/../../front/');
const scriptName = 'script.js';
fs.readFile(scriptPath + scriptName, (err, buf) => {
let scriptContent : string = buf.toString();
let newScript = scriptContent.replace('URL_PLACEHOLDER', newUrl);
fs.writeFile(scriptPath + 'newScript.js', newScript, () => {
console.log('done writing');
});
});
}
And my script is just simple:
const url = URL_PLACEHOLDER;
function foo() {
let req = new XMLHttpRequest();
req.open('GET', url , false);
req.send(null);
if (req.status == 200) {
replaceContent(req.response);
}
}
function replaceContent(content) {
document.getElementById('content').innerHTML = content;
}
I ran into the same issue today and managed to find a solution for it.
The C# code I am using in my CDK program is the following:
// This will at runtime be just a token which refers to the actual JSON in the format {'api':{'baseUrl':'https://your-url'}}
var configJson = stack.ToJsonString(new Dictionary<string, object>
{
["api"] = new Dictionary<string, object>
{
["baseUrl"] = api.Url
}
});
var configFile = new AwsCustomResource(this, "config-file", new AwsCustomResourceProps
{
OnUpdate = new AwsSdkCall
{
Service = "S3",
Action = "putObject",
Parameters = new Dictionary<string, string>
{
["Bucket"] = bucket.BucketName,
["Key"] = "config.json",
["Body"] = configJson,
["ContentType"] = "application /json",
["CacheControl"] = "max -age=0, no-cache, no-store, must-revalidate"
},
PhysicalResourceId = PhysicalResourceId.Of("config"),
},
Policy = AwsCustomResourcePolicy.FromStatements(
new[]
{
new PolicyStatement(new PolicyStatementProps
{
Actions = new[] { "s3:PutObject" },
Resources= new [] { bucket.ArnForObjects("config.json") }
})
})
});
}
You will need to install the following package to have the types available: https://docs.aws.amazon.com/cdk/api/latest/docs/custom-resources-readme.html
It is basically a part of the solution you can find as an answer to this question AWS CDK passing API Gateway URL to static site in same Stack, or at this GitHub repository: https://github.com/jogold/cloudstructs/blob/master/src/static-website/index.ts#L134

How to build a unit test expected query for Nest 6.x with nested values

This is a follow up question for another post which I had asked (and been answer) how to build a unit test to a specific query in ElasticSearch using Nest (in c#).
this is a code sample from the answer which I was given in which I build an expected query for "normal" tags:
var expected = new
{
query = new {
#bool = new {
must = new object[] {
new {
#bool = new {
should = new object[] {
new {
match = new {
title = new {
query = "Kibana"
}
}
},
new {
match = new {
title = new {
query = "Elasticsearch",
boost = 2d
}
}
}
},
}
},
new {
#bool = new {
filter = new [] {
new {
range = new {
score = new {
gt = 0d
}
}
}
}
}
}
}
}
}
};
I didn't managed to turn one of those tags into a nested one. i.e. field.title
In my tested query object I created this property using JsonProperty in the following way:
[JsonProperty(PropertyName = "field.title")]
public object { get; set; }
but I didn't mange to do something similar in my unit test to 'mock' his creation.
Any help will be deeply appreciated.

DynamoDB UpdateItem Doesn't Work

This is pretty much straight out of the documentation (http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.Modifying.html), but like most things AWS, it just doesn't work and isn't documented well at all.
The following fails with "The document path provided in the update expression is invalid for update"
var messageId = "f244ed33-d678-4763-8f46-0f06514d5139"
var sequenceId = "00000000-0000-0000-0000-000000000000"
var now = DateTime.UtcNow;
var lastActivityThreshold = now.Subtract(TimeSpan.FromSeconds(10));
var response = client.UpdateItem(new UpdateItemRequest
{
TableName = "TheTable",
Key = new Dictionary<string, AttributeValue> { { "SequenceId", new AttributeValue { S = sequenceId } } },
ExpressionAttributeValues = new Dictionary<string, AttributeValue>
{
{":MessageId", new AttributeValue {S = messageId}},
{":Now", new AttributeValue {N = now.Ticks.ToString()}},
{":LastActivityThreshold", new AttributeValue {N = lastActivityThreshold.Ticks.ToString() }},
},
UpdateExpression = "REMOVE Messages[0] SET LastActivity = :Now",
ConditionExpression = "Messages[0] <> :MessageId AND (LastActivity <= :LastActivityThreshold OR attribute_not_exists(LastActivity))",
ReturnValues = ReturnValue.UPDATED_NEW
});
This is the document I'm trying to update (as seen in JSON view in the AWS Management Console):
{
"LastActivity": {
"N": "635753575712635873"
},
"Messages": {
"SS": [
"f244ed33-d678-4763-8f46-0f06514d5139",
"f668d2a5-3a4a-4564-8384-5b5a51c9bad3"
]
},
"SequenceId": {
"S": "00000000-0000-0000-0000-000000000000"
}
}
I've tried many variations of the code above, striaght down to removing all ExpressionAttributeValues and the ConditionExpression and just using REMOVE Messages[0], but it doesn't work and throws the same error.
It looks like you're trying to apply a document path to a non JSON item. There's no concept of ordering in a set, so if you want to remove the first item, you'll need to load it into memory and iterate over it. In short, you'll need to use a list in this case.