Can someone help me write a Cloud formation script to update output location of Athena primary workgroup. When i run below code, getting error message "Invalid request provided: primary workGroup could not be created (Service: Athena, Status Code: 400, Request ID: 9945209c-6999-4e8b-bd3d-a3af13b4ac4f)"
Resources:
MyAthenaWorkGroup:
Type: AWS::Athena::WorkGroup
Properties:
Name: primary
Description: My WorkGroup Updated
State: DISABLED
WorkGroupConfigurationUpdates:
BytesScannedCutoffPerQuery: 10000000
EnforceWorkGroupConfiguration: true
PublishCloudWatchMetricsEnabled: true
RequesterPaysEnabled: false
ResultConfigurationUpdates:
EncryptionConfiguration:
EncryptionOption: SSE_S3
OutputLocation: s3://test/
you need to "create a change set to import an existing resource into a new or existing stack.", because the 'primary' workgroup was created outside your stack.
So you need to use the option: "Create stack with existing resources":
You also need to add DeletionPolicy/UpdateReplacePolicy.
An complete working example is:
Resources:
AthenaPrimaryWorkGroup:
Type: AWS::Athena::WorkGroup
Properties:
Name: primary
State: ENABLED
WorkGroupConfigurationUpdates:
BytesScannedCutoffPerQuery: 1000000000
EnforceWorkGroupConfiguration: true
PublishCloudWatchMetricsEnabled: true
RequesterPaysEnabled: false
ResultConfigurationUpdates:
OutputLocation: s3://MY-Athena-results
DeletionPolicy: Retain
UpdateReplacePolicy: Retain
More detail at: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-import.html
Why not name the workgroup in your cloudformation stack something other than "primary", and let the stack manage the workgroup resource completely? Then, depending on how the rest of the system is set up, it may work out to use this new workgroup instead of the primary one. Even if you do get it working to alter the primary workgroup output location in CF, I think it's clear that's going upstream against CF's most natural usage patterns.
Related
I'm working on a CloudFormation template and noticed an intricacy with Lambda that's new for me. Here's a snippet:
VersionLambdaRandom:
Type: AWS::Lambda::Version
Properties:
Description: Version testing.
FunctionName: !Sub "arn:${AWS::Partition}:lambda:${AWS::Region}:${AWS::AccountId}:function:${LambdaRandom}"
DependsOn: LambdaRandom
AliasLambdaRandom:
Type: AWS::Lambda::Alias
Properties:
FunctionName: !Sub "arn:${AWS::Partition}:lambda:${AWS::Region}:${AWS::AccountId}:function:${LambdaRandom}"
FunctionVersion: '1'
Name: TestDeploymentAlias
DependsOn:
- LambdaRandom
- VersionLambdaRandom
In the process of spinning this stack up and down, which deletes both the Version and Alias along with the Random Lambda that is not included here, I've noticed the version number doesn't reset when I create the stack again. Essentially, I'll create the stack again and will get a failure during the deployment along the lines of:
Function not found: arn:aws:lambda:us-east-1:<Acct#>:function:RandomLambdaFunction:1 (Service: AWSLambdaInternal; Status Code: 404; Error Code: ResourceNotFoundException)
Looking in the console, I can see a version of the newly created function was created successfully; however, it's now at version 2 rather than version 1. This seems like strange behavior to me since I completely deleted the stack, which includes deleting the lambda and the version.
What can I do to ensure the version I create when creating a stack always remains at one, besides switching the function name each time?
Lambda versions are immutable. That's the entire point of versioning - you can't change them once published, which includes deleting them. You have to delete your function as well to get read of its versions.
I am creating an APIGateway using Cloudformation. When attempting to create via the AWS CF Console I am recieving this error:
The EmptyModel resource is a AWS::ApiGateway::Model object that looks like this:
EmptyModel:
Type: "AWS::ApiGateway::Model"
Properties:
RestApiId: !Ref ApiGatewayRestApi
Name: "Empty"
Description: "This is a default empty schema model"
Schema: |
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title" : "Empty Schema",
"type" : "object"
}
ContentType: "application/json"
I am referencing this model on every one of my AWS::ApiGateway::Method objects in the CF Template like this:
What am I doing wrong? I used Former2 to reverse engineer my current api and get some of this template for the new api I am creating. So I am wondering if there is just something weird in this? Any help is hella appreciated.
I was able to identify this issue finally through much trial and error. Apparently Models are shared between all of the APIs in your account (or maybe just region)
So the error was indicating that there was already a model called "Empty" and that is because there was one, in a different API. I changed the name to "EmptyModel" and it worked great!
If you used former2 to create your template from existing resources, you can't just deploy the template obtained, as you will get the errors you are getting.
Instead you have to modify the template and import your resources to CloudFormation. Or easier, you have to delete existing resources, and then re-create them using CloudFormation.
I am trying to add another path to AWS Lambda that has been generated by code star.
I have tried to add the following events (in different commits) with the same failure behavior.
HelloWorld:
Type: AWS::Serverless::Function
Properties:
Events:
API:
Type:
Properties:
Path: /{proxy+}
Method: ANY
HelloWorld:
Type: AWS::Serverless::Function
Properties:
Events:
Decks_GetEvent:
Type: Api
Properties:
Path: /decks
Method: get
GenerateChangeSet step in code pipeline fails with the following unhelpful message:
Action execution failed
Failed to create change set. Current status: FAILED
When going to 'Details' Link of GenerateChangeSet step. It just takes me to Stack view without any particular stack or execution event selected.
What are the steps to troubleshoot this? Any suggestions on what could be causing this?
If it doesn't want to create a changeset, you presumably have a syntax error in your document. However, since you've not shared the CloudFormation template, that is hard to determine.
If you want to troubleshoot this, you can try to manually create a change set (that might provide more error information), or you can use cfn-lint to do some static code analysis on your CloudFormation template.
I need to configure a lambda via serverless.yml to use different provision concurrency for different environments. Below is my lambda configuration:
myLambda:
handler: src/lambdas
name: myLambda
provisionedConcurrency: ${self:custom.pc}
...
custom:
pc: ${env:PC}
The value PC is loaded from environment variable. It works for values greater than 0 but I can't set a value 0 in one environment. What I want to do is to disable provision concurrency in dev environment.
I have read through this doc https://forum.serverless.com/t/conditional-serverless-yml-based-on-stage/1763/3 but it doesn't seem to help in my case.
How can I set provisionedConcurrency conditional based on environment?
Method 1: Stage-based variables via default values
This is a fairly simple trick by using a cascading value variable. The first value is the one you want, the second one being a default, or fallback value. Also called cascading variables.
// serverless.yml
provider:
stage: "dev"
custom:
provisionedConcurrency:
live: 100
staging: 50
other: 10
myLambda:
handler: src/lambdas
name: myLambda
provisionedConcurrency: ${self:custom.provisionedConcurrency.${self:provider.stage}, self:custom.provisionedConcurrency.other}
This above with stage set to dev will default to "other" value of 10, but if you set stage via serverless deploy --stage live then it will use the live value of 100.
See here for more details: https://www.serverless.com/framework/docs/providers/aws/guide/variables#syntax
Method 2: Asynchonous Value via Javascript
You can use an js include and put your conditional logic there. It's called "asynchronous value support". Basically, this allows you to put logic in a javascript file which you include and it can return different values depending on various things (like, what AWS account you're on, or if certain variables are set, or whatever). Basically, it allows you to do this...
provisionedConcurrency: ${file(./detect_env.js):get_provisioned_concurrency}
Which works if you create a javascript file in this folder called detect_env.js, and it has the contents similar to...
module.exports.get_provisioned_concurrency = () => {
if ("put logic to detect which env you are deploying to, eg for live") {
return Promise.resolve('100');
} else {
// Otherwise fallback to 10
return Promise.resolve('10');
}
}
For more info see: https://www.serverless.com/framework/docs/providers/aws/guide/variables#with-a-new-variables-resolver
I felt I had to reply here even though this was asked months ago because none of the answers were even remotely close to the right answer and I really felt sorry for the author or anyone who lands here.
For really sticky problems, I find it's useful to go to the Cloudformation script instead and use the Cloudformation Intrinsic Functions.
For this case, if you know all the environments you could use Fn::FindInMap
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-findinmap.html
Or if it's JUST production which needs 0 then you could use the conditional Fn::If and a boolean Condition test in the Cloudformation template to test if environment equals production, use 0, else use the templated value from SLS.
Potential SLS:
resources:
Conditions:
UseZero: !Equals ["production", ${provider.stage}]
Resources:
myLambda:
ProvisionedConcurrency: !If [UseZero, 0, ${self:custom.pc}]
You can explicitly remove the ProvisionedConcurrency property as well if you want:
resources:
Conditions:
UseZero: !Equals ["production", ${provider.stage}]
Resources:
myLambda:
ProvisionedConcurrency: !If [UseZero, AWS::NoValue, ${self:custom.pc}]
Edit: You can still use SLS to deploy; it simply compiles into a Cloudformation JSON template which you can explicitly modify with the SLS resources field.
The Serverless Framework provides a really useful dashboard tool with a feature called Parameters. Essentially what it lets you do is connect your service to it then you can set different values for different stages and then use those values in your serverless.yml with syntax like ${param:VARAIBLE_NANE_HERE} and it gets replaced at deploy time with the right value for whatever stage you are currently deploying. Super handy. There are also a bunch of other features in the dashboard such as monitoring and troubleshooting.
You can find out more about Parameters at the official documentation here: https://www.serverless.com/framework/docs/guides/parameters/
And how to get started with the dashboard here: https://www.serverless.com/framework/docs/guides/dashboard#enabling-the-dashboard-on-existing-serverless-framework-services
Just using a variable with a null value for dev environments during on deploy/package and SLS will skip this property:
provisionedConcurrency: ${self:custom.variables.provisionedConcurrency}
When creating AWS::MSK::Cluster with Cloud Formation I am not able to set Tags in the usual way:
Tags:
- Key: Name
Value: !Ref Identifier
Because of this error:
Property validation failure: [Value of property {/Tags} does not match type {Map}]
As of the time of writing, the documentation states that, instead of the usual Type: List of Tag, I should use: Type: Json.
Also the same documentation states that:
You can specify tags in JSON or in YAML, depending on which format you use for your template
After further investigation (and AWS support help), the working (only on creation) example looks like this:
Tags:
Name: !Ref Identifier
Additionally, tags cannot be modified (the docs actually state that tags change require replacement), when tried a slightly confusing error shows up:
CloudFormation cannot update a stack when a custom-named resource requires replacing. Rename kafka-eu-west-1-dev and update the stack again.