I am using AWS data-pipeline to export a DDB table, but when I activate I get an error:
Web service limit exceeded: Exceeded number of concurrent executions. Please set the field 'maxActiveInstances' to a higher value in your pipeline or wait for the currenly running executions to complete before trying again (Service: DataPipeline; Status Code: 400; Error Code: InvalidRequestException; Request ID: efbf9847-49fb-11e8-abef-1da37c3550b5)
How do I set this maxActiveInstances property using the AWS UI?
You can set it as a property on your Ec2Resource[1] (or EmrActivity[2]) object. Using the UI, click Edit Pipeline, click on Resources on the right hand side of the screen (it's a collapsable menu). There should be an Ec2Resource object. There should be a drop down on this object called "Add an additional field" and you should see max active instances in the drop down.
[1]https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-ec2resource.html
[2] https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-emractivity.html
We ran into this too. For an on-demand pipeline, it looks like after a certain number of retries, you have to give it time to finish terminating the provisioned resources before you will be allowed to try again.
Solution: Patience.
With an on-demand pipline you can specify it in the 'Default object', like this
{
"objects": [
{
"failureAndRerunMode": "CASCADE",
"scheduleType": "ONDEMAND",
"name": "Default",
"id": "Default",
"maxActiveInstances": "5"
},
...
I couldn't add it in Architect, I had to create another pipeline from the json. But once that was done I could edit it in Architect (under 'Others' section).
Related
So I've created a logging alert policy on google cloud that monitors the project's logs and sends an alert if it finds a log that matches a certain query. This is all good and fine, but whenever it does send an email alert, it's barebones. I am unable to include anything useful in the email alert such as the actual message, the user must instead click on "View incident" and go to the specified timeframe of when the alert happened.
Is there no way to include the message? As far as I can tell viewing the gcp Using Markdown and variables in documentation templates doc on this.
I'm only really able to use ${resource.label.x} which isn't really all that useful because it already includes most of that stuff by default in the alert.
Could I have something like ${jsonPayload.message}? It didn't work when I tried it.
Probably (!) not.
To be clear, the alerting policies track metrics (not logs) and you've created a log-based metric that you're using as the basis for an alert.
There's information loss between the underlying log (that contains e.g. jsonPayload) and the metric that's produced from it (which probably does not). You can create Log-based metrics labels using expressions that include the underlying log entry fields.
However, per the example in Google's docs, you'd want to consider a limited (enum) type for these values (e.g. HTTP status although that may be too broad too) rather than a potentially infinite jsonPayload.
It is possible. Suppose you need to pass "jsonPayload.message" present in your GCP log to documentation section in your policy. You need to use "label_extractor" feature to extract your log message.
I will share a policy creation JSON file template wherein you can pass "jsonPayload.message" in the documentation section in your policy.
policy_json = {
"display_name": "<policy_name>",
"documentation": {
"content": "I have the extracted the log message:${log.extracted_label.msg}",
"mime_type": "text/markdown"
},
"user_labels": {},
"conditions": [
{
"display_name": "<condition_name>",
"condition_matched_log": {
"filter": "<filter_condition>",
"label_extractors": {
"msg": "EXTRACT(jsonPayload.message)"
}
}
}
],
"alert_strategy": {
"notification_rate_limit": {
"period": "300s"
},
"auto_close": "604800s"
},
"combiner": "OR",
"enabled": True,
"notification_channels": [
"<notification_channel>"
]
}
When I launch an AWS CodeBuild project from the web interface, I can choose "Start Build" to start the build project with its normal configuration. Alternatively I can choose "Start build with overrides", which lets me specify, amongst others, custom environment variables for the build job.
From AWS EventBridge (events -> Rules -> Create rule), I can create a scheduled event to trigger the codebuild job, and this works. How though in EventBridge do I specify environment variable overrides for a scheduled CodeBuild job?
I presume it's possible somehow by using "additional settings" -> "Configure target input", which allows specification and templating of event JSON. I'm not sure though how how to work out, beyond blind trial and error, what this JSON should look like (to override environment variables in my case). In other words, where do I find the JSON spec for events sent to CodeBuild?
There are an number of similar questions here: e.g. AWS EventBridge scheduled events with custom details? and AWS Cloudwatch (EventBridge) Event Rule for AWS Batch with Environment Variables , but I can't find the specifics for CodeBuild jobs. I've tried the CDK docs at e.g. https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_events_targets.CodeBuildProjectProps.html , but am little wiser. I've also tried capturing the events output by EventBridge, to see what the event WITHOUT overrides looks like, but have not managed. Submitting the below (and a few variations: e.g. as "detail") as an "input constant" triggers the job, but the environment variables do not take effect:
{
"ContainerOverrides": {
"Environment": [{
"Name": "SOME_VAR",
"Value": "override value"
}]
}
}
There is also CodeBuild API reference at https://docs.aws.amazon.com/codebuild/latest/APIReference/API_StartBuild.html#API_StartBuild_RequestSyntax. EDIT: this seems to be the correct reference (as per my answer below).
The rule target's event input template should match the structure of the CodeBuild API StartBuild action input. In the StartBuild action, environment variable overrides have a key of "environmentVariablesOverride" and value of an array of EnvironmentVariable objects.
Here is a sample target input transformer with one constant env var and another whose value is taken from the event payload's detail-type:
Input path:
{ "detail-type": "$.detail-type" }
Input template:
{"environmentVariablesOverride": [
{"name":"MY_VAR","type":"PLAINTEXT","value":"foo"},
{"name":"MY_DYNAMIC_VAR","type":"PLAINTEXT","value":<detail-type>}]
}
I got this to work using an "input constant" like this:
{
"environmentVariablesOverride": [{
"name": "SOME_VAR",
"type": "PLAINTEXT",
"value": "override value"
}]
}
In other words, you can ignore the fields in the sample events in EventBridge, and the overrides do not need to be specified in a "detail" field.
I used the Code Build "StartBuild" API docs at https://docs.aws.amazon.com/codebuild/latest/APIReference/API_StartBuild.html#API_StartBuild_RequestSyntax to find this format. I would presume (but have not tested) that other fields show here would work similarly (and that the API reference for other services would work similarly when using EventBridge: can anyone confirm?).
For over a year I have been able to publish a ASP.NET Core Web API application using Visual Studio 2019 by selecting "Publish to AWS Lambda..." without incident (via a right click on the project). Until yesterday. Now it consistently fails to publish and rolls back.
The following two reasons are given as to why it has failed.
1 validation error detected: Value 'AWSLambdaFullAccess' at 'policyArn' failed to satisfy constraint: Member must have length greater than or equal to 20 (Service: AmazonIdentityManagement; Status Code: 400; Error Code: ValidationError; Request ID: ...; Proxy: null)
The following resource(s) failed to create: [AspNetCoreFunctionRole, Bucket]. Rollback requested by user.
I have looked at AWSLambdaFullAccess and AWSLambda_FullAccess and the other things and just have no model to follow or even know what it is referring to in any sense where I can imagine a fruitful path to proceed. What exactly is the "Member" it is referring to? Extensive research has yielded nothing of use.
I want to successfully publish my Web API. What can I look into to proceed?
This may not be the correct or ideal solution, I tried this approach and it worked
Step 1:
Changed the Access from "AWSLambdaFullAccess" to "AWSLambda_FullAccess" in serverless.template
"Resources": {
"AspNetCoreFunction": {
"Type": "AWS::Serverless::Function",
"Properties": {
"Handler": "SampleAPI::SampleAPI.LambdaEntryPoint::FunctionHandlerAsync",
"Runtime": "dotnetcore3.1",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [
"AWSLambda_FullAccess"
],
"Environment": {
"Variables": {
"AppS3Bucket": {
Lambda publishing was successful after this step.
Step 2:
Then I faced an issue in accessing the DynamoDb table. I went to IAM role added the DynamoDb Execution role. (Previously I don't remember adding this role explicitly)
According to https://docs.aws.amazon.com/lambda/latest/dg/access-control-identity-based.html the AWSLambdaFullAccess policy has just been deprecated and as a result my stack which I tried to update was stuck in UPDATE_ROLLBACK_FAILED.
To fix this I had to take the following steps:
Manually continue the rollback of the stack from the CloudFormation page and ensuring that I was skipping the role which was referencing AWSLambdaFullAccess.
Change my AWSLambdaFullAccess reference to AWSLambda_FullAccess in the CloudFormation template
Update the stack using my newly updated CloudFormation template
Hope this is able to help someone!
On AWS cloud watch we have one dashboard per environment.
Each dashboard has N plots.
Some plots, use the Auto Scaling Group Name (ASG) to find the data to plot.
Example of such a plot (edit, tab source):
{
"metrics": [
[ "production", "mem_used_percent", "AutoScalingGroupName", "awseb-e-rv8y2igice-stack-AWSEBAutoScalingGroup-3T5YOK67T3FD" ]
],
... other params removed for brevity ...
"title": "Used Memory (%)",
}
Every time we deploy, the ASG name changes (deploy using code-deploy with Elastic Bean Stalk (EBS) configuration files from source).
I need to manually find the new name and update the N plots one by one.
The strange thing is that this happens for production and staging environments, but not for integration.
All 3 should be copies of one another, with different settings from the EBS configuration files, so I don't know what is going on.
In any case, what (I think) I need is one of:
option 1: prevent the ASG name change upon deploy
option 2: dynamically update the plots with the new name
option 3: plot the same data without using the ASG name (but alternatives I find are EC2 instance ID that changes and ImageId and InstanceType that are common to more than one EC2, so won't work either)
My online-search-foo has turned out empty.
More Info:
I'm publishing these metrics with the cloud watch agent, by adjusting the conf file, as per the docs here:
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-on-EC2-Instance.html
Have a look at CloudWatch Search Expression Syntax. It allows you to use tokens for searching, e.g.:
SEARCH(' {AWS/CWAgent, AutoScalingGroupName} MetricName="mem_used_percent" rv8y2igice', 'Average', 300)
which would replace the entry for metrics like so:
"metrics": [
[ { "expression": "SEARCH(' {AWS/CWAgent, AutoScalingGroupName} MetricName=\"mem_used_percent\" rv8y2igice', 'Average', 300)", "label": "Expression1", "id": "e1" } ]
]
Simply search the desired result in the console, results that match the search appear.
To graph, all of the metrics that match your search, choose Graph search
and find the accurate search expression that you want in the Details on the Graphed metrics tab.
SEARCH('{CWAgent,AutoScalingGroupName,ImageId,InstanceId,InstanceType} mem_used_percent', 'Average', 300)
I am new to terraform - I have created remote tfstate in s3, and now there are some manual changes too that are done in my AWS infrastructure. I need to import those manual changes into tfstate.
I used the import command for some resources, but for some resources such as IAM policy etc, there is no such import command.
Also some resources such as DB are changed with new parameters added, and I need to import them as well. When I try to import those changes it says:
Error importing: 1 error(s) occurred:
* Can't import aws_security_group.Q8SgProdAdminSshInt, would collide
with an existing resource.
Please remove or rename this resource before continuing.
Any help would be appreciated. Thanks.
Before directly answering this question I think some context would help:
Behind the scenes, Terraform maintains a state file that contains a mapping from the resources in your configuration to the objects in the underlying provider API. When you create a new object with Terraform, the id of the object that was created is automatically saved in the state so that future commands can locate the referenced object for read, update, and delete operations.
terraform import, then, is a different way to create an entry in the state file. Rather than creating a new object and recording its id, instead the user provides an id on the command line. Terraform reads the object with that id and adds the result to the state file, after which it is indistinguishable in the state from a resource that Terraform created itself.
So with all of that said, let's address your questions one-by-one.
Importing Resources That Don't Support terraform import
Since each resource requires a small amount of validation and data-fetching code to do an import, not all resources are supported for import at this time.
Given what we know about what terraform import does from the above, in theory it's possible to skip Terraform's validation of the provided id and instead manually add the resource to the state. This is an advanced operation and must be done with care to avoid corrupting the state.
First, retrieve the state into a local file that you'll use for your local work:
terraform state pull >manual-import.tfstate
This will create a file manual-import.tfstate that you can open in a text editor. It uses JSON syntax, so though its internal structure is not documented as a stable format we can carefully edit it as long as we remain consistent with the expected structure.
It's simplest to locate an existing resource that is in the same module as where you want to import and duplicate and edit it. Let's assume we have a resources object like this:
"resources": {
"null_resource.foo": {
"type": "null_resource",
"depends_on": [],
"primary": {
"id": "5897853859325638329",
"attributes": {
"id": "5897853859325638329"
},
"meta": {},
"tainted": false
},
"deposed": [],
"provider": ""
}
},
Each attribute within this resources object corresponds to a resource in your configuration. The attribute name is the type and name of the resource. In this case, the resource type is null_resource and the attribute name is foo. In your case you might see something like aws_instance.server here.
The id attributes are, for many resources (but not all!), the main thing that needs to be populated. So we can duplicate this structure for a hypothetical IAM policy:
"resources": {
"null_resource.foo": {
"type": "null_resource",
"depends_on": [],
"primary": {
"id": "5897853859325638329",
"attributes": {
"id": "5897853859325638329"
},
"meta": {},
"tainted": false
},
"deposed": [],
"provider": ""
},
"aws_iam_policy.example": {
"type": "aws_iam_policy",
"depends_on": [],
"primary": {
"id": "?????",
"attributes": {
"id": "?????"
},
"meta": {},
"tainted": false
},
"deposed": [],
"provider": ""
}
},
The challenge at this step is to figure out what sort of id this resource requires. The only sure-fire way to know this is to read the code, which tells me that this resource expects the id to be the full ARN of the policy.
With that knowledge, we replace the two ????? sequences in the above example with the ARN of the policy we want to import.
After making manual changes to the state it's necessary to update the serial number at the top-level of the file. Terraform expects that any new change will have a higher serial number, so we can increment this number.
After completing the updates, we must upload the updated state file back into Terraform:
terraform state push manual-import.tfstate
Finally we can ask Terraform to refresh the state to make sure it worked:
terraform refresh
Again, this is a pretty risky process since the state file is Terraform's record of its relationship with the underlying system and it can be hard to recover if the content of this file is lost. It's often easier to simply replace a resource than to go to all of this effort, unless it's already serving a critical role in your infrastructure and there is no graceful migration strategy available.
Imports Colliding With Existing Resources
The error message given in your question is talking about an import "colliding" with an existing resource:
Error importing: 1 error(s) occurred:
* Can't import aws_security_group.Q8SgProdAdminSshInt, would collide with an existing resource.
Please remove or rename this resource before continuing.
The meaning of this message is that when Terraform tried to write the new resource to the state file it found a resource entry already present for the name aws_security_group.Q8SgProdAdminSshInt. This suggests that either it was already imported or that a new security group was already created by Terraform itself.
You can inspect the attributes of the existing resource in state:
terraform state show aws_security_group.Q8SgProdAdminSshInt
Compare the data returned with the security group you were trying to import. If the ids match then there's nothing left to do, since the resource was already imported.
If the ids don't match then you need to figure out which of the two objects is the one you want to keep. If you'd like to keep the one that Terraform already has, you can manually delete the one you were trying to import.
If you'd like to keep the one you were trying to import instead, you can drop the unwanted one from the Terraform state to make way for the import to succeed:
terraform state rm aws_security_group.Q8SgProdAdminSshInt
Note that this just makes Terraform "forget" the resource; it will still exist in EC2, and will need to be deleted manually via the console, command line tools, or API. Be sure to note down its id before deleting it to ensure that you can find it in order to to clean it up.
For resources you have to use import function or manually add terraform state like block..
or if there is any change in config like you mentioned dB configs.. if the dB resource is managed by terraform remote state.. terraform refresh will help you..