I'm using Packer to query AWS to find an AMI to use as a source AMI. I'd like to find AMI by tags. Here is my code.
"source_ami_filter": {
"filters": {
"tag": "type=Ubuntu Base"
},
"owners": ["self"],
"most_recent": true
}
which receives this error
amazon-ebs: Error querying AMI: InvalidParameterValue: The filter 'Filter.tag' is invalid
I can't for the life of me figure out how to format that filter. Any help would be greatly appreciated.
Your sample code is very close, but the tag name should be specified in the filters key instead of the value.
This modification of your code should work to find the AMI with a "type" tag containing the value "Ubuntu Base":
"source_ami_filter": {
"filters": {
"tag:type": "Ubuntu Base"
},
"owners": ["self"],
"most_recent": true
}
The Packer documentation for source_ami_filter explains that "any filter described in the docs for DescribeImages is valid."
Then the AWS EC2 documentation for DescribeImages shows that a filter for a value contained in a given tag should use the format, tag:key=value:
tag:key=value - The key/value combination of a tag assigned to the resource. Specify the key of the tag in the filter name and the value of the tag in the filter value. For example, for the tag Purpose=X, specify tag:Purpose for the filter name and X for the filter value.
Related
I am trying to copy AWS SSM parameter(for cloudwatch) from one region to another. I have the json which is created as a String in one region.
I am trying to write a terraform script to create this ssm parameter in another region.
According to the terraform documentation, I need to do this
resource "aws_ssm_parameter" "foo" {
name = "foo"
type = "String"
value = "bar"
}
In my case value is a json. Is there a way to store the json in a file and pass this file as value to the above resource? I tried using jsonencode,
resource "aws_ssm_parameter" "my-cloudwatch" {
name = "my-cloudwatch"
type = "String"
value = jsonencode({my-json})
that did not work either. I am getting this error
Extra characters after interpolation expression I believe this is because the json has characters like quotes and colon.
Any idea?
I tested the following & this worked for me:
resource "aws_ssm_parameter" "my-cloudwatch" {
name = "my-cloudwatch"
type = "String"
#value = file("${path.module}/ssm-param.json")
value = jsonencode(file("${path.module}/files/ssm-param.json"))
}
./files/ssm-param.json content:
{
"Value": "Something"
}
and the parameter store value looks like this:
"{\n \"Value\": \"Something\"\n}"
I just faced this issue the $ in the CW config is causing the problem. Use $$
"Note: If you specify the template as a literal string instead of loading a file, the inline template must use double dollar signs (like $${hello}) to prevent Terraform from interpolating values from the configuration into the string. "
https://www.terraform.io/docs/configuration-0-11/interpolation.html
"metrics": {
"append_dimensions": {
"AutoScalingGroupName": "$${aws:AutoScalingGroupName}",
"ImageId": "$${aws:ImageId}",
"InstanceId": "$${aws:InstanceId}",
"InstanceType": "$${aws:InstanceType}"
},
I prefer Pauls aproach though.
You need to insert your json with escaped quotas, is a little trick in AWS, and you need to parse this when retrieve:
const value = JSON.parse(Value)
Example of insert:
"Value": "\"{\"flag\":\"market_store\",\"app\":\"ios\",\"enabled\":\"false\"}\"",
I'm creating a AWS glue crawler using API, I have created a crawler already and it is working fine. Now i want to implemt IncrementalCrawler functionality to it. When I try to give Recrawler parameter for update_crawler() it is throwing me with the error. The same thing is happening is happening with create_crawler() also.
mycode
`crawler_args = {"Name": args['CRAWLER_NAME'], "Role": args['IAM_ROLE'], "DatabaseName": args['DB_NAME'],"RecrawlPolicy": {"RecrawlBehavior": "CRAWL_NEW_FOLDERS_ONLY"}
"Targets":
{"S3Targets": [{"Path": f"s3:/../{src_obj}/"}]},
"TablePrefix": ""}
glue.create_crawler(**crawler_args)`
"Parameter validation failed:
Unknown parameter in input: "RecrawlPolicy", must be one of: Name, Role, DatabaseName, Description, Targets, Schedule, Classifiers, TablePrefix, SchemaChangePolicy, Configuration, CrawlerSecurityConfiguration"
I believe it is because of the boto3 version im using, if that is the case how to update boto3 to latest version.
It looks like you are missing a comma, can you try:
crawler_args = {"Name": args['CRAWLER_NAME'], "Role": args['IAM_ROLE'], "DatabaseName": args['DB_NAME'],"RecrawlPolicy": {"RecrawlBehavior": "CRAWL_NEW_FOLDERS_ONLY"},
"Targets":
{"S3Targets": [{"Path": f"s3:/../{src_obj}/"}]},
"TablePrefix": ""}
I have my product uploaded to AWS as an AMI through Hashicorp's Packer. Now I'ld like to automate the last step, publishing it to the marketplace. The product already exists, it's only about adding a revision.
After reading this article, the API_StartChangeSet doc, this add revisions user guide & fiddling with the marketplace console, I think I just have to
aws marketplace-catalog start-change-set --catalog AWSMarketplace --change-set-name "$VERSION" --change-set '[ {"ChangeType": "AddRevisions", "Entity": {"Identifier": "REDACTED#29","Type": "ServerProduct#1.0"}, "Details": "{\"DataSetArn\": \"?????\", \"RevisionArns\": [\"?????\"] }" ]'
I'm having a hard time coming up with "Details" part. I've my AMI id. I guess that goes in the RevisionsArns ? What should I put in the DataSetArn, the "EntityArn" from the output of aws marketplace-catalog describe-entity --catalog AWSMarketplace --entity-id REDACTED ?
Details facet here is just a product type specific facet, encoded as json string. For the AMI that you are offering in the AWS Marketplace, it could include support information, region availability or any other info that provides a descriptive text regarding your change. For example:
"Details": "{\"Description\":{}, \"PromotionalResources\":{}, \"RegionAvailability\":{}, \"SupportInformation\":{}}",
The example you found does not necessarily mean that you have to have EntityArn and RevisionsArns. The Details facet is used as an information describing the details of your change.
Check here.
Turns out I didn't found the good documentation, my last link being about AWS Data Exchange, whose "Details" field's contents were confusing.
Here the relevant documentation: Marketplace catalog AMI add version, and here's the snippet I was looking for
"Details": "{
\"Version\": {
\"VersionTitle\": \"*My new title*\",
\"ReleaseNotes\": \"*My new Release notes*\"
},
\"DeliveryOptions\": [
{
\"Details\": {
\"AmiDeliveryOptionDetails\": {
\"AmiSource\": {
\"AmiId\": \"ami-1234567890abcdef\",
\"AccessRoleArn\": \"arn:aws:iam::12345678901:role/AwsMarketplaceAmiIngestion\",
\"UserName\": \"ec2-user\",
\"OperatingSystemName\": \"AMAZONLINUX\",
\"OperatingSystemVersion\": \"Amazon Linux 2 AMI 2.0.20210126.0 x86_64 HVM gp2\"
},
\"UsageInstructions\": \"Easy to use AMI\",
\"RecommendedInstanceType\": \"m4.xlarge\",
\"SecurityGroups\": [
{
\"IpProtocol\": \"tcp\",
\"FromPort\": 443,
\"ToPort\": 443,
\"IpRanges\": [
\"0.0.0.0/0\"
]
}
]
}
}
}
]
}"
I am currently assigning AWS media-live channels to a specific group by a custom tag and want to get the (CostExplorer) GetReservationUtilization for a group's channels by filtering by tag. The AWS documentation for GetReservationUtilization lists the Filtering options as:
"Filter": {
.
.
"Tags": {
"Key": "string",
"MatchOptions": [ "string" ],
"Values": [ "string" ]
}
.
.
}
I interpret it as it should be possible to sort by a custom set tag via:
"Key": "Group",
"Value": [customId]
But I get an error that says "An error occurred (ValidationException) when calling the GetReservationUtilization operation: Tags expression is not allowed, allowed expression(s): And, Not, Dimensions"
Feels like I have tried everything possible but I cant seem to get it to work.
Have you looked at the examples boto3 documentation?
Seems you may need to wrap the tag element inside of the And, Not or supply dimensions
For anyone coming here in the future, sorting reservation-utilization by the tag dimension is currently not supported. The following dimensions are supported:
AZ
CACHE_ENGINE
DEPLOYMENT_OPTION
INSTANCE_TYPE
LINKED_ACCOUNT
OPERATING_SYSTEM
PLATFORM
REGION
SERVICE
SCOPE
TENANCY
As specified in the AWS API docs.
We have two types of SNS messages coming in:
1. has MessageAttributes empty like this:
"MessageAttributes": {}
2. has MessageAttributes coming in like this:
"MessageAttributes": {
"Generator": {
"Type": "String",
"Value": "some-service"
}
}
I would like to use a filter subscription policy that ignores the second type but passes the first type to the subscriber.
So I tried this for the policy:
{
"Generator": [
{
"exists": false
}
]
}
I thought this would mean it will only pass along messages that do NOT contain the Generator key in MessageAttributes
However I am seeing now that no messages are getting passed along.
The AWS Subscription Filter docs seem to support this as a solution, but they only show the opposite way of checking that a key does exist, so I'm not sure if they support checking a key doesn't exist: https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html#attribute-key-matching
Is this possible?
The answer from #David Adams is out of date. See the Attribute key matching docs.
Use "exists": false to return incoming messages that don't include the specified attribute.
It is now possible to exclude any messages that have a particular key by using the policy:
{
"key": [
{
"exists": false
}
]
}
Late response but may be helpful to someone.
Filtering out by lack of existance is not possible. See the bottom of https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html#attribute-key-matching
Note: You cannot use the exists operator to match messages in which an attribute does not exist. Filtering will NOT match any messages if you set [{"exists": false}].
You could pass a string 'null' or similar to the generator attribute if it is non existant maybe?