I am currently assigning AWS media-live channels to a specific group by a custom tag and want to get the (CostExplorer) GetReservationUtilization for a group's channels by filtering by tag. The AWS documentation for GetReservationUtilization lists the Filtering options as:
"Filter": {
.
.
"Tags": {
"Key": "string",
"MatchOptions": [ "string" ],
"Values": [ "string" ]
}
.
.
}
I interpret it as it should be possible to sort by a custom set tag via:
"Key": "Group",
"Value": [customId]
But I get an error that says "An error occurred (ValidationException) when calling the GetReservationUtilization operation: Tags expression is not allowed, allowed expression(s): And, Not, Dimensions"
Feels like I have tried everything possible but I cant seem to get it to work.
Have you looked at the examples boto3 documentation?
Seems you may need to wrap the tag element inside of the And, Not or supply dimensions
For anyone coming here in the future, sorting reservation-utilization by the tag dimension is currently not supported. The following dimensions are supported:
AZ
CACHE_ENGINE
DEPLOYMENT_OPTION
INSTANCE_TYPE
LINKED_ACCOUNT
OPERATING_SYSTEM
PLATFORM
REGION
SERVICE
SCOPE
TENANCY
As specified in the AWS API docs.
Related
I have my product uploaded to AWS as an AMI through Hashicorp's Packer. Now I'ld like to automate the last step, publishing it to the marketplace. The product already exists, it's only about adding a revision.
After reading this article, the API_StartChangeSet doc, this add revisions user guide & fiddling with the marketplace console, I think I just have to
aws marketplace-catalog start-change-set --catalog AWSMarketplace --change-set-name "$VERSION" --change-set '[ {"ChangeType": "AddRevisions", "Entity": {"Identifier": "REDACTED#29","Type": "ServerProduct#1.0"}, "Details": "{\"DataSetArn\": \"?????\", \"RevisionArns\": [\"?????\"] }" ]'
I'm having a hard time coming up with "Details" part. I've my AMI id. I guess that goes in the RevisionsArns ? What should I put in the DataSetArn, the "EntityArn" from the output of aws marketplace-catalog describe-entity --catalog AWSMarketplace --entity-id REDACTED ?
Details facet here is just a product type specific facet, encoded as json string. For the AMI that you are offering in the AWS Marketplace, it could include support information, region availability or any other info that provides a descriptive text regarding your change. For example:
"Details": "{\"Description\":{}, \"PromotionalResources\":{}, \"RegionAvailability\":{}, \"SupportInformation\":{}}",
The example you found does not necessarily mean that you have to have EntityArn and RevisionsArns. The Details facet is used as an information describing the details of your change.
Check here.
Turns out I didn't found the good documentation, my last link being about AWS Data Exchange, whose "Details" field's contents were confusing.
Here the relevant documentation: Marketplace catalog AMI add version, and here's the snippet I was looking for
"Details": "{
\"Version\": {
\"VersionTitle\": \"*My new title*\",
\"ReleaseNotes\": \"*My new Release notes*\"
},
\"DeliveryOptions\": [
{
\"Details\": {
\"AmiDeliveryOptionDetails\": {
\"AmiSource\": {
\"AmiId\": \"ami-1234567890abcdef\",
\"AccessRoleArn\": \"arn:aws:iam::12345678901:role/AwsMarketplaceAmiIngestion\",
\"UserName\": \"ec2-user\",
\"OperatingSystemName\": \"AMAZONLINUX\",
\"OperatingSystemVersion\": \"Amazon Linux 2 AMI 2.0.20210126.0 x86_64 HVM gp2\"
},
\"UsageInstructions\": \"Easy to use AMI\",
\"RecommendedInstanceType\": \"m4.xlarge\",
\"SecurityGroups\": [
{
\"IpProtocol\": \"tcp\",
\"FromPort\": 443,
\"ToPort\": 443,
\"IpRanges\": [
\"0.0.0.0/0\"
]
}
]
}
}
}
]
}"
I am using cloudwatch subscription filter which automatically sends logs to elasticsearch aws and then I use Kibana from there. The issue is that everyday cloudwatch creates a new indice due to which I have to manually create the new index pattern each day in kibana. Accordingly I will have to create new monitors and alerts in kibana as well each day. I have to automate this somehow. Also if there is better option with which I can go forward would be great. I know datadog is one good option.
Typical work flow will look like this (there are other methods)
Choose a pattern when creating an index. Like staff-202001, staff-202002, etc
Add each index to an alias. Like staff
This can be achieved in multiple ways, easiest is to create a template with index pattern , alias and mapping.
Example: Any new index created matching the pattern staff-* will be assigned with given mapping and attached to alias staff and we can query staff instead of individual indexes and setup alerts.
We can use cwl--aws-containerinsights-eks-cluster-for-test-host to run queries.
POST _template/cwl--aws-containerinsights-eks-cluster-for-test-host
{
"index_patterns": [
"cwl--aws-containerinsights-eks-cluster-for-test-host-*"
],
"mappings": {
"properties": {
"id": {
"type": "keyword"
},
"firstName": {
"type": "text"
},
"lastName": {
"type": "text"
}
}
},
"aliases": {
"cwl--aws-containerinsights-eks-cluster-for-test-host": {}
}
}
Note: If unsure of mapping, we can remove mapping section.
I am trying to do this in AWS elasticsearch, whereby I create a template for the pattern application-logs-*, and then I want to apply a index policy log-rotation-policy for all indexes which match that expression. I have created my policy successfully, but when I try to create a template like so:
PUT _template/application-logs
{
"index_patterns" : [
"application-logs-*"
],
"settings" : {
"index.lifecycle.name": "log-rotation-policy",
}
}
I get an error:
"type": "illegal_argument_exception",
"reason": "unknown setting [index.policy_id] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"
The AWS documentation is extremely vague,
Ok sorry I thought I would post this answer anyway because as I was writing this I figured out the problem, the correct key o use is: opendistro.index_state_management.policy_id so it should be:
PUT _template/application-logs
{
"index_patterns" : [
"application-logs-*"
],
"settings" : {
"opendistro.index_state_management.policy_id": "log-rotation-policy",
}
}
I found the answer here.
We have two types of SNS messages coming in:
1. has MessageAttributes empty like this:
"MessageAttributes": {}
2. has MessageAttributes coming in like this:
"MessageAttributes": {
"Generator": {
"Type": "String",
"Value": "some-service"
}
}
I would like to use a filter subscription policy that ignores the second type but passes the first type to the subscriber.
So I tried this for the policy:
{
"Generator": [
{
"exists": false
}
]
}
I thought this would mean it will only pass along messages that do NOT contain the Generator key in MessageAttributes
However I am seeing now that no messages are getting passed along.
The AWS Subscription Filter docs seem to support this as a solution, but they only show the opposite way of checking that a key does exist, so I'm not sure if they support checking a key doesn't exist: https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html#attribute-key-matching
Is this possible?
The answer from #David Adams is out of date. See the Attribute key matching docs.
Use "exists": false to return incoming messages that don't include the specified attribute.
It is now possible to exclude any messages that have a particular key by using the policy:
{
"key": [
{
"exists": false
}
]
}
Late response but may be helpful to someone.
Filtering out by lack of existance is not possible. See the bottom of https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html#attribute-key-matching
Note: You cannot use the exists operator to match messages in which an attribute does not exist. Filtering will NOT match any messages if you set [{"exists": false}].
You could pass a string 'null' or similar to the generator attribute if it is non existant maybe?
I'm using Packer to query AWS to find an AMI to use as a source AMI. I'd like to find AMI by tags. Here is my code.
"source_ami_filter": {
"filters": {
"tag": "type=Ubuntu Base"
},
"owners": ["self"],
"most_recent": true
}
which receives this error
amazon-ebs: Error querying AMI: InvalidParameterValue: The filter 'Filter.tag' is invalid
I can't for the life of me figure out how to format that filter. Any help would be greatly appreciated.
Your sample code is very close, but the tag name should be specified in the filters key instead of the value.
This modification of your code should work to find the AMI with a "type" tag containing the value "Ubuntu Base":
"source_ami_filter": {
"filters": {
"tag:type": "Ubuntu Base"
},
"owners": ["self"],
"most_recent": true
}
The Packer documentation for source_ami_filter explains that "any filter described in the docs for DescribeImages is valid."
Then the AWS EC2 documentation for DescribeImages shows that a filter for a value contained in a given tag should use the format, tag:key=value:
tag:key=value - The key/value combination of a tag assigned to the resource. Specify the key of the tag in the filter name and the value of the tag in the filter value. For example, for the tag Purpose=X, specify tag:Purpose for the filter name and X for the filter value.