I'm refactoring a Ruby framework that is calling describe_instances and then filtering the response for just the VPC names.
It seems a waste of bandwidth to pull down the data for every instance in the region and then filter out the VPC ids in this way.
When I look at the documentation for filtering server side I see posts doing things like applying filters for all instances of type xx and so on.
What I want to do is pull down all VPC ids as a unique list.
Can anyone point me at an example of how to do that?
Thanks in advance
Never mind, I discovered the describe_vpcs endpoint:
def get_vpc_ids
ec2_object.describe_vpcs[:vpcs].each do |vpc|
#vpc_list.push(vpc[:vpc_id])
end
#vpc_list.uniq!
end
Related
Is there a simple way to retrieve all items from a DynamoDB table using a mapping template in an API Gateway endpoint? I usually use a lambda to process the data before returning it but this is such a simple task that a Lambda seems like an overkill.
I have a table that contains data with the following format:
roleAttributeName roleHierarchyLevel roleIsActive roleName
"admin" 99 true "Admin"
"director" 90 true "Director"
"areaManager" 80 false "Area Manager"
I'm happy with getting the data, doesn't matter the representation as I can later transform it further down in my code.
I've been looking around but all tutorials explain how to get specific bits of data through queries and params like roles/{roleAttributeName} but I just want to hit roles/ and get all items.
All you need to do is
create a resource (without curly braces since we dont need a particular item)
create a get method
use Scan instead of Query in Action while configuring the integration request.
Configurations as follows :
enter image description here
now try test...you should get the response.
to try it out on postman deploy the api first and then use the provided link into postman followed by your resource name.
API Gateway allows you to Proxy DynamoDB as a service. Here you have an interesting tutorial on how to do it (you can ignore the part related to index to make it work).
To retrieve all the items from a table, you can use Scan as the action in API Gateway. Keep in mind that DynamoDB limits the query sizes to 1MB either for Scan and Query actions.
You can also limit your own query before it is automatically done by using the Limit parameter.
AWS DynamoDB Scan Reference
I am looking to setup some CloudFormation stuff that is able to find any email addresses in CloudWatch logs and let us know that one slipped through the cracks. I thought this would be a simple process of using a RegEx pattern that catches all the possible variations and email address can have, and using that as a filter. Having discovered that CloudWatch filtering does not support RegEx I've become a bit stumped as to how to write a filter that can be relied upon to catch any email address.
Has anyone done something similar to this, or know where a good place to start would be?
Amazon has launched a service called CloudWatch insights and it allows to filter messages logs. In the previous link you have examples of queries.
You need to select the CloudWatch Log Group and the period of time in which search.
Example:
fields #message
| sort #timestamp desc
| filter #message like /.*47768.*/
If you're exporting the logs somewhere (Like Sumologic, Datadog etc) thats a better place to do that alerting.
If not and you're exporting them into S3 then a triggered lambda function that runs the check might do the trick. Could be expensive long term though.
The solution that we landed upon was to pass stings through a RegEx pattern that recognises email addresses before they logged into AWS. Replacing any matches with [REDACTED]. Which is simple enough to do in a lambda.
I made aws auto deployment code with boto3 library.
In my code, get all service list and use it.
I have to get lastest service. But I think there is no order option.
(https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html#ECS.Client.list_services)
Sometimes, first element is latest service.
But sometimes, old service is placed in first element.
Is there any option or way to get latest service?
Thanks.
The list_services method does not return details of individual services. It simply lists the services, and returns you a list of identifiers (ARNs) for those services.
To get more details of a given service, you can use describe_services. This allows you to get details of up to 10 services at a time.
So, take the list of service identifiers that you get back from list_services, and pass it to describe_services (with at most 10 service identifiers). Something like this (untested):
list_response = client.list_services(
cluster='xyz',
launchType='EC2'
)
desc_response = client.describe_services(
cluster='xyz',
services=list_response['serviceArns']
)
Note that you will have to do pagination using maxResults / nextToken if there are a lot of results.
Simple question, but I suspect it doesn't have a simple or easy answer. Still, worth asking.
We're creating an implementation for push notifications using AWS with our Web Server running on EC2, sending messages to a queue on SQS, which is dealt with using Lambda, which is sent finally to SNS to be delivered to the iOS/Android apps.
The question I have is this: is there a way to query SNS endpoints based on the custom user data that you can provide on creation? The only way I see to do this so far is to list all the endpoints in a given platform application, and then search through that list for the user data I'm looking for... however, a more direct approach would be far better.
Why I want to do this is simple: if I could attach a User Identifier to these Device Endpoints, and query based on that, I could avoid completely having to save the ARN to our DynamoDB database. It would save a lot of implementation time and complexity.
Let me know what you guys think, even if what you think is that this idea is impractical and stupid, or if searching through all of them is the best way to go about this!
Cheers!
There isn't the ability to have a "where" clause in ListTopics. I see two possibilities:
Create a new SNS topic per user that has some identifiable id in it. So, for example, the ARN would be something like "arn:aws:sns:us-east-1:123456789:know-prefix-user-id". The obvious downside is that you have the potential for a boat load of SNS topics.
Use a service designed for this type of usage like PubNub. Disclaimer - I don't work for PubNub or own stock but have successfully used it in multiple projects. You'll be able to target one or many users this way.
According the the [AWS documentation][1] if you try and create a new Platform Endpoint with the same User Data you should get a response with an exception including the ARN associated with the existing PlatformEndpoint.
It's definitely not ideal, but it would be a round about way of querying the User Data Endpoint attributes via exception.
//Query CustomUserData by exception
CreatePlatformEndpointRequest cpeReq = new CreatePlatformEndpointRequest().withPlatformApplicationArn(applicationArn).withToken("dummyToken").withCustomUserData("username");
CreatePlatformEndpointResult cpeRes = client.createPlatformEndpoint(cpeReq);
You should get an exception with the ARN if an endpoint with the same withCustomUserData exists.
Then you just use that ARN and away you go.
I'm accessing EC2 with the aws-sdk for Ruby. I have an array of instances from describe_instances().
This provides me with the state of the instances and even a state transition reason. But how can I get a time for the state transition?
Edit
So I have:
client=Aws::EC2::Client()
resp =client.describe_instances({ filters })
and I would need
resp.reservations[0].instances[0].state_transition_time #=> Time
similar to
resp.reservations[0].instances[0].state_transition_reason #=> String
This information is not available via the Amazon EC2 API at this time. The aws-sdk gem returns all of the information available from the DescribeInstances operation as documented here: http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html
The State Transition Reason is not always populated with a date and time and may not even be populated at all per the documentation. I have not found any hints in the documentation that specify the conditions in which you DO get a date/time, but in my experience, the date/time are present in the State Transition Reason for between 30 and 90 days. After that, the reason seems to persist, but the date is dropped from the string.
All of the documentation that I can find is listed here:
Attribute Definition
EC2 API - Ruby