In Postman I'm calling the CloudWatch API and the GetLogEvents action. This works well and I'm able to retrieve logs. However, I have an AWS Role per AWS environment, so how do I specify a different role to switch to so I can retrieve logs from, for example, my UAT environment?
Many thanks
Related
I wanted to know if there was a way to track alerts or audit anything that happens with the AWS account like who changed what and why. I did find this https://docs.aws.amazon.com/opensearch-service/latest/developerguide/audit-logs.html where they use a comand line for enabling audit logs on an existing domain: aws opensearch update-domain-config --domain-name my-domain --log-publishing-options "AUDIT_LOGS={CloudWatchLogsLogGroupArn=arn:aws:logs:us-east-1:123456789012:log-group:my-log-group,Enabled=true}" but this is in regard to Amazon OpenSearch Service which I believe is only free for 12 months if you haven't used already. AWS Audit Manager. I am aware there are services that can do this but require a fee and I wanted to know if there were any free options
From the AWS documentation:
With AWS CloudTrail, you can monitor your AWS deployments in the cloud by getting a history of AWS API calls for your account, including API calls made by using the AWS Management Console, the AWS SDKs, the command line tools, and higher-level AWS services. You can also identify which users and accounts called AWS APIs for services that support CloudTrail, the source IP address from which the calls were made, and when the calls occurred. You can integrate CloudTrail into applications using the API, automate trail creation for your organization, check the status of your trails, and control how administrators turn CloudTrail logging on and off.
AWS Config provides a detailed view of the resources associated with your AWS account, including how they are configured, how they are related to one another, and how the configurations and their relationships have changed over time.
Basically, AWS CloudTrail keeps a log of API calls (requests to AWS to do/change stuff), while AWS Config tracks how individual configurations have changed over time (for a limited range of resources, such as Security Group rule being changed).
I'm running my Python web app on an EC2, and I want it to report some custom (app-level) metrics to CloudWatch.
Sample metrics are (uplink) request duration and similar.
From what I understand, I have to either use boto3 or the AWS CLI in order to do that. However:
My app doesn't use boto3 for functionality, so it seems like an overkill to use it just for reporting metrics
I have to be authenticated - unlike with Lambda, just the fact I'm running inside an EC2 does not mean I'm automatically authenticated.
What's the best practice here? My app doesn't have to run on EC2 (can be run it on GCP, Azure, or a custom server), so I really don't want to import boto3 into the code.
You can attach role and have access to AWS cervices without credentials.
It's all depends on your metric. Probably you don't need to use CloudWatch Metrics, but X-Ray in some cases.
If your app doesn't use both3 it's not means what you need to use that, but it's easiest way to call AWS API
If you are using GCP or Azure, please, use their monitoring services.
If you need to collect some custom metrics from anywhere, please, use some metrics service or implement your own API
You can look into CloudWatch Embedded Metrics Format (EMF).
You would need to install and configure CloudWatch Agent on your EC2 instance and then you can use python EMF library to publish metrics.
With this approach:
Your application is not calling CloudWatch APIs directly. CloudWatch agent does the publishing.
You get custom metrics and EMF log entries in CloudWatch Logs, which can then be used with CloudWatch Logs Insights and Contributor Insights.
But you still need to provide a way to CloudWatch agent to authenticate against CloudWatch APIs. On EC2 instances this is done via the role your instance assumes.
I want to call api gateway from our own backend which is a ECS cluster, and I want to use IAM authorization, is there any way to not manually sign the request using Access Key and Secret Key?
For example when Lambda call KMS to decrypt environment variables, no need to configure the AWS SDK. Wondering if there's similar thing for API gateway.
It is definitely possible, even more - it's a security best practice. You can assign IAM roles to all computing services of AWS: Lambda, EC2, ECS, Beanstalk etc. On ECS you can assign IAM roles to your tasks.
It gives a great benefit, which is well described in official docs:
Benefits of Using IAM Roles for Tasks
Credential Isolation: A container can only retrieve credentials for the IAM role that is defined in the task definition to which it belongs; a container never has access to credentials that are intended for another container that belongs to another task.
Authorization: Unauthorized containers cannot access IAM role credentials defined for other tasks.
Auditability: Access and event logging is available through CloudTrail to ensure retrospective auditing. Task credentials have a context of taskArn that is attached to the session, so CloudTrail logs show which task is using which role.
This link will help you: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html
How do I find out what IAM user created certain metrics and alarms?
I tried CLI commands: describe-alarms and list-metrics and they don't seem to provide this information nor when looking at the configuration for the alarms and metrics in the AWS console.
While users call APIs to create resources in AWS, those resources are associated with the AWS Account rather than a user. As long as a user has permission to create resources in the account, the resources will be created/viewed/deleted.
However, you can use AWS CloudTrail to retrieve information about the API calls themselves:
AWS CloudTrail is an AWS service that helps you enable governance, compliance, and operational and risk auditing of your AWS account. Actions taken by a user, role, or an AWS service are recorded as events in CloudTrail. Events include actions taken in the AWS Management Console, AWS Command Line Interface, and AWS SDKs and APIs.
My code is running on an EC2 machine. I use some AWS services inside the code, so I'd like to fail on start-up if those services are unavailable.
For example, I need to be able to write a file to an S3 bucket. This happens after my code's been running for several minutes, so it's painful to discover that the IAM role wasn't configured correctly only after a 5 minute delay.
Is there a way to figure out if I have PutObject permission on a specific S3 bucket+prefix? I don't want to write dummy data to figure it out.
You can programmatically test permissions by the SimulatePrincipalPolicy API
Simulate how a set of IAM policies attached to an IAM entity works with a list of API actions and AWS resources to determine the policies' effective permissions.
Check out the blog post below that introduces the API. From that post:
AWS Identity and Access Management (IAM) has added two new APIs that enable you to automate validation and auditing of permissions for your IAM users, groups, and roles. Using these two APIs, you can call the IAM policy simulator using the AWS CLI or any of the AWS SDKs. Use the new iam:SimulatePrincipalPolicy API to programmatically test your existing IAM policies, which allows you to verify that your policies have the intended effect and to identify which specific statement in a policy grants or denies access to a particular resource or action.
Source:
Introducing New APIs to Help Test Your Access Control Policies
Have you tried the AWS IAM Policy Simulator. You can use it interactively, but it also has some API capabilities that you may be able to use to accomplish what you want.
http://docs.aws.amazon.com/IAM/latest/APIReference/API_SimulateCustomPolicy.html
Option 1: Upload an actual file when you app starts to see if it succeeds.
Option 2: Use dry runs.
Many AWS commands allow for "dry runs". This would let you execute your command at the start without actually doing anything.
The AWS CLI for S3 appears to support dry runs using the --dryrun option:
http://docs.aws.amazon.com/cli/latest/reference/s3/cp.html
The Amazon EC2 docs for "Dry Run" says the following:
Checks whether you have the required permissions for the action, without actually making the request. If you have the required permissions, the request returns DryRunOperation; otherwise, it returns UnauthorizedOperation.
Reference: http://docs.aws.amazon.com/AWSEC2/latest/APIReference/CommonParameters.html