We are working on a POC in AWS Amplify and we have a requirement to store all the deployments logs (individual deploy jobs) for a period of 1 year. At the moment we have a functional application deployed in Amplify and we can see all the access logs in the Amplify console. But the deployment logs are only available through the AWS CLI and I cannot find them anywhere in the console. when I query aws amplify list-jobs --app-id "${AMPLIFY_APP_ID}" --branch-name "${branch}" it will give me the below output,
{
"jobSummaries": [
{
"jobArn": "xxxxxxxxxxxxxx",
"jobId": "xxxxxxxxxxxxxx",
"commitId": "xxxxxxxxxxxxxx",
"commitMessage": "xxxxxxxxxxxxxx",
"commitTime": "xxxxxxxxxxxxxx",
"startTime": "xxxxxxxxxxxxxx",
"status": "SUCCEED",
"endTime": "xxxxxxxxxxxxxx"
}
}
I noticed we can use log groups and the Lambda function to store access logs. I need to find a way to store these deployment logs in an S3 or some kind of Database. Appreciate your suggestions.
Related
AWS has auto-join/auto-enable recording of new resources and accounts in Security Hub, which I am happily using.
I wanted to enable the same kind of setting for AWS Config, which currently require manual setup for all new accounts of my organisation. However, I failed to find anything similar to aforementioned Security Hub feature in Config.
Am I missing it or is it currently not available out-of-box?
#Ido Van Orell I do not think there is a native feature in AWS Config but what you can leverage for this is CloudFormation stacksets.
Create a CloudFormation template that enables AWS Config. Deploy it as stackset with automatic deployment enabled. This should ensure that when a new account joins your org AWS Config is enabled.
If you are using AWS Control Tower by any chance, you also get an event when account is vended which you can run automation on the back off.
{
"detail-type": ["AWS Service Event via CloudTrail"],
"source": ["aws.controltower"],
"detail":
{
"eventName": ["CreateManagedAccount"],
"serviceEventDetails":
{ "createManagedAccountStatus": { "state": ["SUCCEEDED"] } },
},
}
Use AWS Config Aggregator's integration with AWS Organisations to get a single pane of glass view across all accounts and regions for org where Config is enabled. Refer this
I have performed aws configure and ask configure after installing ask-cli.
While setting up new skill using ask new selected NodeJS, AWS with CloudFormation.
Trying to deploy the skill using ask deploy, getting [Error]: CliError: The CloudFormation deploy failed for Alexa region "default": Access Denied.
Tried setting the region in ~/.aws/config and in ~/.aws/credentials, still running into same error.
What should be done to fix the issue?
Skill creation
Error deploying skill
I've been able to deploy.
After running aws configure, I called ask new, and I think the solution was to not select AWS With CloudFormation but AWS Lambda:
I wanted to use an existing skill that I previously created in the web UI. So I created two folders: lambda and skill-package. Then I used ask init saying I don't want to use AWS CloudFormation to deploy:
Next, I added my region in ask-resources.json, under skillInfrastructure:
{
"askcliResourcesVersion": "2020-03-31",
"profiles": {
"default": {
"skillMetadata": {
"src": "./skill-package"
},
"code": {
"default": {
"src": "./lambda"
}
},
"skillInfrastructure": {
"type": "#ask-cli/lambda-deployer",
"userConfig": {
"runtime": "nodejs12.x",
"handler": "index.js",
"awsRegion": "eu-west-1"
}
}
}
}
}
And I finished with ask deploy that worked!
We have developed an AWS Serverless Lambda application using dotnetcore to perform operations on EC2 Instances, say start or stop EC2 instance and integrated with Aws API Gateway.
serverless.template in dotnetcore application
"StartInstanceById" : {
"Type" : "AWS::Serverless::Function",
"Properties": {
"Handler": "EC2_Monitoring_Serverless::EC2_Monitoring_Serverless.Functions::StartInstanceById",
"Runtime": "dotnetcore2.1",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": "arn:aws:iam::2808xxxx1013:role/lamda_start_stop",
"Policies": [ "AWSLambdaBasicExecutionRole" ],
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "/instances",
"Method": "Get"
}
}
}
}
}
The above Lambda function is working fine for starting ec2 instance when I invoking the API gateway url.
For calling these API's, We have created Angular 6 application and provided authentication using Aws Cognito Userpools.
So the cognito user logins into the website and gets all EC2 informations.
If the user wants to stop / start the EC2 instance, user will click on the particular button which invokes the relevant api gateway url of the lambda functions and It's working fine.
Now the question is who performed that action. After so much of research on stackoverflow and aws community forums for knowing who started or stopped the EC2 instances , I found Aws CloudTrail logs the information when user start or stopped the instance.
So I created a trail and I can see the logs in S3 buckets. But in every log I opened, I saw that the role "arn:aws:iam::2808xxxx1013:role/lamda_start_stop" is captured. I know this is because of the Lambda function. But I want to know who really stopped the instance.
Please advice how to capture user details!
The reason lambda execution role is getting printed in cloudtrail, is because it has initiated the process to stop the ec2 instance. Here the role is assumed (instead of actual user).
To print your actual user, you need to implement logs at your lambda, which will print logs to Cloudwatch. You can get the actual user or any other custom information from those logs.
I have been trying to figure out, how to find s3 URL or s3 object name which is created after the codedeploy deployment with new commit ID.
Here is the aws-cli way to list application revisions and their s3 Location:
aws deploy list-application-revisions --application <your application name>
Example output:
{ "revisionType": "S3",
"s3Location": {
"bucket:" "mybucket",
"key": "mys3objectname",
"bundleType": "zip",
"eTag": "ff1e77d70adaedfd14cecba209811a94"
}
}
To construct an s3 url from this, use:
https://s3-<region>.amazonaws.com/<bucket>/<key>
If you need to find your application name, use:
aws deploy list-applications
I'm trying to forward the EC2 Launch logs to cloudwatch from my win 2016-based EC2 instance.
For some reason I can't see the log groups for this specific category.
Here's example of my AWS.EC2.Windows.CloudWatch.json:
{
"IsEnabled": true,
"EngineConfiguration": {
"PollInterval": "00:00:15",
"Components": [
{
"Id": "Ec2Config",
"FullName": "AWS.EC2.Windows.CloudWatch.CustomLog.CustomLogInputComponent,AWS.EC2.Windows.CloudWatch",
"Parameters": {
"LogDirectoryPath": "C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\\Log",
"TimestampFormat": "yyyy-MM-ddTHH:mm:ss.fffZ:",
"Encoding": "UTF-8",
"Filter": "UserdataExecution.log",
"CultureName": "en-US",
"TimeZoneKind": "UTC"
}
},
{
"Id": "EC2ConfigSink",
"FullName": "AWS.EC2.Windows.CloudWatch.CloudWatchLogsOutput,AWS.EC2.Windows.CloudWatch",
"Parameters": {
"Region": "eu-west-1",
"LogGroup": "/my-customer/deployment/ec2config-userdata",
"LogStream": "ec2config-userdata"
}
}
...
I have a few more definitions in this file
...],
"Flows": {
"Flows":
[
"Ec2Config,EC2ConfigSink",
... other references here
]
}
}
Cloudwatch agent starts and doesn't report any errors, I can see data from other sources (some application log files - I skipped the definitions intentionally)
It means the cloudwatch config file is correct and is applied / placed in a correct directory.
Logs are coming through with no problem except for the EC2 launch logs.
I'm wondering if anybody ran into this problem? It works perfectly on Windows 2012 - based images
Apparently, the SSM Agent starts after the EC2 Launch executes UserDatascript. I can see it from the SSM Agent's log file modification timestamps.
Therefore, there's no log forwarding happening during the EC2 Launch.
When the SSM Agent starts and loads the cloudwatch plugin, the log files are already filled with entries and never change (wallpaper log is the only exception) So they never end up in cloudwatch console.
There's been a lot of changes implemented on AWS side: they switch to .Net core, removed EC2 config service and moved the log forwarding logic to SSM Agent (cloudwatch Plugin) for Windows 2016-based AMIs
It looks like the behavior has changed quite significantly too so there's no way to get the EC2 launch logs in cloudwatch (when using AWS toolset-only)
Basically we have to stick to our Application logs only which is very unfortunate. We rely on EC2 launch logs to see if the instance started & successfully executed user data.