Recent Tenable scan highlighted an issue with certain versions of datadog versions. This is also brought to attention in Datadog monitor.
Critical bug in Windows Agent versions 6.14.0 and 6.14.1. See --> http://dtdg.co/win-614-fix <-- for steps to fix the issue.
As the bulk of our servers are hosted on AWS - just wondered if I could query this through AWS CLI to list which servers were using the affected versions.
On the bottom of the infrastructure list, you should see a link called "JSON API permalink". If you query it, this should give you a JSON of all your hosts with their agent version. You can then query it with a quick Python script.
Related
In our company we have hundreds of instances, some Linux and some Windows. I need to check what instances need to have the latest PowerShell installed, therefore I need a list of PowerShell versions for all the instances.
In order to get the information, I was thinking of somehow using the fleet manager (from system manager) to gather this information about all the instances, but not sure how to do it.
Would appreciate any advice.
For anyone looking for it: run command with systems manager is the answer. Use it with the RunPowerShellScript document, have it ask each instance for their $psversiontable.psversion, and dump it into s3 for easier querying
We have an Elastic Beanstalk application and we configure it to automatically publish rotated logs to S3.
In an older deployment of this application, the published logs included the full path, with slashes replaced with underscores, like "_var_log_nginx_rotated_access.log1640678462.gz".
In a new deployment the published logs instead look like "access.log1644962462.gz", without the complete path.
The older deployment ran on the (now deprecated) platform "Tomcat 8.5 with Java 8 running on 64bit Amazon Linux", and the newer deployment runs on the platform "Tomcat 8.5 with Corretto 11 running on 64bit Amazon Linux 2".
Is there any configuration which will allow the published S3 logs to include the full path?
I have one alternative suggestion, you can use post-processing in a Lambda to copy them to any bucket structure you want.
I referred the below link for moving specific logs to specific bucket :
..
https://medium.com/#ravindueranga/aws-elastic-beanstalk-rotated-logs-into-a-separate-s3-bucket-using-aws-lambda-cd3868447b04
..
I hope the information is useful to you as well.
I'm trying to get all our instances (all Windows based) upgraded to IMDSv2 and have been following the advice found here https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html#instance-metadata-transition-to-version-2 and using CloudWatch to find instances making MetadataNoToken calls (i.e. using IMDSv1).
I've found several instances using IMDSv1 this way, but I can't work out how to find out what is making the calls from with the OS.
According to CloudWatch each server is making one call per minute to the IMDSv1 service.
The support article mentions upgrading any AWS SDKs or CLI tools, but the servers in question don't have seem to have any SDKs or CLI tools installed.
Each instance has the following AWS published tools installed on them:
Amazon SSM Agent
Amazon CloudWatch Agent
AWS Tools for Windows
EC2ConfigService
AWS PV Drivers
aws-cfn-bootstrap
I've updated the Amazon SSM Agent and the Amazon CloudWatch Agent to the latest versions. But I can't find any information about how to update the AWS Tools for Windows package.
I've also run TCPView from Sysinternals on the servers and tried to find what process is making calls to the 169.254.169.254 endpoint, but it doesn't seem to pick up any traffic to that address.
I'm reluctant to just disable IMDSv1 and do a scream test as they are production servers.
If anyone has any advice or guidance on how to find what is making the IMDSv1 calls it would be appreciated.
I figured it out in the end, using the £Windows Resource Monitor Network monitor" tool, I found the exectucable that was making the calls.
I've written up the proceess in this blog post:
https://www.greystone.co.uk/2022/03/24/how-greystone-upgraded-its-aws-ec2-instances-to-use-instance-meta-data-service-version-2-imdsv2/
I'm in the process of exploring AWS SSM to apply Patches on multiple AWS accounts. I was able to implement this successfully using respective AWS documents. During my implementation process, I used AWS-RunPatchBaseline document to update my linux instances.
This completely works fine. Post patch installation the instance reboots. The use case which i'm trying to achieve is:
Instance should not reboot after installing patches.
I tried to alter the document to disable reboot process, which did not help me. Also, checked with Amazon Support, they still in the process getting a fix for my request.
Does one have answer for this, either through using a custom document or by modifying the existing document?
Thanks,
Vick
As per amazon we cannot stop or suspend the reboot post patching which is not advisable. However, Amazon is working on a feature request to suspend instance reboot. Wish it happens soon.
I see that Impala 2.3 is only supported on Cloudera CDH 5.5 & above. Impala 2.2 can be installed on Amazon EMR as there is Bootstrap script available on GitHub & you don't require Cloudera installation.
However, I don't see any way to install Cloudera CDH 5.5 or 5.6 on Amazon EMR. I want to install Impala 2.3 so is there any way through which Impala 2.3 can be installed on Amazon EMR?
Well, my previous answer has been deleted as long as "does not provide an answer to the question". I'm not going to argue if it's better to have a partially incorrect answer to this question or if making categorical claims without foundation is a good answer :/.
In any case, I'm not giving up :)
Yes, it's possible to install "anything" on the paper.
Once you launch the EMR cluster, all instances will appear on your EC2 console. The only thing is that you have to be careful assigning the right permissions to access thru SSH to your instances. My suggestion is to create a specific security group with the access and assign this extra security group to the instances using the Advanced configuration of the cluster.
By having the proper configuration, you could ssh into any instance and install anything (you should be able to scp any file or download from internet if you have the proper configuration of your VPC). Note that the user will be "hadoop" instead "ec2-root" but this is documented on the EMR user guide.
Keep in mind that the cluster is "Terminated" so, the EMR instances are volatile and the installation is not going to survive the cluster termination.
On the other hand, using the latest versions of EMR AMIs and the latest capabilities of AWS (I think that it was all the time the case, but, it doesn't matter now) you should be able to create some actions on the bootstrap and install anything you want.
Using the "Advanced configuration" of your cluster, you can access to the "Bootstrap" actions to be executed on your cluster. You could even have different actions depending on the node type (master, core, tasks). You should store your scripts (and/or jar files) on an S3 bucket and made this bucket available to your cluster. On the paper, you could install Impala on these EC2 instances comprising the EMR cluster but I'm not sure if this will work.
For more information, you can read http://docs.aws.amazon.com//emr/latest/ManagementGuide/emr-plan-bootstrap.html
And for a previous version of EMR AMI and not so recent version of Impala you can read https://github.com/awslabs/emr-bootstrap-actions/tree/master/impala
Thanks Mark, you forced me to elaborate better my comment.
No, it is not possible to "install" anything on EMR because it's a PaaS provided by AWS. But if your goal is to run a newer version of Impala on AWS, there is an AWS Quick Start path for installing CDH 5.x (including Impala) that makes the process relatively easy.
http://aws.amazon.com/quickstart/