I'm trying to get all our instances (all Windows based) upgraded to IMDSv2 and have been following the advice found here https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html#instance-metadata-transition-to-version-2 and using CloudWatch to find instances making MetadataNoToken calls (i.e. using IMDSv1).
I've found several instances using IMDSv1 this way, but I can't work out how to find out what is making the calls from with the OS.
According to CloudWatch each server is making one call per minute to the IMDSv1 service.
The support article mentions upgrading any AWS SDKs or CLI tools, but the servers in question don't have seem to have any SDKs or CLI tools installed.
Each instance has the following AWS published tools installed on them:
Amazon SSM Agent
Amazon CloudWatch Agent
AWS Tools for Windows
EC2ConfigService
AWS PV Drivers
aws-cfn-bootstrap
I've updated the Amazon SSM Agent and the Amazon CloudWatch Agent to the latest versions. But I can't find any information about how to update the AWS Tools for Windows package.
I've also run TCPView from Sysinternals on the servers and tried to find what process is making calls to the 169.254.169.254 endpoint, but it doesn't seem to pick up any traffic to that address.
I'm reluctant to just disable IMDSv1 and do a scream test as they are production servers.
If anyone has any advice or guidance on how to find what is making the IMDSv1 calls it would be appreciated.
I figured it out in the end, using the £Windows Resource Monitor Network monitor" tool, I found the exectucable that was making the calls.
I've written up the proceess in this blog post:
https://www.greystone.co.uk/2022/03/24/how-greystone-upgraded-its-aws-ec2-instances-to-use-instance-meta-data-service-version-2-imdsv2/
Related
I was going through the AWS cloudwatch documentation for Cloudwatch agents.
So I came across the documentation which says:
Unified Cloud watch agent is recommended and it has support across all operating systems.
But for old Cloud watch agent(on the path to deprecation) documentation mentions that "it supports the collection of logs from only servers running Linux".
But when I went ahead in documentation in next section (for Old cloudwatch agent):
Documentation mentions that "Using the CloudWatch Logs agent, you can publish log data from Amazon EC2 instances running Linux or Windows Server, and logged events from AWS CloudTrail. We recommend instead using the CloudWatch unified agent to publish your log data."
So I am confused regarding whether the old cloudwatch agent supports EC2 instances running Windows server or not?
Is there anything I am overlooking?
Or is it just a documentation error from AWS side?
Edit:
Getting Started with CloudWatch Logs- This link mentions: Old cloudwatch agent supports the collection of logs from only servers running Linux.
Using old cloudwatch agent - This link mentions: Using the CloudWatch Logs agent, you can publish log data from Amazon EC2 instances running Linux or Windows Server, and logged events from AWS CloudTrail
If you check the next section of "Using old cloudwatch agent", it listed the CloudWatch Logs agent prerequisites, it doesn't include windows server.
Amazon Linux version 2014.03.02 or later. Amazon Linux 2 is not supported
Ubuntu Server version 12.04, 14.04, or 16.04
CentOS version 6, 6.3, 6.4, 6.5, or 7.0
Red Hat Enterprise Linux (RHEL) version 6.5 or 7.0
Debian 8.0
However, from the following URL it is clear that you can use cloudwatch logs (old cloudwatch agent) for windows server.
So, I believe cloudwatch logs (old cloudwatch agent) supports windows server.
https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/QuickStartWindows2016.html
Just was having some similar doubt.
The old cloudwatch logs is going to be deprecated
[...] Recommended – The unified CloudWatch agent. It enables you to collect both logs and advanced metrics with one agent. It offers support across operating systems, including servers running Windows Server. [...]
And the old one as you say, it does not support collecting metrics from windows servers:
[...] The older CloudWatch Logs agent, which supports the collection of logs from only servers running Linux. [...]
Source: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_GettingStarted.html
Recent Tenable scan highlighted an issue with certain versions of datadog versions. This is also brought to attention in Datadog monitor.
Critical bug in Windows Agent versions 6.14.0 and 6.14.1. See --> http://dtdg.co/win-614-fix <-- for steps to fix the issue.
As the bulk of our servers are hosted on AWS - just wondered if I could query this through AWS CLI to list which servers were using the affected versions.
On the bottom of the infrastructure list, you should see a link called "JSON API permalink". If you query it, this should give you a JSON of all your hosts with their agent version. You can then query it with a quick Python script.
We have multiple amazon ec2 instances behind a load balancer. Our build script is written in phing and is integrated with git.
We are looking for a tool (like Jenkins or Amazon code deploy) which could display all the active instances currently behind load balancer and then allow us to select some of them (or select a group defined previously) and then trigger either of the following (whichever is better) -
a build script hosted on the same dedicated server where the tool is hosted.
or the respective build scripts hosted on the selected ec2 instances.
We should be able to do the following -
specify a git branch name, optionally, when we trigger the build script for any group of instances.
be able to roll out in batches of boxes, so as to get some time to monitor load, and then move to next batch if all is good. Best way, I guess, would be to specify a size of the batch (e.g. 10), so that the process waits for a user prompt after rollout on every batch completes.
So, if we have to rollout two different git branches to two groups of instances, we should be able to run them in two steps (if we do not specify batch size).
Would like to know about experiences of people who dealt with something similar.
For CodeDeploy, it supports Git (more precisely, GitHub). It also allows you to deploy only to tagged EC2 instances. If combined with custom DeploymentConfig (http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-create-deployment-configuration.html), you can also control how fast (the size of the batch) to deploy.
I would re-structure the question:
The choices you have for application deployment
and whether the tool has option to perform rolling deployments.
Jenkins is software for CI/CD, which will have to use plugins,custom scripting or leverage an existing orchestration software setup for doing the deployments.
For software orchestration, you have many choices, some of the more famous tools are Chef, puppet, ansible etc.. All of these would need you to manage some kind of centralized setup. All such software support application deployment.
You need to make a decision on whether you would want to invest in maintaining such a setup.
If you decide against such a setup, you have the option of using managed services such as AWS OpsWorks, AWS CodeDeploy, hosted chef etc.
In choosing any of these services, you delegate the management of orchestration software to a vendor, which will ensure the service is up all the time.
AWS code deploy and AWS OpsWorks are managed services on aws and work pretty well on AWS setups.
AWS OpsWorks uses chef under the hood.
AWS CodeDeploy only provides a subset of what OpsWorks provides and is responsible only for deployments. With AWS code deploy you get convenient visualization of your software deployments through AWS console.
With AWS code deploy, you can achieve the goal of partial roll out to ec2 instances.
You can do the same with other tools as well but CodeDeploy on AWS environment will take least amount of work.
CodeDeploy also allows you to deploy from GIT. Please refer to the following aws documentation
http://docs.aws.amazon.com/codedeploy/latest/userguide/github-integ-tutorial.html
The pitfall with code deploy is the fact that the agent that will run on instances has been tested for and is supported for only a limited number of OS combinations.(http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-run-agent.html#how-to-run-agent-supported-oses)
Also in future if you decide to move away from AWS, you will have to redo the deployment related work.
CodeDeploy service only charges you for the underneath AWS resources.
Please find the link to pricing documentation below:
https://aws.amazon.com/codedeploy/pricing/
Can IBM Integration Bus((and /or Websphere message Broker) be implemeted on AWS ? Can my on-premise ESB be migrated to AWS Cloud ?
Thanks in Advance
AWS EC2 allows importing VMs into an AMI then you can start an EC2 instance using that image. If you are new to AWS you can check the link below
https://aws.amazon.com/ec2/vm-import/
However, you should be careful about IIB license and how many machines you can install it on before regesting the AMI in a launch configuration and create an autoscaling group and set a scaling policy that can start instances more that what you purchased.
That's very much possible. There are several possible approaches.
1. IIB on EC2
Installing and configuring IIB on an EC2 instance is very much similar to doing the same in on-premise servers. Only difference is that the physical server is in AWS Cloud. While this approach gives you maximum flexibility to design your architecture any way, it does not take advantage of the basic features of the cloud.
2. Quick Start
IIB is available for deployment under AWS Quick Start. You can read more about this here. This helps you get started quickly by setting up the entire environment in a few clicks. But, if you're planning to migrate your existing architecture to AWS, this may not suit you as the architecture is pre-defined with limited options for customization.
3. IIB on Containers
ACE 11 provides better support for containerization. You can read more about running IIB 10 on containers here and ACE 11 on containers here. After this, the containers can be deployed into fully managed containers such as AWS Elastic Container Service or your own container configuration such as Docker on EC2.
Yes of course, AWS provides the IAAS and you just install whatever you want inside. Make sure you open ports, use specific credentials for the instalation (dont use admin) and everything should work.
IBM also provides docker images of integration bus v10 and APP Connect Enterprise v11. This is true for all their integration tools, MQ, API Management and more.
Not restricted to AWS.
I am writing an application which would enable my company's future clients to be able to launch Amazon aws instances with Windows OS. Hence, I would want to create windows AMIs first. The article below gives a step by step approach for creating EBS-backed windows AMIs using Amazon AWS console.
http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/Creating_EBSbacked_WinAMI.html
But when I tried looking up the methods provided by CreateImageRequest API provided by Amazon SDK, I couldn't quite figure out how to replicate the AWS management console process for creating a windows image in Java. I exhausted my google searches and I came up empty. Has anyone tried this? Is this even possible at all to create Windows images through Amazon SDK?
I found that CreateInstanceRequest API has an option to specify the OS types but the options are limited to Amazon Linux or Ubuntu 12.04 LTS. The documentation for CreateInstanceRequest is below:
http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/opsworks/model/CreateInstanceRequest.html
I would appreciate any useful link or code snippet. Thanks.
I don't completely understand what you are trying to achieve.
Do you want to start an instance in EC2 or with OpsWorks because your second link points to the OpsWorks documentation.
OpsWorks dosen't support Windows right now.