I was able to successfully setup and run tests on ec2 instance by setting up JMeter, Grafana (UI to display results) and a database called influxDB. The only issue is that a user has to logon to the instance to run the test as the test plans need to be uploaded on the instance for the same.
I was hoping if I could leverage SSM (aws run command) by which I can store the test plan in an S3 bucket and then use SSM to take this test plan and run the test from AWS SSM directly instead of logging onto the ec2 instance?
Please note, I would like to still run the test on the ec2 instance but as a user I don't want to login to the instance directly but rather have aws ssm take care of this.
Any insight on the same will be helpful. Thanks!!
I was able to mount an S3 bucket and fix the problem.
The metadata URL from Amazon gives a lot of data but lags tag information. I tried to combine a bunch of different commands and eventually got to the describe-tags CLI command. The problem is that while I can get the Instance ID and the Region easily enough, I cannot get values for tags without dropping credentials onto the box.
I get the following error:
Unable to locate credentials. You can configure credentials by running "aws configure".
The basic command I wind up executing is:
aws ec2 describe-tags --region us-east-1 --filters "Name=resource-id,Values=SOME_ID"
The process I follow is this:
Create an instance with a predefined application on it
Image the instance
Spin up various instances using the image via the Amazon AWS API programmatically
Tag the instances that get spun up with pieces of critical data
Attempt to read the tags from the application
Any way to get around the credentials issue? I figure that the local machine would have access to its own tag metadata without signing in but that doesn't appear to be the case.
If there's no way to get around it, are there any suggestions to pass in the data to the VM without sitting around and waiting for it to start up?
I really don't want to write a process that sits around waiting for the EC2 to finish spinning up, SSH in and then pass in the critical data myself. The data changes on-the-fly and can change between instances that I fire up in order to handle various events.
I would create your EC2 instances with IAM roles for EC2. You don't need to do anything fancy and the credentials are then available on the box. It's easy to restrict the role down to do only what you need.
I might be naive but looking for a good solution to automatically spin up an AWS server with an API.
The use case is to create AWS EC2 instances on a click and maintain the deployments. Ansible is a probable candidate but looking for the core solution of spinning up a new EC2 machine.
Appreciate your help.
Rather than directly calling an API (eg from Java, .Net, Python, etc), you can also use the AWS Command-Line Interface (CLI).
The command you want is run-instances, which will launch a new Amazon EC2 instance.
See: AWS CLI documentation for run-instances
I've just started with AWS and I have some questions.
First, I followed the official documentation on how to launch an instance using AWS SDK for JAVA like this:
AmazonEC2 Client = new AmazonEC2Client(awsCreds);
CreateSecurityGroupRequest csgr = new CreateSecurityGroupRequest();
csgr.withGroupName("Azzouz_group").withDescription("My security group");
IpPermission ipPermission = new IpPermission();
ipPermission.withIpRanges("0.0.0.0/0").withIpProtocol("tcp");
AuthorizeSecurityGroupIngressRequest authorizeSecurityGroupIngressRequest = new AuthorizeSecurityGroupIngressRequest();
authorizeSecurityGroupIngressRequest.withGroupName("Azzouz_group").withIpPermissions(ipPermission);
RunInstancesRequest runInstancesRequest = new RunInstancesRequest();
runInstancesRequest.withImageId("ami-4b814f22")
.withInstanceType("m1.small")
.withMinCount(1)
.withMaxCount(1)
.withKeyName("azzouz_key")
.withSecurityGroups("Azzouz_group");
RunInstancesResult runInstancesResult = Client.runInstances(runInstancesRequest);
RunInstancesResult runInstancesResult = Client.runInstances(runInstancesRequest);
String instanceId = runInstancesResult.getReservation().getInstances().get(0).getInstanceId();
I didn't use the CreateKeyPairRequest part because I want to upload my public key to amazon so when I try to ssh into into I don't have to add -i path/to/key.pem and I have only to mention the key name in my java code ("azzouz_key") , in the next lines, $USER contains azzouz_key:
keypair=$USER # just a name
publickeyfile=$HOME/.ssh/id_rsa.pub
regions=$(aws ec2 describe-regions \
--output text \
--query 'Regions[*].RegionName')
for region in $regions; do
echo $region
aws ec2 import-key-pair \
--region "$region" \
--key-name "$keypair" \
--public-key-material "file://$publickeyfile"
done
what I want now is connect to the instance and automate some stuff. So I 'm heading to make a call to a shell from inside the java code, the script gets an instance id as a parameter, then gets the ip adress ( using aws ec2 describe-instances ), ssh into it and do some stuff.
I wanted to authorize ssh connection to the instance from any ip just as a start(0.0.0.0/0) and I'm not sure if this is what I'm supposed to do.
So, my question is: Is this the best approach?! Should I just use the aws cli to create and manage the instance?! Does just mentioning just the key pair name fits with the mechanism of uploading the public ssh key to amazon?!
Please, I'm just starting, I'm an intern and I dont yet have an access to an amazon account so I can test my work. I'm just working all of this in my mind. THANK YOU VERY MUCH!
my advice is to setup an account on AWS and start using the AWS free tier options.
All in all, it is there and it is for free (just pay attention on what you launch or use in the service).
Apart of that, your question about how to authorize connections over SSH from everywhere, this is done over security groups (http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html)
About what is the best option, this really depends on you.
If you need to launch 2 instances in your life on AWS, then the console is good enough. If you want to orchestrate your hybrid setup, then probably your way is the best.
CLI is an excellent solution for daily operations too.
In simple words, there is not best way or a good or bad approach. It all depends on your needs.
I hope this helps somehow.
Automation is a huge topics.If you want to extend AWS automation using script, Before touching the API/SDK, first, you must design your own AWS resources tags naming.
Tags naming is an implicit ways to reference to AWS resource without explicitly specify the resource-id(e.g. VPC id, EC2-id ,interface-id,etc). In addition, for resource such as EC2 that doesn't allow immediate use of tag during creation, you need to study usage of "client-token".
AWS CLI allow you to do lots of automation, however, to manipulate response result, you need shell script skill to manipulate them. I suggest you pick the AWS SDK language that you are familiar with.
Cloud configuration management tools(there is limited support from tools like Ansible, saltstack,puppet) can be the next step, if you plan to extend the whole source deployment, server configuration.
You may want to consider starting off with Infrastructure as Code. Cloud Formation with Code Pipeline will ensure automated and consistent environment launches and makes you highly valuable in the marketplace.
Both can be launched and managed via the awscli. As your capabilities and the complexity of your IaC increase it may be worth looking into Terraform due to the modularity available compared to CloudFormation.
I have a running AWS instance. I want to set user-data for this instance.
Question is how do I do it either using AWS console or using AWS CLI tools.
You have to stop your instance to change the user-data. The AWS EC2 User Guide has instructions on how to do it: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Stop_Start.html#Using_ChangingAttributesWhileInstanceStopped