I have deployed TeamCity server and Agent to AWS using JetBrains Stack Template (https://www.jetbrains.com/help/teamcity/running-teamcity-stack-in-aws.html)
All seems to be good, my server starts, agent is functional, I have created several builds, etc.
I came to a point, where I want to deploy my application to AWS environment using aws-cli commands.
I am struggling to enable/install aws-cli into agent. My build steps are erroring out with aws: command not found
Does anyone have any ideas?
My progress so far: I have connected to agent EC2 machine via ssh bastion ec2, and I am able to invoke aws --version as ec2-user, but the build agent cannot see aws.
Turns out, my TeamCity agent runs in AWS ECS via docker image https://hub.docker.com/r/jetbrains/teamcity-agent
What I ended up doing is creating my own docker image by using jetbrains one as a base.
I uploaded my docker image to AWS ECS Repository. Afterwards I created a new revision for original task definition. This new revision uses my image instead of original one, therefore I have aws-cli there.
I then added my AWS profile to EC2 host machine and added volume to docker container (via task definition) so that container would be able to access .aws/credentials file.
Dockerfile looks like this:
FROM jetbrains/teamcity-agent
RUN apt-get update && apt-get install -y python-pip
RUN pip install awscli --upgrade --user
ENV PATH="~/.local/bin:${PATH}"
I added the aws-cli in team city agent using remote desktop connection as I used window agent of team city. In the build steps I used Runner Type as command line and executed the aws commands.
for more information you can refer below link where I answered the question:
How to deploy to AWS Elastic Beanstalk on successful Teamcity build
Related
Hi I've been trying to get autodeployment working on AWS. Whenever there's a merge or commit to repo, CodePipeline will detect it and have CodeDeploy update the tagged EC2 instance with the new changes. The app is a simple node.js app which I want to start with docker-compose. I have installed docker-compose + docker on the EC2 instance already and enabled CodeDeploy Agent.
I tested the whole process and it is mostly working except for the part where CodeDeploy fails the deployment because it is unable to run the command docker-compose up -d in my ApplicationStart portion of the appspec.yml. I get the error docker compose cannot be found which is kind of weird because in the BeforeInstall script I download and install docker + docker-compose and set all the permissions. Is there something I'm missing or it is just not meant to happen with CodeDeploy and EC2?
I can confirm when I SSH into the EC2 instance and use the command docker-compose up -d in project root directory it works, but as soon as I try to run the docker-compose command in the script portion of the appspec.yml it fails.
The project repo is here, just in case there's anything I missed: https://github.com/c3ho/simple_crud
I've been using the AWS console to upload a WAR file for deployment. Now I want to do it from the command line. I've been following this guide and see eb init and read the help with eb init --help and eb --help, but the only option is to create a new application.
usage: eb init <application_name> [options ...]
Initializes your directory with the EB CLI. Creates the application.
positional arguments:
application_name application name
How do I link my local source project directory to an existing application in AWS console?
I would expect a command like eb link or something, like how you can just add a Git remote with Heroku and automatically link an existing project to an existing app.
When you perform eb init in the directory containing your source code, eb will prompt you for an application name and an environment name. This way you can link your source code to what ever application/environment is deployed on Beanstalk.
It worked after I got the AWS CLI keys for the project and ran aws configure. I had old keys in ~/.aws/ from a different project from perhaps a decade ago that used a different format. Once I got new keys, that were given permission for these particular apps, and ran aws configure and set the region, then eb init would present a menu of applications to choose from. The command aws elasticbeanstalk describe-applications has to work first before eb can work. I was expecting it would ask for a username and password, like Heroku does.
Install aws and eb command line tools:
Install awscli
Get keys from AWS admin devops.
aws configure (Example Region: 'us-east-1')
aws elasticbeanstalk describe-applications
Install Python
pip install awsebcli --upgrade --user
Add eb to your PATH, probably %USERPROFILE%\AppData\Roaming\Python\Python37\Scripts
eb init
eb list / eb logs / eb ssh / eb status / eb config / eb help
Beanstalk differs from Heroku in this workflow, unless you are using CodeCommit. I am assuming you are just using S3 to store your application versions.
The EBCLI command to do this is:
eb create-application-version
You can specify an application, a version label, as well as either a CodeCommit repository, a codeBuild build, or a source bundle in S3. API docs
You will need to run a separate command before create-application-version to upload to your S3 bucket.
Using the CLI:
aws s3 cp <filename> <s3bucket>
API docs
You can also use the console.
It seems like that guide skips initializing your local git repository. For linking your local source project to beanstalk, make sure you have initialized a local git repository. Then you can link your workspace and application using eb init. more about EB CLI and Git
Based on my understanding, your question is that you had a project directory on your PC and run your app at the localhost, now you want to run it in the AWS Elastic Beanstalk to make it public.
If you have created an EB application in the EB management console and uploaded your bundled source code, the source code becomes an application version, you need to deploy it into one of your environment using the EB management console, like this:
Figure of the management console.
Then the EB platform(container) will take care of that and run your server automatically as long as you set up the command which your app uses to run the server, the proxy, and other configurations either through the EB management console -> [Your environment] -> configuration or using the .ebextensions file.
If everything is well, you can visit your app's home page through the environment URL at that time.
I'm attempting to deploy an app from one ElasticBeanstalk instance to another. Running pip install awsebcli --upgrade --user doesnt install the eb cli tool for some odd reason on the EC2 machine.
Does anyone know the equivalent of eb deploy using only the aws cli options?
This question is a bit confusing. Are you attempting to move code between EC2 instances in your Beanstalk environment?
If I'm assuming correctly, you've pulled/changed your code on one Beanstalk host. And now you're trying to propagate that change to the other instances using the EB CLI. That's not a best practice. Beanstalk has a mechanism to deploy your code to all instances.
The EB CLI is meant to be run from your workstation to push code from your IDE/editor to the Beanstalk hosts in AWS.
Beanstalk keeps a copy of that code revision in S3. And if the Beanstalk environment is load balanced then all instances will be running the same application version when scaling events or deployments occur because it will pull your code from a common source.
But to answer your question:
Does anyone know the equivalent of eb deploy using only the aws cli options?
You're gonna wanna ZIP and upload your code to S3 and note the S3 key and bucket values of where it's located.
Then create a new application version.
% aws elasticbeanstalk create-application-version --application-name="<APPLICATION_NAME>" --version-label="<NEW_VERSION_LABEL>" --source-bundle="{\"S3Bucket\": \"<S3_BUCKET_NAME>\",\"S3Key\": \"<S3_KEY>\"}"
Then deploy your new application version to the running environment.
% aws elasticbeanstalk update-environment --environment-id="<ENVIRONMENT_ID>" --version-label="<NEW_VERSION_LABEL>"
Reading is hard...
Linux requires you to "[a]dd the path to the executable file to your PATH variable"
export PATH=~/.local/bin:$PATH
eb --version now works
I am new to github and AWS. I want to deploy my code directly from my github repository (a simple 'hello world' html page), and onto my EC2 instance. I was following this tutorial http://docs.aws.amazon.com/codedeploy/latest/userguide/github-integ-tutorial.html However on step 4 I am struggling.
It says after 'launched the instance and verified the AWS CodeDeploy agent is running, go to the next step'.
But, how do I verify AWS CodeDeploy Agent is running? It says to follow this link, however i am lost with it http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-run-agent.html#how-to-run-agent-install-windows (windows server)
Where do i put these commands in and where? And do I need the AWS SDK first?
Thanks
You can check if the code deploy agent is running from the command
sudo service codedeploy-agent status
If the command returns an error, the AWS CodeDeploy agent is not installed. Install it as described in To install, uninstall, or reinstall the AWS CodeDeploy agent for Amazon Linux or RHEL
If the AWS CodeDeploy agent is installed and running, you should see a message like The AWS CodeDeploy agent is running.
If you see a message like error: No AWS CodeDeploy agent running, start the service and run the following two commands, one at a time:
sudo service codedeploy-agent start
sudo service codedeploy-agent status
see http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-run-agent.html if you want info for another os type
I want to integrate Atlassian Bamboo with AWS Elastic Beanstalk. Is there anyway to do this?
It depends a bit on your Bamboo and beanstalk config as well as the type of application you are planning to deploy on AWS Beanstalk.
We did some things for Java Web Apps:
Since Bamboo understands maven, you can have a look at the following maven plugin:
http://beanstalker.ingenieux.com.br/beanstalk-maven-plugin/configurations-and-templates.html
We are using it for some environments to create wars and upload them to elastic beanstalk. You can then create a maven task in bamboo to call the plugin.
If you downloaded and installed Bamboo on a machine you own yourself you could use the Elastic Beanstalk command line interface (CLI).
This is probably the most powerful approach, but you need to install the CLI on the bamboo instance. Then you can do almost anything. This approach should also work for other environments besides Java/Tomcat.
Another idea:
If you use Beanstalk using git (i.e. you deploy by making a code change and pushing to Beanstalk), then you can also use the new "Deployment Project" Feature in Bamboo to push the code once it passes all tests.
David's answer provides good options for cross product usage of AWS Elastic Beanstalk (+1). Nowadays I'd recommend the excellent unified AWS Command Line Interface over the now legacy AWS Elastic Beanstalk API Command Line Interface, see the resp. AWS CLI commands for elasticbeanstalk.
If you are looking for a Bamboo specific solution, you might be interested in Utoolity's Tasks for AWS (Bamboo) add-on (commercial, see disclaimer), which provides three dedicated tasks, specifically:
AWS Elastic Beanstalk Application - create, update or delete AWS Elastic Beanstalk applications.
AWS Elastic Beanstalk Application Version - create, update or delete AWS Elastic Beanstalk application versions.
AWS Elastic Beanstalk Environment - create, update, rebuild, restart, swap or terminate AWS Elastic Beanstalk environments and specify configuration settings and advanced options.
Disclaimer: I'm the co-founder of this add-on's vendor, Utoolity.
In case you're interested in C# deployments:
What we do is to simply start the awsdeploy tool (should already be installed on the build server) with a link to the configuration script. I create the environment simply in Visual Studio and when I redeploy the application once, I save the script. Once the script is on the build server, I reference it in the deployment configuration with awsdeploy /r c:\location\of\myscript.txt.
The package itself the is referenced in the AWS deployment configuration script is created at build time with the MSbuild /target:package command and defined as an artifact (default location of the ZIP package is c:\build-dir\...\project\obj\debug\package, but can be overwritten.
Everything works pretty well so far, although I am having problem to start an elastic instance when none is available (e.g. nightly builds).
Take a look at our repo: https://github.com/matzegebbe/docker-aws-login
With that snippet you are able to login with the aws an push images
simple bamboo task script (of course you need docker installed on the agents):
#!/bin/bash
docker images hellmann/awscli | grep -q awscli
[ "$?" -eq "0" ] && exit 0
cat <<'EOF' >> Dockerfile
FROM python
MAINTAINER Mathias Gebbe <mathias.gebbe#hellmann.net>
RUN pip install awscli --ignore-installed six
ENV aws_access_key_id AWS_ACCESS_KEY
ENV aws_secret_access_key AWS_SECRET_ACCESS_KEY
RUN mkdir /root/.aws/
RUN printf "[default]\nregion = eu-west-1\n" > /root/.aws/config
RUN printf "[default]\naws_access_key_id = ${aws_access_key_id}\naws_secret_access_key = ${aws_secret_access_key}\n" > /root/.aws/credentials
ENTRYPOINT ["/bin/bash","-c"]
CMD ["aws ecr get-login"]
EOF
docker build -t hellmann/awscli .
$(docker run --rm hellmann/awscli)