I want to integrate Atlassian Bamboo with AWS Elastic Beanstalk. Is there anyway to do this?
It depends a bit on your Bamboo and beanstalk config as well as the type of application you are planning to deploy on AWS Beanstalk.
We did some things for Java Web Apps:
Since Bamboo understands maven, you can have a look at the following maven plugin:
http://beanstalker.ingenieux.com.br/beanstalk-maven-plugin/configurations-and-templates.html
We are using it for some environments to create wars and upload them to elastic beanstalk. You can then create a maven task in bamboo to call the plugin.
If you downloaded and installed Bamboo on a machine you own yourself you could use the Elastic Beanstalk command line interface (CLI).
This is probably the most powerful approach, but you need to install the CLI on the bamboo instance. Then you can do almost anything. This approach should also work for other environments besides Java/Tomcat.
Another idea:
If you use Beanstalk using git (i.e. you deploy by making a code change and pushing to Beanstalk), then you can also use the new "Deployment Project" Feature in Bamboo to push the code once it passes all tests.
David's answer provides good options for cross product usage of AWS Elastic Beanstalk (+1). Nowadays I'd recommend the excellent unified AWS Command Line Interface over the now legacy AWS Elastic Beanstalk API Command Line Interface, see the resp. AWS CLI commands for elasticbeanstalk.
If you are looking for a Bamboo specific solution, you might be interested in Utoolity's Tasks for AWS (Bamboo) add-on (commercial, see disclaimer), which provides three dedicated tasks, specifically:
AWS Elastic Beanstalk Application - create, update or delete AWS Elastic Beanstalk applications.
AWS Elastic Beanstalk Application Version - create, update or delete AWS Elastic Beanstalk application versions.
AWS Elastic Beanstalk Environment - create, update, rebuild, restart, swap or terminate AWS Elastic Beanstalk environments and specify configuration settings and advanced options.
Disclaimer: I'm the co-founder of this add-on's vendor, Utoolity.
In case you're interested in C# deployments:
What we do is to simply start the awsdeploy tool (should already be installed on the build server) with a link to the configuration script. I create the environment simply in Visual Studio and when I redeploy the application once, I save the script. Once the script is on the build server, I reference it in the deployment configuration with awsdeploy /r c:\location\of\myscript.txt.
The package itself the is referenced in the AWS deployment configuration script is created at build time with the MSbuild /target:package command and defined as an artifact (default location of the ZIP package is c:\build-dir\...\project\obj\debug\package, but can be overwritten.
Everything works pretty well so far, although I am having problem to start an elastic instance when none is available (e.g. nightly builds).
Take a look at our repo: https://github.com/matzegebbe/docker-aws-login
With that snippet you are able to login with the aws an push images
simple bamboo task script (of course you need docker installed on the agents):
#!/bin/bash
docker images hellmann/awscli | grep -q awscli
[ "$?" -eq "0" ] && exit 0
cat <<'EOF' >> Dockerfile
FROM python
MAINTAINER Mathias Gebbe <mathias.gebbe#hellmann.net>
RUN pip install awscli --ignore-installed six
ENV aws_access_key_id AWS_ACCESS_KEY
ENV aws_secret_access_key AWS_SECRET_ACCESS_KEY
RUN mkdir /root/.aws/
RUN printf "[default]\nregion = eu-west-1\n" > /root/.aws/config
RUN printf "[default]\naws_access_key_id = ${aws_access_key_id}\naws_secret_access_key = ${aws_secret_access_key}\n" > /root/.aws/credentials
ENTRYPOINT ["/bin/bash","-c"]
CMD ["aws ecr get-login"]
EOF
docker build -t hellmann/awscli .
$(docker run --rm hellmann/awscli)
Related
Currently, Amazon deprecated Multi-container Docker running on 64bit Amazon Linux.Need migrate to Docker running on 64bit Amazon Linux 2. In 1st version , we used Dockerrun.aws.json v2 to manage multi container docker. In latest version (Docker running on 64bit Amazon Linux 2), we need to use Dockerrun.aws.json v3 or docker-compose. But there is no working example or blogs are available. Can i get working samples ?.
In regards to Elastic Beanstalk and the Docker running on 64bit Amazon Linux 2 platform.
I was struggling too and finally got to the bottom of it. What confused me is that the documentation makes it seem like you can choose to use either, the Dockerrun.aws.json (v3) or a docker-compose.yml in your EB application package.
Then you go looking for the documentation on Dockerrun.aws.json (v3), and you won't find it anywhere.
The reason for this is that, you don't get a choice. If you want to run multiple containers you must include a docker-compose.yml in your application package. The only thing the Dockerrun.aws.json (v3) allows you to do is configure the s3 bucket and key to the location of your container repository authentication file ".dockercfg"
This is essentially the documentation for "Dockerrun.aws.json (v3)" it doesn't support anything similar to the "Dockerrun.aws.json (v2)
{
"AWSEBDockerrunVersion": "3",
"Authentication": {
"bucket": "DOC-EXAMPLE-BUCKET",
"key": "mydockercfg"
}
}
Include a docker-compose.yml and you'll need the dockerrun.aws.json (v3) only if the docker images are in a private repository.
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/single-container-docker-configuration.html
According to AWS Docs, Multi-container Docker running on Amazon Linux can be migrated to ECS on Amazon Linux 2
This option seems to be easier to apply with the CLI than using the Elastic Beanstalk console because it requires 1 single command:
aws elasticbeanstalk update-environment \
--environment-name ${my-env} \
--solution-stack-name "64bit Amazon Linux 2 ${version} running ECS" \
--region ${my-region}
I'd suggest that you first clone the environment you'd like to upgrade, apply the command mentioned above to this copied environment and test it, if everything works as expected then you can use a blue/green deployment to avoid downtime.
I hope this helps someone!
I have deployed TeamCity server and Agent to AWS using JetBrains Stack Template (https://www.jetbrains.com/help/teamcity/running-teamcity-stack-in-aws.html)
All seems to be good, my server starts, agent is functional, I have created several builds, etc.
I came to a point, where I want to deploy my application to AWS environment using aws-cli commands.
I am struggling to enable/install aws-cli into agent. My build steps are erroring out with aws: command not found
Does anyone have any ideas?
My progress so far: I have connected to agent EC2 machine via ssh bastion ec2, and I am able to invoke aws --version as ec2-user, but the build agent cannot see aws.
Turns out, my TeamCity agent runs in AWS ECS via docker image https://hub.docker.com/r/jetbrains/teamcity-agent
What I ended up doing is creating my own docker image by using jetbrains one as a base.
I uploaded my docker image to AWS ECS Repository. Afterwards I created a new revision for original task definition. This new revision uses my image instead of original one, therefore I have aws-cli there.
I then added my AWS profile to EC2 host machine and added volume to docker container (via task definition) so that container would be able to access .aws/credentials file.
Dockerfile looks like this:
FROM jetbrains/teamcity-agent
RUN apt-get update && apt-get install -y python-pip
RUN pip install awscli --upgrade --user
ENV PATH="~/.local/bin:${PATH}"
I added the aws-cli in team city agent using remote desktop connection as I used window agent of team city. In the build steps I used Runner Type as command line and executed the aws commands.
for more information you can refer below link where I answered the question:
How to deploy to AWS Elastic Beanstalk on successful Teamcity build
I've been using the AWS console to upload a WAR file for deployment. Now I want to do it from the command line. I've been following this guide and see eb init and read the help with eb init --help and eb --help, but the only option is to create a new application.
usage: eb init <application_name> [options ...]
Initializes your directory with the EB CLI. Creates the application.
positional arguments:
application_name application name
How do I link my local source project directory to an existing application in AWS console?
I would expect a command like eb link or something, like how you can just add a Git remote with Heroku and automatically link an existing project to an existing app.
When you perform eb init in the directory containing your source code, eb will prompt you for an application name and an environment name. This way you can link your source code to what ever application/environment is deployed on Beanstalk.
It worked after I got the AWS CLI keys for the project and ran aws configure. I had old keys in ~/.aws/ from a different project from perhaps a decade ago that used a different format. Once I got new keys, that were given permission for these particular apps, and ran aws configure and set the region, then eb init would present a menu of applications to choose from. The command aws elasticbeanstalk describe-applications has to work first before eb can work. I was expecting it would ask for a username and password, like Heroku does.
Install aws and eb command line tools:
Install awscli
Get keys from AWS admin devops.
aws configure (Example Region: 'us-east-1')
aws elasticbeanstalk describe-applications
Install Python
pip install awsebcli --upgrade --user
Add eb to your PATH, probably %USERPROFILE%\AppData\Roaming\Python\Python37\Scripts
eb init
eb list / eb logs / eb ssh / eb status / eb config / eb help
Beanstalk differs from Heroku in this workflow, unless you are using CodeCommit. I am assuming you are just using S3 to store your application versions.
The EBCLI command to do this is:
eb create-application-version
You can specify an application, a version label, as well as either a CodeCommit repository, a codeBuild build, or a source bundle in S3. API docs
You will need to run a separate command before create-application-version to upload to your S3 bucket.
Using the CLI:
aws s3 cp <filename> <s3bucket>
API docs
You can also use the console.
It seems like that guide skips initializing your local git repository. For linking your local source project to beanstalk, make sure you have initialized a local git repository. Then you can link your workspace and application using eb init. more about EB CLI and Git
Based on my understanding, your question is that you had a project directory on your PC and run your app at the localhost, now you want to run it in the AWS Elastic Beanstalk to make it public.
If you have created an EB application in the EB management console and uploaded your bundled source code, the source code becomes an application version, you need to deploy it into one of your environment using the EB management console, like this:
Figure of the management console.
Then the EB platform(container) will take care of that and run your server automatically as long as you set up the command which your app uses to run the server, the proxy, and other configurations either through the EB management console -> [Your environment] -> configuration or using the .ebextensions file.
If everything is well, you can visit your app's home page through the environment URL at that time.
I'm attempting to deploy an app from one ElasticBeanstalk instance to another. Running pip install awsebcli --upgrade --user doesnt install the eb cli tool for some odd reason on the EC2 machine.
Does anyone know the equivalent of eb deploy using only the aws cli options?
This question is a bit confusing. Are you attempting to move code between EC2 instances in your Beanstalk environment?
If I'm assuming correctly, you've pulled/changed your code on one Beanstalk host. And now you're trying to propagate that change to the other instances using the EB CLI. That's not a best practice. Beanstalk has a mechanism to deploy your code to all instances.
The EB CLI is meant to be run from your workstation to push code from your IDE/editor to the Beanstalk hosts in AWS.
Beanstalk keeps a copy of that code revision in S3. And if the Beanstalk environment is load balanced then all instances will be running the same application version when scaling events or deployments occur because it will pull your code from a common source.
But to answer your question:
Does anyone know the equivalent of eb deploy using only the aws cli options?
You're gonna wanna ZIP and upload your code to S3 and note the S3 key and bucket values of where it's located.
Then create a new application version.
% aws elasticbeanstalk create-application-version --application-name="<APPLICATION_NAME>" --version-label="<NEW_VERSION_LABEL>" --source-bundle="{\"S3Bucket\": \"<S3_BUCKET_NAME>\",\"S3Key\": \"<S3_KEY>\"}"
Then deploy your new application version to the running environment.
% aws elasticbeanstalk update-environment --environment-id="<ENVIRONMENT_ID>" --version-label="<NEW_VERSION_LABEL>"
Reading is hard...
Linux requires you to "[a]dd the path to the executable file to your PATH variable"
export PATH=~/.local/bin:$PATH
eb --version now works
I am developing an Elastic Beanstalk app. It is a Scala web application, built with sbt. I want to deploy the resulting WAR from the command line to an existing environment.
All I can find is the eb CLI which appears to require you to use git: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3-getting-started.html
Is there not a way to simply specify a WAR and environment name to perform the deployment?
What is the best workaround otherwise? I can upload to S3 from the command line and then use the web app to choose that file, but it's a bit more painful than I wanted.
You can use Elastic Beanstalk CLI (eb) instead of AWS CLI. Just run eb create to create a new environment and eb deploy to update your environment.
You can set specific artifact (your *.war file), by configuring the EB CLI (read: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3-configuration.html#eb-cli3-artifact):
You can tell the EB CLI to deploy a ZIP or WAR file that you generate
as part of a separate build process by adding the following lines to
.elasticbeanstalk/config.yml in your project folder.
deploy:
artifact: path/to/buildartifact.zip
I found a way - use the aws CLI instead. First upload to S3 (I actually use s3cmd) then create an application version:
$ aws elasticbeanstalk create-application-version --application-name untaggeddb --version-label myLabel --source-bundle S3Bucket="bucketName",S3Key="key.war"
I believe the application version can then be deployed with update-environment also using the aws CLI.