In Elastic Beanstalk, code is uploaded to an S3 bucket and then mounted to /var/app/current in the EC2 instance so all the volumes need to be sourced from there.
So my Dockerrun.aws.json looks like this:
"volumes": [
{
"host": {
"sourcePath": "/var/app/current/foo" // <<< problem lies here
},
"name": "foo"
}
Because of this /var/app/current thing, doing $ eb local run fails because the source path on my local machine is ~/bar/foo instead of /var/app/current/foo when deployed. The error is given below:
Mounts denied:
The path /var/app/current
is not shared from OS X and is not known to Docker.
You can configure shared paths from Docker -> Preferences... -> File Sharing.
See https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.
Given that i want to have my eb deployments work locally and remotely with the same config file, how should I overcome this problem?
Stack:
AWS Elastic Beanstalk Multi-container Docker
MacOS, Docker for Mac
By default Docker for Mac shared below paths
/Users
/Volumes
/tmp
/private
So if your path starts with any of these and you can keep both paths on local & AWS then it would work. Else you should share the /var on your Docker for Mac settings
Related
I am following a tutorial to deploy a Flask application with Docker to AWS Elastic Beanstalk (EB). I created an AWS Elastic Container Registry (ECR) and ran some commands which successfully pushed the Docker image to the ECR:
docker build -t app-backend
docker tag app-backend:latest [URL_ID].dkr.ecr.us-east-1.amazonaws.com/app-backend:latest
docker push [URL_ID].dkr.ecr.us-east-1.amazonaws.com/app-backend:latest
Then I tried to deploy to EB:
eb init (selecting a Docker EB application I created on the AWS GUI)
eb deploy
On "eb init" I get the error "Cannot setup CodeCommit because there is no Source Control setup, continuing with initialization", but I assume this can be ignored as it otherwise looked fine. On "eb deploy" though, the deployment fails. In "eb-engine.log" (found in the AWS GUI), I see error messages like:
[ERROR] An error occurred during execution of command [app-deploy] - [Docker Specific Build Application]. Stop running the command. Error: failed to pull docker image: Command /bin/sh -c docker pull [URL_ID].dkr.ecr.us-east-1.amazonaws.com/app-backend:latest failed with error exit status 1. Stderr:failed to register layer: Error processing tar file(exit status 1): write /root/.cache/pip/http/5/e/7/3/b/[long number]: no space left on device
When I manually run the pull command the error references (locally, not from the EB instance), the command seems to respond as expected:
docker pull [URL_ID].dkr.ecr.us-east-1.amazonaws.com/app-backend:latest
What could be causing this deployment failure?
My Dockerrun.aws.json file looks like this:
{
"AWSEBDockerrunVersion": "1",
"Image": {
"Name": "[URL_ID].dkr.ecr.us-east-1.amazonaws.com/app-backend",
"Update": "true"
},
"Ports": [
{
"ContainerPort": 5000,
"HostPort": 5000
}
]
}
I solved this by following how to prevent error "no space left on device" when deploying multi container docker application on AWS beanstalk?.
Basically you find your Elastic Beanstalk instance in the EC2 AWS GUI, you modify the volumes to add space to the EB instance. Then you follow the link in that Stack Overflow post to repartition your EB instance by SSHing into it with eb ssh and then using commands like df -H and lsblk to see how much space in in each partition. And use commands like:
sudo growpart /dev/xvda 1
sudo xfs_growfs -d /
to repartition the hard drive as to use all the new space you added in the AWS EC2 GUI. You can check with df -H and lsblk to see if the repartitioning gave you more space.
Then the eb deploy command should work. If SSH isn't setup yet, you may have to do eb ssh --setup first.
I am following the AWS tutorial that tells us how to mount EFS system to elastic beanstalk instances available at https://aws.amazon.com/premiumsupport/knowledge-center/elastic-beanstalk-mount-efs-volumes/.
I am mounting this folder inside the current folder, as it needs to be accessible by my web application as a path, e.g: public/medias. So I am mounting EFS inside my app public folder and all media will be accessible through the webserver.
The first mount is ok, but after the first deployment, it seems that elastic beanstalk is trying to remove the folder when clearing up the app/current folder for a new deployment, then the deploy fails to remove the mounted unit which is inside current with a message that can not remove directory - Device or resource busy.
It's not possible to mount EFS directories inside current folder for elastic beanstalk? Or I should consider in mounting it outside of the application folder then I could use something like symlink for accessing the files through a web server?
The reason why I am mounting this inside var/app/current/public/media is because it needs to be accessible through https://mywebsite.com/media/myImage.png => this should come from EFS.
I could consider using S3 Buckets instead but all my web application is reading files using a static path and it would be a massive work to migrate that to read from a bucket.
I had this same issue for a while and finally found the answer in this article from AWS Knowledge Center:
Important: You can't mount an Amazon EFS volume directly to the application directory because the contents of /var/app/current are moved to /var/app/current.old whenever you deploy an Elastic Beanstalk application.
The solution is to mount EFS to some other directory and create a symlink between that and your /current directory, or whatever other sub directory you are mounting to in /var/app/current.
I did this by modifying my storage-efs-mountfilesystem.config (in .ebextensions - example efs-mountfilesystem.config) to mount to /efs instead of /var/app/current/wwwroot/user_content, and added a new storage-efs-symlink.config (name it whatever you like) config file in .ebextensions that runs this command:
container_commands:
01_symlink:
command: ln -s /efs ./wwwroot/user_content
Where the first argument is what you specified as your EFS mount point, and the second argument is where your app attempts to read/write to (in my case, /var/app/current/wwwroot/user_content -> ./wwwroot/user_content).
(Note the relative path. It needs to be relative because these commands execute when the app is still in the /var/app/staging, so the /current directory doesn't exist yet. More Here.)
Here was my exact error in eb-engine.log when attempting to deploy an elastic beanstalk application with EFS mounted in /var/app/current: An error occurred during execution of command [app-deploy] - [FlipApplication]. Stop running the command. Error: remove current dir failed: unlinkat /var/app/current/wwwroot/user_content: device or resource busy
If you don't already have a .ebextensions folder, you can create it yourself and make sure it ends up in your application source bundle. More information about .ebextensions here.
I've deployed an Aurelia application to AWS Elastic Beanstalk via AWS ECR and have run into some difficulty. The docker container, when run locally, works perfectly (see below for Dockerfile).
FROM nginx:1.15.8-alpine
COPY dist /usr/share/nginx/html
The deployment works quite well, however when I navigate to the AWS provided endpoint http://docker-tester.***.elasticbeanstalk.com/ I get 502 Bad Gateway
nginx/1.12.1.
I can't figure out what might be the issue. The docker container in question is a simple Hello World example created via the au new command; it's nothing fancy at all.
Below is my Dockerrun.aws.json file
{
"AWSEBDockerrunVersion": "1",
"Image": {
"Name": "***.dkr.ecr.eu-central-1.amazonaws.com/tester:latest",
"Update": "true"
},
"Ports": [
{
"ContainerPort": "8080"
}
],
"Logging": "/var/log/nginx"
}
My Elastic Beanstalk configuration is rather small with an EC2 instance type of t2.micro. I'm using the free tier as an opportunity to learn.
I greatly appreciate any help, or links to some reading that may point in the right direction.
It has nothing to do with your aurelia application. You are missing EXPOSE statement (which is mandatory) in your Dockerfile. You can change it like this.
FROM nginx:1.15.8-alpine
EXPOSE 80
COPY dist /usr/share/nginx/html
If you try to run it without EXPOSE, you will get an error
ERROR: ValidationError - The Dockerfile must list ports to expose on the Docker container. Specify at least one port, and then try again.
You should test your application before pushing it to ElasticBeanstalk
install eb cli (assuming that you have pip, if not then you need to install it as well)
pip install awsebcli --upgrade --user
then initialize local repository for deployment
eb init -p docker <application-name>
and you can test it
eb local run --port <port-number>
So I am using Travis CI to automatically deploy my application to AWS Elasticbeanstalk environment. I have this issue that I need to update the nginx.conf file that is located in the host machine files.
Im running a Single container Docker image inside that host machine.
How can I copy or link the nginx.conf file from docker container to host machines nginx.conf file.
Currently my Dockerrun.aws.json looks like that:
{
"AWSEBDockerrunVersion": "1",
"Image": {
"Name": "some:image:url:here",
"Update": "true"
},
"Ports": [
{
"ContainerPort": "8001"
}
],
"Volumes": [
{
"HostDirectory": "/etc/nginx/nginx.conf",
"ContainerDirectory": "/home/node/app/nginx.conf"
}
]
}
When I tried to use dockerrunversion: 2, it gave me an error on the build that version is wrong.
How can I link those two files with Single Container Docker application?
The "Volumes" key is used to map full volumes, not individual files. See Dockerrun.aws.json file specifications for an explanation.
I know of 2 ways you can solve this problem: 1) Custom AMI or 2) use a Dockerfile with your Dockerrun.aws.json.
1. Build a Custom AMI
The idea behind building a custom AMI is to launch an instance from one of Amazons existing AMIs. You make the changes you need to it (in your case, change the nginx.conf). Finally you create a new AMI from this instance and it will be available to you when you create your environment in Elastic Beanstalk. Here are the detailed steps to create your own AMI and how to use it with Elastic Beanstalk.
2. Use a Dockerfile with your Dockerrun.aws.json
If you dont build your own AMI, you can copy your conf file with the help of a Dockerfile. Dockerfile is a text file that provides commands to Elastic Beanstalk to run to build your custom image. The Docerfile reference details the commands that can be added to a Dockerfile to build your image. You are going to need to to use the Copy command or if the file is simple, you can use Run and echo to build it like in the example here.
Once you create your Dockerfile, you will need to put the Dockerfile and your Dockerrun.aws.json into a directory and create a zip file with both. Provide this to Elastic Beanstalk as your source bundle. Follow this guide to build the source bundle correctly.
I have a node.js application packaged in a docker image hosted in a public repository.
I have deployed that image in an AWS Beanstalk docker application successfully.
The problem is that I was expecting the Beanstalk application to be automatically updated when I update the image in the public repository, as the following configuration sugggests.
Dockerrun.aws.json:
{
"AWSEBDockerrunVersion": "1",
"Image": {
"Name": "peveuve/dynamio-payment-service",
"Update": "true"
},
"Ports": [
{
"ContainerPort": "8000"
}
],
"Logging": "/var/log/dynamio"
}
The Dockerfile is very simple:
FROM node:4.2.1-onbuild
# Environment variables
ENV NODE_ENV test
ENV PORT 8000
# expose application port outside
EXPOSE $PORT
The Amazon documentation is pretty clear on that:
Optionally include the Update key. The default value is "true" and
instructs Elastic Beanstalk to check the repository, pull any updates
to the image, and overwrite any cached images.
But I have to update the Beanstalk application manually by uploading a new version of the Dockerrun.aws.json descriptor. Did I miss something? Is it supposed to work like that?
You can use the aws command-line tool to trigger the update:
aws elasticbeanstalk update-environment --application-name [your_app_name] --environment-name [your_environment_name] --version-label [your_version_label]
You specify the version that contains the Dockerrun.aws.json file, that way a new version won't be added to the application. In this case the Dockerrun file works as the "source" for the application, but it only tells aws to pull the docker image, so it would be redundant to create new versions for the application in Elastic Beanstalk (unless you use specifically tagged docker images in the Dockerrun file)
Links:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_image.html
http://docs.aws.amazon.com/elasticbeanstalk/latest/api/API_UpdateEnvironment.htm
The documentation should be more clear. What they are saying is with update=true:
EBS will do a docker pull before it does a docker run when the application is first started. It will not continually poll docker hub.
In contrast, issuing a docker run without first doing a docker pull will always use the locally stored version of that machine, which may not always be the latest.
In order to acheive what you want, you'll need to set up a webhook on Docker Hub, that calls an application you control, that rebuilds your ELB app.