I am trying to deploy to elastic beanstalk using the cli.
The command I run is
eb create --modules ebtargets/goapi -v
I get the following error
WARNING: You have uncommitted changes.
INFO: Getting version label from git with git-describe
INFO: Uploading archive to s3 location: goapi/app-d4ec2-160630_135740.json
Uploading goapi/app-d4ec2-160630_135740.json to S3. This may take a while.
Upload Complete.
INFO: Creating AppVersion app-d4ec2-160630_135740
--- Waiting for application versions to be pre-processed ---
Finished processing application version app-d4ec2-160630_135740
--- Creating modules ---
ERROR: ServiceError - ApplicationVersion app-d4ec2-160630_135740 must specify an environment name in env.yaml
I have the key EnvironmentName in my env.yaml file. Any help on this?
I had a similar problem and I noticed you have to check your env.yaml into version control otherwise it won't be used:
git add env.yaml
git commit -m "add elastic beanstalk environment config"
Check that you have an .elasticbeanstalk folder into your ebtargets/goapi
If necessary, use eb init --modules ebtargets/goapi.
More information here
Related
I have a simple docker app that is able to run for me locally via docker-compose up, and when I send the .yml file to my friend, they are also able to get it up and running on their local machine. However, when I try to deploy it on Elastic Beanstalk, I get errors (specifically, something related to error:open /var/pids/eb-docker-compose-log.pid: no such file or directory, as I'll show below). I've tried to upload multiple times to Elastic Beanstalk, with the same errors. This is a custom app, but they are the same errors I got when I was trying to follow the instructions on https://docker-curriculum.com/#docker-on-aws. Here is the docker-compose.yml for my current app:
version: "3"
services:
server:
image: mfatigati/shop-server
container_name: shop-server
ports:
- "4000:4000"
client:
image: mfatigati/shop-client
depends_on:
- server
ports:
- "3000:3000"
mfatigati/shop-server and mfatigati/shop-client are both Node.JS apps, i.e., FROM node:16 in their Dockerfile.
To deploy this on AWS, I go to my EB console, and then:
Click "Create Application" to take me to the create app screen
Choose "Docker" as the platform
Choose "Upload local code", and upload the above-mentioned .yml file.
Click "Create Application"
Based on the notes here, I think this should be all I need to do (maybe I'm wrong about that?), but I get errors every time that point me to the eb.engine.log file. I've pasted what seems to be the relevant section below, as it is the only section that mentions errors, and it also reflects what appears in the AWS GUI console. The main problem seems reflected by the bit about error:open /var/pids/eb-docker-compose-log.pid: no such file or directory:
2022/02/14 14:17:23.619888 [ERROR] update processes [cfn-hup eb-docker-events healthd eb-docker-compose-events eb-docker-compose-log docker] pid symlinks failed with error Read pid source file /var/pids/eb-docker-compose-log.pid failed with error:open /var/pids/eb-docker-compose-log.pid: no such file or directory
2022/02/14 14:17:23.619901 [ERROR] An error occurred during execution of command [app-deploy] - [Track pids in healthd]. Stop running the command. Error: update processes [cfn-hup eb-docker-events healthd eb-docker-compose-events eb-docker-compose-log docker] pid symlinks failed with error Read pid source file /var/pids/eb-docker-compose-log.pid failed with error:open /var/pids/eb-docker-compose-log.pid: no such file or directory
2022/02/14 14:17:23.619905 [INFO] Executing cleanup logic
2022/02/14 14:17:23.620005 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1644848243,"severity":"ERROR"}]}]}
Any insight would be greatly appreciated! I'm pasting some screenshots below, in case that helps.
GUI corresponding to step 2; GUI corresponding to step 3; GUI errors
I had the same error:open /var/pids/eb-docker-compose-log.pid: no such file or directory error happening for my Docker Compose app when trying to deploy it to my Elastic Beanstalk environment; I'm not sure if my solution will be the same solution you need, but I hope it points you in the right direction (and helps future devs facing a similar problem).
What caused the error for me:
This ...eb-docker-compose-log.pid: no such file... error was a false error that was triggered by a separate issue; my separate error was actually a problem with my application code not finding the environment variables set in my Elastic Beanstalk environment. See below for how I found the problem, and what I did to fix it.
How I found my real problem:
I downloaded the Full Logs:
go to your EB environment
click Logs on the left nav
click the Request Logs dropdown button (at the top right)
click Full Logs
click the Download link once the full logs are ready
Inside of the logs, I found the real problem in the eb-docker/containers/eb-current-app/eb-stdouterr.log file, the issue being that my application code wasn't able to find the environment variables that were setup in my Elastic Beanstalk Software configuration.
In case you're curious what my error said:
panic: required key ONE_OF_MY_ENV_KEYS missing value
(I also had a couple other errors in this log that I fixed, but fixing the error shown above is what ended up solving the ...eb-docker-compose-log.pid: no such file... error).
How I fixed this error:
I turns out that if you use docker-compose.yml, while setting up your environment variables in your Elastic Beanstalk Software configuration, you have to make sure you use the .env file that Elastic Beanstalk creates for you; otherwise (from my own testing), EB only see's/uses the environment variable keys/values you specify in your own .env file or environment: list you can specify in docker-compose.yml.
NOTE: see the Elastic Beanstalk "Environment properties and Environment Variables" and "Referencing environment variables in containers" sections in the docs here, in particular this bit:
"Elastic Beanstalk generates a Docker Compose environment file called .env in the root directory of your application project. This file stores the environment variables you configured for Elastic Beanstalk.
Note
If you include a .env file in your application bundle, Elastic Beanstalk will not generate an .env file."
I solved my problem by updating my docker-compose.yml file to point to the supposed .env file that EB would create for me (by adding env_file: .env to my services that needed it), i.e.:
version: "3"
services:
my_service1:
# ...
env_file: .env
my_service2:
# ...
env_file: .env
I'm trying to get docker-compose deployment to AWS Elastic Beanstalk working, in which the docker images are pulled from a private registry hosted by GitLab.
The strange thing is that initial deployment works perfectly; It pulls the image from the private registry and starts the containers using docker-compose, and the webpage (served by Django) is accessible through the host.
Deploying a new version using the same docker-compose and the same docker image will result in an error while pulling the docker image:
2021/03/16 09:28:34.957094 [ERROR] An error occurred during execution of command [app-deploy] - [Run Docker Container]. Stop running the command. Error: failed to run docker containers: Command /bin/sh -c docker-compose up -d failed with error exit status 1. Stderr:Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Creating network "current_default" with the default driver
Pulling redis (redis:alpine)...
Pulling mysql (mysql:5.7)...
Pulling project.dockertest(registry.gitlab.com/company/spikes/dockertest:latest)...
Get https://registry.gitlab.com/v2/company/spikes/dockertest/manifests/latest: denied: access forbidden
2021/03/16 09:28:34.957104 [INFO] Executing cleanup logic
Setup
AWS Elastic Beanstalk 64bit Amazon Linux 2/3.2
Gitlab registry credentials are stored within a S3 bucket, with the filename .dockercfg and has the following content:
{
"auths": {
"registry.gitlab.com": {
"auth": "base64 encoded username:personal_access_token"
}
},
"HttpHeaders": {
"User-Agent": "Docker-Client/18.03.1-ce (linux)"
}
}
The repository contains a v3 Dockerrun.aws.json file to refer to the credential file in S3:
{
"AWSEBDockerrunVersion": "3",
"Authentication": {
"bucket": "gitlab-dockercfg",
"key": ".dockercfg"
}
}
Reproduce
Setup docker-compose.yml that uses a service with a private docker image (and can be pulled with the credentials setup in the dockercfg within S3)
Create a new applicatoin that uses the docker-platform.
eb init testapplication --platform=docker --region=eu-west-1
Note: region must be the same as the S3 bucket containing the dockercfg.
Initial deployment (this will succeed)
eb create testapplication-test --branch_default --cname testapplication-test --elb-type=application --instance-types=t2.micro --min-instance=1 --max-instances=4
The initial deployment shows that the image is available and can be started:
2021/03/16 08:58:07.533988 [INFO] save docker tag command: docker tag 5812dfe24a4f redis:alpine
2021/03/16 08:58:07.533993 [INFO] save docker tag command: docker tag f8fcde8b9ae2 mysql:5.7
2021/03/16 08:58:07.533998 [INFO] save docker tag command: docker tag 1dd9b65d6a9f registry.gitlab.com/company/spikes/dockertest:latest
2021/03/16 08:58:07.534010 [INFO] Running command /bin/sh -c docker rm `docker ps -aq`
Without changing anything to the local repository and the remote docker image on the private registry, lets do a redeployment which will trigger the error:
eb deploy testapplication-test
This will fail with the following output:
...
2021-03-16 10:02:28 INFO Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
2021-03-16 10:02:29 ERROR Unsuccessful command execution on instance id(s) 'i-0dc445d118ac14b80'. Aborting the operation.
2021-03-16 10:02:29 ERROR Failed to deploy application.
ERROR: ServiceError - Failed to deploy application.
And logs of the instance show (/var/log/eb-engine.log):
Pulling redis (redis:alpine)...
Pulling mysql (mysql:5.7)...
Pulling project.dockertest (registry.gitlab.com/company/spikes/dockertest:latest)...
Get https://registry.gitlab.com/v2/company/spikes/dockertest/manifests/latest: denied: access forbidden
2021/03/16 10:02:25.902479 [INFO] Executing cleanup logic
Steps I've tried to debug or solve the issue
Rename dockercfg to .dockercfg on S3 (somewhere mentioned on the internet as possible solution)
Use the 'old' docker config format instead of the one generated by docker 1.7+. But later on I figured out that Amazon Linux 2-instances are compatible with the new format together with Dockerrun v3
Having an incorrectly formatted dockercfg on S3 will cause an error deployment regarding the misformatted file (so it actually does something with the dockercfg from S3)
Documentation
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/single-container-docker-configuration.html
I'm out of debug options, and I've no idea where to look any further to debug this problem. Perhaps someone can see what is going wrong here?
First of all, the issue describe above is a bug confirmed by Amazon. To get the deployment working on our side, we've contacted Amazon support.
They've a fix in place which should be released this month, so keep an eye on the changelog of the Elastic beanstalk platform: https://docs.aws.amazon.com/elasticbeanstalk/latest/relnotes/relnotes.html
Although the upcoming release should have the fix, there is a workaround available to get the docker-compose deployment working.
Elastic Beanstalk allows hook to be executed within the deployment, which can be used to fetch the .docker.cfg from a S3 bucket to authenticate with against the private registry.
To do so, create the following file and directories from the root of the project:
File location: .platform/hooks/predeploy/docker_login
#!/bin/bash
aws s3 cp s3://{{bucket_name_to_use}}/.dockercfg ~/.docker/config.json
Important: Add execution rights to this file (for example: chmod +x .platform/hooks/predeploy/docker_login)
To support instance configuration changes, please symlink the hooks directory to confighooks:
ln -s .platform/hooks/ .platform/confighooks/
Updating configuration requires the .dockercfg credentials to be fetched too.
This should enable continuous deployments to the same EB-instance without the authentication errors, because the hook will be execute before the docker image pulling.
Some background:
The docker daemon reads credentials from ~/.docker/config by default on traditional linux systems. On the initial deploy this file will exist on the Elastic Beanstalk instance. On the next deployment this file is removed. Unfortunately, on the next deployment the .dockercfg is not refetched, therefor the docker daemon does not have the correct credentials to authenticate with.
I was dealing the same errors while trying to pull images from a privately hosted GitLab instance. I was able to resolve them by including the email address that was associated with the generated token found in the auth field of the .dockercfg file.
The following file format worked for me:
"registry.gitlab.com" {
"auth": "base64 encoded username:personal_access_token",
"email": "email for personal access token"
}
In my case I used a Project Access Token, which has an e-mail address associated with it once it is created.
The file format in the Elastic Beanstalk documentation for the authentication file here, indicates that this is the required file format, though the versions that it says this format is required for are almost certainly outdated, since we are running Docker ^19.
I recently was able to get my Laravel app deployed using codepipeline on Elastic Beanstalk but ran into a problem. I noticed that my routes where failing because of php.conf Nginx configuration. I had to add a few lines of code to EB's nginx php.conf file to get it to work.
My problem now was that after every deployment, the instance of the application I modified the php.conf file was destroyed and recreated fresh. I wanted a way to dynamically update the file after every successful deployment. I had a version of the file I wanted versioned with my application and so wanted to create a symlink to that file after deployment.
After loads of research, I stumbled on appDeploy Hooks on Elastic Beanstalk that runs post scripts after deployment so did this
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/91_post_deploy_script.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
sudo mkdir /var/testing1
sudo ln -sfn /var/www/html/php.conf.example /etc/nginx/conf.d/elasticbeanstalk/php.conf
sudo mkdir /var/testing
sudo nginx -s reload
And this for some reason does not work. The symlink is not created so my routes are still not working..
I even added some mkdir so am sure the commands in that script runs, none of those commands ran because none of those directories where created.
Please note that if I ssh into the ec2 instance and run the commands there it works. That bash script also exists in the post directory and if I manually run in on the server it works too.
Any pointers to how I could fix this would be helpful. Maybe I am doing something wrong too.
Now I have gotten my scripts to run by following this. However, the script is not running. I am getting an error
2020/06/28 08:22:13.653339 [INFO] Following platform hooks will be executed in order: [01_myconf.config]
2020/06/28 08:22:13.653344 [INFO] Running platform hook: .platform/hooks/postdeploy/01_myconf.config
2020/06/28 08:22:13.653516 [ERROR] An error occurred during execution of command [app-deploy] - [RunPostDeployHooks]. Stop running the command. Error: Command .platform/hooks/postdeploy/01_myconf.config failed with error fork/exec .platform/hooks/postdeploy/01_myconf.config: permission denied
I tried to follow this forum post here to make my file executable by adding to my container command a new command like so:
01_chmod1:
command: "chmod +x .platform/hooks/postdeploy/91_post_deploy_script.sh"
I am still running into the same issue. Permission denied
Sadly, the hooks you are describing (i.e. /opt/elasticbeanstalk/hooks/appdeploy) are for Amazon Linux 1.
Since you are using Amazon Linux 2, as clarified in the comments, the hooks you are trying to use do not apply. Thus they are not being executed.
In Amazon Linux 2, there are new hooks as described here and they are:
prebuild – Files here run after the Elastic Beanstalk platform engine downloads and extracts the application source bundle, and before it sets up and configures the application and web server.
predeploy – Files here run after the Elastic Beanstalk platform engine sets up and configures the application and web server, and before it deploys them to their final runtime location.
postdeploy – Files here run after the Elastic Beanstalk platform engine deploys the application and proxy server.
The use of these new hooks is different than in Amazon Linux 1. Thus you have to either move back to Amazon Linux 1 or migrate your application to Amazon Linux 2.
General migration steps from Amazon Linux 1 to Amazon Linux 2 in EB are described here
Create a folder called .platform in your project root folder and create a file with name 00_myconf.config inside the .platform folder.
.platform/
00_myconf.config
Open 00_myconf.config and add the scripts
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/91_post_deploy_script.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
sudo mkdir /var/testing1
sudo ln -sfn /var/www/html/php.conf.example /etc/nginx/conf.d/elasticbeanstalk/php.conf
sudo mkdir /var/testing
sudo nginx -s reload
Commit your changes or reupload the project. This .platform folder will be considered in each new instance creation and your application will deploy properly in all the new instances Amazon Elastic beanstalk creates.
If you access the documentation here and scroll to the section with the title "Application example with extensions" you can see an example of the folder structure of your .platform folder so it adds your custom configuration to NGINX conf on every deploy.
You can either replace the entire nginx.conf file with your file or add additional configuration files to the conf.d directory
Replace conf file with your file on app deploy:
.platform/nginx/nginx.conf
Add configuration files to nginx.conf:
.platform/nginx/conf.d/custom.conf
I have created a ruby env on amazon elastic beanstalk, but when I try to deploy my rails app from command line using eb deploy I get this error:
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
You need to install git to be able to use gems from git repositories. For help
installing git, please refer to GitHub's tutorial at
https://help.github.com/articles/set-up-git (Executor::NonZeroExitStatus)
[2015-08-09T15:50:38.513Z] INFO [4217] - [CMD-AppDeploy/AppDeployStage0/AppDeployPreHook/10_bundle_install.sh] : Activity failed.
[2015-08-09T15:50:38.513Z] INFO [4217] - [CMD-AppDeploy/AppDeployStage0/AppDeployPreHook] : Activity failed.
[2015-08-09T15:50:38.513Z] INFO [4217] - [CMD-AppDeploy/AppDeployStage0] : Activity failed.
[2015-08-09T15:50:38.514Z] INFO [4217] - [CMD-AppDeploy] : Completed activity. Result:
Command CMD-AppDeploy failed.
So, shall I install git at amazon instance bash directly? will this effect autoscaling?
I don't know if you fixed this, but you need to tell Elastic Beanstalk to install git.
In the root directory of your project, add a folder called .ebextensions.
Create a file inside that folder called (something like) install_git.config (the .config is important).
Add the following lines to that file:
packages:
yum:
git: []
Then redeploy your application, and you shouldn't see that error anymore.
I'm trying to deploy my code to AWS Beanstalk and get this error. I researched that it could be that the number of versions is more than 500, so I deleted a lot of versions. But, I still get this error.
eb deploy
ERROR: No Application Version named 'v0_9_2-76-gf5a4' found.
I also tried
git aws.push
Error: Failed to create the AWS Elastic Beanstalk application version
Edit:
Trying with eb deploy --debug I now get:
Instance: i-2ad238d5 Module: AWSEBAutoScalingGroup ConfigSet: null Command failed on instance. Return code: 1 Output: Error occurred during build: Command hooks failed . Script /opt/elasticbeanstalk/hooks/appdeploy/pre/10_bundle_install.sh failed with returncode 18
ebcli.objects.exceptions.ServiceError: Update environment operation is
complete, but with errors. For more information, see troubleshooting
documentation.
Did you update the file .elasticbeanstalk/config.yml ? It may have a wrong setup.
Make a backup of .elasticbeanstalk/ folder and remove it
Execute eb create
Select the same region you deployed it before. You can check the region on .elasticbeanstalk/config.yml backup
A list with the environments will appear, select the right one
Deploy now
Remove the .elasticbeanstalk/config.yml backup
Check for the .elasticbeanstalk/config.yml file
environment: CORRECT_ENV_NAME
global:
application_name: CORRECT_APP_NAME
In my case, I was doing eb deploy X where X was an environment for a different project.
When I had the error
InvalidParameterValueError: No Application Version named 'app-9f5c-180927_071528' found.
I fixed this by specifying the label I wanted to push up.
eb deploy XXX-env -l XXX.0.0.1
The -l flag is documented AWS EB Deploy Docs
most likely, the deploy is trying an incorrect Elasticbeanstalk Application. it could be because you renamed the application in the AWS console.
so double check you're pointing to the correct elasticbeanstalk Environment and Application. it could be picking out default values from your .elasticbeanstalk/config.yml file.