Your requirements.txt is invalid. AWS Elastic Beanstalk, Django - django

i am new to aws and django. I was tryinh to upload my code to aws elastic beanstalk using code commit while i am getting the following error
2020/03/12 13:42:30.200017 [ERROR] Command timed out after 300 seconds
2020/03/12 13:42:30.202535 [ERROR] Command /bin/sh [-c python3 -m pipenv install -r requirements.txt --skip-lock] failed with error signal: killed
2020/03/12 13:42:30.204812 [ERROR] An error occurred during execution of command [app-deploy] - [SetUpPythonEnvironment]. Stop running the command.
Error: fail to install dependencies with requirements.txt file with error Command /bin/sh [-c python3 -m pipenv install -r requirements.txt --skip-lock] failed with error signal: killed
2020/03/12 13:42:30.204825 [INFO] Executing cleanup logic
2020/03/12 13:42:30.210383 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[]}]}
2020/03/12 13:42:30.211588 [INFO] Platform Engine finished execution on command: app-deploy

Related

Getting error when trying to deploy spring boot + mysql app to AWS using codepipeline

I have build a cicd pipeline using AWS codepipeline.
Source is coming from Github which successed. Build is also successed.
I am using Corretto 11 running on 64bit Amazon Linux 2 as a platform in Elastic Beanstalk as Java 8 running on 64bit Amazon Linux shows as it's being deprecated.
I'm getting error during deployment to AWS Elastic Beanstalk.
Action execution failed
Deployment completed, but with errors: Failed to deploy application. Unsuccessful command execution on instance id(s) 'i-03b7a9c8a86ffde8e'. Aborting the operation. [Instance: i-03b7a9c8a86ffde8e] Command failed on instance. Return code: 1 Output: Engine execution has encountered an error.. Instance deployment failed. For details, see 'eb-engine.log'.
I checked eb-engine.log file and see below error message.
...
...
2021/11/07 05:01:18.765214 [INFO] extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /tmp/extracted_app_source_bundle
2021/11/07 05:01:18.765227 [INFO] Running command /bin/sh -c /usr/bin/unzip -q -o /opt/elasticbeanstalk/deployment/app_source_bundle -d /tmp/extracted_app_source_bundle
2021/11/07 05:01:19.122461 [INFO] finished extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /tmp/extracted_app_source_bundle successfully
2021/11/07 05:01:19.122560 [INFO] app source bundle is zip file ...
2021/11/07 05:01:19.122566 [INFO] extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /var/app/staging/
2021/11/07 05:01:19.122578 [INFO] Running command /bin/sh -c /usr/bin/unzip -q -o /opt/elasticbeanstalk/deployment/app_source_bundle -d /var/app/staging/
2021/11/07 05:01:19.498171 [INFO] finished extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /var/app/staging/ successfully
2021/11/07 05:01:19.498694 [INFO] Executing instruction: RunAppDeployPreBuildHooks
2021/11/07 05:01:19.498712 [INFO] The dir .platform/hooks/prebuild/ does not exist in the application. Skipping this step...
2021/11/07 05:01:19.498717 [INFO] Executing instruction: Java Specific Build Application
2021/11/07 05:01:19.498724 [INFO] no buildfile found, skip building java application
2021/11/07 05:01:19.498731 [INFO] old env file for build tasks does not exist
2021/11/07 05:01:19.498742 [INFO] Executing instruction: CheckProcfileForJavaApplication
2021/11/07 05:01:19.498776 [ERROR] An error occurred during execution of command [app-deploy] - [CheckProcfileForJavaApplication]. Stop running the command. Error: there is no Procfile and no .jar file at root level of your source bundle
2021/11/07 05:01:19.498786 [INFO] Executing cleanup logic
2021/11/07 05:01:19.498869 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1636261279,"severity":"ERROR"}]}]}
...
...
Below is my buildspec.yml file.
version: 0.2
phases:
install:
runtime-versions:
java: corretto11
commands:
- echo install
pre_build:
commands:
- echo pre_build
build:
commands:
- mvn package
- echo build
post_build:
commands:
- echo post_build
artifacts:
files:
- target/course_reviews_backend-0.0.1-SNAPSHOT.jar
PLEASE HELP!!
Any help would be appericated. Thanks in advance.

AWS beanstalk fails installing pip requisites when downloading tensorflow==2.4.0

I'm trying to deploy a flask app that predicts whether a lung x-ray picture is COVID-19 positive using a Tensorflow CNN. It is running on python 3.7
This is my requirements.txt
tensorflow==2.4.0
Flask==1.1.2
Keras==2.4.3
numpy==1.19.5
Pillow==8.3.1
After deployment it, the environment's health status is labelled severe with a '502 Bad gateway' error.
These are the errors in the eb-engine.log
2021/08/23 10:47:40.676374 [INFO] Installing dependencies with requirements.txt by using Pip
2021/08/23 10:47:40.676387 [INFO] Running command /bin/sh -c /var/app/venv/staging-LQM1lest/bin/pip install -r requirements.txt
2021/08/23 10:47:48.296114 [INFO] Collecting tensorflow==2.4.0
Downloading tensorflow-2.4.0-cp38-cp38-manylinux2010_x86_64.whl (394.8 MB)
2021/08/23 10:47:48.300932 [ERROR] An error occurred during execution of command [app-deploy] - [InstallDependency]. Stop running the command. Error: fail to install dependencies with requirements.txt file with error Command /bin/sh -c /var/app/venv/staging-LQM1lest/bin/pip install -r requirements.txt failed with error exit status 2. Stderr:ERROR: Exception:
Traceback (most recent call last):
File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 180, in _main
status = self.run(options, args)
File "... repeats itself until here:
File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/msgpack/fallback.py", line 896, in _pack
return self._buffer.write(obj)
MemoryError
2021/08/23 10:47:48.300957 [INFO] Executing cleanup logic
2021/08/23 10:47:48.309000 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed to install application dependencies. The deployment failed.","timestamp":1629715668,"severity":"ERROR"},{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1629715668,"severity":"ERROR"}]}]}
2021/08/23 10:47:48.312983 [INFO] Platform Engine finished execution on command: app-deploy
2021/08/23 12:34:34.689865 [INFO] Starting...
2021/08/23 12:34:34.689926 [INFO] Starting EBPlatform-PlatformEngine
2021/08/23 12:34:34.689965 [INFO] reading event message file
2021/08/23 12:34:34.690627 [INFO] no eb envtier info file found, skip loading env tier info.
2021/08/23 12:34:34.690716 [INFO] Engine received EB command cfn-hup-exec
why is AWS-Beanstalk failing to install Tensorflow?
I changed the environment EC2 capacity from t2.small to c5.xlarge and the problem was solved.
Your application needs more memory space.

Django/AWS EB: [app-deploy] - [RunAppDeployPreBuildHooks] error (no such file or directory), BUT the file does exist?

I have a Django application and I'm trying to deploy to my AWS EB Environment.
It's a project that my friend and I are working on.
I an trying to run the command eb deploy however though I get this:
Alert: The platform version that your environment is using isn't recommended. There's a recommended version in the same platform branch.
Creating application version archive "app-3d19-210731_133226".
Uploading Prod Rest API/app-3d19-210731_133226.zip to S3. This may take a while.
Upload Complete.
2021-07-31 17:32:29 INFO Environment update is starting.
2021-07-31 17:32:33 INFO Deploying new version to instance(s).
2021-07-31 17:32:36 ERROR Instance deployment failed. For details, see 'eb-engine.log'.
2021-07-31 17:32:39 ERROR [Instance: i-05761282d68083a51] Command failed on instance. Return code: 1 Output: Engine execution has encountered an error..
2021-07-31 17:32:39 INFO Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
2021-07-31 17:32:39 ERROR Unsuccessful command execution on instance id(s) 'i-05761282d68083a51'. Aborting the operation.
2021-07-31 17:32:39 ERROR Failed to deploy application.
ERROR: ServiceError - Failed to deploy application.
I checked the eb-engine.og and this is what I get:
2021/07/31 17:19:47.104584 [INFO] Executing instruction: StageApplication
2021/07/31 17:19:47.111204 [INFO] extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /var/app/staging/
2021/07/31 17:19:47.111230 [INFO] Running command /bin/sh -c /usr/bin/unzip -q -o /opt/elasticbeanstalk/deployment/app_source_bundle -d /var/app/staging/
2021/07/31 17:19:47.154001 [INFO] finished extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /var/app/staging/ successfully
2021/07/31 17:19:47.156956 [INFO] Executing instruction: RunAppDeployPreBuildHooks
2021/07/31 17:19:47.156982 [INFO] Executing platform hooks in .platform/hooks/prebuild/
2021/07/31 17:19:47.157046 [INFO] Following platform hooks will be executed in order: [install_supervisor.sh]
2021/07/31 17:19:47.157060 [INFO] Running platform hook: .platform/hooks/prebuild/install_supervisor.sh
2021/07/31 17:19:47.157342 [ERROR] An error occurred during execution of command [app-deploy] - [RunAppDeployPreBuildHooks]. Stop running the command. Error: Command .platform/hooks/prebuild/install_supervisor.sh failed with error fork/exec .platform/hooks/prebuild/install_supervisor.sh: no such file or directory
2021/07/31 17:19:47.157349 [INFO] Executing cleanup logic
2021/07/31 17:19:47.157565 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1627751987,"severity":"ERROR"}]}]}
2021/07/31 17:19:47.159200 [INFO] Platform Engine finished execution on command: app-deploy
I'm really confused on why this is occuring because for my friend who has the same repo, he is able to eb deploy fine but I can't for some reason.
This is my file structure:
my-app/
├─ .ebextensions/
│ ├─ 01_django.config
├─ .elasticbeanstalk/
│ ├─ config.yml
├─ .platform/
│ ├─ files/
│ │ ├─ supervisor.ini
│ ├─ hooks/
│ │ ├─ prebuild/
│ │ │ ├─ install_supervisor.sh
├─ other files/
where my other files are my actual Django code itself.
01_django.config
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: RestAPI.wsgi:application
aws:elasticbeanstalk:environment:proxy:staticfiles:
/static: static
config.yml
branch-defaults:
main:
environment: Prodrestapi-env
environment-defaults:
Prodrestapi-env:
branch: null
repository: null
global:
application_name: Prod Rest API
default_ec2_keyname: aws-eb
default_platform: Python 3.8 running on 64bit Amazon Linux 2
default_region: us-east-2
include_git_submodules: true
instance_profile: null
platform_name: null
platform_version: null
profile: eb-cli
sc: git
workspace_type: Application
supervisor.ini
# Create celery configuraiton script
[program:celeryd-worker]
; Set full path to celery program if using virtualenv
command=sh /var/app/current/scripts/worker.sh
directory=/var/app/current
; user=nobody
numprocs=1
stdout_logfile=/var/log/celery-worker.log
stderr_logfile=/var/log/celery-worker.log
autostart=true
autorestart=true
startsecs=60
; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 60
; When resorting to send SIGKILL to the program to terminate it
; send SIGKILL to its whole process group instead,
; taking care of its children as well.
killasgroup=true
; if rabbitmq is supervised, set its priority higher
; so it starts first
priority=998
install_supervisor.sh
#!/bin/sh
sudo amazon-linux-extras enable epel
sudo yum install -y epel-release
sudo yum -y update
sudo yum -y install supervisor
sudo systemctl start supervisord
sudo systemctl enable supervisord
sudo cp .platform/files/supervisor.ini /etc/supervisord.d/celery.ini
sudo supervisorctl reread
sudo supervisorctl update
I"m just really confused on why it's not deploying properly because the install_supervisor.sh file is clearly in the folder.
Any and all help will be really appreciate.
Thank you so much!

Docker build timeouts on AWS Beanstalk for Python app

I have a Python2/Django application running on AWS Beanstalk. I use Dockerfile for deployment. Everything was working fine until I added firebase-admin==3.2.1 dependency to my requirements.txt. What happens is that the Docker image build time took longer than usual and Beanstalk build process started to fail. With more analysis, I noticed that a sub-dependency is what takes long time to install, grpcio==1.31.0. It is used by Firebase lib to communicate with Google APIs.
I tried to increase the Command Timeout setting to 20 min and more but nothing changed. EB deployment process consumes the whole time interval and eventually fails. I tried to get into the EC2 instance itself to find anything useful that I can tweak but found nothing.
This is a sample of the eb-engine.log that, I believe, explains the error part.
Stored in directory: /root/.cache/pip/wheels/6f/8e/a7/c5ce8f1742b7d2b9e07cd1064cf201293d157c3c4e1021a74a
Running setup.py bdist_wheel for psutil: started
Running setup.py bdist_wheel for psutil: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/39/a0/f5/c4fa280463e29aea07797acb5312358fefb067c1f4f98e11b1
Running setup.py bdist_wheel for Pyphen: started
Running setup.py bdist_wheel for Pyphen: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/78/e3/76/42853bf2f7573a72fb5fe377a8406c979a5479ad6930506cde
Running setup.py bdist_wheel for cairocffi: started
Running setup.py bdist_wheel for cairocffi: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/3a/2e/92/89e9dcce600a1a78bebe571fdd096669cba354b29c5e1140b9
Running setup.py bdist_wheel for CairoSVG: started
Running setup.py bdist_wheel for CairoSVG: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/b0/f0/12/0c41649d33dbc95ce0188188feefc8f90aa877363322fd3ace
Running setup.py bdist_wheel for gevent: started
Running setup.py bdist_wheel for gevent: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/66/89/6a/7438b05445edc27023a9d8935801d4e70ed962e02541e57fbd
Running setup.py bdist_wheel for httplib2: started
Running setup.py bdist_wheel for httplib2: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/9a/a8/c3/72684de9397a5633e21ba89911174fbc9eb0b1fb9005c6eaf0
Running setup.py bdist_wheel for grpcio: started
Running setup.py bdist_wheel for grpcio: still running...
Running setup.py bdist_wheel for grpcio: still running...
Running setup.py bdist_wheel for grpcio: still running...
2020/11/05 16:12:14.123278 [ERROR] An error occurred during execution of command [app-deploy] - [Docker Specific Build Application]. Stop running the command. Error: failed to build docker image: Command /bin/sh -c docker build -t aws_beanstalk/staging-app /var/app/staging/ failed with error Command timed out after 300 seconds
2020/11/05 16:12:14.123302 [INFO] Executing cleanup logic
2020/11/05 16:12:14.131856 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed to build the Docker image. The deployment failed.","timestamp":1604592734,"severity":"ERROR"},{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1604592734,"severity":"ERROR"}]}]}
2020/11/05 16:12:14.133315 [INFO] Platform Engine finished execution on command: app-deploy
As you will notice in the previous sample that the Docker build process fails after exactly 300 seconds regardless of the Command Timeout setting value.
Stop running the command. Error: failed to build docker image: Command /bin/sh -c docker build -t aws_beanstalk/staging-app /var/app/staging/ failed with error Command timed out after 300 seconds

make.service fails when deploying Go project file to AWS Elastic Beanstalk

overflowers,
I've been struggling for hours on end trying to publish a Go application to AWS Elastic Beanstalk.
Have followed helpful tutorials online for what I first thought was dead simple.
One of the tutorials i followed
Another one of the tutorials i followed
An examples which actually compiles on AWS for me
My project structure is now:
The following files are all in my root folder:
application.go
build.sh
Buildfile
Procfile
application.go
package main
import (
"github.com/gin-gonic/gin"
)
func main() {
router := gin.Default()
router.GET("/user/:name", func(c *gin.Context) {
name := c.Param("name")
c.String(http.StatusOK, "Hello %s", name)
})
router.Run(":5000")
}
build.sh
go get "github.com/gin-gonic/gin" //Have tried without quotation marks
go build application.go
Buildfile
make: ./build.sh
Procfile
web: bin/application
I have also read the documentation on AWS website. And from what I can tell- my code, and the code in the demos referred to above, is correct.
I compress the enire contents of my root folder to a .zip file and upload to AWS through the web portal.
It fails...and in the error logs i can read:
2020/08/28 23:17:38.902079 [INFO] Executing instruction: RunAppDeployPreBuildHooks
2020/08/28 23:17:38.902095 [INFO] The dir .platform/hooks/prebuild/ does not exist in the application. Skipping this step...
2020/08/28 23:17:38.902100 [INFO] Executing instruction: Golang Specific Build Application
2020/08/28 23:17:38.902104 [INFO] checking Buildfile...
2020/08/28 23:17:38.902110 [INFO] building Golang application with Buildfile
2020/08/28 23:17:38.902148 [INFO] Running command /bin/sh -c systemctl show -p PartOf make.service
2020/08/28 23:17:38.908345 [WARN] Warning: process make is already registered...
Deregistering the process ...
2020/08/28 23:17:38.908368 [INFO] Running command /bin/sh -c systemctl show -p PartOf make.service
2020/08/28 23:17:38.913626 [INFO] Running command /bin/sh -c systemctl is-active make.service
2020/08/28 23:17:38.916919 [INFO] Running command /bin/sh -c systemctl disable make.service
2020/08/28 23:17:39.002192 [INFO] Running command /bin/sh -c systemctl daemon-reload
2020/08/28 23:17:39.086778 [INFO] Running command /bin/sh -c systemctl reset-failed
2020/08/28 23:17:39.092606 [INFO] Running command /bin/sh -c systemctl daemon-reload
2020/08/28 23:17:39.166648 [INFO] Running command /bin/sh -c systemctl reset-failed
2020/08/28 23:17:39.170861 [INFO] Running command /bin/sh -c systemctl is-enabled eb-app.target
2020/08/28 23:17:39.174735 [INFO] Running command /bin/sh -c systemctl enable eb-app.target
2020/08/28 23:17:39.251327 [INFO] Running command /bin/sh -c systemctl start eb-app.target
2020/08/28 23:17:39.256334 [INFO] Running command /bin/sh -c systemctl enable make.service
2020/08/28 23:17:39.338502 [INFO] Running command /bin/sh -c systemctl show -p PartOf make.service
2020/08/28 23:17:39.344691 [INFO] Running command /bin/sh -c systemctl is-active make.service
2020/08/28 23:17:39.348280 [INFO] Running command /bin/sh -c systemctl start make.service
2020/08/28 23:17:39.358198 [ERROR] startProcess Failure: starting process "make" failed: Command /bin/sh -c systemctl start make.service failed with error exit status 1. Stderr:Job for make.service failed because the control process exited with error code. See "systemctl status make.service" and "journalctl -xe" for details.
2020/08/28 23:17:39.358222 [ERROR] An error occurred during execution of command [app-deploy] - [Golang Specific Build Application]. Stop running the command. Error: build application failed on command ./build.sh with error: startProcess Failure: starting process "make" failed: Command /bin/sh -c systemctl start make.service failed with error exit status 1. Stderr:Job for make.service failed because the control process exited with error code. See "systemctl status make.service" and "journalctl -xe" for details.
Unfortunately I don't have much experience of this kind of server configuration or any other deployment operations. So I do not understand "process make is already registered..."
...Perhaps someone more experienced in these areas might know what to make of this?
Cheers!
NOTE: The project has been able to compile and run on my computer all the time. It just fails when i try deploying it on the Elastic Beanstalk.
EDIT: Have now also tried deploying with the AWS Command Line Interface. In this case the logfile eb-activity tells me:
Executing: HOME=/tmp /opt/elasticbeanstalk/lib/ruby/bin/ruby /opt/elasticbeanstalk/lib/ruby/bin/foreman start --procfile /tmp/d20200829-3130-13cz1sh/eb-buildtask-0 --root /var/app/staging --env /var/elasticbeanstalk/staging/elasticbeanstalk.env
/opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.2.0/gems/foreman-0.78.0/lib/foreman/process.rb:54:in `spawn': Permission denied - ./build.sh (Errno::EACCES)
from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.2.0/gems/foreman-0.78.0/lib/foreman/process.rb:54:in `block in run'
This may occur when the build.sh script lacks executable permissions. When that's the case "make.service" on beanstalk can't execute the script and an error is generated.
If you are developing on a unix machine try to chmod +x build.sh before deployment.
If you are developing on a windows machine you can't set the unix file attribute. But you can use this trick with Buildfile and deploy this way.
Buildfile
make: chmod +x build.sh; ./build.sh
This will make build.sh executable when the make service runs.
I also encountered this issue when migrating the deployment of my Golang application to the Amazon Linux 2 platform from Amazon Linux 1 platform. I was experiencing a similar error in the build process:
2021/07/10 16:33:21.411538 [ERROR] An error occurred during execution of command [app-deploy] - [Golang Specific Build Application]. Stop running the command. Error: build application failed on command ./build.sh with error: startProcess Failure: starting process "make" failed: Command /bin/sh -c systemctl start make.service failed with error exit status 1. Stderr:Job for make.service failed because the control process exited with error code. See "systemctl status make.service" and "journalctl -xe" for details.
After looking in to the systemctl status, as suggested in the error, I figured out the linux user "webapp" did not seem to have the correct permissions.
I added "webapp" to the sudoers file and now that build process finishes, and the deploy is able to succeed.
# User rules for webapp user
webapp ALL=(ALL) NOPASSWD:ALL