Elastic Beanstalk is infinitely copying a file to the /tmp folder that I created with a config file in .ebextensions. The name of this file is /tmp/mount-efs.sh. This file causes an issue on initialisation of an environment. So I try to get rid of it or at least change the content of it.
I tried already:
deploy an older version, that is not having this file.
Result: The ec2 instance not get deleted, so the file is still there
Upload the zip instead of using the application version
Result: The ec2 instance not get deleted, so the file is still there
delete the file from /tmp/mount-efs.sh
Result: The file immediatly reappears again and its ".bak" file too
Removed the '.config' file from /var/app/staging/.ebextensions/
Result: Same error and the file mount-efs.sh is still created in /tmp folder
I think Elastik Beanstalk is stuck with a version that it thinks works. But the version has an issue. And EB does not allow me to deploy a different version (older or newer).
The stranger thing is, that the version, that EB every time fallback to, did not have the file in the .ebextensions.
I also tried to rebuild the environment.
Result: Fallback is loaded, file is there, issue happens.
from eb-engine.log:
Running command /bin/sh -c /opt/aws/bin/cfn-init -s arn:aws:cloudformation:us-west-2:xxxxxxxxxxxx:stack/awseb-e-xxxxxxxxxxx-stack/nnnnnnnn-nnnn-nnnn-nnnn-xxxxxxxxxxxx -r AWSEBAutoScalingGroup --region us-west-2 --configsets Infra-EmbeddedPreBuild
2022/07/14 20:31:13.403626 [INFO] Error occurred during build: Command 01_mount failed
2022/07/14 20:31:13.403667 [ERROR] An error occurred during execution of command [self-startup] - [PreBuildEbExtension]. Stop running the command. Error: EbExtension build failed. Please refer to /var/log/cfn-init.log for more details.
This error happens every 5 sec. So EB is in an infinite loop here.
So I want to get rid of the /tmp/mount-efs.sh file, or that the content of /tmp/mount-efs.sh is different. I want to do this directly via ssh on the ec2 instance it self.
So my understanding is, that EB runs the config files that I added in .ebextensions. In this files there are files created in the /tmp folder. This files in the /tmp folder run on initialization.
So what file I have to change, so that the changes are recognized in the file, that is created in the /tmp folder (without deployment)?
Or can I stop the initialization loop somehow?
The infinity loop happens because of a command that calls a file in /var/www/html that did not exist. Why this file did not exist is a riddle for me. The whole /var/www/html folder was empty. Normally elastic beanstalk should do the stuff before running the commands, but this is not the case. (create app folder and staging, unzip the source code into staging, copy it into the app/current folder, and create a symlink to the app/current folder)
I was able to solve the issue with the infinity loop by doing the following:
sudo mkdir -p /var/app/staging
cd $_
sudo unzip /opt/elasticbeanstalk/deployment/app_source_bundle
sudo cp -rpv /var/app/staging /var/app/current
sudo rm -rf /var/www/html
sudo ln -s /var/app/current /var/www/html
mkdir -p: creates the directories with parent. so if "app" not exists it will be created before "staging" will be created
$_: Reference to the last folder "in action". here this was /var/app/staging
unzip: unzip the source bundle code into staging
cp -rp: copy recursively (r) and keep ownership and timestamps (p) from "staging" into "current"
rm -rf /var/www/html: deletes the existing HTML folder. Be careful with this command what you delete!
ln -s : creates a symbolic link from /var/www/html to /var/app/current
Related
When deploying an Elastic Beanstalk application, one of my hooks fails with "permission denied". I get the following in /var/log/eb-engine.log:
[INFO] Running platform hook: .platform/hooks/predeploy/collectstatic.sh
[ERROR] An error occurred during execution of command [app-deploy] - [RunAppDeployPreDeployHooks]. Stop running the command. Error: Command .platform/hooks/predeploy/predeploy.sh failed with error fork/exec .platform/hooks/predeploy/predeploy.sh: permission denied
How do I fix this?
According to the docs, Platform hooks need to be executable. Of note, this means they need to be executable according to git, because that's what Elastic Beanstalk uses to deploy.
You can check if they are executable via git ls-files -s .platform; you should see 100755 before any shell files in the output of this command. If you see 100644 before any of your shell files, run git add --chmod=+x -- .platform/*/*/*.sh to make them executable.
Create a file under .ebextensions folder with the right order and name it something like: 001_chmod.config
# This command finds all the files within hooks folder with extension .sh and makes them executable.
container_commands:
01_chmod1:
command: find .platform/hooks/ -type f -iname "*.sh" -exec chmod +x {} \;
Source: https://www.barot.us/running-sidekiq-on-amazon-linux-2/
I have a file that I can't edit but needs to run on in a docker container. Because the file doesn't have an extension, I have to use chmod for setting the file executable. But after I build the docker image from the docker file I always get a "permission denied" error
My docker file:
FROM alpine
COPY . /home/guestuser/bin/gateway
RUN apk add libressl-dev
RUN apk add libffi-dev
RUN pwd
WORKDIR /home/guestuser/bin/.
RUN ["chmod", "+x", "gateway"]
RUN pwd
CMD ["/home/guestuser/bin/gateway"]
EXPOSE 11878
I alwas get this error:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/home/guestuser/bin/gateway\": permission denied": unknown.
As I already mentioned, I am not able to edit the file I want to execute. What am I doing wrong?
You may try this simple one.
FROM alpine
COPY . /home/guestuser/bin/gateway
RUN apk add libressl-dev
RUN apk add libffi-dev
WORKDIR /home/guestuser/bin/
RUN chmod -R 755 /home/guestuser
CMD ["/bin/bash", "/home/guestuser/bin/gateway"]
Otherwise, run sleep command login to container and see your commands works manually
It looks like you are using the exec form of CMD, as shown here
There are two ways to use CMD. The first is the way you are already doing it, in exec form:
CMD ["/home/guestuser/bin/gateway"]
Or you could use shell form:
CMD /home/guestuser/bin/gateway
If you need a shell you could also explicitly call one in exec form, which is what Ganesh was trying to suggest.
CMD ["sh", "/home/guestuser/bin/gateway"]
But if that syntax is correct, why didn't it work?
Well, because this is assuming that gateway is a file. The issue is... it probably isn't.
When you run this command:
COPY . /home/guestuser/bin/gateway
From the reference:
Multiple resources may be specified but the paths of files and directories will be interpreted as relative to the source of the context of the build.
You are copying the entire contents of the build context into the directory /home/guestuser/bin/gateway. If you want to copy a specific file, you should name it explicitly rather than using . The COPY command's syntax is source first, then destination, as shown here.
So when you are trying to execute gateway, you are probably "executing" a directory named gateway. So long as there is more than one file in the build context, gateway will be a directory. That can include the Dockerfile itself, so even if the build context is a folder with just the Dockerfile and the script you want to run, you'll still pull in both files, which turns gateway itself into a directory.
Tests you can try
As proof that your Dockerfile CMD syntax is correct, try changing that CMD to something like this:
CMD ["top"]
Similarly, you can remove the CMD and just run the container in interactive mode. It will drop you in your WORKDIR, which is empty except for the gateway directory, complete with the contents of whatever directory structure was pulled in during the build process.
So, to make this work, change your COPY line to name the script you want:
COPY somescript /home/guestuser/bin/gateway
Other notes:
your default user here is root, so you don't need to chmod gateway
RUN pwd will only show the first time you build the container
I have .env in my code. I copy it to s3. I want to delete it from my GitHub and beanstalk download it when it starts. which directory I should use?
I see my code is on
/var/app/current/
/var/www/html/
,...
I want to use .ebextensions
commands:
01_get_env_vars:
command: aws s3 cp s3://test/.env DIRECTORY
does it have a better solution?
Probably the best way would be to use container_commands instead of commands.
The reason is that conatiner_commands run in the staging folder /var/app/staging:
The specified commands run as the root user, and are processed in alphabetical order by name. Container commands are run from the staging directory, where your source code is extracted prior to being deployed to the application server.
Thus your code could be:
container_commands:
01_get_env_vars:
command: aws s3 cp s3://test/.env .
where DIRECTORY is replaced by ..
I have a problem downloading entire folder in GCP. How should I download the whole bucket? I run this code in GCP Shell Environment:
gsutil -m cp -R gs://my-uniquename-bucket ./C:\Users\Myname\Desktop\Bucket
and I get an error message: "CommandException: Destination URL must name a directory, bucket, or bucket subdirectory for the multiple source form of the cp command. CommandException: 7 files/objects could not be transferred."
Could someone please point out the mistake in the code line?
To download an entire bucket You must install google cloud SDK
then run this command
gsutil -m cp -R gs://project-bucket-name path/to/local
where path/to/local is your path of local storage of your machine
The error lies within the destination URL as specified by the error message.
I run this code in GCP Shell Environment
Remember that you are running the command from the Cloud Shell and not in a local terminal or Windows Command Line. Thus, it is throwing that error because it cannot find the path you specified. If you inspect the Cloud Shell's file system/structure, it resembles more that of a Unix environment in which you can specify the destination like such instead: ~/bucketfiles/. Even a simple gsutil -m cp -R gs://bucket-name.appspot.com ./ will work since Cloud Shell can identify the ./ directory which is the current directory.
A workaround to this issue is to perform the command on your Windows Command Line. You would have to install Google Cloud SDK beforehand.
Alternatively, this can also be done in Cloud Shell, albeit with an extra step:
Download the bucket objects by running gsutil -m cp -R gs://bucket-name ~/ which will download it into the home directory in Cloud Shell
Transfer the files downloaded in the ~/ (home) directory from Cloud Shell to the local machine either through the User Interface or by running gcloud alpha cloud-shell scp
Your destination path is invalid:
./C:\Users\Myname\Desktop\Bucket
Change to:
/Users/Myname/Desktop/Bucket
C: is a reserved device name. You cannot specify reserved device names in a relative path. ./C: is not valid.
There is not a one-button solution for downloading a full bucket to your local machine through the Cloud Shell.
The best option for an environment like yours (only using the Cloud Shell interface, without gcloud installed on your local system), is to follow a series of steps:
Downloading the whole bucket on the Cloud Shell environment
Zip the contents of the bucket
Upload the zipped file
Download the file through the browser
Clean up:
Delete the local files (local in the context of the Cloud Shell)
Delete the zipped bucket file
Unzip the bucket locally
This has the advantage of only having to download a single file on your local machine.
This might seem a lot of steps for a non-developer, but it's actually pretty simple:
First, run this on the Cloud Shell:
mkdir /tmp/bucket-contents/
gsutil -m cp -R gs://my-uniquename-bucket /tmp/bucket-contents/
pushd /tmp/bucket-contents/
zip -r /tmp/zipped-bucket.zip .
popd
gsutil cp /tmp/zipped-bucket.zip gs://my-uniquename-bucket/zipped-bucket.zip
Then, download the zipped file through this link: https://storage.cloud.google.com/my-uniquename-bucket/zipped-bucket.zip
Finally, clean up:
rm -rf /tmp/bucket-contents
rm /tmp/zipped-bucket.zip
gsutil rm gs://my-uniquename-bucket/zipped-bucket.zip
After these steps, you'll have a zipped-bucket.zip file in your local system that you can unzip with the tool of your choice.
Note that this might not work if you have too much data in your bucket and the Cloud Shell environment can't store all the data, but you could repeat the same steps on folders instead of buckets to have a manageable size.
I've linked a git branch to my Elastic Beanstalk environment and using git aws.push it deploys correctly.
I've now added a .extensions directory which contains a config script which should be creating a couple of directories. However, nothing appears to be happening.
I understand that the .extensions directory should be copied across to the ec2 instance as well but I'm not seeing it.
I've checked eb-tools.log and it's not mentioned in the upload.
Is there something additional that's required?
The script contains:
commands:
cache:
command: mkdir /tmp/cache
items:
command: mkdir /tmp/cache/items
chmod:
command: chmod -R 644 /tmp
You can find the run logs for this at /var/log/cfn-init.log.
In here I could see that the mkdir commands had worked initially but subsequently failed as the directory already existed.
Turns out that eb extensions run commands in alphabetical order so I had to change the commands to:
01command1:
02command2:
etc.
From this point on it worked fine.
Something else that was confusing me is that the .ebextensions directory in my local git repo was not appearing on the target instance directory. this is because once it's been run it will delete the directory.
Double check that your local script file has a .config extension. I was having a similar problem because my local file was called .ebextensions/01_stuff.yaml and it was fixed once I renamed it to .ebextensions/01_stuff.config.