I am running the following script. (intentionally i hide the keys of course).
It is basically a copy paste from the readme.md.
Enviroment details:
- I have windows 10.
- running this script on git bash enviroment.
- docker version is 18.03.1-ce
docker container run \
--env AWS_ACCESS_KEY_ID=aaaaaaa \
--env AWS_SECRET_ACCESS_KEY=bbbbbbb \
-v $PWD:/data \
garland/aws-cli-docker \
aws s3 sync . s3://www.typing-coacher.net
i am getting the following error:
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: Mount denied:
The source path "C:/projects/docker;C"
doesn't exist and is not known to Docker.
See 'C:\Program Files\Docker\Docker\Resources\bin\docker.exe run --help'.
the folder path that actually exist is: C:/projects/docker
Your Git Bash environment will evaluate $PWD to /c/projects/docker instead of C:\projects\docker. Docker daemon will not be able to find that path.
Walkarounds:
Use Winodows shell or PowerShell.
Use absolute path instead of $PWD.
Related
Elastic Beanstalk is infinitely copying a file to the /tmp folder that I created with a config file in .ebextensions. The name of this file is /tmp/mount-efs.sh. This file causes an issue on initialisation of an environment. So I try to get rid of it or at least change the content of it.
I tried already:
deploy an older version, that is not having this file.
Result: The ec2 instance not get deleted, so the file is still there
Upload the zip instead of using the application version
Result: The ec2 instance not get deleted, so the file is still there
delete the file from /tmp/mount-efs.sh
Result: The file immediatly reappears again and its ".bak" file too
Removed the '.config' file from /var/app/staging/.ebextensions/
Result: Same error and the file mount-efs.sh is still created in /tmp folder
I think Elastik Beanstalk is stuck with a version that it thinks works. But the version has an issue. And EB does not allow me to deploy a different version (older or newer).
The stranger thing is, that the version, that EB every time fallback to, did not have the file in the .ebextensions.
I also tried to rebuild the environment.
Result: Fallback is loaded, file is there, issue happens.
from eb-engine.log:
Running command /bin/sh -c /opt/aws/bin/cfn-init -s arn:aws:cloudformation:us-west-2:xxxxxxxxxxxx:stack/awseb-e-xxxxxxxxxxx-stack/nnnnnnnn-nnnn-nnnn-nnnn-xxxxxxxxxxxx -r AWSEBAutoScalingGroup --region us-west-2 --configsets Infra-EmbeddedPreBuild
2022/07/14 20:31:13.403626 [INFO] Error occurred during build: Command 01_mount failed
2022/07/14 20:31:13.403667 [ERROR] An error occurred during execution of command [self-startup] - [PreBuildEbExtension]. Stop running the command. Error: EbExtension build failed. Please refer to /var/log/cfn-init.log for more details.
This error happens every 5 sec. So EB is in an infinite loop here.
So I want to get rid of the /tmp/mount-efs.sh file, or that the content of /tmp/mount-efs.sh is different. I want to do this directly via ssh on the ec2 instance it self.
So my understanding is, that EB runs the config files that I added in .ebextensions. In this files there are files created in the /tmp folder. This files in the /tmp folder run on initialization.
So what file I have to change, so that the changes are recognized in the file, that is created in the /tmp folder (without deployment)?
Or can I stop the initialization loop somehow?
The infinity loop happens because of a command that calls a file in /var/www/html that did not exist. Why this file did not exist is a riddle for me. The whole /var/www/html folder was empty. Normally elastic beanstalk should do the stuff before running the commands, but this is not the case. (create app folder and staging, unzip the source code into staging, copy it into the app/current folder, and create a symlink to the app/current folder)
I was able to solve the issue with the infinity loop by doing the following:
sudo mkdir -p /var/app/staging
cd $_
sudo unzip /opt/elasticbeanstalk/deployment/app_source_bundle
sudo cp -rpv /var/app/staging /var/app/current
sudo rm -rf /var/www/html
sudo ln -s /var/app/current /var/www/html
mkdir -p: creates the directories with parent. so if "app" not exists it will be created before "staging" will be created
$_: Reference to the last folder "in action". here this was /var/app/staging
unzip: unzip the source bundle code into staging
cp -rp: copy recursively (r) and keep ownership and timestamps (p) from "staging" into "current"
rm -rf /var/www/html: deletes the existing HTML folder. Be careful with this command what you delete!
ln -s : creates a symbolic link from /var/www/html to /var/app/current
When deploying an Elastic Beanstalk application, one of my hooks fails with "permission denied". I get the following in /var/log/eb-engine.log:
[INFO] Running platform hook: .platform/hooks/predeploy/collectstatic.sh
[ERROR] An error occurred during execution of command [app-deploy] - [RunAppDeployPreDeployHooks]. Stop running the command. Error: Command .platform/hooks/predeploy/predeploy.sh failed with error fork/exec .platform/hooks/predeploy/predeploy.sh: permission denied
How do I fix this?
According to the docs, Platform hooks need to be executable. Of note, this means they need to be executable according to git, because that's what Elastic Beanstalk uses to deploy.
You can check if they are executable via git ls-files -s .platform; you should see 100755 before any shell files in the output of this command. If you see 100644 before any of your shell files, run git add --chmod=+x -- .platform/*/*/*.sh to make them executable.
Create a file under .ebextensions folder with the right order and name it something like: 001_chmod.config
# This command finds all the files within hooks folder with extension .sh and makes them executable.
container_commands:
01_chmod1:
command: find .platform/hooks/ -type f -iname "*.sh" -exec chmod +x {} \;
Source: https://www.barot.us/running-sidekiq-on-amazon-linux-2/
I am working on a project, where I need to read Text File from S3 bucket using Boto3. Then need to dockerize my application. I Implemented my code using Boto3, its running Perfectly fine(Note: Taking two arguments through argparser with switches -p and -n). But when I try the same using Docker run
PS **my Working Directory** > docker run --rm Image_name -p Argument1 -n Argument2
<class 'botocore.exceptions.ProfileNotFound'> Code.py 92
I searched lot of things on it, my understanding is AWS container directory is unable to locate my Credential File and config file stored in my home directory/.aws folder.
What I Tried:
1. Path mounting as below:
PS **my Working Directory** > docker run --rm -it -v %userprofile%\.aws:/root/.aws
amazon/aws-cli
docker: Error response from daemon: %!u(string=is not a valid Windows path)serprofile%!\
(MISSING).aws.
See 'docker run --help'.
I completely don't understand what's wrong with the syntax. I tried manually inputting my user profile directory in the %userprofile% as C:/Users/Deepak
Then Strangely WSL2(backend) popup comes saying using passing containers on windows may poorly work.
I am not sure what it means. Does it have any effect on Docker containers build on windows environment?
2. I moved my Credential and Config file in my working directory as well and tried below code:
PS my Working Directory > docker run --rm -it -v ${PWD}:/root/.aws amazon/aws-cli Image_name -p
Argument1 -n Argument2
In Build Step, I've added Send files or execute command over SSh -> SSH Publishers -> Exec command, I'm trying to run aws command to copy file from ec2 to s3. The same command runs fine when I execute it over the terminal, but via jenkins it simply returns:
bash: aws: command not found
The command is
cd ~/.local/bin/ && aws s3 cp /home/ec2-user/lambda_test/lambda_function.zip s3://temp-airflow-us/lambda_function.zip
Based on the comments.
The solution was to use the following command:
cd ~/.local/bin/ && ./aws s3 cp /home/ec2-user/lambda_test/lambda_function.zip s3://temp-airflow-us/lambda_function.zip
since aws is not available in PATH env variable.
command not found indicates that the aws utility is not on $PATH for the jenkins user.
To confirm, sudo su -l jenkins and then issue the command which aws - this will most likely return no results.
You have two options:
use the full path (likely /usr/local/bin/aws)
add /usr/local/bin to the jenkins user's $PATH
I need my Makefile to work in both Linux and Windows so the accepted answer is not an option for me.
I diagnosed the problem by adding the following to the top of my build script:
whoami
which aws
env|grep PATH
This returned:
root
which: no aws in (/sbin:/bin:/usr/sbin:/usr/bin)
PATH=/sbin:/bin:/usr/sbin:/usr/bin
Bizarrely, the path does not include /usr/local/bin, even though the interactive shell on the Jenkins host includes it. The fix is simple enough, create a symlink on the Jenkins host:
ln -s /usr/local/bin/aws /bin/aws
Now the aws command can be found by scripts running in Jenkins (in /bin).
So I am using cloudformation for my AWS setup, I am trying to run composer but for some reason no matter what command I put in my userdata section I always can an error, this is my error:
php /usr/local/bin/composer.phar create-project composer/satis /var/www/satis --stability=dev
[RuntimeException]
The HOME or COMPOSER_HOME environment variable must be set for composer to run correctly
This is my code within the userdata section:
"#composer\n",
"curl -sS https://getcomposer.org/installer | php\n",
"mv composer.phar /usr/local/bin/composer.phar\n",
"#satis\n",
"php /usr/local/bin/composer.phar create-project composer/satis /var/www/satis --stability=dev\n",
Does anyone have any ideas why this might not work and should I should be doing ?
Composer is looking for the location of the .composer directory. Export the HOME or COMPOSER_HOME env variable, e.g. : HOME=/root php /usr/local/bin/composer.phar create-project composer/satis /var/www/satis --stability=dev, it will work fine then.
I had the similar issue with amazon linux ami 2, it was showing in the log All settings correct for using Composer. The HOME or COMPOSER_HOME environment variable must be set for composer to run correctly, but it was not installed at all. Below is the way to fix it. Might be helpful to somebody rather waisting 2,3 hours!
sudo curl -sS https://getcomposer.org/installer | sudo php
mv composer.phar /usr/bin/composer
chmod +x /usr/bin/composer
export COMPOSER_HOME=/root
Agree with Ntwobike's answer.
When launching AWS EC2 instances I was installing composer by running an Ansible playbook during in the user data script run. (The user data script is called by cloud-init during the instance build process).
For some reason at this point in the build the $HOME environment variable is not set. So I needed to add 'export HOME=/root' - e.g.
# These need to be set to enable the composer installer to run. It is probably due to an issue
# with the $HOME variable not yet being set at this point in the instance creation.
export HOME=/root
ansible-playbook --extra-vars "target=localhost" playbooks/debian-9/drush.yml