In the newly created google container builder I am unable to fetch git tags during a build. During the build process the default cloning does not seem to fetch git tags. I added a custom build process which calls git fetch --tags but this results in the error:
Fetching origin
git: 'credential-gcloud.sh' is not a git command. See 'git --help'.
fatal: could not read Username for 'https://source.developers.google.com': No such device or address
# cloudbuild.yaml
#!/bin/bash
openssl aes-256-cbc -k "$ENC_TOKEN" -in gcr_env_vars.sh.enc -out gcr_env_vars.sh -
source gcr_env_vars.sh
env
git config --global url.https://${CI_USER_TOKEN}#github.com/.insteadOf git#github.com:
pushd vendor
git submodule update --init --recursive
popd
docker build -t gcr.io/project-compute/continuous-deploy/project-ui:$COMMIT_SHA -f /workspace/installer/docker/ui/Dockerfile .
docker build -t gcr.io/project-compute/continuous-deploy/project-auth:$COMMIT_SHA -f /workspace/installer/docker/auth/Dockerfile .
This worked for me, as the first build step:
- name: gcr.io/cloud-builders/git
args: [fetch, --depth=100]
To be clear, you want all tags to be available in the Git repo, not just to trigger on tag changes? In the latter, the triggering tag should be available IIUC.
I'll defer to someone on the Container Builder team for a more detailed explanation, but that error tells me that they used gcloud to clone the Google Cloud Source Repository (GCSR), which configures a Git credential helper named as such. They likely did this in another container before invoking yours, or on the host. Since gcloud and/or the gcloud credential helper aren't available in your container, you can't authenticate properly with GCSR.
You can learn a bit more about the credential helper here.
Related
I am using an ebs app on linux 2 platforms, and I need to clone a directory during deployment to get configfiles for my app.
I did a predeploy hook so that the files are there when the app starts after deployment: /.platform/hooks/predeploy/01_import
After deployment in a predeploy hook, the files are not there.
When I run the exact same script in a postdeploy hook, the files are there.
So the command works, I see the predeploy hook is running (I see the echo text in the log), but the files are not present. Anyone knows why?
#!/bin/bash
mkdir /var/app/current/config
echo Adding github in known hosts
ssh-keyscan -H github.com >> /home/webapp/.ssh/known_hosts
echo Done Adding github in known hosts
echo deleting old flows
echo cloning
git -c core.sshCommand="ssh -i /etc/pki/tls/certs/githubKey" clone -b dev --single-branch <mygithub> /var/app/current/config
echo done cloning
In predeploy stage, the new code is deployed to /var/app/staging, not /var/app/current.
/var/app/current is actually overwritten by staging if the new staging deployment is successful.
So in predeploy, I've cloned to staging instead, and it works.
This is not well documented in AWS docs; this helped me.
Trying to use AWS Amplify to deploy a multi-repo dendron wiki which has a mix of public and private github repositories.
Amplify can be associated with a single repo but there doesn't seem to be a built-in way to pull in additional private repositories.
Create a custom deploy key for the private repo in github
generate the key
ssh-keygen -f deploy_key -N ""
Encode the deploy key as a base64 encoded env variable for amplitude
cat deploy_key | base64 | tr -d \\n
add this as a hosting environment variable (eg. DEPLOY_KEY)
Modify the amplify.yml file to make use of the deploy key
there's 2 key steps
adding deploy key to ssh-agent
WARNING: this implementation will print the $DEPLOY_KEY to stdout
disabling StrictHostKeyChecking
NOTE: amplify does not have a $HOME/.ssh folder by default so you'll need to create one as part of the deployment process
relevant excerpt below
- ...
- eval "$(ssh-agent -s)"
- ssh-add <(echo "$DEPLOY_KEY" | base64 -d)
- echo "disable strict host key check"
- mkdir ~/.ssh
- touch ~/.ssh/config
- 'echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
- ...
full build file here
Now you should be able to use git to clone the private repo.
For a more detailed writeup as well as alternatives and gotchas, see here
For GitLab repositories:
Create a deploy token in GitLab
Set env variables (eg. DEPLOY_TOKEN_USERNAME, DEPLOY_TONE_PASSWORD) in amplify panel
Add this line to the amplify config
- git config --global url."https://$DEPLOY_TOKEN_USERNAME:$DEPLOY_TOKEN_PASSWORD#gitlab.com/.../repo.git".insteadOf "https://gitlab.com/.../repo.git"
I have a Django project that I deploy on a server using CircleCI. The server is a basic cloud server, and I can SSH into it.
I set up the deployment section of my circle.yml file, and everything is working fine. I would like to automatically perform some actions on the server after the deployment (such as migrating the database or reloading gunicorn).
I there a way to do that with CircleCI? I looked in the docs but couldn't find anything related to this particular problem. I also tried to put ssh user#my_server_ip after my deployment step, but then I get stuck and cannot perform any action. I can successfully SSH in, but the rest of the commands is not called.
Here is what my ideal circle.yml file would look like:
deployment:
staging:
branch: develop
commands:
- rsync --update ./requirements.txt user#server:/home/user/requirements.txt
- rsync -r --update ./myapp/ user#server:/home/user/myapp/
- ssh user#server
- workon myapp_venv
- cd /home/user/
- pip install -r requirements.txt
I solved the problem by putting a post_deploy.sh file on the server, and putting this line on the circle.yml:
ssh -i ~/.ssh/id_myhost user#server 'post_deploy.sh'
It executes the instructions in the post_deploy.sh file, which is exactly what I wanted.
So I am using cloudformation for my AWS setup, I am trying to run composer but for some reason no matter what command I put in my userdata section I always can an error, this is my error:
php /usr/local/bin/composer.phar create-project composer/satis /var/www/satis --stability=dev
[RuntimeException]
The HOME or COMPOSER_HOME environment variable must be set for composer to run correctly
This is my code within the userdata section:
"#composer\n",
"curl -sS https://getcomposer.org/installer | php\n",
"mv composer.phar /usr/local/bin/composer.phar\n",
"#satis\n",
"php /usr/local/bin/composer.phar create-project composer/satis /var/www/satis --stability=dev\n",
Does anyone have any ideas why this might not work and should I should be doing ?
Composer is looking for the location of the .composer directory. Export the HOME or COMPOSER_HOME env variable, e.g. : HOME=/root php /usr/local/bin/composer.phar create-project composer/satis /var/www/satis --stability=dev, it will work fine then.
I had the similar issue with amazon linux ami 2, it was showing in the log All settings correct for using Composer. The HOME or COMPOSER_HOME environment variable must be set for composer to run correctly, but it was not installed at all. Below is the way to fix it. Might be helpful to somebody rather waisting 2,3 hours!
sudo curl -sS https://getcomposer.org/installer | sudo php
mv composer.phar /usr/bin/composer
chmod +x /usr/bin/composer
export COMPOSER_HOME=/root
Agree with Ntwobike's answer.
When launching AWS EC2 instances I was installing composer by running an Ansible playbook during in the user data script run. (The user data script is called by cloud-init during the instance build process).
For some reason at this point in the build the $HOME environment variable is not set. So I needed to add 'export HOME=/root' - e.g.
# These need to be set to enable the composer installer to run. It is probably due to an issue
# with the $HOME variable not yet being set at this point in the instance creation.
export HOME=/root
ansible-playbook --extra-vars "target=localhost" playbooks/debian-9/drush.yml
I have installed reviewboard. I added local GIT repository to it. During creation of review requests, the repository is available to be selected. But, when I am selecting a file from the repository and trying to add it as a Diff it says 'The selected file does not appear to be a diff.'. Please let me know if anyone has any answer for the question. Thanks....
git diff <filename1> ><filename2>.diff
This can be used for generating the diff file.
Some helpful tips for reviewboard are:
Log Settings:
Check/Tick - Enable logging
Log directory: /var/www/reviewboard
Log Level: Debug
Review board git configuration steps::
$ git config --global user.name "Chalpat Rauth"
$ git config --global user.email chalpat.rauth#ap.sony.com
You can veryify the entries in vim ~/.gitconfig
$ ssh-keygen -t rsa
copy the public key ./root/.ssh/id_rsa.pub to gitlab as a new key
chmod 700 -R /root/.ssh/
git clone git#gitlab.csx.sony.co.jp:testtest.git
During configuration in ReviewBoard:
Hosting service: None - Custom Repository
Repository Type: Git
Path: /var/www/reviewboard/code/testtest/.git
Note the below:
In path: /var/www/reviewboard/code/testtest/helloworld/src/test/java/com/sony/csx
git add <file_name>
git commit -m "This is second commit"
git push
git diff HEAD >DiffForReview
LDAP Settings::
Check/Tick - Allow anonymous read-only access
Authentication Method: LDAP
LDAP Server: ldap://ldap.csx.sony.co.jp
LDAP Base DN: dc=csx,dc=sony,dc=co,dc=jp
Surname Attribute: csxUsername1
Full Name Attribute: csxUsernameF
E-Mail LDAP Attribute: mail
User Mask: uid=%s