I am trying to bump a version automatically when deploying a project into a QA environment.
I have the following code
args:
- '-c'
- |
set -x
npm install
npm i -g #nrwl/cli
npm version --git-tag-version false --commit-hooks false $(TZ=UTC0 git show --quiet --date='format-local:%Y' --format="%cd").$(TZ=UTC0 git show --quiet --date='format-local:%m%d' --format="%cd").${BUILD_ID}-${SHORT_SHA}
nx run-many --target=build --all --configuration=qa-1 --parallel
nx run-many --target=deploy-qa-1 --all --configuration=qa-1 --token "$$FIREBASE_TOKEN" --parallel
This works, but I would like to shorten the BUILD_ID to only 8 digits and not the full strings.
I tried using ${BUILD_ID::8} ${$(echo BUILD_ID::8)} but what I get is a empty string.
How to achieve it ?
Currently this is not possible in Cloud Build, you can't shorten the value of substitutions by making a substring within the args of a build.
However, if you'd like to have this implemented into Cloud Build, you can open a Feature Request into Google's Issue tracker system, so that this can be considered by their product team.
This seems to work
substitutions:
_SHORT_BUILD: ${BUILD_ID:0:8}
then you can use ${_SHORT_BUILD}
Related
Trying to run a simple bash script on google cloud build. Trying to run it it says it cannot find it, even though ls shows it is there
I've set up a build trigger on google cloud to run a simple test repository on pushes to main branch
The test repository has just two files: the cloudbuild yaml and a simple testfile.sh bash script
Cloudbuild yaml tells it to run this testfile.sh file, but says it cannot find it even though a simple ls arg shows it
I've tried like every combination of ways to run a bash file:
with/without '-c' argument
with/without '.' argument
with/without file shebang
cloudbuild.yaml:
steps:
- name: 'ubuntu'
entrypoint: 'bash'
args: ['-c', 'testfile.sh']
testfile.sh:
echo "Go suck it, world!"
gcloud builds log <log-id>:
starting build "640c5ba5-5906-4296-a80c-9adc54ee84bb"
FETCHSOURCE
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /workspace/.git/
From https://source.developers.google.com/p/test-wtf-2734586432/r/test-files
* branch 1d6fc0b27c09cb3421a242764dfe28bc115bf8f5 -> FETCH_HEAD
HEAD is now at 1d6fc0b Fix typo in entrypoint
BUILD
Pulling image: ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
Digest: sha256:adf73ca014822ad8237623d388cedf4d5346aa72c270c5acc01431cc93e18e2d
Status: Downloaded newer image for ubuntu:latest
docker.io/library/ubuntu:latest
bash: testfile.sh: command not found
ERROR
ERROR: build step 0 "ubuntu" failed: step exited with non-zero status: 127
I fixed it
Had to get rid of the '-c' from the args list
I am trying to include the Container Analyis API link in a Cloud Build pipeline.This is a beta component and with command line I need to install it first:
gcloud components install beta local-extract
then I can run the on demand container analyis (if the container is present locally):
gcloud beta artifacts docker images scan ubuntu:latest
My question is how I can use component like beta local-extract within Cloud Build ?
I tried to do a fist step and install the missing componentL
## Update components
- name: 'gcr.io/cloud-builders/gcloud'
args: ['components', 'install', 'beta', 'local-extract', '-q']
id: Update component
but as soon as I move to the next step the update is gone (since it is not in the container)
I also tried to install the component and then run the scan using (& or ;) but it is failling:
## Run vulnerability scan
- name: 'gcr.io/cloud-builders/gcloud'
args: ['components', 'install', 'beta', 'local-extract', '-q', ';', 'gcloud', 'beta', 'artifacts', 'docker', 'images', 'scan', 'ubuntu:latest', '--location=europe']
id: Run vulnaribility scan
and I get:
Already have image (with digest): gcr.io/cloud-builders/gcloud
ERROR: (gcloud.components.install) unrecognized arguments:
;
gcloud
beta
artifacts
docker
images
scan
ubuntu:latest
--location=europe (did you mean '--project'?)
To search the help text of gcloud commands, run:
gcloud help -- SEARCH_TERMS
so my question are:
how can I run "gcloud beta artifacts docker images scan ubuntu:latest" within Cloud Build ?
bonus: from the previous command how can I get the "scan" output value that I will need to pass as a parameter to my next step ? (I guess it should be something with --format)
You should try the cloud-sdk docker image:
https://github.com/GoogleCloudPlatform/cloud-sdk-docker
The Cloud Build team (implicitly?) recommends it:
https://github.com/GoogleCloudPlatform/cloud-builders/tree/master/gcloud
With the cloud-sdk-docker container you can change the entrypoint to bash pipe gcloud commands together. Here is an (ugly) example:
https://github.com/GoogleCloudPlatform/functions-framework-cpp/blob/d3a40821ff0c7716bfc5d2ca1037bcce4750f2d6/ci/build-examples.yaml#L419-L432
As to your bonus question. Yes, --format=value(the.name.of.the.field) is probably what you want. The trick is to know the name of the field. I usually start with --format=json on my development workstation to figure out the name.
The problem comes from Cloud Build. It cache some often used images and if you want to use a brand new feature in GCLOUD CLI the cache can be too old.
I performed a test tonight, the version is 326 in cache. the 328 has just been released. So, the cached version has 2 weeks old, maybe too old for your feature. It could be worse in your region!
The solution to fix this, is to explicitly request the latest version.
Go to this url gcr.io/cloud-builders/gcloud
Copy the latest version
Paste the full version name in the step of your Cloud Build pipeline.
The side effect is a longer build. Indeed, because this latest image isn't cached, it has to be downloaded in Cloud Build.
I have a simple project but i have no experiences with GitLab-ci. I use ROS and cmake to build my project on my local machine(ubuntu18-04).
now I want to build my project on GitLab but this doesn't look easy for me.
Steps:
1-) installed binary runners from here
2-) registered runners from here for Linux
- used docker as an executor (like gitlabci; i have no experience with docker)
- selected ruby:2.6 default image
3-) Now' i can see my runner under Settings > CI/CD -> Runners
4-) Creating example .yml provided from gitlab
build-job:
stage: build
script:
- echo "Hello, $GITLAB_USER_LOGIN!"
test-job1:
stage: test
script:
- echo "This job tests something"
test-job2:
stage: test
script:
- echo "This job tests something, but takes more time than test-job1."
- echo "After the echo commands complete, it runs the sleep command for 20 seconds"
- echo "which simulates a test that runs 20 seconds longer than test-job1"
- sleep 20
deploy-prod:
stage: deploy
script:
- echo "This job deploys something from the $CI_COMMIT_BRANCH branch."
And its working!
But now I want to build my code on gitlab-ci. File structure:
scripts
->build.sh
src
->Cmakelist.txt
->codes.cpp
binaries
->outputs ll be here.
.gitlab-ci.yml
build.sh doing everything that I want:
...
mkdir build
cd build
cmake .. -GNinja
ninja
So I need just run it. But I don't know how to install prerequisites.
Which system exactly am I working on right now and how can I install prerequisites? (ubuntu 18.04 - docker - runner.. I just got mixed up )
Which system exactly am I working on right now
You clicked selected ruby:2.6 default image so you are on ruby:2.6. You may then browse docker hub : https://hub.docker.com/_/ruby and dockerfile https://github.com/docker-library/ruby/blob/8e49e25b591d4cfa6324b6dada4f16629a1e51ce/2.6/buster/Dockerfile - I see it has "buster" which is the name of one of debian releases, so I guess it's debian.
how can I install prerequisites?
That depends on the image your are using, different linux distributions use different package mangers. I usually take a look at wiki package manager.
You could just something like:
build:
image: ubuntu
script:
- apt-get install -y cmake gcc whatever-else-like-you-have-on-your-machine
- ./scripts/build.sh
I've created a GitHub repo for my simple custom recipe:
my-cookbook/
|- recipes/
|- appsetup.rb
I've added the repo to Custom Chef Recipes as https://github.com/my-github-user/my-github-repo.git
I've added my-cookbook::appsetup to the Setup "cycle".
I know it's executed, because it fails to load if I mess up the syntax.
This is my appsetup.rb:
node[:deploy].each do |app_name, deploy|
script "install_composer" do
interpreter "bash"
user "root"
cwd "#{deploy[:deploy_to]}/current"
code "curl -sS https://getcomposer.org/installer | php && php composer.phar install --no-dev"
end
end
When I log into the instance by SSH with the ubuntu user, composer isn't installed.
I've also tried the following to no avail (A nodejs install):
node[:deploy].each do |app_name, deploy|
execute "installing node" do
command "add-apt-repository --yes ppa:chris-lea/node.js && apt-get update && sudo apt-get install python-software-properties python g++ make nodejs"
end
end
Node doesn't get installed, and there are no errors in the log. The only references to the cookbook in the log just says:
[2014-03-31T13:26:04+00:00] INFO: OpsWorks Custom Run List: ["opsworks_initial_setup", "ssh_host_keys", "ssh_users", "mysql::client", "dependencies", "ebs", "opsworks_ganglia::client", "opsworks_stack_state_sync", "mod_php5_apache2", "my-cookbook::appsetup", "deploy::default", "deploy::php", "test_suite", "opsworks_cleanup"]
...
2014-03-31T13:26:04+00:00] INFO: New Run List expands to ["opsworks_initial_setup", "ssh_host_keys", "ssh_users", "mysql::client", "dependencies", "ebs", "opsworks_ganglia::client", "opsworks_stack_state_sync", "mod_php5_apache2", "my-cookbook::appsetup", "deploy::default", "deploy::php", "test_suite", "opsworks_cleanup"]
...
[2014-03-31T13:26:05+00:00] DEBUG: Loading Recipe my-cookbook::appsetup via include_recipe
[2014-03-31T13:26:05+00:00] DEBUG: Found recipe appsetup in cookbook my-cookbook
Am I missing some critical step somewhere? The recipe is clearly recognized and loaded, but doesn't seem to be executed.
(The following are fictitious names: my-github-user, my-github-repo, my-cookbook)
I know you've abandoned the cookbook but I'm almost 100% sure it's because you don't have a metadata.rb file in the root of your cookbook.
Your cookbook name should not contain a dash. I had the same problem, replacing by '_' solved it for me.
If those commands are failing silently, it could be that your use of && is obscuring a failure.
As for add-apt-repository, that is an interactive command. Try using the "--yes" option to answer yes by default, making it no longer interactive.
If you do not execute your command successfully, you will not find the files in the current directory. Check inside the last release folder to see if it had been put there.
It maybe prudent to check if you got the right directory etc setup by changing the CWD to : /tmp
How can one get the build_number (and other build metadata) from within the after_script command in Travis-CI?
What have been tried already:
The documentation on build configuration says this, in the IRC notification section:
You also have the possibility to customize the message that will be
sent to the channel(s) with a template:
notifications:
irc:
channels:
- "irc.freenode.org#travis"
- "irc.freenode.org#some-other-channel"
template:
- "%{repository} (%{commit}) : %{message} %{foo} "
- "Build details: %{build_url}"
You can interpolate the following variables:
repository: your GitHub repo URL.
build_number: build number.
branch: branch build name.
commit: shorten commit SHA
author: commit author name.
message: travis message to the build.
compare_url: commit change view URL.
build_url: URL of the build detail.
Trying to get this to work within an after_script command as below, did not work at all:
language: java
after_script:
- git commit -a -m "Committed by Travis-CI build number: %{build_number}"
It behaved as if .travis.yml file was absent/invalid (even though it did pass the Travis-CI YAML validation here).
It seems as though this should be doable, but could not find any sample that does this.
Could someone point me in the right direction?
The string replacements you can do for IRC output only work there unfortunately. They're only meant to be used for notifications in general, to customize the output, but are currently only available for IRC.
There's still a way to get the current build number, by accessing the TRAVIS_JOB_ID environment variable. If you change your script to the following line, things should work as expected:
after_success:
- git commit -a -m "Committed by Travis-CI build number: $TRAVIS_JOB_ID"
I use this in my deploy script:
git commit -am "Auto deploy from Travis CI build $TRAVIS_BUILD_ID"
More on Travis CI Documentation.