In my Hudson log I see error or fails, but hudson says ,that build is success.
Is it right?
This is what hudson does (run ant few times)
call f:\runGenericAntBuild.bat %WORKSPACE% f:\general-build.properties %WORKSPACE%\build\buildProjects.xml deploy %BUILD_NUMBER% %SVN_REVISION%
call f:\runGenericAntBuild.bat %WORKSPACE% f:\general-build.properties %WORKSPACE%\build\buildProjects.xml MyJavaProject %BUILD_NUMBER% %SVN_REVISION%
call f:\runGenericAntBuild.bat %WORKSPACE% f:\general-build.properties %WORKSPACE%\build\buildProjects.xml buildGrails %BUILD_NUMBER% %SVN_REVISION%
call f:\runGrailsClean.bat %WORKSPACE%\MyProject
Thanks!
Have you got a single build step that is a batch step and it contains all four call commands? If yes, thats your problem.
Your batch looks like its not doing anything with errors in any of the steps, so the status of the batch is the status of the last command (your clean step)
Either split it into four separate build steps within Hudson/Jenkins, or handle the errors in the batch script.
e.g.
call f:\runGenericAntBuild.bat your params here || exit /b 1
call f:\runGenericAntBuild.bat next params here || exit /b 2
call f:\runGenericAntBuild.bat third param here || exit /b 3
call f:\runGrailsClean.bat more param as needed || exit /b 4
This way the batch will exit with an error of 1-4 depending on the step that failed.
According to me Jenkins/Hudson is able to complete the Job successfully because it is able to complete the Job that is assigned to him. You might have not set any property in your buildProjects.xml file that says the build must fail on errors. There is a property that declares a build to fail on error.
Related
I'm working on a TCL project, for which version control is based on Git. So, in order to produce good-quality code, I set up put execution of the tests in pre-commit hook.
However, even if they are executed (trace is shown in command-line), and one of the tests is failed, Git performs the commit. So I launched the hook manually to check the error code, and I figured out that it is null, explaining why Git does not stop:
$ .git/hooks/pre-commit
++++ FlattenResult-test PASSED
(...)
==== CheckF69F70 FAILED
==== Content of test case:
(...)
==== CheckF69F70 FAILED
$ echo $?
0
(Launching the tests script with tclsh also results in $? to be 0.)
So my question is about this last line: why is $? equal to 0, when one of the tcl tests is failed? And how can I achieve a simple pre-commit hook that stops on failure?
I read and reread the tcltest documentation, but saw no setting or information about this error code. And I would really like not to have to parse the tcl tests output, to check if ERROR or FAILED is present...
Edit: versions
TCL version : 8.5
tcltest version: 2.3.4
This depends on how you run your test suite. Normally you run a file called tests/all.tcl which may look something like this:
package require Tcl 8.6
package require tcltest 2.5
namespace import tcltest::*
configure -testdir [file dirname [file normalize [info script]]] {*}$argv
runAllTests
That final runAllTests returns a boolean indicating success (0) or failure (1). You can use that to generate an exit code by changing the last line to:
exit [runAllTests]
I use this redefinition in some of my test scripts:
# Exit non-zero if any tests fail.
# tcltest's `cleanupTests` resets the numTests array, so capture it first.
proc cleanupTests {} {
set failed [expr {$::tcltest::numTests(Failed) > 0}]
uplevel 1 ::tcltest::cleanupTests
if {$failed} then {exit 1}
}
After some research, I could make it work, even though several factors were against me:
I have to use an old TCL version (8.5) with tcltest version 2.3.4, in which runAllTests returns nothing;
I forgot to write cleanupTests at the end of test scripts, as the documentation is not really clear about its usage. (It is not clearer now. I just figured out it is needed if you want to get your tests run by runAllTests, which is really not obvious).
And here is my solution, mostly based on Hai's DevBits blog post:
all.tcl
package require tcltest
::tcltest::configure (...)
proc ::tcltest::cleanupTestsHook {} {
variable numTests
set ::exitCode [expr {$numTests(Total) == 0 || $numTests(Failed) > 0}]
}
::tcltest::runAllTests
exit $exitCode
Some thoughts about it:
I added $numTests(Total) == 0 as a failure condition: this means that no tests was found, which is clearly an erroneous condition;
This doesn't catch exceptions in the configuration of the tests, for instance a source command that points to a non-existing file, revealing some failure in tests scaffolding. This would be catched as error in other test framewords (ah, pytest, I miss you!)
We have a project managed in Gitlab, with CI pipeline for builds and tests (pytest, Google tests). Two or three of our test cases in Google tests fail. But Gitlab consider that the test stage is successful. Is it because the success percentage is more than 90% (an arbitrary value) ? Is there a way to make the stage (and thus the complete pipeline) fail if we don't get 100% of success ?
Here is a screenshot of the pipeline summary:
Here is the yml script of the stage:
test_unit_test:
stage: test
needs: ["build", "build_unit_test"]
image: $DOCKER_IMAGE
rules:
- if: '$CI_PIPELINE_SOURCE != "merge_request_event"'
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
script: |
ZIPNAME=`cat _VERSION_.txt`
./scripts/gitlab-ci/stage-unittests.sh test_unit_test_report.xml $ZIPNAME
artifacts:
reports:
junit: test_unit_test_report.xml
expire_in: 1 week
Thank you for any help.
Regards.
Gitlab CI/CD jobs don't care what the script areas are doing (so they don't look at, for example, test pass percentages). The only they things use to determine if a job passed or failed are exit codes and the allow_failure keyword.
After each command in the before_script, script, and after_script sections are executed, the Gitlab Runner checks the exit code of the command to see if it is 0. If it is non-zero, the command is considered a failure, and if the allow_failure keyword is not set to true for the job, the job fails.
So, for your job, even though the tests are failing, somehow the script is existing with exit code 0, meaning the command itself finished successfully. The command in this case is:
ZIPNAME=$(cat _VERSION_.txt)
./scripts/gitlab-ci/stage-unittests.sh test_unit_test_report.xml $ZIPNAME
NOTE: I replaced your backticks '`' with the $(command) syntax explained here which does the same thing (execute this command) but has some advantages over '`command`' including nesting and easier use in markdown where '`' indicates code formatting.
So, since you are calling a script (./scripts/gitlab-ci/stage-unittests.sh) to run your tests, that script itself is finishing successfully, so the job finishes successfully. Take a look at that script to see if you can tell why it finishes successfully even though the tests fail.
I am trying to figure out a way to check if the file exists before running a session.
I currently have a command line task before a sessions that does the following:
IF EXIST TEST*.TXT
EXIT 0
ELSE
EXIT 1
I want to use the ErrorCode value on the link task, but I have no luck getting it to work. Is there a way to do this?
I use Command task with this command:
test -f $PMTargetFileDir/fdata.txt; echo $((1/$?))
1) If the file exists $? equals to 0.
2) If the file does not exist $? equals to 1.
So, in 1) $((1/$?)) will cause th error (division by zero). After the Command task you can add two or one of these links: first with the condition $TaskName.PrevTaskStatus=SUCCEEDED, and second with $TaskName.PrevTaskStatus=FAILED.
Create a command task just before your main session
Link this command task to main session and put link condition as $prevtaskstatus=succeeded
and write below code in Command task
head -2 /testfiledir/test.txt
Command task will fail if file does not exist and succeeds if file exist.
Script 1 has imported Script 2
Script 1 invokes a method on Script 2
If some condition is met, Script 2 must exit and Script 1 must resume.
If this is possible, how do I achieve it and how Script 1 understands Script 2 has exited?
I do aware of return statements, but function is nested and hence 'return' would return to the previous function. I want to stop the entire script (Script2), without affecting Script 1.
Thanks in advance.
When I run an Execute shell build step to execute a script and that script returns 0, Jenkins flags the build as SUCCESS, otherwise it flags it as FAILURE which is the expected default behaviour as 0 means no errors and any other value represents an error.
Is there a way to mark a build as SUCCESS only if the return value matches a specific value other than 0 (e.g. 1,2,3...)?
PS: in case you're wondering why I'm looking for that, this will allow me to perform unit testing of Jenkins itself as my scripts are written to return different exit values depending on various factors, thus allowing me to expect certain values depending on certain setup mistakes and making sure my whole Jenkins integration picks up on those.
Alright, I went on IRC #jenkins and no-one new about a plugin to set a particular job status depending on a particular exit code :( I managed to do what I wanted by creating an Execute shell step with the following content:
bash -c "/path/to/myscript.sh; if [ "\$?" == "$EXPECTED_EXIT_CODE" ]; then exit 0; else exit 1; fi"
-Running the script under bash -c allows catching the exit code and prevents Jenkins from stopping build execution when that exit code is different than 0 (which it normally does).
-\$? is interpreted as $? after the script execution and represents its exit code.
-$EXPECTED_EXIT_CODE is one of my job parameters which defines the exit code I'm expecting.
-The if statement simply does the following: if I get the expected exit code, exit with 0 so that the build is marked as SUCCESS, else exit with 1 so that the build is marked as FAILURE.
/path/to/myscript.sh || if [ "$?" == "$EXPECTED_EXIT_CODE" ]; then continue; else exit 1; fi
I would use continue instead of exit 0 in case you have other items below that you need to run through.
Can handle it via the Text-finder Plugin:
Have your script print the exit-code it is about to exit with, like: Failed on XXX - Exiting with RC 2
Use the Text-finder Plugin to catch that error-message and mark the build as 'Failed' or 'Unstable',for example, if you decide RC 2, 3 and 4 should mark the build as 'Unstable', look for text in this pattern: Exiting with RC [2-4].
Create a wrapper for your shell script. Have that wrapper execute your tests and then set the resturn value according to whatever criteria you want.
I do it like this:
set +e
./myscript.sh
rc="$?"
set -e
if [ "$rc" == "$EXPECTED_CODE_1" ]; then
#...actions 1 (if required)
exit 0
elif [ "$rc" == "$EXPECTED_CODE_2" ]; then
#...actions 2 (if required)
exit 0
else
#...actions else (if required)
exit "$rc"
fi
echo "End of script" #Should never happen, just to indicate there's nothing further
Here +e is to avoid default Jenkins behavior to report FAILURE on any sneeze during your script execution. Then get back with -e.
So that you can handle your exit code as appropriate, else eventually FAIL with the returned code.
robocopy "srcDir" "destDir" /"copyOption" if %ERRORLEVEL% LEQ 2 exit 0
If robocopy exit code is less than or equal to 2 then it will exit successfully.
Robocopy Exit Codes:
0×00 0 No errors occurred, and no copying was done.
The source and destination directory trees are completely synchronized.
0×01 1 One or more files were copied successfully (that is, new files have arrived).
0×02 2 Some Extra files or directories were detected. No files were copied
Examine the output log for details.
0×04 4 Some Mismatched files or directories were detected.
Examine the output log. Housekeeping might be required.
0×08 8 Some files or directories could not be copied
(copy errors occurred and the retry limit was exceeded).
Check these errors further.
0×10 16 Serious error. Robocopy did not copy any files.
Either a usage error or an error due to insufficient access privileges
on the source or destination directories.