I am going to use CDash with CMake/CTest on my C++ project.
In order to enable CDash and customize settings, like
"MEMORYCHECK_SUPPRESSIONS_FILE", "DART_TESTING_TIMEOUT", I added the following lines in the root CMakeLists.txt
set(MEMORYCHECK_SUPPRESSIONS_FILE "${CMAKE_SOURCE_DIR}/valgrind.supp")
set(DART_TESTING_TIMEOUT "120")
include(CTest)
However, the generated "DartConfiguration.tcl" does not contain my settings at all
( MemoryCheckSuppressionFile is empty and TimeOut is still the default value )
I found that, for example, if I pass -DDART_TESTING_TIMEOUT=STRING:120 , it works , but it fails if specifying them in the CMakeLists.txt.
Thank you in advance :)
DartConfiguration.tcl
# Dynamic analisys and coverage
PurifyCommand:
ValgrindCommand:
ValgrindCommandOptions:
MemoryCheckCommand: /usr/bin/valgrind
MemoryCheckCommandOptions:
MemoryCheckSuppressionFile:
CoverageCommand: /usr/bin/gcov
# Testing options
# TimeOut is the amount of time in seconds to wait for processes
# to complete during testing. After TimeOut seconds, the
# process will be summaily terminated.
# Currently set to 25 -9.0.0.71596-0inutes
TimeOut: 1500
There are three possible solutions:
You create cache variables. This also creates a GUI entry for the variable, which is not always what you want for automatic testing: SET(DART_TESTING_TIMEOUT "120" CACHE STRING "")
You specify your options with a simple "set" command, but in a file called DartConfig.cmake instead of the main CMakeLists.txt . This file gets parsed to create the DartConfiguration.tcl
You use CTest scripting to set up your dartclient: http://www.cmake.org/Wiki/CMake_Scripting_Of_CTest
Related
I'm running an Experiment with multiple trials and I would like to log the STDOUT for the process for each trial in a file in the directory created in ray_results for that trial.
I have set log_to_driver to false since the driver couldn't keep up and I have set redirect_worker_output to true but that doesn't seem to result in what I want either.
Perhaps there is an option in ray.tune.run? I have set verbose=2 but that doesn't work.
All the time when debugging cmake code, I find myself writing things like the following:
message( "[some_filename.cmake]: some message about what is going on here." )
In C++, I use macros to automatically log the file name and line number - is this possible in cmake? Ideally, I'd like to write a macro that with the following:
log_info( "some message about what is going on here." )
It would print to console:
|info | some_filename.cmake[72] some message about what is going on here.
With CMAKE_CURRENT_LIST_FILE and CMAKE_CURRENT_LIST_LINE this should be possible.
However, using CMAKE_CURRENT_LIST_LINE directly in the macro will always give you the line in the macro, not where the macro was used (at least with CMake 3.6.1). Thus you need to pass it as an argument:
macro(log_info _line)
# get the path of the CMakeLists file evaluating this macro
# relative to the project's root source directory
file(RELATIVE_PATH _curr_file
"${PROJECT_SOURCE_DIR}"
"${CMAKE_CURRENT_LIST_FILE}")
message("|info | ${_curr_file}[${_line}] ${ARGN}")
endmacro(log_info)
And use it as
log_info(${CMAKE_CURRENT_LIST_LINE} "a info message")
A more advanced solution would be:
macro(log _level _line)
# get the path of the CMakeLists file evaluating this macro
# relative to the project's root source directory
file(RELATIVE_PATH _curr_file
"${PROJECT_SOURCE_DIR}"
"${CMAKE_CURRENT_LIST_FILE}")
set(level_padded " ")
if("${_level}" MATCHES [Dd][Ee][Bb][Uu][Gg])
set(level_padded "debug ")
elseif("${_level}" MATCHES [Ii][Nn][Ff][Oo])
set(level_padded "info ")
elseif("${_level}" MATCHES [Ee][Rr][Rr][Oo][Rr])
set(level_padded "error ")
endif()
message("|${level_padded} | ${_curr_file}[${_line}] ${ARGN}")
endmacro(log)
Usage:
log(debug ${CMAKE_CURRENT_LIST_LINE} "debug info")
log(Info ${CMAKE_CURRENT_LIST_LINE} "info message")
log(ERROR ${CMAKE_CURRENT_LIST_LINE} "error message")
I don't think this is possible in a general manner. CMake does not offer much help with regard to debugging or introspection.
I can imagine a macro that uses global variables which must be added by the user at the beginning of each macro or function. In the end, it wouldn't reduce the amount of manual work but clutter the code.
Maybe you can use a function / macro of your editor. Some editors can insert the current file name, the current time stamp or a pseudo-random UUID by key stroke. Whenever you want to log a message, just add the value and you can later search the code for this unique sequence. Nut sure whether this is a better workflow for you.
The best place would be a feature request for CMake. You are not the only one who could profit from such a feature. Maybe Kitware or someone else is willing to develop and upstream a log macro.
When go test is ran it runs your files ending in _test.go by running the functions that start in the format TestXxx and use the (*t testing.T) module. I was wondering if each function in the _test.go file were ran concurrently or if it definitively ran each function separately? Does it create a go routine for each one? If it does create a go routine for each one, can I monitor the go routines in some way? Is it ever possible to do something like golibrary.GoRoutines() and get an instance for each one and monitor them some how or something like that?
Note: this question assumes you using the testing framework that comes with go (testing).
Yes, tests are executed as goroutines and, thus, executed concurrently.
However, tests do not run in parallel by default as pointed out by #jacobsa. To enable parallel execution you would have to call t.Parallel() in your test case and set GOMAXPROCS appropriately or supply -parallel N (which is set to GOMAXPROCS by default).
The simplest solution for your case when running tests in parallel would be to have a global slice for port numbers and a global atomically incremented index that serves as association between test and port from the slice. That way you control the port numbers and have one port for each test. Example:
import "sync/atomic"
var ports [...]uint64 = {10, 5, 55}
var portIndex uint32
func nextPort() uint32 {
return atomic.AddUint32(&portIndex, 1)
}
Yes, as other answers already pointed, tests inside one _test.go file are running in parallel only if you add t.Parallel() and run with -parallel tag.
BUT - by default the command go test runs in parallel tests for different packages, and default is the number of CPUs available (see go help build)
So to disable parallel execution you should run tests like this
go test -p 1 ./...
Yes if you add this :
t.Parallel() to your functions.
like this:
func TestMyFunction(t *testing.T) {
t.Parallel()
//your test code here
}
There are two aspects of "parallel tests" in go.
Testing of packages (folders) in parallel. Controlled by -p flag.
go help build:
-p n
the number of programs, such as build commands or
test binaries, that can be run in parallel.
The default is GOMAXPROCS, normally the number of CPUs available.
Testing of tests within a package. Controlled by -parallel flag, and tests need this enabled with t.Parallel().
go help testflag:
-parallel n
Allow parallel execution of test functions that call t.Parallel.
The value of this flag is the maximum number of tests to run
simultaneously; by default, it is set to the value of GOMAXPROCS.
Note that -parallel only applies within a single test binary.
The 'go test' command may run tests for different packages
in parallel as well, according to the setting of the -p flag
(see 'go help build').
For #1, you can observe this directly by having tests in multiple packages, and forcing them to be long running with a sleep. Then ps aux and grep for go running and you'll see separate processes in parallel.
Or run with -x detail flag. That pretty clearly shows what’s going on. The tests are launched once per package (e.g. given have *_test.go files in several package. Note I’m running go test -x ./... -v -count=1 |& ruby -pe 'print Time.now.strftime("[%Y-%m-%d %H:%M:%S] ")' |& tee temp.out with various configurations of -p=N -parallel=M, you can see in parallel them running:
[2022-05-09 17:22:26] $WORK/b065/understandinit.test -test.paniconexit0 -test.timeout=10m0s -test.parallel=1 -test.v=true -test.count=1
[2022-05-09 17:22:26] $WORK/b001/main.test -test.paniconexit0 -test.timeout=10m0s -test.parallel=1 -test.v=true -test.count=1
[2022-05-09 17:22:26] $WORK/b062/mainsub.test -test.paniconexit0 -test.timeout=10m0s -test.parallel=1 -test.v=true -test.count=1
You can run them concurrently by flagging the test with t.Parallel
and then run the test using the -parallel flag.
You can see other testing flags here
The go test command has support for the -c flag, described as follows:
-c Compile the test binary to pkg.test but do not run it.
(Where pkg is the last element of the package's import path.)
As far as I understand, generating a binary like this is the way to run it interactively using GDB. However, since the test binary is created by combining the source and test files temporarily in some /tmp/ directory, this is what happens when I run list in gdb:
Loading Go Runtime support.
(gdb) list
42 github.com/<username>/<project>/_test/_testmain.go: No such file or directory.
This means I cannot happily inspect the Go source code in GDB like I'm used to. I know it is possible to force the temporary directory to stay by passing the -work flag to the go test command, but then it is still a huge hassle since the binary is not created in that directory and such. I was wondering if anyone found a clean solution to this problem.
Go 1.5 has been released, and there is still no officially sanctioned Go debugger. I haven't had much success using GDB for effectively debugging Go programs or test binaries. However, I have had success using Delve, a non-official debugger that is still undergoing development: https://github.com/derekparker/delve
To run your test code in the debugger, simply install delve:
go get -u github.com/derekparker/delve/cmd/dlv
... and then start the tests in the debugger from within your workspace:
dlv test
From the debugger prompt, you can single-step, set breakpoints, etc.
Give it a whirl!
Unfortunately, this appears to be a known issue that's not going to be fixed. See this discussion:
https://groups.google.com/forum/#!topic/golang-nuts/nIA09gp3eNU
I've seen two solutions to this problem.
1) create a .gdbinit file with a set substitute-path command to
redirect gdb to the actual location of the source. This file could be
generated by the go tool but you'd risk overwriting someone's custom
.gdbinit file and would tie the go tool to gdb which seems like a bad
idea.
2) Replace the source file paths in the executable (which are pointing
to /tmp/...) with the location they reside on disk. This is
straightforward if the real path is shorter then the /tmp/... path.
This would likely require additional support from the compiler /
linker to make this solution more generic.
It spawned this issue on the Go Google Code issue tracker, to which the decision ended up being:
https://code.google.com/p/go/issues/detail?id=2881
This is annoying, but it is the least of many annoying possibilities.
As a rule, the go tool should not be scribbling in the source
directories, which might not even be writable, and it shouldn't be
leaving files elsewhere after it exits. There is next to nothing
interesting in _testmain.go. People testing with gdb can break on
testing.Main instead.
Russ
Status: Unfortunate
So, in short, it sucks, and while you can work around it and GDB a test executable, the development team is unlikely to make it as easy as it could be for you.
I'm still new to the golang game but for what it's worth basic debugging seems to work.
The list command you're trying to work can be used so long as you're already at a breakpoint somewhere in your code. For example:
(gdb) b aws.go:54
Breakpoint 1 at 0x61841: file /Users/mat/gocode/src/github.com/stellar/deliverator/aws/aws.go, line 54.
(gdb) r
Starting program: /Users/mat/gocode/src/github.com/stellar/deliverator/aws/aws.test
[snip: some arnings about BinaryCache]
Breakpoint 1, github.com/stellar/deliverator/aws.imageIsNewer (latest=0xc2081fe2d0, ami=0xc2081fe3c0, ~r2=false)
at /Users/mat/gocode/src/github.com/stellar/deliverator/aws/aws.go:54
54 layout := "2006-01-02T15:04:05.000Z"
(gdb) list
49 func imageIsNewer(latest *ec2.Image, ami *ec2.Image) bool {
50 if latest == nil {
51 return true
52 }
53
54 layout := "2006-01-02T15:04:05.000Z"
55
56 amiCreationTime, amiErr := time.Parse(layout, *ami.CreationDate)
57 if amiErr != nil {
58 panic(amiErr)
This is just after running the following in the aws subdir of my project:
go test -c
gdb aws.test
As an additional caveat, it does seem very selective about where breakpoints can be placed. Seems like it has to be an expression but that conclusion is only via experimentation.
If you're willing to use tools besides GDB, check out godebug. To use it, first install with:
go get github.com/mailgun/godebug
Next, insert a breakpoint somewhere by adding the following statement to your code:
_ = "breakpoint"
Now run your tests with the godebug test command.
godebug test
It supports many of the parameters from the go test command.
-test.bench string
regular expression per path component to select benchmarks to run
-test.benchmem
print memory allocations for benchmarks
-test.benchtime duration
approximate run time for each benchmark (default 1s)
-test.blockprofile string
write a goroutine blocking profile to the named file after execution
-test.blockprofilerate int
if >= 0, calls runtime.SetBlockProfileRate() (default 1)
-test.count n
run tests and benchmarks n times (default 1)
-test.coverprofile string
write a coverage profile to the named file after execution
-test.cpu string
comma-separated list of number of CPUs to use for each test
-test.cpuprofile string
write a cpu profile to the named file during execution
-test.memprofile string
write a memory profile to the named file after execution
-test.memprofilerate int
if >=0, sets runtime.MemProfileRate
-test.outputdir string
directory in which to write profiles
-test.parallel int
maximum test parallelism (default 4)
-test.run string
regular expression to select tests and examples to run
-test.short
run smaller test suite to save time
-test.timeout duration
if positive, sets an aggregate time limit for all tests
-test.trace string
write an execution trace to the named file after execution
-test.v
verbose: print additional output
At the moment I have to hard code the names of servers on my vimrc in order to either make it different on the remote machine. This is done by conditional statement using hostname() function in vim. I want to make the conditional to be based on the status of remote connection and not on the hostname. So...
The first possible solution I found was using the following bash command in system():
cat /proc/$PPID/status | head -1 | cut -f2
This does not work because I use GNU screen and this will not detect my connection status properly.
The second possible solution I am exploring right now is using who am i This reliably shows whether or not remote connection has been made from which client, but I have trouble getting it working with system()
if substitute(system('who am i'), "theclient", ????, "") == ""
...
How could I get ???? to extract my client name somehow??
Even if the second solution works, allowing me to use .vimrc for many different remote machines, it is still tied to one client. I want the conditional to work in all remote session, regardless of the client name. So I am wondering, is this possible?
The following line allows me to create a variable that detects the remote connection status:
let g:remoteSession = ($STY == "")
Now you can surround the lines that you want to be ignored in the remote connection via:
if g:remoteSession
...
endif
On a side note, I do not know how expensive it is look up the environment variable compared to the global variable, but I am guessing the difference is negligible. The system call in an environment like cygwin where fork() is inefficient, it is worth doing the optimization.
Instead of adding conditional logic to a shared ~/.vimrc, you could alternatively source system-local settings. I use the following:
" Source system-specific .vimrc first.
if filereadable(expand('~/local/.vimrc'))
source ~/local/.vimrc
endif
" Stop sourcing if inclusion guard exists.
if exists('g:loaded_vimrc')
finish
endif
" Common settings of .vimrc here...
I find this more scalable than trying to maintain an ever-changing list of hostnames in a central location.