I had some difficulties in overriding perl lib when testing perl codes when the .pl or .pm has use lib or unshift #INC, my question is:
Is it a bad idea to use use lib or unshift #INC in the production code since they are hard to test? The prove -lvr cannot override these too.
Code test.pl
push #INC, '/push/inc/lowest_priority';
use lib "/top/priority/use/lib/second_priority";
unshift #INC, "/unshift/inc/lib/first_priority";
foreach my $inc (#INC){
print "INC=>$inc\n";
}
set perl env
export PERL5LIB=/export/PERL5LIB/env/lib:$PERL5LIB
Output of perl -I/cmd/Iinclude/lib/ test.pl
INC=>/unshift/inc/lib/first_priority
INC=>/top/priority/use/lib/second_priority
INC=>/cmd/Iinclude/lib/
INC=>/export/PERL5LIB/env/lib
INC=>/usr/local/lib64/perl5
INC=>/usr/local/share/perl5
INC=>/usr/lib64/perl5/vendor_perl
INC=>/usr/share/perl5/vendor_perl
INC=>/usr/lib64/perl5
INC=>/usr/share/perl5
INC=>/push/inc/lowest_priority
I don't hard-code paths unless I have no other options.
In general, you don't want to hardcode things that you can provide to your program in some other fashion so it can respond to whatever environment it's in rather than only the environment where you developed it. One of those environments could be your testing environment.
You can set the library search path from outside the program, and that makes it more flexible.
And, since you hard-code them and add them at runtime, they are going to come after anything you've set previously. Here's what happens in your setting:
You start with the default #INC.
You start to "run" your program, you start the compilation phase. It compiles the entire program before it executes run-time statements.
As it compiles, it encounters the use lib and executes that pragma immediately. Now /top/priority/use/lib/second_priority is at the beginning of #INC.
For the rest of the compilation phase, /top/priority/use/lib/second_priority is the first thing in #INC. That's where subsequent use calls will look for things.
The compilation phase finishes and the program transitions into the run phase.
It encounters the push and executes that. Now /push/inc/lowest_priority is the last element of #INC.
It skips over the use lib because the compilation phase handled the pragma.
It encounters the unshift and executes that. Now /unshift/inc/lib/first_priority is the first item in #INC.
Subsequent require calls (a runtime feature) will look in /unshift/inc/lib/first_priority first.
I don't know where you expect to find the library you expected to load, but you have to supply the full path to it. There may be extra directories under the lib/ that matter and you haven't accounted for.
I might be misunderstanding your problem but local::lib allows you to "manually" tune your module path. You should be able to use it to control what paths are used for your test environment.
Related
A post in this (Are system() calls evil?) thread says:
Your program's privileges are inherited by its spawned programs. If your application ever runs as a privileged user, all someone has to do is put their own program with the name of the thing you shell out too, and then can execute arbitrary code (this implies you should never run a program that uses system as root or setuid root).
But system("PAUSE") and system("CLS") shell to the OS, so how could a hacker possibly intervene if it ONLY shells to a specific secure location on the hard-drive?
Does explicitly flush—by using fflush or _flushall—or closing any stream before calling system eliminate all risk?
The system function passes command to the command interpreter, which executes the string as an operating-system command. system uses the COMSPEC and PATH environment variables to locate the command-interpreter file CMD.exe. If command is NULL, the function just checks whether the command interpreter exists.
You must explicitly flush—by using fflush or _flushall—or close any stream before you call system.
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/system-wsystem
In case, there are any doubts here's the actual snippet from the MS' implementation (very simple and straightforward):
// omitted for brevity
argv[1] = _T("/c");
argv[2] = (_TSCHAR *) command;
argv[3] = NULL;
/* If there is a COMSPEC defined, try spawning the shell */
/* Do not try to spawn the null string */
if (argv[0])
{
// calls spawnve on value of COMSPEC vairable, if present
// omitted for brevity
}
/* No COMSPEC so set argv[0] to what COMSPEC should be. */
argv[0] = _T("cmd.exe");
/* Let the _spawnvpe routine do the path search and spawn. */
retval = (int)_tspawnvpe(_P_WAIT,argv[0],argv,NULL);
// clean-up part omitted
As to concerns of what _tspawnvpe may actually be doing, the answer is: nothing magical. The exact invocation sequence for spawnvpe and friends goes as following (as anybody with licensed version of MSVC can easily learn by inspecting the spanwnvpe.c source file):
Do some sanity checks on parameters
Try to invoke _tspawnve on the passed file name. spawnve will succeed if the file name represents an absolute path to an executable or a valid path relative to the current working directory. No further checks are done - so yes, if a file named cmd.exe exists in current directory it will be invoked first in the context of system() call discussed.
In a loop: obtain the next path element using `_getpath()
Append the file name to the path element
Pass the resulted path to spwanvpe, check if it was successful
That's it. No special tricks/checks involved.
The original question references POSIX not windows. Here there is no COMSPEC (there is SHELL but system() deliberately does not use it); however /bin/sh is completely, utterly vulnerable.
Suppose /opt/vuln/program does system("/bin/ls"); Looks completely harmless, right? Nope!
$ PATH=. IFS='/ ' /opt/vuln/program
This runs the program called bin in the current directory. Oops. Defending against this kind of thing is so difficult it should be left to the extreme experts, like the guys who wrote sudo. Sanitizing environment is extremely hard.
So you might be thinking what is that system() api for. I don't actually know why it was created, but if you wanted to do a feature like ftp has where !command is executed locally in the shell you could do ... else if (terminalline[0] == '!') system(terminalline+1); else ... Since it's going to be completely insecure anyway there's no point in making it secure. Of course a truly modern use case wouldn't do it that way because system() doesn't look at $SHELL but oh well.
BACKGROUND
We have testers for our embedded GUI product and when a tester declares "test failed", sometimes it's hard for us developers to reproduce the exact problem because we don't have the exact trace of what happened.
We do currently have a logging framework but us devs have to manually input those logging statements in the code which is fine . . . except when a hard-to-reproduce bug occurs and we didn't have a log statement at the 'right' location and then when we re-build, re-run the test with the same steps, we get a different result.
ISSUE
We would like a solution wherein the compiler produces extra instrumentation code that allows us to see the exact sequence of events including, at the minimum:
function enter/exit (already provided by -finstrument-functions)
control-flow statement enter i.e. entering if/else, which case statement we jumped to
The log would look like this:
int main() entered
if-line 5 entered
else-line 10 entered
void EventLoop() entered
. . .
Some additional nice-to-haves are
Parameter values on function entry AND exit (for pass-by-reference types)
Function return value
QUESTION
Are there any gcc tools or even paid tools that can do this instrumentation automatically?
You can either use gdb for that, and you can automate that (I've got a tool for that in the works, you find it here or you can try to use gcov.
The gcov implementation is so, that it loads the latest gcov data when you start the program. But: you can manually dump and load the data. If you call __gcov_flush it will dump the current data and reset the current state. However: if you do that several times it will always merge the data, so you'd also need to rename the gcov data file then.
I'm trying to understand source code of SDL-1.2.15, and to find out how it renders stuff on windows. But I can't find where the rendering is happening. I looked inside SDL-1.2.15/src/video folder, and there is a ton of subfolders, and I don't know what any of these stands for. See for yourself.
aalib/ directfb/ ipod/ os2fslib/ quartz/ windib/
ataricommon/ dummy/ maccommon/ photon/ riscos/ windx5/
bwindow/ fbcon/ macdsp/ picogui/ svga/ wscons/
caca/ gapi/ macrom/ ps2gs/ symbian/ x11/
dc/ gem/ nanox/ ps3/ vgl/ xbios/
dga/ ggi/ nds/ qtopia/ wincommon/ Xext/
Is this documented somewhere? This is a pretty popular library, so it probably is documented, right? Right? What's the point of having source code if you can't even understand it, if you can't find functions you are using.
While not all the names are self-explanatory, they contain some hints.
directfb, fbcon (framebuffer console) and X (x11, Xext) are output layers on Linux (unix).
The ones starting with win indicate they are for Windows. More specifically, windib should be about device independent bitmaps (DIBs), dx5 about DirectX 5, and wincommon about some common stuff. Indeed, using grep shows that (only) these folders contain Windows-specific code:
grep -r windows.h src/video/*
[ lists files in the win* folders ]
You could also just compile the package on Windows and see which files were compiled (which folders contain object files)
However, to find out what it actually does, you should rather study the function you're interested in (e.g. SDL_BlitSurface), look at it's implementation, and then look at the implementation of the functions it uses. Start in SDL_video.h (and notice that SDL_BlitSurface is just a define).
You should use some tool to search the code base. Grep or some IDE. Or both.
First of all, why not SDL2?
These are different SDL's video drivers. You can get what driver is used by your program by calling SDL_VideoDriverName. Which driver will be used determined by target platform (e.g. operating system - most drivers are platform-specific), environment variable SDL_VIDEODRIVER, or calling side.
The go test command has support for the -c flag, described as follows:
-c Compile the test binary to pkg.test but do not run it.
(Where pkg is the last element of the package's import path.)
As far as I understand, generating a binary like this is the way to run it interactively using GDB. However, since the test binary is created by combining the source and test files temporarily in some /tmp/ directory, this is what happens when I run list in gdb:
Loading Go Runtime support.
(gdb) list
42 github.com/<username>/<project>/_test/_testmain.go: No such file or directory.
This means I cannot happily inspect the Go source code in GDB like I'm used to. I know it is possible to force the temporary directory to stay by passing the -work flag to the go test command, but then it is still a huge hassle since the binary is not created in that directory and such. I was wondering if anyone found a clean solution to this problem.
Go 1.5 has been released, and there is still no officially sanctioned Go debugger. I haven't had much success using GDB for effectively debugging Go programs or test binaries. However, I have had success using Delve, a non-official debugger that is still undergoing development: https://github.com/derekparker/delve
To run your test code in the debugger, simply install delve:
go get -u github.com/derekparker/delve/cmd/dlv
... and then start the tests in the debugger from within your workspace:
dlv test
From the debugger prompt, you can single-step, set breakpoints, etc.
Give it a whirl!
Unfortunately, this appears to be a known issue that's not going to be fixed. See this discussion:
https://groups.google.com/forum/#!topic/golang-nuts/nIA09gp3eNU
I've seen two solutions to this problem.
1) create a .gdbinit file with a set substitute-path command to
redirect gdb to the actual location of the source. This file could be
generated by the go tool but you'd risk overwriting someone's custom
.gdbinit file and would tie the go tool to gdb which seems like a bad
idea.
2) Replace the source file paths in the executable (which are pointing
to /tmp/...) with the location they reside on disk. This is
straightforward if the real path is shorter then the /tmp/... path.
This would likely require additional support from the compiler /
linker to make this solution more generic.
It spawned this issue on the Go Google Code issue tracker, to which the decision ended up being:
https://code.google.com/p/go/issues/detail?id=2881
This is annoying, but it is the least of many annoying possibilities.
As a rule, the go tool should not be scribbling in the source
directories, which might not even be writable, and it shouldn't be
leaving files elsewhere after it exits. There is next to nothing
interesting in _testmain.go. People testing with gdb can break on
testing.Main instead.
Russ
Status: Unfortunate
So, in short, it sucks, and while you can work around it and GDB a test executable, the development team is unlikely to make it as easy as it could be for you.
I'm still new to the golang game but for what it's worth basic debugging seems to work.
The list command you're trying to work can be used so long as you're already at a breakpoint somewhere in your code. For example:
(gdb) b aws.go:54
Breakpoint 1 at 0x61841: file /Users/mat/gocode/src/github.com/stellar/deliverator/aws/aws.go, line 54.
(gdb) r
Starting program: /Users/mat/gocode/src/github.com/stellar/deliverator/aws/aws.test
[snip: some arnings about BinaryCache]
Breakpoint 1, github.com/stellar/deliverator/aws.imageIsNewer (latest=0xc2081fe2d0, ami=0xc2081fe3c0, ~r2=false)
at /Users/mat/gocode/src/github.com/stellar/deliverator/aws/aws.go:54
54 layout := "2006-01-02T15:04:05.000Z"
(gdb) list
49 func imageIsNewer(latest *ec2.Image, ami *ec2.Image) bool {
50 if latest == nil {
51 return true
52 }
53
54 layout := "2006-01-02T15:04:05.000Z"
55
56 amiCreationTime, amiErr := time.Parse(layout, *ami.CreationDate)
57 if amiErr != nil {
58 panic(amiErr)
This is just after running the following in the aws subdir of my project:
go test -c
gdb aws.test
As an additional caveat, it does seem very selective about where breakpoints can be placed. Seems like it has to be an expression but that conclusion is only via experimentation.
If you're willing to use tools besides GDB, check out godebug. To use it, first install with:
go get github.com/mailgun/godebug
Next, insert a breakpoint somewhere by adding the following statement to your code:
_ = "breakpoint"
Now run your tests with the godebug test command.
godebug test
It supports many of the parameters from the go test command.
-test.bench string
regular expression per path component to select benchmarks to run
-test.benchmem
print memory allocations for benchmarks
-test.benchtime duration
approximate run time for each benchmark (default 1s)
-test.blockprofile string
write a goroutine blocking profile to the named file after execution
-test.blockprofilerate int
if >= 0, calls runtime.SetBlockProfileRate() (default 1)
-test.count n
run tests and benchmarks n times (default 1)
-test.coverprofile string
write a coverage profile to the named file after execution
-test.cpu string
comma-separated list of number of CPUs to use for each test
-test.cpuprofile string
write a cpu profile to the named file during execution
-test.memprofile string
write a memory profile to the named file after execution
-test.memprofilerate int
if >=0, sets runtime.MemProfileRate
-test.outputdir string
directory in which to write profiles
-test.parallel int
maximum test parallelism (default 4)
-test.run string
regular expression to select tests and examples to run
-test.short
run smaller test suite to save time
-test.timeout duration
if positive, sets an aggregate time limit for all tests
-test.trace string
write an execution trace to the named file after execution
-test.v
verbose: print additional output
I have a C++ program which is mainly used for video processing. Inside the program, I am launching the system command in order to obtain pass the processed videos to some other binaries to postprocess them.
I have a while loop towards infinite and I am launching the system command inside the loop every time. The thing is that at a certain point I am receiving the -1 return code from the system command. What could be the reason for that?
Inside the system command I am just calling a binary file with the adequate parameters from the main project.
The system command which I want to execute is actually a shell file.
In this file I am extracting the main feature from the video and passing them through a SVM model from a 3D party library in order to obtain the the desired classification.
./LiveGestureKernel ./Video ./SvmVideo
./mat4libsvm31 -l SvmVideoLabels < SvmVideo > temp_test_file
./svm-predict temp_test_file svm_model temp_output_file
cat < temp_output_file
rm -f temp_*
After a certain number of iterations through the while loop, it just won't execute the script file and I cannot figure out the reason for this. Thanks!
If you get -1 from the call to system(), you should first examine the contents of errno - that will most likely tell you what your specific problem is.
The one thing to watch out for is that the return value from system is an implementation-defined one in the case where you pass it a non-NULL command, so it's possible that -1 may be coming from your actual executable.
Your best bet in that case is to print out (or otherwise log) the command being executed on failure (and possibly all the time), so that you can check what happens with the same arguments when you execute it directly from a command line or shell.