I have a small Go app with a cli using flags and I've been asked to make it more testable.
My app gets called on the command line like
deploy.exe <task> <command> -tenant tenant_name -validate -package "c:\some dir\\"
Based on which task and command a different execution path is called and ultimately a func residing in another package is called like:
if command == "db" {
dbhelper.RunDBCmds(*tenant, *validate, *package)
}
I need to write unit tests for just the flag parsing, without calling the actual functions at the end.
I'm fairly new to Go and I'm struggling figuring out how to accomplish this. I've thought about adding my Os.Args() and Flag parsing to a function that takes input and outputs a pointer of sorts to the RunDBCmds(*tenant, ...) func. However, I'm just not sure I can accomplish returning a pointer to a function.
I'd appreciate any advice on how I can make my code more testable without actually invoking the functions.
If all of your tasks/commands have different sets of flags I would end up with introducing some kind of Command abstraction. The best example can be found in the Go source code itself:
base.Commands = []*base.Command{
work.CmdBuild,
clean.CmdClean,
doc.CmdDoc,
envcmd.CmdEnv,
bug.CmdBug,
fix.CmdFix,
//...
}
Each command can have its own flag.FlagSet to parse command-specific flags:
// A Command is an implementation of a go command
// like go build or go fix.
type Command struct {
// Run runs the command.
// The args are the arguments after the command name.
Run func(cmd *Command, args []string)
// UsageLine is the one-line usage message.
// The first word in the line is taken to be the command name.
UsageLine string
// Short is the short description shown in the 'go help' output.
Short string
// Long is the long message shown in the 'go help <this-command>' output.
Long string
// Flag is a set of flags specific to this command.
Flag flag.FlagSet
// CustomFlags indicates that the command will do its own
// flag parsing.
CustomFlags bool
}
Using this approach you can separate commands' flags parsing from the execution.
Related
A post in this (Are system() calls evil?) thread says:
Your program's privileges are inherited by its spawned programs. If your application ever runs as a privileged user, all someone has to do is put their own program with the name of the thing you shell out too, and then can execute arbitrary code (this implies you should never run a program that uses system as root or setuid root).
But system("PAUSE") and system("CLS") shell to the OS, so how could a hacker possibly intervene if it ONLY shells to a specific secure location on the hard-drive?
Does explicitly flush—by using fflush or _flushall—or closing any stream before calling system eliminate all risk?
The system function passes command to the command interpreter, which executes the string as an operating-system command. system uses the COMSPEC and PATH environment variables to locate the command-interpreter file CMD.exe. If command is NULL, the function just checks whether the command interpreter exists.
You must explicitly flush—by using fflush or _flushall—or close any stream before you call system.
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/system-wsystem
In case, there are any doubts here's the actual snippet from the MS' implementation (very simple and straightforward):
// omitted for brevity
argv[1] = _T("/c");
argv[2] = (_TSCHAR *) command;
argv[3] = NULL;
/* If there is a COMSPEC defined, try spawning the shell */
/* Do not try to spawn the null string */
if (argv[0])
{
// calls spawnve on value of COMSPEC vairable, if present
// omitted for brevity
}
/* No COMSPEC so set argv[0] to what COMSPEC should be. */
argv[0] = _T("cmd.exe");
/* Let the _spawnvpe routine do the path search and spawn. */
retval = (int)_tspawnvpe(_P_WAIT,argv[0],argv,NULL);
// clean-up part omitted
As to concerns of what _tspawnvpe may actually be doing, the answer is: nothing magical. The exact invocation sequence for spawnvpe and friends goes as following (as anybody with licensed version of MSVC can easily learn by inspecting the spanwnvpe.c source file):
Do some sanity checks on parameters
Try to invoke _tspawnve on the passed file name. spawnve will succeed if the file name represents an absolute path to an executable or a valid path relative to the current working directory. No further checks are done - so yes, if a file named cmd.exe exists in current directory it will be invoked first in the context of system() call discussed.
In a loop: obtain the next path element using `_getpath()
Append the file name to the path element
Pass the resulted path to spwanvpe, check if it was successful
That's it. No special tricks/checks involved.
The original question references POSIX not windows. Here there is no COMSPEC (there is SHELL but system() deliberately does not use it); however /bin/sh is completely, utterly vulnerable.
Suppose /opt/vuln/program does system("/bin/ls"); Looks completely harmless, right? Nope!
$ PATH=. IFS='/ ' /opt/vuln/program
This runs the program called bin in the current directory. Oops. Defending against this kind of thing is so difficult it should be left to the extreme experts, like the guys who wrote sudo. Sanitizing environment is extremely hard.
So you might be thinking what is that system() api for. I don't actually know why it was created, but if you wanted to do a feature like ftp has where !command is executed locally in the shell you could do ... else if (terminalline[0] == '!') system(terminalline+1); else ... Since it's going to be completely insecure anyway there's no point in making it secure. Of course a truly modern use case wouldn't do it that way because system() doesn't look at $SHELL but oh well.
I have a c++ program that run a command and pass some arguments to it. The code is as follow:
int RunApplication(fs::path applicationPathName,std::string arguments)
{
std::string applicationShortPath=GetShortFileName(applicationPathName);
std::string cmd="\""+applicationShortPath +"\" "+ arguments+" >>log.txt 2>&1 \"";
std::cout<<cmd<<std::endl;
int result=std::system(cmd.c_str());
return result;
}
When I run system command, the cmd window appears shortly and then closes, but the result is 1 and the cmd was not run (the command should generate output which is not generated).
To check that the cmd is correct, I stopped the application just before system line and copy/ paste cmd content to a cmd window and it worked.
I am wondering how can I find why application is not run in system()?
the cmd has this value just before running it:
"D:/DEVELO~3/x64/Debug/enfuse.exe" -w --hard-mask --exposure-weight=1 --saturation-weight=0.328 --contrast-weight=0.164 -o "C:/Users/m/AppData/Local/Temp/1.tif" "C:/Users/m/AppData/Local/Temp/1.jpg" "C:/Users/m/AppData/Local/Temp/2.jpg" >>log.txt 2>&1 "
How can I find why it is not working?
Is there any way that I set the system so it doesn't close cmd window so I can inspect it?
is there any better way to run a command on OS?
Does Boost has any solution for this?
Edit
After running it with cmd /k, I get this error message:
The input line is too long.
How can I fix it other than reducing cmd line?
There are two different things here: if you have to start a suprocess, "system" is not the best way of doing it (better to use the proper API, like CreateProcess, or a multiplatform wrapper, but avoid to go through the command interpreter, to avoid to open to potential malware injection).
But in this case system() is probably the right way to go since you in fact need the command interpreter (you cannot manage things like >>log.txt 2>&1 with only a process creation.)
The problem looks like a failure in the called program: may be the path is not correct or some of the files it has to work with are not existent or accessible with appropriate-permission and so on.
One of the firt thing to do: open a command prompt and paste the string you posted, in there. Does it run? Does it say something about any error?
Another thing to check is how escape sequence are used in C++ literals: to get a '\', you need '\\' since the first is the escape for the second (like \n, or \t etc.). Although it seems not the case, here, it is one of the most common mistakes.
Use cmd /k to keep the terminal: http://ss64.com/nt/cmd.html
Or just spawn cmd.exe instead and inspect the environment, permissions, etc. You can manually paste that command to see whether it would work from that shell. If it does, you know that paths, permssions and environment are ok, so you have some other issue on your hands (argument escaping, character encoding issues)
Check here How to execute a command and get output of command within C++ using POSIX?
Boost.Process is not official yet http://www.highscore.de/boost/process/
I am developing an application for work that allows the users to quickly set environment variables on a terminal basis. By setting the path in each terminal we ensure files with the same name in different directories aren't causing application testing to be problematic. I am Using Qt to build the program which is c++ based and all the datatypes are foundationally the same.
I am using the following code to invoke commands in the terminal from which the application launches from using system(). I can run commands into the bash just fine with code; however, I run into a problem when I attempt to use a command with arguments. This is probably why source doesn't seem to work right as the source command is followed by the filename. It would appear that I drop the argument appended after the bash command.
My Code:
void assignTerminalToPath(QString path)
{
QString data = "";
QString currentUsersHomeDirectory = QDir::homePath();
QString tmpScriptLocation = currentUsersHomeDirectory;
QByteArray ba;
tmpScriptLocation += "/.tmpSourceFile";
QFile tmpSourceFile(tmpScriptLocation);
if(tmpSourceFile.open(QFile::WriteOnly | QFile::Truncate))
{
QTextStream output(&tmpSourceFile);
data.append("export PATH=.:");
data.append(path);
data.append(":$PATH");
output << QString("#!/bin/bash\n");
output << data;
tmpSourceFile.close();
}
data.clear();
data.append("/bin/bash -c source ");
data.append(tmpScriptLocation);
ba = data.toLatin1();
const char *cStr = ba.data();
system(cStr);
}
Perhaps I'm not referencing bash correctly and I need something outside of -c?
Reference Execute shell/bash command using C/C++
Thanks for any help in advance!
source is not a program that you can call, it is embedded bash command. It is designed to be processed by bash without invoking another copy of bash, such that environment variables can be changed in current bash copy.
However, you cannot call source as part of system(). And even if you did succeed at that, its effects to change environment variables would be completely lost for caller app once system() has returned.
Try a command to envelop with parameters in double quotes ("command - arg1 - to arg2") to transfer in the function system().
used:
char *com = "\"command -arg1 -arg2\"";
system(com);
I have a C++ program which is mainly used for video processing. Inside the program, I am launching the system command in order to obtain pass the processed videos to some other binaries to postprocess them.
I have a while loop towards infinite and I am launching the system command inside the loop every time. The thing is that at a certain point I am receiving the -1 return code from the system command. What could be the reason for that?
Inside the system command I am just calling a binary file with the adequate parameters from the main project.
The system command which I want to execute is actually a shell file.
In this file I am extracting the main feature from the video and passing them through a SVM model from a 3D party library in order to obtain the the desired classification.
./LiveGestureKernel ./Video ./SvmVideo
./mat4libsvm31 -l SvmVideoLabels < SvmVideo > temp_test_file
./svm-predict temp_test_file svm_model temp_output_file
cat < temp_output_file
rm -f temp_*
After a certain number of iterations through the while loop, it just won't execute the script file and I cannot figure out the reason for this. Thanks!
If you get -1 from the call to system(), you should first examine the contents of errno - that will most likely tell you what your specific problem is.
The one thing to watch out for is that the return value from system is an implementation-defined one in the case where you pass it a non-NULL command, so it's possible that -1 may be coming from your actual executable.
Your best bet in that case is to print out (or otherwise log) the command being executed on failure (and possibly all the time), so that you can check what happens with the same arguments when you execute it directly from a command line or shell.
On a Linux platform, I have C++ code that goes like this:
// ...
std::string myDir;
myDir = argv[1]; // myDir is initialized using user input from the command line.
std::string command;
command = "mkdir " + myDir;
if (system(command.c_str()) != 0) {
return 1;
}
// continue....
Is passing user input to a system() call safe at all?
Should the user input be escaped / sanitized?
How?
How could the above code be exploited for malicious purposes?
Thanks.
Just don't use system. Prefer execl.
execl ("/bin/mkdir", "mkdir", myDir, (char *)0);
That way, myDir is always passed as a single argument to mkdir, and the shell isn't involved. Note that you need to fork if you use this method.
But if this is not just an example, you should use the mkdir C function:
mkdir(myDir, someMode);
Using system() call with command line parameters without sanitizing the input can be highly insecure.
The potential security threat could be a user passing the following as directory name
somedir ; rm -rf /
To prevent this , use a mixture of the following
use getopt to ensure your input is
sanitized
sanitize the input
use execl instead of system to execute
the command
The best option would be to use all three
Further to Matthew's answer, don't spawn a shell process unless you absolutely need it. If you use a fork/execl combination, individual parameters will never be parsed so don't need to be escaped. Beware of null characters however which will still prematurely terminate the parameter (this is not a security problem in some cases).
I assume mkdir is just an example, as mkdir can trivially be called from C++ much more easily than these subprocess suggestions.
Reviving this ancient question as I ran into the same problem and the top answers, based on fork() + execl(), weren't working for me. (They create a separate process, whereas I wanted to use async to launch the command in a thread and have the system call stay in-process to share state more easily.) So I'll give an alternative solution.
It's not usually safe to pass user input as-is, especially if the utility is designed to be sudo'd; in order to sanitize it, instead of composing the string to be executed yourself, use environment variables, which the shell has built-in escape mechanisms for.
For your example:
// ...
std::string myDir;
myDir = argv[1]; // myDir is initialized using user input from the command line.
setenv("MY_DIR", myDir, 1);
if (system("mkdir \"${MY_DIR}\"") != 0) {
return 1;
}
// continue....