I have a program that needs to run a program we'll call externalProg in parallel on our linux (CentOS) cluster - or rather, it needs to run many instances of externalProg, each on different cores. Each "thread" creates 3 files based on a few parameters - the inputs to externalProg, a command file to tell externalProg how to execute my file, and a bash script to set up the environment (calls a setup script provided by the manufacturer) and actually call externalProg with my inputs.
Since this needs to be parallel with an unknown number of concurrent threads and I don't want to risk overwriting another thread's files, I am creating temp files using
mkstemp("PREFIX_XXXXXX")
for these input files. After the external program runs, I extract the relevant data and store it, and close the temp files (therefore deleting them).
We'll call the files created (Which actually have a name based on the template above)
tmpInputs - Inputs to externalProg
tmpCommand - Input that tells externalProg how to execute tmpInputs
tmpBash - bash script to set up and call externalProg with my inputs
The file tmpBash looks something like
source /path/to/setup/script # Sets up environment variables
externalProg < /path/to/tmpCommand
where tmpCommand is just a simple text file.
The problem I'm having is actually executing the bash script. Within my program, I call
ostringstream launchcmd;
launchcmd << "bash " << path_to_tmpBash
system(launchcmd.str().c_str());
But nothing happens. No error, no warning, no 'file not found' or permission denied or anything. I have verified that the files are being created and have the correct content. The rest of the code after system() is executed successfully (Though it fails since externalProg wasn't run).
Strangely, if I go back to the terminal and type
bash /path/to/tmpBash
then externalProg is executed successfully. I have also cout'd the launchcmd string, copy and pasted that in to the terminal, which also works successfully. For some reason, this only fails when called within my program.
After a bit of experimentation, I've determined that system() calls /bin/sh on our cluster. If I change launchcmd to look like
/path/to/tmpBash
(So that the full command should look like /bin/sh /path/to/tmpBash), I get a permission denied error, which is no surprise. The problem is that I can't chmod +x the tmpBash file while it's still open, and if I close the file, it gets deleted - so I'm not sure how to address that.
Is there something obviously wrong I'm doing, or does system() have some nuance that I'm missing?
edit: I wanted to add that I can successfully call things like
system("echo $PATH")
and get the expected results (in this case, my default $PATH).
Two separate ideas:
Change your SHELL environment variable to be /bin/bash, then call system(),
or:
Use execve directly `execve('/bin/bash', ['/path/to/tmpBash'], environ)
Related
I am trying to mount ecryptfs from within a C++ program. I can definitely mount it without it asking questions by issuing this command at the prompt:
sudo mount -t ecryptfs -o "rw,key=passphrase:passphrase_passwd=geoff,ecryptfs_cipher=aes,ecryptfs_key_bytes=32,ecryptfs_passthrough=n,ecryptfs_enable_filename_crypto=n,no_sig_cache" ~/source/ ~/target/
Note that in reality, I am passing a full canonical path in case that matters.
But from within the program I get failure with errno=EINVAL after trying by using the mount() function with the same arguments:
mount("~/source/", "~/target/", "ecryptfs", MS_NODEV, "rw,key=passphrase:passphrase_passwd=geoff,ecryptfs_cipher=aes,ecryptfs_key_bytes=32,ecryptfs_passthrough=n,ecryptfs_enable_filename_crypto=n,no_sig_cache")
The program does launch with root privileges and I have checked that I have CAP_SYS_ADMIN.
The mount() function returns -1 and sets errno to EINVAL.
Have I got the arguments correct? Is this maybe a privileges issue?
EDIT: I got it to work by executing mount externally via system(), but would still like to use the function because of reasons.
I believe this is because mount -t ecryptfs is actually calling the helper executable mount.ecryptfs, and it's processing some of the options (in particular, key=) itself. What's actually passed to the kernel is different (you can see this by looking at /proc/mounts afterward).
If you look closely at https://manpages.ubuntu.com/manpages/kinetic/en/man7/ecryptfs.7.html, key= and ecryptfs_enable_filename_crypto= are listed under "MOUNT HELPER OPTIONS" - the actual kernel module's options are ecryptfs_sig=(fekek_sig) and ecryptfs_fnek_sig=(fnek_sig).
So, if you want to bypass the helper and do the mount directly, you'd need to load the tokens into the kernel's keyring with https://man7.org/linux/man-pages/man2/keyctl.2.html and replace key= with the resulting token signatures, like mount.ecryptfs did.
It does appear that there is a libecrytpfs with functions in ecryptfs.h like ecryptfs_add_passphrase_key_to_keyring which you can (presumably, not tested) use to do this in a way matching the mount.ecryptfs
I have a c++ program that run a command and pass some arguments to it. The code is as follow:
int RunApplication(fs::path applicationPathName,std::string arguments)
{
std::string applicationShortPath=GetShortFileName(applicationPathName);
std::string cmd="\""+applicationShortPath +"\" "+ arguments+" >>log.txt 2>&1 \"";
std::cout<<cmd<<std::endl;
int result=std::system(cmd.c_str());
return result;
}
When I run system command, the cmd window appears shortly and then closes, but the result is 1 and the cmd was not run (the command should generate output which is not generated).
To check that the cmd is correct, I stopped the application just before system line and copy/ paste cmd content to a cmd window and it worked.
I am wondering how can I find why application is not run in system()?
the cmd has this value just before running it:
"D:/DEVELO~3/x64/Debug/enfuse.exe" -w --hard-mask --exposure-weight=1 --saturation-weight=0.328 --contrast-weight=0.164 -o "C:/Users/m/AppData/Local/Temp/1.tif" "C:/Users/m/AppData/Local/Temp/1.jpg" "C:/Users/m/AppData/Local/Temp/2.jpg" >>log.txt 2>&1 "
How can I find why it is not working?
Is there any way that I set the system so it doesn't close cmd window so I can inspect it?
is there any better way to run a command on OS?
Does Boost has any solution for this?
Edit
After running it with cmd /k, I get this error message:
The input line is too long.
How can I fix it other than reducing cmd line?
There are two different things here: if you have to start a suprocess, "system" is not the best way of doing it (better to use the proper API, like CreateProcess, or a multiplatform wrapper, but avoid to go through the command interpreter, to avoid to open to potential malware injection).
But in this case system() is probably the right way to go since you in fact need the command interpreter (you cannot manage things like >>log.txt 2>&1 with only a process creation.)
The problem looks like a failure in the called program: may be the path is not correct or some of the files it has to work with are not existent or accessible with appropriate-permission and so on.
One of the firt thing to do: open a command prompt and paste the string you posted, in there. Does it run? Does it say something about any error?
Another thing to check is how escape sequence are used in C++ literals: to get a '\', you need '\\' since the first is the escape for the second (like \n, or \t etc.). Although it seems not the case, here, it is one of the most common mistakes.
Use cmd /k to keep the terminal: http://ss64.com/nt/cmd.html
Or just spawn cmd.exe instead and inspect the environment, permissions, etc. You can manually paste that command to see whether it would work from that shell. If it does, you know that paths, permssions and environment are ok, so you have some other issue on your hands (argument escaping, character encoding issues)
Check here How to execute a command and get output of command within C++ using POSIX?
Boost.Process is not official yet http://www.highscore.de/boost/process/
The goal of this code is to create a stack trace whenever a sigterm/sigint/sigsegv/etc is caught. In order to not rely on memory management inside of the C++ code in the case of a sigsegv, I have decided to write a bash script that will receive the PID and memory addresses in the trace array.
The Sig events are being caught.
Below is where I generate the call to the bash script
trace_size = backtrace(trace, 16);
trace[1] = (void *)ctx->rsi;
messages = backtrace_symbols(trace, trace_size);
char syscom[356] = {0};
sprintf(syscom,"bash_crash_supp.sh %d", getpid());
for (i=1; i<(trace_size-1) && i < 10; ++i)
{
sprintf(syscom,"%s %p",syscom,trace[i]);
}
Below is where my issue arises. The command in syscom is generating correctly. I can stop the code before the following popen, run the command in a terminal, and it functions correctly.
However running the script directly from the c++ code does not seem to work.
setuid(0);
FILE *bashCommand = popen(syscom,"r");
char buf[256] = {0};
while(fgets(buf,sizeof(buf),bashCommand) != 0) {
LogMessage(LOG_WARNING, "%s\n", buf);
}
pclose(bashCommand);
exit(sig);
The purpose of the bash script is to get the offset from /proc/pid/maps, and then use that to run addr2line to get the file name/line number.
strResult=$(sudo cat /proc/"$1"/maps | grep "target_file" | grep -m 1 '[0-9a-fA-F]')
offset=$( cut -d '-' -f 1 <<< "$strResult");
However offset is getting 0 when I run from the c++ code, but when I run the exact same command (that is stored in syscom in the c++ code) in a terminal, I get the expected output.
I have been trying to fix this for a while. Permissions are most likely the issue, but I've tried to work around these with every way I know of/have seen via google. The user trying to run the script (currently running the c++ code) is apache.
The fix does not need to worry about the security of the box. If something as simple as "chmod 777 /proc -r" worked, that would have been the solution (sadly the OS doesn't let me mess do such commands with /proc).
Things I've already tried:
Adding `` around the command that's stored in syscom from the c++ code
chown the script to apache
chmod 4755 on the bash_crash_supp.sh script, allowing it to always fire as root.
I have added apache to sudoers (visudo), allowing them to run sudo without using a password
I have added a sub file to sudoers.d (just in case) that does the same as above
I have looked into objdump, however it does not give me either the offset or the file/line num for an addr (from what I can see)
I have setuid(0) in the c++ code to set the current user to root
Command generated in C++
bash_crash_supp.sh 25817 0x7f4bfe600ec8 0x7f4bf28f7400 0x7f4bf28f83c6 0x7f4bf2904f02 0x7f4bfdf0fbb0 0x7f4bfdf1346e 0x7f4bfdf1eb30 0x7f4bfdf1b9a8 0x7f4bfdf176b8
Params in bash:
25817 0x7f4bfe600ec8 0x7f4bf28f7400 0x7f4bf28f83c6 0x7f4bf2904f02 0x7f4bfdf0fbb0 0x7f4bfdf1346e 0x7f4bfdf1eb30 0x7f4bfdf1b9a8 0x7f4bfdf176b8
Can anyone think of any other ways to solve this?
Long story short, almost all Unix-based systems ignore setuid on any interpreted script (anything with a shebang #!) as a security precaution.
You may use setuid on actual executables, but not the shell scripts themselves. If you're willing to take a massive security risk, you can make a wrapper executable to run the shell script and give the executable setuid.
For more information, see this question on the Unix StackExchange: https://unix.stackexchange.com/a/2910
I have been having problems trying to run an external program from a python program that was generated from a trigger in a postgres 9.2 database. The trigger works. It writes to a file. I had tried just running the external program but the permissions would not allow it to run. I was able to create a folder (using os.system(“mkdir”) ). The owner of the folder is NETWORK SERVICE.
I need to run a program called sdktest. When I try to run it no response happens so I think that means that the python program does not have enough permissions (with an owner of NETWORK SERVICE) to run it.
I have been having my program copy files that it needs into that directory so they would have the correct permissions and that has worked to some degree but the program that I need to run is the last one and it is not running because it does not have enough permissions.
My python program runs a C++ program called PG_QB_Connector which calls sdktest.
Is there any way I can change the owner of the process to be a “normal” owner? Is there a better way to do this? Basically I just need to have this C++ program have eniough perms to run correctly.
BTW, when I run the C++ program by hand, the line that runs the sdktest program runs correctly, however, when I run it from the postgres/python it does not do anything...
I have Windows 7, python 3.2. The other 2 questions that I asked about this are located here and here
The python program:
CREATE or replace FUNCTION scalesmyone (thename text)
RETURNS int
AS $$
a=5
f = open('C:\\JUNK\\frompython.txt','w')
f.write(thename)
f.close()
import os
os.system('"mkdir C:\\TEMPWITHOWNER"')
os.system('"mkdir C:\\TEMPWITHOWNER\\addcustomer"')
os.system('"copy C:\\JUNK\\junk.txt C:\\TEMPWITHOWNER\\addcustomer"')
os.system('"copy C:\\BATfiles\\junk6.txt C:\\TEMPWITHOWNER\\addcustomer"')
os.system('"copy C:\\BATfiles\\run_addcust.bat C:\\TEMPWITHOWNER\\addcustomer"')
os.system('"copy C:\\Workfiles\\PG_QB_Connector.exe C:\\TEMPWITHOWNER\\addcustomer"')
os.system('"copy C:\\Workfiles\\sdktest.exe C:\\TEMPWITHOWNER\\addcustomer"')
import subprocess
return_code = subprocess.call(["C:\\TEMPWITHOWNER\\addcustomer\\PG_QB_Connector.exe", '"hello"'])
$$ LANGUAGE plpython3u;
The C++ program that is called from the python program and calls sdktest.exe is below
command = "copy C:\\Workfiles\\AddCustomerFROMWEB.xml C:\\TEMPWITHOWNER\\addcustomer\\AddCustomerFROMWEB.xml";
system(command.c_str());
//everything except for the qb file is in my local folder
command = "C:\\TEMPWITHOWNER\\addcustomer\\sdktest.exe \"C:\\Users\\Public\\Documents\\Intuit\\QuickBooks\\Company Files\\Shain Software.qbw\" C:\\TEMPWITHOWNER\\addcustomer\\AddCustomerFROMWEB.xml C:\\TEMPWITHOWNER\\addcustomer\\outputfromsdktestofaddcust.xml";
system(command.c_str());
It sounds like you want to invoke a command-line program from within a PostgreSQL trigger or function.
A usually-better alternative is to have the trigger send a NOTIFY and have a process with a PostgreSQL connection LISTENing for notifications. When a notification comes in, the process can start your program. This is the approach I would recommend; it's a lot cleaner and it means your program doesn't have to run under PostgreSQL's user ID. See NOTIFY and LISTEN.
If you really need to run commands from inside Pg:
You can use PL/Pythonu with os.system or subprocess.check_call; PL/Perlu with system(); etc. All these can run commands from inside Pg if you need to. You can't invoke programs directly from PostgreSQL, you need to use one of the 'untrusted' (meaning fully privileged, not sandboxed) procedural languages to invoke external executables. PL/TCL can probably do it too.
Update:
Your Python code as shown above has several problems:
Using os.system in Python to copy files is just wrong. Use the shutil library: http://docs.python.org/3/library/shutil.html to copy files, and the simple os.mkdir command to create directories.
The double-layered quoting looks wrong; didn't you mean to quote only each argument not the whole command? You should be using subprocess.call instead of os.system anyway.
Your final subprocess.call invocation appears OK, but fails to check the error code so you'll never know if it went wrong; you should use subprocess.check_call instead.
The C++ code also appears to fail to check for errors from the system() invocations so you'll never know if the command it runs fails.
Like the Python code, copying files in C++ by using the copy shell command is generally wrong. Microsoft Windows provides the CopyFile function for this; equivalents or alternatives exist on other platforms and you can use portable-but-less-efficient stream copying too.
I have a C++ program which is mainly used for video processing. Inside the program, I am launching the system command in order to obtain pass the processed videos to some other binaries to postprocess them.
I have a while loop towards infinite and I am launching the system command inside the loop every time. The thing is that at a certain point I am receiving the -1 return code from the system command. What could be the reason for that?
Inside the system command I am just calling a binary file with the adequate parameters from the main project.
The system command which I want to execute is actually a shell file.
In this file I am extracting the main feature from the video and passing them through a SVM model from a 3D party library in order to obtain the the desired classification.
./LiveGestureKernel ./Video ./SvmVideo
./mat4libsvm31 -l SvmVideoLabels < SvmVideo > temp_test_file
./svm-predict temp_test_file svm_model temp_output_file
cat < temp_output_file
rm -f temp_*
After a certain number of iterations through the while loop, it just won't execute the script file and I cannot figure out the reason for this. Thanks!
If you get -1 from the call to system(), you should first examine the contents of errno - that will most likely tell you what your specific problem is.
The one thing to watch out for is that the return value from system is an implementation-defined one in the case where you pass it a non-NULL command, so it's possible that -1 may be coming from your actual executable.
Your best bet in that case is to print out (or otherwise log) the command being executed on failure (and possibly all the time), so that you can check what happens with the same arguments when you execute it directly from a command line or shell.