I want to execute multiple processes concurrently from a lua script e.g.
os.execute("cmd1")
os.execute("cmd2")
os.execute("cmd3")
where cmd1,2 and 3 are continually running processes. When i do this as above, cmd2 and 3 will only run when cmd1 is finished. Any idea on this? Should i be using "fork" or something equivalent?
Thanks in advance
You've got several solutions to your problem:
Depending on your operating system shell, you might use & to put tasks into the background. For example: os.execute('(sleep 10&& echo bar) & echo foo')
Lua Posix comes with a posix.fork() function
Lua Lanes gives you multithreading in Lua, so you might be able to just use os.execute in separate lanes (note that 'threads' in a Lua context usually refers to coroutines instead of native OS threads).
(answer mostly copied from Call popen with environment)
There is an os.spawn function in the ExtensionProposal API.
You can use it as follows:
require"ex"
local proc, err = os.spawn{
command = e.."/bin/aprogr",
args = {
"arg1",
"arg2",
-- etc
},
env = {
A = 100, -- I assume it tostrings the value
B = "Hi",
C = "Test",
},
-- you can also specify stdin, stdout, and stderr
-- see the proposal page for more info
}
if not proc then
error("Failed to aprogrinate! "..tostring(err))
end
-- if you want to wait for the process to finish:
local exitcode = proc:wait()
lua-ex-pai provides implementations for POSIX and Windows.
It allows the spawning of multiple concurrent processes.
You can find precompiled binaries of this implementation bundled with the LuaForWindows distribution.
Here is a more concise version of your use case:
require"ex"
local cmd1_out = io.pipe()
local cmd1_proc = assert(os.spawn("cmd", {
stdout = cmd1_out,
}))
local cmd2_out = io.pipe()
local cmd2_proc = assert(os.spawn("cmd", {
stdout = cmd1_out,
}))
-- perform actions with cmd1 and cmd2
That's because Lua is single threaded. To run it concurrently you'll need to provide a multi-threaded solution for Lua (not coroutines, because they're microthreads!), like lua pthreads.
Try simply adding & at the end of your commands:
os.execute("cmd1 &")
os.execute("cmd2 &")
os.execute("cmd3 &")
This should work on an operative system. On windows there might be a way to to the same, but I have no idea of what it is.
Related
I have a program that calls a shell script tool that I made that goes through a directory and zips up files and gets the checksum value and calls some other tools to upload the files. The operation takes roughly 3 to 4 minutes.
I call the script like this:
int result = system("/bin/sh /path/to/my/script");
I've also got the same result by using the exec() family of functions:
int child = fork();
if(child == 0) {
execl( "/bin/sh", "sh", "/path/to/my/script", (char*)0 );
}
I know with exec you can redirect output to the parent program so it can read the output of the command line tools, but other than that when should you use system as opposed to exec?
Ignoring for the time being that use of system is portable while use of exec family of functions is not portable...
When you combine use of exec family of functions with other POSIX functions such as pipe, dup, wait, you get a lot more control over how to pass data between the parent process and the child process.
When you don't need any of those controls, i.e. you just want to execute a command, then using system is preferable, IMO.
The first system call in your question will do the same, what you are doing in the next piece of code (fork and execl)
From documentation:
The system() library function uses fork(2) to create a child process
that executes the shell command specified in command using execl(3)
http://man7.org/linux/man-pages/man3/system.3.html
We have a C++ program, sometimes this program need to execute a user defined batch/shell/ant script. We are not able to control how this script runs. Is there a way to get the return code from C++ program?
Something like:
exec a script.sh > status.tmp ?
We need to support both Windows and Linux.
Any ideas?
Another simple of going this is get the return using the marco WEXITSTATUS. Pretty much the same way that you get return values of child process using waitpid call (in Unix based systems).
Here is the sample program. I have one C/C++ program, and one simple bash script.
Sample bash script
#!/bin/bash
echo "I am in Script"
exit 5;
Sample C/C++ program
int i, ret = system("./b.sh 2>&1 > /dev/null");
i=WEXITSTATUS(ret);
printf("My val= %d\n",i);
Output
./a.out
My val= 5
If you want more advanced approach to have multiple return code from the script or want an interactive session then perhaps you should use popen
Hope this helps.
in linux just use
int ret=system("myshellscrtipt.sh");
since the return value of the script is the return value of the system function.
In Windows I'dont't know wether there is a similar function.
If you used the Qt toolkit you could do something like this
QProcess process;
process.start( "yourShellCommand", QStringList( args );
and this would be really cross-platform..
In bash the status code is stored in a special variable:
C:/myprogram.exe
echo $?
On a Linux platform, I have C++ code that goes like this:
// ...
std::string myDir;
myDir = argv[1]; // myDir is initialized using user input from the command line.
std::string command;
command = "mkdir " + myDir;
if (system(command.c_str()) != 0) {
return 1;
}
// continue....
Is passing user input to a system() call safe at all?
Should the user input be escaped / sanitized?
How?
How could the above code be exploited for malicious purposes?
Thanks.
Just don't use system. Prefer execl.
execl ("/bin/mkdir", "mkdir", myDir, (char *)0);
That way, myDir is always passed as a single argument to mkdir, and the shell isn't involved. Note that you need to fork if you use this method.
But if this is not just an example, you should use the mkdir C function:
mkdir(myDir, someMode);
Using system() call with command line parameters without sanitizing the input can be highly insecure.
The potential security threat could be a user passing the following as directory name
somedir ; rm -rf /
To prevent this , use a mixture of the following
use getopt to ensure your input is
sanitized
sanitize the input
use execl instead of system to execute
the command
The best option would be to use all three
Further to Matthew's answer, don't spawn a shell process unless you absolutely need it. If you use a fork/execl combination, individual parameters will never be parsed so don't need to be escaped. Beware of null characters however which will still prematurely terminate the parameter (this is not a security problem in some cases).
I assume mkdir is just an example, as mkdir can trivially be called from C++ much more easily than these subprocess suggestions.
Reviving this ancient question as I ran into the same problem and the top answers, based on fork() + execl(), weren't working for me. (They create a separate process, whereas I wanted to use async to launch the command in a thread and have the system call stay in-process to share state more easily.) So I'll give an alternative solution.
It's not usually safe to pass user input as-is, especially if the utility is designed to be sudo'd; in order to sanitize it, instead of composing the string to be executed yourself, use environment variables, which the shell has built-in escape mechanisms for.
For your example:
// ...
std::string myDir;
myDir = argv[1]; // myDir is initialized using user input from the command line.
setenv("MY_DIR", myDir, 1);
if (system("mkdir \"${MY_DIR}\"") != 0) {
return 1;
}
// continue....
I'm working on a init for an initramfs in C++ for Linux. This script is used to unlock the DM-Crypt w/ LUKS encrypted drive, and set the LVM drives to be available.
Since I don't want to have to reimplement the functionality of cryptsetup and gpg I am using system calls to call the executables. Using a system call to call gpg works fine if I have the system fully brought up already (I already have a bash script based initramfs that works fine in bringing it up, and I use grub to edit the command line to bring it up using the old initramfs). However, in the initramfs it never even acts like it gets called. Even commands like system("echo BLAH"); fail.
So, does anyone have any input?
Edit: So I figured out what was causing my errors. I have no clue as to why it would cause errors, but I found it.
In order to allow hotplugging, I needed to write /sbin/mdev to /proc/sys/kernel/hotplug...however I ended up switching around the parameters (on a function I wrote myself no less) so I was writing /proc/sys/kernel/hotplug to /sbin/mdev.
I have no clue as to why that would cause the problem, however it did.
Amardeep is right, system() on POSIX type systems runs the command through /bin/sh.
I doubt you actually have a legitimate need to invoke these programs you speak of through a Bourne shell. A good reason would be if you needed them to have the default set of environment variables, but since /etc/profile is probably also unavailable so early in the boot process, I don't see how that can be the case here.
Instead, use the standard fork()/exec() pattern:
int system_alternative(const char* pgm, char *const argv[])
{
pid_t pid = fork();
if (pid > 0) {
// We're the parent, so wait for child to finish
int status;
waitpid(pid, &status, 0);
return status;
}
else if (pid == 0) {
// We're the child, so run the specified program. Our exit status will
// be that of the child program unless the execv() syscall fails.
return execv(pgm, argv);
}
else {
// Something horrible happened, like system out of memory
return -1;
}
}
If you need to read stdout from the called process or send data to its stdin, you'll need to do some standard handle redirection via pipe() or dup2() in there.
You can learn all about this sort of thing in any good Unix programming book. I recommend Advanced Programming in the UNIX Environment by W. Richard Stevens. The second edition coauthored by Rago adds material to cover platforms that appeared since Stevens wrote the first edition, like Linux and OS X, but basics like this haven't changed since the original edition.
I believe the system() function executes your command in a shell. Is the shell executable mounted and available that early in your startup process? You might want to look into using fork() and execve().
EDIT: Be sure your cryptography tools are also on a mounted volume.
what do you have in initramfs ? You could do the following :
int main() {
return system("echo hello world");
}
And then strace it in an initscript like this :
strace -o myprog.log myprog
Look at the log once your system is booted
I'm currently developing an application that happens to require some file preprocessing before actually reading the data.
Doing it externally was not a possibility so I came up with a fork & execve of "cut options filename | sort | uniq -c" etc... and I execute it like that.
However I thought that maybe there was already another option to reuse all those ancient and good working tools directly in my code and not having to invoke them through a shell.
I am currently looking at busybox to see if there is an easy way of statically link and programatically call those utils but no luck yet.
Arkaitz, the answer no, because of how you've phrased the question.
You ask for "another option to reuse all those ancient and good working tools directly in my code and not having to invoke them through a shell"
The problem with that is, the proper and accepted way of reusing all those ancient and good working tools is exactly what you're saying you want to avoid - invoking them via a shell (or at least, firing them up as child processes via popen for example) - and it's definitely not recommend to try to subsume, copy, or duplicate these tools into your code.
The UNIX (and Linux) model for data manipulation is robust and proven - why would you want to avoid it?
The 'ancient' tools were built for use by the shell, not to be built/linked into an executable. There are, however, more recent tools that kinda do lot of what you showed on your command line preprocessor: iostreams with extractors (to replace cut), std::sort and std::unique to replace the respective programs...
struct S { string col1, col3;
bool operator<( const S& s ) { return col1 < s.col1; }
};
vector<S> v;
while( cin ) {
S s;
string dummy;
cin >> s.col1 >> dummy >> col3 >> dummy;
v.push_back( s );
}
sort(v.begin(), v.end(), S::smaller );
unique( v.begin(), v.end() );
Not too complicated, I think.
Try popen().
char buffer [ BUFFER_SIZE ];
FILE * f = popen( "cut options filename | sort | uniq -c", "r" );
while( /*NOT*/! feof(f) )
fgets( buffer, BUFFER_SIZE, f );
pclose( f );
Reference: How to execute a command and get output of command within C++ using POSIX?
You have to do it through the shell, but it's easier to use "system" call.
while(something) {
int ret = system("foo");
if (WIFSIGNALED(ret) &&
(WTERMSIG(ret) == SIGINT || WTERMSIG(ret) == SIGQUIT))
break;
}
Just write another useful 'ancient and good' tool ;) and read all data from stdin and return it to stdout.
cat *.txt | grep 'blabla' | sort | my_new_tool | tee -o res_file
The nice way to do it is:
Create 2 pipes
Fork a new process
Replace stdin and stdout for child process with pipes using dup2 function
exec a command you'd like
Write and read from parent process using pipes
busybox was my first thought as well, although you might also want to consider embedding a scripting engine like Python and doing these kind of manipulations in Python scripts.
I would definitely not try to strip this kind of functionality out of GNU command line tools since they have grown significantly since the early UNIX days and sprouted an awful lot of options.
If the busybox code seems too hard to adapt, then the next place I would look would be Minix source code. Look under Previous Versions and pick one of the version 1 or 2 Minixes because those were written as teaching code so they tend to be clearer and simpler.
If you do not want to call external commands (whether by exec, popen or system etc) but do not want to modify the source of these utilities and compile them into your code (relatively easy, just change 'main' to 'main_cut' etc), then the only remaining option I see is to embed the utilities inside your code and either extract them at runtime or dynamically create a filing system by pointing at the data inside your code (eg using a floppy or cd image and writing a FUSE module that picks up the disk image data from a ram address). All of which seems like a lot of work just to make this look like a single neatly-packaged utility.
Personally, if i really had to do this, I'd get the source of all those utils and compile them in as external calls. Of course you'd no longer have pipes easily available, you'd either have to use temp files for preprocessing, or something more complicated involving co-routines. Or maybe sockets. Lots of work and messy whatever you do!