I am learning C++ [Java background fwiw] and trying to write a UNIX shell as a project. I am running into a funny little problem with tokenizing the input for execution. The tok function is getting called twice and I'm not sure why. My current test code is the following:
#include <iostream>
#include <vector>
#include <sstream>
#include <unistd.h>
#include <signal.h>
#include <string.h>
#include <sys/wait.h>
#include <sys/types.h>
using namespace std;
void tok(string, char**);
int main(){
const char* EXIT = "exit";
string input;
cout << "shell>> ";
getline(cin, input);
pid_t pid = fork();
char* args[64]; //arbitrary size, 64 possible whitespace-delimited tokens in command
tok(input, args);
return 0;
}
//copied from http://stackoverflow.com/questions/14265581/parse-split-a-string-in-c-using-string-delimiter-standard-c
void tok(string inStr, char** args){
int last = 0, next = 0, i = 0;
while( (next = inStr.find(' ', last)) != -1){
cout << i++ << ": " << inStr.substr(last, next-last) << endl;
*args++ = strdup(inStr.substr(last, next-last).c_str());
last = next + 1;
}
cout << i++ << ": " << inStr.substr(last) << endl;
*args++ = strdup(inStr.substr(last).c_str());
*args = '\0';
cout << "done tokenizing..." << endl;
}
My output when I actually run the program is:
$ ./a.out
shell>> ls -l
0: ls
1: -l
done tokenizing...
0: ls
1: -l
done tokenizing...
I'm not sure why it would do that. Can anyone guide me in the right direction please? Thank you
The fork function returns twice, once in the original process and once in the newly-created, forked process. Both of those processes then call tok.
There doesn't seem to be any clear reason why you called fork. So the fix may be as simple as eliminating the call to fork.
When you call fork, you create two processes. Each process has nearly the exact same state except for the respective pid_t you receive. If that value is greater than 0, then you are in the parent process (main), and otherwise you are in the child (or fork failed).
Without performing a check on the returned pid_t, both processes will call tok, resulting in the double call behavior you witnessed.
Hide the call behind a check on pid like so:
pid_t pid = fork();
if (pid > 0) // have parent process call tok
{
char* args[64]; //arbitrary size, 64 possible whitespace-delimited tokens in command
tok(input, args);
}
To see what else parent and child processes have in common (or not): check the docs
following code may work fine
#include <iostream>
#include <vector>
#include <sstream>
#include <unistd.h>
#include <signal.h>
#include <string.h>
#include <sys/wait.h>
#include <sys/types.h>
using namespace std;
void tok(string, char**);
int main(){
const char* EXIT = "exit";
string input;
cout << "shell>> ";
getline(cin, input);
// pid_t pid = fork();
char* args[64];
tok(input, args);
return 0;
}
void tok(string inStr, char** args){
int last = 0, next = 0, i = 0;
while( (next = inStr.find(' ', last)) != -1){
cout << i++ << ": " << inStr.substr(last, next-last) << endl;
*args++ = strdup(inStr.substr(last, next-last).c_str());
last = next + 1;
}
cout << i++ << ": " << inStr.substr(last) << endl;
*args++ = strdup(inStr.substr(last).c_str());
*args = '\0';
cout << "done tokenizing..." << endl;
}
Related
I am new to C++ and Linux and I am confused on how to correctly pass an integer parameter using execlp() to a child class. I tried following the parameter requirements for this system call, however, the argument is not passing the correct value when I am executing the program. The char conversions is what is making me confused.
In the programs below, the parent accepts gender name pairs from the terminal. Next, it uses the fork() and exec() system calls where it passes the child number, gender, and name to the child program. The child program will output the statement. The output for the child numbers is off (should be: 1,2,3,4,...ect.). However, the output value is blank. Is it because of how I am initializing the argument in the execlp() system call?
Below is my code for the parent program - parent.cc:
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>
#include <iostream>
using namespace std;
int main (int argc, char* argv[])
{
int i, num = 0;
/* Holds the value of the number of pairs */
int pairs = (argc - 1) / 2;
pid_t pid;
cout << "I have " << pairs << " children." << endl;
/* Perform all child process depending on the number of name-gender pairs*/
for (i = 1; i <= argc-1; i+=2)
{
pid = fork();
/* Child process */
if (pid == 0)
{
char n[] = {char(num++)};
execlp("./c",n,argv[i],argv[i+1], NULL);
}
/* Parent will wait until all child processes finish */
wait(NULL);
}
cout << "All child process terminated. Parent exits." << endl;
/* Exits program */
return 0;
}
Here is my child program- child.cc
#include <iostream>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
using namespace std;
int main(int argc, char* argv[])
{
/* When child process starts, it will print out this statement */
cout << "Child # " << argv[0] << ": I am a " << argv[1] << ", and my name is " << argv[2] << endl;
exit(0);
}
Here is my output in the terminal:
g++ -o parent.cc
g++ -o c child.cc
./p boy Mark girl Emily boy Daniel girl Hailey
I have 4 children.
Child # : I am a boy, and my name is Mark
Child # : I am a girl, and my name is Emily
Child # : I am a boy, and my name is Daniel
Child # : I am a girl, and my Hailey
All child process terminated. Parent exits.
There's two and a half issues:
You are not correctly converting an integer to a string, so the value is wrong
You are incrementing the number in the child process, so the next child process won't see the update
argv[0] is conventionally the program name, and failing to follow this convention means that e.g. ./c 1 boy Mark will not work in a shell, and that the child process can't easily be substituted for something written in a different language.
Here's the updated parent process:
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>
#include <iostream>
using namespace std;
int main (int argc, char* argv[])
{
int i, num = 0;
/* Holds the value of the number of pairs */
int pairs = (argc - 1) / 2;
pid_t pid;
cout << "I have " << pairs << " children." << endl;
/* Perform all child process depending on the number of name-gender pairs*/
for (i = 1; i <= argc-1; i+=2)
{
pid = fork();
/* Child process */
if (pid == 0)
{
string str = to_string(num);
execlp("./c",
"./c", // Conventionally the first argument is the program name
str.c_str(), // Pass in the correctly formatted number
argv[i],argv[i+1], NULL);
}
// Increment in the parent process so that the change is not lost
num++;
/* Parent will wait until all child processes finish */
wait(NULL);
}
cout << "All child process terminated. Parent exits." << endl;
/* Exits program */
return 0;
}
Accordingly, the child should access argv[1], 2, and 3, and not 0, 1 and 2.
I'm trying to port a program from Windows to Linux.
I encountered a problem when I found out that there isn't a "real" ReadProcessMemory counterpart on Linux; I searched for an alternative and I found ptrace, a powerful process debugger.
I quickly coded two small console applications in C++ to test ptrace, before using it in the program.
TestApp
This is the tracee; it keeps printing two integers every 50 milliseconds while increasing their value by 1 every time.
#include <QCoreApplication>
#include <QThread>
#include <iostream>
using namespace std;
class Sleeper : public QThread
{
public:
static void usleep(unsigned long usecs){QThread::usleep(usecs);}
static void msleep(unsigned long msecs){QThread::msleep(msecs);}
static void sleep(unsigned long secs){QThread::sleep(secs);}
};
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
int value = 145;
int i = 0;
do {
cout << "i: " << i << " " << "Value: " << value << endl;
value++;
i++;
Sleeper::msleep(50);
} while (true);
return a.exec();
}
MemoryTest
This is the tracer; it asks for the process name and retrieves the PID using the command pidof -s, then ptrace attaches to the process and retrieves the memory address' value every 500 milliseconds, for 10 times.
#include <QCoreApplication>
#include <QThread>
#include <iostream>
#include <string>
#include <sys/ptrace.h>
#include <errno.h>
using namespace std;
class Sleeper : public QThread
{
public:
static void usleep(unsigned long usecs){QThread::usleep(usecs);}
static void msleep(unsigned long msecs){QThread::msleep(msecs);}
static void sleep(unsigned long secs){QThread::sleep(secs);}
};
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
char process_name[50];
cout << "Process name: ";
cin >> process_name;
char command[sizeof(process_name) + sizeof("pidof -s ")];
snprintf(command, sizeof(command), "pidof -s %s", process_name);
FILE* shell = popen(command, "r");
char pidI[sizeof(shell)];
fgets(pidI, sizeof(pidI), shell);
pclose(shell);
pid_t pid = atoi(pidI);
cout << "The PID is " << pid << endl;
long status = ptrace(PTRACE_ATTACH, pid, NULL, NULL);
cout << "Status: " << status << endl;
cout << "Error: " << errno << endl;
unsigned long addr = 0x012345; // Example address, not the true one
int i = 0;
do {
status = ptrace(PTRACE_PEEKDATA, pid, addr, NULL);
cout << "Status: " << status << endl;
cout << "Error: " << errno << endl;
i++;
Sleeper::msleep(500);
} while (i < 10);
status = ptrace(PTRACE_DETACH, pid, NULL, NULL);
cout << "Status: " << status << endl;
cout << "Error: " << errno << endl;
return a.exec();
}
Everything works fine, but TestApp is paused (SIGSTOP) until ptrace detaches from it.
Also, when it attaches to the process, the status is 0 and the error is 2; the first time it tries to retrieve the memory address value it fails with status -1 and error 3. Is it normal?
Is there a way to prevent ptrace from sending the SIGSTOP signal to the process?
I already tried using PTRACE_SEIZE instead of PTRACE_ATTACH, but it doesn't work: status -1 and error 3.
Update: Using Sleeper in MemoryTest before the "do-while" loop fixes the problem of the first memory address value retrieval, even if the value of seconds, milliseconds or microseconds is 0. Why?
After a lot of research I'm pretty sure that there isn't a way to use ptrace without stopping the process.
I found a real ReadProcessMemory counterpart, called process_vm_readv, which is much more simple.
I'm posting the code in the hope of helping someone who is in my (previous) situation.
Many thanks to mkrautz for his help coding MemoryTest with this beautiful function.
#include <QCoreApplication>
#include <QThread>
#include <sys/uio.h>
#include <stdint.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <iostream>
using namespace std;
class Sleeper : public QThread
{
public:
static void usleep(unsigned long usecs){QThread::usleep(usecs);}
static void msleep(unsigned long msecs){QThread::msleep(msecs);}
static void sleep(unsigned long secs){QThread::sleep(secs);}
};
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
char process_name[50];
cout << "Process name: ";
cin >> process_name;
char command[sizeof(process_name) + sizeof("pidof -s ")];
snprintf(command, sizeof(command), "pidof -s %s", process_name);
FILE* shell = popen(command, "r");
char pidI[sizeof(shell)];
fgets(pidI, sizeof(pidI), shell);
pclose(shell);
pid_t pid = atoi(pidI);
cout << "The PID is " << pid << endl;
if (pid == 0)
return false;
struct iovec in;
in.iov_base = (void *) 0x012345; // Example address, not the true one
in.iov_len = 4;
uint32_t foo;
struct iovec out;
out.iov_base = &foo;
out.iov_len = sizeof(foo);
do {
ssize_t nread = process_vm_readv(pid, &out, 1, &in, 1, 0);
if (nread == -1) {
fprintf(stderr, "error: %s", strerror(errno));
} else if (nread != in.iov_len) {
fprintf(stderr, "error: short read of %li bytes", (ssize_t)nread);
}
cout << foo << endl;
Sleeper::msleep(500);
} while (true);
return a.exec();
}
Davide,
Have you had a look at the /proc filesystem? It contains memory map files that can be used to peek at the full process space. You can also write in the space to set a breakpoint. There is a wealth of other information in /proc as well.
The PTRACE_CONT command can be used to continue a process. Generally, the target will be paused with a PTRACE_ATTACH when the debugger attaches.
The man page says PTRACE_SIEZE should not pause the process. What flavor and version of Linux are you using? PTRACE_SIEZE has been around for quite awhile so I'm not sure why you are having trouble there.
I note the addr value is set to 0x12345. Is this a valid address in the target space? Or was that just an example? How is the stack address of interest (&value) communicated between the two processes?
I'm not too sure about the return codes. Generally a 0 means all is well, the errno may just be a hangover value from the last error.
--Matt
When this program runs it goes through the loop in the parent then switches to the child when it writes to the pipe. In the child the pipe that reads just causes the program to stop.
Current example output:
Parent 4741 14087 (only this line when 5 more lines are expected)
Expected output(with randomly generated numbers):
Parent 4741 14087
Child 4740 47082
Parent 4741 11345
Child 4740 99017
Parent 4741 96744
Child 4740 98653
(when given the variable 3 and the last number is a randomly generated number)
#include <stdio.h>
#include <iostream>
#include <unistd.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#include <time.h>
#include <ctime>
using namespace std;
int main (int argc, char *argv[]) {
int pid = fork(), temp, randNum, count, pipeName[2], pipeName2[2];
string conver;
pipe(pipeName);
conver = argv[1];
temp = atoi(conver.c_str());
char letter;
if (pid == 0) { //child
srand((unsigned)time(NULL) * getpid() );
//closing unused pipes
close(pipeName2[1]);
close(pipeName[0]);
//loop to switch between processes
for(int i=0; i<temp; i++) {
count = read(pipeName2[0], &letter, 20);
randNum = rand();
cout << "Child " << getpid() << " " << randNum << endl;
write(pipeName[1], "x", 1);
}
close(pipeName2[0]);
close(pipeName[1]);
}
else { //parent
srand((unsigned)time(NULL) * getpid() );
pipe(pipeName2);
//closing unused pipes
close(pipeName2[0]);
close(pipeName[1]);
//loop to switch between processes
for(int i=0; i<temp; i++) {
if(i != 0)
count = read(pipeName[0], &letter, 20);
randNum = rand();
cout << "Parent " << getpid() << " " << randNum << endl;
write(pipeName2[1], "x", 1);
}
close(pipeName[0]);
close(pipeName2[1]);
}
}
The program ends when it hits the read from pipe line in the child.
Your principal mistake is fork()ing before you initialize the pipes. Both parent and child thus have their own private (not shared via fd inheritance) pipe pair named pipeName, and only the parent initializes pipeName2 with pipe fds.
For the parent, there's simply no data to read behind pipeName[0]. For the child ... who knows what fd it is writing to in pipeName2[1]? If you're lucky that fails with EBADF.
So, first pipe() twice, and then fork(), and see if that improves things.
The following program, crashes with
libc++abi.dylib: terminating with uncaught exception of type stxxl::io_error: Error in virtual void stxxl::ufs_file_base::lock() : fcntl(,F_SETLK,) path=/var/tmp/stxxl fd=5 : Resource temporarily unavailable: unspecified iostream_category error
Abort trap: 6
This looks a lot like my two threads are trying to use the same file handler/file for doing updates to the stxxl file til /var/tmp.
Is there a trick for having multiple threads using multiple files in stxxl?
#include <stxxl/queue>
#include <iostream>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <stdlib.h>
void test() {
typedef stxxl::queue<unsigned int> queue;
queue my_queue;
for(unsigned long long i = 0; i != 1024L * 1024 * 1024; i++)
my_queue.push(10);
std::cout << "queue_size " << my_queue.size() << std::endl;
while(my_queue.size() != 0)
my_queue.pop();
std::cout << "queue_size " << my_queue.size() << std::endl;
}
int main()
{
pid_t pid;
pid_t cpid;
int status;
pid = fork();
if (pid == 0)
{
test();
exit(0);
} else
{
test();
if ((cpid=wait(&status)) == pid){
std::cout << "Child " << pid << " returned" << std::endl;
}
}
return 0;
}
With STXXL 1.4.0 you can also use "###" in a .stxxl config file. The "###" is replaced with the current pid when files are opened.
Note that the disk files are automatically opened when the first STXXL functions are called. So one must delay such calls until after the fork(), which like you did in your example.
I found the answer my self, a solution is to give every thread its own virtual disk, this I have done like this:
stxxl::config * cfg = stxxl::config::get_instance();
std::string result;
std::ostringstream convert;
convert << id; // the id of the stream, to make th filenames different
result = convert.str();
std::string base_file = "/var/tmp/stxxl" + result;
stxxl::disk_config disk1(base_file, 100 * 1024 * 1024, "syscall autogrow delete_on_exit");
cfg->add_disk(disk1);
I am struggling with process creation and piping the child process' output into a string of the parent process. I got it working on Windows (using CreatePipe and CreateProcess and ReadFile), but can't seem to get the exact analog on Unix to work. This is my code:
#include <spawn.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/wait.h>
#include <iostream>
#include <string>
#include <vector>
using namespace std;
int main()
{
int exit_code;
int cout_pipe[2];
int cerr_pipe[2];
posix_spawn_file_actions_t action;
if(pipe(cout_pipe) || pipe(cerr_pipe))
cout << "pipe returned an error.\n";
posix_spawn_file_actions_init(&action);
posix_spawn_file_actions_addclose(&action, cout_pipe[0]);
posix_spawn_file_actions_addclose(&action, cerr_pipe[0]);
posix_spawn_file_actions_adddup2(&action, cout_pipe[1], 1);
posix_spawn_file_actions_adddup2(&action, cerr_pipe[1], 2);
posix_spawn_file_actions_addclose(&action, cout_pipe[1]);
posix_spawn_file_actions_addclose(&action, cerr_pipe[1]);
vector<string> argmem = {"bla"};
vector<char*> args = {&argmem[0][0], nullptr}; // I don't want to call new.
pid_t pid;
if(posix_spawnp(&pid, "echo", &action, NULL, &args[0], NULL) != 0)
cout << "posix_spawnp failed with error: " << strerror(errno) << "\n";
//close(cout_pipe[0]);
//close(cerr_pipe[0]);
close(cout_pipe[1]);
close(cerr_pipe[1]);
waitpid(pid,&exit_code,0);
cout << "exit code: " << exit_code << "\n";
// Read from pipes
const size_t buffer_size = 1024;
string buffer;
buffer.resize(buffer_size);
ssize_t bytes_read = read(cout_pipe[0], &buffer[0], buffer_size);
while ((bytes_read = read(cout_pipe[0], &buffer[0], buffer_size)) > 0)
{
cout << "read " << bytes_read << " bytes from stdout.\n";
cout << buffer.substr(0, static_cast<size_t>(bytes_read)+1) << "\n";
bytes_read = read(cout_pipe[0], &buffer[0], buffer_size);
}
if(bytes_read == -1)
cout << "Failure reading from stdout pipe.\n";
while ((bytes_read = read(cerr_pipe[0], &buffer[0], buffer_size)) > 0)
{
cout << "read " << bytes_read << " bytes from stderr.\n";
cout << buffer.substr(0, static_cast<size_t>(bytes_read)+1) << "\n";
bytes_read = read(cout_pipe[0], &buffer[0], buffer_size);
}
if(bytes_read == -1)
cout << "Failure reading from stderr pipe.\n";
posix_spawn_file_actions_destroy(&action);
}
The output is:
exit code: 0
So I suppose everything is working except the actual piping. What is wrong here? I also wonder if there is a way to read the piped bytes in a waitpid loop, but when I try that, the parent process hangs infinitely.
posix_spawn is interesting and useful, which makes this question worth necromancing -- even if it is no longer relevant to the OP.
There are some significant bugs in the code as posted. I suspect that some of these were the result of hacking in desperation, but I don't know which was the original bug:
The args array does not include the argv[0] that would represent the executable name. This results in the echo program never seeing the intended argv[1] ("bla").
The read() function is called from different places in a way that just doesn't make sense. A correct way to do this would be to only call read as part of the control expression for the while loops.
waitpid() is called before reading from the pipes. This prevents the I/O from completing (in non-trivial cases at least).
A more subtle issue with this code is that attempts to read all of the child's stdout before reading anything from stderr. In principle, this could cause the child to block while attempting to write to stderr, thus preventing the program from completing. Creating an efficient solution to this is more complicated as it requires that you can read from whichever pipe has available data. I used poll() for this. Another approach would be to use multiple threads.
Additionally, I have used sh (the command shell, i.e. bash) as the child process. This provides a great deal of additional flexibility, such as running a pipeline instead of a single executable. In particular, though, using sh provides the simple convenience of not having to manage the parsing of the command-line.
/*BINFMTCXX: -std=c++11 -Wall -Werror
*/
#include <spawn.h> // see manpages-posix-dev
#include <poll.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/wait.h>
#include <iostream>
#include <string>
#include <vector>
using namespace std;
int main()
{
int exit_code;
int cout_pipe[2];
int cerr_pipe[2];
posix_spawn_file_actions_t action;
if(pipe(cout_pipe) || pipe(cerr_pipe))
cout << "pipe returned an error.\n";
posix_spawn_file_actions_init(&action);
posix_spawn_file_actions_addclose(&action, cout_pipe[0]);
posix_spawn_file_actions_addclose(&action, cerr_pipe[0]);
posix_spawn_file_actions_adddup2(&action, cout_pipe[1], 1);
posix_spawn_file_actions_adddup2(&action, cerr_pipe[1], 2);
posix_spawn_file_actions_addclose(&action, cout_pipe[1]);
posix_spawn_file_actions_addclose(&action, cerr_pipe[1]);
//string command = "echo bla"; // example #1
string command = "pgmcrater -width 64 -height 9 |pgmtopbm |pnmtoplainpnm";
string argsmem[] = {"sh","-c"}; // allows non-const access to literals
char * args[] = {&argsmem[0][0],&argsmem[1][0],&command[0],nullptr};
pid_t pid;
if(posix_spawnp(&pid, args[0], &action, NULL, &args[0], NULL) != 0)
cout << "posix_spawnp failed with error: " << strerror(errno) << "\n";
close(cout_pipe[1]), close(cerr_pipe[1]); // close child-side of pipes
// Read from pipes
string buffer(1024,' ');
std::vector<pollfd> plist = { {cout_pipe[0],POLLIN}, {cerr_pipe[0],POLLIN} };
for ( int rval; (rval=poll(&plist[0],plist.size(),/*timeout*/-1))>0; ) {
if ( plist[0].revents&POLLIN) {
int bytes_read = read(cout_pipe[0], &buffer[0], buffer.length());
cout << "read " << bytes_read << " bytes from stdout.\n";
cout << buffer.substr(0, static_cast<size_t>(bytes_read)) << "\n";
}
else if ( plist[1].revents&POLLIN ) {
int bytes_read = read(cerr_pipe[0], &buffer[0], buffer.length());
cout << "read " << bytes_read << " bytes from stderr.\n";
cout << buffer.substr(0, static_cast<size_t>(bytes_read)) << "\n";
}
else break; // nothing left to read
}
waitpid(pid,&exit_code,0);
cout << "exit code: " << exit_code << "\n";
posix_spawn_file_actions_destroy(&action);
}