I've been trying to mimic & in my dummy shell.
The foreground process works fine, but as soon as I include the & symbol, it doesn't behave as expected. The program shows unexpected behavior. It first executes the process (which should not be executed as a foreground process) and then it just freezes until I press the Enter key.
Here is the snippet of my code.
if(background)
{
int bgpid;
pid_t fork_return;
fork_return = fork();
if(fork_return == 0)
{
setpgid(0,0);
if(execvp(path, args) == -1)
{
bgpid = getpid();
cout<<"Error\n";
return 1;
}
else if(fork_return != -1)
{
addToTable(bgpid);
return 1;
}
}else{
court<<"ERROR\n";
return 1;
}
}
Output Image is also attached here
I am implementing an asynchronous log writing mechanism for my project's multithreaded application. Below is the partial code of the part where the error occurs.
void CTraceFileWriterThread::run()
{
bool fShoudIRun = shouldThreadsRun(); // Some global function which decided if operations need to stop. Not really relevant here. Assume "true" value.
while(fShoudIRun)
{
std::string nextMessage = fetchNext();
if( !nextMessage.empty() )
{
process(nextMessage);
}
else
{
fShoudIRun = shouldThreadsRun();
condVarTraceWriter.wait();
}
}
}
//This is the consumer. This is in my thread with lower priority
std::string CTraceFileWriterThread::fetchNext()
{
// When there are a lot of logs, I mean A LOT, I believe the
// control stays in this function for a long time and an other
// thread calling the "add" function is not able to acquire the lock
// since its held here.
std::string message;
if( !writeQueue.empty() )
{
writeQueueMutex.lock(); // Obj of our wrapper around pthread_mutex_lock
message = writeQueue.front();
writeQueue.pop(); // std::queue
writeQueueMutex.unLock() ;
}
return message;
}
// This is the producer and is called from multiple threads.
void CTraceFileWriterThread::add( std::string outputString ) {
if ( !outputString.empty() )
{
// crashes here while trying to acquire the lock when there are lots of
// logs in prod systems.
writeQueueMutex.lock();
const size_t writeQueueSize = writeQueue.size();
if ( writeQueueSize == maximumWriteQueueCapacity )
{
outputString.append ("\n queue full, discarding traces, traces are incomplete" );
}
if ( writeQueueSize <= maximumWriteQueueCapacity )
{
bool wasEmpty = writeQueue.empty();
writeQueue.push(outputString);
condVarTraceWriter.post(); // will be waiting in a function which calls "fetchNext"
}
writeQueueMutex.unLock();
}
int wrapperMutex::lock() {
//#[ operation lock()
int iRetval;
int iRetry = 10;
do
{
//
iRetry--;
tRfcErrno = pthread_mutex_lock (&tMutex);
if ( (tRfcErrno == EINTR) || (tRfcErrno == EAGAIN) )
{
iRetval = RFC_ERROR;
(void)sched_yield();
}
else if (tRfcErrno != EOK)
{
iRetval = RFC_ERROR;
iRetry = 0;
}
else
{
iRetval = RFC_OK;
iRetry = 0;
}
} while (iRetry > 0);
return iRetval;
//#]
}
I generated the core dump and analysed it with GDB and here are some findings
Program terminated with signal 11, Segmentation fault.
"Errno=45" at the add function where I am trying to acquire the lock. The wrapper we have around pthread_mutex_lock tries to acquire the lock for around 10 times before it gives up.
The code works fine when there are fewer logs. Also, we do not have C++11 or further and hence restricted to mutex of QNX. Any help is appreciated as I am looking at this issue for over a month with little progress. Please ask if anymore info is required.
Running my listen function in a seperate thread seems to use up a lot of CPU
Is it considered ok to use Delays to reduce cpu usage or am I using threads all wrong ?
// Running in a seperate Thread
void Server::listen()
{
while (m_running)
{
if (SDLNet_UDP_Recv(m_socket, m_packet) > 0)
{
//Handle Packet Function
}
}
}
From the SDLNet_UDP_Recv reference
This is a non-blocking call, meaning if there's no data ready to be received the function will return.
That means if there's nothing to receive then SDLNet_UDP_Recv will return immediately with 0 and your loop will iterate and call SDLNet_UDP_Recv again which returns 0 and so on. This loop will never sleep of pause, so of course it will use as much CPU as it can.
A possible solution is indeed to add some kind of delay or sleep in the loop.
I would suggest something like
while (m_running)
{
int res;
while (m_running && (res = SDLNet_UDP_Recv(...)) > 0)
{
// Handle message
}
if (res < 0)
{
// Handle error
}
else if (m_running /* && res == 0 */)
{
// Small delay or sleep
}
}
I wrote a simple program that spawns 10 threads, each thread opens a database (common to all the threads), or creates it (with "Write-Ahead Log" option) if open fails, creates a table on the database and then it goes into an infinite loop in which it adds one row at the time into its table. I found out that the program leaks about 2 handles every 5 minutes, I tried a tool called Memory Verify which tells me that the leaked handles are SQLite3 file locks (line 34034 on the version 3.7.13) but I am not sure whether the bug is in SQLite or in the way I use it.
I haven't specified any compiler option to build SQLite3 so it is built as Multi-Thread and as far as I understand Multi-Thread should work fine in my case as every threads has its own SQLite connection.
To open or create a database I use the following code:
bool Create()
{
int iFlags = 0;
iFlags = iFlags | SQLITE_OPEN_READWRITE | SQLITE_OPEN_NOMUTEX | SQLITE_OPEN_CREATE;
return sqlite3_open_v2(dbName_sm.c_str(), &pHandle_m, iFlags, 0) == SQLITE_OK;
}
bool Open()
{
int iFlags = 0;
iFlags = iFlags | SQLITE_OPEN_READWRITE | SQLITE_OPEN_NOMUTEX;
return sqlite3_open_v2(dbName_sm.c_str(), &pHandle_m, iFlags, 0) == SQLITE_OK;
}
The hard loop in every thread calls ExecuteQuery which does prepare, step and finalize of an INSERT statement:
bool ExecuteQuery(const std::string& statement)
{
bool res = Prepare(statement);
if(!res)
{
return false;;
}
SQLiteStatus status = Step();
Finalize();
res = (ESuccess == status || EDatabaseDone == status);
return res;
}
bool Prepare(const std::string& statement)
{
return sqlite3_prepare_v2(pHandle_m, statement.c_str(), -1, &pStmt_m, 0) == SQLITE_OK;
}
enum SQLiteStatus { ESuccess, EDatabaseDone, EDatabaseTimeout, EDatabaseError };
SQLiteStatus Step()
{
int iRet = sqlite3_step(pStmt_m);
if (iRet == SQLITE_DONE)
{
return EDatabaseDone;
}
else if (iRet == SQLITE_BUSY)
{
return EDatabaseTimeout;
}
else if (iRet != SQLITE_ROW)
{
return EDatabaseError;
}
return ESuccess;
}
bool Finalize()
{
int iRet = sqlite3_finalize(pStmt_m);
pStmt_m = 0;
return iRet == SQLITE_OK;
}
Do you guys see any mistake in my code or is it a known issue in SQLite? I tried to google it for a couple of days but I couldn't find anything about it.
Thank you very much for your help.
Regards,
Andrea
P.S. I forgot to say that I am running my test on a WinXP 64bit PC, the compiler is VS2010, the application is compiled in 32bit, SQLite version is 3.7.13...
check whether you have sqlite3_reset after every sqlite3_step because this is one case that might causes leaks. after preparing a statement with sqlite3_prepare and executing it with sqlite3_step,you need to always reset it with sqlite3_reset.
Using Linux and C++, I would like a function that does the following:
string f(string s)
{
string r = system("foo < s");
return r;
}
Obviously the above doesn't work, but you get the idea. I have a string s that I would like to pass as the standard input of a child process execution of application "foo", and then I would like to record its standard output to string r and then return it.
What combination of Linux syscalls or POSIX functions should I use?
I'm using Linux 3.0 and do not need the solution to work with older systems.
The code provided by eerpini does not work as written. Note, for example, that the pipe ends that are closed in the parent are used afterwards. Look at
close(wpipefd[1]);
and the subsequent write to that closed descriptor. This is just transposition, but it shows this code has never been used. Below is a version that I have tested. Unfortunately, I changed the code style, so this was not accepted as an edit of eerpini's code.
The only structural change is that I only redirect the I/O in the child (note the dup2 calls are only in the child path.) This is very important, because otherwise the parent's I/O gets messed up. Thanks to eerpini for the initial answer, which I used in developing this one.
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#define PIPE_READ 0
#define PIPE_WRITE 1
int createChild(const char* szCommand, char* const aArguments[], char* const aEnvironment[], const char* szMessage) {
int aStdinPipe[2];
int aStdoutPipe[2];
int nChild;
char nChar;
int nResult;
if (pipe(aStdinPipe) < 0) {
perror("allocating pipe for child input redirect");
return -1;
}
if (pipe(aStdoutPipe) < 0) {
close(aStdinPipe[PIPE_READ]);
close(aStdinPipe[PIPE_WRITE]);
perror("allocating pipe for child output redirect");
return -1;
}
nChild = fork();
if (0 == nChild) {
// child continues here
// redirect stdin
if (dup2(aStdinPipe[PIPE_READ], STDIN_FILENO) == -1) {
exit(errno);
}
// redirect stdout
if (dup2(aStdoutPipe[PIPE_WRITE], STDOUT_FILENO) == -1) {
exit(errno);
}
// redirect stderr
if (dup2(aStdoutPipe[PIPE_WRITE], STDERR_FILENO) == -1) {
exit(errno);
}
// all these are for use by parent only
close(aStdinPipe[PIPE_READ]);
close(aStdinPipe[PIPE_WRITE]);
close(aStdoutPipe[PIPE_READ]);
close(aStdoutPipe[PIPE_WRITE]);
// run child process image
// replace this with any exec* function find easier to use ("man exec")
nResult = execve(szCommand, aArguments, aEnvironment);
// if we get here at all, an error occurred, but we are in the child
// process, so just exit
exit(nResult);
} else if (nChild > 0) {
// parent continues here
// close unused file descriptors, these are for child only
close(aStdinPipe[PIPE_READ]);
close(aStdoutPipe[PIPE_WRITE]);
// Include error check here
if (NULL != szMessage) {
write(aStdinPipe[PIPE_WRITE], szMessage, strlen(szMessage));
}
// Just a char by char read here, you can change it accordingly
while (read(aStdoutPipe[PIPE_READ], &nChar, 1) == 1) {
write(STDOUT_FILENO, &nChar, 1);
}
// done with these in this example program, you would normally keep these
// open of course as long as you want to talk to the child
close(aStdinPipe[PIPE_WRITE]);
close(aStdoutPipe[PIPE_READ]);
} else {
// failed to create child
close(aStdinPipe[PIPE_READ]);
close(aStdinPipe[PIPE_WRITE]);
close(aStdoutPipe[PIPE_READ]);
close(aStdoutPipe[PIPE_WRITE]);
}
return nChild;
}
Since you want bidirectional access to the process, you would have to do what popen does behind the scenes explicitly with pipes. I am not sure if any of this will change in C++, but here is a pure C example :
void piped(char *str){
int wpipefd[2];
int rpipefd[2];
int defout, defin;
defout = dup(stdout);
defin = dup (stdin);
if(pipe(wpipefd) < 0){
perror("Pipe");
exit(EXIT_FAILURE);
}
if(pipe(rpipefd) < 0){
perror("Pipe");
exit(EXIT_FAILURE);
}
if(dup2(wpipefd[0], 0) == -1){
perror("dup2");
exit(EXIT_FAILURE);
}
if(dup2(rpipefd[1], 1) == -1){
perror("dup2");
exit(EXIT_FAILURE);
}
if(fork() == 0){
close(defout);
close(defin);
close(wpipefd[0]);
close(wpipefd[1]);
close(rpipefd[0]);
close(rpipefd[1]);
//Call exec here. Use the exec* family of functions according to your need
}
else{
if(dup2(defin, 0) == -1){
perror("dup2");
exit(EXIT_FAILURE);
}
if(dup2(defout, 1) == -1){
perror("dup2");
exit(EXIT_FAILURE);
}
close(defout);
close(defin);
close(wpipefd[1]);
close(rpipefd[0]);
//Include error check here
write(wpipefd[1], str, strlen(str));
//Just a char by char read here, you can change it accordingly
while(read(rpipefd[0], &ch, 1) != -1){
write(stdout, &ch, 1);
}
}
}
Effectively you do this :
Create pipes and redirect the stdout and stdin to the ends of the two pipes (note that in linux, pipe() creates unidirectional pipes, so you need to use two pipes for your purpose).
Exec will now start a new process which has the ends of the pipes for stdin and stdout.
Close the unused descriptors, write the string to the pipe and then start reading whatever the process might dump to the other pipe.
dup() is used to create a duplicate entry in the file descriptor table. While dup2() changes what the descriptor points to.
Note : As mentioned by Ammo# in his solution, what I provided above is more or less a template, it will not run if you just tried to execute the code since clearly there is a exec* (family of functions) missing, so the child will terminate almost immediately after the fork().
Ammo's code has some error handling bugs. The child process is returning after dup failure instead of exiting. Perhaps the child dups can be replaced with:
if (dup2(aStdinPipe[PIPE_READ], STDIN_FILENO) == -1 ||
dup2(aStdoutPipe[PIPE_WRITE], STDOUT_FILENO) == -1 ||
dup2(aStdoutPipe[PIPE_WRITE], STDERR_FILENO) == -1
)
{
exit(errno);
}
// all these are for use by parent only
close(aStdinPipe[PIPE_READ]);
close(aStdinPipe[PIPE_WRITE]);
close(aStdoutPipe[PIPE_READ]);
close(aStdoutPipe[PIPE_WRITE]);