Global variable in fork() child process - c++

I have encountered an issue when trying to make a duplicate check function.
The main objective is to store data from child process (based on the check() function result that returns true or false) to parent process. I already tried global variables but didn't work.
This is my code :
...
for(j=0; j<indIP; j++)
{
fflush(stdout);
if (!(fork()))
{
char* this_ip = strdup(ip[j]);
if(is_duplicate_check(this_ip,"file1"))
{
if(debugLevel >= 2) printf("Duplicate IP %s\n", this_ip);
}
else if( is_duplicate_check(this_ip,"file2"))
{
if(debugLevel >= 2) printf("Duplicate IP %s\n",this_ip);
}
else
{
if(debugLevel >= 2) printf("Checking IP [%d of %d] -> [%s]\n",current_combo,possible_combinations,ip[j]);
checkauth(this_ip);
}
exit(0);
}
else
{
numforks++;
current_combo += trys;
if (current_combo > possible_combinations)
{
break;
}
if (numforks >= maxf)
{
wait(NULL);
numforks--;
}
}
indInterface++;
if(indInterface>=countInterface) indInterface=0;
}
puts("Finalizing...");
while(numforks>0)
{
printf("Waiting for the child processes [%d] are finished ....\n", numforks);
wait(NULL);
numforks--;
}
puts("Script completed!");
return 0;
...
In short the program reads ips from a text file and checks them if they are a certain GEO CODE location. The is_duplicate_check checks if the ip was not already checked, but it is working by storing ips to file1 and file2, and when it arrives on 100.000 records it is making a lot of load.
Now all i want is to store ips from child process in an array and check before calling check() function or not.
I already tried with
static int *glob_var;
glob_var = (int *) mmap(NULL, sizeof *glob_var, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
glob_var++;
printf("glob_var = [%d]\n",glob_var);
allways gives me something like (-1232557773525 ...), no use.
How can i solve this?

A fork duplicates the memory. So when you store the variable it is different memory from the parent process.
If you can use threads you can have the behavior where you share a global variable (because threads share memory).
Or if you have to fork you can look at
How to use shared memory with Linux in C
for shared memory usage or have a look at the boost version:
http://www.boost.org/doc/libs/1_60_0/doc/html/interprocess/sharedmemorybetweenprocesses.html#interprocess.sharedmemorybetweenprocesses.sharedmemory

Related

Understanding unix child processes that use semaphore and shared memory

I'm going to do my best to ask this question with the understanding that I have.
I'm doing a programming assignment (let's just get that out of the way now) that uses C or C++ on a Unix server to fork four children and use semaphore and shared memory to update a global variable. I'm not sure I have an issue yet, but my lack of understanding has me questioning my structure. Here it is:
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <sys/sem.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#define NUM_REPEATS 10
#define SEM_KEY 1111
#define SHM_KEY 2222
int globalCounter = 0;
/***** Test function for confriming a process type ******/
int checkProcessType(const char *whoami)
{
printf("I am a %s. My pid is:%d my ppid is %d\n",
whoami, getpid(), getppid() );
for(int i = 1; i<=3; i++){
printf("%s counting %d\n", whoami, i);
}
return 1;
}
void
int main (void) {
pid_t process_id; // PID (child or zero)
int sharedMemID; // Shared memory ID
int sharedMemSize; // shared memory size
struct my_mem * sharedMemPointer; // pointer to the attached shared memory
// Definition of shared memory //
struct my_mem {
long counter;
int parent;
int child;
};
// Gathering size of shared memory in bytes //
sharedMemSize = sizeof(my_mem);
if(sharedMemSize <= 0){
perror("error collection shared memory size: Exiting...\n");
exit(0);
}
// Creating Shared Memory //
sharedMemID = shmget(SHM_KEY, sharedMemSize, 0666 | IPC_CREAT);
if (sharedMemID < 0) {
perror("Creating shared memory has failed: Exiting...");
exit(0);
}
// Attaching Shared Memory //
sharedMemPointer = (struct my_mem *)shmat(sharedMemID, NULL, 0);
if (sharedMemPointer == (struct my_mem*) -1) {
perror("Attaching shared memory has failed. Exiting...\n");
exit(0);
}
// Initializing Shared Memory //
sharedMemPointer->counter = 0;
sharedMemPointer->parent = 0;
sharedMemPointer->child = 0;
pid_t adder, reader1, reader2, reader3;
adder = fork();
if(adder > 0)
{
// In parent
reader1 = fork();
if(reader1 > 0)
{
// In parent
reader2 = fork();
if(reader2 > 0)
{
//In parent
reader3 = fork();
if (reader3 > 0)
{
//In parent
}
else if (reader3 < 0)
{
// Error
perror("fork() error");
}
else
{
// In reader3
}
}
else if(reader2 < 0)
{
//Error
perror("fork() error");
}
else
{
// In reader2
}
}
else if(reader1 < 0)
{
// Error
perror("fork() error");
}
else
{
// In reader1
}
}
else if(adder < 0 )
{
// Error
perror("fork() error");
}
else
{
// In adder
//LOOP here for global var in critical section
}
}
Just some info of what I'm doing (I think), I'm creating a hunk of shared memory that will contain a variable, lets call it counter that will strictly be updated by adder and by the parent which becomes a subtractor after all child processes are active. I'm still trying to figure out the semaphore stuff that I will be using so adder and subtractor execute in critical section, but my main question is this.
How can I know where I am in this structure? My adder should have a loop that will do some job (update global var), and the parent/subtractor should have a loop for its job (also update global var). And all the readers can look at any time. Does the loop placement for parent/subtractor matter? I basically have 3 locations I know I'll be in parent. But since all children need to be created first does it have to be in the last conditional after my third fork where I know I'm in parent? When I use my test method I get scattered outputs, meaning child one can be after parent's output, then child three, etc. It's never in any order, and from what I understand of fork that's expected.
I really have like three questions going on, but I need to first wrap my head around the structure. So let me just try to say this again concisely without any junk cause I'm hung up on loop and critical section placement that isn't even written up yet.
More directly, when does parent know the existence of all children and with this structure can one child do a task and somehow come back to it (i.e. adder/first child adding to global variable once, exits, and some other child can do its thing etc).
I still feel like I'm not asking the right thing, and I believe this is due to still trying to grasp concepts. Hopefully my stammering will kind of show what I'm stuck on conceptually. If not I can clarify.

Multiple threads writing to same socket causing issues

I have written a client/server application where the server spawns multiple threads depending upon the request from client.
These threads are expected to send some data to the client(string).
The problem is, data gets overwritten on the client side. How do I tackle this issue ?
I have already read some other threads on similar issue but unable to find the exact solution.
Here is my client code to receive data.
while(1)
{
char buff[MAX_BUFF];
int bytes_read = read(sd,buff,MAX_BUFF);
if(bytes_read == 0)
{
break;
}
else if(bytes_read > 0)
{
if(buff[bytes_read-1]=='$')
{
buff[bytes_read-1]='\0';
cout<<buff;
}
else
{
cout<<buff;
}
}
}
Server Thread code :
void send_data(int sd,char *data)
{
write(sd,data,strlen(data));
cout<<data;
}
void *calcWordCount(void *arg)
{
tdata *tmp = (tdata *)arg;
string line = tmp->line;
string s = tmp->arg;
int sd = tmp->sd_c;
int line_no = tmp->line_no;
int startpos = 0;
int finds = 0;
while ((startpos = line.find(s, startpos)) != std::string::npos)
{
++finds;
startpos+=1;
pthread_mutex_lock(&myMux);
tcount++;
pthread_mutex_unlock(&myMux);
}
pthread_mutex_lock(&mapMux);
int t=wcount[s];
wcount[s]=t+finds;
pthread_mutex_unlock(&mapMux);
char buff[MAX_BUFF];
sprintf(buff,"%s",s.c_str());
sprintf(buff+strlen(buff),"%s"," occured ");
sprintf(buff+strlen(buff),"%d",finds);
sprintf(buff+strlen(buff),"%s"," times on line ");
sprintf(buff+strlen(buff),"%d",line_no);
sprintf(buff+strlen(buff),"\n",strlen("\n"));
send_data(sd,buff);
delete (tdata*)arg;
}
On the server side make sure the shared resource (the socket, along with its associated internal buffer) is protected against the concurrent access.
Define and implement an application level protocol used by the server to make it possible for the client to distinguish what the different threads sent.
As an additional note: One cannot rely on read()/write() reading/writing as much bytes as those two functions were told to read/write. It is an essential necessity to check their return value to learn how much bytes those functions actually read/wrote and loop around them until all data that was intended to be read/written had been read/written.
You should put some mutex to your socket.
When a thread use the socket it should block the socket.
Some mutex example.
I can't help you more without the server code. Because the problem is probably in the server.

C++ Map Iteration and Stack Corruption

I am trying to use a system of maps to store and update data for a chat server. The application is mutlithreaded and uses a lock system to prevent multiple threads from accessing the data.
The problem is this: when a client is removed individually from the map, it is ok. However, when I try to call multiple closes, it leaves some in the memory. If I at any point call ::clear() on the map, it causes a debug assertion error with either "Iterator not compatible" or similar. The code will work the first time (tested using 80+ consoles connected as a test), but due to it leaving chunks behind, will not work again. I have tried researching ways, and I have written systems to stop the code execution until each process has completed. I appreciate any help so far, and I have attached the relevant code snippets.
//portion of server code that handles shutting down
DWORD WINAPI runserver(void *params) {
runserverPARAMS *p = (runserverPARAMS*)params;
/*Server stuff*/
serverquit = 0;
//client based cleanup
vector<int> tokill;
map<int,int>::iterator it = clientsockets.begin();
while(it != clientsockets.end()) {
tokill.push_back(it->first);
++it;
}
for(;;) {
for each (int x in tokill) {
clientquit[x] = 1;
while(clientoffline[x] != 1) {
//haulting execution until thread has terminated
}
destoryclient(x);
}
}
//client thread based cleanup complete.
return 0;
}
//clientioprelim
DWORD WINAPI clientioprelim(void* params) {
CLIENTthreadparams *inparams = (CLIENTthreadparams *)params;
/*Socket stuff*/
for(;;) {
/**/
}
else {
if(clientquit[inparams->clientid] == 1)
break;
}
}
clientoffline[inparams->clientid] = 1;
return 0;
}
int LOCKED; //exported as extern via libraries.h so it's visible to other source files
void destoryclient(int clientid) {
for(;;) {
if(LOCKED == 0) {
LOCKED = 1;
shutdown(clientsockets[clientid], 2);
closesocket(clientsockets[clientid]);
if((clientsockets.count(clientid) != 0) && (clientsockets.find(clientid) != clientsockets.end()))
clientsockets.erase(clientsockets.find(clientid));
if((clientname.count(clientid) != 0) && (clientname.find(clientid) != clientname.end()))
clientname.erase(clientname.find(clientid));
if((clientusername.count(clientid) != 0) && (clientusername.find(clientid) != clientusername.end()))
clientusername.erase(clientusername.find(clientid));
if((clientaddr.count(clientid) != 0) && (clientaddr.find(clientid) != clientaddr.end()))
clientaddr.erase(clientusername.find(clientid));
if((clientcontacts.count(clientid) != 0) && (clientcontacts.find(clientid) != clientcontacts.end()))
clientcontacts.erase(clientcontacts.find(clientid));
if((clientquit.count(clientid) != 0) && (clientquit.find(clientid) != clientquit.end()))
clientquit.erase(clientquit.find(clientid));
if((clientthreads.count(clientid) != 0) && (clientthreads.find(clientid) != clientthreads.end()))
clientthreads.erase(clientthreads.find(clientid));
LOCKED = 0;
break;
}
}
return;
}
Are you really using an int for locking or was it just a simplification of the code? If you really use an int: this won't work and the critical section can be entered twice (or more) simultaneously, if both threads check the variable before one assigns to it (simplified). See mutexes in Wikipedia for reference. You could either use some sort of mutex provided by windows or boost thread instead of the int.

Linux: Executing child process with piped stdin/stdout

Using Linux and C++, I would like a function that does the following:
string f(string s)
{
string r = system("foo < s");
return r;
}
Obviously the above doesn't work, but you get the idea. I have a string s that I would like to pass as the standard input of a child process execution of application "foo", and then I would like to record its standard output to string r and then return it.
What combination of Linux syscalls or POSIX functions should I use?
I'm using Linux 3.0 and do not need the solution to work with older systems.
The code provided by eerpini does not work as written. Note, for example, that the pipe ends that are closed in the parent are used afterwards. Look at
close(wpipefd[1]);
and the subsequent write to that closed descriptor. This is just transposition, but it shows this code has never been used. Below is a version that I have tested. Unfortunately, I changed the code style, so this was not accepted as an edit of eerpini's code.
The only structural change is that I only redirect the I/O in the child (note the dup2 calls are only in the child path.) This is very important, because otherwise the parent's I/O gets messed up. Thanks to eerpini for the initial answer, which I used in developing this one.
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#define PIPE_READ 0
#define PIPE_WRITE 1
int createChild(const char* szCommand, char* const aArguments[], char* const aEnvironment[], const char* szMessage) {
int aStdinPipe[2];
int aStdoutPipe[2];
int nChild;
char nChar;
int nResult;
if (pipe(aStdinPipe) < 0) {
perror("allocating pipe for child input redirect");
return -1;
}
if (pipe(aStdoutPipe) < 0) {
close(aStdinPipe[PIPE_READ]);
close(aStdinPipe[PIPE_WRITE]);
perror("allocating pipe for child output redirect");
return -1;
}
nChild = fork();
if (0 == nChild) {
// child continues here
// redirect stdin
if (dup2(aStdinPipe[PIPE_READ], STDIN_FILENO) == -1) {
exit(errno);
}
// redirect stdout
if (dup2(aStdoutPipe[PIPE_WRITE], STDOUT_FILENO) == -1) {
exit(errno);
}
// redirect stderr
if (dup2(aStdoutPipe[PIPE_WRITE], STDERR_FILENO) == -1) {
exit(errno);
}
// all these are for use by parent only
close(aStdinPipe[PIPE_READ]);
close(aStdinPipe[PIPE_WRITE]);
close(aStdoutPipe[PIPE_READ]);
close(aStdoutPipe[PIPE_WRITE]);
// run child process image
// replace this with any exec* function find easier to use ("man exec")
nResult = execve(szCommand, aArguments, aEnvironment);
// if we get here at all, an error occurred, but we are in the child
// process, so just exit
exit(nResult);
} else if (nChild > 0) {
// parent continues here
// close unused file descriptors, these are for child only
close(aStdinPipe[PIPE_READ]);
close(aStdoutPipe[PIPE_WRITE]);
// Include error check here
if (NULL != szMessage) {
write(aStdinPipe[PIPE_WRITE], szMessage, strlen(szMessage));
}
// Just a char by char read here, you can change it accordingly
while (read(aStdoutPipe[PIPE_READ], &nChar, 1) == 1) {
write(STDOUT_FILENO, &nChar, 1);
}
// done with these in this example program, you would normally keep these
// open of course as long as you want to talk to the child
close(aStdinPipe[PIPE_WRITE]);
close(aStdoutPipe[PIPE_READ]);
} else {
// failed to create child
close(aStdinPipe[PIPE_READ]);
close(aStdinPipe[PIPE_WRITE]);
close(aStdoutPipe[PIPE_READ]);
close(aStdoutPipe[PIPE_WRITE]);
}
return nChild;
}
Since you want bidirectional access to the process, you would have to do what popen does behind the scenes explicitly with pipes. I am not sure if any of this will change in C++, but here is a pure C example :
void piped(char *str){
int wpipefd[2];
int rpipefd[2];
int defout, defin;
defout = dup(stdout);
defin = dup (stdin);
if(pipe(wpipefd) < 0){
perror("Pipe");
exit(EXIT_FAILURE);
}
if(pipe(rpipefd) < 0){
perror("Pipe");
exit(EXIT_FAILURE);
}
if(dup2(wpipefd[0], 0) == -1){
perror("dup2");
exit(EXIT_FAILURE);
}
if(dup2(rpipefd[1], 1) == -1){
perror("dup2");
exit(EXIT_FAILURE);
}
if(fork() == 0){
close(defout);
close(defin);
close(wpipefd[0]);
close(wpipefd[1]);
close(rpipefd[0]);
close(rpipefd[1]);
//Call exec here. Use the exec* family of functions according to your need
}
else{
if(dup2(defin, 0) == -1){
perror("dup2");
exit(EXIT_FAILURE);
}
if(dup2(defout, 1) == -1){
perror("dup2");
exit(EXIT_FAILURE);
}
close(defout);
close(defin);
close(wpipefd[1]);
close(rpipefd[0]);
//Include error check here
write(wpipefd[1], str, strlen(str));
//Just a char by char read here, you can change it accordingly
while(read(rpipefd[0], &ch, 1) != -1){
write(stdout, &ch, 1);
}
}
}
Effectively you do this :
Create pipes and redirect the stdout and stdin to the ends of the two pipes (note that in linux, pipe() creates unidirectional pipes, so you need to use two pipes for your purpose).
Exec will now start a new process which has the ends of the pipes for stdin and stdout.
Close the unused descriptors, write the string to the pipe and then start reading whatever the process might dump to the other pipe.
dup() is used to create a duplicate entry in the file descriptor table. While dup2() changes what the descriptor points to.
Note : As mentioned by Ammo# in his solution, what I provided above is more or less a template, it will not run if you just tried to execute the code since clearly there is a exec* (family of functions) missing, so the child will terminate almost immediately after the fork().
Ammo's code has some error handling bugs. The child process is returning after dup failure instead of exiting. Perhaps the child dups can be replaced with:
if (dup2(aStdinPipe[PIPE_READ], STDIN_FILENO) == -1 ||
dup2(aStdoutPipe[PIPE_WRITE], STDOUT_FILENO) == -1 ||
dup2(aStdoutPipe[PIPE_WRITE], STDERR_FILENO) == -1
)
{
exit(errno);
}
// all these are for use by parent only
close(aStdinPipe[PIPE_READ]);
close(aStdinPipe[PIPE_WRITE]);
close(aStdoutPipe[PIPE_READ]);
close(aStdoutPipe[PIPE_WRITE]);

Reading Shared Memory from x86 to x64 and vice versa on OSX

If I create a SM from 64 bit application and open it on 32 bit application it fails.
//for 64 bit
shared_memory_object( create_only, "test" , read_write) ;
// for 32 bit
shared_memory_object (open_only, "test", read_write);
file created by 64bit application is at path as below:
/private/tmp/boost_interprocess/AD21A54E000000000000000000000000/test
where as file searched by 32 bit application is at path
/private/tmp/boost_interprocess/AD21A54E00000000/test
Thus 32 bit applications cannot read the file.
I am using boost 1.47.0 on Mac OS X.
Is it a bug? Do I have to do some settings use some Macros in order to fix it? Has any one encountered this problem before?
Is it important that the shared memory be backed by a file? If not, you might consider using the underlying Unix shared memory APIs: shmget, shmat, shmdt, and shmctl, all declared in sys/shm.h. I have found them to be very easy to use.
// create some shared memory
int id = shmget(0x12345678, 1024 * 1024, IPC_CREAT | 0666);
if (id >= 0)
{
void* p = shmat(id, 0, 0);
if (p != (void*)-1)
{
initialize_shared_memory(p);
// detach from the shared memory when we are done;
// it will still exist, waiting for another process to access it
shmdt(p);
}
else
{
handle_error();
}
}
else
{
handle_error();
}
Another process would use something like this to access the shared memory:
// access the shared memory
int id = shmget(0x12345678, 0, 0);
if (id >= 0)
{
// find out how big it is
struct shmid_ds info = { { 0 } };
if (shmctl(id, IPC_STAT, &info) == 0)
printf("%d bytes of shared memory\n", (int)info.shm_segsz);
else
handle_error();
// get its address
void* p = shmat(id, 0, 0);
if (p != (void*)-1)
{
do_something(p);
// detach from the shared memory; it still exists, but we can't get to it
shmdt(p);
}
else
{
handle_error();
}
}
else
{
handle_error();
}
Then, when all processes are done with the shared memory, use shmctl(id, IPC_RMID, 0) to release it back to the system.
You can use the ipcs and ipcrm tools on the command line to manage shared memory. They are useful for cleaning up mistakes when first writing shared memory code.
All that being said, I am not sure about sharing memory between 32-bit and 64-bit programs. I recommend trying the Unix APIs and if they fail, it probably cannot be done. They are, after all, what Boost uses in its implementation.
I found the solution to the problem and as expected it is a bug.
This Bug is present in tmp_dir_helpers.hpp file.
inline void get_bootstamp(std::string &s, bool add = false)
{
...
std::size_t char_counter = 0;
long fields[2] = { result.tv_sec, result.tv_usec };
for(std::size_t field = 0; field != 2; ++field){
for(std::size_t i = 0; i != sizeof(long); ++i){
const char *ptr = (const char *)&fields[field];
bootstamp_str[char_counter++] = Characters[(ptr[i]&0xF0)>>4];
bootstamp_str[char_counter++] = Characters[(ptr[i]&0x0F)];
}
...
}
Where as it should have been some thing like this..
**long long** fields[2] = { result.tv_sec, result.tv_usec };
for(std::size_t field = 0; field != 2; ++field){
for(std::size_t i = 0; i != sizeof(**long long**); ++i)
I have created a ticket in boost for this bug.
Thank you.