I am developing an application based on one library.
I am facing problem related to communication in between Parent Process and Forked Process from Parent process.
I need to access Function in the library, and pointer for library is in Parent process and i am calling functins of library from Forked process using pointer in Parent Process. Function in Parent process get called from Forked process but currusponding function in Library stays in bloked state as it should be called from Parent Process only not from Forked process.
What should be the solution for this problem.
***Update:
Library which i mentioned is not exatly loaded library it has one class and i have instantiated that class through my Parent Process and then the library will create its own threds and keep running in it,
So when i do call to library through Parent Process it goes through and when i call Library functions using the Parent Pointer through Forked Process it does not do through. It does not segements but it gets bloked in the function call.
After the fork, both your processes have the library loaded and any existing pointers are valid in both of them, but there's no further connection between them in terms of function calls or access to data. If the parent calls a function, the parent will run the function. If the child calls a function, the child will run the function. There's no notion of one process calling a function in another process.
If you want your two processes to communicate, you need to write code to make them do so. Read about interprocess communication to learn about the various ways to do that. One simple option is to call pipe() prior to forking, and after the fork, have the parent close one of the file descriptors and the child close the other. Now you have a way for one process to send messages to the other. Do that twice and you have two-way communication. You can make the parent run a loop that waits for messages from the child via one pipe, acts upon them, and sends back the results via the other pipe.
You don't say what OS you are using, but generally pointers are not valid across processes. Most OSes give each process its own virtual memory space, so address 0x12345678 may be a pointer to something in one process, but not even an available valid address in another.
If the forked process wants to call a function, it will have to gain access to it itself (link or open the library itself, etc.)
If you're trying to share memory across two processes, but the same executable, then you should be using threads instead of forking a separate process. As others have mentioned, forking gives you a separate memory space, so you can't pass a pointer. With threads, you'd share the same memory space.
You have reached the Inter Process Communication (IPC) problem, where one program wants to make another one do something.
Since you fork()ed your child process, it now lives on his own, and to be able to execute a function on the parent process you'll have to figure out a way for them to communicate:
Child : Dad, please execute this function with these arguments, and give me the result in this pointer, please.
The problem is very widely known, you have many solutions, one of which is to design your own IPC language and implement a Remote Procedure Call (RPC) over it.
Now, people have solved the problem before, so you can take a look at some of these things:
IPC Methods
pipes
sockets (unix and network)
message queues
shared memory
RPC protocols
D-Bus
Corba
TPL (not an RPC protocol, but you can build one with it)
Related
In a multithreaded Linux/C++-program, I want to use fork() with a signal handler for SIGCHLD.
In the child process I use open() to create two new file descriptors, sendfile() and close(), then the child exits.
I planned to use fork() to implement the following requirements:
The threads in the parent process shall be able to
detect the normal termination of the child process, and in that case shall be able to create another fork() doing the open()/sendfile()/close() for a range of files
kill the sendfile()-child process in case of a specific event and detect the intentional termination to clean up
For requirement 1 I could just wait for the result of sendfile().
Requirement 2 is why I think I need to use fork() in the first place.
After reading the following posts
Threads and fork(). How can I deal with that?
fork in multi-threaded program
I think that my solution might not be a good one.
My questions are:
Is there any other solution to implement requirement 2 ?
Or how can I make sure that the library calls open(), close() and sendfile() will be okay?
Update:
The program will run on a Busybox Linux / ARM
I've assumed that I should use sendfile() for having the most efficient file transfer due to several posts I've read regarding this topic.
A safe way to implement my requirement could be using fork() and exec*() with cp, with the disadvantage that the file transfer might be less efficient
Update 2:
it's sufficient to fork() once in case of a specific event (instead of once per file) since I switched to exec*() with rsync in the child process. However the program needs invoke that rsync always in case of a specific event.
You can use threads, but forcefully terminating threads typically leads to memory leaks and other problems.
My linux experience is somewhat limited, but I would probably try to fork the program early, before it gets multithreaded. Now that you have two instances, the single threaded instance can be safely used to manage the starting and stopping of additional instances.
I know the answer to "why is it this way" is because the language was invented so, but it seems like a lot of wasted effort that fork() spawns a copy of the process that called it. Perhaps it is useful sometimes, but surely the majority of time someone wants to start a new process its not to be a duplicate of the calling one? Why does fork create an identical process and not an empty one or one defined by passing an argument?
From yolinux
The fork() system call will spawn a new child process which is an
identical process to the parent except that has a new system process
ID
In other words when is it useful to start with a copy of the parent process?
One big advantage of having the parent process duplicated in the child is that it allows the parent program to make customizations to the child process' environment before executing it. For example, the parent might want to read the child process' stdout, in which case it needs to set up the pipes in order to allow it to read that before execing the new program.
It's also not as bad as it sounds, efficiency wise. The whole thing is implemented on Linux using copy-on-write semantics for the process' memory (except in the special cases noted in the man page):
Under Linux (and in most unices since version 7, parent of all unices alive now), fork() is implemented using copy-on-write pages, so the only
penalty that it incurs is the time and memory required to duplicate the
parent's page tables (which can be also copy-on-write), and to create a unique task structure for the child.
There are some very legitimate uses of the fork system call. Here are a few examples:
Memory saving. Because fork on any modern UNIX/Linux system shares memory between the child and parent (via copy-on-write semantics), a parent process can load some static data which can be instantly shared to a child process. The zygote process on Android does this: it preloads the Java (Dalvik) runtime and many classes, then simply forks to create new application processes on demand (which inherit a copy of the parent's runtime and loaded classes).
Time saving. A process can perform some expensive initialization procedure (such as Apache loading configuration files and modules), then fork off workers to perform tasks which use the preloaded initialization data.
Arbitrary process customization. On systems that have direct process creation methods (e.g. Windows with CreateProcess, QNX with spawn, etc., these direct process creation APIs tend to be very complex since every possible customization of the process has to be specified in the function call itself. By contrast, with fork/exec, a process can just fork, perform customizations via standard system calls (close, signal, dup, etc.) and then exec when it's ready. fork/exec is consequently one of the simplest process creation APIs in existence, yet simultaneously one of the most powerful and flexible.
To be fair, fork also has its fair share of problems. For example, it doesn't play nice with multithreaded programs: only one thread is created in the new process, and locks are not correctly closed (leading to the necessity of atfork handlers to reset lock states across a fork).
Contrary to all expectations, it's mainly fork that makes process creation so incredibly fast on Unices.
AFAIK, on Linux, the actual process memory is not copied upon fork, the child starts with the same virtual memory mapping as the parent, and pages are copied only where and when the child makes changes. The majority of pages are read-only code anyway, so they are never copied. This is called copy-on-write.
Use cases where copying the parent process is useful:
Shells
When you say cat foo >bar, the shell forks, and in the child process (still the shell) prepares the redirection, and then execs cat foo. The executed program runs under the same PID as the child shell and inherits all open file descriptors. You would not believe how easy it is to write a basic Unix shell.
Daemons (services)
Daemons run in the background. Many of them fork after some initial preparation, the parent exits, and the child detaches from the terminal and remains running in the background.
Network servers
Many networking daemons have to handle multiple connections at the same time. Example sshd. The main daemon runs as root and listens for new connections on port 22. When a new connection comes in it forks a child. The child just keeps the new socket representing that connection, authenticates the user, drops privileges and so on.
Etc
Why fork()? It had nothing to do with C. C was itself only coming into existence at the time. It's because of the way the original UNIX memory page and process management worked, it was trivial to cause a process to be paged out, and then paged back in at a different location, without unloading the first copy of the process.
In The Evolution of the Unix Time-sharing System (http://cm.bell-labs.com/cm/cs/who/dmr/hist.html), Dennis Ritchie says "In fact, the PDP-7's fork call required precisely 27 lines of assembly code." See the link for more.
Threads are evil. With threads, you essentially have a number of processes all with access to the same memory space, which can dance all over each others' values. There's no memory protection at all. See The Art of Unix Programming, Chapter 7 (http://www.faqs.org/docs/artu/ch07s03.html#id2923889) for a fuller explanation.
Is it possible to use hooks to jump to another process' running memory, and then jump back, without anything like a DLL injection?
For instance, if process A has a procedure foo and process B has a procedure bar with an identical prototype to foo (used for a hook), is it possible to hook foo to jmp to bar, assuming both processes are running?
EDIT: This needs to be done on Windows.
A process by definition is a sandbox. If you even by mistake step outside your address space it's raised and caught as a SIG_USR signal and reported as a segmentation fault
Having said that there are inter process communication mechanisms such as shared memory -shmem, Pipes and sockets that you can use to communicate across processes.
Edit :
There are RPC ( remote procedure calls) mechanisms available as well Such as CORBA That provide remote method invocation.
Each process in Windows (as well as in Unix and probably vast majority of other modern OS) has it's own virtual memory space, which usually maps to different physical addresses. Consequently injecting a DLL into the address space of another process is the only method to hook anything in that process. On the other hand once you have a DLL in that process, there are plenty of things you can do there, eg. spawn your own thread and communicate with the parent process using Windows messages (as one of the easiest communication methods in Windows).
I am making a server and I use fork() for creating child processes but I have doubts about that. Here are some:
Why do you need to close main socket in a child process and new connection accepted socket in the parent process? (after accepting a new connection) I thought that sockets are only integers with some id which are used to access opened sockets in some system-wide object which is only acceptable through system functions calls. In that case fork would only copy the integer but would have no affect on socket opened.
I have checked and found out that if I fork a process inside a class method, all members are copied. Well, I have found out it is copy-on-edit so does it mean that my server class will be copied in each child that uses a non-constant function? How do I make some memory shared between all such processes (like a list of tasks and each child puts something into it while the parent is reading stuff from it)? I guess fork is not the right function for that. What is the best way?
P.S. I am pretty sure I know the answer to the second question, which is clone(), but just wanted to make sure that's the right function.
Sockets in Unix are File descriptors and they are indeed integers, as seen by the user, but they really are indexes into a table that the kernel maintains per process. In this table each file descriptor (FD) refers to an open file description (OFD) that are system-wide objects maintained in kernel. When you do fork() the opened file descriptors are duplicated and both child's and parent's point to the same OFD. Having two FDs that refer to the same OFD is not usually a problem, but particularly with sockets they can have subtle problems, as the connection is closed only when you close all the FDs that refer to it.
You should really consider using threads (do not close the sockets if you use threads!). clone is a linux system call and is not intended to be used directly. Your alternative is to use shared memory but it is kind of more complex.
The int is a handle, but the socket itself is still associated with the process. The child closes the listening socket mainly for safety reasons (it doesn't need it, and if the child ever spawns another process, that process would inherit the socket as well); the server process closes the new connection's socket because otherwise the connection would remain open until the server process exits (the socket exists as long as at least one process still has a handle to it).
You either want multithreading or a proper shared memory approach. This is where the fun begins.
Shared memory between independent processes comes with interesting problems, but also provides otherwise impossible capabilities (for example, you can restart the master server process and leave the processes serving open connections running, which is difficult to get right as two different versions of the service then have to talk to each other, but allows seamless upgrades without disconnecting clients or interrupting service).
Sharing memory between threads is relatively easy, but threads share the same set of file descriptors, so you do not win much here.
Last, there is a third alternative: an event loop watching multiple sockets, giving attention to each only if something is actually happening. Look at the documentation for the select and poll functions.
Forking duplicates file descriptors, so you have to close the duplicate.
Forking also effectively copies all memory (though in practice it's copy-on-write, so it's not very expensive). You make a new process which is entirely separate from the parent process, unless you explicitly set up some shared memory.
Maybe you intended to spawn a new thread rather than forking a new process?
I think you might want to look through this book as a reference on fork().
Yes you do need to close the socket bound to listen in the child and accepted socket in the parent. The integers aka file handles point to real structures see this so unless you want the kernel to dump a new connection on a child or parent being able to send the data to the connected client you might want to prevent this outright.
To share data for between the processes the best way is shared memory. The book I referred you to will have quite a bit of information regarding that too. In general if you need to share memory without shared memory then you might want to look at threads.
P.S. I'm not sure which clone() method you are referring to. Object copying is done via copy constructors.
HI
I am writing a part of a server which should dispatch some other child processes.
Because I want to wait on some of the processes, and dispatch others without waiting for completion, I use double fork for the second kind of processes (thus avoiding the zombie processes).
Problem is, my server holds a lot of memory, so forking takes a long time (even the copy-on-write fork used in Linux which copies only the paging tables)
I want to replace the fork() with vfork(), and it's easy for the second fork (as it only calls execve() in the child), but I couldn't find any way I can replace the first one.
Does anyone know how I can do that?
Thanks!
The server is a linux (RH5U4), written in C++.
Why not simply have the newly exec'd process do another fork itself? That way only a small simple process will have its page tables copied?
EDIT:
Of course the parent would have to do a short-duration wait() to clean up the zombie from that one, but the grandchild process could then run for as long as it wanted.
vfork() can only be used to fork and then call exec or exit. Also, vfork() will block the parent process until the child calls _exit or exec, which is almost certainly not the behavior that you want.
The reason for this is that vfork() doesn't make any copies of any of the data, including the stack, for the new process. So everything is shared, and it is very easy to accidentally change something that the parent process cannot handle. Since the data is shared without copies, the parent process cannot continue running at the same time as the child, so it must wait for the child to _exit or call exec so it is no longer using the data when the parent starts to modify it.
I think that what you really want to do is to make use of SIGCHLD and maintain a list of child processes. You can then do away with the double fork by having your main process be notified when children change state (mostly, when they die) and perform some action on them based on that. You can also keep track of of any of your child processes take longer than expected to complete (because you stored their creation time in your list) and take action if they go crazy and never complete.
Don't double fork. Handle SIGCHLD to save errno, call wait, restore errno.
I believe you can use the answer to another question I asked, for a similar reason. You can vfork() + exec() to an executable which forks again. See setuid() before calling execv() in vfork() / clone()