i know little about pipes but have used one to connect two processes in my code in visual C++. The pipe is working well, but I need to add error handling to the same, hence wanted to know what will happen to a pipe if the server creating it crashed and how do I recognize it from client process?
Also what will happen if the client process tried accessing the same pipe, after the server crash, if no error handling is put in place?
Edit:
What impact will be there on the memory if i keep creating new pipes (say by using system time as pipe name) while the previous was broken because of a server crash? Will these broken pipes be removed from the memory?
IIRC the ReadFile or WriteFile function will return FALSE and GetLastError() will return STATUS_PIPE_DISCONNECTED
I guess this kind of handling is implemented in your code, if not you should better add it ;-)
I just want to throw this out there.
If you want a survivable method for transferring data between two applications, you might consider using MSMQ or even bringing in BizTalk or another message platform.
There are several things to consider:
what happens if the server is rebooted or loses power?
What happens if the server application becomes unresponsive?
What happens if the server application is killed or goes away completely?
What is the appropriate response of a client application in each of the above?
Each of those contexts represent a potential loss of data. If the data loss is unacceptable then named pipes is not the mechanism you should be using. Instead you need to persist the messages somehow.
MSMQ, storing to a database, or even leveraging Biztalk can take care of the survivability of the message itself.
If 1 or 3 happens, then the named pipe goes away and must be recreated by a new instance of your server application. If #2 happens, then the pipe won't go away until someone either reboots the server or kills the server app and starts it again.
Regardless, the client application needs to handle the above issues. They boil down to connection failed problems. Depending on what the client does you might have it move into a wait state and let it ping the server every so often to see if it has come back again.
Without knowing the nature of the data and communication processes involved its hard to recommend a proper approach.
Related
I am currently developing the server part of a game (MMORPG) and I am stuck on a point that seems to me quite important: how to manage the packets received by the clients and their logic?
Let me explain: I know how to get a connection from a client, how to store the socket of this client but I don't know how to manage packets that it will send later and apply the modifications on the server (all asynchronously).
I had thought of 2 solutions:
1) As soon as the server detects a client connection, it creates a thread for the client. So there is 1 thread per client that will handle the packets of a single client. But in this case, the more clients there are, the more processor will be called right?
2) As soon as the server detects a new client, it stores it in a list. A thread will loop on the client list and see if the current client is sending a packet. If so, it manages it. But this solution also poses a problem: how to manage this packet? Create a new thread specifically for this packet? But I come back to the starting point: too many packets will overload the machine.
A friend offered me a third solution: make a mixture of both. In this way, a thread would take care of NB_MAX_CLIENT.
I would like to know if there are other ways of doing that.
I'm on Windows. I develop with Visual Studio in C ++ and I use the Winsocks.
Thanks in advance and sorry for my bad english.
As soon as the server detects a client connection, it creates a thread for the client. So there is 1 thread per client that will handle the packets of a single client. But in this case, the more clients there are, the more processor will be called right?
This is fairly common unless you are running out of RAM from the stacks that each thread requires (typically OS threads require an OS stack per physical thread). The other issue is too many context switches that might make you consider otherwise.
Avoiding the thread issue is really difficult because you lose the ability to do anything per client without pivoting off a data structure since you have no idea what stack will handle the next packet.
As soon as the server detects a new client, it stores it in a list. A thread will loop on the client list and see if the current client is sending a packet. If so, it manages it. But this solution also poses a problem: how to manage this packet?
Typically you setup a producer consumer set of threads for this. One producer gets each packet and sends it to a queue which is then consumed by some number of worker threads that just handle each item.
Honestly doing this correctly requires a ton of work (as in an example of it was a major piece of technology that Netflix developed) you probably should avoid it to simplify things.
Especially since RAM is cheap and 1MB per thread requires concurrency that will knock you over from other problems before your dedicated thread stacks kill you. (Similarly when context switches become your biggest issue you are pretty far along unless you are doing something unrelated to this discussion wrong).
I am new to creating Windows applications in C++. My task is to write two cpp files, one of which will send a number (x) to the other one, the other one will evaluate f(x) and send it back to the first one. I should implement it using Messages. Couldn't get anything specific online, Could someone pls give me a clue, where to start?
Great thanx!
Are you talking about window messages? If so, the sending app could use SendMessage, which would cause the receiving app to get its window procedure executed. Of course, this means that the receiving app needs to create a window whose window handle is somehow made available to the sending app.
You can do it in several ways.
Using WM_COPYDATA message to pass the data
Allocating global memory to pass data and sending your own message, such that second program can read the data from memory
Sending a message (if two ints suit your needs to pass data)
Using named pipes
Using TCP/IP local connection (peer to peer or through a server)
Look at ZeroMQ (http://zeromq.org ; cross-platform, LGPL). It is a very simple, lightweight and powerfull library. From the very basic level you can use it to exchange UDP-style datagrams, but through reliable transport (TCP or some variants). Also you have cancelling support, time-based polling and advanced network schemes (which are non-needed in your case). I've selected it for a similar task, and it performs very well.
I have a server program which should run full time a day. If I want to change some parameters of it, Is there any way rather than shut down then restart way?
There are quite a few ways of doing this, including, but almost certainly not limited to:
You can maintain the parameters in a separate file so that the program will periodically check that file and update its internal information.
Similar to (1) but you can send some sort of signal to the application to get it to immediately re-read the file.
You can do either (1) or (2) but using shared memory rather than a configuration file.
You can have your program sit at the server end of an IPC conversation, so that a client can open up a connection to it to provide new parameters. Anything from a simple message queue to a full-blown HTTP server and associated pages.
Of course, all of these tend to need a fair amount of work in your program to get it to look for the new information.
You should take that into account when making your decision. By far the quickest solution to implement is to just (cleanly) kill off the process at something like 11:55pm then immediately restart it. It's simpler because your code probably already has the ability to load the information on startup, so this could be a simple cron one-liner.
Some people speak of laziness as a bad thing but that's not always the case :-)
If the Server maintains many alive connections from clients, restarting the server process is the last way you should consider. Except reloading configuration files, inserting a proxy process between clients and server can be another way.
The proxy process is Responsible for 2 things.
a. Maintaining the connection from clients and forwarding packets to Server for handling.
b. Judging weather the current server process(Server A) is alive and if it not, switching to another server(Server B) automatically.
Then you can change parameters by restart server without worrying about interrupting clients since there is always two(or more) servers running.
I have two applications running on my machine. One is supposed to hand in the work and other is supposed to do the work. How can I make sure that the first application/process is in wait state. I can verify via the resources its consuming, but that does not guarantee so. What tools should I use?
Your 2 applications shoud communicate. There are a lot of ways to do that:
Send messages through sockets. This way the 2 processes can run on different machines if you use normal network sockets instead of local ones.
If you are using C you can use semaphores with semget/semop/semctl. There should be interfaces for that in other languages.
Named pipes block until there is both a read and a write operation in progress. You can use that for synchronisation.
Signals are also good for this. In C it is called sendmsg/recvmsg.
DBUS can also be used and has bindings for variuos languages.
Update: If you can't modify the processing application then it is harder. You have to rely on some signs that indicate the progress. (I am assuming you processing application reads a file, does some processing then writes the result to an output file.) Do you know the final size the result should be? If so you need to check the size repeatedly (or whenever it changes).
If you don't know the size but you know how the processing works you may be able to use that. For example the processing is done when the output file is closed. You can use strace to see all the system calls including the close. You can replace the close() function with the LD_PRELOAD environment variable (on windows you have to replace dlls). This way you can sort of modify the processing program without actually recompiling or even having access to its source.
you can use named pipes - the first app will read from it but it will be blank and hence it will keep waiting (blocked). The second app will write into it when it wants the first one to continue.
Nothing can guarantee that your application is in waiting state. You have to pass it some work and get back a response. It might be transactions or not - application can confirm that it got the message to process before it starts to process it or after it was processed (successfully or not). If it does not wait, passing a piece of work should fail. Whether when trying to write to a TCP/IP socket or other means, or if timeout occurs. This depends on implementation, what kind of transport you are using and other requirements.
There is actually a way of figuring out if the process (thread) is in blocking state and waiting for data on a socket (or other source), but that means that client should be on the same computer and have access privileges required to do that, but that makes no sense other than debugging, which you can do using any debugger anyway.
Overall, the idea of making sure that application is waiting for data before trying to pass it that data smells bad. Not to mention the racing condition - what if you checked and it was OK, and when you actually tried to send the data, you found out that application is not waiting at that time (even if that is microseconds).
I have a simple c++ application that generates reports on the back end of my web app (simple LAMP setup). The problem is the back end loads a data file that takes about 1.5GB in memory. This won't scale very well if multiple users are running it simultaneously, so my thought is to split into several programs :
Program A is the main executable that is always running on the server, and always has the data loaded, and can actually run reports.
Program B is spawned from php, and makes a simple request to program A to get the info it needs, and returns the data.
So my questions are these:
What is a good mechanism for B to ask A to do something?
How should it work when A has nothing to do? I don't really want to be polling for tasks or otherwise spinning my tires.
Use a named mutex/event, basically what this does is allows one thread (process A in your case) to sit there hanging out waiting. Then process B comes along, needing something done, and signals the mutex/event this wakes up process A, and you proceed.
If you are on Microsoft :
Mutex, Event
Ipc on linux works differently, but has the same capability:
Linux Stuff
Or alternatively, for the c++ portion you can use one of the boost IPC libraries, which are multi-platform. I'm not sure what PHP has available, but it will no doubt have something equivalent.
Use TCP sockets running on localhost.
Make the C++ application a daemon.
The PHP front-end creates a persistent connection to the daemon. pfsockopen
When a request is made, the PHP sends a request to the daemon which then processes and sends it all back. PHP Sockets C++ Sockets
EDIT
Added some links for reference. I might have some really bad C code that uses sockets of interprocess communication somewhere, but nothing handy.
IPC is easy on C++, just call the POSIX C API.
But what you're asking would be much better served by a queue manager. Make the background daemon wait for a message on the queue, and the frontend PHP just add there the specifications of the task it wants processed. Some queue managers allow the result of the task to be added to the same object, or you can define a new queue for the finish messages.
One of the best known high-performance queue manager is RabbitMQ. Another one very easy to use is MemcacheQ.
Or, you could just add a table to MySQL for tasks, the background process just queries periodically for unfinished ones. This works and can be very reliable (sometimes called Ghetto queues), but break down at high tasks/second.