I have a collection of executables that regularly update a collection of files every couple of minutes 24/7. I am thinking about writing a single monitoring program that will continuously check the last write time (using the function stat()) of all these files so that if any have not been updated recently enough it can ring an alarm. My concern though is that perhaps the very act of calling stat() may cause a program that is attempting to write to that file, to fail. Need I worry?... and if so is there an alternative way to achieve my goal?
Yes, a stat call can be thought of as atomic, in that all the information it returns is guaranteed to be consistent. If you call stat at the same instant some other process is writing to the file, there should be no possibility that, say, the other process's write is reflected in st_mtime but not st_size.
And in any case, there's certainly no possibility that calling stat at the same instant some other process is writing to the file could cause that other process to fail. (That would be a serious and quite unacceptable bug in the operating system -- one of an OS'es main jobs is to ensure that unrelated processes can't accidentally interact with each other in such ways. This lack-of-interference property isn't usually what we mean by "atomic", though.)
With that said, though, the usual way to monitor a process is via its process ID. And there are probably plenty of prewritten packages out there to help you manage one or more processes that are supposed to run continuously, giving you clean start/stop and monitoring capabilities. (See s6 as an example. I know nothing about this package and am not recommending it; it's just the first one I came across in a web search.)
Another possibility, if you have any kind of IPC mechanism set up between your processes, is to set up a periodic heartbeat that each one publishes, so that a watchdog timer somewhere can detect a process dying.
If you want to keep monitoring your processes by the timeliness of the files they write, though, that sounds like a perfectly fine technique also.
Related
I have made a program in c++ for changing the password of a system and I wanna run it for every 2 hours,then I end up with two choice in c++ ,one is Sleep(ms) and the other is using recent thread lib this_thread::sleep_for(2h)[ 2h using std::chrono_literals].
The doubt I have been wandering is, does long pausing an exe will work the way we want, is it any other better way than what i mentioned?
I have also planned to put my exe as a windows service.
any other better way than what i mentioned?
Yes.
I suggest, that you do not pause the program at. Simply do the thing, and exit.
Extract the scheduling part to a separate program. You don't even need to write this scheduler, because it already exists on most operating systems.
If you have some task that must be run periodically with long periods of waiting, you should use a program or script, that does the task and exits, and a scheduler, which handles the waiting. There're also questions you need to consider, for example:
do you need to start your task if the scheduled time was missed (due to reboot, for example)
do you allow several of your tasks to run at once, if time it takes to complete is longer than wait period
What you're trying to do is to implement a scheduler yourself. If this is what you want, then sleep is a posix function, and chrono::thread::sleep_for is cross-platform, so it's better to use the second one.
However, it's not generally recommended to implement schedulers, moreover, so simple ones.
I have a software (c++) that runs few processes (each process is a major system itself).
The processes have communication with each other via xml-rpc or boost asio
I want to be able to freeze or stop all processes at a given moment and be able to raise the system (all processes) later to the same state as before hibernating.
How can I do that in c++?
Would it be feasible due to the fact that the processes communicates with each other?
The big picture is that you need to get the system to a stable consistent state, then persist that state in some re-creatable form.
You can in principle write such code, the degree of difficulty depends on your application. You will need to figure out things such as:
How the processes agree that they are in a consistent state. You may need to define some new "Get ready to hibernate" and "I'm ready" messages.
For each process you need to figure out how to persist and recover it's state. Depending upon the complexity of any live data structures that may be quite tricky. On the other hand, if your processes are stateless then this could be really easy.
You'll need to devise a scheme for managing the sets of hibernated data, how you determine a consistent set across all the processes.
I see this as significant coding effort, the degree of difficulty will depend on the complexity of your application and the quality of its implementation. In a well structured application such major "replumbing" exercises often go surprisingly simply.
Unless you're an OS - no, it won't be possible.
What you need to do instead is to make sure that each process can do it for itself (i.e.: write a functionality that allows saving and restoring the states for each of the processes), and also to accommodate for inconsistencies in the communication (for example - to ensure ACK on the messages, and resend if saved state without receiving ACK).
It's feasible if done right, but it's easier said than done, of course, and assumes you can actually change the processes.
Well,
the other answers are fine. There is another rather "exotic" way which may solve this quickly, but it may be overkill or not suitable. But who knows ? So just in case...
I suggest to run your program into a virtual machine (I mean for example a linux with vmware) and pause/wake up this virtual machine at will.
If you are using an inter-process communication method which is not disrupted by this kind of operation, it may work and save you a lot of time.
Good luck.
Usually developing applications I am used to print to console in order to get useful debugging/tracing information. The application I am working now since it is multi-threaded sometimes I see my printf overlapping each other.
I tried to synchronize the screen using a mutex but I end up in slowing and blocking the app. How to solve this issue?
I am aware of MT logging libraries but in using them, since I log too much, I slow ( a bit ) my app.
I was thinking to the following idea..instead of logging within my applications why not log outside it? I would like to send logging information via socket to a second application process that actually print out on the screen.
Are you aware of any library already doing this?
I use Linux/gcc.
thanks
afg
You have 3 options. In increasing order of complexity:
Just use a simple mutex within each thread. The mutex is shared by all threads.
Send all the output to a single thread that does nothing but the logging.
Send all the output to a separate logging application.
Under most circumstances, I would go with #2. #1 is fine as a starting point, but in all but the most trivial applications you can run in to problems serializing the application. #2 is still very simple, and simple is a good thing, but it is also quite scalable. You still end up doing the processing in the main application, but for the vast majority of applications you gain nothing by spinning this off to it's own, dedicated application.
Number 3 is what you're going to do in preformance-critical server type applications, but the minimal performance gain you get with this approach is 1: very difficult to achieve, 2: very easy to screw up, and 3: not the only or even most compelling reason people generally take this approach. Rather, people typically take this approach when they need the logging service to be seperated from the applications using it.
Which OS are you using?
Not sure about specific library's, but one of the classical approaches to this sort of problem is to use a logging queue, which is worked by a writer thread, who's job is purely to write the log file.
You need to be aware, either with a threaded approach, or a multi-process approach that the write queue may back up, meaning it needs to be managed, either by discarding entries or by slowing down your application (which is obviously easier if it's the threaded approach).
It's also common to have some way of categorising your logging output, so that you can have one section of your code logging at a high level, whilst another section of your code logs at a much lower level. This makes it much easier to manage the amount of output that's being written to files and offers you the option of releasing the code with the logging in it, but turned off so that it can be used for fault diagnosis when installed.
As I know critical section has less weight.
Critical section
Using critical section
If you use gcc, you could use atomic accesses. Link.
Frankly, a Mutex is the only way you really want to do that, so it's always going to be slow in your case because you're using so many print statements.... so to solve your question then, don't use so many print_f statements; that's your problem to begin with.
Okay, is your solution using a mutex to print? Perhaps you should have a mutex to a message queue which another thread is processing to print; that has a potential hang up, but I think will be faster. So, use an active logging thread that spins waiting for incoming messages to print. The networking solution could work too, but that requires more work; try this first.
What you can do is to have one queue per thread, and have the logging thread routinely go through each of these and post the message somewhere.
This is fairly easy to set up and the amount of contention can be very low (just a pointer swap or two, which can be done w/o locking anything).
I am on MacOSX.
I am writing a multi threaded program.
One thread does logging.
The non-logging threads may crash at any time.
What conventions should I adopt in the logger / what guarantees can I have?
I would prefer a solution where even if I crash during part of a write, previous writes still go to disk, and when reading back the log, I can figure out "ah, I wrote 100 complete enties, then I crashed on the 101th".
Thanks!
I program on Linux, not MacOSX, but probably it's the same there.
If only one thread in your program logs, it means that you buffer the logging data in this logging thread and then it writes it to a file probably some larger portion to avoid too many I/O operations and make the logging process faster.
The bad thing is that if one thread segfaults, the whole process is destroyed along with the buffered data.
The solutions (for Linux) I know of are:
Transfer the logging data through a socket, without using buffering logging thread (syslog for example). In this case the OS will probably take care of the data, written to the socket, and even if your application crashes, the data should be received on the other end and logged successfully.
Don's use logging thread, every thread can log synchronously to a file. In this case the losses of the log data after the crash should be very small or none. It's slower though.
I don't know better solutions for this problem yet, it would be interesting to learn ones though.
As Dmitry says, there's only a few options to ensure you actually capture he logging output. Do you really need to write your own? And does it really need to be on another thread? This may introduce a timing window for a crash to miss logs, when you normally want to log synchronously.
The syslog facility on Unix is the standard means for reliable logging for system services. It essentially solves these sorts of problems you describe (ie. logs are processed out-of-process, so if you crash, your logs still get saved).
If your application is aimed only at Mac OS X, you should have a look at the Apple System Log facility (ASL). It provides a more sophisticated API than syslog and a superset of its functionality.
I'm looking for a way to restart a thread, either from inside that thread's context or from outside the thread, possibly from within another process. (Any of these options will work.) I am aware of the difficulty of hibernating entire processes, and I'm pretty sure that those same difficulties attend to threads. However, I'm asking anyway in the hopes that someone has some insight.
My goal is to pause, save to file, and restart a running thread from its exact context with no modification to that thread's code, or rather, modification in only a small area - i.e., I can't go writing serialization functions throughout the code. The main block of code must be unmodified, and will not have any global/system handles (file handles, sockets, mutexes, etc.) Really down-and-dirty details like CPU registers do not need to be saved; but basically the heap, stack, and program counter should be saved, and anything else required to get the thread running again logically correctly from its save point. The resulting state of the program should be no different, if it was saved or not.
This is for a debugging program for high-reliability software; the goal is to run simulations of the software with various scripts for input, and be able to pause a running simulation and then restart it again later - or get the sim to a branch point, save it, make lots of copies and then run further simulations from the common starting point. This is why the main program cannot be modified.
The main thread language is in C++, and should run on Windows and Linux, however if there is a way to only do this on one system, then that's acceptable too.
Thanks in advance.
I think what you're asking is much more complicated than you think. I am not too familiar with Windows programming but here are some of the difficulties you'll face in Linux.
A saved thread can only be restored from the root process that originally spawned the thread, otherwise the dynamic libraries would be broken. Because of this saving to disk is essentially meaningless. The reason is dynamic libraries are loaded at different address each time they're loaded. The only way around this would be to take complete control of dynamically linking, no small feat. It's possible, but pretty scary.
The suspended thread will have variables in the the heap. You'd need to be able to find all globals 'owned' by the thread. The 'owned' state of any piece of the heap cannot be determined. In the future it may be possible with the C++0x's garbage collection ABI. You can't just assume the whole stack belongs to the thread to be paused. The main thread uses the heap when creating threads. So blowing away the heap when deserializing the paused thread would break the main thread.
You need to address the issues with globals. And not just the globals from created in the threads. Globals (or statics) can and often are created in dynamic libraries.
There are more resources to a program than just memory. You have file handles, network sockets, database connections, etc. A file handle is just a number. serializing its memory is completely meaningless without the context of the process the file was opened in.
All that said. I don't think the core problem is impossible, just that you should consider a different approach.
Anyway to try to implement this the thread to paused needs to be in a known state. I imagine the thread to be stoped would call a library function meant the halt the process so it could be resumed.
I think the linux system call fork is your friend. Fork perfectly duplicates a process. Have the system run to the desired point and fork. One fork wait to fork others. The second fork runs one set of input.
once it completes the first fork can for again. Again the second fork can run another set of input.
continue ad infinitum.
Threads run in the context of a process. So if you want to do anything like persist a thread state to disk, you need to "hibernate" the entire process.
You will need to serialise the entire set of the processes data. And you'll need to store the current thread execution point. I think serialising the process is do-able (check out boost::serialize) but the thread stop point is a lot more difficult. I would put places where it can be stopped through the code, but as you say, you cannot modify the code.
Given that problem, you're looking at virtualising the platform the app is running on, and using its suspend functionality to pause the entire thing. You might find more information about how to do this in the virtualisation vendor's features, eg Xen.
As the whole logical address space of the program is part of the thread's context, you would have to hibernate the whole process.
If you can guarantee that the thread only uses local variables, you could save its stack. It is easy to suspend a thread with pthreads, but I don't see how you could access its stack from outside then.
The way you would have to do this is via VM Snapshots; get a copy of VMWare Workstation, then you can write code to automate starting/stopping/snapshotting the machine at different points. Any other approach is pretty untenable, as while you might be able to freeze and dethaw a process, you can't reconstruct the system state it expects (all the stuff that Caspin mentions like file handles et al.)