I'm writing an images-processing program in C++. For this purpose, I have modified a third party program (an edge detector) into a static library that I use in my program.
It seams the original edge detector relied on the OS to clean up the memory after the main function has been executed. Unfortunately, after I have modified this original code, the main function became a "common" repeatedly called function and no automatic cleaning is therefore performed. The result is a huge memory leak every time the function is called.
I'm not able to exhaustively review the whole code of the detector to fix this. I'd like therefore to ask: In general, is there a way to separate a "subroutine" of the whole program (in my case the detector) from the rest and to force the OS to clean up after the subroutine as if it was a stand-alone program? Could there be a solution with the use of threads, for example?
Thank you for your replies.
If you are using a *nix platform, perhaps you could fork the library call.
You could run it in a separate process that would be kicked in from your program.
There are ways to pipe the child-process'es stdin, stdout, so you can control it.
What you could also try is to use valgrind to detect the leaks and fix them.
If you are on linux, you can google for: fork() or system() functions to create a child process.
Related
A while back I made a post regarding creating a dll, for the purpose of injection, that will cause the host application to trigger an Nvidia Optimus laptop to "awaken" the dGpu. This being necessary because of the pathetic system nvidia created here which results in MANY applications not recognizing the presence of the power dGpu, and instead using the integrated intel gpu. (Specifically some video processing apps which take hours longer using Intel's than it would Nvidia's). That post was here.
Suffice to say, I moved to work in Antarctica and gave up on the project. I just picked it back up years later and decided to learn (enough) C++ to program it here. I have created the DLL, and if I place the DX code in a function, then call that function from a host "caller" program.. IT WORKS!!! However, if I put that code in the DLLMAIN, and then simply load that dll from my "caller" program (without actually calling a specific function)... the procedure executes!!! However, when it gets to the part of the code where CREATEDEVICE is run, it crashes. I have since learned this is due to an issue called deadlock, or loaderlock.. i'm not sure which. I understand the concept, but don't have anywhere NEAR the C++ understanding to develop a workaround.
So basically.. can I run my procedure in DLLMAIN using some workaround? Maybe spawning an independent thread somehow (so DLLMAIN can finish executing to it's return?) Thanks for any info. I'll include the vcproject source code here.. but it's a Frankenstein of things I found online.. so don't look for elegance- I know next to nothing about C++ programming! http://s000.tinyupload.com/index.php?file_id=07876333208461296171
The loader lock is a lock which is per-process and owned just after you call LoadLibrary, until just before the LoadLibrary returns. It is intended to ensure the process correctly accounts for the loaded DLLs and their order.
There is very little code which can be added in DllMain which doesn't run the risk of a fail, as any Windows call which may cause IPC can fall fowl of the loader-lock.
If you can create a thread from outside the process, or create a second function you can call directly, then this will be a better solution
I want to change the timezone for Linux system. I know there are many ways.
One way is to use tzset() function and another is to call 'timedatectl' command from 'popen()' function.
I am using second approach i.e, using "popen()".
I just want to ask is it a good programming practice to use "popen()" in your code?
Also, I am carefully calling "pclose()" for every "popen()".
There is nothing wrong about popen in general, if you really need a child process to do a specific job for you.
popen creates a pipe allowing you to either read the output (what it wrote to stdout) of the child process or write input to its stdin - but not both at the same time.
If you are not interested in either option, you possibly might prefer calling system instead (however, system will wait for the process to terminate, in contrast to popen - pclose waits for).
But why would you want to create a separate process, if you can do the same job by simply calling an ordinary function (system call or not)? You are creating a lot of overhead using a process then (process must be initialised and hooked into OS, it needs its own memory for executable code, heap and stack, ...)!
It gets a little more complicated, if the job in question requires a considerable amount of time and you cannot afford to wait for the function to complete, but need to do some other stuff. However, in such a case, I'd rather create a thread only and again call the function from there...
popen() invokes a shell to run the command which is an extra unnecessary layer of indirection. Plus there are all sorts of security pitfalls, for instance, you don't have control over the environment - or which shell actually gets invoked.
I'd say it's fine for prototypes and proofs of concept, but for production code you should use fork(), one of the execs and pipes for IO.
EDIT
If there is a function equivalent to doinf something by invoking a command, always use that first. For example, if you can achieve what you want with tzset(), always use that in preference to spawning a new process.
I wrote a tool in C++ using wxWidgets for the GUI and IBM ILOG Cplex to solve an optimization problem.
In one of the functions called by the wx event handler, I invoke the IBM ILOG Cplex Optimizer which is itself multi-threaded code.
I realize that this causes indererministic bugs with non-sensical memory contents.
Since I have no experince in writing multi-threaded code and would like to get away without spending three weeks learning how to do it, I would like to know:
Is there is some safe, possibly inelegant way to avoid problems here? (More elegant, maybe, than writing a file to disc, calling a different task through the OS and reading the output back in).
Is it a bad idea to launch Cplex threads from a wx thread?
Is it generally a bad idea to use two libraries that might use different libraries internally to implement multi-threading? (I have no idea what there is except pthreads and what is used by either cplex or wx).
Any help and background information is appreciated.
Based on my experience, the rule is:
every wxWdiget function call that change the display must be made in the wxWidget thread
I don't know much about Cplex, but if you say it's multithreaded, chances are you are calling an asynchronous function and you handle the results in a call back. The callback is most definitely not called withing the wxWidget thread. If you then try to display the results within the callback, you are breaking the rule stated above. That's when you'll get nice little bugs, which in my case usually materialize as heap corruption.
To fix that you must pass the results of your callback to the wxWidget thread and display them in that thread. There's many way to do it, but the global mechanism is to trigger a custom event on wxWigdet that get passed to the wxWidget thread.
Check this link, http://wiki.wxwidgets.org/Custom_Events you need to use
wxEvtHandler::AddPendingEvent(wxEvent& event)
I have a boost threadpool which I use to do certain tasks. I also have a Sensor class that has the pure virtual function doWork(int total) = 0;. Whenever it is requested, my main process gets the necessary Sensor pointer and tells the threadpool to run Sensor::doWork(int total).
threadpool->schedule(boost::bind(&Sensor::doWork,this,123456));
I am dynamically loading libraries of type Sensor, thus it is out of my control if someone else has faulty coding which results in SEGFAULTS and such. So is there a way for me to (in my main process) handle any errors thrown by Sensor::doWork(int total), clean up the thread, delete that sensor object and notify the console what and where the error has occurred?
Really the only way to handle a segmentation fault here is to run Sensor::doWork in a completely separate process.
In UNIX, this involves using fork (or some other similar means), running Sensor::doWork in the child process, and then somehow shuttling the results back to the parent process.
I assume similar means are available in Windows.
EDIT: I thought I'd flesh out a bit some of the things you can do.
Solution #1: you can work with processes in the same fashion as you would threads. For example, you could create process pool that sit there in a loop of
Wait for a task to be passed in over a pipe or queue or some similar object
Perform the task
Return the results over a pipe or queue or some similar object
And since you're executing the tasks in the other processes, you're protected against them crashing. The main difficulty with this solution is actually communicating between processes; maybe boost's interprocess library will help with that. I've mainly done this sort of thing in python, which has a standard multiprocessing module that handles this stuff for you.
Solution #2: You could divide your application into "safe" and "risky" portions that run in different processes. The "risky" portion executes the Sensor::doWork methods and anything else you might want to do in that process -- but only work that is acceptable to be spontaneously lost if it crashes. The "safe" portion deals with any precious information that you cannot afford to lose, and monitors the "risky" portion, performing some recovery operations when the child crashes. And, of course, whatever other work you decide you want to do in the safe part.
If you got a SIGSEGV, even if you caught it you have no guarantee about your program state so there's pretty much no way to recover.
If you're working with 3rd party libraries, and they're buggy, and the library maintainer won't fix it (and you don't have the source) then your only recourse is to run the third party library from within a totally separate binary that talks to the main binary by some means. See for example firefox and plugin-container.
You might want to register a function callback to catch SIGSEV. In C this can be done using signal. Be aware, however, there is not much you can do, when the OS sends you a SIGSEV (note that it isn't required to). You don't really know in what state your program is in, I'd guess. If for example the heap got corrupt, new and delete operations may fail, so even a plain simple
std::cout << std::string("hello world") << std::endl;
statement, might not work since memory from the heap needs to be allocated.
Best, Christoph
When I exit my C++ program it crashes with errors like:
EAccessViolation with mesage 'Access violation at address 0...
and
Abnormal Program Termination
It is probably caused by some destructor because it happens only when the application exits. I use a few external libraries and cannot find the code that causes it. Is there a function that forces immediate program exit (something like kill in Linux) so that memory would have to be freed by the operating system? I could use this function in app exit event.
I know that it would be a terrible solution because it'd just hide the problem.
I'm just asking out of sheer curiosity, so please don't give me -1 :)
I tried exit(0) from stdlib but it didn't help.
EDIT:
Thanks for your numerous replies:)
I use Builder C++ 6 (I know it's outdated but for some reasons I had to use it). My app uses library to neural networks (FANN). Using the debugger I found that program crashes in:
~neural_net()
{
destroy();
}
destroy() calls multiple time another function fann_safe_free(ptr), that is:
#define fann_safe_free(x) {if(x) { free(x); x = NULL; }}
The library works great, problem only appears when it does cleaning. That's why I asked about so brutal solution. My app is multi-threaded but other threads operate on different data.
I will analyze my code for the n-th time(the bug must be somewhere), thanks for all your tips :)
You should fix the problem.
First step: find at check all functions you register with atexit() (not many I hope)
Second step: find all global variables and check their destructors.
Third Step: find all static function variables check their destructors.
But otherwise you can abort.
Note: abort is for Abnormal program termination.
abort()
The difference: (note letting an application leave the main function is the equivalent of exit())
exit()
Call the functions registered with the atexit(3) function, in the reverse order of their registration. This includes the destruction of all global (static storage duration) variables.
Flush all open output streams.
Close all open streams.
Unlink all files created with the tmpfile(3) function.
abort()
Flush all open output streams.
Close all open streams.
It's a terrible solution for more than one reason. It will hide the problem (maybe), but it could also corrupt data, depending on the nature of your application.
Why don't you use a debugger and try to find out what is causing the error?
If your application is multi-threaded, you should make sure that all threads are properly shut down before exiting the application. This is a fairly common cause of that type of error on exit, when a background thread is attempting to use memory/objects that have already been destructed.
Edit:
based on your updated question, I have the following suggestions:
Try to find out more specifically what is causing the crash in the destructor.
The first thing I would do is make sure that it's not trying to destruct a NULL object. When you get your crash in ~neural_net in your debugger, check your "this" pointer to make sure it's not NULL. If it is, then check your call-stack and see where it's being destructed, and do a check to make sure it's not NULL before calling delete.
If it's not NULL, then I would unroll that macro in destroy, so you can see if it's crashing on the call to free.
You could try calling abort(); (declared in <stdlib.h> and in <process.h>)
The version in VisualC++, however, will print a warning message as it exits: "This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information."
On Linux/UNIX you can use _exit:
#include <unistd.h>
void _exit(int status);
The function _exit() is like exit(), but does not call any functions registered with atexit() or on_exit(). Whether it flushes standard I/O buffers and removes temporary files created with tmpfile(3) is implementation dependent. On the other hand, _exit() does close open file descriptors, and this may cause an unknown delay, waiting for pending output to finish. If the delay is undesired, it may be useful to call functions like tcflush() before calling _exit(). Whether any pending I/O is cancelled, and which pending I/O may be cancelled upon _exit(), is implementation-dependent.
Have you tried the gruesome step by step? If you're project/solution is simply to large to do so maybe you could try segmenting it assuming you use a modular build and test each component indivdually. Without any code or visible destructors abstract advice is all I can give you I'm afraid. But nonetheless I hope trying to minimize the debugging field will help in some way.
Good luck with getting an answer :)
That immediate program exit (and yes, that's a terrible solution) is abort()
That happens most likely because a NULL pointer is being accessed. Depending on your OS try getting a stack trace and identify the culprit, don't just exit.
If you use linux, valgrind should solve your problem.
but if it is windows, try one of these: MemoryValidator, BoundsChecker or other tools like these.
Simply close your application is not the best way to deal with bugs ...