I wrote a tool in C++ using wxWidgets for the GUI and IBM ILOG Cplex to solve an optimization problem.
In one of the functions called by the wx event handler, I invoke the IBM ILOG Cplex Optimizer which is itself multi-threaded code.
I realize that this causes indererministic bugs with non-sensical memory contents.
Since I have no experince in writing multi-threaded code and would like to get away without spending three weeks learning how to do it, I would like to know:
Is there is some safe, possibly inelegant way to avoid problems here? (More elegant, maybe, than writing a file to disc, calling a different task through the OS and reading the output back in).
Is it a bad idea to launch Cplex threads from a wx thread?
Is it generally a bad idea to use two libraries that might use different libraries internally to implement multi-threading? (I have no idea what there is except pthreads and what is used by either cplex or wx).
Any help and background information is appreciated.
Based on my experience, the rule is:
every wxWdiget function call that change the display must be made in the wxWidget thread
I don't know much about Cplex, but if you say it's multithreaded, chances are you are calling an asynchronous function and you handle the results in a call back. The callback is most definitely not called withing the wxWidget thread. If you then try to display the results within the callback, you are breaking the rule stated above. That's when you'll get nice little bugs, which in my case usually materialize as heap corruption.
To fix that you must pass the results of your callback to the wxWidget thread and display them in that thread. There's many way to do it, but the global mechanism is to trigger a custom event on wxWigdet that get passed to the wxWidget thread.
Check this link, http://wiki.wxwidgets.org/Custom_Events you need to use
wxEvtHandler::AddPendingEvent(wxEvent& event)
Related
I'm designing the threading architecture for my game engine, and I have reached a point where I am stumped.
The engine is partially inspired by Grimrock's engine, where they put as much as they could into LuaJIT, with some things, including low level systems, written in C++.
This seemed like a good plan, given that LuaJIT is easy to use, and I can continue to add API functions in C++ and expand it further. Faster iteration is nice, the ability to have a custom IDE attached to the game and edit the code while it runs is an interesting option, and serializing from Lua is also easy.
But I am stumped on how to go about adding threading. I know Lua has coroutines, but that is not true threading; it's basically to keep Lua from stalling as it waits for code that takes too long.
I originally had in mind to have the main thread running in Lua and calling C++ functions which are dispatched to the scheduler, but I can't find enough information on how Lua functions. I do know that when Lua calls a C++ function it runs outside of the state, so theoretically it may be possible.
I also don't know whether, if Lua makes such a call that is not supposed to return anything, it will hang on the function until it's done.
And I'm not sure whether the task scheduler runs in the main thread, or if it is simply all worker threads pulling data from a queue.
Basically meaning that, instead of everything running at once, it waits for the game state update before doing anything.
Does anyone have any ideas, or suggestions for threading?
In general, a single lua_State * is not thread safe. It's written in pure C and meant to go very fast. It's not safe to allow exceptions go through it either. There's no locks in there and no way for it to protect itself.
If you want to run multiple lua scripts simultaneously in separate threads, the most straightforward way is to use luaL_newstate() separately in each thread, initialize each of them, and load and run scripts in each of them. They can talk to the C++ safely as long as your callbacks use locks when necessary. At least, that's how I would try to do it.
There are various things you could do to speed it up, for instance, if you are loading copies of a single script in each of the threads, you could compile it to lua bytecode before you launch any of the threads, then put the buffer into shared memory, and have the scripts load the shared byte code without changing. That's most likely an unnecessary optimization though, depending on your application.
I have a small C++ dll that has 2 callback functions that retrieve information from another dll.
These 2 callback functions are being called repeatedly in more than 1 thread.
They both add information to the same global Cstring variable.
I have another function that the program that uses this dll will call that reads this variable.
It is rare, but sometimes I get a crash and its definitely due to this global variable being read/written to at the same time by 2 different functions.
I am not very experienced with multithreads, so I don't really know what to do.
Any suggestions?
here is a previous question I posted about the same problem with a bit more info..(and some of the code).
One of the users assisted me in confirming that it was a multithread issue and we didnt get much further than that.
C++ DLL crash (reading/writing crash related I think)
Have a read of Thread Synchronization for Beginners.
If you're using MFC then CMutex may be appropriate.
You have to create a critical section on this variable. In Windows, you can do it by using Mutex Objects.
I have a boost threadpool which I use to do certain tasks. I also have a Sensor class that has the pure virtual function doWork(int total) = 0;. Whenever it is requested, my main process gets the necessary Sensor pointer and tells the threadpool to run Sensor::doWork(int total).
threadpool->schedule(boost::bind(&Sensor::doWork,this,123456));
I am dynamically loading libraries of type Sensor, thus it is out of my control if someone else has faulty coding which results in SEGFAULTS and such. So is there a way for me to (in my main process) handle any errors thrown by Sensor::doWork(int total), clean up the thread, delete that sensor object and notify the console what and where the error has occurred?
Really the only way to handle a segmentation fault here is to run Sensor::doWork in a completely separate process.
In UNIX, this involves using fork (or some other similar means), running Sensor::doWork in the child process, and then somehow shuttling the results back to the parent process.
I assume similar means are available in Windows.
EDIT: I thought I'd flesh out a bit some of the things you can do.
Solution #1: you can work with processes in the same fashion as you would threads. For example, you could create process pool that sit there in a loop of
Wait for a task to be passed in over a pipe or queue or some similar object
Perform the task
Return the results over a pipe or queue or some similar object
And since you're executing the tasks in the other processes, you're protected against them crashing. The main difficulty with this solution is actually communicating between processes; maybe boost's interprocess library will help with that. I've mainly done this sort of thing in python, which has a standard multiprocessing module that handles this stuff for you.
Solution #2: You could divide your application into "safe" and "risky" portions that run in different processes. The "risky" portion executes the Sensor::doWork methods and anything else you might want to do in that process -- but only work that is acceptable to be spontaneously lost if it crashes. The "safe" portion deals with any precious information that you cannot afford to lose, and monitors the "risky" portion, performing some recovery operations when the child crashes. And, of course, whatever other work you decide you want to do in the safe part.
If you got a SIGSEGV, even if you caught it you have no guarantee about your program state so there's pretty much no way to recover.
If you're working with 3rd party libraries, and they're buggy, and the library maintainer won't fix it (and you don't have the source) then your only recourse is to run the third party library from within a totally separate binary that talks to the main binary by some means. See for example firefox and plugin-container.
You might want to register a function callback to catch SIGSEV. In C this can be done using signal. Be aware, however, there is not much you can do, when the OS sends you a SIGSEV (note that it isn't required to). You don't really know in what state your program is in, I'd guess. If for example the heap got corrupt, new and delete operations may fail, so even a plain simple
std::cout << std::string("hello world") << std::endl;
statement, might not work since memory from the heap needs to be allocated.
Best, Christoph
I was recently making a very simple application that just printed matrix-effect out to the console. So I made it in a very easy to use function void drawLine(int startX, int startY, int lineLength,int speed);. The problem now is that it only prints one line at a time. Of course I need to make this application multi-threaded, but I found myself struggling because I've never done this in C++, only in C# and in C# it is very easy to do compared to C++.
I did some research and tried to create 3 threads with CreateThread and then launching them with WaitForMultipleObjects. But output is very weird and doesn't seem correct. And it also leads me to next problem even if this would work correctly. Imagine that I want to launch 15+ lines on my console, does that mean that I need to create 15 different threads?
Note that this is not something important, it's just something I just created because I was bored and also because I want to learn threading with C++. I can, of course use boost libraries, but I want to create example for myself w/o using it.
Here is a screenshot with 1 thread only just to make it more clear:
This problem has nothing to do with threading- or rather, this problem absolutely doesn't require threading. A simple object-orientated design should easily allow you to draw multiple lines in a frame- how do you think single-threaded games draw thousands or more of vertices in a frame?
More importantly, only recent (Windows Vista or later) renderers for Windows allow multi-threaded rendering- that is, Direct2D/Direct3D11. Other renderers like D3D9 hold internal locks that prevent multi-threaded rendering because their run-times and the graphics drivers can't handle it.
If you do have a concurrency-safe renderer like Direct2D, then it should be relatively trivial to render from multiple threads- and you shouldn't be using the native Windows threading API. I see that you're using Visual Studio 2010- use the Concurrency runtime. The WinAPI only provides threading primitives- using them directly would be akin to writing in assembler. At least use boost::thread. Threading is not really about implementation, it's about good design, and design is irrelevant of what library you use to implement it.
Edit: Wait a minute, you're using the console? That's not thread-safe at all. That's insanity. The console exists for the most basic I/O ever, not for this. What's most likely is that C# just synchronizes it for you and C++ doesn't.
The function drawLine is probably one loop with a command to position the cursor, following with printing the character (if not then show us the function code). Those two instructions must execute sequentially, without some other instruction from other thread messing with the execution. So, introduce a lock (AKA critical section) that will guarantee that these two instructions execute orderly. Order of functions would be something like this:
EnterCriticalSection
SetConsoleCursorPosition
SetConsoleTextAttribute
WriteConsole
LeaveCriticalSection
The critical section is shared among all threads.
Single threaded approach is also an option, because the matrix strings don't interact. Simple solution would be something like this: Keep in memory data of all matrix strings, and just draw them one after another, in single thread. Sleep a little after all strings are drawn. Hopefully, the animation will look plausible.
More complex solution would be to use simple solution, but to have two screen buffers (with CreateConsoleScreenBuffer and SetConsoleActiveScreenBuffer), and switch them constantly to achieve instant drawing.
I'm developing a project and I have to make a wrapper to some hardware functions.
We have to write and read data to and from a non-volatile memory. I have a library with the read and write functions from the seller company. The problem is that these functions should be called with a delay between each call due to hardware characteristics.
So my solution is to start a thread, make a queue and make my own read and write functions. So every time my functions are called, the data will be stored on the queue and then in the loop thread will be actually read or written on the memory. My functions will use a mutex to synchronize the access to the queue. My wrapper is going to be on a dll. The main module will call my dll init function once to start the thread, and then it will call my read/write functions many times from different threads.
My questions is: Is it safe to do this? the original functions are non reentrant. I don't know if this is going to be a problem. Is there a better way to do this?
Any help will be appreciated.
Sorry I forgot something:
-The language to be used is C++
-The main program will call my wrapper dll but also will call other modules (dlls) that are going to call the wrapper dll.
Adding a mediator in this context is a pretty typical solution so you aren't out in the weeds here. I would say you would need to implement this because the original functions are not reentrant. Assuming, of course, that you own the access to the hardware. (i.e. You are the driver.) If other people can get access to the same piece of hardware, then you're going to have to come up with some higher level contract. Your thread then provides the ordered access to the driver. You'll find that the mediator will also allow you to throttle.
The hard part it seems is knowing when it is okay to make the next call to the device. Does it have some sort of flag to let you know it is ready for reads and writes? Some other questions: How do you plan to communicate state to your clients? Since you are providing an async interface, you'll need to have some sort of error callback registration, etc. Take a look at a normal async driver interface for ideas.
But overall, sounds like a good strategy to start with. As another poster mentioned, more specifics would be nice.