Our code has just started crashing due to a thread calling a memory alloc function and losing the pointer to the memory pool.
The pointer is initialised before the threads are started, but when the thread uses it to call the memory alloc code, it's zero.
In out init code we have
poolptr = InitMemoryPool ()
This sets it to a non zero memory address
In our .mm code on the thread we have
unsigned byte * p=(unsigned byte * ) MyAlloc ( poolptr, amount )
When the code gets into the MyAlloc function, poolptr is 0
Do I need my poolptr pointer to be volatile ? Even so, it's value is set up before the thread starts and never changes, so if the compiler is assuming it's a const, why doesn't it have it set correctly ?
Also, this has worked fine for years - and just started going wrong yesterday, simultaneously on two peoples machines.
Any ideas ?
This, what you mentioned, I don't do. What eventually worked for me is as follows:
I call my function or method and put in that function or method local instances of an class on the heap via command "new". Data that is to be returned is also paid respect to. Triggering a new thread will have access to that heap area if the heap area is a simple parameter. I.e., t= new thread( parameter);
void* function_or_method() {
clist *lstp;
string *_ps;
bool b;
try {
lstp= NULL;
lstp= new clist;
_ps= new string;
lstp->set( (void *)_ps);
mathclass *math;
thread *_thread;
math= new mathclass();
if((NULL==math))
throw Exception();
b= math->set( lstp);
if(! b) {
throw Exception();
}
_thread= new thread( math);
_thread->join();
delete _thread;
_thread= NULL;
} catch(const exception& e) {
clog <<"exception: logging" <<endl;
}
return (void*)lstp;
}
Okay, this is just C++ as well as C. I hope it will help a bit.
Related
I have a class and a library (lwip). For some reasons I need to call library's function of thread creation like :
/** The only thread function:
* Creates a new thread
* #param name human-readable name for the thread (used for debugging purposes)
* #param thread thread-function
* #param arg parameter passed to 'thread'
* #param stacksize stack size in bytes for the new thread (may be ignored by ports)
* #param prio priority of the new thread (may be ignored by ports) */
sys_thread_t sys_thread_new(const char *name, lwip_thread_fn thread, void *arg, int
stacksize, int prio);
Inside this function we call pthread:
code = pthread_create(&tmp,NULL,(void *(*)(void *)) function, arg);
My call looks like :
sys_thread_new("main_thread",(lwip_thread_fn)&this->main_thread, NULL,
DEFAULT_THREAD_STACKSIZE,DEFAULT_THREAD_PRIO);
My class method works fine, but I need to change some fielsd of CURRENT class (like 'state'
or else) I have an Idea to pass a pointer to current class to that thread and in thread function change class fields. Some kind of:
sys_thread_new("main_thread",(lwip_thread_fn)&this->main_thread, (void*)this,
DEFAULT_THREAD_STACKSIZE, DEFAULT_THREAD_PRIO);
Then in main_thread:
void lwip::main_thread(void *arg) {
lwip *p = (lwip*)arg;
p->state = 1;
}
Something like that. But it seems I do something wrong -
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff6e8e700 (LWP 4985)]
0x0000000000403a75 in lwip::main_thread (this=0x7fffffffe4f0, arg=0x80) at
../src/lwip.cpp:50
50 p->state = 1;
There are two problems here: If the main_thread member function is a static member function, you pass a pointer to it using &lwip::main_thread, no casting should be needed. If the function is not static, then you must make it static.
The other problem is that if the instance (this) you pass to the thread function is destructed, the thread function now has a pointer to a destructed object. Be careful with temporary object or passing instances by value.
If the actual thread function can't be static, you can easily solve it with a static wrapper function:
class lwip
{
...
private:
void main_thread() { ... }
static void* main_thread_wrapper(void* arg)
{
reinterpret_cast<lwip*>(arg)->main_thread();
return nullptr;
}
};
...
sys_thread_new("main_thread", &lwip::main_thread_wrapper, this,
DEFAULT_THREAD_STACKSIZE,DEFAULT_THREAD_PRIO);
If you have to cast the pointer to function to get
pthread_create to compile, you have undefined behavior.
If the goal is to call a member function in a different thread,
you need to wrap the call in an extern "C" function. This
means no members and no templates; in the simplest case:
extern "C" void*
startThread( void* p )
{
static_cast<T*>(p)->f();
}
and pass the address of startThread as third argument and
a pointer to the object as fourth. If inheritance is involved,
you must ensure that the fourth argument has the same type as
that in the cast in startThread, e.g.:
pthread_create( &tmp, nullptr, &startThread, static_cast<Base*>( pointerToDerived ) );
if startThread casts to Base*.
If you need arguments to the function as well, you need to pass
a pointer to a struct with both the pointer to the object and
the additional arguments. You also need to ensure that the
lifetime of this struct is sufficient, so that there is no risk
of the thread accessing an already inexistant object. This
often means an additional conditional variable, to ensure that
the thread calling pthread_create doesn't continue before the
new thread has made a copy of all of the relevant data. (Both
Boost threads and the C++11 threads do this for you. It's only
necessary if you need additional data, other than just the
pointer to the object, in the new thread.)
This can get painful if you need to do it for many different
types, and downright impossible if the class in question is
a template. In such cases, one common solution is to use
a Thread object, along the lines of:
class Thread
{
public:
virtual void* run() = 0;
};
and a starter function:
namespace {
extern "C" void*
doStartThread( void* p )
{
return static_cast<Thread*>( p )->run();
}
}
pthread_t
startThread( Thread* thread )
{
pthread_t results;
if ( pthread_create( &results, nullptr, doStartThread, thread ) != 0 ) {
throw std::runtime_error( "Could not create thread" );
}
}
Afterwards, you inherit from Thread, overriding the run
function with whatever you want (and adding any additional data
you might need); the derived class can even be a template.
Again, the lifetime of the Thread object is an issue; the
solution I've usually used has been to require it to be
dynamically allocated, and then delete it at the end of
doStartThread. It's a very good idea to catch it in an
std::unique_ptr in doStartThread, although you still want to
catch exceptions in this function, since otherwise they will
kill the process. And don't forget the delete if
pthread_create fails (since the caller has passed over
ownership. If you really want to be sure:
namespace {
extern "C" void*
doStartThread( void* p )
{
std::unique_ptr<Thread*> object( static_cast<Thread*>( p ) );
try {
return object->run();
} catch ( ... ) {
return somethingElseToReportTheError;
}
}
}
pthread_t
startThread( std::unique_ptr<Thread> thread )
{
pthread_t results;
if ( pthread_create( &results, nullptr, doStartThread, thread.get() ) != 0 ) {
throw std::runtime_error( "Could not create thread" );
}
thread.release(); // AFTER the call has succeeded!
}
I've used this technique successfully in a number of
applications (using std::auto_ptr, since there was no
std::unique_ptr then); typically, the fact that you need to
use dynamic allocation is not an issue, and it solves the
lifetime issue quite nicely. (The alternative would be to use
a conditional variable, blocking the original thread until the
new thread had copied everything over.)
Note that by using a unique_ptr in the interface, you
effectively block the calling thread from further access to the
thread object, by robbing it of its pointer to the object. This
offers an additional guarantee with regards to thread safety.
But of course, this additional guarantee (and the solution to
the lifetime issues) only applies to the Thread object itself,
and not to anything it might point to.
I have overloaded new function but unfortunetly never been able to execute global handler for requesting more memory access on my compiler. I also don't understand as per below code snippet if we invoke the
global handler for requesting more memory how it is gling to allocate to P.
I appreciate if anybody can through some light on this
void * Pool:: operator new ( size_t size ) throw( const char *)
{
int n=0;
while(1)
{
void *p = malloc (100000000L);
if(p==0)
{
new_handler ghd= set_new_handler(0);//deinstall curent handler
set_new_handler(ghd);// install global handler for more memory access
if(ghd)
(*ghd)();
else
throw "out of memory exception";
}
else
{
return p;
}
}
}
To have any effect, some other part of the program must have installed a global handler previously. That handler must also have some kind of memory to release when the handler is called (perhaps some buffers or cache that can be discarded).
The default new_handler is just a null pointer, so your code is very likely to end up throwing an exception.
Also, I would have thrown a bad_alloc exception to be consistent with other operator new overloads.
Here are two things to discuss, the first is using new_handler, the second is overloading operator new.
set_new_handler()
When you want use a new_handler, you have to register it. It is typically the first thing to do after entering main(). The handler should also be provided by you.
#include <iostream>
#include <new>
void noMemory() throw()
{
std::cout << "no memory" << std::endl;
exit(-1);
}
int main()
{
set_new_handler(noMemory);
// this will probably fail and noMemory() will be called
char *c = new char[100000000L];
std::cout << "end" << std::endl;
}
When no memory can be allocated, your registered handler will be called, and you have the chance to free up some memory. When the handler returns, operator new will give another try to allocate the amount of memory you requested.
operator new
The structure of the default operator new is something similar you presented. From the point of the new_handler the important part is the while(1) loop, since it is responsible for trying to get memory after called the new_handler.
There is two way out of this while(1) loop:
getting a valid pointer
throwing an exception
You have to have this in mind when you provide a new_handler, because if you can not do anything to free up memory you should deinstall the handler (or terminating or throwing an exception), otherwise you can stuck in an endless loop.
I guess omitting parameter size in your code is just for test purpose.
Also see Scott Meyers' Effective C++ Item 7 for details. As operator new must return a valid pointer even with parameter size = 0, the first thing to do in your operator new should be overwriting size to 1 in case of the user want to allocate 0 number of bytes. This trick is simple and fairly effective.
I have a program that processes neural spike data that is broadcast in UDP packets on a local network.
My current program has two threads a UI thread and a worker thread. The worker thread simply listens for data packets, parses them and makes them available to the UI thread for display and processing. My current implementation works just fine. However for a variety of reasons I'm trying to re-write the program in C++ using an Object Oriented approach.
The current working program initialized the 2nd thread with:
pthread_t netThread;
net = NetCom::initUdpRx(host,port);
pthread_create(&netThread, NULL, getNetSpike, (void *)NULL);
Here is the getNetSpike function that is called by the new thread:
void *getNetSpike(void *ptr){
while(true)
{
spike_net_t s;
NetCom::rxSpike(net, &s);
spikeBuff[writeIdx] = s;
writeIdx = incrementIdx(writeIdx);
nSpikes+=1;
totalSpikesRead++;
}
}
Now in my new OO version of the program I setup the 2nd thread in much the same way:
void SpikePlot::initNetworkRxThread(){
pthread_t netThread;
net = NetCom::initUdpRx(host,port);
pthread_create(&netThread, NULL, networkThreadFunc, this);
}
However, because pthead_create takes a pointer to a void function and not a pointer to an object's member method I needed to create this simple function that wraps the SpikePlot.getNetworSpikePacket() method
void *networkThreadFunc(void *ptr){
SpikePlot *sp = reinterpret_cast<SpikePlot *>(ptr);
while(true)
{
sp->getNetworkSpikePacket();
}
}
Which then calls the getNetworkSpikePacket() method:
void SpikePlot::getNetworkSpikePacket(){
spike_net_t s;
NetCom::rxSpike(net, &s);
spikeBuff[writeIdx] = s; // <--- SegFault/BusError occurs on this line
writeIdx = incrementIdx(writeIdx);
nSpikes+=1;
totalSpikesRead++;
}
The code for the two implementations is nearly identical but the 2nd implementation (OO version) crashes with a SegFault or BusError after the first packet that is read. Using printf I've narrowed down which line is causing the error:
spikeBuff[writeIdx] = s;
and for the life of me I can't figure out why its causing my program to crash.
What am I doing wrong here?
Update:
I define spikeBuff as a private member of the class:
class SpikePlot{
private:
static int const MAX_SPIKE_BUFF_SIZE = 50;
spike_net_t spikeBuff[MAX_SPIKE_BUFF_SIZE];
....
}
Then in the SpikePlot constructor I call:
bzero(&spikeBuff, sizeof(spikeBuff));
and set:
writeIdx =0;
Update 2: Ok something really weird is going on with my index variables. To test their sanity I changed getNetworkSpikePacket to:
void TetrodePlot::getNetworkSpikePacket(){
printf("Before:writeIdx:%d nspikes:%d totSpike:%d\n", writeIdx, nSpikes, totalSpikesRead);
spike_net_t s;
NetCom::rxSpike(net, &s);
// spikeBuff[writeIdx] = s;
writeIdx++;// = incrementIdx(writeIdx);
// if (writeIdx>=MAX_SPIKE_BUFF_SIZE)
// writeIdx = 0;
nSpikes += 1;
totalSpikesRead += 1;
printf("After:writeIdx:%d nspikes:%d totSpike:%d\n\n", writeIdx, nSpikes, totalSpikesRead);
}
And I get the following output to the console:
Before:writeIdx:0 nspikes:0 totSpike:0
After:writeIdx:1 nspikes:32763 totSpike:2053729378
Before:writeIdx:1 nspikes:32763 totSpike:2053729378
After:writeIdx:1 nspikes:0 totSpike:1
Before:writeIdx:1 nspikes:0 totSpike:1
After:writeIdx:32768 nspikes:32768 totSpike:260289889
Before:writeIdx:32768 nspikes:32768 totSpike:260289889
After:writeIdx:32768 nspikes:32768 totSpike:260289890
This method is the only method where I update their values (besides the constructor where I set them to 0). All other uses of these variables are read only.
I'm going to go on a limb here and say all your problems are caused by the zeroing out of the spike_net_t array.
In C++ you must not zero out objects with non-[insert word for 'struct-like' here] members. i.e. if you have an object that contains a complex object (a std string, a vector, etc. etc.) you cannot zero it out, as this destroys the initialization of the object done in the constructor.
This may be wrong but....
You seemed to move the wait loop logic out of the method and into the static wrapper. With nothing holding the worker thread open, perhaps that thread terminates after the first time you wait for a UDP packet, so second time around, sp in the static method now points to an instance that has left scope and been destructed?
Can you try to assert(sp) in the wrapper before trying to call its getNetworkSpikePacket()?
It looks like your reinterpret_cast might be causing some problems. When you call pthread_create, you are passing in "this" which is a SpikePlot*, but inside networkThreadFunc, you are casting it to a TetrodePlot*.
Are SpikePlot and TetrodePlot related? This isn't called out in what you've posted.
If you are allocating the spikeBuff array anywhere then make sure you are allocating sufficient storage so writeIdx is not an out-of-bounds index.
I'd also check that initNetworkRxThread is being called on an allocated instance of spikePlot object (and not on just a declared pointer).
This question is a refinement of this one, which went in a different direction than expected.
In my multithreaded application, the main thread creates parameters and stores them:
typedef struct {
int parameter1;
double parameter2;
float* parameter3;
} jobParams;
typedef struct {
int ID;
void* params;
} jobData;
std::vector<jobData> jobs;
// main thread
for (int i = 0; i < nbJobs; ++i) {
jobParams* p = new jobParams;
// fill and store params
jobData data;
data.ID = i;
data.params = p;
jobs.push_back(data);
}
// start threads and wait for their execution
// delete parameters
for (int i = 0; i < jobs.size(); ++i) {
delete jobs[i].params;
}
Then, each thread gets a pointer to a set of parameters, and calls a job function with it:
// thread (generic for any job function and any type of params)
jobData* job = main->getNextParams();
jobFunction(job->ID, job->params);
The whole thing takes void* as argument to be able to use any structure for the parameters, but then the job function casts it back to the right struct:
void* jobFunction(void* param) {
jobParams* params = (jobParams*) param;
// do stuff
return 0;
}
My problem is the following: if I delete params at the end of jobFunction(), it works perfectly. However, I'd prefer to have the deletion taken care of by the threads or the main thread, such that I don't have to remember to delete the params for each jobFunction() that I write.
If I try to delete params just after calling jobFunction() in the treads, or even in the main thread after being sure that all threads are done (and thus the params are not needed anymore), I get a heap corruption error:
HEAP[prog]: Invalid Address specified to RtlFreeHeap( 02E90000, 03C2EE38 )
I'm using Visual Studio 2008 Pro, and I thus can't use valgrind or other *nix tools for debugging. All access to the main thread from the "child threads" are synchronized using a mutex, so the problem is not that I delete the same parameters twice.
In fact, by using VS memory viewer, I know that the memory pointed by the jobParams pointer does not change between the end of jobFunction() and the point where I try to delete it (either in the main thread or in the "child threads").
I added the definition of both structures, as well as the way I'd like to delete the params.
Just as a thought .. can you try
for (int i = 0; i < jobs.size(); ++i) {
delete (jobParams*)jobs[i].params;
}
newing a type jobParams and then deleteing a void* might be the cause of your problems.
Is there any reason you store params as a void* in jobData? I'd argue if you wish to have different types of jobParams then you should be using an inheritance hierarchy and not blindly casting to a void*.
That sort of bug generally means you have a data race somewhere. Does main->getNextParams() do the right thing even if it's called by several threads at once? If it gives the same params to both, you could have a double-free in your hands.
Also, instead of
jobFunction(jobData->ID, jobData->params);
You probably meant
jobFunction(job->ID, job->params);
To debug it you could add a deleted member to the jobParams class and set that to true instead of actually deleting the object. Then see check the deleted flag in every method of jobParams and throw an exception if it's true. Then see where the exception gets thrown.
Could someone please help with this? I have the following:
// part_1
if (pthread_create(&threadID, NULL, ThreadMain,
(void *) clientSocket) != 0) {
cerr << "Unable to create thread" << endl;
exit(1);
}
// part_2
void *ThreadMain(void *clientSocket) {
pthread_detach(pthread_self());
...
delete (TCPSocket *) clientSocket;
return NULL;
}
I would to have part_2 in part_1 ( I mean without calling TreadMain() function )
thanks for your replies
What if you could do this?
pthread_create() has to create a new stack for the new thread to use. If pthread_create() copied the existing thread's old stack into a separate block of memory to create a new stack, then what would happen to local variables that point to other local variables? The local variables on the new stack would point to the old stack. When the thread using the old stack (the one that called pthread_create()) returns or overwrites them, those variables start pointing to invalid data.
But what about fork()? Why doesn't fork() take a function pointer like pthread_create()? Well, unlike pthread_create(), the new thread of execution created by fork() doesn't have to live in the same address space as the old thread of execution, because they are in separate processes. fork() clones the parent process's virtual address space, stack, heap, and all. The child process's stack variables have the same virtual addresses as the corresponding variables in the parent process. So all of the pointers in the new child process continue to be valid*, no matter what the old parent process does with the memory.
* Nitpicker's corner: excluding pointers that were invalid to begin with, as well as memory that is deliberately shared
If all you want to do is simply move the function for part2 inside part1, you can create a local class inside of part1, with a static member function...
class LocalFunctor
{
public:
static void *ThreadFunc(void* clientSocket)
{
pthread_detach(pthread_self());
...
delete (TCPSocket *) clientSocket;
return NULL;
}
};
then call LocalFunctor::ThreadFunc within pthread_create
pthread_create(&threadID, NULL, LocalFunctor::ThreadFunc,(void *) clientSocket)
If you're going to do this more than once, look at boost::thread or wrap this up inside a template helper class.
You could get an Apple Macintosh computer with OS 10.6 and start programming with Grand Central Dispatch. Apple added some new C compiler features that do almost exactly what you seem to want. They're called Blocks.