Passing handle to C++ classes across a C API boundary - c++

Question

I am writing a library in C++, but want it to have a C API, which should also be thread safe. One thing that the API needs to do is to pass back and forth handles (e.g. a structure containing a reference or pointer) of objects created within the library. These objects need to be destroyed at some point, so any handles to such an object that were still in existence would them become invalid.
EDIT: We cannot assume that each handle is only used within a single client thread. In particular I want to handle the case where there are two client threads accessing the same resource simultaneously, one trying to destroy it while the other tries to modify it
There are two paradigms for dealing with this. One is the smart pointer paradigm, for example boost::shared_ptr or std::shared_ptr, which make sure the object is only destroyed when there are no more references to it. However, as I understand it, such pointers aren't possible to implement in C (as it doesn't support constructors and destructors), so this approach won't work in my case. I don't want to rely on the user to call a "release" function for every instance of a handle they obtain, since they inevitably won't.
Another paradigm is to simply destroy the object within the library, and have any subsequent function calls which pass a handle to that object as an input simply return an error code. My question is what techniques or libraries are available to implement this approach, specifically in a multi-threaded application?
EDIT:
The library should of course handle all the allocation of deallocation of internal memory itself. Smart pointers may be also used within the library; they may not be passed across the API, however.
EDIT:
More detail on handles might be used on the client side: the client might have two threads, one of which creates an object. That thread might pass the handle to the second thread, or the second thread might get it from the library using a "find_object" function. The second thread might then continuously update the object, but while that is going on the first thread might destroy the object, making all handles to the object invalid.
I appreciate rough suggestions of an approach - I have come up with some of these myself too. However, the nitty gritty details of things like how a C++ class instance is retrieved and locked, given a handle, while other threads may be attempting to destroy that object, are non-trivial, so I'm really after answers like "I have done the following and it works without fail." or, even better, "This library implements what you're after sensibly and safely".

Answer 1

IMHO handles (e.g. simple integer numbers) that are kept as key values in a map of smart pointers internally might be a viable solution.
Though you'll need to have a mechanism that guarantees a handle that was freed by a particular client won't destroy the map entry, as long the handle is still used by other clients in a multithreaded environment.
UPDATE:
At least #user1610015 answer isn't such a bad idea (if you're working with a COM enabled environment). In any case you'll need to keep track with some reference counting of your internally managed class instances.
I'm not so experienced with the C++11 or boosts smart pointer features, and how it's possible to intercept or override reference counting mechanism with these. But e.g. Alexandrescou's loki library smart pointer considerations may let you implement an appropriate policy on how you're going to handle the reference counting and which interfaces can access this or not.
It should be possible with Alexandrescou's Smart Pointer to provide a ref counting mechanism that supports access via a C-API and C++ internal implementation concurrently and thread safe.

Answer 2

One thing that the API needs to do is to pass back and forth handles
Ok so far
(e.g. a structure containing a reference or pointer)
Why? A "handle" is just a way to identify an object. Doesn't necessarily mean it has to hold the reference or pointer.
One is the smart pointer paradigm, for example boost::shared_ptr or std::shared_ptr, which make sure the object is only destroyed when there are no more references to it.
Sure, a
map<int, boost::shared_ptr<my_object>>
might work fine here if you want to use it for your memory deallocation mechanism.
simply destroy the object within the library,
This can exist with smart pointers, its not one or the other.
have any subsequent function calls which pass a handle to that object as an input simply return an error code.
sure sounds good.
If your library is responsible for allocating memory, then it should be responsible for deallocating memory.
I would return simple integer "handles" from the library _GetNewObject() method.
Your library needs a map of handles to internal objects. No one outside the library should see the objects from the C interface.
All the library methods should take a handle as their first parameter.
For the multi-threaded stuff, do two threads need to access the same object? If so, you'll need to put in some sort of locking that occurs when a C API function is entered, and released before it leaves. You will have to make a decision if you want the code outside the library to know about this locking (you probably don't), a C function that calls the library function will probably just want to get the return value and not worry about the lock/unlock.
So your library needs:
an interface to allocate and deallocate objects, viewed as handles by the outside
an interface to do stuff given the handles.
EDIT: MORE INFO
Within the library I would use a Factory pattern to create new objects. The Factory should hand out a shared_ptr after the object allocation. This way everything else in the library just uses shared_ptr, and the cleanup will be fairly automatic (i.e. the factory doesn't have to store a list of what is created to remember to clean up, and no one has to call delete explicitly). Store the shared_ptr in the map with the handle. You'll probably need some sort of static counter along with a GetNextHandle() function to get the next available handle and to deal with wrap around (depending on how many objects are created and destroyed within the lifetime of the running program).
Next, put your shared pointer into a Proxy. The proxy should be very lightweight and you can have many Proxy objects per actual object. Each proxy will hold a private shared_ptr and whatever thread / mutex object you choose to use (you haven't given any information about this, so its hard to be any more specific). When a Proxy is created, it should acquire the mutex, and release on destruction (i.e. RAII for releasing the lock).
You haven't included any information on how to determine if you want to create a new object or find an existing object, and how two different threads would "find" the same object. However, lets assume that you have a GetObject() with enough parameters to uniquely identify each object and return the handle from the map, if the objects exists.
In this case, each of your visible extern C library functions would accept an object handle and:
Create a new Proxy for the given handle. In the Proxy constructor, the Proxy would look in the map to find the handle, if it doesn't exist, ask the Factory to create one (or return an error, your choice here). The Proxy would then acquire the lock. Your function would then get the pointer from the Proxy and use it. When the function exits, the Proxy goes out of scope, releases the lock, and decrements the reference counter.
If two functions are running in different threads, as long as a Proxy exists in one of the functions, the object will still exist. The other function could ask the library to delete the object which would remove the reference from the map. Once all the other functions with active Proxy objects finish, the final shared_ptr will go out of scope and the object will be deleted.
You can do most of this generically with templates, or write concrete classes.
EDIT: MORE INFO
The Proxy will be a small class. It will have a shared_ptr, and have a lock. You would instantiate a Proxy within the scope of the extern C function called by the client (note, this is actually a C++ function with all the benefits such as being able to use C++ classes). The proxy is small and should go on the stack (do not new and delete this, more trouble than its worth, just make a scoped variable and let C++ do the work for you). The proxy would use the RAII pattern to get a copy of the shared_ptr (which would increment the reference count of the shared_ptr) and acquire the lock at construction. When the Proxy goes out of scope, the shared_ptr it has is destroyed thus decrementing the reference count. The Proxy destructor should release the lock. BTW, you may want to think about blocking and how you want your thread mutexes to work. I don't know enough about your specific mutex implementation to suggest anything yet.
The map will contain the "master" shared_ptr that all others are copied from. However, this is flexible and decoupled because once a Proxy gets a shared_ptr from the map, it doesn't have to worry about "giving it back". The shared_ptr in the map can be removed (i.e. the object no longer "exists" to the Factory), but there can still be Proxy classes that have a shared_ptr, so the actual object will still exist as long as something is using it.

Answer 3

When code uses handles, it is almost always responsible for calling a dispose handle function. And operating on a disposed handle is then illegal.
Handles are usually either pointers to forward decl struct with no body, or pointers to structwith a preable field or two (maybe to help debugging on the C side) that are incomplete. Creation and destruction of them occur inside the API.
In the API you have a full view ofthe contents of the handle, which need not be a C struct -- it can have unique ptrs or whatever. You will have to delete the handle manually, but that is inevidable.
As noted below, another possible handle is a guid, with inside the API having a map from guid to data. This is slow, but the space of guids is practically infinite, so you can detect use of erased handles and return an error. Note that failure to return a handle leaks the resource, but this eliminates dangling pointer segfaults at a modsst runtime cost.

Answer 4

Another way is to expose a COM API. This has the benefit that it's object-oriented, unlike a C API. But it still can be used from C. It would look like this:
C++:
// Instantiation:
ISomeObject* pObject;
HRESULT hr = CoCreateInstance(..., IID_PPV_ARGS(&pObject));
// Use:
hr = pObject->SomeMethod(...);
// Cleanup:
pObject->Release();
C:
// Instantiation:
ISomeObject* pObject;
HRESULT hr = CoCreateInstance(..., IID_PPV_ARGS(&pObject));
// Use:
hr = (*pObject->lpVtbl->SomeMethod)(pObject, ...);
// Cleanup:
(*pObject->lpVtbl->Release)(pObject);
Also, if the client is C++, it can use a COM smart pointer like ATL's CComPtr to automate memory management. So the C++ code could be turned into:
// Instantiation:
CComPtr<ISomeObject> pSomeObject;
HRESULT hr = pSomeObject.CoCreateInstance(...);
// Use:
hr = pSomeObject->SomeMethod(...);
// Cleanup is done automatically at the end of the current scope

Answer 5

Perhaps I have too much time on my hands... but I've thought about this a few times and decided to just go ahead and implement it. C++ calleable, no external libs. Totally reinvented the wheel, just for fun (if you can call coding on a sunday fun.)
Note, synchronization is not here, because I don't know what OS you are on...
SmartPointers.h:
#ifndef SMARTPOINTER_H
#define SMARTPOINTER_H
#ifdef __cplusplus
extern "C" {
#endif
#ifndef __cplusplus
#define bool int
#define true (1 == 1)
#define false (1 == 0)
#endif
// Forward declarations
struct tSmartPtr;
typedef struct tSmartPtr SmartPtr;
struct tSmartPtrRef;
typedef struct tSmartPtrRef SmartPtrRef;
// Type used to describe the object referenced.
typedef void * RefObjectPtr;
// Type used to describe the object that owns a reference.
typedef void * OwnerPtr;
// "Virtual" destructor, called when all references are freed.
typedef void (*ObjectDestructorFunctionPtr)(RefObjectPtr pObjectToDestruct);
// Create a smart pointer to the object pObjectToReference, and pass a destructor that knows how to delete the object.
SmartPtr *SmartPtrCreate( RefObjectPtr pObjectToReference, ObjectDestructorFunctionPtr Destructor );
// Make a new reference to the object, pass in a pointer to the object that will own the reference. Returns a new object reference.
SmartPtrRef *SmartPtrMakeRef( SmartPtr *pSmartPtr, OwnerPtr pReferenceOwner );
// Remove a reference to an object, pass in a pointer to the object that owns the reference. If the last reference is removed, the object destructor is called.
bool SmartPtrRemoveRef( SmartPtr *pSmartPtr, OwnerPtr pReferenceOwner );
// Remove a reference via a pointer to the smart reference itself.
// Calls the destructor if all references are removed.
// Does SmartPtrRemoveRef() using internal pointers, so call either this or SmartPtrRemoveRef(), not both.
bool SmartPtrRefRemoveRef( SmartPtrRef *pRef );
// Get the pointer to the object that the SmartPointer points to.
void *SmartPtrRefGetObjectPtr( SmartPtrRef *pRef );
#ifdef __cplusplus
}
#endif
#endif // #ifndef SMARTPOINTER_H
SmartPointers.c:
#include "SmartPointers.h"
#include <string.h>
#include <stdlib.h>
#include <assert.h>
typedef struct tLinkedListNode {
struct tLinkedListNode *pNext;
} LinkedListNode;
typedef struct tLinkedList {
LinkedListNode dummyNode;
} LinkedList;
struct tSmartPtrRef {
LinkedListNode listNode;
OwnerPtr pReferenceOwner;
RefObjectPtr pObjectReferenced;
struct tSmartPtr *pSmartPtr;
};
struct tSmartPtr {
OwnerPtr pObjectRef;
ObjectDestructorFunctionPtr ObjectDestructorFnPtr;
LinkedList refList;
};
// Initialize singly linked list
static void LinkedListInit( LinkedList *pList )
{
pList->dummyNode.pNext = &pList->dummyNode;
}
// Add a node to the list
static void LinkedListAddNode( LinkedList *pList, LinkedListNode *pNode )
{
pNode->pNext = pList->dummyNode.pNext;
pList->dummyNode.pNext = pNode;
}
// Remove a node from the list
static bool LinkedListRemoveNode( LinkedList *pList, LinkedListNode *pNode )
{
bool removed = false;
LinkedListNode *pPrev = &pList->dummyNode;
while (pPrev->pNext != &pList->dummyNode) {
if (pPrev->pNext == pNode) {
pPrev->pNext = pNode->pNext;
removed = true;
break;
}
pPrev = pPrev->pNext;
}
return removed;
}
// Return true if list is empty.
static bool LinkedListIsEmpty( LinkedList *pList )
{
return (pList->dummyNode.pNext == &pList->dummyNode);
}
// Find a reference by pReferenceOwner
static SmartPtrRef * SmartPtrFindRef( SmartPtr *pSmartPtr, OwnerPtr pReferenceOwner )
{
SmartPtrRef *pFoundNode = NULL;
LinkedList * const pList = &pSmartPtr->refList;
LinkedListNode *pIter = pList->dummyNode.pNext;
while ((pIter != &pList->dummyNode) && (NULL == pFoundNode)) {
SmartPtrRef *pCmpNode = (SmartPtrRef *)pIter;
if (pCmpNode->pReferenceOwner == pReferenceOwner) {
pFoundNode = pCmpNode;
}
pIter = pIter->pNext;
}
return pFoundNode;
}
// Commented in header.
SmartPtrRef *SmartPtrMakeRef( SmartPtr *pSmartPtr, OwnerPtr pReferenceOwner )
{
// TODO: Synchronization here!
SmartPtrRef *pRef = (SmartPtrRef *)malloc(sizeof(SmartPtrRef) );
LinkedListAddNode( &pSmartPtr->refList, &pRef->listNode );
pRef->pReferenceOwner = pReferenceOwner;
pRef->pObjectReferenced = pSmartPtr->pObjectRef;
pRef->pSmartPtr = pSmartPtr;
return pRef;
}
// Commented in header.
bool SmartPtrRemoveRef( SmartPtr *pSmartPtr, OwnerPtr pReferenceOwner )
{
// TODO: Synchronization here!
SmartPtrRef *pRef = SmartPtrFindRef( pSmartPtr, pReferenceOwner );
if (NULL != pRef) {
assert( LinkedListRemoveNode( &pSmartPtr->refList, &pRef->listNode ) );
pRef->pReferenceOwner = NULL;
pRef->pObjectReferenced = NULL;
free( pRef );
if (LinkedListIsEmpty( &pSmartPtr->refList ) ) {
pSmartPtr->ObjectDestructorFnPtr( pSmartPtr->pObjectRef );
}
}
return (NULL != pRef);
}
// Commented in header.
bool SmartPtrRefRemoveRef( SmartPtrRef *pRef )
{
return SmartPtrRemoveRef( pRef->pSmartPtr, pRef->pReferenceOwner );
}
// Commented in header.
void *SmartPtrRefGetObjectPtr( SmartPtrRef *pRef )
{
return pRef->pObjectReferenced;
}
// Commented in header.
SmartPtr *SmartPtrCreate( void *pObjectToReference, ObjectDestructorFunctionPtr Destructor )
{
SmartPtr *pThis = (SmartPtr *)malloc( sizeof( SmartPtr ) );
memset( pThis, 0, sizeof( SmartPtr ) );
LinkedListInit( &pThis->refList );
pThis->ObjectDestructorFnPtr = Destructor;
pThis->pObjectRef = pObjectToReference;
return pThis;
}
And a test program (main.cpp)
// SmartPtrs.cpp : Defines the entry point for the console application.
//
#include "stdafx.h"
#include "SmartPointers.h"
#include <assert.h>
#include <stdlib.h>
#include <string.h>
typedef struct tMyRefObj {
int refs;
SmartPtr *pPointerToMe;
bool deleted;
} MyRefObj;
static bool objDestructed = false;
static MyRefObj *MyObjectGetReference( MyRefObj *pThis, void *pObjectReferencing )
{
// TODO: Synchronization here...
pThis->refs++;
SmartPtrRef * const pRef = SmartPtrMakeRef( pThis->pPointerToMe, pObjectReferencing );
return (MyRefObj *)SmartPtrRefGetObjectPtr( pRef );
}
static void MyObjectRemoveReference( MyRefObj *pThis, void *pObjectReferencing )
{
// TODO: Synchronization here...
pThis->refs--;
assert( SmartPtrRemoveRef( pThis->pPointerToMe, pObjectReferencing ) );
}
static void MyObjectDestructorFunction(void *pObjectToDestruct)
{
MyRefObj *pThis = (MyRefObj *)pObjectToDestruct;
assert( pThis->refs == 0 );
free( pThis );
objDestructed = true;
}
static MyRefObj *MyObjectConstructor( void )
{
MyRefObj *pMyRefObj =new MyRefObj;
memset( pMyRefObj, 0, sizeof( MyRefObj ) );
pMyRefObj->pPointerToMe = SmartPtrCreate( pMyRefObj, MyObjectDestructorFunction );
return pMyRefObj;
}
#define ARRSIZE 125
int main(int argc, char* argv[])
{
int i;
// Array of references
MyRefObj *refArray[ARRSIZE];
// Create an object to take references of.
MyRefObj *pNewObj = MyObjectConstructor();
// Create a bunch of references.
for (i = 0; i < ARRSIZE; i++) {
refArray[i] = MyObjectGetReference( pNewObj, &refArray[i] );
}
assert( pNewObj->refs == ARRSIZE );
for (i = 0; i < ARRSIZE; i++) {
MyObjectRemoveReference( pNewObj, &refArray[i] );
refArray[i] = NULL;
}
assert(objDestructed);
return 0;
}