Im trying to start a new thread using CreateThread and pass the pointee of a shared_pointer as an argument. If i was to use good old ATL, it would call Ccomptr's operator () and increase the ref count and eventually retrieve the pointer.
I'm trying to do the same using STL's shared_ptr, but I'm getting an error where conversion between std::shared_ptr to LPVOID doesn't exist.
std::shared_ptr<X> m_pSettings = std::make_shared(new ....);
if (nullptr == (m_hBackgroundThread = ::CreateThread(
nullptr, // default security attributes
0, // use default stack size
ThreadProc, // thread function name
m_pSettings, // argument to thread function
0, // use default creation flags
0)))
{
LOG_ERROR(L"Failed to create thread");
return false;
}
The correct answer, as you have already been told, is to use std::thread:
// Hey, look: No magic nullptrs and zeros nobody ever changes to non-defaults anyway!
std::thread t(ThreadProc, m_pSettings);
// Don't forget to
t.detach();
// if you let the std::thread object destruct without first joining it and making sure
// the thread finishes executing
Not only std::thread solves your problem, it has a multitude of other benefits such as allowing you to pass multiple parameters without fuss, it copies/moves/does-the-right-thing with the arguments you pass, etc. and it doesn't require you to resort to any supposedly clever but actually not hacks. If you use std::async instead of std::thread you can even get in the caller thread the exception that caused the thread function to unwind if there was one.
The alternative of passing CreateThread the address of your shared_ptr and then dereferencing it inside the ThreadProc might work, if you're extremely careful - at the very least you have to make sure the caller's shared_ptr doesn't go away until the ThreadProc is done with it - but that's just asking for bad things to happen.
Sure, if you had no alternatives that's what you do, but given std::thread and friends doing anything else is wrong and a waste of time.
Also, your comparison to ATL seems to be founded on a misunderstanding. The difference is not between ATL's smart pointers that are extra-clever and and are able in some magical way to keep the reference count even when giving you the raw pointer (though I am unaware of an CComPtr::operator()) while C++ standard library smart pointers are lame and don't do that. The difference is between special object who manage keep their own reference count and manage their own lifetimes given that you declare when you use them (i.e. call AddRef) and arbitrary resources.
A smart pointer for COM (be it CComPtr or _com_ptr_t or any half-baked lame class you fine online) is basically a wrapper around the COM object's innate capability to manage its own lifetime.
You could have just as much added AddRed, Release, etc. (with the associated semantics, obviously) to your class X and wrapped it in a COM smart pointer...
You could pass shared_ptr object by pointer like this: &m_pSettings
Then dereference it in your ThreadProc :
std::shared_ptr<X> pSettings = *static_cast<std::shared_ptr<X>*>(lpParam);
Probably you need to copy your shared_ptr first before passing pointer to it into CreateTrhead so it'll be in scope while ThreadProc running.
Related
In most of my programming now a day I put everything in a smart pointer and forget about it. The resource is properly managed 99.9% of the time. It's really great and way better than a garbage collection mechanism.
However, once in a while, the resource being held by a smart pointer needs to be explicitly freed before we can reallocate a new instance of it. Something like this:
r = std::make_shared<my_resource>(with_this_id);
r->do_work();
...
r->do_more_work();
...
r->do_even_more_work();
r.reset();
r = std::make_shared<my_resource>(with_this_id);
...
If I miss the r.reset() call, the resource may either be using a large mount of memory or disk space and re-allocating without first resetting is likely to cause problems on smaller computers. Either that, or the resource is locked so it can't be reallocated until explicitly freed.
Is there a pattern/algorithm/something which handles such a case in a cleaner manner?
I see basically two ways to approach this. The first is to wrap the reset-assign sequence into one function and never assign directly. I would probably do this like such
template<typename T, typename ...Args>
void reset_and_assign(std::shared_ptr<T> &ptr, Args ... &&args) { //in-out parameter to avoid copy since you cannot rvo on a parameter
ptr.reset();
ptr.reset(new T(std::forward<Args> args...));
}
This is pretty easy and fast to do but it won't save you from accidentally calling the assignment directly and I don't see a way to do this if you keep using shared_ptr.
The other alternative is writing a wrapper around shared_ptr that just forwards most function calls and changes reset and assignment such that it will first deallocate and then create the new resource. This is a bit work to do and it is easy to get a bug into this (especially if you mess up some universal references if you try to save yourself some constructors). It will also be annoying to interact with other code that uses std smart pointers and will be a major refactoring process. But you cannot mess it up by accidentally calling an assignment (at least probably not).
Also note that the standard library does reset intentionally in this order such that we don't delete the old resource if the new allocation throws.
Although it may sound bizarre, I believe this can be better expressed as move semantics. Here is why:
The new resource object you create is a replacement of the old one. Assume that the replacement is transparent to your users (i.e. they perceive the new resource object and the old one as the same). Therefore, it is as if you performed a imagined copy of the old object to obtain the new one, and then destroy the old one. This matches the intention of move semantics.
So my_resource should have a move constructor:
my_resource(my_resource&& old) {
old.reset();
/* code to init the new resource object */
}
// replace
ptr = std::make_unique<my_resource>(std::move(*ptr));
Normally you just put the reset() inside the destructor of the my_resource class.
If this isn't suitable for whatever reason, upon instantiation, you can put custom destructor function into the std::shared_ptr that calls the reset() and then deletes the resource.
If the shared_ptr doesn't reach the refcount 0 - are you sure you use it correctly? There is std::weak_ptr - a utility class for shared_ptr - made for the whole purpose of when you need both safety and timely deallocation.
Also, why use make_shared and instatiate new my_resource instead of simply using some init function of my_resource that will automatically call reset() if needed?
I am writing code that utilizes COM interfaces. I am basing my code on examples that I have found online. I do not want to utilize smart pointers in this case because I want to understand the basics of COM and not just have a smart pointer class do all of the work for me.
In order to frame my questions, let's assume I have a class similar to the following:
public class TestClass
{
private:
IUnknown *m_pUnknown;
public:
TestClass();
void AssignValue();
}
TestClass::TestClass()
{
m_pUnknown = NULL;
}
void TestClass::AssignValue()
{
IUnknown *pUnknown = NULL;
//Assign value to pUnknown here - not relevant to my questions
m_pUnknown = pUnknown;
pUnknown->Release();
}
Now on to my specific questions.
1) The examples I've seen to not use AddRef() when initializing a value, such as in the class constructor. Does the AddRef() happen "automatically" behind the scenes when a COM pointer is first assigned a value?
2) Although my code example does not show it, it is my understanding that in the AssignValue() method, when you assign a second value to overwrite the value of pUnknown (originally set in the class constructor), Release() is automatically called. After assigning the new value to pUnknown its reference count stands at zero. I need to call pUnknown->AddRef() immediately after the reassignment. Is my understanding correct?
Notes: I assume we are ignoring exceptions for simplicity here. If this was for real, you would want to use smart pointers to help keep things straight in the presence of exceptions. Similarly, I am not worrying about proper copying or destruction of instances of your example class or multi-threading. (Your raw pointers cannot be used from different threads as simply as you might assume.)
First, You need to make any necessary calls to COM. The only way anything might happen "automatically" behind the scenes would be if you were using smart pointers to do them.
1) The examples you refer to have to be getting their COM interface pointers from somewhere. This would be by making COM calls, e.g., CoCreateInstance() and QueryInterface(). These calls are passed the address of your raw pointer and set that raw pointer to the appropriate value. If they weren't also implicitly AddRef'ed, the reference count might be 0 and COM could delete the associated COM object before your program could do anything about it. So such COM calls must include an implicit AddRef() on your behalf. You are responsible for a Release() to match this implicit AddRef() that you instigated with one of these other calls.
2a) Raw pointers are raw pointers. Their value is garbage until you arrange for them to be set to something valid. In particular, assigning a value to one will NOT auto-magically call a function. Assigning to a raw pointer to an interface does not call Release() - you need to do that at the appropriate time. In your post, it appears that you are "overwriting" a raw pointer that had previously been set to NULL, hence there was no existing COM interface instance in the picture. There could not have been an AddRef() on something that doesn't exist, and must not be a Release() on something that isn't there.
2b)
Some of the code you indicated by a comment in your example is very relevant, but can easily be inferred. You have a local raw pointer variable, pUnknown. In the absent code, you presumably use a COM call that obtains an interface pointer, implicitly AddRefs it, and fills in your raw pointer with the proper value to use it. This gives you the responsibility for one corresponding Release() when you are done with it.
Next, you set a member raw pointer variable (m_pUnknown) with this same value. Depending on the previous use of this member variable, you might have needed to call Release() with its former value before doing this.
You now have 2 raw pointers set to the value to work with this COM interface instance and responsibility for one Release() due to 1 implicit AddRef() call. There are two ways to deal with this, but neither is quite what you have in your sample.
The first, most straightforward, and proper approach (which others have correctly pointed out & I skipped passed in the first version of this answer) is one AddRef() and one Release() per pointer. Your code is missing this for m_pUnknown. This requires adding m_pUnknown->AddRef() immediately after the assignment to m_pUnknown and 1 corresponding call to Release() "someplace else" when you are done using the current interface pointer from m_pUnknown. One usual candidate for this "someplace else" in your code is in the class destructor.
The second approach is more efficient, but less obvious. Even if you decide not to use it, you may see it, so should at least be aware of it. Following the first approach you would have the code sequence:
m_pUnknown = pUnknown;
m_pUnknown->AddRef();
pUnknown->Release();
Since pUnknown and m_pUnknown are set the same here, the Release() is immediately undoing the AddRef(). In this circumstance, eliding this AddRef/Release pair is reference count neutral and saves 2 round trips into COM. My mental model for this is a transfer of the interface and reference count from one pointer to the other. (With smart pointers it would look like newPtr.Attach( oldPtr.Detach() ); ) This approach leaves you with the original/not shown implicit AddRef() and needing to add the same m_pUnknown->Release() "someplace else" as in the first alternative.
In either approach, you exactly match AddRefs (implicit or explicit) with Releases for each interface and never go to a 0 reference count until you are done with the interface. Once you do hit 0, you do not attempt to use the value in the pointer.
Avi Berger already posted a great answer, but here is the same thing stated another way in case it helps with understanding.
In COM, reference counting is done within the COM object. The COM runtime will destruct and free an object whose reference count reaches 0. (This might be delayed by some time from the point of the count hitting 0).
Everything else is a convention. The usual convention amongst C++ COM programmers is that raw interface pointers should be treated as owning pointers. This concept means that any time a pointer points to a COM object, the pointer owns that object.
Using this terminology, the object may have multiple owners at any one time, and the object will be destroyed when nobody owns it.
However, raw pointers in C++ don't have ownership semantics built in. So you have to implement it yourself by making function calls:
Call AddRef on an interface pointer when that pointer takes ownership of an object. (You'll need to be aware of which Windows API functions or other library functions already do this, to avoid you doing it twice)
Call Release on an interface pointer when that pointer is about to stop owning an object.
The benefit of smart pointers is that they make it impossible for you to forget to call Release when an interface pointer stops owning an object. This includes the following cases:
Pointer goes out of scope.
Pointer is made to stop pointing to the object, by using assignment operator.
So, looking at your sample code. You have the pointer m_pUnknown. You want this pointer to take ownership of the object, so the code should be:
m_pUnknown = pUnknown;
m_pUnknown->AddRef();
You will also need to add code to your class destructor and your class assignment operator to call m_pUnknown->Release(). I would very strongly recommend wrapping these calls in the smallest class possible (that is, write your own smart pointer and make TestClass have that smart pointer as a member variable). Assuming of course you don't want to use an existing COM smart pointer class for pedagogical reasons.
The call pUnknown->Release(); is correct because pUnknown currently owns the object, and the pointer is about to stop owning the object due to the fact that it will be destroyed when the function block ends.
You may observe that it would be possible to remove both of the lines m_pUnknown->AddRef() and pUnknown->Release(). The code will behave exactly the same. However , it is better to follow the convention outlined above. Sticking to a convention helps yourself to avoid errors and it also helps other coders to understand your code.
To put it another way, the usual convention is to think of the pointer as having a reference count of either 0 or 1, even though the reference counting is not actually implemented that way.
First, my apologies. My attempt to simplify my code for the sake of clarity turned out to be misguided. However, I believe my questions were answered. If I may, I will summarize.
1) Any COM object that is assigned a value other than NULL needs to be immediately followed by AddRef() unless the AddRef() was implicitly handled (as is the case with some Windows API calls).
2) Any reassignment of value to a COM pointer, assuming that the "before" value is not NULL must be immediately proceeded by Release(). AddRef() would then by needed as mentioned in #1.
3) Any COM variable whose value needs to be preserved beyond its current scope requires that it have a reference count of at least 1 upon exiting its said scope. This may mean that an AddRef() is required.
Would this be a fair summary? Did I miss anything?
In boost.thread's start function, the source code is something like that:
bool thread::start_thread_noexcept()
{
uintptr_t const new_thread = _beginthreadex(
0,
0,
&thread_start_function,
thread_info.get(),
CREATE_SUSPENDED,
&thread_info->id);
if (!new_thread)
{
return false;
}
// why call this line?
intrusive_ptr_add_ref(thread_info.get());
thread_info->thread_handle = (detail::win32::handle)(new_thread);
ResumeThread(thread_info->thread_handle);
return true;
}
thread_info is a intrusive smart pointer which points to the thread information data, before calling the intrusive_ptr_add_ref, the count is already 1, I don't know why call the intrusive_ptr_add_ref mannually here. I think Intrusive smart pointer's job should be calling the intrusive_ptr_add_ref and intrusive_ptr_release automatically.
I've tried to step through the source code but didn't find any clue.
Can anyone tell me
1. why call intrusive_ptr_add_ref manually here?
2. In what condition when using the intrusive_ptr, I should call intrusive_ptr_add_ref manually?
Thanks, Sincerely.
why call intrusive_ptr_add_ref manually here?
To represent the sharing of ownership of the pointer.
_beginthreadex was passed thread_info.get() as a parameter. This parameter will be passed to thread_start_function when the thread starts. And this function expects the pointer to remain valid until that happens.
Now, _beginthreadex is a simple function. It's not a variadic template that can take arbitrary parameters or anything. It takes exactly and only a naked pointer, and passes exactly that to the start function.
It is very possible for the person creating the boost::thread to call thread::detach before thread_start_function ever gets called. And if that happened, then the thread_info intrusive pointer would be destroyed, thus causing the destruction of its contained object.
And that leaves _beginthreadex with a destroyed pointer. That's bad.
What _beginthreadex needs to do is claim ownership of the intrusvie pointer. But since the API doesn't take a boost::intrusive_ptr, how do you do that?
By bumping the reference count. The reference count increase is how _beginthreadex claims ownership of the object.
I have been working with d3d11 for quite a while now, and after discovering the directx debugger, i've recently discovered that my program is leaking memory everywhere from all the com objects that aren't releasing properly. After a bit of snooping around and hours of staring at the code, i've developed some methods to isolate where i'm getting these unexpected increases to the ref counts.
first off, all of the objects are wrapped in std::shared_ptrs with custom deleters that call their respective release function. I did this so i would never have to call addref, and the first call to release, the one in the deleter, would only be called when the object went out of scope. It would look something like this:
// in D3D11Renderer.h
...
// declaration
std::shared_ptr<ID3D11Device *> m_Device;
...
// after call to ID3D11CreateDeviceAndSwapChain
m_Device.reset(device, [](ID3D11Device * ptr){ptr->Release();})
Problem is certain random functions in the api calls will just randomly increase the ref count, expecting me to have to deal with it later.
something i found useful in diagnosis was a function that looked like this:
template <typename T>
int getRefCount(T object)
{
object->AddRef();
return object->Release();
}
which, just increments and decrements that count to obtain the current count of refs on that object. Using this, i found that, just before the release in the custom deleter is called, there are 10 outstanding references to the 1 ID3D11Device i created. Curious, i backtracked slowly, calling this function all the way back through the program, right up to where i originally created it. Funny thing, just after i first create the object, (even before the shared_ptr takes ownership), the number of outstanding refs is already 3! This occurs immediately after this here.
result = D3D11CreateDeviceAndSwapChain(NULL, D3D_DRIVER_TYPE_HARDWARE, NULL, 0, &featureLevel, 1,
D3D11_SDK_VERSION, &swapChainDesc, &swapChain, &device, NULL, &deviceContext);
if(FAILED(result))
{
return false;
}
this is the first time i call any such function that creates the device, and when i check to see how many refs there are right after, and it says 3! So clearly, I'm misunderstanding something about the way these com objects are supposed to be handled. Is there any such way that i can just manually delete them, rather then use there behind-the-scenes ref counting nonsense?
Every time you create a buffer or a shader or anything that depends on the device, that object will likely contain a reference to the device so will bump up it's reference count to ensure it's not deleted while it is still using it.
It sounds like your approach might well work overall, as you'll essentially keep one single reference to the device in your code to stop it being deleted, and when all your internal references are gone release it. However d3d will still be doing it's own reference counting and so the reference count will only drop to zero when you release every reference to every other related object. Even just creating the swap chain and device will make back buffers and so on that likely need to maintain a reference to the device.
I tried this same idea for a while... And in the end found it much easier to just
#include <atlbase>
Then use
CComPtr<ID311Device> m_Device
as that's pretty much exactly what that class is designed for and it's more lightweight than std::shader_ptr as the objects already have a reference counter in them so there is no need to keep a separate one.
Using shared_ptr is not correct for Direct3D (COM) objects, even if you use custom deleters.
First, COM objects use intrusive reference counting, which means the reference count is stored in the object itself. shared_ptr on the other hand uses non-intrusive reference counting, which means the reference count is stored in the smart poiter object. Therefore, using shared_ptr for COM objects means that you have two separate, independent reference counts: the COM object's, and the shared_ptr's.
Second, using a custom deleter solves the problem of properly releasing the object, but it doesn't solve the problem of properly acquiring a reference to the object. Assigning a COM object to a shared_ptr will increment the reference count of the shared_ptr, but not the object's.
That explains why you're leaking objects: D3D methods increment objects' reference counts, but you are using shared_ptrs that decrement the object's reference count only once for the entire lifetime of the object (when all shared_ptrs pointing to the object are destroyed).
So, you need to use a COM smart pointer, such as ATL's CComPtr.
I came across a leak in a Direct3D application of mine, and I ended up correcting it, but I think the cause of the leak was due to my misunderstanding of how Direct3D handles its memory and interfaces.
I haven't been able to find a definitive article/tutorial on it (please provide one if you have one), but from what I've gathered, it works as such:
Every time you call a Get method, the number of references for the object returned is incremented. So if I call GetRenderTarget, the surface being rendered to has its reference count incremented.
Calling Release on the interface decrements its reference count. These first two points combined essentially mean: every time you get an interface, release it after you're done with it.
When the reference count reaches 0, the instance is deleted.
I'm not entirely sure if this is correct, but it seems to work in practice. If someone could clarify/confirm how it works, that'd be great.
P.S, are there any safeguards implemented in releasing interfaces? Calling Release any number of times on the back buffer doesn't seem to do any damage (which is a good thing, but I'm not sure why it doesn't).
Direct3D is based on COM, which is a technology that's at least 15 years old. Seems many people claim COM is dead and for that reason many overlook it, but reality is that there are many things in windows including Direct3D and MS's new Media Foundation that are all based on COM.
I strongly suggest you take a look at general COM programming. There are plenty of books and resources, but many of them are rather old but that's ok because the root of the technology hasn't changed for a very long time.
Basically what you've observed is interface reference counting. COM is based purely on accessing objects via interfaces, which all derive from the base interface, IUnknown. IUnknown implements methods AddRef() and Release() and it is the responsibility of your application to call AddRef() whenever you store a local copy of a pointer and to call Release() whenever that local copy is no longer needed.
When you have methods with interface out parameters (i.e. IFoo** ppObj ), that means the callee is giving you back an interface and now that you have it, it is still your responsibility to call Release() whenever you are done with it.
Once you get the hang of it, I'd suggest you start using CComPtr smart class for storing local and member variables (still pass raw interface values between function calls, no need for smart pointer parameter types). It will take care of all your reference counting. Also don't make it a practice of calling release "any number" of times. It might work today because the object is implemented as a singleton, or maybe something else is holding on to it, but that might change with next patch or next release. Always follow the rules. If you have an interface, when you don't need it call Release() exactly once. If you made a copy of interface pointer, make sure to call AddRef() exactly once.
The application of addref/release semantics is much wider than COM technology. There is simple rule one CreateObject() (or CreateTexture, or GetRenderTarget, or GetBackBuffer, etc...) have to be confronted with one Release(), one AddRef() have to be confronted with one Release().
In COM IUnknown::Release() returns number of references to object. It may delude you and you can think:
"Hm... I just call Release() until it return 0 and I will have no leaks. ???? PROFIT!!!!!111" <-- That is wrong! AddRef might be called by Direct3D itself or by 3rd_party library you pass this object to, or something else outside your app. One Release for one AddRef. You should call Release when you don't need object anymore, don't waste system resources.
You said:
Calling Release any number of times on the back buffer doesn't seem to do any damage
That means nothing. May be The Universe like you so much or you just too lucky to not get exceptions from D3D.
Smart pointers (such as CComPtr) could make your life much easier if you will use them. In this case you don't need to call Release explicitly, it is called in CComPtr dtor if it is assigned to some object.
void get_surface(IDirect3DDevice9 *pDevice)
{
IDirect3DSurface9 *surf0;
IDirect3DSurface9 *surf1;
CComPtr<IDirect3DSurface9> surf2;
CComPtr<IDirect3DSurface9> surf3;
CComPtr<IDirect3DSurface9> surf4;
pDevice->GetRenderTarget( 0, surf0 ); // surface reference counter incremented, you should call Release() for this
surf1 = surf0; // surface reference count is not incremented, you shouldn't call Release() for this
pDevice->GetRenderTarget( 0, surf2 ); // surface reference counter incremented
CComPtr<IDirect3DSurface9> surf3 = surf0; // surface reference counter incremented
surf0->Release(); // release for pDevice->GetRenderTarget( 0, surf0 );
surf2.Release(); // .Release() used not ->Release() - it is important
surf4.Release(); // nothing happens because surf4 == 0
} // surf3.Release() is called in surf3 destructor
Also you may #define D3D_DEBUG_INFObefore including direct 3d headers and switch to debug d3d runtime. It is helpful in finding leaks in d3d app.
May the CComPtr Force be with you.
D3D objects are COM objects, and they use a basic reference counting system to manage the lifetime of the object. (See wikipedia for more info about the Component Object Model, or the MSDN article Managing Object Lifetimes)
The reference count is modified purely through the AddRef/Release methods, and certain other functions call those methods.
Creating the object as well as calling certain Get methods that return an object derived from the IUnknown class will call AddRef internally to increment the reference count, so you will need to call Release for each call when you are finished with the object.
If you pass the object to another function or class that stores a copy of the point (even temporarily) that class/function should call AddRef to ensure that the object is not freed while it is using it (and Release to signal it is done).
When the reference counter reaches 0 from a call to Release the object is signalled that it may be a good time to delete the held resources, but it may not happen immediately. There is also no protection for calling Release multiple times. The reference counter will not become negative, but it will not perform any other sanity checking (because it can't really) so you can cause application instability by trying to release references you don't hold.
Yes, you are correct. This is called reference counting and it ensures that objects are alive as long as they are being used, and no longer. You can use a variety of smart pointers to enforce this rule- both shared_ptr and (C++11) unique_ptr allow for custom deleters to call Release(). This makes it easy to control the lifetime of Direct3D objects just like you would for any other object in your application. You don't need to start including ATL libraries and CComPtr to use smart pointers with COM interfaces.