CreateRemoteThread failing with ERROR_NOT_ENOUGH_MEMORY - c++

This is quite strange question, but, I believe, this is on-topic for SO.
Intro:
I have an service, written in C#, which calls my C++ library. C++ library execute some 3rdparty software via WinExec.
3rdparty software injects DLL via CreateRemoteThread. I don't have source files for this software.
Main part
I have 2 PCs - Win2008 and Win10.
For Win10 - this frankenstein is working flawlessly, Service runs DLL, DLL runs 3rdparty DLL injector, DLL injector injects stuff.
For Win2008 things are different. If I run 3rdparty DLL injector from CMD - it works flawlessly. But if I run service - Injector returns, that he got ERROR_NOT_ENOUGH_MEMORY from CreateRemoteThread.
Service is working from LocalService account, and everything is OK on Windows 10. I am looking for possible ideas\clues, why there is a problem with SERVICE (remember, CMD works fine) and ONLY for Windows 2008.

This issue might be related to creating a remote thread across privilege levels, as explained in the following blog article:
Injecting Code Into Privileged Win32 Processes
With XP SP2 and later (2003, Vista) some new security measures prevent the traditional CreateRemoteThread() function from working properly. You should be able to open the process, allocate memory on its heap, and write data to the allocated region, but when trying to invoke the remote thread, it will fail with ERROR_NOT_ENOUGH_MEMORY.
...
For XP SP2 I did a little debugging and found that inside CreateRemoteThread(), there is a call to ZwCreateThread() which is an export from ntdll.dll. The call is made while specifying that the thread should start suspended, which it does properly, however down the road still inside CreateRemoteThread() before ZwResumeThread() is called, there is a call to CsrClientCallServer() which fails and eventually leads to the error message.
The article explains some different ways of injecting remote threads on different version of Windows to avoid the error, ending with this conclusion:
At this point, we can successfully execute remote threads into privileged processes across all target platforms, but as mentioned before, its pretty messy. We're using three different, largely undocumented functions and auto-detecting which one to use based on the OS version.
The better solution is to create a secondary program that adds a service object (your injector program) to the service control manager database on the target system. Since you're administrator, which is required anyway, you'll be able to add these entries and start the service. This will enable the injector program to run with different access rights than normal code, and the traditional CreateRemoteThread() will work properly on Windows 2000, all of XP, and 2003/Vista. The API functions for adding and controlling the service are documented by MSDN and remain consistent across all of the platforms.
So, what is learned is that we can use a number of different functions to inject code into privileged remote processes, including RtlCreateUserThread() on XP SP2, and NtCreateThreadEx() on Vista, but the optimal way is to install a temporary service and allow CreateRemoteThread() to be the single API that accomplishes the task for all platforms.
Of course, none of this really matters since you don't have the source code for the injector and thus cannot change how it works.
Also, you can't create remote threads across session boundaries, either. Calling WinExec() in a service will run the injector process in the same session as the service, ie session 0. If it is trying to inject into a process that is running in a user session, that will never work. This would also explain why running the injector from CMD works, if CMD is running in the same session as the process that is being injected into.

I encountered the same issue today and this seems to be the issue-
Prior to Windows 8, Terminal Services isolates each terminal session by design. Therefore, CreateRemoteThread fails if the target process is in a different session than the calling process.
This explains why your code works on Windows 10 but not on Windows 7/2008.
Source: https://msdn.microsoft.com/en-us/library/windows/desktop/dd405484(v=vs.85).aspx

Related

Unexpected IConnectionPointImpl::Unadvise call on Windows Embedded Compact 7

We have a bigger software running on Win CE6 without problems. The core functionality is implemented in a COM server DLL that provides connection points. The COM client program registers event handlers for the connection points on program startup to get status notifications etc. On program exit it unregisters the handlers by calling the corresponding IConnectionPointImpl::Unadvise methods.
Now, we are porting the program to run on Win EC 7. The new Board Support Package (BSP) for Win EC 7 works well. There are also different versions with different options, created at different times with different sources from Microsoft, but our software always show the same issue.
On program startup, ~10s after launch, IConnectionPointImpl::Unadvise is called unexpectedly on all registered event handlers. We only have one method in our source code that calls IConnectionPointImpl::Unadvise and this is definitely not executed.
The issue appears ~95%, but sometimes the program starts and runs without problems. We cannot use the Debugger because of the size of the program, the performance is very poor.
We guess, that the COM runtime calls the IConnectionPointImpl::Unadvise methods for some reasons. But we have no idea, how to prevent this.
Has anybody observed the same issue? Is there a solution/workaround available? Thanks.
So we finally found how solve this problem.
We remove our dependency on MarshalByReObject and replace it by a proper implementation of ISerializable.
That allow us to load properly inside custom AppDomain our assembly and events are not loose anymore.
But this has a side effect on path where assembly a configuration file are loaded. To solve this we also implement an AppDomain.AssemblyResolve event which allow us to redirect the loading in a proper place.
I hope this can help you ;)

COM Surrogate Server Timeout

I have a Win32/MFC application that depends on two separate STA COM DLL servers that I created many years ago using C++/ATL. These are large DLL servers with multiple interfaces and are also successfully used in other contexts and client programs. Several years ago, I had to create 64-bit versions of these 32-bit servers, and my 32-bit MFC app needed to be able to use either the 32-bit or 64-bit version of the DLL COM server (chosen with a checkbox).
Because a 32-bit process can't load a 64-bit COM server DLL in-process, I worked around this by having the MFC app create the 64-bit servers in the system surrogate (DLLHOST.EXE) by replacing
CoCreateInstance(..., CLSCTX_INPROC_SERVER, ...)
with
CoCreateInstance(..., CLSCTX_LOCAL_SERVER | CLSCTX_ACTIVATE_64_BIT_SERVER, ...)
Some updates were required, like adding an interface to copy environment variables into the server process and set the server/surrogate's working directory (the surrogate starts in SYSTEM32), but the other interfaces were all remoteable. This all seems to work perfectly and I can now use the 32-bit and 64-bit servers interchangeably from the 32-bit app by flipping a switch.
There is, however, one problem that I haven't been able to solve: making the surrogate quickly terminate when the client releases the last interface. The surrogate hangs around for 3-5 seconds after all remote interfaces are released by the MFC client -- presumably an optimization, hoping the client will come back. If the MFC app re-launches the server with CoCreateInstance() during that 3-5 seconds, it reconnects to the same "dirty" surrogate. The server code is not serially re-usable (it packages up many thousands of lines of legacy ANSI "C" code with lots of static variables) so reconnecting to the same instance is just not possible.
I worked around this several years ago by having the startup interface return a COM error code indicating the server is waiting to be recycled (better than a crash). However, the servers are launched when the end user presses a toolbar button in the MFC app, so this means the user gets a message like "wait a few seconds and try again". That works, but the bad part is that every fresh launch attempt resets the 3-5 second counter that keeps the surrogate from exiting. And impatient users are complaining. I'll add this all works perfectly in-process, with CoFreeUnusedLibraries() working as expected.
I tried a number of things already -- everything short of coding an ExitProcess() in the server, which seems inappropriate. There seems to be no way to tell the surrogate that the application is complete and should not wait for more connections. The MS documentation claims omitting the RunAs attribute in the AppID might help (I had it set to "Interactive User") but it didn't. It also mentions REGCLS_SINGLUSE but then says "Do not set REGCLS_SINGLUSE or REGCLS_MULTIPLEUSE when you register a surrogate for DLL servers" and "REGCLS_SINGLUSE and REGCLS_MULTIPLEUSE should not be used for DLL servers loaded into surrogates." and I don't have control over what the surrogate's class factory as far as I know.
It looks like COM+ might provide some control over recycling, as it seems to have a RecycleActivationLimit option that I might be able to set to 0, but I have no idea what it would take to convert this into a COM+ server.
The other possibility is to write a custom surrogate.
If there's no easy answer, I might just resort to greying out the button until the server vanishes -- but since I can't probe the server without extending its lifetime, I guess I could add a shared mutex and wait for it to vanish. Ugh.
Is RecycleActivationLimit somehow available to regular COM applications? Any other suggestions are most welcome.

LoadLibrary fails sometimes when starting as service

We have a 64bit application running as service on Windows 7 64bit. When we start a service manually, it works fine everytime. When the service is started automatically with system startup, sometimes (like 1 in 10) it fails. Problem is with LoadLibrary function which is in fact the first thing it does. It is a load of our DLL where the most of the code is. I can't reproduce it on my computers, so I am just sending special testing binaries to colleague abroad. I added text outputs to file using unbuffered write operation so I can see exactly where it disappears. There is no crash dump, there is no exception, there is no NULL retrived, it just exits somewhere in LoadLibrary and service is not running after system startup. Our library does not have DllMain, but if I add it, it does not get there (in fail case). I tried delay loading of dependent system libraries, no success. It seems like sometimes something is not yet started in Windows and so the initialization fails, I am not sure. But adding sleep before LoadLibrary does not solve the problem (too late to sleep?). Do you have any ideas what is going on and how to proceed? I am currently going to remove the code from our DLL step by step to remove the lib dependencies one by one to see if in some moment it will start to work. I will be happy to provide additional information when needed.
Lenovo RapidBoot Shield app was the reason.
"RapidBoot Shield works by delaying non-critical applications and services to help your system boot faster. However, in some instances, RapidBoot Shield may delay an application and/or service that is critical to the system startup. This can cause the system to boot slower than expected or some applications may be unable to start normally."

Whats going to happen with COleTemplateServer::RegisterAll() in windows 7

Ok, so we have a C++ app that runs fine in Windows XP.
It has the following code in the initialization
// Register all OLE server (factories) as running. This enables the
// OLE libraries to create objects from other applications.
COleObjectFactory::RegisterAll();
Now like i said, it works fine in Windows XP, but as far I understand the program tries to register its COM interface Which is fine in XP, but this may be a problem in Windows Vista and Windows 7 because of the UAC. Especially if its is run as a standard user (with no elevated privileges).
If i understand it correctly this is needed for the program to run properly, but it cant execute this code without elevated privileges. If it will run every time the app runs (this usually runs when CWinApp::init() is run)
Before you say just use admin privileges, the user will not have them, there is no way to change that
So, now my questions are:
1) am i correct in my assumptions?
2) if I am correct, what is the best way around this? Can i just remove this? Do i need to set up some other thing? (we changed some VB modules to use a XML file instead of stuff in the registry
PS: the modules compiles into DLLs
PPS: UAC MUST be on
Take note that:
The documentation for these functions makes no mention of any privilege requirements; and
nobody online seems to be having trouble with these functions in limited privilege environments; and
it's 2012, I think someone would have noticed if these functions didn't work under UAC by now.
So (with nothing to suggest otherwise) I'd say It Just Works.
Notwithstanding the above, I looked at the implementation of COleObjectFactory::RegisterAll() and COleObjectFactory::UpdateRegistryAll().
RegisterAll
Ultimately calling RegisterAll ends up in olefact.cpp:135 where CoRegisterClassObject is called. From MSDN:
Registers an EXE class object with OLE so other applications can connect to it.
I believe this registration will be limited to the current user's session and the lifetime of the application. The Remarks section touches on privileges (As of Windows Server 2003...) but doesn't provide anything concrete.
There's an object known as the Running Object Table (ROT) that can be retrieved via GetRunningObjectTable. The documentation has this snippet:
Each workstation has a local ROT that maintains a table of the objects that have been registered as running on that computer.
The COM Elevation Moniker has some more information about the ROT and privileges (it suggests processes of various privilege levels work fine with it). The links on the left-hand side might help, too.
Overall it seems there's nothing to suggest that CoRegisterClassObject requires administrator permissions.
UpdateRegistryAll
This function ends up in olefact.cpp:375 where it opens HKEY_CLASSES_ROOT. At this point the documentation gets a bit better:
Registry functions such as RegOpenKeyEx or RegQueryValueEx allow you to specify the HKEY_CLASSES_ROOT key. When you call these functions from a process running in the interactive user account, the system merges the default settings in HKEY_LOCAL_MACHINE\Software\Classes with the interactive user's settings at HKEY_CURRENT_USER\Software\Classes.
Further on:
If you write keys to a key under HKEY_CLASSES_ROOT, the system stores the information under HKEY_LOCAL_MACHINE\Software\Classes
The documentation doesn't define what happens when you try to write to HKEY_CLASSES_ROOT under limited privileges (i.e. a standard user can't write to HKLM), but I believe that you'll end up writing to HKCU instead.
And finally, note:
Windows Server 2003 and Windows XP/2000: Applications can register dependent COM objects to either the per-machine or per-user COM configuration store (HKEY_LOCAL_MACHINE\Software\Classes or HKEY_CURRENT_USER\Software\Classes).
So if it falls through to HKCU, you should be fine.
Caveat Implementor: Don't rely on implementation details.

Windows Shell Extension Not Calling Initialize

I was hoping somebody here had some insight into what could be causing this problem. I've implemented several COM extensions for Explorer that provide ShellIconOverlays and a ContextMenu, and the overlays work perfectly. The Context Menu works fine when I click on the desktop but when I right click in any explorer instance, I can see the interface being queried in the debugger and an instance of IShellExtInit being generated but the initialize function doesn't get called in the explorer instances, but it is called fine from the desktop and a ContextMenu item is queried immediately after.
Has anybody here seen anything like this before?
If you're debugging a shell extension, chances are that you've had occasions to terminate the running explorer.exe process and start a new one. When you started a new one, was it running with the same integrity level as the original?
Do your Explorer settings say to browse files in a new process? If so, is that process running with the same integrity level as the original?
Also, since you're running a debugger, chances are that you built a debug build. Does explorer.exe sometimes try to load the debug build of your DLL and sometimes try to load the release build of your DLL?
OK, I run into the exact same problem here, and it turns out that the issue has to do with
ThreadingModel = Apartment
Basically, what I think you are experiencing, is that the second thread of explorer.exe (desktop runs in STA thread) uses the default (legacy) ThreadingModel - and expects your COM to implement IMarshal to do IPC. Apartment ThreadingModel allows multiple instances of your IShellExt class to co-exist.
Caveat - If you are using ActiveQt to develop Context Menu Shell Extensions, there are few more tricks to use.