I'm trying to find a way to resolve a memory address and a DLL file name for a particular API when it's mapped into a process. Most of this can be resolved with the import/export tables in a DLL and by analyzing the Import Address Table of a mapped module. That is for most functions.
But the issue happens with some forwarded functions. An example of such function happens to be DeleteProcThreadAttributeList on my Windows 10 system. So for instance, if I build a test 32-bit process with such function, or better yet, let's use a 32-bit version of cmd.exe from c:\windows\syswow64\cmd.exe image, and then try to analyze its import table. It turns out that this function is imported from the API Set with a virtual name API-MS-WIN-CORE-PROCESSTHREADS-L1-1-2.DLL:
To find an actual file that it redirects to I do the following:
HMODULE hMM = ::LoadLibraryEx(L"API-MS-WIN-CORE-PROCESSTHREADS-L1-1-2.DLL",
NULL, DONT_RESOLVE_DLL_REFERENCES);
WCHAR buffModPath[MAX_PATH];
::GetModuleFileNameEx(::GetCurrentProcess(), hMM, buffModPath, _countof(buffModPath));
::FreeLibrary(hMM);
Which gives me buffModPath as C:\Windows\System32\KERNEL32.DLL.
Since I'm calling it from a 32-bit process, I now inspect the export table of c:\windows\syswow64\KERNEL32.DLL module:
that shows that DeleteProcThreadAttributeList is forwarded to api-ms-win-core-processthreads-l1-1-0.DeleteProcThreadAttributeList.
OK, I then use my method again to resolve redirection of the virtual api-ms-win-core-processthreads-l1-1-0.dll API set:
HMODULE hMM = ::LoadLibraryEx(L"api-ms-win-core-processthreads-l1-1-0.dll",
NULL, DONT_RESOLVE_DLL_REFERENCES);
WCHAR buffModPath[MAX_PATH];
::GetModuleFileNameEx(::GetCurrentProcess(), hMM, buffModPath, _countof(buffModPath));
::FreeLibrary(hMM);
Which gives me C:\Windows\System32\KERNEL32.DLL, that brings me back to where I started.
So how do I resolve this circular dependency to actual DLL address/entry point the way OS module loader does it?
I think whatever happens if you GetProcAddress one of these exports is supposed to be a black box.
The PEB has a ApiSetMap member in Windows 7+ that contains set information that the loader uses. The format of this data has changed at least 3 times but the mapping is not just from "API-*.dll" to "*32.dll".
You can read the Microsoft patent of this concept here.
Related
Short:
In my c++ project i need to read/write extended file properties. I managed it with using alternate data streams (ADS). My problem is, for opening the ADS i need to use the CreateFile API. But it is not fulfilling my needs. NtCreateFile will fullfill all my needs. (Or alternatively NtSetEaFile and NtQueryEaFile) But NtCreateFile is not directly accessible from a win32 console application.
I know i can use this function easily via GetProcAdress. But i like to know the opinion of you all, if i did miss something? Some other libs are using this pattern already, for example Chromium (https://github.com/chromium-googlesource-mirror/chromium/blob/1c1996b75d3611f56d14e2b30e7ae4eabc101486/src/sandbox/src/win_utils.cc function: ResolveNTFunctionPtr)
But im uncertain, because the c++ project is not a hobby project and i ask myself if it is dangerous or not.
I guess NtCreateFile is maybe the securest way to do, because it is well documented and supported by winternl.h header. Especially because this method is unchanged since windows 2000. But what is with NtSetEaFile, NtQueryEaFile which are fitting my needs perfectly. They are only half documented. A documentation for ZwSetEaFile and ZwQueryEaFile exist (unchanged since windows 2000).
Reason why i want to do that:
I want to write and read extended properties from files via ADS. But in case of writing the extended property of a given file the first time, i need to open the file with OPEN_ALWAYS. In case of file is not existing it will create a new file, even if i only access not the content stream of the file. To avoid this i get first the handle of the original file and check with this HANDLE if the file still exist.
But i dont want to blog any file with reduced access rights, because from my point of view that is a very bad pattern. The user needs to have full access to any file any time. Because of that we open all HANDLES with the flag FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE. And now i have the race.
auto hFile = CreateFileW(originalPath, …, FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE, …).
// this is the little race: if somebody at least rename originalPath the
// second CreateFileW call will cause the creation of a empty file with the
// path originalPath (the old path).
auto hADS = CreateFileW(originalPath + adsName, …, FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE, OPEN_ALWAYS, …).
This is a main issue, especially because this happens from time to time in our tests. NtCreateFile will fix it, because i can create the second HANDLE with the help of the first HANDLE. Because of that no race. Or NtSetEaFile and NtQueryEaFile will help, because i only need one HANDLE.
The thing is, that the application needs not to be save for the future, because ADS works only on NTFS anyway. And who knows when NTFS will be exchanged. But i dont want a flaky behaviour. I want to trust this Methods. I I am fine if the API will change in the future and the software needs to adapt to it. But i want to be sure, that all Windows higher or equal then 7 can deal with it. Somebody some experience to share? I would like to hear them very much.
This question is wrong. Your proposed solution for your problem, is not using NtCreateFile, but use CreateFile with dwCreationDisposition set to the OPEN_EXISTING.
From documentation:
OPEN_EXISTING
Opens a file or device, only if it exists. If the specified file or
device does not exist, the function fails and the last-error code is
set to ERROR_FILE_NOT_FOUND.
Simply open file if exists and set whatever you want. If file is renamed, CreateFile returns ERROR_FILE_NOT_FOUND.
THE PROBLEM
Now, to your proposed solution, what is better method or why is not possible use ntdll.dll in win32 console application (???).
Again, your "better" method - GetProcAddress is "wrong" same as using linking against ntdll.dll. In Windows 11, or Windows 12 or Windows 3030 the function may be removed and both solutions (statical vs. dynamical import) will be fail.
It is not really unsecure to use this kind of APIs if their is a documentation. In case of NtSetEaFile, NtQueryEaFile and NtCreateFile you can find a description inside of Microsoft's Doc. (keep in mind NtXxx == ZwXxx)
But this API can change in the future and Microsoft does not guarantee that it will provides the same methods in the next Windows version. If you can, use the public API, because then you are safe. If not it is a case by case decision. In this case the three methods from the API are unchanged since Windows2000. Plus for example NtSetEaFile and NtQueryEaFile is used by Microsoft for WSL (Windows Subsystem for Linux). And especially NtCreateFile is used by a wide range of OpenSource Projects. So it is very unlikely that this API will change.
In my use case another aspect is important. Because I wanted to use ADS, but ADS is only supported by NTFS. So using ADS does not ensure future compatibility as well. So it was very clear for me using NtSetEaFile and NtQueryEaFile.
But how you can use this kind of APIs? Dynamic or static linking is possible. It depends on your needs what is better. In case of static linking you need to download the last WDK (Windows Driver Kit) and link against the ntdll.lib. In case of dynamic linking you can access the dll directly via GetModuleHandle and finding out the address of the method with GetProcAddress. Under Windows ntdll.dll is accessible from any application. In both cases you don't have directly a header file. You have to define the header file by yourself or use WDK to get them.
In my project dynamic linking was the best choice. The reason was, that on every windows the right implementation will be choosen and in case the method is not available i have the chance to deactivate the feature in my software instead of crash. Microsoft is recommending the dynamic way, because of the last reason.
Simple PseudoCode (dynamic case):
typedef struct _FILE_FULL_EA_INFORMATION {
ULONG NextEntryOffset;
UCHAR Flags;
UCHAR EaNameLength;
USHORT EaValueLength;
CHAR EaName[1];
} FILE_FULL_EA_INFORMATION, *PFILE_FULL_EA_INFORMATION;
typedef struct _IO_STATUS_BLOCK {
union {
NTSTATUS Status;
PVOID Pointer;
};
ULONG_PTR Information;
} IO_STATUS_BLOCK, *PIO_STATUS_BLOCK;
typedef NTSTATUS(WINAPI *NtSetEaFileFunction)(IN HANDLE FileHandle,
OUT PIO_STATUS_BLOCK
IoStatusBlock,
IN PVOID Buffer,
IN ULONG Length);
HMODULE ntdll = GetModuleHandle(L"ntdll.dll");
NtSetEaFileFunction function = nullptr;
FARPROC *function_ptr = reinterpret_cast<FARPROC *>(&function);
*function_ptr = GetProcAddress(ntdll, "NtQueryEaFile");
// function could be used normally.
The other answer is incorrect. The reason is that the reason of my problem is, that I need to use OPEN_ALWAYS. Of course, if you don't need this flag, everything is fine. But in my case there is a point where I needed to create the ADS. And it will not be created without the OPEN_ALWAYS flag.
I have written a C++ code for application, in which there are some variables that must have different values for every user will use it ( lets call it the variable X for simplicity)
X have different values for different user. This (X)should be not changed and also embedded in the exe itself ( so I can't read it from a file or any other similar solution)
I don't want to distribute the source code then compile. Instead, I want a method that makes me edit the final exe directly without need to compile ( it is just value of variable X which differs !) Is this possible ?
My idea to do this is if I can force this (X) at a constant memory location, I can then edit its value easily from Hex-editor as example. ( I mean the same ideas when hackers writes cheat tool for a certain game )
Is the mechanism of fixed memory position possible?
Is there any other idea to make what I want?
I hope my question is clear enough
In this answer I'll use Visual Studio 2017 Community Edition because I wanted to be sure to a have a development environment fully compatible with Windows.
I'll present five methods, from the most maintainable to the less. Of course the focus of this answer in strictly limited to the goal of "sharing" a C++ variable with an external tool.
Security of such an operation is a different topic and ultimately a futile attempt anyway.
Method 1 - Resources
Windows APIs1 and the PE2 support embedding resources in an executable3.
Resources are typically images, icons or localized strings but they can be anything - including raw binary data.
With Visual Studio is quite easy to add a resource: In the Solution Explorer > Resource files > Add > New item > Resource > Resource file (.rc)
This will open the Resource view, right-click on Resource.rc and select Add resource....
It's possible to create the standard resources but we need a Custom... type that we can call RAW.
This will create a new binary resource, gives it an ID and makes a few files in the solution.
Switching back to the Solution explorer we can see these new files and eventually edit the .bin file with a better hex editor than the VS's integrated one.
Of particular interest is the resource.h file that we can include to have the definition for the resource id, in my case it was IDR_RAW1.
After the bin file has been crafted we are ready to read it in the application, the pattern to use is the usual one - I don't feel like going over these API one more time a new answer so I'll link the Official documentation and provides a sample code:
#include <Windows.h>
#include "resource.h"
int WINAPI WinMain(HMODULE hModule, HMODULE hPrevModule, LPSTR lpCmdLine, int showCmd)
{
//Get an handle to our resource
HRSRC hRes = FindResource(hModule, MAKEINTRESOURCE(IDR_RAW1), "RAW");
//Load the resource (Compatibility reasons make this use two APIs)
HGLOBAL hResData = LoadResource(hModule, hRes);
LPVOID ptrData = LockResource(hResData);
/*
ptrData is out binary content. Here is assumed it was a ASCIIZ string
*/
MessageBox(NULL, (LPCSTR)ptrData, "Title", MB_ICONINFORMATION);
return 0;
}
Resources are good because they allow for an easy integration with other automatic build tools: it's easy to add a build step before the resources are compiled to generate them on the fly.
It is also very easy to alter them after the exe file as been generated - CFF Explorer III is a simple and effective tools to edit a PE module's resources.
It's even possible to replace a resource entirely thereby not limiting ourselves to keeping the new resource the same size as the old one.
Just open the module in CFF, select Resource editor, browse to the raw resource and edit/replace it. Then save.
Method 2 - PE exports
Executable are ordinary PE module just like Dlls, the difference is really a batter of a bit.
Just like Dlls can exports functions and variables4 so can exes.
With VC++ the way to tag a symbol as exported is __declspec(dllexport):
#include <Windows.h>
__declspec(dllexport) char var[30] = "Hello, cruel world!";
int WINAPI WinMain(HMODULE hModule, HMODULE hPrevModule, LPSTR lpCmdLine, int showCmd)
{
MessageBox(NULL, var, "Title 2", MB_ICONINFORMATION);
return 0;
}
The C++ side of the matter is little affected.
The editing of the PE module is less user friendly but still very easy for everyone to follow.
With CFF open the export directory, all the exports will be listed.
C++ compilers have to mangle variables names when they can be shared due to the C++ features they support - so you won't find a nice name like var in the exports but something like ?var##3PADA.
The export name doesn't really fulfil any goal in this context but you must be able to identify the correct export.
This should be easy since it's very likely to be only one.
CFF will show you the function RVA, this is the RVA (relative to the image base) of the variable, you can easily convert it into a file offset or simply use the Address converted integrated in CFF.
This will open an hex editor and points you at the right bytes.
Method 3 - Map files
If you don't want to have a PE exports pointing right at your variable you can tell VS to generate a MAP file.
Map files will list all the symbols exported by an object file (note: an object file, not a PE module).
So you must make sure a variable, in this case, is exported by your translation unit - this is the default case for "global" variables but make sure to remember to not attach the static linkage modified to it and eventually make it volatile to prevent the compiler from eliminating it during the constants folding step.
#include "Windows.h"
//extern is redundant, I use it only for documenting the intention
//volatile is a hack to prevent constant folding in this simple case
extern volatile int var2 = 3;
int WINAPI WinMain(HMODULE hModule, HMODULE hPrevModule, LPSTR lpCmdLine, int showCmd)
{
//A simple use of an int
return var2;
}
A MAP file will be generated in the output dir, along with the exe, inside it's present a row like this one:
0003:00000018 ?var2##3HC 00403018 Source.obj
This gives you the VA of the variable (403018) that you can use in CFF Address translator.
Method 4 - PE scan
You can initialise the variable with an unique value.
To be able to do so the variable size must be big enough that the probability that a random sequence of bits of equal size end up with the same value is negligible.
For example, if the var is a QWORD the probability of finding, in the PE module, another QWORD with the same value is very low (one in 264) but if the var is a byte then the probability is just one in 256.
Eventually, add a marker variable (I'd use a random array of 16 bytes) before the variable to mark it (i.e. act as the unique value).
To modify the PE use an hex editor to look for that unique value, this will give you the offset of the var to edit.
Method 5 - Reverse engineering
After each release, reverse engine the application (this is easy as you can debug it with VS along with the sources) and look where the compiler allocated the variable.
Take note of the RVA (nota bene: RVA not VA, the VA is variable) and then use CFF to edit the exe.
This requires a reverse engineering analysis each time a new release is built.
1 To be correct, "Win32" APIs.
2 I strongly advice the reader to be at least accustomized with the PE file format as I must assume so to keep this answer in topic and short. Having no understanding of the PE file format will likely result in no understanding of the question as a whole.
3 Actually, in any PE module.
4 Symbols in general.
I have an application that relies heavily on plugins.
On startup it scans a directory for DLLs and loads them one by one, looking for ones that implement a certain exported function. However - if someone were to rename a different type of file to *.dll and put it in the directory, that file would then also be loaded by LoadLibrary(). LoadLibrary() doesn't like that and produces an error [dialog].
Is there a way to simply ignore invalid / incompatible .dll files (either detecting them prior to the call or have LoadLibrary() return NULL rather than throwing a fit)?
You need to set the error mode for your process. Do this once and for all at startup:
UINT oldMode = SetErrorMode(0);
SetErrorMode(oldMode | SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX);
After you've set the process error mode, when LoadLibrary fails no dialog box will be displayed and LoadLibrary will return NULL.
The documentation says:
Best practice is that all applications call the process-wide SetErrorMode function with a parameter of SEM_FAILCRITICALERRORS at startup. This is to prevent error mode dialogs from hanging the application.
I also recommend adding SEM_NOOPENFILEERRORBOX for reasons that I guess should be obvious.
http://msdn.microsoft.com/en-us/library/windows/desktop/ms684175%28v=vs.85%29.aspx
Remarks
To enable or disable error messages displayed by the loader during DLL loads, use the SetErrorMode function.
Don't forget to restore the error mode after you are finished.
If you wanted to do this yourself, in a primitive manner you could parse the PE header to identify obvious cases of bad DLL files, however it is not really possible to determine if a DLL is valid and loadable without a full PE loader which requires a lot of work and is already built into the operating system.
Updated answer after feedback in comments from djgandy and Remy Lebeau. Now a complete
function, better preservation of error mode at entry:
// This function will load the DLL named by pszPath if it is a valid library.
// If the function succeeds, it will return a valid HMODULE for the DLL. This
// handle should be passed to FreeLibrary when it is no longer needed.
// If the function fails, it will return NULL and no annoying dialog boxes will
// be displayed. It is therefore up to the caller to notify the user about what
// happened or take any other appropriate action. The reason for failure can
// be obtained from GetLastError(). Common problems:
// ERROR_BAD_EXE_FORMAT - Bad DLL (tested function with text file)
// ERROR_MOD_NOT_FOUND - Missing DLL (tested with file that did not exist)
//
// Module-loading functions can return several other errors, look at winerror.h
// list starting at ERROR_INVALID_MODULETYPE
//
// Obviously, since it's just a wrapper around LoadLibrary this function is not
// safe to call from DllMain.
//
// NB: GetErrorMode() is only available on Vista / Server 2003 or later.
HMODULE LoadLibraryIfValid(LPCTSTR pszPath)
{
HMODULE hModule = NULL;
UINT prevErrorMode = GetErrorMode();
SetErrorMode(prevErrorMode | SEM_FAILCRITICALERRORS);
hModule = LoadLibrary(pszPath);
SetErrorMode(prevErrorMode);
return hModule;
}
If targeting Windows 7 / Server 2008 R2 or later, the Get/SetThreadErrorMode()
functions are available, but might not be worth it or even a good alternative
(discussion in comments, below)
If anyone cared enough to put the time into it (I sure don't), a version of
this function could easily be written using GetModuleHandle for kernel32 and
GetProcAddress to be compatible with earlier versions of Windows as well as
provide a global/per-thread error mode option for platforms that support it
(truly pointless because it's only changed for the duration of one call anyway).
This is the largest commentary-to-code ratio in my life.
I'm trying to figure out how to load the AppDomainManager assembly in a managed clr hosting scenario from a folder different from the native .exe file.
Has anyone done an IHostAssemblyStore::ProvideAssembly implementation,
that will load the AppDomainManager assembly from a folder, and would like to share it?
I'm new to this WinApi/OLE/whatever programming, and I have no idea how to
get an IStream* from a file on disk in c++.
Or is there another way load the AppDomainManager for the DefaultDomain from an arbitrary folder?
SHCreateStreamOnFile will load the assembly from disc into an IStream, e.g.:
HRESULT hr = SHCreateStreamOnFile(assemblyPath, STGM_READWRITE, ppStmAssemblyImage);
Pitfall:
The CLR will call your ProvideAssembly implementation several times for one assembly.
Be sure to use the postPolicyIdentity in the AssemblyBindInfo to return the "right" bits at the right call considering the processor architecture:
If your assembly was build "Any CPU" but you return it when the postPolicyIdentity contains e.g. "processorarchitecture=amd64" at the end the runtimeHost->Start() call will fail.
Instead you have to return COR_E_FILENOTFOUND (0x80070002) until the processorarchitecture matches.
This is what I did:
DWORD dwReturn;
MCI_OPEN_PARMS mciOpenParms;
mciOpenParms.lpstrDeviceType = _T("MPEGvideo");
mciOpenParms.lpstrElementName = m_tmpFileName;
dwReturn = mciSendCommand(NULL, MCI_OPEN,
MCI_OPEN_TYPE | MCI_OPEN_ELEMENT,
(DWORD)(LPVOID) &mciOpenParms);
if (dwReturn)
{
wchar_t chError[100];
mciGetErrorString(dwReturn,chError,sizeof(chError));
//report the error here
}
When I run the code, I see that dwReturn is 266 and chError is set to "Unknown problem while loading the specified device driver". What could be wrong?
Note: I also tried "mpegvideo" instead of "MPEGvideo"; it didn't help. Where are these things documented anyway?
It works for me. Maybe (likely) you have the MCI register all screwed up. Or maybe the file you are opening is corrupted somehow.
As far as I know these device names are not documented anywhere. But you can find the ones configured in your system in the registry: HKLM\Software\Microsoft\Windows NT\CurrentVersion\MCI32. My system has:
AVIVideo
CDAudio
MPEGVideo
Sequencer
WaveAudio
And given that the MCI is not used much nowadays, I'd say that they are pretty standard.
If you're specifying MCI_OPEN_ELEMENT then mciOpenParms.lpstrDeviceType must be null.
See the Remarks section.
*To use automatic type selection (via the entries in the registry), assign the filename and file extension to the lpstrElementName member of the structure identified by lpOpen, set the lpstrDeviceType member to NULL, and set the MCI_OPEN_ELEMENT flag.*