Executing rundll32.exe with CreateProcess - c++

Question

I've created a DLL and would like to execute one of the functions using the rundll32.exe command on windows.
Using rundll32.exe, it runs correctly from the command line; however, I'd like to call it (rundll32.exe) from a separate program. I cannot directly call the function from my code due to 32/64 bit compatibility issues in the underlying libraries I'm using (Easyhook).
Below is what I'm using in an attempt to run the dll function:
STARTUPINFO si;
PROCESS_INFORMATION pi;
ZeroMemory( &si, sizeof(si));
si.cb = sizeof(si);
ZeroMemory( &pi, sizeof(pi));
LPCTSTR application = "C:\\Windows\\system32\\rundll32.exe";
LPTSTR cmd = "C:\\Projects\\Test\\mydll.dll,MyFunc";
BOOL cpRes = CreateProcess(application,
cmd,
NULL,
NULL,
FALSE,
0,
NULL,
NULL,
&si,
&pi);
if(cpRes == 0) {
cout << "ERROR\n";
cout << GetLastError() << endl;
} else {
cout << "DLL Launched!" << endl;
}
CloseHandle(pi.hProcess);
CloseHandle(pi.hThread);
The output to my console is always DLL Launched; however, I do not see the effects of my DLL actually being called (currently stubbed out in such a way that the command writes to a file).
If I swap out the application with something such as C:\\Windows\\system32\\notepad.exe, the program successfully runs.
For completion, here's the body of MyFunc:
ofstream file;
file.open("C:\\Projects\\Test\\test.txt");
file << "I wrote to a file!";
file.close();
Is there any reason CreateProcess cannot be used with rundll32? While reading over this I found several warnings about LoadLibrary() and DLLMain but it doesn't seem like they're relevant to this.
More Clarification:
This is currently a 32-bit application (allegedly) launching the 32-bit rundll32.exe (Logic will be added later to call the 32 or 64 bit version).
My dll is as follows:
extern "C" __declspec(dllexport) void CALLBACK MyFunc(HWND hwnd, HINSTANCE hinst, LPSTR lpszCmdLine, int nCmdShow);
void CALLBACK MyFunc(HWND hwnd, HINSTANCE hinst, LPSTR lpszCmdLine, int nCmdShow) { ... }
Which also has a .def file with:
EXPORTS
MyFunc
Running
C:\Windows\system32\rundll32.exe C:\Projects\Test\mydll.dll,MyFunc
produces the expected results.
Update
Setting application to NULL and including the rundll32.exe in cmd as mentioned in the comments seems to work.
Relevant Docs:
CreateProcess
RunDll32.exe

Answer 1

Per the CreateProcess() documentation:
If both lpApplicationName and lpCommandLine are non-NULL, the null-terminated string pointed to by lpApplicationName specifies the module to execute, and the null-terminated string pointed to by lpCommandLine specifies the command line. The new process can use GetCommandLine to retrieve the entire command line. Console processes written in C can use the argc and argv arguments to parse the command line. Because argv[0] is the module name, C programmers generally repeat the module name as the first token in the command line.
You are not repeating rundll32.exe as the first command-line token.
So, if you continue using the lpApplicationName parameter, then change this:
LPCTSTR application = "C:\\Windows\\system32\\rundll32.exe";
LPTSTR cmd = "C:\\Projects\\Test\\mydll.dll,MyFunc";
To this instead:
LPCTSTR application = TEXT("C:\\Windows\\system32\\rundll32.exe");
LPTSTR cmd = TEXT("C:\\Windows\\system32\\rundll32.exe C:\\Projects\\Test\\mydll.dll,MyFunc");
Note that you are currently compiling for ANSI/MBCS (by virtue of the fact that you are passing narrow strings to CreateProcess()). If you ever update the project to compile for Unicode, use this instead:
TCHAR cmd[] = TEXT("C:\\Windows\\system32\\rundll32.exe C:\\Projects\\Test\\mydll.dll,MyFunc");
This is because the documentation states:
lpCommandLine [in, out, optional]
...
The Unicode version of this function, CreateProcessW, can modify the contents of this string. Therefore, this parameter cannot be a pointer to read-only memory (such as a const variable or a literal string). If this parameter is a constant string, the function may cause an access violation.
You might consider changing cmd into a TCHAR[] array anyway, even in ANSI/MBCS, so you can do something like this:
LPCTSTR application = TEXT("C:\\Windows\\system32\\rundll32.exe");
TCHAR cmd[(MAX_PATH*2)+10];
wsprintf(cmd, TEXT("%s %s,%s"), application, TEXT("C:\\Projects\\Test\\mydll.dll"), TEXT("MyFunc"));
Either way, by passing the module filename as the first token in the lpCommandLine parameter, you can then set the lpApplicationName parameter to NULL:
The lpApplicationName parameter can be NULL. In that case, the module name must be the first white space–delimited token in the lpCommandLine string.
Let CreateProcess() setup the correct command-line to pass to rundll32.exe for you:
TCHAR cmd[] = TEXT("C:\\Windows\\system32\\rundll32.exe C:\\Projects\\Test\\mydll.dll,MyFunc");
BOOL cpRes = CreateProcess(NULL, cmd, ...);