Modifying the stack on Windows, TIB and exceptions - c++

Question

The origin of my question effectively stems from wanting to provide an implementation of pthreads on Windows which supports user provide stacks. Specifically, pthread_attr_setstack should do something meaningful. My actual requirements are a bit more involved than this but this is good enough for the purpose of the post.
There are no public Win APIs for providing a stack in either the Fiber or Thread APIs. I've searched around for sneaky backdoors, workarounds and hacks, there's nothing going. In fact, I looked that the winpthread source for inspiration and that ignores any stack provided to pthread_attr_setstack.
Instead I tried the following "solution" to see if it would work. I create a Fiber using the usual combination of ConvertThreadToFiber, CreateFiberEx and SwitchToFiber. In CreateFiberEx I provide a minimal stack size. In the entry point of the fibre I then allocate memory for a stack, change the TIB fields: "Stack Base" and "Stack Limit" appropriately (see here: http://en.wikipedia.org/wiki/Win32_Thread_Information_Block) and then set ESP to the high address of my stack.
(In a real world case I would setup the stack better than this and change EIP as well so that this step behaves more like the posix funciton swapcontext, but you get the idea).
If I make any OS calls when on this different stack then I'm pretty much screwed (printf for example dies). However this isn't an issue for me. I can ensure that I never make sure calls when on my custom stack (hence why I said my actual requirements are a bit more involved). Except...I need exceptions to work. And they don't! Specifically, if I try to throw and catch an exception on my modified stack then I get an assert
Unhandled exception at 0xXXXXXXXX ....
So my (vague) question is, does anyone have any insight as to how exceptions and a custom stack might not be playing nicely together? I appreciate that this is totally unsupported and can happily except nil response or "go away". In fact, I've pretty much decided that I need a different solution and, despite this involving compromise, I'm likely to use one. However, curiosity gets the better of me so I'd like to know why this doesn't work.
On a related note, I wondered how Cygwin dealt with this for ucontext. The source here http://szupervigyor.ddsi.hu/source/in/openjdk-6-6b18-1.8.13/cacao-0.99.4/src/vm/jit/i386/cygwin/ucontext.c uses GetThreadContext/SetThreadContext to implement ucontext. However, from experimentation I see that this also fails when an exception is thrown from inside a new context. In fact the SetThreadContext call doesn't even update the TIB block!
EDIT (based on the answer from #avakar)
The following code, which is very similar to yours, demonstrates the same failure. The difference is that I don't start the second thread suspended but suspend it then try to change context. This code exhibits the error I was describing when the try-catch block is hit in foo. Perhaps this simply isn't legal. One notable thing is that in this situation the ExceptionList member of the TIB is a valid pointer when modifyThreadContext is called, whereas in your example it's -1. Manually editing this doesn't help.
As mentioned in my comment to your answer. This isn't precisely what I need. I would like to switch contexts from the thread I'm current on. However, the docs for SetThreadContext warn not to call this on an active thread. So I'm guessing that if the below code doesn't work then I have no chance of making it work on a single thread.
namespace
{
HANDLE ghSemaphore = 0;
void foo()
{
try
{
throw 6;
}
catch(...){}
ExitThread(0);
}
void modifyThreadContext(HANDLE thread)
{
typedef NTSTATUS WINAPI NtQueryInformationThread_t(HANDLE ThreadHandle, DWORD ThreadInformationClass, PVOID ThreadInformation, ULONG ThreadInformationLength, PULONG ReturnLength);
HMODULE hNtdll = LoadLibraryW(L"ntdll.dll");
auto NtQueryInformationThread = (NtQueryInformationThread_t *)GetProcAddress(hNtdll, "NtQueryInformationThread");
DWORD stackSize = 1024 * 1024;
void * mystack = VirtualAlloc(0, stackSize, MEM_COMMIT, PAGE_READWRITE);
DWORD threadInfo[7];
NtQueryInformationThread(thread, 0, threadInfo, sizeof threadInfo, 0);
NT_TIB * tib = (NT_TIB *)threadInfo[1];
CONTEXT ctx = {};
ctx.ContextFlags = CONTEXT_ALL;
GetThreadContext(thread, &ctx);
ctx.Esp = (DWORD)mystack + stackSize - ((DWORD)tib->StackBase - ctx.Esp);
ctx.Eip = (DWORD)&foo;
tib->StackBase = (PVOID)((DWORD)mystack + stackSize);
tib->StackLimit = (PVOID)((DWORD)mystack);
SetThreadContext(thread, &ctx);
}
DWORD CALLBACK threadMain(LPVOID)
{
ReleaseSemaphore(ghSemaphore, 1, NULL);
while (1)
Sleep(10000);
// Never gets here
return 1;
}
} // namespace
int main()
{
ghSemaphore = CreateSemaphore(NULL, 0, 1, NULL);
HANDLE th = CreateThread(0, 0, threadMain, 0, 0, 0);
while (WaitForSingleObject(ghSemaphore, INFINITE) != WAIT_OBJECT_0);
SuspendThread(th);
modifyThreadContext(th);
ResumeThread(th);
while (WaitForSingleObject(th, 10) != WAIT_OBJECT_0);
return 0;
}

Answer 1

Both exceptions and printf work for me, and I don't see why they shouldn't. If you post your code, we can try to pinpoint what's going on.
#include <windows.h>
#include <stdio.h>
DWORD CALLBACK ThreadProc(LPVOID)
{
try
{
throw 1;
}
catch (int i)
{
printf("%d\n", i);
}
return 0;
}
typedef NTSTATUS WINAPI NtQueryInformationThread_t(HANDLE ThreadHandle, DWORD ThreadInformationClass, PVOID ThreadInformation, ULONG ThreadInformationLength, PULONG ReturnLength);
int main()
{
HMODULE hNtdll = LoadLibraryW(L"ntdll.dll");
auto NtQueryInformationThread = (NtQueryInformationThread_t *)GetProcAddress(hNtdll, "NtQueryInformationThread");
DWORD stackSize = 1024 * 1024;
void * mystack = VirtualAlloc(0, stackSize, MEM_COMMIT, PAGE_READWRITE);
DWORD dwThreadId;
HANDLE hThread = CreateThread(0, 0, &ThreadProc, 0, CREATE_SUSPENDED, &dwThreadId);
DWORD threadInfo[7];
NtQueryInformationThread(hThread, 0, threadInfo, sizeof threadInfo, 0);
NT_TIB * tib = (NT_TIB *)threadInfo[1];
CONTEXT ctx = {};
ctx.ContextFlags = CONTEXT_ALL;
GetThreadContext(hThread, &ctx);
ctx.Esp = (DWORD)mystack + stackSize - ((DWORD)tib->StackBase - ctx.Esp);
tib->StackBase = (PVOID)((DWORD)mystack + stackSize);
tib->StackLimit = (PVOID)((DWORD)mystack);
SetThreadContext(hThread, &ctx);
ResumeThread(hThread);
WaitForSingleObject(hThread, INFINITE);
}