I'm interested in using API spying/hijacking to implement some core features of a project I'm working on. It's been mentioned in this question as well, but that wasn't really on topic so I figured it'd be better with a question of its own for this.,
I'd like to gather as much information as possible on this, different techniques/libraries (MS Detours, IAT patching) or other suggestions.
Also, it'd be especially interesting to know if someone has any real production experience of using such techniques -- can they be made stable enough for production code or is this strictly a technique for research? Does it work properly over multiple versions of windows? How bug prone is it?
Personal experiences and external links both appreciated.
I implemented syringe.dll (L-GPL) instead of MS Detours (we did not like the license requirements or huge payment for x64 support) it works fantastically well, I ported it from Win32 to Win64, we have been using in our off-the-self commercial applications for around 2 years now.
We use it for very simple reasons really its to provide a presentation frame work for re-packing, re-branding the same compiled application as many different products, we do general filtering and replacment for string, general resource, toolbar, and menus.
Being L-GPL'd we supply the source, copyright etc, and only dynamically link to the library.
Hooking standard WinAPI functions is relatively safe since they're not going to change much in the near future, if at all, since Microsoft does it's
best to keep the WinAPI backwards compatible between versions.
Standard WinAPI hooking, I'd say, is generally stable and safe.
Hooking anything else, as in the target program's internals, is a different story.
Regardless of the target program, the hooking itself is usually a solid practice. The weakest link of the process is usually finding the correct spot,
and hanging on to it.
The smallest change in the application can and will change the addresses of functions, not to mention dynamic libraries and so forth.
In gamehacking, where hooking is standard practice, this has been defeated to some degree with "sigscanning", a technique first developed by LanceVorgin on the somewhat infamous
MPC boards. It works by scanning the executable image for the static parts of a function, the actual instruction bytes that won't change unless the
function's action is modified.
Sigscanning is obviously better than using static address tables, but it will also fail eventually, when the target application is changed enough.
Example implementation of sigscanning in c++ can be found here.
I've been using standard IAT hooking techniques for a few years now and it works well has been nice and stable and ported to x64 with no problems. The main problems I've had have been more to do with how I inject the hooks in the first place, it took a fair while to work out how best to suspend managed processes at the 'right' point in their start up so that injection was reliable and early enough for me. My injector uses the Win32 debug API and whilst this made it easy to suspend unmanaged processes it took a bit of trial and error to get managed processes suspended at an appropriate time.
My uses for IAT have mostly been for writing test tools, I have a deadlock detection program which is detailed here: http://www.lenholgate.com/blog/2006/04/deadlock-detection-tool-updates.html, a GetTickCount() controlling program which is available for download from here http://www.lenholgate.com/blog/2006/04/tickshifter-v02.html
and a time shifting application which is still under development.
Something a lot of people forget is that windows dll's are compiled as hot-patchable images(MSDN).
Hot-patching is the best way to do WinAPI detours, as its clean and simple, and preserves the original function, meaning no inline assembly needs to be used, only slightly adjusted function pointers.
A small hot patching tutorial can be found here.
Related
Background: I'm working on porting a large project that was developed in C++ exclusively for unix systems, to be compatible with windows to make way for an eventual windows distribution. I don't have very much experience in Windows development, but I'd like to get whatever I can do, done right, before senior developers move in and take over.
Question: So for a while, I've been looking up Windows versions for all the unix/posix calls that are used in the software, most coming from dirent.h, unistd.h, and some ones under sys, like sys/stat.h or sys/types.h. And although it'll take a lot of work to modify the programs to adopt the new Win32 API calling conventions and return types (and sometimes all new functions), it'll probably work, eventually.
But I've been seeing this come up frequently, the fact that MinGW, I guess, includes many native unix calls and functionalities as part of the GCC environment, and can translate them into windows-compatible calls so you can compile on Windows, and use said compiled program on Windows easily. In fact, one of the similar questions I read in the sidebar talks about just that. What I don't understand, and seem to have trouble understanding, is exactly the extent of this built-in translation functionality, and where I can find a list of system stuff that'll work with this.
Sorry this post is a little unstructured and that I'm so green with this, but I only have 2 weeks to try to accomplish something with this before I get swapped with a senior developer.
No. MinGW does not attempt to implement UNIX functions on Windows. It cannot replicate most system calls with no effort.
However, Cygwin does do that.
Can I use the WRL library in C++ as a replacement for ATL to write a COM component? And if yes, would I be able to use it on older desktop Windows systems, like Windows XP?
I'm pretty sure that the answer to the first question is positive as I found this tutorial on MSDN:
http://msdn.microsoft.com/en-us/library/jj822931.aspx
but what about non-Windows 8 systems?
Well, sure you can. You can write a COM server in pure C++ code without any helper classes at all. You can write one in C if you really want to, although that's a violation of the Geneva Convention on Programmer's Rights in most jurisdictions.
What is probably not so visible is what is missing. WRL just doesn't make it any easier to write a server. Or maintain it. What you'll miss out on:
No help whatsoever to implement IDispatch. Obsoleted in WinRT but still quite important in regular servers. The only way to consume the server from a scripting language, IDispatch provides the late binding support.
No help whatsoever for registration. Obsoleted in WinRT, required for regular servers. Note how the DllRegisterServer() entrypoint is missing. The page you linked patches around the problem by requiring you to write a registry script by hand. That works, it is not exactly something you'd want to maintain or document so IT staff can get it right.
No wrappers for Automation types like BSTR, VARIANT and SAFEARRAY. Obsoleted in WinRT along with IDispatch. You can still fall back to <comutil.h> however.
No help whatsoever from the wizards that are built into Visual Studio that help you get it right. COM servers are awkward because definitions appear in multiple places in your code. And need to be an exact match. You get no help to ensure that CalculatorComponent is an exact match with the IDL, it is entirely up to you. Compiler errors you get from making small mistakes, particularly when maintaining it, can be painful.
And a whole bunch if smaller stuff not worth mentioning, like apartments, ActiveX, aggregation, error info, etc. Hurts pretty bad when you need it though. A possible advantage of WRL is that there's less mystical glue, ATL has a fair amount of it that considerably raises the abstraction level. That was intentional but it has to be learned. There's none at all when you use pure C++ of course, albeit that you have to write too much of it yourself.
Yes. You can write a standard COM component.
There is a sample for this directly in the docs.
And no: Such a COM component will only run on Windows 8 and later...
Perhaps some of you people have heard of http://suckless.org/ and their set of Unix tools. Basically, they're a set of programs that each aim to do one thing but do it well, while still being as simple and resource-light as possible.
I've been trying to find a way to reproduce this style of programming on Windows with C++ but all the libraries I know of would produce binaries that are huge with respect to their function. Even the simplest of anything Qt, for example, is generally several megabytes large. I'm not against packaging dependencies along with distributables but I wouldn't want to do it to that level.
Binary size is not one of my main goals but simplicity is and big libraries like these are, by construction, not simple. If binary size were your primary concern you could use runtime compression just like kkreiger or malware.
A possibility would be to go commando and use only ISO Standard C++ libraries but rebuilding a sockets or networking system for a small single-purpose application is not really something anyone would want to be troubled with.
For some reason I thought there was some general-purpouse library that Windows developers could count on everyone and their grandma having readily accessible but now I don't know if anything like that exists. What can you use to write code that adheres to the Unix Philosophy but for Windows targets?
You should target the Win32 API directly. You can't get much lower level than that. In the Windows world, everything directly or indirectly wraps the SDK functions, including the so-called "standard C++ libraries".
Alternatively, you could use something like MFC or WTL, which are relatively thin C++ wrappers over the Win32 API. Because of the overhead of the class libraries, such programs will be slightly smaller than those created using only the SDK, but nowadays, the actual overhead is completely insignificant.
The desires expressed in your question are precisely why I learned and still use the Win32 API today, so that's definitely what I would go with. Plus, your programs will look and feel native, which is way better than the crap most "cross-platform GUI toolkits" put out. The advantages of this can't be underestimated.
But if you just open up Visual Studio and compile a simple little SDK "Hello World" app, it'll still be awfully large. Kilobytes, to be sure, but that still seems like a lot for about the simplest app imaginable. If you really need to cut things down further, you can try telling Visual Studio not to link to the C runtime libraries and define your own main entrypoint. This does mean that you'll have to implement all of your own startup initialization code, but this can reduce the size of a trivial app substantially.
Matt Pietrek had this same idea some years ago, although you'll probably want to take time to "modernize" his original code significantly if you decide to go this route.
FLTK is a popular cross platform minimal gui toolkit.
Or a popular alternative if you don't need too much detailed interaction is just to fire up a minimal embedded webserver and do all the 'gui' in html in a browser.
I'm interning in a company for the summer and I've to look at different ways of looking at the current codebase (C++,MFC, around 100K lines) and using state machines to model the current program.
I've been reading a couple papers and CPP2XMi looks like it may be some use to try to build sequence diagrams as a start.
The end goal is to gauge the feasibility of moving away from microsoft as an O/S and look at development (possibly in another language) on *nix.
I've also started looking at the MFC dependancies to see if we could just port the current C++ code.
I've had the program running through WINE and performance-wise, it seems acceptable but I still need to investigate other solutions as this will only work on X86 while we have other solutions running running on MIPS and ARM.
Any other ideas or caveats I could look at?
The first thing I would look at is where do I use mfc and other non portable stuff. If the only place there is mfc is in the interface layer for example you then can isolate the work.
If there is no such separation I would look at the fesablity of creating some sections of the code that are isolated and portable. Once you have a base of portability you can begin abstracting all of the services rendered by the non portable code. Any way you slice it though MFC to Nix is a big change and will require a significant amount of work. One other possibility is to see if you can run it under a windows emulator.
From reading through the wxWidgets book, it seems very similar to MFC. You might have a look at it.
I would first look into whether the GUI is separated from the rest of the application. With MFC, this includes limiting use of utility classes like CString to GUI-only code.
If the code is well-factored in this way, the easiest thing to do is probably to leave the MFC GUI code alone, and simply build a new GUI for your other platforms using the native GUI library of choice for each new platform. This will give a proper native appearance and behavior to the application that is really difficult to achieve any other way.
If the application logic is intermixed with the GUI code, it's a good time to ask whether you could devote resources to creating a proper separation, with the goal of doing the above once you've achieved separation. This is risky, from a business standpoint, because it can look like you have made a lot of effort and merely ended up back where you started. It isn't until you start work on the new GUI atop the refactored application that your sponsors see any real progress.
You can also look at portable GUI libraries like wxWidgets and Qt.
I have programmed for both MFC and wxWidgets, and they are conceptually very similar. I have never had to port code from one to the other, but I did once port from Borland's OWL to MFC, which was a similar experience. This sort of thing is not particularly difficult; it's just a grind. I can only recommend doing it when you have multiple reasons for dropping the old GUI library. For instance, perhaps you were also thinking of dropping Visual C++ entirely, or switching from Professional to Express, losing access to MFC. If you were planning on sticking with VC++ Professional (or above), it becomes difficult to justify throwing away your MFC GUI.
I once ported a big COM library from MFC to portable code. I used the STL and boost to replace all the MFC bits. For example, CString => std::string and VARIANT => boost::any.
It took forever, but it was mostly straightforward replacement and tweaking. Fortunately it didn't have any gui code-- it was a data processing library.
I am interested in hearing about peoples experience with embedding mono (open source implementation of .NET) in a C/C++ application. How is it to distribute such an application and what are the dependencies? I have tested on OS X and mono comes as a huge framework (hundreds of MB). Do users of my app all need this big framework or can it be stripped down or everything be compiled into the main executable.
I previously have experience with embedding Lua in a C++ app, and that works really well because I can link statically the whole lua interpreter in with my main executable. So I have no external dependencies. Is it possible to do something similar with mono?
Any Lua people here who can comment on how they found mono compared to Lua?
PS: By embedding I mean a C++ application which initializes a mono environment and loads a .NET assembly and executes it and then allows for communication between say C# code in assembly and C++ methods in main executable.
You should probably also take a look at Mono's Small Footprint page that describes how you can embed a smaller runtime. Heck, they do it themselves with Moonlight.
I hope that helps.
This is 2 years old question. So situation may become different now.
For me, most important point was GC. I embedded Lua for interactive-apps and games, because incremental GC required. Currently Lua 5.1 has precise, incremental GC, but I couldn't found any proof of incremental or precise GC on Mono. So memory will leak (even it's very tiny!), and apps will struggle intermittently.
People says GC pause can be solved by tuning some parameters and pooling objects, but as I experienced, It never be solved without any kind of distribution GC load over time approach in GC. Generational GC is one of distribution algorithm, but it's too rough, and almost not helpful.
Because you can't control lifetime pattern or reuse instance by pooling the objects used in code that not yours. (such as basic class library)
So I don't recommend the C# platform (Mono or .NET, at least yet) for interactive/(soft)realtime apps.
Edit
I don't know whether any incremental/concurrent approached GC is presented on Mono or .NET. If you can sure about they offer the kind of GC, of course, it's fine to use :)