This is a terrible idea, but i'm seeing if it's even feasible before I walk down this road.
I have to write a Win32 C++ program that can dynamically load a library based on a file that has serialized information on what dll, function, signature, and arguements to use. Loading the library is trivial (LoadLibraryEx works fine). Then getting the function pointer is easy (not a big deal, the GetProcAdderss takes care of this). However the rest is tricky.
Here's my plan of attack, feel free to let me know if this isn't the best approach:
Open the serialized information from a file on what DLL to load, and what function to execute.
LoadLibraryEx to bring in the DLL
GetProcAddress to get the function pointer (after casting the byte array to a string)
Write the arguments (which are read in as a byte array) to memory in bytes.
Get the address to the beginning of each argument (i'll know from serialization what the size of each argument is).
Using assembly jump to the beginning of the function pointer, push the addresses on the heap to the arguments in the stack (in reverse order).
Execute and get back the return value address (as a void * ?)
Use the memory address of the return value (that I got from assembly) and the size (which I got from the serialization) of the return type value and write the raw bytes back to a file.
Keep in mind my limitations:
I will never know except for run-time what the signature, dll, function name is.
It is always read in from a file.
Is there a better approach, will this approach even work?
Update
For anyone who comes poking in this thread to learn more, I found a solution. In C you can dynamically load a library using dlopen (there's a winlib of this for ease of use). Once loaded you can dynamically execute functions using libffi (supports mac/ios/win 64 and 32bit). This only gets you to C functions and primitive types (pointer,uint,int,double,float) and thats about it. However using macosx objective-c bridge you can access objective-c by loading libobjc (osx's native obj-c to c "toll free" bridge). Then through that dynamically create obj-c and c++ classes. A similar technique can be done on windows using C# and its marshaling capabilities.
This ends up with HIGH overhead, and you must be VERY careful about your memory, in addition don't mix pointers from C/C#/C++. Finally, whatever you do, at runtime. BE ABSOLUTELY SURE YOU KNOW YOUR TYPES.... seriously. BTW, libffi/cinvoke, amazing libraries.
There are existing libraries that can do what you describe, such as C/Invoke:
http://www.nongnu.org/cinvoke/
General rule, that if you have a terrible idea, drop it and find a good one.
If the signature is not known at all, what you describe will fall on face. Suppose your call works for my function as it is. I change the function from __stdcall to __cdecl or back, and you will crash.
Also you don't handle the return.
If you relax the "unknown" to allow some limitations, like fixing a return type and calling convention, you are somewhat ahead -- then really you can emulate the call but what is it good for? This whole thing sounds like a self-service hack-me engine.
The normal way is to publish some fixed interface (using function signatures), and make the DLL support it. Or arrange some uniform data transfer mechanism.
I'd suggest you to describe what you're after really and post ask that, maybe on Programmers SE.
Related
I am fairly new to C++ and working with DLLs. I have a main application that aggregates results from different measurements. As the measurements are different from case to case I decided to put them into external DLLs so they can be loaded at runtime (they simply all export the same function). The idea is to just load them like this so the aggregator can be extended depending on the runtime needs:
typedef int (*measure)(measurement &dataHolder);
int callM() {
[...]
measurement dataHolder;
lib = LoadLibraryA("measureDeviceTypeA.dll");
measure measureFunc = (measure)GetProcAddress(lib, "measureFunc");
pluginFunc(dataHolder);
[...] // close the lib and load the next one depending on found Devices
}
This works pretty well for simple datatypes (depending on the actual definition of the struct "measurement") such as this:
typedef struct measurement {
DWORD realPBS;
DWORD imaginaryPBS;
int a;
} measurement;
Now there also may be a string of arbitrary length (char representations of results). I would like to put them into the measurement struct as well and fill them inside the actual worker function inside the DLL. My first assumption was that it would be easy to just use std::string, which works sometimes and sometimes not (as it will reallocate memory on std::string().append() and this might break (access violation) depending on the actual runtime environment of the program and the dll). I read here and here that returning a string from a function is a bad idea.
So what would be the "proper" C++ way of returning arbitrary length strings from such a call? Is it helpful at all to pass a struct to the DLL or should I split it into separate calls? I don't want to have pointers dangling around or unfreed memory when I close the DLL again.
This won't work with std::string, as noted by Dani in the comments. The problem is that std::string is a type that belongs to your implementation, and different C++ implementations have different std::strings.
For DLL's specifically (Microsoft), you do have another alternative. COM is an ancient technology, but it still works today and is unlikely to go away ever. And it has its own string type, BSTR. Visual Studio provides a helper C++ class bstr_t for your own code, but on the interface you'd use the plain BSTR from _bstr_t::GetBSTR.
BSTR relies on the Windows allocator SysAllocString from OleAut32.dll
The problem is, that the string data is often allocated on the heap, so it has to be freed / managed somehow.
You could think, hey std::string is returned by value - so why I need to care about memory management. The problem is that usually only very small strings are stored "inside" the class. For larger strings the string class contains a pointer to some "heap-storage".
Dlls can be used from and with different programming languages - which is the reason that dlls do not share a "memory manager", freeing in the dll would fail.
To solve this you need to have two function calls, one which returns a pointer / handle to the data and one to free it. Or the caller could give the callee some pointer where it wants the data to be stored. You need for that a maximum-byte-count, too.
As you can see, there are some reasons why you should avoid these APIs - but it is not always possible. See for example the Windows API (there you can find both approaches).
Another approach would be to ensure a shared memory manager, but this is tricky somehow because it must be done really early!
I'm new to c++ and Access. I'm working with a project calls dll (compiled by c++) from Access.
I want to understand how are the parameters passed into the dll.
The input data for dll is prepared in Access, and we call the dll from Access.
We associate "RunFunction" with the dll we want to call.
The line in Access calls the dll:
Results = RunFunction(Data.age, Data.calendar, Data.timesheet, Data.extra)
The cpp code that complies the dll:
double __stdcall RunFunction(double * iData, double(*iCalendar)[100], double(*iTimesheet)[100])
First question, from the cpp code, I found the *iData(in c++) actually contains all info from Data (in Access).
Why it could happen? I thought only Data.age is passed into *iData, not the whole Data?
Second question, the the RunFunction from Access has four input parameters, while c++ only takes three, why it doesn't this cause any issue?
First, consider that inside Access the value of Data.age might be inside a buffer containing the entire record or some other structure. So when the address of that one value is passed to you in C++, you can explore neighboring addresses and see what’s in it. Don’t do that!
Second, look at the way __stdcall works. It was designed in the early days of C when function arguments were not checked at all! You can pass fewer or more parameters on the caller side and not mess up the stack. If you pass extra, no big deal. If you leave off some, then using the rightmost names in the function will give garbage values and witing to them can cause all sorts of problems including clobbering the return address.
I'm designing a JNI interface that passes string parameters from Java to C++. I need high performance and have been able to use Direct ByteBuffer and String.getBytes() to do that fairly well, but the penalty for passing strings to C/C++ still remains fairly high. I recently read about the Open JDK's Unsafe class. This excellent page got me started, but I'm finding Unsafe to be woefully, but understandably poorly documented.
I'm wondering, if I use the Unsafe class to obtain a pointer to a string and pass it to C++, is there a risk that the object has moved before the C++ code is entered? And even while C++ is executing? Or are these addresses provided by the Unsafe code somehow pinned? If they aren't pinned, how are these Unsafe pointers ever useful?
Unsafe is not meant to interop with JNI. So obtained via Unsafe could change any time (even in parallel with your C++).
JNI API has ability to pin object in memory to access array content (in HotSpot JVM it would block GC thus may have negative effect on GC pause duration).
In particular, Get*ArrayElements would pin array until you explicitly do Release*ArrayElements. GetStringChars work similar way.
Direct ByteBuffer hold pointer to memory buffer outside of heap, hense this buffer is not moving and you can access it for Native code.
I've read the Java source for java.misc.Unsafe and have a bit more insight.
Unsafe has at least two ways of dealing with memory.
allocateMemory/reallocateMemory/freeMemory/etc -- As far as I can tell this allocation of memory is outside the heap so faces no GC'ing challenges. I have indirectly tested this and it seems that the long returned is simply a pointer to the memory. It seems very likely that this type of memory is safe to pass through JNI to native code. And the application Java code should be able to quickly modify/query it before and after JNI calls by using some of the other intrinsic Unsafe methods that support this style of memory pointer.
object+offset - These methods accept a pointer to an object and an "offset" token to indicate where in the object to fetch/modify the value. The objects presumably are always in the Java heap, but passing the object to these methods probably helps resolve GC complications. It does sounds like the "offset" is sometimes a "cookie" rather than an actual offset, but it also sounds like that in the case of arrays, arrayBaseOffset() returns an "offset" that one can manipulate arithmetically. I don't know if this object+offset is safe for JNI code. I don't see a method to generate a pointer directly to the Java object in the heap that one could (dangerously) pass through JNI. One could pass an object and offset, but given the cost of passing Objects through JNI, this approach is not appealing anyway.
Like (1), the code associated with the page I referenced in my posting is probably pretty safe for JNI interactions. It takes the object+offset approach when dealing with String, but uses approach (1) when dealing with the direct ByteBuffer, which always reside outside the Java heap. Direct ByteBuffer's are very JNI friendly and often they can be used in ways that avoids the JNI Object passing costs I allude to in my comment to Tom above.
This is something that recently crossed my mind, quoting from wikipedia: "To initialize a function pointer, you must give it the address of a function in your program."
So, I can't make it point to an arbitrary memory address but what if i overwrite the memory at the address of the function with a piece of data the same size as before and than invoke it via pointer ? If such data corresponds to an actual function and the two functions have matching signatures the latter should be invoked instead of the first.
Is it theoretically possible ?
I apologize if this is impossible due to some very obvious reason that i should be aware of.
If you're writing something like a JIT, which generates native code on the fly, then yes you could do all of those things.
However, in order to generate native code you obviously need to know some implementation details of the system you're on, including how its function pointers work and what special measures need to be taken for executable code. For one example, on some systems after modifying memory containing code you need to flush the instruction cache before you can safely execute the new code. You can't do any of this portably using standard C or C++.
You might find when you come to overwrite the function, that you can only do it for functions that your program generated at runtime. Functions that are part of the running executable are liable to be marked write-protected by the OS.
The issue you may run into is the Data Execution Prevention. It tries to keep you from executing data as code or allowing code to be written to like data. You can turn it off on Windows. Some compilers/oses may also place code into const-like sections of memory that the OS/hardware protect. The standard says nothing about what should or should not work when you write an array of bytes to a memory location and then call a function that includes jmping to that location. It's all dependent on your hardware and your OS.
While the standard does not provide any guarantees as of what would happen if you make a function pointer that does not refer to a function, in real life and in your particular implementation and knowing the platform you may be able to do that with raw data.
I have seen example programs that created a char array with the appropriate binary code and have it execute by doing careful casting of pointers. So in practice, and in a non-portable way you can achieve that behavior.
It is possible, with caveats given in other answers. You definitely do not want to overwrite memory at some existing function's address with custom code, though. Not only is typically executable memory not writeable, but you have no guarantees as to how the compiler might have used that code. For all you know, the code may be shared by many functions that you think you're not modifying.
So, what you need to do is:
Allocate one or more memory pages from the system.
Write your custom machine code into them.
Mark the pages as non-writable and executable.
Run the code, and there's two ways of doing it:
Cast the address of the pages you got in #1 to a function pointer, and call the pointer.
Execute the code in another thread. You're passing the pointer to code directly to a system API or framework function that starts the thread.
Your question is confusingly worded.
You can reassign function pointers and you can assign them to null. Same with member pointers. Unless you declare them const, you can reassign them and yes the new function will be called instead. You can also assign them to null. The signatures must match exactly. Use std::function instead.
You cannot "overwrite the memory at the address of a function". You probably can indeed do it some way, but just do not. You're writing into your program code and are likely to screw it up badly.
I'm writing an abstraction layer on top of some graphics API (DirectX9 and DirectX11) and I would like your opinion.
Traditionally I would create a base class for each concept I want to abstract.
So in typical OO fashion I would have for example a class Shader and 2 subclasses DX9Shader and DX11Shader.
I would repeat the process for textures, etc... and when I need to instantiate them I have an abstract factory that will return the appropriate subclass depending on the current graphics API.
Following RAII, the returned pointer would be encapsulated in a std::shared_ptr.
So far so good but in my case there are a few problems with this approach:
I need to come up with a public interface that encapsulate the functionality of both APIs (and other APIs in the future).
The derived class are stored in separate DLLs (one for DX9, one for DX11 etc...) and having a shared_ptr to them in the client is a curse: on exit the graphic dlls are unloaded and if the client still has a shared_ptr to one of the graphics objects boom, crash due to calling code from unloaded DLL.
This prompted me to re-design the way I do things:
I thought I could just return raw pointers to the resources and have the graphics API clean after itself but there's still the issue of dangling pointers on the client side and the interface problem.
I even considered manual reference counting like COM but I thought that would be a step backwards (correct me if I'm wrong, coming from the shared_ptr world, manual reference counting seems primitive).
Then I saw the work of Humus where all his graphics classes are represented by integer IDs (much like what OpenGL does).
Creating a new object only returns its integer ID, and stores the pointer internally; it's all perfectly opaque!
The classes that represent the abstraction (such as DX9Shader etc...) are all hidden behind the device API which is the only interface.
If one wants to set a texture, it's just a matter of calling device->SetTexture(ID) and the rest happens behind the scenes.
The downfall is that the hidden part of the API is bloated, there is a lot of boiler plate code required to make it work and I'm not a fan of a do-it-all class.
Any ideas/thoughts ?
You say that the main problem is that a DLL is unloaded while still having a pointer to its internals. Well... don't do that. You have a class instance, who's members are implemented in that DLL. It is fundamentally an error for that DLL to be unloaded so long as those class instances exist.
You therefore need to be responsible in how you use this abstraction. Just as you need to be responsible with any code you load from a DLL: stuff that comes from the DLL must be cleaned up before you unload the DLL. How you do that is up to you. You could have an internal reference count that gets incremented for every object the DLL returns and only unload the DLL after all referenced objects go away. Or anything, really.
After all, even if you use these opaque numbers or whatever, what happens if you call one of those API functions on that number when the DLL is unloaded? Oops... So it doesn't really buy you any protection. You have to be responsible either way.
The downsides of the number method that you may not be thinking about are:
Reduced ability to know what an object actually is. API calls can fail because you passed a number that isn't really an object. Or worse, what happens if you pass a shader object into a function that takes a texture? Maybe we're talking about a function that takes a shader and a texture, and you accidentally forget the order of the arguments? The rules of C++ wouldn't allow that code to even compile if those were object pointers. But with integers? It's all good; you'd only get runtime errors.
Performance. Every API call will have to look this number up in a hashtable or something to get an actual pointer to work with. If it's a hashtable (ie: an array), then it's probably fairly minor. But it's still an indirection. And since your abstraction seems very low-level, any performance loss at this level can really hurt in performance-critical situations.
Lack of RAII and other scoping mechanisms. Sure, you could write a shared_ptr-esque object that would create and delete them. But you wouldn't have to do that if you were using an actual pointer.
It just doesn't seem worthwhile.
Does it matter? To the user of the object, it is just an opaque handle. its actual implementation type doesn't matter, as long as I can pass the handle to your API functions and have them do stuff with the object.
You can change the implementation of these handles easily, so make it whatever is easier for you now.
Just declare the handle type as a typedef of either a pointer or an integer, and make sure that all client code uses the typedef name, then the client code doesn't depend on the specific type you chose to represent your handles.
Go for the simple solution now, and if/when you run into problems because that was too simple, change it.
Regarding your p. 2: Client is always unloaded before libraries.
Every process has its library dependency tree, with .exe as tree root, user Dll at intermediate levels, and system libraries at low level. Process is loaded from low to high level, tree root (exe) is loaded last. Process is unloaded starting from the root, low-level libraries are unloaded last. This is done to prevent situations you are talking about.
Of course, if you load/unload libraries manually, this order is changed, and you are responsible to keep pointers valid.