I have a few Libraries defined in C++ source code. My problem is whenever a library is removed or a new one is added, something has to call the libraries "Initialize" function.
An example would be: A class for Network sockets called CLuaNet which has a function called CLuaNet::Initialize(*State);. How do I call this function in my project since I cannot predict the library name? If possible I would like this "linking" to be done at compile time, something like a macro.
Whenever a new instance of a Lua environment is opened, every library's initialize function has to be called with an the Lua state as argument.
The project runs on different architectures (X86/X64/ARMv6/ARMv7) and operating systems, making compiled libraries for every possible platform and OS combination is not feasible (Windows - DLL, Linux - SO, etc..). This is intended to be a server application.
EDIT: I am not using DLLs or SOs - everything is compiled into one executable for portability.
Note: I dont have alot of experience in project design/managment. Id
like to hear opinions and tips about my approach on this.
The normal mechanism for coping with name finding, is to create another well named function which returns sufficient meta data to allow you to call it.
extern "C" DLLEXPORT const MetaInfo & InfoFunction()
{
static MetaInfo luaNet(
CLuaNet::Initialize,
CLuaNet::Finalize,
CLuaNet::SomethingElse );
return luaNet; // Give main application information on how to call me
}
Where DLLEXPORT declares a function to be exported from the DLL/shared object.
MetaInfo is compiled into the DLL/shared object and has a set of well-defined slots which perform useful functions.
e.g.
struct MetaInfo {
int (*initializeFunction)( lua_State * L );
int (*closeFunction)( lua_State * L );
int (*utilityFunction)( lua_State* L );
MetaInfo( int (*)(lua_State *) init,
int (*)( lua_State*) fini,
int (*)( lua_State*) other ) :
initializeFunction( init),
closeFunction( fini ),
utilityFunction( other ) {}
};
I'm trying to provide my own entry point to a visual c++ executable using the linker switch "/entry". The associated Microsoft documentation goes into some detail about how to manually initialise the c runtime in a dll, but I can't grasp how to initialise it in my exe. The best I've got is that I need to call _CRT_init, but I don't know if that's a function or a macro, or where it might be defined, and Visual Studio (and msbuild) don't recognise the identifier, so no hints there.
The rationale here is that I'm attempting to unit test, with Google Test, an executable, and it won't link because main() clashes. The two approaches mentioned in the GTest FAQ aren't really generalisable and would require considerable rework to about 30 legacy executables to implement. Renaming each unit test application's main() seems like a super-easy approach, if only I could initialise the c runtime.
So I searched through the C Runtime source this afternoon, and the answer is "no". You cannot provide your own executable entry point not named main if you want to use the C Runtime. Quite aside from _CRT_INIT() being the an initialisation function for a dll (mainCRTStartup() is one of the initialisation functions for executables), mainCRTStartup() and its like call main() by name.
You could initialize the CRT in a standalone wrapper module which has
main; obj1
Rename the main of all existing exe modules to xx_main; obj2, obj3.... objn
Link obj1 (main) with (obj2, obj3.... objn).
To test a foo.cc file, you need to compile and link it into your unit
test program
What's the issue with this scheme ?
Though not specifically documented, it is possible to initialize the CRT in an executable in the same way it can be done in a DLL.
EXTERN_C BOOL WINAPI _CRT_INIT( HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpReserved );
// ThreadProc for CreateRemoteThread, a possible use case.
DWORD WINAPI ArbitraryAlternateEntryPoint( LPVOID )
{
_CRT_INIT( GetModuleHandle( NULL ), DLL_PROCESS_ATTACH, NULL );
// CRT functions can be used here.
_CRT_INIT( GetModuleHandle( NULL ), DLL_PROCESS_DETACH, NULL );
return 0;
}
The documentation suggests that _CRT_INIT should be called for each new thread in addition to the initial call, but in practice this is not necessary.
I want to define an array of things, like event handlers. The contents of
this array is completely known at compile time, but is defined among
multiple compilation units, distributed amongst multiple libraries that
are fairly decoupled, at least until the final (static) link. I'd like
to keep it that way too - so adding or deleting a compilation unit will
also automatically manage the event handler without having to modify a
central list of event handlers.
Here's an example of what I'd like to do (but does not work).
central.h:
typedef void (*callback_t)(void);
callback_t callbacks[];
central.c:
#include "central.h"
void do_callbacks(void) {
int i;
for (i = 0; i < sizeof(callbacks) / sizeof(*callbacks); ++i)
callbacks[i]();
}
foo.c:
#include "central.h"
void callback_foo(void) { }
callback_t callbacks[] = {
&callback_foo
};
bar.c:
#include "central.h"
void callback_bar(void) { }
callback_t callbacks[] = {
&callback_bar
};
What I'd like to happen is to get a single callbacks array, which contains
two elements: &callback_foo and &callback_bar. With the code above, there's
obviously two problems:
The callbacks array is defined multiple times.
sizeof(callbacks) isn't known when compiling central.c.
It seems to me that the first point could be solved by having the linker merge
the two callbacks symbols instead of throwing an error (possibly through some
attribute on the variable), but I'm not sure if there is something like that.
Even if there is, the sizeof problem should somehow also be solved.
I realize that a common solution to this problem is to just have a startup
function or constructor that "registers" the callback. However, I can see only
two ways to implement this:
Use dynamic memory (realloc) for the callbacks array.
Use static memory with a fixed (bigger than usually needed) size.
Since I'm running on a microcontroller platform (Arduino) with limited memory,
neither of these approaches appeal to me. And given that the entire contents of
the array is known at compile time, I'm hoping for a way to let the compiler
also see this.
I've found this and this solution, but those require a custom
linker script, which is not feasible in the compilation environment I'm
running (especially not since this would require explicitely naming each
of these special arrays in the linker script, so just having a single
linker script addition doesn't work here).
This solution is the best I found so far. It uses a linked list
that is filled at runtime, but uses memory allocated statically in each
compile unit seperately (e.g. a next pointer is allocated with each
function pointer). Still, the overhead of these next pointers should not
be required - is there any better approach?
Perhaps having a dynamic solution combined with link-time optimization can
somehow result in a static allocation?
Suggestions on alternative approaches are also welcome, though the required
elements are having a static list of things, and memory efficiency.
Furthermore:
Using C++ is fine, I just used some C code above for illustrating the problem, most Arduino code is C++ anyway.
I'm using gcc / avr-gcc and though I'd prefer a portable solution, something that is gcc only is also ok.
I have template support available, but not STL.
In the Arduino environment that I use, I have not Makefile or other way to easily run some custom code at compiletime, so I'm looking for something that can be entirely implemented in the code.
As commented in some previous answer, the best option is to use a custom linker script (with a KEEP(*(SORT(.whatever.*))) input section).
Anyway, it can be done without modifying the linker scripts (working sample code below), at least at some platforms with gcc (tested on xtensa embedded device and cygwin)
Assumptions:
We want to avoid using RAM as much as possible (embedded)
We do not want the calling module to know anything about the modules with callbacks (it is a lib)
No fixed size for the list (unknown size at library compile time)
I am using GCC. The principle may work on other compilers, but I have not tested it
Callback funtions in this sample receive no arguments, but it is quite simple to modify if needed
How to do it:
We need the linker to somehow allocate at link time an array of pointers to functions
As we do not know the size of the array, we also need the linker to somehow mark the end of the array
This is quite specific, as the right way is using a custom linker script, but it happens to be feasible without doing so if we find a section in the standard linker script that is always "kept" and "sorted".
Normally, this is true for the .ctors.* input sections (the standard requires C++ constructors to be executed in order by function name, and it is implemented like this in standard ld scripts), so we can hack a little and give it a try.
Just take into account that it may not work for all platforms (I have tested it in xtensa embedded architecture and CygWIN, but this is a hacking trick, so...).
Also, as we are putting the pointers in the constructors section, we need to use one byte of RAM (for the whole program) to skip the callback code during C runtime init.
test.c:
A library that registers a module called test, and calls its callbacks at some point
#include "callback.h"
CALLBACK_LIST(test);
void do_something_and_call_the_callbacks(void) {
// ... doing something here ...
CALLBACKS(test);
// ... doing something else ...
}
callme1.c:
Client code registering two callbacks for module test. The generated functions have no name (indeed they do have a name, but it is magically generated to be unique inside the compilation unit)
#include <stdio.h>
#include "callback.h"
CALLBACK(test) {
printf("%s: %s\n", __FILE__, __FUNCTION__);
}
CALLBACK(test) {
printf("%s: %s\n", __FILE__, __FUNCTION__);
}
void callme1(void) {} // stub to be called in the test sample to include the compilation unit. Not needed in real code...
callme2.c:
Client code registering another callback for module test...
#include <stdio.h>
#include "callback.h"
CALLBACK(test) {
printf("%s: %s\n", __FILE__, __FUNCTION__);
}
void callme2(void) {} // stub to be called in the test sample to include the compilation unit. Not needed in real code...
callback.h:
And the magic...
#ifndef __CALLBACK_H__
#define __CALLBACK_H__
#ifdef __cplusplus
extern "C" {
#endif
typedef void (* callback)(void);
int __attribute__((weak)) _callback_ctor_stub = 0;
#ifdef __cplusplus
}
#endif
#define _PASTE(a, b) a ## b
#define PASTE(a, b) _PASTE(a, b)
#define CALLBACK(module) \
static inline void PASTE(_ ## module ## _callback_, __LINE__)(void); \
static void PASTE(_ ## module ## _callback_ctor_, __LINE__)(void); \
static __attribute__((section(".ctors.callback." #module "$2"))) __attribute__((used)) const callback PASTE(__ ## module ## _callback_, __LINE__) = PASTE(_ ## module ## _callback_ctor_, __LINE__); \
static void PASTE(_ ## module ## _callback_ctor_, __LINE__)(void) { \
if(_callback_ctor_stub) PASTE(_ ## module ## _callback_, __LINE__)(); \
} \
inline void PASTE(_ ## module ## _callback_, __LINE__)(void)
#define CALLBACK_LIST(module) \
static __attribute__((section(".ctors.callback." #module "$1"))) const callback _ ## module ## _callbacks_start[0] = {}; \
static __attribute__((section(".ctors.callback." #module "$3"))) const callback _ ## module ## _callbacks_end[0] = {}
#define CALLBACKS(module) do { \
const callback *cb; \
_callback_ctor_stub = 1; \
for(cb = _ ## module ## _callbacks_start ; cb < _ ## module ## _callbacks_end ; cb++) (*cb)(); \
} while(0)
#endif
main.c:
If you want to give it a try... this the entry point for a standalone program (tested and working on gcc-cygwin)
void do_something_and_call_the_callbacks(void);
int main() {
do_something_and_call_the_callbacks();
}
output:
This is the (relevant) output in my embedded device. The function names are generated at callback.h and can have duplicates, as the functions are static
app/callme1.c: _test_callback_8
app/callme1.c: _test_callback_4
app/callme2.c: _test_callback_4
And in CygWIN...
$ gcc -c -o callme1.o callme1.c
$ gcc -c -o callme2.o callme2.c
$ gcc -c -o test.o test.c
$ gcc -c -o main.o main.c
$ gcc -o testme test.o callme1.o callme2.o main.o
$ ./testme
callme1.c: _test_callback_4
callme1.c: _test_callback_8
callme2.c: _test_callback_4
linker map:
This is the relevant part of the map file generated by the linker
*(SORT(.ctors.*))
.ctors.callback.test$1 0x4024f040 0x0 .build/testme.a(test.o)
.ctors.callback.test$2 0x4024f040 0x8 .build/testme.a(callme1.o)
.ctors.callback.test$2 0x4024f048 0x4 .build/testme.a(callme2.o)
.ctors.callback.test$3 0x4024f04c 0x0 .build/testme.a(test.o)
Try to solve the actual problem. What you need are multiple callback functions, that are defined in various modules, that aren't in the slightest related to each other.
What you have done though, is to place a global variable in a header file, which is accessible by every module including that header. This introduces a tight coupling between all such files, even though they are not related to each other. Furthermore, it seems only the callback handler .c function needs to actually call the functions, yet they are exposed to the whole program.
So the actual problem here is the program design and nothing else.
And there is actually no apparent reason why you need to allocate this array at compile time. The only sane reason would be to save RAM, but that's of course is a valid reason for an embedded system. In which case the array should be declared as const and initialized at compile time.
You can keep something similar to your design if storing the array as read-write objects. Or if the array must be a read-only one for the purpose of saving RAM, you must do a drastic re-design.
I'll give both versions, consider which one is most suitable for your case:
RAM-based read/write array
(Advantage: flexible, can be changed in runtime. Disadvantages: RAM consumption. Slight over-head code for initialization. RAM is more exposed to bugs than flash.)
Let the callback.h and callback.c from a module which is only concerned with the handling of the callback functions. This module is responsible for how the callbacks are allocated and when they are executed.
In callback.h define a type for the callback functions. This should be a function pointer type just as you have done. But remove the variable declaration from the .h file.
In callback.c, declare the callback array of functions as
static callback_t callbacks [LARGE_ENOUGH_FOR_WORST_CASE];
There is no way you can avoid "LARGE_ENOUGH_FOR_WORST_CASE". You are on an embedded system with limited RAM, so you have to actually consider what the worst-case scenario is and reserve enough memory for that, no more, no less. On a microcontroller embedded system, there are no such things as "usually needed" nor "lets save some RAM for other processes". Your MCU either has enough memory to cover the worst case scenario, or it does not, in which case no amount of clever allocations will save you.
In callback.c, declare a size variable that keeps track of how much of the callback array that has been initialized. static size_t callback_size;.
Write an init function void callback_init(void) which initializes the callback module. The prototype should be in the .h file and the caller is responsible for executing it once, at program startup.
Inside the init function, set callback_size to 0. The reason I propose to do this in runtime is because you have an embedded system where a .bss segment may not be present or even undesired. You might not even have a copy-down code that initializes all static variables to zero. Such behavior is non-conformant with the C standard but very common in embedded systems. Therefore, never write code which relies on static variables getting automatically initialized to zero.
Write a function void callback_add (callback_t* callback);. Every module that includes your callback module will call this function to add their specific callback functions to the list.
Keep your do_callbacks function as it is (though as a minor remark, consider renaming to callback_traverse, callback_run or similar).
Flash-based read-only array
(Advantages: saves RAM, true read-only memory safe from memory corruption bugs. Disadvantages: less flexible, depends on every module used in the project, possibly slightly slower access because it's in flash.)
In this case, you'll have to turn the whole program upside-down. By the nature of compile-time solutions, it will be a whole lot more "hard-coded".
Instead of having multiple unrelated modules including a callback handler module, you'll have to make the callback handler module include everything else. The individual modules still don't know when a callback will get executed or where it is allocated. They just declare one or several functions as callbacks. The callback module is then responsible for adding every such callback function to its array at compile-time.
// callback.c
#include "timer_module.h"
#include "spi_module.h"
...
static const callback_t CALLBACKS [] =
{
&timer_callback1,
&timer_callback2,
&spi_callback,
...
};
The advantage of this is that you'll automatically get the worst case scenario handed to you by your own program. The size of the array is now known at compile time, it is simply sizeof(CALLBACKS)/sizeof(callback_t).
Of course this isn't nearly as elegant as the generic callback module. You get a tight coupling from the callback module to every other module in the project, but not the other way around. Essentially, the callback.c is a "main()".
You can still use a function pointer typedef in callback.h though, but it is no longer actually needed: the individual modules must ensure that they have their callback functions written in the desired format anyhow, with or without such a type present.
I too am faced with a similar problem:
...need are multiple callback functions, that are defined in various
modules, that aren't in the slightest related to each other.
Mine is C, on Atmel XMega processor. You mentioned that you are using GCC. The following doesn't solve your problem, it is a variant on the above #1 solution. It exploits the __attribute__((weak)) directive.
1) For each optional module, have a unique (per module name) but similar (per purpose) callback function. E.g.
fooModule.c:
void foo_eventCallback(void) {
// do the foo response here
}
barModule.c:
void bar_eventCallback(void) {
// do the bar response here
}
yakModule.c:
void yak_eventCallback(void) {
// do the yak response here
}
2) Have a callback start point that looks something like:
__attribute__((weak)) void foo_eventCallback(void) { }
__attribute__((weak)) void bar_eventCallback(void) { }
__attribute__((weak)) void yak_eventCallback(void) { }
void functionThatExcitesCallback(void) {
foo_eventCallback();
foo_eventCallback();
foo_eventCallback();
}
The __attribute__((weak)) qualifier basically creates a default implementation with an empty body, which the linker will replace with a different variant IF it finds a non-weak variant by the same name. It doesn't make it completely decoupled, unfortunately. But you can at least put this big super-set-of-all-callbacks in one and only one place, and not get into header file hell with it. And then your different compilation units basically replace the subsets of the superset that they want to. I would love it if there was a way to do this with using the same named function in all modules and just have those called based on what's linked, but haven't yet found something that does that.
I am writing a lib that intercepts the calls to malloc and free at runtime by running the program with LD_PRELOAD=mylib myexe.
The calls to malloc and free are intercept fine. My problem is that there's another function in mylib that I also want intercepted when LD_PRELOAD is used and I cannot figure out why it doesn't 'just work' like the calls to malloc and free.
In mylib.c:
void* malloc(size_t s)
{
return doMyMalloc();
}
void free(void* p)
{
doMyFree(p);
}
void otherThing(size_t)
{
doThing();
}
In myexe.cpp:
#include <malloc.h>
extern "C" void otherThing(size_t); // Compile with -Wl,--unresolved-symbols=ignore-all
int main(int argc, char* argv[])
{
void* x = malloc(1000); // Successfully intercepted.
free(x); // Successfully intercepted.
otherThing(1); // Segfault.
}
One way I have managed to get it to work is by doing:
typedef void (*FUNC)(size_t);
FUNC otherThing = NULL;
int main(int argc, char* argv[])
{
otherThing = (FUNC)dlsym(RTLD_NEXT, "otherThing");
otherThing(1); // Successfully calls mylib's otherThing().
}
but I don't want to write all this code; I don't have to do it for malloc and free.
It's ok if the program crashes if the LD_PRELOAD prefix is missing.
I feel like you're applying a single solution (LD_PRELOAD) to solve two different problems. First, you want to patch malloc() and free(). You've got that working--great. Next, you want to have a runtime "plug-in" system where you don't link against any library at build time, but only do so at runtime. This is typically done using dlopen() and dlsym(), and I recommend you use those.
The idea is that you don't want to specify a particular implementation of otherThing() at build time, but you do need to have some implementation at runtime (or you rightly expect to crash). So let's be explicit about it and use dlsym() to resolve the function name at runtime, of course with error detection in case it's not found.
As for where to define otherThing(), it can be in a totally separate file given to dlopen(), or in mylib (in which case pass NULL as the filename to dlopen()).
It's a bit of a gotcha. There are several posts relating to this on the internet, but I'll try to break it down to 'getting it to work'.
If this is under Linux, then what has happened is that the app has been compiled to not be able to use the external symbol. The quickest solution is to add the same compilation flags to the main application as are used in the library i.e. add the -fPIC flag to the compilation of the main application, just like it is done for the library.
Rather than using the -Wl,--unresolved-symbols=ignore-all flag, you should be using the __attribute__ ((weak)) for the function, like:
extern "C" void otherThing(size_t) __attribute__ ((weak);
And checking it for NULL at run-time, which will allow you to determine if it's been set or not.
By compiling the main application in the same way as a .so, you are implicitly allowing it to be used itself as a target for LD_PRELOAD as, by the manual page:
LD_PRELOAD
A list of additional, user-specified, ELF shared libraries to be
loaded before all others. The items of the list can be separated by
spaces or colons. This can be used to selectively override
functions in other shared libraries.
I all,
I have a win32 application and several DLLs that must use a global variable. In each dll I put
extern MYTYPE* myvariable = NULL;
and in the main program I have
MYTYPE* myvariable = NULL;
mavariable = new MYTYPE();
....
now, when the DLLs are loaded myvariable is NULL and I cannot use it. How can I share the instance of the main program with all the DLLs?
You should make some changes in your program. If it is possible you can just move myvariable from the EXE to one from DLL. Then you can continue to use import libraries.
It general you can export functions or data from an EXE, but in the most cases there are less sense to do this. So you can see this very seldom. For example WinWord.exe or Excel.exe do this.
If really need to export frunction or data from EXE and use it in the DLL you should use dynamical binding with respect of GetProcAddress and GetModuleHandle(NULL). You can do such kind of manual binding inside of DllMain. The address of myvariable of the EXE you can save in the local myvariable of the DLL.
maybe you're looking for this: Windows & C++: extern & __declspec(dllimport)