I'm a GStreamer user/programmer but I had never use GLib directly. Recenty I decided to use GLib for building a simple GObject and take advantages of signal implementation. (I'm a Windows programmer)
I have develop a simple static library with the GObject definition, implementation. The main app link statically with this library and link dinamically with other library linked statically with the first one too.
If I call
DummyObj *dummy = (DummtyObj *) g_object_new(DUMMY_OBJ_TYPE, NULL);
from the main app it works, but if inside the dynamic library I try to build a DummyObj instance with the same function, it fails, in the output I can read
cannot register exisiting type ...
g_once_init_leave_ assertion 'initialization_value != 0' failed
g_object_new: assertion 'G_TYPE_IS_OBJECT (object_tye)' failed
Instead, if is the dynamic library the first one to call
DummyObj *dummy = (DummtyObj *) g_object_new(DUMMY_OBJ_TYPE, NULL);
after if the main app call this function it fails with the same error.
Is like if the first context? that initialize the object is the only one that can create instances of this kind of objects.
I'm a little bit confused about this. In GStreamer I can create new plugins in my main app, inside other plugins, dynamic libraries, I have never see these errors
I hope I have explained well, the english is not my native language and I think that the issue is not easy to explain.
Thanks a lot
It seems that first call to g_object_new in every context try to register the TYPE in a hash_table. The first one can register the TYPE but the second always fail with the same error. Looking the code I'm not able to detect why the second call try to register again the type... the function check_type_name_I in gtype.c fails but I don't know why g_type_register_static is call in both cases.
Before glib 2.32 you were required to initialize the thread system (used by the g_once_... family functions) by calling once (and only once) g_thread_init(). Furthermore, before glib 2.36 you had to initialize the type system with g_type_init().
Knowing that g_type_init():
internally calls g_thread_init by itself, protecting from multiple calls by checking g_thread_get_initialized() on glib < 2.32;
resolves to a nop function on glib >= 2.36;
I think that you can solve your issue in a backward compatible way by just calling g_type_init() at startup.
Related
Say I have a function
def pyfunc():
print("ayy lmao")
return 4
and I want to call it in c++
int j = (int)python.pyfunc();
how exactly would I do that?
You might want to have a look into this:https://docs.python.org/2/extending/extending.html
In order to call a Python function from C++, you have to embed Python
in your C++ application. To do this, you have to:
Load the Python DLL. How you do this is system dependent:
LoadLibrary under Windows, dlopen under Unix. If the Python DLL is
in the usual path you use for DLLs (%path% under Windows,
LD_LIBRARY_PATH under Unix), this will happen automatically if you try
calling any function in the Python C interface. Manual loading will
give you more control with regards to version, etc.
Once the library has been loaded, you have to call the function
Py_Initialize() to initialize it. You may want to call
Py_SetProgramName() or Py_SetPythonHome() first to establish the
environment.
Your function is in a module, so you'll have to load that:
PyImport_ImportModule. If the module isn't in the standard path,
you'll have to add its location to sys.path: use
PyImport_ImportModule to get the module "sys", then
PyObject_GetAttrString to get the attribute "path". The path
attribute is a list, so you can use any of the list functions to add
whatever is needed to it.
Your function is an attribute of the module, so you use
PyObject_GetAttrString on the module to get an instance of the
function. Once you've got that, you pack the arguments into a tuple or
a dictionary (for keyword arguments), and use PyObject_Call to call
it.
All of the functions, and everything that is necessary, is documented
(extremely well, in fact) in https://docs.python.org/2/c-api/. You'll
be particularly interested in the sections on "Embedding Python" and
"Importing Modules", along with the more general utilities ("Object
Protocol", etc.). You'll also need to understand the general principles
with regards to how the Python/C API works—things like reference
counting and borrowed vs. owned references; you'll probably want to read
all of the sections in the Introduction first.
And of course, despite the overall quality of the documentation, it's
not perfect. A couple of times, I've had to plunge into the Python
sources to figure out what was going on. (Typically, when I'm getting
an error back from Python, to find out what it's actually complaining
about.)
Out of curiosity, what exactly happens when an application compiled with the MSVCR is loaded, resp. how does the loader of Windows actually initialize the CRT? For what I have gathered so far, when the program as well as all the imported libraries are loaded into memory and all relocations are done, the CRT startup code (_CRT_INIT()?) initializes all global initializers in the .CRT$XC* sections and calls the user defined main() function. I hope this is correct so far.
But let's assume, for the sake of explanation, a program that is not using the MSVCR (e.g. an application built with Cygwin GCC or other compilers) tries to load a library at runtime, requiring the CRT, using a custom loader/runtime linker, so no LoadLibrary() involved. How would the loader/linker has to handle CRT initialization? Would it have to manually initialize all "objects" in said sections, does it have to do something else to make the internal wiring of the library work properly, or would it have to just call _CRT_INIT() (which unpractically is defined in the runtime itself and not exported anywhere as far as I am aware). Would this mix-up even work in any way, assuming that the non-CRT application and the CRT-library would not pass any objects, exceptions and things the like between them?
I would be very interested in finding out, because I can't quite make out what the CRT has an effect on the actual loading process...
Any information is very appreciated, thanks!
The entrypoint for an executable image is selected with the /ENTRY linker option. The defaults it uses are documented well in the MSDN Library article. They are the CRT entrypoint.
If you want to replace the CRT then either pick the same name or use the /ENTRY option explicitly when you link. You'll also need /NODEFAULTLIB to prevent it from linking the regular .lib
Each library compiled against the C++ runtime is calling _DllMainCRTStartup when it's loaded. _DllMainCRTStartup calls _CRT_INIT, which initializes the C/C++ run-time library and invokes C++ constructors on static, non-local variables.
The PE format contains an optional header that has a slot called 'addressofentrypoint', this slot calls a function that will call _DllMainCRTStartup which fires the initialization chain.
after _DllMainCRTStartup finishes the initialization phase it will call your own implemented DllMain() function.
When you learn about programming, someone will tell you that "the first thing that happens is that the code runs in main. But that's a bit like when you learn about atoms in School, they are fairly well organized and operate acorrding to strict rules. If you later go to a Nuclear/Particle Physics class at university, those simple/strict rules are much more detailed and don't always apply, etc.
When you link a C or C++ program the CRT contains some code something like this:
start()
{
CRT_init();
...
Global_Object_Constructors();
...
exit(main());
}
So the initialization is done by the C runtime library itself, BEFORE it calls your main.
A DLL has a DllMain that is executed by LoadLibrary() - this is responsible for initializing/creating global objects in the DLL, and if you don't use LoadLibrary() [e.g. loading the DLL into memory yourself] then you would have to ensure that objects are created and initialized.
We're working on a relatively large-scale C++ project where we chose at the very beginning to use protobuf as our Lingua Franca for stored and transmitted data.
We had our first problem because of end-of-program memory leaks due to the protobuf generated classes metadata that are stored as static pointer, allocated during the first call to the constructor and never deallocated. We found a nice function provide by Mr. Google to do this clean-up:
google::protobuf::ShutdownProtobufLibrary();
Works fine except there is no symmetric call, so once it's done you can no longer use anything . You have to do that exactly one time in your executable. We did what any lazy developper would have done:
struct LIBPROTOBUF_EXPORT Resource
{
~Resource()
{
google::protobuf::ShutdownProtobufLibrary();
}
};
bool registerShutdownAtExit()
{
static Resource cleaner;
return true;
}
And we added in the protobuf generation of cc files a:
static bool protobufResource = mlv::protobuf::registerShutdownAtExit();
It worked fine for several months.
Then we added the support for dynamically loadable plugins (dlls) in our tool. Some of them using protobuf. Unloading of the plugins worked fine, but when more than one of them used protobuf, we had a nice little crash when unloading the last one.
The reason: the last to unload would destroy the cleaner instance, itself trying to google::protobuf::ShutdownProtobufLibrary(), itself trying to destroy metadata of unloaded types... CRASH.
Long story short: are we condemned to either have lots of "normal" memory leaks or a crash when closing our tool. Does anyone have experienced the same problem and found a better solution? Is my diagnosis bad?
Like johnathon suggested in his comment, use a reference counting scheme, or register your destruction routine with atexit. Such a routine is free-standing, but that could work fine for your case.
Relevant documentation:
MSDN
POSIX
Edit: You're right, it's basically the same thing. Didn't think this through.
Another suggestion: Use a global resource singleton for all protobuf-using plugins. This one has a global destructor, which is only registered when a plugin first uses the protobuf library. Or just set a flag whenver it's used, then call ShutdownProtobufLibrary only if the flag is set.
Unfortunately I can't post the source code for this, but I will try to set it up as best I can.
I have a case where dynamic_cast fails to cast to a derived class type, and I know it should succeed (ie, I know the actual type of the instance).
Also typeid for a heap allocated object doesn't equal the typeid for a stack allocated object!! IE,
Foo mstack;
Foo*mheap = new Foo();
typeid(mstack) == typeid(*mheap); // returns FALSE!?
So there is clearly a RTTI problem somewhere. The class implementation (for both base and derived classes) is in one shared library, the malfunctioning code is in a second shared library which is loaded as a Python module in the Python interpreter (all on linux, same problem when using either gcc 4 or Intel C++ compiler). If I write a simple little test executable that links both shared libraries, everything works fine. I've tried --export-dynamic when linking the shared libraries without success (looks like it's intended for use with executables).
Anybody have any pointers for where to look? Is there something particular about the way Python uses dlopen() that causes this kind of problem?
This is caused by Python loading the extension module with RTLD_LOCAL, and the solution is to force Python to load it with RTLD_GLOBAL instead (see OP's comment).
Sooooo I'm writing a script interpreter. And basically, I want some classes and functions stored in a DLL, but I want the DLL to look for functions within the programs that are linking to it, like,
program dll
----------------------------------------------------
send code to dll-----> parse code
|
v
code contains a function,
that isn't contained in the DLL
|
list of functions in <------/
program
|
v
corresponding function,
user-defined in the
program--process the
passed argument here
|
\--------------> return value sent back
to the parsing function
I was wondering basically, how do I compile a DLL with gcc? Well, I'm using a windows port of gcc. Once I compile a .dll containing my classes and functions, how do I link to it with my program? How do I use the classes and functions in the DLL? Can the DLL call functions from the program linking to it? If I make a class { ... } object; in the DLL, then when the DLL is loaded by the program, will object be available to the program? Thanks in advance, I really need to know how to work with DLLs in C++ before I can continue with this project.
"Can you add more detail as to why you want the DLL to call functions in the main program?"
I thought the diagram sort of explained it... the program using the DLL passes a piece of code to the DLL, which parses the code, and if function calls are found in said code then corresponding functions within the DLL are called... for example, if I passed "a = sqrt(100)" then the DLL parser function would find the function call to sqrt(), and within the DLL would be a corresponding sqrt() function which would calculate the square root of the argument passed to it, and then it would take the return value from that function and put it into variable a... just like any other program, but if a corresponding handler for the sqrt() function isn't found within the DLL (there would be a list of natively supported functions) then it would call a similar function which would reside within the program using the DLL to see if there are any user-defined functions by that name.
So, say you loaded the DLL into the program giving your program the ability to interpret scripts of this particular language, the program could call the DLLs to process single lines of code or hand it filenames of scripts to process... but if you want to add a command into the script which suits the purpose of your program, you could say set a boolean value in the DLL telling it that you are adding functions to its language and then create a function in your code which would list the functions you are adding (the DLL would call it with the name of the function it wants, if that function is a user-defined one contained within your code, the function would call the corresponding function with the argument passed to it by the DLL, the return the return value of the user-defined function back to the DLL, and if it didn't exist, it would return an error code or NULL or something). I'm starting to see that I'll have to find another way around this to make the function calls go one way only
This link explains how to do it in a basic way.
In a big picture view, when you make a dll, you are making a library which is loaded at runtime. It contains a number of symbols which are exported. These symbols are typically references to methods or functions, plus compiler/linker goo.
When you normally build a static library, there is a minimum of goo and the linker pulls in the code it needs and repackages it for you in your executable.
In a dll, you actually get two end products (three really- just wait): a dll and a stub library. The stub is a static library that looks exactly like your regular static library, except that instead of executing your code, each stub is typically a jump instruction to a common routine. The common routine loads your dll, gets the address of the routine that you want to call, then patches up the original jump instruction to go there so when you call it again, you end up in your dll.
The third end product is usually a header file that tells you all about the data types in your library.
So your steps are: create your headers and code, build a dll, build a stub library from the headers/code/some list of exported functions. End code will link to the stub library which will load up the dll and fix up the jump table.
Compiler/linker goo includes things like making sure the runtime libraries are where they're needed, making sure that static constructors are executed, making sure that static destructors are registered for later execution, etc, etc, etc.
Now as to your main problem: how do I write extensible code in a dll? There are a number of possible ways - a typical way is to define a pure abstract class (aka interface) that defines a behavior and either pass that in to a processing routine or to create a routine for registering interfaces to do work, then the processing routine asks the registrar for an object to handle a piece of work for it.
On the detail of what you plan to solve, perhaps you should look at an extendible parser like lua instead of building your own.
To your more specific focus.
A DLL is (typically?) meant to be complete in and of itself, or explicitly know what other libraries to use to complete itself.
What I mean by that is, you cannot have a method implicitly provided by the calling application to complete the DLLs functionality.
You could however make part of your API the provision of methods from a calling app, thus making the DLL fully contained and the passing of knowledge explicit.
How do I use the classes and functions in the DLL?
Include the headers in your code, when the module (exe or another dll) is linked the dlls are checked for completness.
Can the DLL call functions from the program linking to it?
Yes, but it has to be told about them at run time.
If I make a class { ... } object; in the DLL, then when the DLL is loaded by the program, will object be available to the program?
Yes it will be available, however there are some restrictions you need to be aware about. Such as in the area of memory management it is important to either:
Link all modules sharing memory with the same memory management dll (typically c runtime)
Ensure that the memory is allocated and dealloccated only in the same module.
allocate on the stack
Examples!
Here is a basic idea of passing functions to the dll, however in your case may not be most helpfull as you need to know up front what other functions you want provided.
// parser.h
struct functions {
void *fred (int );
};
parse( string, functions );
// program.cpp
parse( "a = sqrt(); fred(a);", functions );
What you need is a way of registering functions(and their details with the dll.)
The bigger problem here is the details bit. But skipping over that you might do something like wxWidgets does with class registration. When method_fred is contructed by your app it will call the constructor and register with the dll through usage off methodInfo. Parser can lookup methodInfo for methods available.
// parser.h
class method_base { };
class methodInfo {
static void register(factory);
static map<string,factory> m_methods;
}
// program.cpp
class method_fred : public method_base {
static method* factory(string args);
static methodInfo _methoinfo;
}
methodInfo method_fred::_methoinfo("fred",method_fred::factory);
This sounds like a job for data structures.
Create a struct containing your keywords and the function associated with each one.
struct keyword {
const char *keyword;
int (*f)(int arg);
};
struct keyword keywords[max_keywords] = {
"db_connect", &db_connect,
}
Then write a function in your DLL that you pass the address of this array to:
plugin_register(keywords);
Then inside the DLL it can do:
keywords[0].f = &plugin_db_connect;
With this method, the code to handle script keywords remains in the main program while the DLL manipulates the data structures to get its own functions called.
Taking it to C++, make the struct a class instead that contains a std::vector or std::map or whatever of keywords and some functions to manipulate them.
Winrawr, before you go on, read this first:
Any improvements on the GCC/Windows DLLs/C++ STL front?
Basically, you may run into problems when passing STL strings around your DLLs, and you may also have trouble with exceptions flying across DLL boundaries, although it's not something I have experienced (yet).
You could always load the dll at runtime with load library