The way I understand it native addons for node are just shared objects or dynamically linked libraries on the operating system. Normally though, dynamically linked libraries are linked at load time. But in node you can require() modules dynamically. So how does this work? Is it more like dynamic loading with dlopen and the function pointers?
nodejs supports addons that are written in other languages (such as C/C++). As such, those other languages can use all OS facilities to dynamically load other DLLs, either system DLLs or their own DLLs.
If you want to read more about the nodejs add-on process, you can see here: Nodejs Addons.
Built in library support (such as the fs module) has the same ability.
Indeed native addons for node are dynamically loaded. Taken from this post:
C/C++ addons are just dynamic libraries, and Node loads those libraries with a function in libuv called uv_dlopen. That provides a uniform interface for loading dynamic libraries on both Windows and Unix-like systems.
Related
I'm currently developing a shared library on Windows (dll) with c++.
The library depends on another external library.
What is the best way to link them together?
Link the external library as a static library into my shared lib?
Link it as a shared library and provide the dll to the application who is using my shared lib?
For the second case what happens if i create an application which uses my own created shared library and also the external library as a shared library?
If for example my shared library is build with the external library version 1.1 and the application uses the newer Version for example 1.3 ? Now the dlls should be different but how could i provide them to the main application?
Are there some best practices or recommendations on how to handle such a situation?
This really depends on what you want to do and how you are deploying your library.
shared/dynamic libraries (dll's on windows) have several advantages over static libraries
They can be distributed externally from the application, allowing your application binary to be smaller
They can be updated externally to the application
They are slightly more efficient as code is only executed if its needed rather than being bundled with the executable
Of course they have several weaknesses too
They can be distributed externally from your application and updated externally - allowing dll injection attacks
Trying to release dynamic libraries with the executable is one of the most painful and horrible things to do (especially on windows where there is no RPATH)
You may have to use a dll, dependent on your external libraries licensing, Qt for example requires shared library linking in many cases (not all).
Standard convention is usually to offer shared and static versions of your library with the shared version being completely linked to other shared libraries and the static version being an ar static library (includes all dependencies). The shared library variant then offers instructions on linking (i.e a .pc (pkgconfig) file) which specifies the versions of the other libraries to link to (i.e v1.1 of x.dll) in order to successfully compile/link.
TL:DR; Static/Dynamic library and data sharing in UE4.
So all libraries are static or dynamic but no "mixed" solution?
Long description:
I'm browsing the UE4 source code and found that they are using a modular system. So they're building libraries (static or dynamic, based on the IS_MONOLITHIC define) and they are linked based on the build configuration files (like Core.Build.cs).
Let's read the "OpenGLDrv" module's Build file. It depends on the Core and Engine (and so on) libraries, so I guess they are linked to the OpenGLDrv library.
And checking the Launch module, in the Build file they set up dynamic module dependencies for the D3DXX and the OpenGLDrv libraries.
And here comes my problems:
If they are dynamically loaded modules, they have to be dynamic libraries (dll in Windows). If it's not true, are they linked statically?
If linked statically, how are they handling the global/static data sharing? Both the Launch and OpenGLDrv depends on the Core module which is linked statically to both libraries.
If the OpenGLDrv is a dynamic library and the Launch and Core are not, the data sharing problem still exists.
More specifically
I've tried to implement a similar system. I'm using visual studio and my current target platform is Windows only (but I'm writing the code in a platform-independent way).
Currently I have 5 following modules:
Core: It has the really core things like math classes, logger, misc, and the module manager.
Modules: The base classes for the "implementable" modules: InputDevice, GraphicsDevice and so on. The Core is linked to this.
OGLDevice: This supposed to be a dynamic library. It implements the OpenGL-specific GraphicsDevice. Both the Core and Modules are linked to it.
Engine: This is the "main library" which. Both the Core and Modules are linked to this library. It should load the OGLDevice dynamically.
Application: The main executable. The Engine is linked to this.
Here is a pretty cool image drawn by myself in Paint:
The best design-image ever
If all the 4 libraries are static, there aren't any problem with the dependencies. However, this way the OGLDevice is not a dynamically loaded library, because it is linked to the Engine. Maybe the device itself is not instantiated but still linked to the project.
If the Core, Modules and Engine libraries are static but the OGLDevice is dynamic, data sharing problems comes up. First, the Core is "loaded" by the Engine (because it's statically linked to it) but when the Engine loads the OGLDevice library, it will also "load" its own Core. So every static and global are duplicated.
If all modules would be dynamic, I guess it should work. Is there any way to mix the static and dynamic library thing?
Or how is the UE4 doing this?
Is it possible to build shared libraries (e.g. *.so, *.dll) using languages other than C or C++?
What is the underlying requirement to build a shared library? Is it that the language be capable of compiling to a native binary?
I'm particularly interested to find out if it is possible to build a shared library from Prolog.
E.g. could I build a .so using Prolog, which I could then link to from another language, C, Java, Python etc.
You can build a shared library with a number of languages. One I use regularly is Delphi, which can create dll's for windows.
For Prolog, not sure if you can create a shared library, it would need some sort of run time environment, if any did I'd start with GNU-Prolog. The problem isn't technical so much as licensing.
There are prologs around that can be used as a shared library, and linked in with your app, written in anything that can include a shared library (most languages). A quick google turned up these http://www.swi-prolog.org/FAQ/, http://www.gprolog.org/#platform
I am creating an application that supports modules in the form of dlls that are loaded dynamically at runtime. The code is laid out in the following way:
core - static library
This has a mechanism to load shared libraries and call a "create" function that returns a new Module object (uses a shared header).
module shared library (linked against core static library)
This module uses the shared Module header and also uses other classes from the core library (hence why it is linked against the core library). It is built to include all symbols from static libraries.
test application executable (linked against core static library)
I am getting funky, and seemingly sporadic behavior. They always end up in access violations but it seems that member variables that I very explicitly set (integers) will print out in later functions as garbage (i have verified that they are not being deleted earlier). This only ever seems to happen if they dynamic library is loaded (even if I never call the create function).
My main question is, is there are danger here that the symbols in the shared library will conflict with the symbols in the executable (since they came from the same static library) and cause problems even though they are from the exact same static library?
I can't speak for Linux and OS X behavior, but on Windows, the following is exactly what is happening. Since you say you also want to compile on Windows, this is relevant.
The problem you are experiencing is that you actually have multiple versions of everything in the core. Each module and the application itself has its own copy of the core, and their variables are not shared. This includes the C runtime, so things like new/delete across module boundaries are fraught with peril.
To verify that this is what is happening, create a simple test: set a global in the core to a value in your test application, then from from your dynamically loaded code try to access that global and see what you get. I will wager that you will see that your store to the global will not be reflected!
Solutions:
1) Make core a shared dynamic library. This may or may not be an option for you.
2) Operate extremely carefully with the knowledge of the above; All CRT and/or your own core state will not be shared, so you must make sure things will be allocated/destroyed on their own side of the module boundaries, among other things.
My own application is designed almost identically to yours; ie a static library with shared code needed by both the application and the modules, and then dynamically loaded plugins loaded by the application core.
What I do for all shared core state that must be accessed across modules is that the first thing each module does after loading is have its "core pointer" set to an instantiation of the core libraries in the application. This ensures that all modules are working with the same data.
I am building a Java application that uses a shared library written in C++ and compiled for different operating systems. The problem is, that this shared library itself depends on an additional library it normally finds under the appropriate environment variable (PATH, LIBRARY_PATH or LD_LIBRARY_PATH).
I can - but don't want to - set these environment variables. I'd rather load the needed shared libraries from a given path at runtime - just like a plugin. And no - I don't want any starter application that starts a new process with a new environment. Does anybody know how to achieve this?
I know that this must be possible, as one of the libraries I use is capable of loading its plugins from a given path. Of course I'd prefer platform independent code, but if this ain't possible, seperate solutions for Windows, Linux and MacOS would also do it.
EDIT
I should have mentioned that the shared library I'd wish to use is object oriented, which means that a binding of single functions won't do it.
Un UNIX/Linux systems you can use dlopen. The issue then is you have to fetch all symbols you need via dlsym
Simple example:
typedef int (*some_func)(char *param);
void *myso = dlopen("/path/to/my.so", RTLD_NOW);
some_func *func = dlsym(myso, "function_name_to_fetch");
func("foo");
dlclose(myso);
Will load the .so and execute function_name_to_fetch() from in there. See the man page dlopen(1) for more.
On Windows, you can use LoadLibrary, and on Linux, dlopen. The APIs are extremely similar and can load a so/dll directly by providing the full path. That works if it is a run-time dependency (after loading, you "link" by calling GetProcAddress/dlsym.)
I concur with the other posters about the use of dlopen and LoadLibrary. The libltdl gives you a platform-independent interface to these functions.
I do not think you can do it for it.
Most Dlls have some sort of init() function that must be called after it have been loaded, and sometime that init() function needs some parameters and returns some handle to be used to call the dll's functions. Do you know the definition of the additional library?
Then, the first library can not simply look if the DLL X is in RAM only by using its name. The one it needs can be in a different directory or a different build/version. The OS will recognize the library if the full path is the same than another one already loaded and it will share it instead of loading it a second time.
The other library can load its plugins from another path because it written to not depend on PATH and they are his own plugins.
Have you try to update the process's environment variables from the code before loading the Dll? That will not depends on a starter process.