C++ DLLs: how to create the corresponding include headers? - c++

Problem
I have a C++ project and create a library A from it. If I now link another project B with this library A, I of course also have to provide an include path for A's headers, so I just use A's source folder. But A's headers contain symbols that aren't exported. I feel like this is not the correct way to do it, but don't know better. A specific thing that makes me feel like this is incorrect is that my IDE suggests the symbols that aren't exported.
I'd guess the solution would be to create an include folder besides the source folder where the same headers are in but only with the exported symbols. So at build-time, every symbol with PROJECTAPI should be automatically copied over to the corresponding headers in the include folder. But if I google, I don't find such a function for e.g. cmake.
So what would be the recommended way here? Is there a functionality to create such an include folder?
Example
example.cpp of project B
#include <A/main.hpp>
int main() {
ex::World w("Earth");
w.say_hello();
//IDE wouldn't see this as error: w.private();
}
main.cpp of project A
#include <iostream>
#include "main.hpp"
namespace ex {
void World::say_hello() {
std::cout << "Hello, World from " << m_name << std::endl;
}
World::World(std::string name)
: m_name(name)
{}
void World::hidden() {
std::cout << "Not exported" << std::endl;
}
}
main.hpp of project A
#include <string>
#ifndef PROJECTAPI
# ifdef example_EXPORTS
# define PROJECTAPI __declspec(dllexport)
# else
# define PROJECTAPI __declspec(dllimport)
# endif
#endif
namespace ex {
class World {
private:
std::string m_name;
public:
void PROJECTAPI say_hello();
PROJECTAPI World(std::string name);
void hidden();
};
}
Edit: private isn't a good method name

You are looking for the PIMPL idiom. "PIMPL" is short for "Pointer to IMPLementation". The idea is that, at the cost of a pointer indirection, you hide the implementation data and private methods in an inner class whose definition is opaque to the API consumer.
This approach is especially effective if you need to provide ABI stability.
Herb Sutter has a great GOTW on this here: https://herbsutter.com/gotw/_100/
Here is a full example that's close-ish to your code:
$ tree
.
├── CMakeLists.txt
├── include
│   └── world.h
├── main.cpp
└── src
├── world.cpp
└── world_priv.h
In ./include/world.h (the public header)
#ifndef WORLD_H
#define WORLD_H
#include <memory>
#include <string>
#include "world_export.h"
namespace ex {
class World {
public:
WORLD_EXPORT World(std::string name);
WORLD_EXPORT ~World() /* = default */;
void WORLD_EXPORT say_hello();
private:
class Impl;
std::unique_ptr<Impl> pImpl;
};
} // namespace ex
#endif
In ./src/world_priv.h:
#ifndef WORLD_PRIV_H
#define WORLD_PRIV_H
#include "world.h"
namespace ex {
class World::Impl {
public:
Impl(std::string name) : name(std::move(name)) {}
void say_hello();
void hidden();
private:
std::string name;
};
} // namespace ex
#endif
In ./src/world.cpp:
#include <iostream>
#include "world_priv.h"
namespace ex {
World::World(std::string name)
: pImpl(std::make_unique<Impl>(std::move(name))) {}
World::~World() = default;
void World::say_hello() { pImpl->say_hello(); }
void World::Impl::say_hello() {
std::cout << "Hello, World from " << name << "\n";
}
void World::Impl::hidden() { std::cout << "Not exported" << std::endl; }
} // namespace ex
In main.cpp:
#include <world.h>
int main() {
ex::World w("Earth");
w.say_hello();
}
Finally, here's the build:
cmake_minimum_required(VERSION 3.21)
project(pimpl_example)
option(BUILD_SHARED_LIBS "Build world as shared rather than static" ON)
# Library
include(GenerateExportHeader)
set(CMAKE_CXX_VISIBILITY_PRESET hidden)
set(CMAKE_VISIBILITY_INLINES_HIDDEN 1)
add_library(world src/world.cpp src/world_priv.h include/world.h)
add_library(world::world ALIAS world)
target_include_directories(
world PRIVATE "$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/src>"
PUBLIC "$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>"
)
generate_export_header(world EXPORT_FILE_NAME include/world_export.h)
target_compile_definitions(
world PUBLIC "$<$<NOT:$<BOOL:${BUILD_SHARED_LIBS}>>:WORLD_STATIC_DEFINE>")
target_include_directories(
world PUBLIC "$<BUILD_INTERFACE:${CMAKE_CURRENT_BINARY_DIR}/include>")
# Application
add_executable(app main.cpp)
target_link_libraries(app PRIVATE world::world)
This doesn't include install rules or anything, but it's ready for those to be written.
Building it:
$ cmake -G Ninja -S . -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo
...
$ cmake --build build
$ ./build/app
Hello, World from Earth
$ $ nm ./build/libworld.so | c++filt | grep ' T ' | uniq
0000000000001460 T ex::World::say_hello()
00000000000012e0 T ex::World::World(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)
00000000000013c0 T ex::World::~World()
You can see that only the API of ex::World is exported by the library. All the private details are hidden both in the code and in the library itself.
On Windows:
>dumpbin /EXPORTS build\world.dll
Microsoft (R) COFF/PE Dumper Version 14.28.29915.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file build\world.dll
File Type: DLL
Section contains the following exports for world.dll
00000000 characteristics
FFFFFFFF time date stamp
0.00 version
1 ordinal base
3 number of functions
3 number of names
ordinal hint RVA name
1 0 00001390 ??0World#ex##QEAA#V?$basic_string#DU?$char_traits#D#std##V?$allocator#D#2##std###Z
2 1 00001550 ??1World#ex##QEAA#XZ
3 2 000015D0 ?say_hello#World#ex##QEAAXXZ
Summary
1000 .data
1000 .pdata
2000 .rdata
1000 .reloc
1000 .rsrc
2000 .text

Normally, a simple way to tackle that would be to create separate public and private headers ahead of time, and only expose the public ones to the user.
Here's a simple project structure that would accomplish that:
- lib_a
- include
- main.hpp
- src
- main_private.hpp
- main.cpp
Now, obviously, that won't work for the code you posted, since the declarations you want to separate belong to the same class. But that's just a symptom of the fact that what you are trying to do is unfortunately not allowed.
From the standard basic.def.odr:
There can be more than one definition of a
(13.1) class type ([class]),
[...]
in a program provided that each definition appears in a different translation unit and the definitions satisfy the following requirements.
[...]
Each such definition shall consist of the same sequence of tokens [...]
In other words, if you put a class in a public header, it has to be identical to the one that was used when compiling the library.
As much as it would be convenient, putting "half-a-class" in a public header is just not allowed.

Related

Dynamically linking a shared library from a pybind11-wrapped code

I am trying to add python bindings to a medium-sized C++ scientific code (some tens of thousands LOCs). I have managed to make it work without too many issues, but I have now incurred in an issue which I am incapable of solving myself. The code is organized as follows:
All the classes and data structures are compiled in a library libcommon.a
Executables are created by linking this library
pybind11 is used to create a core.so python module
The bindings for the "main" parts work fine. Indeed, simulations launched from the standalone code or from python give the exact same results.
However, the code also supports a plugin-like system which can load shared libraries at runtime. These shared libraries contain classes that inherit from interfaces defined in the main code. It turns out that if I try to link these shared libraries from python I get the infamous "undefined symbol" errors. I have checked that these symbols are in the core.so module (using nm -D). In fact, simulations that perform the dynamic linking with the standalone code works perfectly (within the same folder and with the same input). Somehow, the shared lib cannot find the right symbols when called through python, but it has no issues when loaded by the standalone code. I am using CMake to build the system.
What follows is a MCE. Copy each file in a folder, copy (or link) the pybind11 folder in the same place and use the following commands:
mkdir build
cd build
cmake ..
make
which will generate a standalone binary and a python module. The standalone executable will produce the correct output. By contrast, using the following commands in python3 (that, at least in my head, should be equivalent) yields an error:
import core
b = core.load_plugin()
main.cpp
#include "Base.h"
#include "plugin_loader.h"
#include <iostream>
int main() {
Base *d = load_plugin();
if(d == NULL) {
std::cerr << "No lib found" << std::endl;
return 1;
}
d->foo();
return 0;
}
Base.h
#ifndef BASE
#define BASE
struct Base {
Base();
virtual ~Base();
virtual void foo();
};
#endif
Base.cpp
#include "Base.h"
#include <iostream>
Base::Base() {}
Base::~Base() {}
void Base::foo() {
std::cout << "Hey, it's Base!" << std::endl;
}
plugin_loader.h
#ifndef LOADER
#define LOADER
#include "Base.h"
Base *load_plugin();
#endif
plugin_loader.cpp
#include "plugin_loader.h"
#include <dlfcn.h>
#include <iostream>
typedef Base* make_base();
Base *load_plugin() {
void *handle = dlopen("./Derived.so", RTLD_LAZY | RTLD_GLOBAL);
const char *dl_error = dlerror();
if(dl_error != nullptr) {
std::cerr << "Caught an error while opening shared library: " << dl_error << std::endl;
return NULL;
}
make_base *entry = (make_base *) dlsym(handle, "make");
return (Base *) entry();
}
Derived.h
#include "Base.h"
struct Derived : public Base {
Derived();
virtual ~Derived();
void foo() override;
};
extern "C" Base *make() {
return new Derived();
}
Derived.cpp
#include "Derived.h"
#include <iostream>
Derived::Derived() {}
Derived::~Derived() {}
void Derived::foo() {
std::cout << "Hey, it's Derived!" << std::endl;
}
bindings.cpp
#include <pybind11/pybind11.h>
#include "Base.h"
#include "plugin_loader.h"
PYBIND11_MODULE(core, m) {
pybind11::class_<Base, std::shared_ptr<Base>> base(m, "Base");
base.def(pybind11::init<>());
base.def("foo", &Base::foo);
m.def("load_plugin", &load_plugin);
}
CMakeLists.txt
PROJECT(foobar)
# compile the library
ADD_LIBRARY(common SHARED Base.cpp plugin_loader.cpp)
TARGET_LINK_LIBRARIES(common ${CMAKE_DL_LIBS})
SET_TARGET_PROPERTIES(common PROPERTIES POSITION_INDEPENDENT_CODE ON)
# compile the standalone code
ADD_EXECUTABLE(standalone main.cpp)
TARGET_LINK_LIBRARIES(standalone common)
# compile the "plugin"
SET(CMAKE_SHARED_LIBRARY_PREFIX "")
ADD_LIBRARY(Derived SHARED Derived.cpp)
# compile the bindings
ADD_SUBDIRECTORY(pybind11)
INCLUDE_DIRECTORIES( ${PROJECT_SOURCE_DIR}/pybind11/include )
FIND_PACKAGE( PythonLibs 3 REQUIRED )
INCLUDE_DIRECTORIES( ${PYTHON_INCLUDE_DIRS} )
ADD_LIBRARY(_oxpy_lib STATIC bindings.cpp)
TARGET_LINK_LIBRARIES(_oxpy_lib ${PYTHON_LIBRARIES} common)
SET_TARGET_PROPERTIES(_oxpy_lib PROPERTIES POSITION_INDEPENDENT_CODE ON)
pybind11_add_module(core SHARED bindings.cpp)
TARGET_LINK_LIBRARIES(core PRIVATE _oxpy_lib)
You are right, symbols from imported library are not visible because core loaded without RTLD_GLOBAL flag set. You can fix that with a couple of extra lines on python side:
import sys, os
sys.setdlopenflags(os.RTLD_GLOBAL | os.RTLD_LAZY)
import core
b = core.load_plugin()
From sys.setdlopenflags() doc:
To share symbols across extension modules, call as sys.setdlopenflags(os.RTLD_GLOBAL). Symbolic names for the flag values can be found in the os module (RTLD_xxx constants, e.g. os.RTLD_LAZY).

How to make CPP files visible to linker in ESP8266 Eclipse project

There is a "hello world" project in Eclipse IDE that is supposed to compile against ESP8266 RTOS SDK.
File structure is as follows
I added one C++ class to it and put it into its own folder. Here is the class header
#ifndef MAIN_BLINKER_BLINKER_H_
#define MAIN_BLINKER_BLINKER_H_
class Blinker {
public:
Blinker( int period );
int Period() const;
private:
int period_;
};
#endif /* MAIN_BLINKER_BLINKER_H_ */
and the definitions
#include "Blinker.h"
Blinker::Blinker( int period ) :
period_( period )
{}
int Blinker::Period() const {
return this->period_;
}
Main.cpp file is like this
#include <stdio.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "blinker/Blinker.h"
extern "C" {
void app_main()
{
auto blnk = Blinker( 3000 );
int i = 0;
while ( 1 ) {
printf( "[%d] Hello beautiful world!\n", i );
i++;
vTaskDelay( blnk.Period() / portTICK_PERIOD_MS );
}
}
}
It compiles but fails at final stage because the linker (or what is supposed to be a linker in xtensa toolchain) does not see definitions of Blinker methods. This is what I get in the build log
If I put class files next to main.cpp file, the build succeeds. However with time there will be hundreds of files, and without any grouping it will quickly turn into an unmanageable mess.
Alternatively I could put this class into top-level components folder and equip it with empty component.mk file. This would also make the build system happy, however it would force me to use ugly header includes like ../components/blinker/Blinker.h, which I would like to avoid.
So the question is how to make build system aware of .c and .cpp files residing in subfolders of main folder?
you can set COMPONENT_SRCDIRS in "main" component.mk file
see:
https://docs.espressif.com/projects/esp8266-rtos-sdk/en/latest/api-guides/build-system.html#example-component-makefiles
Try to add blinker/Blinker.cpp to your CMakeLists.txt.
Take a look at
How to add new source file in CMakeLists.txt?

CMake link not subfolder

I’m new to cmake.
I want to create code to create instances of some classes (like ClassA) and collect them in a handler class. For this i have created a template class Creator.
In each class implementation a instance of this class is created with Creator class. (see ClassA.cpp line 8)
I have following folder structure
├── CMakeLists.txt
├── main.cpp
└── SubFolder
├── ClassA.cpp
├── ClassA.h
├── CMakeLists.txt
└── Creator.h
./main.cpp
#include <iostream>
#include "SubFolder/ClassA.h"
int main(int argc, char **argv) {
//classA a;
std::cout << std::endl << "Hello, world!" << std::endl;
return 0;
}
./CMakeLists.txt
cmake_minimum_required(VERSION 2.8)
project(teststaticcmake)
add_executable(teststaticcmake main.cpp)
add_subdirectory(SubFolder)
target_link_libraries(teststaticcmake SubFolder)
install(TARGETS teststaticcmake RUNTIME DESTINATION bin)
SubFolder/ClassA.h
#ifndef __CLASSA__
#define __CLASSA__
class classA
{
public:
classA();
};
#endif //__CLASSA__
SubFolder/ClassA.cpp
#include "ClassA.h"
#include "Creator.h"
classA::classA()
{
}
classA* pClassA = Creator<classA>::create();
SubFolder/Creator.h
#ifndef __CREATOR__
#define __CREATOR__
#include <iostream>
template<typename T>
class Creator
{
public:
static T* create()
{
T* p = new T();
// Do Something here
// ... like output
std::cout << std::endl << "created: " << p;
return p;
}
};
#endif //__CREATOR__
SubFolder/CMakeLists.txt
add_library(SubFolder ClassA.cpp)
I compile this project and run it. So I get only the output "Hello, world!".
When I remove the comment (main.cpp line 5) a instance of ClassA is used. So I get also the output of class Creator. The code for ClassA is linked.
When I move the class ClassA to root directory it works also.
I have also tried to use parameters like PUBLIC_LINK, debug and general for target_link_libraries. But nothing works.
My intention use a Collection Class in this main.cpp file and get the instanced object from the collection. In the main.ccp file i don't want to know each instanced class because all class ClassA ... ClassZ have the same interface (not shown in this example).
How can i force the link of "unused" code?
Edit: Do don't know if it's neccessary. I use KDevelop4.
See How to force gcc to link an unused static library
I've tested your code with GNU 4.8.1 compilers and in your example just replace your target_link_libraries() line with:
target_link_libraries(
teststaticcmake
PRIVATE
"-Wl,--whole-archive"
SubFolder
"-Wl,--no-whole-archive"
)
From target_link_libraries() documentation:
A link flag: Item names starting with -, but not -l or -framework, are treated as linker flags. Note that such flags will be treated like any other library link item for purposes of transitive dependencies, so they are generally safe to specify only as private link items that will not propagate to dependents.
More References
How to force inclusion of an object file in a static library when linking into executable?

Static variables with build-in types in C++ DLL is not initialized

I need to build DLL lib from C++ code which rely on some global static variables. Static variables of user defined classes initialized just fine while variables with primitive types (int) does not. Even more, attempt to change value of such variables lead to memory errors. What am I doing wrong?
I was able to replicate this error in a few lines example, here is my code:
mylib.hpp
#include <string>
class AAA
{
public:
static int & get_static_int() { return static_int_; }
private:
static int static_int_;
};
mylib.cpp
#include "mylib.hpp"
#include <iostream>
int AAA::static_int_ = 42;
class BBB
{
public:
BBB() { std::cout << "BBB constructor!!!" << std::endl; };
};
static BBB b;
mylib.def:
LIBRARY mylib
EXPORTS
?static_int_#AAA##0HA
main.cpp
#include <mylib.hpp>
#include <iostream>
int main()
{
std::cout << "get_static_int():" << AAA::get_static_int() << std::endl;
AAA::get_static_int() = 10; // Exception!
}
build.bat
cl /c mylib.cpp /I. /Fomylib.obj /MD /EHsc
link /MACHINE:X64 /dll mylib.obj /DEF:mylib.def /out:mylib.dll msvcrt.lib kernel32.lib oleaut32.lib
cl main.cpp mylib.lib /I. /EHsc /MD
output:
>main
BBB constructor!!!
get_static_int():403973631
<windows complain>
I am aware of similar question: Loading DLL not initializing static C++ classes and adding "-entry:_DllMainCRTStartup" to link options does not fix this problem.
I am also aware of proper way to initialize singletons in C++ but for this projects i need to port rather large Linux project and modifying/refactoring it will not be practical.
Thanks!
Using a def file isn't the best way of doing this - you should save yourself the pain by using dllexport.
If you must use def files because it has been dictated, change your def file to
LIBRARY mylib
EXPORTS
?get_static_int#AAA##SAAAHXZ
As you have discovered, the function is inlined so you need to uninline it. You could use /Ob0 on the cl line but I've never found that to work. It still gets inlined and you will still get the exception. mylib.hpp should be
class AAA
{
public:
static int & get_static_int();
private:
static int static_int_;
};
Add the following to mylib.cpp
int& AAA::get_static_int()
{
return static_int_;
}
If you do it this way, then it should run without throwing the exception.

Calling function from different files in C++ error

I would like to use one function from Stats.cpp in Application.cpp. Here are my code snippets:
In Stats.h:
#ifndef STATS_H
#define STATS_H
class Stats
{
public:
void generateStat(int i);
};
#endif
In Stats.cpp:
#include Stats.h
void generateStat(int i)
{
//some process code here
}
In Application.cpp:
int main()
{
generateStat(10);
}
I get an "unresolved external symbol" error however I don't know what I else I would need to include in order for Application.cpp. Any thoughts?
In Stats.cpp
you need to define generateStat like following :
#include Stats.h
void Stats:: generateStat(int i) // Notice the syntax, use of :: operator
{
//some process code here
}
Then create object of class Stats, use it to call the public member function generateStat
Stats s;
s.generateStat( 10 ) ;
Build the application using :
g++ -o stats Stats.cpp Application.cpp -I.
generateStat is part of your Stats class. You need to instantiate a Stats object (along with the necessary includes for Stats.h in your main class)
For example,
Stats stat;
stat.generateStat(i);
Also, your function definition needs to include the class name Stats::generateStat.
The same error msg occured 2 weeks ago (at work).
At first glance --- Try:
void Stats::generateStat(int i) {
//some process code here }
The class name was missing. Hence, unresolved.
btw Concerning your header --- another issue, this #ifndef directive should not be necessary cause you should declare Stats only once in a namespace.
#ifndef CLASS_H
#define CLASS_H
#include "Class.h"
#endif
This is a generic example - Usable in cpp files.
EDIT: Now, I saw your invocation (main method in your case). You need an object instance to invoke your method.
Stats* stats = new Stats(); //add a default constructor if not done
stats->generateStat(77);
// any other stats stuff ......
// in posterior to the last use
delete(stats);
In your header:
Stats::Stats(){}; //with an empty body - no need to write it again in the cpp file