emscripten Linking globals named symbol multiply defined - c++

I have a c++ CMAKE (VERSION 3.10.2 -std=c++17 ) project that i am able to compile and link with bought gcc and clang. Bought of them produce the target binaries which work as expected. Recently i decided to to try and add another target i.e. webassembly. The project is compiling as expect, however when the EMscripten build is being executed i.e. in the linking phase i get the following error:
Elapsed time: 1 s. (time), 0.002241 s. (clock)
[100%] Linking CXX executable wasmExec.js
cd /Projects/time/time.cpp/build/src/wasm && /usr/bin/cmake -E cmake_link_script CMakeFiles/wasmExec.dir/link.txt --verbose=1
/Projects/emscripten/emsdk/emscripten/1.38.12/em++ -s WASM=1 -s NO_EXIT_RUNTIME=1 -s VERBOSE=1 --pre-js /Projects/time/time.cpp/src/wasm/preModule.js -s DEMANGLE_SUPPORT=1 -s DISABLE_EXCEPTION_CATCHING=0 -s ERROR_ON_UNDEFINED_SYMBOLS=0 #CMakeFiles/wasmExec.dir/objects1.rsp -o wasmExec.js #CMakeFiles/wasmExec.dir/linklibs.rsp
error: Linking globals named '_ZTVN9timeproto3time8defaults20TimeDefaultParametersE': symbol multiply defined!
WARNING:root:Note: Input file "#/tmp/tmpUeJ6zc.response" did not exist.
ERROR:root:Failed to run llvm optimizations:
When i do
c++filt _ZTVN9timeproto3time8defaults20TimeDefaultParametersE
i get
vtable for timeproto::time::defaults::TimeDefaultParameters
from another answer by Stackoverflow i.e.
Possible reasons for symbol multiply defined other than 'extern'
i do understand that i have defined this class more then once, however my problem is that i can not locate that place where i have made the mistake with the second definition. In the previous answer the person had the hint i.e. the cpp file where that he has made that mistake but in my case emscipten is not so generous.
This class is used all over the code base in many many files and after long manual searching i was not able to find anything that can point me at least to the localtion of the second definition. Thus i was hoping that someone can help me with the following questions
1) how can this be troubleshoot further in order to find where exactly the second defintion of the class is occuring, maybe a flag by gcc or clang ?
2) why this error is beeing displayed only when I am trying to compile/build the webassmbly target. The regular Linux64 build target is successefull and the test are also working correctly.
3) I am running cmake with following "add_definitions" i.e.
if(UNIX)
add_definitions(" -pedantic -pedantic-errors -W ")
add_definitions(" -Wall -Wextra -Werror -Wshadow -Wnon-virtual-dtor ")
add_definitions(" -v ")
# add_definitions(" -Worl-style-cast -Wcast-align ")
# add_definitions(" -Wunused -Woverloaded-virtual ")
add_definitions(" -g ")
endif(UNIX)
if the TimeDefaultParameters has been defined more then once should't clang not complain also for linux build with the above "add_definitions" ?
here is the code below TimeDefaultParameters.cpp This is a very simple file that does not contain any object instead it has 43 "static const uint32_t" variables.
#include "TimeDefaultParameters.h"
namespace timeproto::time::defaults
{
TimeDefaultParameters::TimeDefaultParameters() {
}
TimeDefaultParameters::~TimeDefaultParameters() {
}
const uint32_t TimeDefaultParameters::SIGNED_SHORT_MAX_VALUE = 32767;
.... (another 42 static const uint32_t)
}
and the header file TimeDefaultParameters.h:
#ifndef _TIME_DEFAULT_PARAMETERS_
#define _TIME_DEFAULT_PARAMETERS_
#include <stdint.h>
namespace timeproto::time::defaults
{
class TimeDefaultParameters final
{
public:
explicit TimeDefaultParameters();
virtual ~TimeDefaultParameters();
static const uint32_t SIGNED_SHORT_MAX_VALUE;
.....
.... (another 42 static const uint32_t)
};
}
#endif //#ifndef _TIME_DEFAULT_PARAMETERS_
in cmake i have the set my target properties like:
set_target_properties(wasmExec PROPERTIES LINK_FLAGS "-s WASM=1 -s NO_EXIT_RUNTIME=1 -s VERBOSE=1 --pre-js /Projects/time/time.cpp/src/wasm/preModule.js -s DEMANGLE_SUPPORT=1 -s DISABLE_EXCEPTION_CATCHING=0 -s ERROR_ON_UNDEFINED_SYMBOLS=0" )
this is how i am calling cmake to make the build from withing the build directory
emconfigure cmake -DCMAKE_BUILD_TYPE=Emscripten -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=/Projects/emscripten/emsdk/emscripten/1.38.12/cmake/Modules/Platform/Emscripten.cmake ../
make -j8
any ideas are greatly appreciated.
ADDITION: 05 January 2020
I was able to find a workaround for this problem but i still it leaves some questions of the nature of the error.
The class in question was part of the archive that was created and loaded dynamically i.e. i had used in the CMAKE part for this library "set(LIB_TYPE SHARED)".
here is full example how cmake generated that archive i.e. the CMakeLists.txt.
set( TIME_DEFAULTS_SRC
...
TimeDefaultParameters.h TimeDefaultParameters.cpp
...
)
set(LIB_TYPE STATIC)
#set(LIB_TYPE SHARED)
add_library(time_defaults ${LIB_TYPE} ${TIME_DEFAULTS_SRC} )
target_include_directories(time_defaults PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}/")
I have changed from dynamic to static and i was able to create the wasm no errors were shown. During the compilation i also saw somewhere in between of the compilation process some warrning i.e. :
WARNING:root:When Emscripten compiles to a typical native suffix for shared libraries (.so, .dylib, .dll) then it emits an LLVM bitcode file. You should then compile that to an emscripten SIDE_MODULE (using that flag) with suffix .wasm (for wasm) or .js (for asm.js). (You may also want to adapt your build system to emit the more standard suffix for a file with LLVM bitcode, '.bc', which would avoid this warning.)
this warrning is now gone. But it is very easy to oversee thing like that especially if the compilation process is taking long time. However my understanding is that the very first error message tells us , "look you have made duplicate definition of some symbol in your code go find the place and make sure that the class is define only once". That was exactly what i was doing i.e. searching the code base for that duplicate definition. Thus now the question is: Why emscripten have a problem does with dynamic linking i.e. i know that it is officially supported i.e.
https://webassembly.org/docs/dynamic-linking/
and is that the source of the error at all or is something else?
Why this error disappears when i change to static. I can reproduce this by simply changing library type!
I think i already found the answer here
https://github.com/emscripten-core/emscripten/wiki/Linking

So the solution in my case was to find the occurrences in the CMAKE file where the library was added dynamically and change that to static linking i.e.
#set(LIB_TYPE SHARED)
set(LIB_TYPE STATIC)

Related

Using CMake and Clang++ with C++20 Modules support [duplicate]

Clang and MSVC already supports Modules TS from unfinished C++20 standard.
Can I build my modules based project with CMake or other build system and how?
I tried build2, it supports modules and it works very well, but i have a question about it's dependency management (UPD: question is closed).
CMake currently does not support C++20 modules.
See also the relevant issue in the CMake issue tracker. Note that supporting modules requires far more support from the build system than inserting a new compiler option. It fundamentally changes how dependencies between source files have to be handled during the build: In a pre-modules world all cpp source files can be built independently in any order. With modules that is no longer true, which has implications not only for CMake itself, but also for the downstream build system.
Take a look at the CMake Fortran modules paper for the gory details. From a build system's point of view, Fortran's modules behave very similar to the C++20 modules.
Update: CMake 3.20 introduces experimental support for Modules with the Ninja Generator (and only for Ninja). Details can be found in the respective pull request. At this stage, this feature is still highly experimental and not intended for production use. If you intend to play around with this anyway, you really should be reading both the Fortran modules paper and the dependency format paper to understand what you're getting into.
This works on Linux Manjaro (same as Arch), but should work on any Unix OS. Of course, you need to build with new clang (tested with clang-10).
helloworld.cpp:
export module helloworld;
import <cstdio>;
export void hello() { puts("Hello world!"); }
main.cpp:
import helloworld; // import declaration
int main() {
hello();
}
CMakeLists.txt:
cmake_minimum_required(VERSION 3.16)
project(main)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(PREBUILT_MODULE_PATH ${CMAKE_BINARY_DIR}/modules)
function(add_module name)
file(MAKE_DIRECTORY ${PREBUILT_MODULE_PATH})
add_custom_target(${name}.pcm
COMMAND
${CMAKE_CXX_COMPILER}
-std=c++20
-stdlib=libc++
-fmodules
-c
${CMAKE_CURRENT_SOURCE_DIR}/${ARGN}
-Xclang -emit-module-interface
-o ${PREBUILT_MODULE_PATH}/${name}.pcm
)
endfunction()
add_compile_options(-fmodules)
add_compile_options(-stdlib=libc++)
add_compile_options(-fbuiltin-module-map)
add_compile_options(-fimplicit-module-maps)
add_compile_options(-fprebuilt-module-path=${PREBUILT_MODULE_PATH})
add_module(helloworld helloworld.cpp)
add_executable(main
main.cpp
helloworld.cpp
)
add_dependencies(main helloworld.pcm)
Assuming that you're using gcc 11 with a Makefile generator, the following code should work even without CMake support for C++20:
cmake_minimum_required(VERSION 3.19) # Lower versions should also be supported
project(cpp20-modules)
# Add target to build iostream module
add_custom_target(std_modules ALL
COMMAND ${CMAKE_COMMAND} -E echo "Building standard library modules"
COMMAND g++ -fmodules-ts -std=c++20 -c -x c++-system-header iostream
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
)
# Function to set up modules in GCC
function (prepare_for_module TGT)
target_compile_options(${TGT} PUBLIC -fmodules-ts)
set_property(TARGET ${TGT} PROPERTY CXX_STANDARD 20)
set_property(TARGET ${TGT} PROPERTY CXX_EXTENSIONS OFF)
add_dependencies(${TGT} std_modules)
endfunction()
# Program name and sources
set (TARGET prog)
set (SOURCES main.cpp)
set (MODULES mymod.cpp)
# Setup program modules object library
set (MODULE_TARGET prog-modules)
add_library(${MODULE_TARGET} OBJECT ${MODULES})
prepare_for_module(${MODULE_TARGET})
# Setup executable
add_executable(${TARGET} ${SOURCES})
prepare_for_module(${TARGET})
# Add modules to application using object library
target_link_libraries(${TARGET} PRIVATE ${MODULE_TARGET})
Some explanation:
A custom target is added to build the standard library modules, in case you want to include standard library header units (search for "Standard Library Header Units" here). For simplicity, I just added iostream here.
Next, a function is added to conveniently enable C++20 and Modules TS for targets
We first create an object library to build the user modules
Finally, we create our executable and link it to the object library created in the previous step.
Not consider the following main.cpp:
import mymod;
int main() {
helloModule();
}
and mymod.cpp:
module;
export module mymod;
import <iostream>;
export void helloModule() {
std::cout << "Hello module!\n";
}
Using the above CMakeLists.txt, your example should compile fine (successfully tested in Ubuntu WSL with gcc 1.11.0).
Update:
Sometimes when changing the CMakeLists.txt and recompiling, you may encounter an error
error: import "/usr/include/c++/11/iostream" has CRC mismatch
Probably the reason is that every new module will attempt to build the standard library modules, but I'm not sure. Unfortunately I didn't find a proper solution to this (avoiding rebuild if the gcm.cache directory already exists is bad if you want to add new standard modules, and doing it per-module is a maintenance nightmare). My Q&D solution is to delete ${CMAKE_BINARY_DIR}/gcm.cache and rebuild the modules. I'm happy for better suggestions though.
CMake ships with experimental support for C++20 modules:
https://gitlab.kitware.com/cmake/cmake/-/blob/master/Help/dev/experimental.rst
This is tracked in this issue:
https://gitlab.kitware.com/cmake/cmake/-/issues/18355
There is also a CMakeCXXModules repository that adds support for modules to CMake.
https://github.com/NTSFka/CMakeCxxModules
While waiting for proper C++20 modules support in CMake, I've found that if using MSVC Windows, for right now you can make-believe it's there by hacking around the build instead of around CMakeLists.txt: continously generate with latest VS generator, and open/build the .sln with VS2020. The IFC dependency chain gets taken care of automatically (import <iostream>; just works). Haven't tried Windows clang or cross-compiling. It's not ideal but for now at least another decently workable alternative today, so far.
Important afterthought: use .cppm and .ixx extensions.
CMake does not currently support C++20 modules like the others have stated. However, module support for Fortran is very similar, and perhaps this could be easily changed to support modules in C++20.
http://fortranwiki.org/fortran/show/Build+tools
Now, perhaps there i an easy way to modify this to support C++20 directly. Not sure. It is worth exploring and doing a pull request should you resolve it.
Add MSVC version (revised from #warchantua 's answer):
cmake_minimum_required(VERSION 3.16)
project(Cpp20)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(PREBUILT_MODULE_DIR ${CMAKE_BINARY_DIR}/modules)
set(STD_MODULES_DIR "D:/MSVC/VC/Tools/MSVC/14.29.30133/ifc/x64") # macro "$(VC_IFCPath)" in MSVC
function(add_module name)
file(MAKE_DIRECTORY ${PREBUILT_MODULE_DIR})
add_custom_target(${name}.ifc
COMMAND
${CMAKE_CXX_COMPILER}
/std:c++latest
/stdIfcDir ${STD_MODULES_DIR}
/experimental:module
/c
/EHsc
/MD
${CMAKE_CURRENT_SOURCE_DIR}/${ARGN}
/module:export
/ifcOutput
${PREBUILT_MODULE_DIR}/${name}.ifc
/Fo${PREBUILT_MODULE_DIR}/${name}.obj
)
endfunction()
set(CUSTOM_MODULES_DIR ${CMAKE_CURRENT_SOURCE_DIR}/modules)
add_module(my_module ${CUSTOM_MODULES_DIR}/my_module.ixx)
add_executable(test
test.cpp
)
target_compile_options(test
BEFORE
PRIVATE
/std:c++latest
/experimental:module
/stdIfcDir ${STD_MODULES_DIR}
/ifcSearchDir ${PREBUILT_MODULE_DIR}
/reference my_module=${PREBUILT_MODULE_DIR}/my_module.ifc
/EHsc
/MD
)
target_link_libraries(test ${PREBUILT_MODULE_DIR}/my_module.obj)
add_dependencies(test my_module.ifc)
With C++20 Modules the file compilation order matters, which is totally new. That's why the implementation is complicated and still experimental in 2023. Please read the authors blogpost
I was not able to find Cmake support for modules. Here is an example how to use modules using clang. I am using Mac and this example works ok on my system. It took me quite a while to figure this out so unsure how general this is across linux or Windows.
Source code in file driver.cxx
import hello;
int main() { say_hello("Modules"); }
Source code in file hello.cxx
#include <iostream>
module hello;
void say_hello(const char *n) {
std::cout << "Hello, " << n << "!" << std::endl;
}
Source code in file hello.mxx
export module hello;
export void say_hello (const char* name);
And to compile the code with above source files, here are command lines on terminal
clang++ \
-std=c++2a \
-fmodules-ts \
--precompile \
-x c++-module \
-Xclang -fmodules-embed-all-files \
-Xclang -fmodules-codegen \
-Xclang -fmodules-debuginfo \
-o hello.pcm hello.mxx
clang++ -std=c++2a -fmodules-ts -o hello.pcm.o -c hello.pcm
clang++ -std=c++2a -fmodules-ts -x c++ -o hello.o \
-fmodule-file=hello.pcm -c hello.cxx
clang++ -std=c++2a -fmodules-ts -x c++ -o driver.o \
-fmodule-file=hello=hello.pcm -c driver.cxx
clang++ -o hello hello.pcm.o driver.o hello.o
and to get clean start on next compile
rm -f *.o
rm -f hello
rm -f hello.pcm
expected output
./hello
Hello, Modules!
Hope this helps, all the best.

Interaction of fno-exceptions and POSITION INDEPENDENT CODE

I ran into a rather bizarre issue while building a dynamic library. Here are details with a small example:
A simple file called static.h whose contents are:
#pragma once
#include <string>
std::string static_speak();
static.cpp looks like this:
#include "static.h"
std::string static_speak() {
return "I am static";
}
one can build a static library with these two files (using cmake) as:
add_library(static
static.cpp
)
Now, consider another file called shared.cpp whose contents are:
#include "static.h"
std::string dynamic_speak() {
return static_speak() + " I am dynamic";
}
One can try to build a dynamic library (again using cmake) as:
add_library(shared SHARED
shared.cpp
)
target_link_libraries(shared PRIVATE
static
)
When one tries to build the above, one will run into the following error:
[4/4] Linking CXX shared library libshared.so
FAILED: libshared.so
: && /opt/vatic/bin/clang++ -fPIC -g -shared -Wl,-soname,libshared.so -o libshared.so CMakeFiles/shared.dir/shared.cpp.o libstatic.a && :
/usr/bin/ld: libstatic.a(static.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
This makes sense. We didn't compile static with POSITION_INDEPENDENT_CODE. That is easily fixable via:
add_library(static
static.cpp
)
set_target_properties(static
PROPERTIES
POSITION_INDEPENDENT_CODE ON
)
Everything works perfectly now when one compiles shared library.
Now here is the problem. Let's say I didn't enable POSITION_INDEPENDENT_CODE but instead disabled exceptions (!) in my code as:
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-exceptions")
Now when I try to compile shared, everything still works!!
How are exceptions and fPIC related to each other?
Here is a repo to reproduce the issue:
https://github.com/skgbanga/shared
How are exceptions and fPIC related to each other?
They aren't (mostly).
What happens is that compiling with -fno-exceptions changes the relocations (certain exception-related data is no longer referenced), and because of that, there is no relocation which causes the error at link time.
You can confirm this by building with and without -fno-exceptions, and comparing output from objdump -dr CMakeFiles/static.dir/static.cpp.o.
P.S. I could not reproduce your problem using g++ (Debian 9.3.0-8), because it defaults to building with -fPIE, and both links succeed.
And if I add -fno-pie, then both links fail the same way.

How to use c++20 modules with CMake?

Clang and MSVC already supports Modules TS from unfinished C++20 standard.
Can I build my modules based project with CMake or other build system and how?
I tried build2, it supports modules and it works very well, but i have a question about it's dependency management (UPD: question is closed).
CMake currently does not support C++20 modules.
See also the relevant issue in the CMake issue tracker. Note that supporting modules requires far more support from the build system than inserting a new compiler option. It fundamentally changes how dependencies between source files have to be handled during the build: In a pre-modules world all cpp source files can be built independently in any order. With modules that is no longer true, which has implications not only for CMake itself, but also for the downstream build system.
Take a look at the CMake Fortran modules paper for the gory details. From a build system's point of view, Fortran's modules behave very similar to the C++20 modules.
Update: CMake 3.20 introduces experimental support for Modules with the Ninja Generator (and only for Ninja). Details can be found in the respective pull request. At this stage, this feature is still highly experimental and not intended for production use. If you intend to play around with this anyway, you really should be reading both the Fortran modules paper and the dependency format paper to understand what you're getting into.
This works on Linux Manjaro (same as Arch), but should work on any Unix OS. Of course, you need to build with new clang (tested with clang-10).
helloworld.cpp:
export module helloworld;
import <cstdio>;
export void hello() { puts("Hello world!"); }
main.cpp:
import helloworld; // import declaration
int main() {
hello();
}
CMakeLists.txt:
cmake_minimum_required(VERSION 3.16)
project(main)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(PREBUILT_MODULE_PATH ${CMAKE_BINARY_DIR}/modules)
function(add_module name)
file(MAKE_DIRECTORY ${PREBUILT_MODULE_PATH})
add_custom_target(${name}.pcm
COMMAND
${CMAKE_CXX_COMPILER}
-std=c++20
-stdlib=libc++
-fmodules
-c
${CMAKE_CURRENT_SOURCE_DIR}/${ARGN}
-Xclang -emit-module-interface
-o ${PREBUILT_MODULE_PATH}/${name}.pcm
)
endfunction()
add_compile_options(-fmodules)
add_compile_options(-stdlib=libc++)
add_compile_options(-fbuiltin-module-map)
add_compile_options(-fimplicit-module-maps)
add_compile_options(-fprebuilt-module-path=${PREBUILT_MODULE_PATH})
add_module(helloworld helloworld.cpp)
add_executable(main
main.cpp
helloworld.cpp
)
add_dependencies(main helloworld.pcm)
Assuming that you're using gcc 11 with a Makefile generator, the following code should work even without CMake support for C++20:
cmake_minimum_required(VERSION 3.19) # Lower versions should also be supported
project(cpp20-modules)
# Add target to build iostream module
add_custom_target(std_modules ALL
COMMAND ${CMAKE_COMMAND} -E echo "Building standard library modules"
COMMAND g++ -fmodules-ts -std=c++20 -c -x c++-system-header iostream
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
)
# Function to set up modules in GCC
function (prepare_for_module TGT)
target_compile_options(${TGT} PUBLIC -fmodules-ts)
set_property(TARGET ${TGT} PROPERTY CXX_STANDARD 20)
set_property(TARGET ${TGT} PROPERTY CXX_EXTENSIONS OFF)
add_dependencies(${TGT} std_modules)
endfunction()
# Program name and sources
set (TARGET prog)
set (SOURCES main.cpp)
set (MODULES mymod.cpp)
# Setup program modules object library
set (MODULE_TARGET prog-modules)
add_library(${MODULE_TARGET} OBJECT ${MODULES})
prepare_for_module(${MODULE_TARGET})
# Setup executable
add_executable(${TARGET} ${SOURCES})
prepare_for_module(${TARGET})
# Add modules to application using object library
target_link_libraries(${TARGET} PRIVATE ${MODULE_TARGET})
Some explanation:
A custom target is added to build the standard library modules, in case you want to include standard library header units (search for "Standard Library Header Units" here). For simplicity, I just added iostream here.
Next, a function is added to conveniently enable C++20 and Modules TS for targets
We first create an object library to build the user modules
Finally, we create our executable and link it to the object library created in the previous step.
Not consider the following main.cpp:
import mymod;
int main() {
helloModule();
}
and mymod.cpp:
module;
export module mymod;
import <iostream>;
export void helloModule() {
std::cout << "Hello module!\n";
}
Using the above CMakeLists.txt, your example should compile fine (successfully tested in Ubuntu WSL with gcc 1.11.0).
Update:
Sometimes when changing the CMakeLists.txt and recompiling, you may encounter an error
error: import "/usr/include/c++/11/iostream" has CRC mismatch
Probably the reason is that every new module will attempt to build the standard library modules, but I'm not sure. Unfortunately I didn't find a proper solution to this (avoiding rebuild if the gcm.cache directory already exists is bad if you want to add new standard modules, and doing it per-module is a maintenance nightmare). My Q&D solution is to delete ${CMAKE_BINARY_DIR}/gcm.cache and rebuild the modules. I'm happy for better suggestions though.
CMake ships with experimental support for C++20 modules:
https://gitlab.kitware.com/cmake/cmake/-/blob/master/Help/dev/experimental.rst
This is tracked in this issue:
https://gitlab.kitware.com/cmake/cmake/-/issues/18355
There is also a CMakeCXXModules repository that adds support for modules to CMake.
https://github.com/NTSFka/CMakeCxxModules
While waiting for proper C++20 modules support in CMake, I've found that if using MSVC Windows, for right now you can make-believe it's there by hacking around the build instead of around CMakeLists.txt: continously generate with latest VS generator, and open/build the .sln with VS2020. The IFC dependency chain gets taken care of automatically (import <iostream>; just works). Haven't tried Windows clang or cross-compiling. It's not ideal but for now at least another decently workable alternative today, so far.
Important afterthought: use .cppm and .ixx extensions.
CMake does not currently support C++20 modules like the others have stated. However, module support for Fortran is very similar, and perhaps this could be easily changed to support modules in C++20.
http://fortranwiki.org/fortran/show/Build+tools
Now, perhaps there i an easy way to modify this to support C++20 directly. Not sure. It is worth exploring and doing a pull request should you resolve it.
Add MSVC version (revised from #warchantua 's answer):
cmake_minimum_required(VERSION 3.16)
project(Cpp20)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(PREBUILT_MODULE_DIR ${CMAKE_BINARY_DIR}/modules)
set(STD_MODULES_DIR "D:/MSVC/VC/Tools/MSVC/14.29.30133/ifc/x64") # macro "$(VC_IFCPath)" in MSVC
function(add_module name)
file(MAKE_DIRECTORY ${PREBUILT_MODULE_DIR})
add_custom_target(${name}.ifc
COMMAND
${CMAKE_CXX_COMPILER}
/std:c++latest
/stdIfcDir ${STD_MODULES_DIR}
/experimental:module
/c
/EHsc
/MD
${CMAKE_CURRENT_SOURCE_DIR}/${ARGN}
/module:export
/ifcOutput
${PREBUILT_MODULE_DIR}/${name}.ifc
/Fo${PREBUILT_MODULE_DIR}/${name}.obj
)
endfunction()
set(CUSTOM_MODULES_DIR ${CMAKE_CURRENT_SOURCE_DIR}/modules)
add_module(my_module ${CUSTOM_MODULES_DIR}/my_module.ixx)
add_executable(test
test.cpp
)
target_compile_options(test
BEFORE
PRIVATE
/std:c++latest
/experimental:module
/stdIfcDir ${STD_MODULES_DIR}
/ifcSearchDir ${PREBUILT_MODULE_DIR}
/reference my_module=${PREBUILT_MODULE_DIR}/my_module.ifc
/EHsc
/MD
)
target_link_libraries(test ${PREBUILT_MODULE_DIR}/my_module.obj)
add_dependencies(test my_module.ifc)
With C++20 Modules the file compilation order matters, which is totally new. That's why the implementation is complicated and still experimental in 2023. Please read the authors blogpost
I was not able to find Cmake support for modules. Here is an example how to use modules using clang. I am using Mac and this example works ok on my system. It took me quite a while to figure this out so unsure how general this is across linux or Windows.
Source code in file driver.cxx
import hello;
int main() { say_hello("Modules"); }
Source code in file hello.cxx
#include <iostream>
module hello;
void say_hello(const char *n) {
std::cout << "Hello, " << n << "!" << std::endl;
}
Source code in file hello.mxx
export module hello;
export void say_hello (const char* name);
And to compile the code with above source files, here are command lines on terminal
clang++ \
-std=c++2a \
-fmodules-ts \
--precompile \
-x c++-module \
-Xclang -fmodules-embed-all-files \
-Xclang -fmodules-codegen \
-Xclang -fmodules-debuginfo \
-o hello.pcm hello.mxx
clang++ -std=c++2a -fmodules-ts -o hello.pcm.o -c hello.pcm
clang++ -std=c++2a -fmodules-ts -x c++ -o hello.o \
-fmodule-file=hello.pcm -c hello.cxx
clang++ -std=c++2a -fmodules-ts -x c++ -o driver.o \
-fmodule-file=hello=hello.pcm -c driver.cxx
clang++ -o hello hello.pcm.o driver.o hello.o
and to get clean start on next compile
rm -f *.o
rm -f hello
rm -f hello.pcm
expected output
./hello
Hello, Modules!
Hope this helps, all the best.

Compiling an external library on Linux

Good Day Everyone,
N.B - This problem has been solved - I have provided my own solution in the answer section however the solution provided by Jonathan is much shorter. Nevertheless, this was the following question I originally posted:
I am basically trying to compile a serial library (for UART communication) on Linux however I am not really sure how to correctly compile (I have mentioned what I have done so far below), any suggestions would be highly valuable. I am using the serialib library - which is composed of 2 main files (serialib.h and serialib.cpp) , you may directly view the source code of these files here (scroll all the way to the bottom and view the files in new tabs): http://serialib.free.fr/html/classserialib.html
I transferred these files (serialib.h and serialib.cpp) to my BeagleBone Black micro-controller which is running Debian (Wheezy) , g++/gcc (Debian 4.6.3-14) 4.6.3. I wrote my own program (uart.cpp is my file name) to access the functions provided by this library, this is what I wrote:
#include <iostream>
#include "serialib.h"
#ifdef __linux__
#define DEVICE_PORT "/dev/ttyO1"
#endif
int main()
{
serialib LS;
return 0;
}
So as you can see I am trying to access the 'seriallib' class. serialib.h, serialib.cpp and uart.cpp are all in the home directory. I also manually added the iostream library in serialib.cpp as I did not see it being declared in the original source code.
Now I am really unsure of how to compile such external libraries but so far I tried the following steps:
g++ -c -Wall -Werror -fPIC serialib.c to convert to PIC which gives the following error:
distcc[3142] (dcc_parse_hosts) Warning: /home/debian/.distcc/zeroconf/hosts contained no hosts; can't distribute work
distcc[3142] (dcc_zeroconf_add_hosts) CRITICAL! failed to parse host file.
distcc[3142] (dcc_build_somewhere) Warning: failed to distribute, running locally instead
g++ serialib.cpp -L /home/debian/serialib.h which gives the following error:
/usr/lib/gcc/arm-linux-gnueabihf/4.6/../../../arm-linux-gnueabihf/crt1.o: In function _start':
(.text+0x30): undefined reference tomain'
collect2: ld returned 1 exit status
distcc[3210] ERROR: compile serialib.cpp on localhost failed
As of now I am still finding out how to compile this and if I manage to work this out then I'll post my solution here too. Once again any suggestion will be highly valuable. Thank you all :) .
g++ -c -Wall -Werror -fPIC serialib.c to convert to PIC which gives the following error:
The "error" is not an error, it's a warning, telling you that your distcc setup is broken, but that it compiled locally.
That command doesn't "convert to PIC", it compiles the file serialib.c and produces a compiled object file, serialib.o
g++ serialib.cpp -L /home/debian/serialib.h
This is just nonsense. It tries to build a program from serialib.cpp and use the directory /home/debian/serialib.h (which isn't a directory!) to find libraries.
You don't need to "compile a library" you can just compile both the source files and link them together into a program. Either:
g++ -c serialib.cpp
g++ -c uart.cpp
g++ serialib.o uart.o -o uart
Or all in one command:
g++ serialib.cpp uart.cpp -o uart
You should read An Introduction to GCC to understand the commands, not just enter bogus commands without understanding them.
I have found a solution to this problem, hope this helps for all the future readers with similar problems. I have my own source code uart.cpp (Given in the question) which I want to compile, the external library is serialib that contains two main files (serialib.h and serialib.cpp), you will want to replace the following commands with respect to the files you have
Step 1: Compiling with position independent code
g++ -c -Wall -Werror -fpic serialib.cpp
Step 2: Creating a shared library
g++ -shared -o libserialib.so serialib.o , here the library is libserialib.so.
Step 3: Linking your source code with library
g++ -L /home/debian -lserialib uart.cpp -o uart
g++ -L /home/debian -Wall -o test uart.cpp -lserialib
You may save the library at a different path and you may have a different name of course. Suppose you have a library called libabc.so at the directory /home/user/myDir then the commands will be like:
g++ -L /home/user/myDir -labc your_code.cpp -o your_code
g++ -L /home/user/myDir -Wall -o test your_code.cpp -labc
test is out own program, lserialib is actually looking for libserialib.so and not serialib.o as gcc/g++ assumes all libraries start with lib and end with .so or .a and you can see the same goes for labc as it will look for libabc.so thus it is important to make sure your library name begins with lib and ends with .so or .a
Step 4: Making library available at run time
Here we provide the path where the library is actually stored, I saved it in the directory /home/debian which is why my command looks like:
export LD_LIBRARY_PATH=/home/debian:$LD_LIBRARY_PATH
if your library is saved at /path/to/file then the command will look like:
export LD_LIBRARY_PATH=/path/to/file:$LD_LIBRARY_PATH
This is to help the loader find the shared library and to view this path: echo $LD_LIBRARY_PATH and to unset this: unset LD_LIBRARY_PATH
To execute the program type either ./test or ./uart and in case of any modification to the main source code (uart.cpp in this case) , simply repeat step 3. I found the following link very useful: http://www.cprogramming.com/tutorial/shared-libraries-linux-gcc.html . Thank you to all of you who took time to read this question and especially those who gave me suggestions. If anyone has more or better solutions, feel free to post them here to assist future readers :).

Linking error when compiling Crypto++ for ARMHF

I'm trying to compile the crypto++ library to run for the armhf architecture. I'm following the method provided in this answer. I tweaked the setenv-embed.sh to match my system's configuration. The output of running . ./setenv-embed.sh is
CPP: /usr/bin/arm-linux-gnueabihf-cpp
CXX: /usr/bin/arm-linux-gnueabihf-g++
AR: /usr/bin/arm-linux-gnueabihf-ar
LD: /usr/bin/arm-linux-gnueabihf-ld
RANLIB: /usr/bin/arm-linux-gnueabihf-gcc-ranlib-4.8
ARM_EMBEDDED_TOOLCHAIN: /usr/bin
ARM_EMBEDDED_CXX_HEADERS: /usr/arm-linux-gnueabihf/include/c++/4.8.2
ARM_EMBEDDED_FLAGS: -march=armv7-a mfloat-abi=hard -mfpu=neon -I/usr/arm-linux-gnueabihf/include/c++/4.8.2 -I/usr/arm-linux-gnueabihf/include/c++/4.8.2/arm-linux-gnueabihf
ARM_EMBEDDED_SYSROOT: /usr/arm-linux-gnueabihf
which indicates that the correct compilers have been found. However, when I build the library using make I run into the following error
/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/../../../../arm-linux-gnueabihf/bin/‌​ld: cannot find /usr/arm-linux-gnueabihf/lib/libc.so.6 inside /usr/arm-linux-gnueabihf
/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/../../../../arm-linux-gnueabihf/bin/‌​ld: cannot find /usr/arm-linux-gnueabihf/lib/libc_nonshared.a inside /usr/arm-linux-gnueabihf
/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/../../../../arm-linux-gnueabihf/bin/‌​ld: cannot find /usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3 inside /usr/arm-linux-gnueabihf
But when I open the location /usr/arm-linux-gnueabihf/lib I can find all the three error files mentioned above ie libc.so.6, libc_nonshared.a and ld-linux-armhf.so.3
I'm trying to compile the library for Beaglebone, if that helps.
Update 1:
The results of running make -f GNUmakefile-cross system after doing a fresh git pull
hassan#hassan-Inspiron-7537:~/cryptopp-armhf$ make -f GNUmakefile-cross system
CXX: /usr/bin/arm-linux-gnueabihf-g++
CXXFLAGS: -DNDEBUG -g2 -Os -Wall -Wextra -DCRYPTOPP_DISABLE_ASM -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb -I/usr/arm-linux-gnueabihf/include/c++/4.8.2 -I/usr/arm-linux-gnueabihf/include/c++/4.8.2/arm-linux-gnueabihf --sysroot=/usr/arm-linux-gnueabihf -Wno-type-limits -Wno-unknown-pragmas
LDLIBS:
GCC_COMPILER: 1
CLANG_COMPILER: 0
INTEL_COMPILER: 0
UNALIGNED_ACCESS:
UNAME: Linux hassan-Inspiron-7537 3.13.0-35-generic #62-Ubuntu SMP Fri Aug 15 01:58:42 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
MACHINE:
SYSTEM:
RELEASE:
make: Nothing to be done for `system'.
The problem is simple. It is in the --sysroot option. The value of this option is /usr/arm-linux-gnueabihf/ and it is used by the linker and the resulting library folder becomes
/usr/arm-linux-gnueabihf/usr/arm-linux-gnueabihf/lib/
I removed the --sysroot option from line 68 in the file GNUmakefile-cross and everything compiled and linked OK.
However, I couldn't run the example on my BeagleBone Black because of mismatch of some shared libraries versions. But this wasn't a real problem for me, because in my application I link crypto++ statically, not dynamically.
Based on Crosswalking's research I think I can explain what is going on. I don't think I agree with the assessment "The problem is simple. It is in the --sysroot option" since the Crypto++ environment script and makefile are doing things as expected.
I think Crosswalking's answer could be how to work around it; but see open questions below. The following is from Crypto++ Issue 134: setenv-embedded.sh and GNUmakefile-cross:
I think this another distro problem, similar to g++-arm-linux-gnueabi
cannot compile a C++ program with
--sysroot.
It might be a Ubuntu problem or a Debian problem if it is coming from
upstream.
When cross-compiling, we expect the following (using ARMHF):
SYSROOT is /usr/arm-linux-gnueabihf
INCLUDEDIR is /usr/arm-linux-gnueabihf/include
LIBDIR is /usr/arm-linux-gnueabihf/lib
BINDIR is /usr/arm-linux-gnueabihf/bin
How LIBDIR morphed into into
/usr/arm-linux-gnueabihf/usr/arm-linux-gnueabihf/lib/ (i.e.,
$SYSROOT/$SYSROOT/lib) is a mystery. But in all fairness, building
GCC is not a trivial task.
You should probably file a bug report with Debian or Ubuntu (or
whomever provides the toolchain).
The open question for me is, since $SYSROOT/lib is messed up, then is $SYSROOT/include messed up, too?
If the include directory is also messed up, then the cross compile is using the host's include files, and not the target include files. That will create hard to diagnose problems later.
If both $SYSROOT/include and $SYSROOT/lib are messed up, then its not enough to simply remove --sysroot. Effectively, this is what has to be done:
# Exported by setenv-embedded
export=ARM_EMBEDDED_SYSROOT=/usr/arm-linux-gnueabihf
# Used by the makefile
-I $ARM_EMBEDDED_SYSROOT/$ARM_EMBEDDED_SYSROOT/include
-L $ARM_EMBEDDED_SYSROOT/$ARM_EMBEDDED_SYSROOT/lib
Which means we should be able to do the following:
# Exported by setenv-embedded
export=ARM_EMBEDDED_SYSROOT=/usr/arm-linux-gnueabihf/usr/arm-linux-gnueabihf
# Used by the makefile
--sysroot="$ARM_EMBEDDED_SYSROOT"
Finally, this looks a lot like Ubuntu's Bug 1375071: g++-arm-linux-gnueabi cannot compile a C++ program with --sysroot. The bug report specifically calls out ... the built-in paths use an extra "/usr/arm-linux-gnueabi".
We need the paths:
A) /usr/arm-linux-gnueabi/include/c++/4.7.3 B)
/usr/arm-linux-gnueabi/include/c++/4.7.3/arm-linux-gnueabi
But the built-in paths tries to use:
C) /usr/arm-linux-gnueabi/usr/arm-linux-gnueabi/include/c++/4.7.3
D)
/usr/arm-linux-gnueabi/usr/arm-linux-gnueabi/include/c++/4.7.3/arm-linux-gnueabi/sf
E)
/usr/arm-linux-gnueabi/usr/arm-linux-gnueabi/include/c++/4.7.3/backward
Notice the built-in paths use an extra "/usr/arm-linux-gnueabi"