I am trying to build custom NS3 module which depends on some static library. This static library depends on NS3 module.
Platform: Ubuntu 16.04 x64
Toolchain: GCC 5.4.0
I will refer to my custom NS3 module as mymodule
I will refer to the library which mymodule depends on as mylib
I will refer to the program which links with mymodule and mylib as myprog
wscript for mymodule:
def build(bld):
module = bld.create_ns3_module('mymodule', ['network'])
module.features = 'c cxx cxxstlib ns3module'
module.source = [
'model/mymodule.cc' ]
# Make a dependency to some other static lib:
bld.env.INCLUDES_MYLIB = [ "some/include/path" ]
bld.env.LIB_MYLIB = ['mylib']
bld.env.LIBPATH_MYLIB = [ "some/path" ]
module.use.append('MYLIB')
# Create a program which uses mymodule
p = bld.create_ns3_program('myprog', ['core', 'mymodule'])
p.source = 'prog.cpp'
headers = bld(features='ns3header')
headers.module = 'mymodule'
headers.source = ['model/mymodule.h']
When I do ./waf build it fails: LD cannot link myprog because mylib has unresolved symbols. This failure is actually expected because mylib and mymodule are codependent and should be linked in non-standard way.
Workarounds:
If I build myprog by hand and use -Wl,--start-group
-lns3.26-mymodule-debug -lmylib -Wl,--end-group it links perfectly fine and works as expected.
If I combine two static libs by hand (using ar -M script) and then run ./waf build it also works fine.
The question: How can I integrate one of the workarounds above into wscript?
it looks like a known problem with order of static libs inclusion. The behavior has changed in waf 1.9, due to this problem.
One workaround might be to use the linkflags attribute of program. You should prefer the use of STLIB_MYLIB and STLIBPATH_MYLIB as mylib is static. In waf 1.9 with he correct order of libs it might suffice.
Anyway, use -v to see the command line generated by waf, it might help !
Related
I am building a test hello-world executable module linking to a simple shared object library 'libshared' which, in turn, links to a static library 'libsodium' that was built in the 'depends' directory.
OS is Ubuntu 16.04.6 LTS.
When I build this for Linux the build is okay.
When I build this for Mingw this libtool error message is printed:
*** Warning: This system cannot link to static lib archive /home/ubuntu/repo/test-dll/depends/x86_64-w64-mingw32/share/../lib/libsodium.la.
*** I have the capability to make that library automatically link in when
*** you link to this library. But I can only do this if you have a
*** shared version of the library, which you do not appear to have.
and recipe for target 'libshared.la' failed.
The Makefile.am:
.PHONY: gen
.INTERMEDIATE: $
# shared lib
lib_LTLIBRARIES = libshared.la
libshared_la_SOURCES = \
src/example_dll.c
libshared_la_CPPFLAGS = -I$(top_srcdir)/src -I /src
libshared_la_CFLAGS = -DBUILDING_EXAMPLE_DLL -O2 -Wno-unused-parameter
libshared_la_LDFLAGS = -no-undefined
libshared_la_LIBADD = -lsodium
#libshared_la_LIBADD = $(prefix)/lib/libsodium.la
# executable
inst_PROGRAMS = testexe
instdir=$(prefix)/bin
testexe_SOURCES = \
src/example_exe.c
testexe_CFLAGS = -DBUILDING_EXAMPLE_DLL -O2 -Wno-unused-parameter
testexe_CPPFLAGS = -I$(top_srcdir)/src
testexe_LDADD = libshared.la
How I invoke configure and make (build_win.sh):
export HOST=x86_64-w64-mingw32
CXX=x86_64-w64-mingw32-g++-posix
CC=x86_64-w64-mingw32-gcc-posix
PREFIX="$(pwd)/depends/$HOST"
set -eu -o pipefail
set -x
cd depends/ && make HOST=$HOST V=1 NO_QT=1
cd ../
./autogen.sh
CONFIG_SITE=$PWD/depends/x86_64-w64-mingw32/share/config.site ./configure --prefix="${PREFIX}" --host=x86_64-w64-mingw32
CC="${CC} -g " CXX="${CXX} -g " make V=1
configure.ac params is trivial
The only way I managed to build it was if I manually moved libsodium.la into the root of the distribution and 'libsodium.a' in the '.libs' subdirectory and manually corrected libsodium.la parameters 'libdir to '' (was '/home/ubuntu/repo/test-dll/depends/x86_64-w64-mingw32/share/../lib/libsodium.a') and 'installed' to 'no' (was 'yes').
So my questions are:
Why is it trying to link libsodium as a shared library when it is in 'depends' directory and set as 'installed=yes' (I need to link it as static)?
Could I have correct automake settings for MINGW to link my shared library to static libs in 'depends' directory?
Just found a solution (or workaround) how to make this link on Mingw (and produce dll):
instead of libshared_la_LIBADD = -lsodium that causes libtool to search for the libsodium shared library, I just passed an option -Wl,-lsodium to the linker in libshared_la_CFLAGS.
Platforms tested: Linux Mint 17, Ubuntu 14.04
Full example: this repository.
What am I doing and why?
I'm trying to build an R package which wraps a subset of the CAF library using Rcpp and RcppEigen.
I have successfully linked an R package to a system level install of CAF (example here) (Note: ABSEIR no longer uses CAF, 2/5/2015), but I'd like to have a good way to deploy CAF to machines without administrator access, and as a result simplify the installation of other CAF dependent packages (yes, I'm aware that R doesn't directly support linking against other compiled packages, but it seems like others have successfully circumvented this limitation).
What's the problem?
I'm building two shared objects during package compilation in addition to the package shared object (RcppCAF.so): libcaf_core.so and libcaf_io.so. These are successfully compiled and linked, but the package fails to load, claiming:
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/home/grantbrown/R/x86_64-pc-linux-gnu-library/3.2/RcppCAF/libs/RcppCAF.so':
librcaf_core.so: cannot open shared object file: No such file or directory
What have I tried?
In addition to numerous failed Makevars configurations, I've found that
if I manually set LD_LIBRARY_PATH to the folder containing the compiled code, the package will successfully install. Obviously, I'd like to avoid this step by finding a way to tell R where to look for these dependencies. I have attempted to use the inst folder for this purpose to no effect. My Makevars file is as follows:
ROOT_DIR := $(abspath .)
$(info The compilation root directory is: $(ROOT_DIR))
$(info The name of the shared library to be created is: $(SHLIB))
$(info The place R should look for librcaf_core.so is: $(abspath ./libcaf_core))
$(info The place R should look for librcaf_io.so is: $(abspath ./libcaf_io))
SOURCES = $(wildcard ./*.cpp)
SOURCES1 = $(wildcard ./libcaf_core/*.cpp)
SOURCES2 = $(wildcard ./libcaf_io/*.cpp)
OBJECTS = $(SOURCES:.cpp=.o)
OBJECTS1 = $(SOURCES1:.cpp=.o)
OBJECTS2 = $(SOURCES2:.cpp=.o)
PKG_CPPFLAGS+= -std=c++11 -Dlibcaf_core_shared_EXPORTS -Wall -pedantic -pthread -fPIC -O2 -g -fPIC -I../inst -I../inst/libcaf_core -I../inst/libcaf_io
PKG_LIBS += -L$(abspath ./libcaf_core) -lrcaf_core -L$(abspath ./libcaf_io) -lrcaf_io
all: $(SHLIB)
$(SHLIB): $(OBJECTS) libcaf_core/librcaf_core.so libcaf_io/librcaf_io.so
libcaf_core/librcaf_core.so: $(OBJECTS1)
g++ -o libcaf_core/librcaf_core.so $(OBJECTS1) $(PKG_CPPFLAGS) -shared
libcaf_io/librcaf_io.so: $(OBJECTS2)
g++ -o libcaf_io/librcaf_io.so $(OBJECTS2) $(PKG_CPPFLAGS) -shared
There are a lot of threads on StackOverflow and mailing lists which deal with problems loading shared objects, but I couldn't find anyone with precisely the same issue. I've even done similar things in the past with no issues, so I'm having trouble figuring out why R can't find my shared objects. Any suggestions?
Edit
Dirk has suggested compiling to a single shared object, which I'm now working on. In the "Using Makevars" section of "Writing R Extensions", however, it does seems to imply that building dependencies should be possible:
"If you want to create and then link to a library, say using code in a subdirectory, use something like
.PHONY: all mylibs
all: $(SHLIB)
$(SHLIB): mylibs
mylibs:
(cd subdir; make)
"
The easiest approach, which you also seem to have found as per your most recent commit is to just have R 'glue' all object code into a single shared library.
That tends to "just work" but it is a little costly as the library needs to be rebuilt. We could look into packaging CAF as an external library which would make RcppCAF more lightweight.
I am trying to link a compiled research experiment project, built in C/C++.
The project is dependant on HyperNEAT and robot simulation software WeBots.
I have cloned and built the HyperNEAT project successfully (in that project there are other dependancies such as Boost, TinyXML, JGTL (custom library) and other unrelated subprojects).
I have made a makefile including all neccesary header search paths and library paths, and compiling the two main .cpp files:
/ModHyperNEAT/mod_ctrler7.cpp
/ModSupervisor/mod_supervisor.cpp
works, giving me 2 .o files.
However, in the make link step, when I want to create (separate) executables of both files, I am getting the 'undefined symbols for architecture x86_64' error (see pastebin here: http://pastebin.com/kiwwCcUf). It seems that C++ standard datatypes and functions such as
std::string::end() const cannot be found.
I have googled and searched SO for answers regarding this, and it seems that either libraries are missing or binary incompatible if i understand correctly, but the libraries are there and both projects have been compiled with the -lstdc++ flag.
This is the make link step (and the used macro's from the makefile) :
CC = gcc
CFLAGS = -v -g -lstdc++ -Wall -Wno-error -ferror-limit=100 -fmessage-length=0
DEFINES = -DHCUBE_NOGUI -DTIXML_USE_STL
FLAGS = $(CFLAGS) $(DEFINES)
LIB_TINYXML = -L/Users/michahell/Documents/projects_c++/HyperNEAT/tinyxmldll/out
LIB_HYPERNEAT = -L/Users/michahell/Documents/projects_c++/HyperNEAT/NE/HyperNEAT/out
LIB_BOOST = -L/usr/local/Cellar/boost/1.57.0/lib
LIB_WEBOTS = -I/Applications/Webots/lib
LIBS = $(LIB_TINYXML) $(LIB_HYPERNEAT) $(LIB_BOOST) $(LIB_WEBOTS)
LIBFLAGS = -ltinyxmlpluslib -lboost_filesystem-mt -lboost_random-mt -lboost_system-mt -lNEATLib_d -lHypercube_NEAT_Base_d
WEBOTS_DYLIB = -dylib_file /Applications/Webots/lib/libController.dylib:/Applications/Webots/lib/libController.dylib
$(CC) $(FLAGS) $(LIBS) ./mod_ctrler7.o $(WEBOTS_DYLIB) $(LIBFLAGS)
I found out that to link to .dylib's I had to use a specific flag and specify the full path, hence the $(WEBOTS_DYLIB) macro.
I am using the -lstdc++ flag because in the HyperNEAT project that flag was also used for building that library. If i exclude this flag, i get a lot of errors during compilation (libc++ and libstdc++ incompatibility as I now understand). All of the library paths check out, and .a and/or .dylib files are present.
My knowledge of C/C++ and GCC tooling is very limited, as I have never had to use it before.
I think it might have to do with the fact that the HyperNEAT project contains a Boost 1.57.0 distribution which is used for their build, and that i have a separate (using homebrew) Boost version installed on my system, which is the same version:
$ brew info boost
boost: stable 1.57.0 (bottled), HEAD
http://www.boost.org
/usr/local/Cellar/boost/1.57.0 (10572 files, 439M) *
What could be the cause of this error failing my link step? Anyone should be able to reproduce my linker errors if both HyperNEAT and my project are cloned and put their root dirs in the same location. WeBots should be downloaded but only for the header includes and libraries. And of course my makefile paths should be modified.
If anyone can give me tips on how to solve this problem, i would GREATLY appreciate it!
It turns out that, for some reason, I had to include the lstdc++ flag to the library link flags and not as a compiler flag, AND the stdlib=libstdc++ as compiler flag.
I'm trying to build simplest Boost.Asio tutorial example "timer1" (it's in timer.cpp) with waf on Debian squeeze, python 2.6.6.
root#ds:/var/timer# ls
timer.cpp wscript
wscript here is a config of waf:
#! /usr/bin/env python
top = '.'
out = '.'
def options(opt):
opt.load('compiler_cxx')
def configure(conf):
conf.load('compiler_cxx')
conf.env['LINKFLAGS'] = '--verbose -L/usr/local/lib -lboost_system'
def build(bld):
tgen = bld.new_task_gen()
tgen.features = 'cxx cxxprogram'
tgen.source = 'timer.cpp'
tgen.target = 'timer'
tgen.includes = '.'
tgen.update_outputs = True
waf configure is successfull.
But waf --verbose build finishes with error (I inserted <*> below to mark a line)
Waf: Entering directory `/var/timer'
[1/2] cxx: timer.cpp -> timer.cpp.1.o
[2/2] cxxprogram: timer.cpp.1.o -> timer
timer.cpp.1.o: In function `__static_initialization_and_destruction_0(int, int)':
timer.cpp:(.text+0x103): undefined reference to `boost::system::generic_category()'
timer.cpp:(.text+0x10f): undefined reference to `boost::system::generic_category()'
...
Build failed
-> task in 'timer' failed (exit status 1):
{task 38446736: cxxprogram timer.cpp.1.o -> timer}
<*>
['/usr/bin/g++', '--verbose -L/usr/local/lib -lboost_system', 'timer.cpp.1.o', '-o', '/var/timer/timer', '-Wl,-Bstatic', '-Wl,-Bdynamic']
It seems gcc called by waf didn't find boost_system library during linkage.
But everything works fine if I build example by gcc without waf.
root#ds:/var/timer# /usr/bin/g++ --verbose -I/var/flake/lib/boost timer.cpp -c -o timer.cpp.1.o
...
<**>
root#ds:/var/timer# /usr/bin/g++ --verbose -L/usr/local/lib -lboost_system timer.cpp.1.o -o timer -Wl,-Bstatic -Wl,-Bdynamic
...
root#ds:/var/timer# ls
timer.cpp timer.cpp.1.o timer wscript
As you can see command line used by waf (marked by <*> ) is identical with command line marked by <**>. But the result is completely different. Why? And how can I force waf to build that thing? Solution from here doesn't work for me too. Also tried
...
opt.tool_options('boost')
...
conf.load('compiler_cxx boost')
conf.check_boost()
...
tgen.uselib = 'BOOST'
...
but without any effect
And another question. Output of gcc --verbose is much more extensive than output of waf --verbose. It seemed to me that verbose option must force waf to display all such info. Why isn't that true? waf option -vvv doesn't display this info as well.
If you use the boost tool, you should check the waf boost example in playground/boost, and you will see that check_boost takes arguments. It will solve your issue.
Alternatively if you don't use the tool, and using a real OS where libraries are set up in an accessible folder, you can just do:
conf.env.LIB_BOOST = ['boost_system']
...
bld(
...,
use='BOOST',
...,
)
Note for your future scripts:
avoid using CFLAGS, CXXFLAGS, LINKFLAGS as it's not portable.
don't set out='.' if possible
use bld(...) vs. the old syntax
The reason was that older Boost 1.42 binaries bundled with Debian were in /usr/lib. When I found that I tried to make /usr/local/lib more prioritive (latest Boost binaries built by b2 reside there). But I couldn't make waf to do that and than just uninstalled Boost 1.42 binaries completely and everything worked after that
I'm trying to create a shared object (.so) that will make it so, by including one shared object with -lboost, I implicitly include all the boost libraries. Here's what I tried:
#!/bin/sh
BOOST_LIBS="-lboost_date_time-gcc43-mt -lboost_filesystem-gcc43-mt"
#truncated for brevity
g++ $BOOST_LIBS -shared -Wl,-soname,libboost.so.1 -o libboost.so.1.0
ln -si libboost.so.1.0 libboost.so.1
ln -si libboost.so.1 libboost.so
After placing all 3 created files (libboost.so libboost.so.1 libboost.so.1.0) in the same directory as all the boost libraries, I tried compiling a test program with it (which depends on -lboost_date_time-gcc43-mt):
g++ -lboost test.cpp
Doing this, I got the same undefined reference message as not having -lboost. Having -lboost_date_time-gcc43-mt works, but that's too wordy :) How do I get -lboost to automatically bring in the other shared libraries?
You don't. Not really, anyway.
The linker is stripping out all of the symbol dependencies because the .so doesn't use them.
You can get around this, perhaps, by writing a linker script that declares all of the symbols you need as EXTERN() dependencies. But this implies that you'll need to list all of the mangled names for the symbols you need. Not at all worth the effort, IMO.
I don't have a solution for creating a dummy '.so', but I do have something that will simplify your life... I highly suggest that you try using cross-platform make (CMake). In CMake, linking against those libraries is easy:
FIND_PACKAGE(Boost 1.37 COMPONENTS date_time filesystem REQUIRED)
ADD_EXECUTABLE(myexecutable ${myexecutable_SRCS})
TARGET_LINK_LIBRARIES(myexecutable ${Boost_LIBRARIES})
The commands above, if placed in a "CMakeLists.txt" file, is all you need to:
Verify that Boost 1.37 or later is installed, with the "date_time" and "filesystem" libraries installed.
Create an executable named "myexecutable" from the sources listed in the corresponding variable.
Link the executable "myexecutable" against the boost "date_time" and "filesystem" libraries.
See also: Why the KDE project switched to CMake.
Actually, making one .so depend on all boost .so files is quite possible (but might not actually help you). I've just tried this:
$ export BOOST_ROOT=/home/ghost/Work/Boost/boost-svn
$ g++ -shared -Wl,-soname,libboost.so -o libboost.so $BOOST_ROOT/stage/lib/libboost_program_options.so
$ g++ -L . -I $BOOST_ROOT first.cpp -lboost -Wl,-R$BOOST_ROOT/stage/lib
$ LD_LIBRARY_PATH=.:$BOOST_ROOT/stage/lib ./a.out
And it did work. However, note that dancing with -R and LD_LIBRARY_PATH. I don't know an way how you can include the path to Boost .so inside your libboost.so so that they are used both for linking and actually running the application. I can include rpath inside libboost.so just fine, but it's ignored when resolving symbols for the application.