Creating dynamic library on Mac OSX using clang++ - c++

I wanted to learn how to make a shared library in C++. So I found some logic to generate Mandelbrot PPM images, and encapsulated it in src/mandelbrot.cpp (with accompanying header file include/mandelbrot.cpp). Directory tree looks like:
$ tree
.
├── Makefile
├── include
│ └── mandelbrot.h
├── lib
│ └── mandelbrot.dylib
└── src
├── client.cpp
├── main
├── main.cpp
├── mandelbrot.cpp
└── mandelbrot.o
The goal is for src/client.cpp to use lib/mandelbrot.dylib to draw the fractal without having access to src/mandelbrot.cpp or src/mandelbrot.o.
Makefile:
.PHONY=clean
main: src/mandelbrot.o
clang++ src/main.cpp -o src/main src/mandelbrot.o -I ./include
src/mandelbrot.o:
clang++ -c src/mandelbrot.cpp -o src/mandelbrot.o -I ./include
clean:
rm src/*.o
rm lib/*.dylib
lib/mandelbrot.dylib:
clang++ -dynamiclib -o lib/mandelbrot.dylib src/mandelbrot.cpp -I ./include
src/client: lib/mandelbrot.dylib
clang++ src/client.cpp -o src/client -L ./lib -I ./include
Running the executable without the dylib works:
$ make main
clang++ src/main.cpp -o src/main src/mandelbrot.o -I ./include
$ ./src/main # runs fine!
But I can't get my shared library to link when compiling src/client.cpp:
$ make src/client
clang++ src/client.cpp -o src/client -L ./lib -I ./include
Undefined symbols for architecture x86_64:
"fractal::Mandelbrot::writeToFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)", referenced from:
_main in client-e344c7.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [src/client] Error 1
Even though the symbol seems to be in the dylib:
$ file lib/mandelbrot.dylib
lib/mandelbrot.dylib: Mach-O 64-bit dynamically linked shared library x86_64
$ nm -C lib/mandelbrot.dylib | grep writeToFile
0000000000001460 T fractal::Mandelbrot::writeToFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)
I'm trying to dig in deeper and understand the linking/compilation process. How do I make the setup I have work to demonstrate what I want to do?

Try naming your library like libmandelbrot.dylib to match the -lmandelbrot flag

Credit goes to #Daniel for finding the answer, but a bit about why.
I can make the compilation work by renaming mandelbrot.dylib -> libmandelbrot.dylib as suggested by #daniel's answer.
I dug into man ld (the linker) and found the following definition:
-lx This option tells the linker to search for libx.dylib or libx.a in the library search path. If string x is of the form y.o, then that file is searched for in the same places, but without prepending
`lib' or appending `.a' or `.dylib' to the filename.
Thus, if you want to arbitrarily name your dynamically loaded libraries (and for some reason don't like the magic of prepending libxxxx to the name, you'd have to do this (I have confirmed this works):
lib/whatever.o:
clang++ -dynamiclib -o lib/whatever.o src/mandelbrot.cpp -I ./include
src/client: lib/whatever.o
clang++ src/client.cpp -o src/client -L ./lib -I ./include -l whatever.o
TL;DR = By having the -l option as a filename ending in .o, the linker looks directly in the directory specified by -L for the exact filename specified in -l. Otherwise -l foo -L /path/ searches for /path/libfoo.[dylib|a]

Related

G++ stops working when used in makefile

I'm currently trying to use a makefile to automate my build process which I am still new to. When I run the g++ command:
g++ -I includes -L lib -lSDL2_test -lSDL2 -lSDL2main main.o -o main.exe
from the following makefile:
main: main.o
g++ -I includes -L lib -lSDL2_test -lSDL2 -lSDL2main main.o -o main.exe
main.o: main.cpp
g++ -c main.cpp
I get clang: error: linker command failed with exit code 1
However when running the same g++ command from the same directory as the makefile everything compiles correctly and I don't get the error. So how do I fix this?
EDIT:
I've done some testing and found out the problem with the makefile is that the -I includes g++ flag is not enabling the main.o file to find SDL.H, hopefully that helps to narrow down the problem.
#G.M's comment isn't unrelated - it's the correct answer to the problem. The include files are needed during the compile stage (the main.o: main.cpp rule), NOT during the link stage (i.e. the main: main.o rule.)
Move the -I includes flag to the second rule, and try again.

Custom shared library cannot be found even when search path defined

I have the following files:
.
├── combinatorics
│   ├── permutation.cpp
│   └── permutation.h
├── main.cpp
└── math
├── basic.cpp
└── basic.h
and am trying to create a shared library by doing these steps:
g++ -c -Wall -std=c++14 -fpic combinatorics/permutation.cpp -o permutations.o
g++ -c -Wall -std=c++14 -fpic math/basic.cpp -o mathbasic.o
g++ -shared -o combmath.so permutations.o mathbasic.o
g++ -Wall -std=c++14 main.cpp -L. -o output -lcombmath
but I am getting the following message:
/usr/bin/ld: cannot find -lcombmath
collect2: error: ld returned 1 exit status
what am I doing wrong?
The linkage option -lfoo directs the linker to search for a file
libfoo.so (shared library) or libfoo.a (static library) in the specified
linker search directories (-L/x/y/z) and then in the linker's default
search directories, stopping when it finds either one of those files and
preferring libfoo.so if it finds both in the same directory.
For your linkage:
g++ -Wall -std=c++14 main.cpp -L. -o output -lcombmath
to succeed, therefore, you need previously to have built libcombmath.so
in the same directory with the command:
g++ -shared -o libcombmath.so permutations.o mathbasic.o
and not built combmath.so, as you have done with the command:
g++ -shared -o combmath.so permutations.o mathbasic.o

Linker command - No rule to make target for 3rd party lib

In my code I make reference this pugixml like this:
#include "pugi/pugixml.hpp"
When compiling I get this error:
main in main-bf0b72.o
"pugi::xml_node::children(char const*) const", referenced from:
_main in main-bf0b72.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [main] Error 1
From another question I was told to pugi as an additional translation unit and link them accordingly like this in my Makefile:
CC = g++
CFLAGS = -g -Wall -std=c++11
SRC = src
INCLUDES = include
TARGET = main
all: $(TARGET)
$(TARGET): $(SRC)/$(TARGET).cpp pugi/pugixml.cpp
$(CC) $(CFLAGS) -I $(INCLUDES) -o $# $^
clean:
$(RM) $(TARGET)
However when I try to run my Makefile after this change I get this error:
make: *** No rule to make target `pugi/pugixml.cpp', needed by `main'. Stop.
I'm really not sure what I am meant to be doing. Should pugi have it's own Makefile to build it individually, or should I give it is't own "target" in my Makefile?
Edit
This is my file system:
root/Makefile
root/src/main.cpp
root/include/pugi/pugixml.hpp
root/include/pugi/pugixml.cpp
root/include/pugi/pugiconfig.hpp
In your Makefile you try to compile pugi/pugixml.cpp file, but on filesystem it located in include/pugi/pugixml.cpp (relatively to Makefile itself). You should:
Create root/pugi directory and put pugixml.cpp file there
OR
Or replace pugi/pugixml.cpp with include/pugi/pugixml.cpp in your Makefile (but keeping source files in include/ subdirectory is bad idea).

Makefile fails to see main.o one out of two times

Ok, I'm trying to build a makefile in order to compile a program on different environments. On Linux (Mint) it works just fine, but under Mac OS I get this error:
Undefined symbols for architecture x86_64:
"_main", referenced from:
implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [build] Error 1
With the '-v':
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.1.0
Thread model: posix
"/Library/Developer/CommandLineTools/usr/bin/ld" -demangle -dynamic -arch x86_64 -macosx_version_min 10.9.0 -w -o bin/Darwin/main ./cfprintf.o ./file.o ./inputmanager.o ./neditfile.o ./neditmanager.o ./neditscreen.o ./n_string.o ./sys_booter.o ./sys_display.o ./sys_manager.o ./sys_screen.o -lc++ -lSystem /Library/Developer/CommandLineTools/usr/bin/../lib/clang/5.1/lib/darwin/libclang_rt.osx.a
Undefined symbols for architecture x86_64:
"_main", referenced from:
implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [build] Error 1
Now here's the makefile:
CC = clang++
LIB =
INC = -I src/
OPTIONS = -w
OBJS = $(shell find ./src/obj -name '*.cpp')
IO = $(shell find ./src/io -name '*.cpp')
SYS = $(shell find ./src/sys -name '*.cpp')
APPS = $(shell find ./src/apps -name '*.cpp')
EXEC = bin/$(shell uname)/main
.PHONY: dummy build all objects io sys apps incBC clear run
build: clear incBC objects io sys apps
#echo "-> Compiling main... "
#$(CC) -c src/main.cpp $(INC) $(OPTIONS)
#echo "-> Building environment... "
#$(CC) $(shell find . -name '*.o') $(LIB) $(OPTIONS) -o $(EXEC) -v
#echo "-> Removing all objects file from compilation... "
#rm *.o
objects:
#echo "-> Building objects files and classes... "
#$(CC) -c $(OBJS) $(INC) $(OPTIONS)
io:
#echo "-> Building input/output files and classes..."
#$(CC) -c $(IO) $(INC) $(OPTIONS)
sys:
#echo "-> Building system files and classes..."
#$(CC) -c $(SYS) $(INC) $(OPTIONS)
apps:
#echo "-> Building all apps files and classes..."
#$(CC) -c $(APPS) $(INC) $(OPTIONS)
incBC:
#echo "-> Updated build count..."
#./incBC.sh
clear:
clear
run:
cd bin/$(shell uname)/ && clear && ./main
Now what's (al least to me) really weird, is that when I type make it fails one out of two times... LITERALLY one out of two! And when it does fail, I get the error message above. What I've noticed from that message is that, in the ld command, there is no main.o which is probably why I get the undefined reference. My question is: why? Why specifically one out of two times, and how can I fix this?
This is a bizarre makefile. Why not just write a shell script? There's nothing in this makefile that takes any advantage of make itself: it will always rebuild everything every time. The point of writing a makefile, instead of a shell script, is so you can avoid rebuilding objects that haven't changes.
Anyway, the reason you're seeing failures is that you're using make's $(shell ...) function inside of a recipe. This is not necessary, because a recipe is already running in a shell. The reason this fails for you is that make will expand the entire recipe (all lines) first, before it starts your recipe (it has to do this obviously). That means that because you're using $(shell ...) that find command is run before the recipe runs, and before the recipe runs you haven't compiled your main.o, so the find command doesn't include it in the list of object files found. Then the next time you run it, main.o exists from the previous run so the find command finds it.
Change this line:
#$(CC) $(shell find . -name '*.o') $(LIB) $(OPTIONS) -o $(EXEC) -v
to this:
#$(CC) `find . -name '*.o'` $(LIB) $(OPTIONS) -o $(EXEC) -v
and it will work.
Although it's still not very useful as a makefile.

boost and cpp-netlib make compile ERROR

[SOLVED] created symlinks from /usr/lib/lib/* to /usr/lib*
[UPDATE 3] NEW VERSION:
Ok, I think I fixed something
use find / -name "libboost_system.*"
outout was
/usr/include/boost/lib/libboost_system.so
/usr/include/boost/lib/libboost_system.a
/usr/include/boost/lib/libboost_system.so.1.46.1
/usr/lib/lib/libboost_system.so
/usr/lib/lib/libboost_system.a
/usr/lib/lib/libboost_system.so.1.46.1
/usr/local/include/boost_1_46_1/bin.v2/libs/system/build/gcc-4.4.3/release/link-static/threading-multi/libboost_system.a
/usr/local/include/boost_1_46_1/bin.v2/libs/system/build/gcc-4.4.3/release/threading-multi/libboost_system.so.1.46.1
/usr/local/lib/libboost_system.so
/usr/local/lib/libboost_system.a
/usr/local/lib/libboost_system.so.1.46.1
/root/tmp/boost_1_46_1/bin.v2/libs/system/build/gcc-4.4.3/release/link-static/threading-multi/libboost_system.a
/root/tmp/boost_1_46_1/bin.v2/libs/system/build/gcc-4.4.3/release/threading-multi/libboost_system.so.1.46.1
why are these files in /usr/lib/lib ? and is it a problem ?
and the ls -l /usr/lib/lib | grep boost_system
ls -l /usr/lib/lib | grep boost_system
-rw-r--r-- 1 root root 21574 2011-05-09 15:15 libboost_system.a
lrwxrwxrwx 1 root root 25 2011-05-09 15:15 libboost_system.so -> libboost_system.so.1.46.1
-rwxr-xr-x 1 root root 20053 2011-05-09 15:15 libboost_system.so.1.46.1
atm my makefile looks like
LIBPATH=-I/usr/local/include/cpp-netlib
LIBS=$(LIBPATH) -lboost_system -lboost_filesystem -lboost_thread -lpthread
LD=g++ -g
CPP=g++ -c -g $(LIBS)
P=.
OBJ=$(P)/tmp/main.o $(P)/tmp/CLink.o $(P)/tmp/CFetcher.o
main: $(OBJ); $(LD) $(OBJ) $(LIBS) -o $#
$(P)/tmp/CLink.o: $(P)/src/CLink.cpp $(P)/include/CLink.h; $(CPP) -c $< -o $#
$(P)/tmp/CFetcher.o: $(P)/src/CFetcher.cpp $(P)/include/CFetcher.h; $(CPP) -c $< -o $#
$(P)/tmp/main.o: $(P)/src/main.cpp $(P)/include/CLink.h $(P)/include/CFetcher.h ; $(CPP) -c $< -o $#
all:
touch $(P)/tmp/*.o;
touch main;
rm -f $(P)/tmp/*.o;
rm -f main;
make main;
The Compiler output is lie
g++ -c -g -I/usr/local/include/cpp-netlib -lboost_system -lboost_filesystem -lboost_thread -lpthread -c src/main.cpp -o tmp/main.o
g++ -c -g -I/usr/local/include/cpp-netlib -lboost_system -lboost_filesystem -lboost_thread -lpthread -c src/CLink.cpp -o tmp/CLink.o
g++ -c -g -I/usr/local/include/cpp-netlib -lboost_system -lboost_filesystem -lboost_thread -lpthread -c src/CFetcher.cpp -o tmp/CFetcher.o
g++ -g ./tmp/main.o ./tmp/CLink.o ./tmp/CFetcher.o -I/usr/local/include/cpp-netlib -lboost_system -lboost_filesystem -lboost_thread -lpthread -o main
So for me all looks nice but when i try to run the program
./main
./main: error while loading shared libraries: libboost_system.so.1.46.1: cannot open shared object file: No such file or directory
The -l flags must come after the source files on linker command-line.
Yes, that means you'll have to split the LD definition to LD and LIBS, put all the -L and -l flags in the later and change the link command to:
$(LD) $(OBJ) $(LIBS) -o $#
The library (.so (dynamic) or .a (static)) files have to be the same version as the headers. While there are boost 1.46.1 headers installed in /usr/local/include/boost_1_46_1/, the corresponding library files don't seem to be installed at all. The only installed libraries are version 1.40.0 in /usr/lib, so the linker finds those (/usr/lib would be searched by default even if you didn't include the -L/usr/lib flag), but they don't contain the symbols expected by 1.46.1.
Note that when linking against shared library (using shared libraries is strongly recommended in Linux), the linker looks for the file with .so extension, but that is usually symlink to a file with added version suffix and the linker reads it and records the target name in the binary. That way programs compiled against the .1.40 will continue to work when 1.46 is installed, because the libboost*.so.1.40.0 may (and have to) stay around after the .so is redirected to the 1.46.1 version.
It should be even possible to install like:
/usr/local/lib/boost_1_46_1/libboost_system-mt.so -> /usr/local/lib/libboost_system-mt.so.1.46
/usr/local/lib/libboost_system-mt.so.1.46 -> /usr/local/lib/libboost_system-mt.so.1.46.1
/usr/local/lib/libboost_system-mt.so.1.46.1
and compile using -L/usr/local/lib/boost_1_46_1, though I currently can't find any package that would do it to confirm this. This way you could have development files for multiple versions installed and switch between them using explicit -I and -L flags while dynamic linker would still find the runtime files, for which it only looks in /usr/local/lib, /usr/lib and /lib (it can be configured in /etc/ld.so.conf, but that's the default).
Execute your application with strace. This will show the location were you app is looking for your boost libs. In my case an app was looking in /usr/lib/x86_64-linux-gnu for boost libs where the actual location was /usr/lib/lib. A simple export statement adding the paths for the boost shared libs in my case LD_LIBRARY_PATH=/usr/lib/lib worked a treat.
output from strace
open("/usr/lib/x86_64-linux-gnu/libboost_system.so.1.46.1", O_RDONLY) = -1 ENOENT (No such file or directory)
exit_group(127) = ?