I'm trying to build bandicoot with armadillo.
These are the flags in the makefile for the benchmark tests:
LIB_FLAGS = -lOpenCL -lnvrtc -lcudart -lcublas -lcuda -lcurand -lcusolver -L /home/charles/Downloads/clBLAS-2.12.0-Linux-x64/lib64 -lclBLAS
#CXX_FLAGS = -std=c++11 -Wshadow -Wall -pedantic -g -I../include/ -O0 -DCOOT_USE_U64S64
CXX_FLAGS = -std=c++11 -Wshadow -Wall -pedantic -I../include/ -I /home/charles/Downloads/clBLAS-2.12.0-Linux-x64/include -I /home/charles/Downloads/OpenCL-Wrapper-master/src/OpenCL/include/CL
-DCOOT_NO_DEBUG -O3
I get errors when I change the config.hpp file for armadillo:
#define ARMA_BLAS_UNDERSCORE
When I comment it out I get:
undefined reference to 'dsyrk_'
So I've not been able to build it with armadillo. Has anyone faced this problem and been able to solve it?
Related
I was building some Cython extensions, and have to link it against a static library (it has CUDA code in them, so have to be static):
running build_ext
building 'k3lib' extension
gcc -pthread -B /home/kelvin/anaconda3/envs/torch/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/kelvin/repos/tools/include -I/home/kelvin/anaconda3/envs/torch/include/python3.8 -c main.cpp -o build/temp.linux-x86_64-3.8/main.o -O3 -march=native
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
g++ -pthread -shared -B /home/kelvin/anaconda3/envs/torch/compiler_compat -L/home/kelvin/anaconda3/envs/torch/lib -Wl,-rpath=/home/kelvin/anaconda3/envs/torch/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.8/main.o /home/kelvin/repos/tools/include/libk2.a -L/home/kelvin/repos/tools/include -lk2 -o build/lib.linux-x86_64-3.8/k3lib.cpython-38-x86_64-linux-gnu.so -static -Wl,-Bstatic -flinker-output=exec
However, Cython's g++ compile command includes the options -shared -fPIC by default. I tried a number of options at the end of the command via this setup file (the static library is at $(LOCAL_INCLUDE)/libk2.a):
includes = [os.getenv("LOCAL_INCLUDE")]
ext_modules = [
Extension("k3lib", sources=["main.pyx"],
libraries=["k2"], include_dirs=includes, library_dirs=includes, language="c++",
extra_compile_args=["-O3", "-march=native"], extra_objects=[f"{includes[0]}/libk2.a"],
extra_link_args=['-static', '-Wl,-Bstatic', '-flinker-output=exec'])
]
#extra_objects=[f"{includes[0]}/libk2.a"]
#extra_link_args=['-static']
setup(name="k3lib", ext_modules=cythonize(ext_modules, language_level="3"))
Still, g++ thinks that I want to build a shared library, and thus the error message. Is there a way to override the -shared option? I'm planning to go into Cython's files and edit them myself, but was wondering is there a simpler way?
Context: I was following this question on SO but can't replicate their success.
After reading this question, I wrote a Makefile that starts with
CXX=g++
CXXFLAGS= -std=c++17 -Wall -O3 -g
ifeq ( $( shell uname ), "Linux" )
CXXFLAGS += -fopenmp
endif
LIBS= -pthread
INCLUDES = -I.
TARGETS= my targets...
I need to pass the -fopenmp flag only if I'm compiling on linux and not when I'm compiling on Mac.
My problem is that this don't work and the flag never get passed.
gmake's syntax is quite sensitive to whitespace (especially gmake macros). Additionally, the output of uname does not include quotes.
This should be:
CXX=g++
CXXFLAGS= -std=c++17 -Wall -O3 -g
ifeq ($(shell uname),Linux)
CXXFLAGS += -fopenmp
endif
zz:
echo $(CXXFLAGS)
Result:
$ make zz
echo -std=c++17 -Wall -O3 -g -fopenmp
-std=c++17 -Wall -O3 -g -fopenmp
I have a package I'm writing in R that has a boatload of complicated C++ code which takes a while to compile.
When I change the code, I'd like to quickly rebuild the package so I can test it. However, R CMD build seems to start from scratch each time, rather than using my code's makefiles to do only what is needed.
Is there a way to quickly do repeated builds of a package in R for testing?
I am a little overdue a short blog post on this, but I mentioned it a couple of times before: use ccache. It helps dramatically when the files don't change (ie when you just alter help pages), or when few files changes. Caching is a very clever trick, and package is robust.
On Ubuntu/Debian: sudo apt-get install ccache followed by e.g. this in your ~/.R/Makevars:
VER=
CCACHE=ccache
#CCACHE=
CC=$(CCACHE) gcc$(VER)
CXX=$(CCACHE) g++$(VER)
CXX11=$(CCACHE) g++$(VER)
CXX14=$(CCACHE) g++$(VER)
That also allows to switch between g++ versions. Changing it to clang++ is left as an exercise to the reader ;-)
Besides this, see the options to R CMD build and R CMD INSTALL to skip vignette and/or manual building to further speed up re-builds.
Illustration: Here is re-install of Rcpp itself (fresh from a git pull) where the first installation takes 21.9 seconds on my (decent) machine at work, the second then only takes 1.4 seconds thanks to ccache:
~/git/rcpp(master)$ time R CMD INSTALL .
* installing to library ‘/usr/local/lib/R/site-library’
* installing *source* package ‘Rcpp’ ...
** libs
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c Date.cpp -o Date.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c Module.cpp -o Module.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c Rcpp_init.cpp -o Rcpp_init.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c api.cpp -o api.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c attributes.cpp -o attributes.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c barrier.cpp -o barrier.o
g++ -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o Rcpp.so Date.o Module.o Rcpp_init.o api.o attributes.o barrier.o -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/Rcpp/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (Rcpp)
real 0m21.917s
user 0m21.388s
sys 0m2.304s
~/git/rcpp(master)$ ./cleanup
~/git/rcpp(master)$ time R CMD INSTALL .
* installing to library ‘/usr/local/lib/R/site-library’
* installing *source* package ‘Rcpp’ ...
** libs
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c Date.cpp -o Date.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c Module.cpp -o Module.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c Rcpp_init.cpp -o Rcpp_init.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c api.cpp -o api.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c attributes.cpp -o attributes.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c barrier.cpp -o barrier.o
g++ -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o Rcpp.so Date.o Module.o Rcpp_init.o api.o attributes.o barrier.o -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/Rcpp/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (Rcpp)
real 0m1.444s
user 0m1.380s
sys 0m1.452s
~/git/rcpp(master)$
Dirk's answer above pointed me in the right direction, but was insufficient. Since he has requested that I not append the final steps I required, I do so here.
My question could not have been answered without the following.
Dirk's answer did not work off the bat for me. If compilation still seems slow to you, try running:
ccache -s
The result will look a little like this
cache directory /home/myuser/.ccache
primary config /home/myuser/.ccache/ccache.conf
secondary config (readonly) /etc/ccache.conf
cache hit (direct) 0
cache hit (preprocessed) 0
cache miss 989
cache hit rate 0 %
called for link 12
preprocessor error 12
cleanups performed 16
files in cache 177
cache size 31.1 MB
max cache size 5.0 GB
Note that cache isn't getting hit, which means ccache isn't doing anything.
You can use:
export CCACHE_LOGFILE=ccache.log
R CMD build .
to do some debugging, though this wasn't helpful for me.
What fixed things was to run:
export CCACHE_NOHASHDIR=true
R CMD build .
As it turns out, ccache sometimes takes the location of a file into account. R CMD build . appears to build in a temporary directory, so the location of the files was changing each time.
I would like to set the C++ compiler flag to -O0 in the Makevars of an Rcpp project.
If I take a look at /etc/R/Makeconf, I see that the compilation command seems to be
$(CXX) $(ALL_CPPFLAGS) $(ALL_CXXFLAGS) -c $< -o $#
Since
ALL_CXXFLAGS = $(R_XTRA_CXXFLAGS) $(PKG_CXXFLAGS) $(CXXPICFLAGS) $(SHLIB_CXXFLAGS) $(CXXFLAGS)
I can edit in the Makevars the variable $(PKG_CXXFLAGS) to add headers for specific libraries, but I am not satisfied with CXXFLAGS = -O3 -pipe -g $(LTO). I would also like to be able to do that directly in the Makevars, so as to tune each project according to my needs.
When I edit CXXFLAGS in the Makevar, nothing happens. Is is possible to adjust that variable ? Is another approach possible ? I know that I can edit ~/.R/Makevars, and switch as requested. I wondered if there was a more robust approach.
You generally want the PKG_* variants in your local file, e.g. ~/.R/Makevars.
Here is a (shortened, edited) portion of mine:
## for C code
CFLAGS= -O3 -g0 -Wall -pipe -pedantic -std=gnu99
## for C++ code
#CXXFLAGS= -g -O3 -Wall -pipe -Wno-unused -pedantic -std=c++11
CXXFLAGS= -g -O3 -Wall -pipe -Wno-unused -pedantic
## for Fortran code
#FFLAGS=-g -O3 -Wall -pipe
FFLAGS=-O3 -g0 -Wall -pipe
## for Fortran 95 code
#FCFLAGS=-g -O3 -Wall -pipe
FCFLAGS=-O3 -g0 -Wall -pipe
VER=-4.8
CC=ccache gcc$(VER)
CXX=ccache g++$(VER)
SHLIB_CXXLD=g++$(VER)
FC=ccache gfortran
F77=ccache gfortran
MAKE=make -j8
The other (system-global) approach is to create and edit /etc/R/Makeconf.site (or, when /etc/R/ does not exist, $RHOME/etc/R/Makeconf.site.
When compiling my code I run into an issue as follows:
io.cpp:21: undefined reference to `PQconnectdb'
as well as all other instances of missing postgres function calls occurring in my code. Obviously this is a linking problem, I'm just not sure what the link issue is.
I'm compiling with the following:
mpiCC -c -O2 -g -Wall -Werror -I /usr/include/postgresql/ decisioning_mpi.cpp
g++ -c -O2 -g -Wall -Werror -I /usr/include/postgresql/ io.cpp
g++ -c -O2 -g -Wall -Werror -I /usr/include/postgresql/ calculations.cpp
g++ -c -O2 -g -Wall -Werror -I /usr/include/postgresql/ rules.cpp
g++ -c -O2 -g -Wall -Werror -I /usr/include/postgresql/ Instrument.cpp
g++ -c -O2 -g -Wall -Werror -I /usr/include/postgresql/ Backtest_Parameter_CPO.cpp
g++ -c -O2 -g -Wall -Werror -I /usr/include/postgresql/ Backtest_Trade_CPO.cpp
g++ -c -O2 -g -Wall -Werror -I /usr/include/postgresql/ Data_Bar.cpp
mpiCC -o decisioning_mpi -O2 -g -Wall -Werror -L/usr/lib -lm -lpq decisioning_mpi.o
io.o calculations.o rules.o Instrument.o Backtest_Parameter_CPO.o Backtest_Trade_CPO.o Data_Bar.o
It should be noted that this is the correct directory for libpq-fe.h and that I'm linking pq, so I'm not exactly sure why the postgres functions aren't linking correctly. I'm running Ubuntu 12.04 and installed psql (PostgreSQL) 9.1.6 from synaptic. As well I'll short circuit this, I am using #include "libpq-fe.h".
Any ideas on how I can get this linking issue resolved?
put -L/usr/lib/ -lm -lpq in the end of link command, the linker can then find the symbols
mpiCC -o decisioning_mpi -O2 -g -Wall -Werror decisioning_mpi.o io.o \
calculations.o rules.o Instrument.o Backtest_Parameter_CPO.o \
Backtest_Trade_CPO.o Data_Bar.o -L/usr/lib -lm -lpq
GCC Link Reference:
http://www.network-theory.co.uk/docs/gccintro/gccintro_18.html