I would like to set the C++ compiler flag to -O0 in the Makevars of an Rcpp project.
If I take a look at /etc/R/Makeconf, I see that the compilation command seems to be
$(CXX) $(ALL_CPPFLAGS) $(ALL_CXXFLAGS) -c $< -o $#
Since
ALL_CXXFLAGS = $(R_XTRA_CXXFLAGS) $(PKG_CXXFLAGS) $(CXXPICFLAGS) $(SHLIB_CXXFLAGS) $(CXXFLAGS)
I can edit in the Makevars the variable $(PKG_CXXFLAGS) to add headers for specific libraries, but I am not satisfied with CXXFLAGS = -O3 -pipe -g $(LTO). I would also like to be able to do that directly in the Makevars, so as to tune each project according to my needs.
When I edit CXXFLAGS in the Makevar, nothing happens. Is is possible to adjust that variable ? Is another approach possible ? I know that I can edit ~/.R/Makevars, and switch as requested. I wondered if there was a more robust approach.
You generally want the PKG_* variants in your local file, e.g. ~/.R/Makevars.
Here is a (shortened, edited) portion of mine:
## for C code
CFLAGS= -O3 -g0 -Wall -pipe -pedantic -std=gnu99
## for C++ code
#CXXFLAGS= -g -O3 -Wall -pipe -Wno-unused -pedantic -std=c++11
CXXFLAGS= -g -O3 -Wall -pipe -Wno-unused -pedantic
## for Fortran code
#FFLAGS=-g -O3 -Wall -pipe
FFLAGS=-O3 -g0 -Wall -pipe
## for Fortran 95 code
#FCFLAGS=-g -O3 -Wall -pipe
FCFLAGS=-O3 -g0 -Wall -pipe
VER=-4.8
CC=ccache gcc$(VER)
CXX=ccache g++$(VER)
SHLIB_CXXLD=g++$(VER)
FC=ccache gfortran
F77=ccache gfortran
MAKE=make -j8
The other (system-global) approach is to create and edit /etc/R/Makeconf.site (or, when /etc/R/ does not exist, $RHOME/etc/R/Makeconf.site.
Related
After reading this question, I wrote a Makefile that starts with
CXX=g++
CXXFLAGS= -std=c++17 -Wall -O3 -g
ifeq ( $( shell uname ), "Linux" )
CXXFLAGS += -fopenmp
endif
LIBS= -pthread
INCLUDES = -I.
TARGETS= my targets...
I need to pass the -fopenmp flag only if I'm compiling on linux and not when I'm compiling on Mac.
My problem is that this don't work and the flag never get passed.
gmake's syntax is quite sensitive to whitespace (especially gmake macros). Additionally, the output of uname does not include quotes.
This should be:
CXX=g++
CXXFLAGS= -std=c++17 -Wall -O3 -g
ifeq ($(shell uname),Linux)
CXXFLAGS += -fopenmp
endif
zz:
echo $(CXXFLAGS)
Result:
$ make zz
echo -std=c++17 -Wall -O3 -g -fopenmp
-std=c++17 -Wall -O3 -g -fopenmp
I am compiling an example application from the SDK repository of a third party vendor. I receive an error that one of the C++ header's (algorithm) cannot be found:
if [ ! -d .deps/ ]; then mkdir -p .deps/; fi
/opt/llvm-3.8.0/bin/clang++ -M -isystem/opt/tbricks/sdk/include64 -I../../.. -I../../../.. -I./../../../.. -DLINUX -DLINUX64 -DTB_USE_RCU -DURCU_INLINE_SMALL_FUNCTIONS -DU_USING_ICU_NAMESPACE=0 -DNDEBUG -D_POSIX_PTHREAD_SEMANTICS -fPIC -D_GNU_SOURCE -DTB_USE_RCU -DTB_USE_RCU -D_GLIBCXX_USE_CXX11_ABI=0 -m64 --gcc-toolchain=/opt/gcc-5.2.0 -flto=full -std=gnu++14 -D_GLIBCXX_DEPRECATED= -pipe -fno-omit-frame-pointer -ffast-math -fno-finite-math-only -pthread -march=core2 -mtune=corei7 -g -O3 -Qunused-arguments -fnon-call-exceptions -fvisibility=hidden -fvisibility-inlines-hidden -Wall -Wextra -Wshadow -Wpointer-arith -Wno-self-assign -Wno-unused-function -Wno-gnu-empty-initializer -Wno-unused-parameter -Wno-ignored-qualifiers -Wno-mismatched-tags -Wno-unused-local-typedef -Wno-parentheses-equality -Wno-unused-private-field -Wno-missing-field-initializers -Wno-missing-braces -Werror=return-type -Werror=overloaded-virtual -DSTRATEGY_BUILD_PROFILE=\"release\" ../../../../shared/Helpers.cpp > .deps/Helpers.o.d
../../../../shared/Helpers.cpp:14:10: fatal error: 'algorithm' file not found
#include <algorithm>
What sets the location path to search for C++ header files, such as algorithm? Is there anything I can grep for within makefiles?
Either install g++ alongside (you need libstdc++) or use LLVM libc++ and specify it with -stdlib=libc++
I have a package I'm writing in R that has a boatload of complicated C++ code which takes a while to compile.
When I change the code, I'd like to quickly rebuild the package so I can test it. However, R CMD build seems to start from scratch each time, rather than using my code's makefiles to do only what is needed.
Is there a way to quickly do repeated builds of a package in R for testing?
I am a little overdue a short blog post on this, but I mentioned it a couple of times before: use ccache. It helps dramatically when the files don't change (ie when you just alter help pages), or when few files changes. Caching is a very clever trick, and package is robust.
On Ubuntu/Debian: sudo apt-get install ccache followed by e.g. this in your ~/.R/Makevars:
VER=
CCACHE=ccache
#CCACHE=
CC=$(CCACHE) gcc$(VER)
CXX=$(CCACHE) g++$(VER)
CXX11=$(CCACHE) g++$(VER)
CXX14=$(CCACHE) g++$(VER)
That also allows to switch between g++ versions. Changing it to clang++ is left as an exercise to the reader ;-)
Besides this, see the options to R CMD build and R CMD INSTALL to skip vignette and/or manual building to further speed up re-builds.
Illustration: Here is re-install of Rcpp itself (fresh from a git pull) where the first installation takes 21.9 seconds on my (decent) machine at work, the second then only takes 1.4 seconds thanks to ccache:
~/git/rcpp(master)$ time R CMD INSTALL .
* installing to library ‘/usr/local/lib/R/site-library’
* installing *source* package ‘Rcpp’ ...
** libs
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c Date.cpp -o Date.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c Module.cpp -o Module.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c Rcpp_init.cpp -o Rcpp_init.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c api.cpp -o api.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c attributes.cpp -o attributes.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c barrier.cpp -o barrier.o
g++ -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o Rcpp.so Date.o Module.o Rcpp_init.o api.o attributes.o barrier.o -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/Rcpp/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (Rcpp)
real 0m21.917s
user 0m21.388s
sys 0m2.304s
~/git/rcpp(master)$ ./cleanup
~/git/rcpp(master)$ time R CMD INSTALL .
* installing to library ‘/usr/local/lib/R/site-library’
* installing *source* package ‘Rcpp’ ...
** libs
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c Date.cpp -o Date.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c Module.cpp -o Module.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c Rcpp_init.cpp -o Rcpp_init.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c api.cpp -o api.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c attributes.cpp -o attributes.o
ccache g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O3 -Wall -pipe -pedantic -Wextra -Wno-empty-body -Wno-unused -c barrier.cpp -o barrier.o
g++ -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o Rcpp.so Date.o Module.o Rcpp_init.o api.o attributes.o barrier.o -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/Rcpp/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (Rcpp)
real 0m1.444s
user 0m1.380s
sys 0m1.452s
~/git/rcpp(master)$
Dirk's answer above pointed me in the right direction, but was insufficient. Since he has requested that I not append the final steps I required, I do so here.
My question could not have been answered without the following.
Dirk's answer did not work off the bat for me. If compilation still seems slow to you, try running:
ccache -s
The result will look a little like this
cache directory /home/myuser/.ccache
primary config /home/myuser/.ccache/ccache.conf
secondary config (readonly) /etc/ccache.conf
cache hit (direct) 0
cache hit (preprocessed) 0
cache miss 989
cache hit rate 0 %
called for link 12
preprocessor error 12
cleanups performed 16
files in cache 177
cache size 31.1 MB
max cache size 5.0 GB
Note that cache isn't getting hit, which means ccache isn't doing anything.
You can use:
export CCACHE_LOGFILE=ccache.log
R CMD build .
to do some debugging, though this wasn't helpful for me.
What fixed things was to run:
export CCACHE_NOHASHDIR=true
R CMD build .
As it turns out, ccache sometimes takes the location of a file into account. R CMD build . appears to build in a temporary directory, so the location of the files was changing each time.
I have this standalone C++ code
that I'm trying to wrap in an R
package.
My problem is that I absolutely
want it to be compiled with the
-O3 flag on.
So in the src/Makevars file
I put:
PKG_CPPFLAGS = -I../inst/include
PKG_CXXFLAGS = -O3
CXX_STD = CXX11
and still when I install my package on my
machine, I see:
g++ -std=c++0x -I/usr/share/R/include -DNDEBUG -I../inst/include -O3 -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c mycppfunctions.cpp -o mycppfunctions.o
g++ -std=c++0x -shared -Wl,-Bsymbolic-functions -Wl,-z,relro -o mycppfunctions.so mycppfunctions.o -L/usr/lib/R/lib -lR
(the dreaded -O2 flag appears to the right)
so my question is: how can I overwrite the
cpp flags used when g++ is invoked by R CMD?
Edit:
Recently, in another package, I found a way to do
something similar for a F77 code (also in an R package).
Basically, by adding this to the Makevars:
PKG_FFLAGS = $(FPICFLAGS) $(SHLIB_FFLAGS)
all: $(SHLIB)
otherf77foo.o: otherf77foo.f
$(F77) $(PGK_FFLAGS) -O3 -pipe -g -c -o otherf77foo.o otherf77foo.f
but I don't know how to do the same for a cpp code...
Edit2:
So, doing this is totally possible. Dirk Eddelbuettel question 'b)' from his answer below
guided me to the solution. So, all I had to do was to
place this in the src/Makevars file:
mycppfoo.o: mycppfoo.cpp
g++ -std=c++0x -I/usr/share/R/include -DNDEBUG -I../inst/include -fpic -g -O3 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c mycppfoo.cpp -o mycppfoo.o
g++ -std=c++0x -shared -Wl,-Bsymbolic-functions -Wl,-z,relro -o mycppfoo.so mycppfoo.o -L/usr/lib/R/lib -lR
and my problem was solved!
You can't (as per a comment by Simon Urbanek on r-devel a while back).
But it should not matter as AFAIK the right-most value wins. And R puts its values to the left, and lets you add your values (eg via CXX_FLAGS from, say, ~/.R/Makevars or PKG_CXXFLAGS from the src/Makevars in your package) to the right.
So just override with -O3 and it should be -O3.
For what it is worth, my current values in ~/.R/Makevars are:
CFLAGS += -O3 -Wall -pipe -pedantic -std=gnu99
CXXFLAGS += -O3 -Wall -pipe -Wno-unused -pedantic
and you could of course throw in -mnative or your specific CPU identifier.
Lastly, if you really wanted you could edit /etc/R/Makeconf but you'd have to do that
after each upgrade of the R package. And as I argue here you do not need to as the scheme suggested here should work.
Edit: In response to your edit:
a) The clear recommendation on r-devel (please check the archives) is that you should avoid Makefile logic if you can. IIRC this echoed in the Writing R Extension manual.
b) You declared a rule to build an .o (object) file from an .f (source) file. Did you try doing the same with cpp instead of f?
Lastly, you have not explained exactly why the world is coming to an end if your file is built with -O2 rather than -O3. You should understand that as an author of source, you can't fully control with which compiler options (let alone compiler versions) people will build your package.
newedit: Okay I'm a fool. It solved the problem for Rcpp (which I don't care about), but it doesn't work for the github.com/ohdsi/cyclops.git package that I do care about. That one still gets -O2 stuck right-most. This is ridiculous. Control over command-line parameters might be the single most important piece of this entire operation. R needs a better build system.
edit: Of course after days of trouble, I figure it out right after posting. My problem was that I was using the CXX_STD = CXX11 flag. Apparently with this flag you need to use CXX11FLAGS += .... So if your Makevars file contains CXX11FLAGS += -O0 -Wall it will correctly put this to the right of the -O2 flag if you're using C++11.
No matter what I do I can't get -O0 to show up on the right. I have the following in my ~/.R/Makevars:
CFLAGS += -O0 -Wall
CXXFLAGS += -O0 -Wall
CPPFLAGS += -O0 -Wall
PKG_CFLAGS += -O0 -Wall
PKG_CXXFLAGS += -O0 -Wall
PKG_CPPFLAGS += -O0 -Wall
I have installed Rcpp from source (as a test...I'm not interested in it directly) using
install.packages(getwd(), repos = NULL, type = "source")
and that did correctly use -O0.
With my current configuration, I end up getting three different -O0's to the left and final -O2 on the right. Has anyone else run into this problem?
The software I'm installing is at github.com/ohdsi/cyclops.git, though I'm not sure what that would be important.
I have Googled everywhere, and have not managed to figure out how to remove the -fpermissive flag from my Qt-generated Makefiles. The -fpermissive flag is defaultly there, even if I do QMAKE_CXXFLAGS -= -fpermissive in my Qt project file, attempting to remove it. Here is my Makefile where the flags are:
CFLAGS = -fpermissive -finline-functions -Wno-long-long -g -Wall $(DEFINES)
CXXFLAGS = -fpermissive -finline-functions -Wno-long-long -g -fexceptions -mthreads -frtti -Wall $(DEFINES)
How can I remove these?
My QtCreator-generated makefiles don't have -fpermissive anywhere in them, at least as far as I can Ctrl+F.
Check your project's Build Settings, and make sure you don't have -fpermissive in there as command-line arguments.
This is the the same two lines in my QMake-generated makefile:
CFLAGS = -pipe -fno-keep-inline-dllexport -O2 -Wall -Wextra $(DEFINES)
CXXFLAGS = -pipe -fno-keep-inline-dllexport -O2 -std=c++11 -frtti -Wall -Wextra -fexceptions -mthreads $(DEFINES)
My makefiles also references a file called 'qmake.conf', which for me is located at:
C:/Qt/Qt5.0.1/5.0.1/mingw47_32/mkspecs/win32-g++/qmake.conf
It looks like the qmake.conf is used to help generate makefiles for different compilers and platforms, and can add arguments to your makefile.