I have a Qt project that I'm compiling with GCC and MinGW for Ubuntu and Windows.
I got a requirement to harden it by adding the following LDFLAGS:
Stack execution protection: LDFLAGS="-z noexecstack"
Data relocation and protection (RELRO): LDLFAGS="-z relro -z now"
The question is can this be done with .pro file and how? I found it easy to add LFLAGS and CFLAGS in the project file but couldn't find anything for LDFLAGS. Even the output Makefiles don't seem to have any LDFLAGS defined.
One way I found after long googling was to add QMAKE_CFLAGS_RELEASE += "--noexecstack" in the .pro file but I'm not convinced this is the right way.
After the line above, the generated Makefile looks like this:
CC = gcc
CXX = g++
DEFINES = -DUNICODE -DMY_LIBRARY -DQT_NO_DEBUG -DQT_NETWORK_LIB -DQT_CORE_LIB
CFLAGS = -pipe -fno-keep-inline-dllexport -O2 -fstack-protector-strong -O2 -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security --noexecstack -Wall -Wextra $(DEFINES)
CXXFLAGS = -pipe -fno-keep-inline-dllexport -O2 -frtti -Wall -Wextra -fexceptions -mthreads $(DEFINES)
LINKER = g++
LFLAGS = -Wl,-s -shared -Wl,-subsystem,windows -mthreads -Wl,--out-implib,C:\libmylib0.a
noexecstack appears in the CLFAGS list but not sure if that's alright. CFLAGS is not the same as LDFLAGS. It doesn't seems to validate the command either since --thisdoesntexist seemed to go through as well when I tried.
Thank you in advance.
EDIT:
Based on Gwen's answer I tried adding QMAKE_LFLAGS += "-z noexecstack -z relro -z now" but this produced an error from ld.exe:
error: unrecognized option '-z'
EDIT2:
Tool versions:
C:\Qt\Qt5.5.0\Tools\mingw492_32\bin>ld.exe -v
GNU ld (GNU Binutils) 2.24
C:\Qt\Qt5.5.0\Tools\mingw492_32\bin>g++.exe --version
g++.exe (i686-posix-dwarf-rev1, Built by MinGW-W64 project) 4.9.2
With my configuration (QtCreator + Visual C++ compiler), the LFLAGS defined in the makefile is given to the linker, contrary to what is stated in the GNU make documentation:
LDFLAGS
: Extra flags to give to compilers when they are supposed to invoke the linker[...]
LFLAGS
: Extra flags to give to Lex.
I think you should try adding QMAKE_LFLAGS += "-z noexecstack -z relro -z now" to your .pro file, empty your build folder, re-run qmake, and see if the option is given to the linker.
Related
I'm building a library (libproj) using g++-9 and CMake (on Xenial, on Travis), so I can statically link it in a Rust crate. My build.rs sets up and runs CMake with the following config:
cmake proj-7.0.1
-DBUILD_SHARED_LIBS=ON
-DBUILD_TESTING=OFF
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_CXX_FLAGS="-std=c++11 -fPIC"
-DCMAKE_INSTALL_PREFIX=[path snipped]/out
-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64
-DCMAKE_C_COMPILER=/usr/bin/gcc-9
-DCMAKE_CXX_COMPILER=/usr/bin/g++-9
-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64
-DCMAKE_ASM_COMPILER=/usr/bin/gcc-9
Which builds and installs libproj successfully.
I then tell cargo to statically link it:
cargo:root=[path snipped]/out
cargo:rustc-link-search=native=[path snipped]/out/lib
cargo:rustc-link-lib=static=proj
However, the link step fails, saying I can't use relocation in a shared object:
note: /usr/bin/ld: [path snipped]/out/lib/libproj.a(4D_api.cpp.o): relocation
R_X86_64_32 against `.rodata.str1.8' can not be used when making a
shared object; recompile with -fPIC`
[path snipped]/out/lib/libproj.a(4D_api.cpp.o): error adding symbols: Bad value
collect2: error: ld returned 1 exit status
What am I doing wrong here? I've also tried to build libproj as a static lib by setting BUILD_SHARED_LIBS to OFF as per the install instructions, but that hasn't had any effect.
UPDATE:
I've managed to enable PIC for g++ (by default it's only enabled for gcc)
running: "cmake" "-Wdev" "--debug-output" "[snipped]/proj-7.0.1" "-DCMAKE_CXX_FLAGS=-std=c++11" "-DCMAKE_CXX_FLAGS=-fPIC" "-DBUILD_SHARED_LIBS=OFF" "-DCMAKE_INSTALL_PREFIX=[snipped]/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_C_COMPILER=/usr/bin/gcc-9" "-DCMAKE_CXX_COMPILER=/usr/bin/g++-9" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_ASM_COMPILER=/usr/bin/gcc-9" "-DCMAKE_BUILD_TYPE=Debug" "-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON"
Resulting in the following g++-9 invocation, which includes -fPIC:
/usr/bin/g++-9 -DCURL_ENABLED -DMUTEX_pthread -DPROJ_LIB=\"/[snip]/proj\" -DTIFF_ENABLED -I/[snip]/src -I/[snip]/include -I/[snip]/src -I/usr/include/x86_64-linux-gnu -fPIC -g -fvisibility=hidden -Wall -Wextra -Wswitch -Wshadow -Wunused-parameter -Wmissing-declarations -Wformat -Wformat-security -std=c++11 -o CMakeFiles/proj.dir/aasincos.cpp.o -c /[snip]/proj/proj-7.0.1/src/aasincos.cpp
However, I know get a completely different error (thousands of lines), that looks like my static library hasn't been correctly linked to the c++11 stdlib:
Error:
undefined reference to std::allocator<char>::allocator()
UPDATE 2:
I've managed to remove the -nodefaultlibs flag from ld. The latest g++ invocation:
/usr/bin/g++-9 -I/[snipped]/src -I/[snipped]/include -I/[snipped]/src -isystem /[snipped]/include -isystem /[snipped]/googletest -fPIC -g -fvisibility=hidden -Wall -Wextra -Wswitch -Wshadow -Wunused-parameter -Wmissing-declarations -Wformat -Wformat-security -pthread -std=c++11 -o CMakeFiles/proj_context_test.dir/proj_context_test.cpp.o -c /[snipped]/proj_context_test.cpp
And I've managed to add lstdc++ to the linker invocation:
"cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/[snip]/lib" "/home/travis/build/georust/proj-sys/target/debug/deps/proj_sys-95cbd73f2c3f0bde.20tasg77xuhsj5z6.rcgu.o" "/home/travis/build/georust/proj-sys/target/debug/deps/proj_sys-95cbd73f2c3f0bde.57y4784ihm3qw715.rcgu.o" "-o" "/[snip]/deps/proj_sys-95cbd73f2c3f0bde" "/home/travis/build/georust/proj-sys/target/debug/deps/proj_sys-95cbd73f2c3f0bde.3kiafs23968zh4mj.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-L" "/home/travis/build/georust/proj-sys/target/debug/deps" "-L" "/home/travis/build/georust/proj-sys/target/debug/build/proj-sys-aff2ac9d43b77886/out/lib" "-L" "/home/travis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-lsqlite3" "-lcurl" "-ltiff" "-lstdc++" "-Wl,-Bstatic" "-Wl,--whole-archive" "-lproj" "-Wl,--no-whole-archive" [trimmed rlib details] "-Wl,--end-group" "/home/travis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-2541f1e09df1c67d.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil"
But the undefined reference error remains.
As I can see from your example, you have quotation marks around CMAKE_CXX_FLAGS and do not have ones around CMAKE_C_FLAGS. It looks odd.
I have built libgproj myself with cmake + your flags and with make VERBOSE=1.
I can see, that C-files compiled without fPIC, m64 and fdata-sections. I suppose, that the cause is in the quotation marks around CMAKE_C_FLAGS. I saw your log in Travis, and there is configuration for cmake from cargo:
"cmake" "/home/travis/build/georust/proj-sys/PROJSRC/proj/proj-7.0.1" "-DBUILD_SHARED_LIBS=ON" "-DBUILD_TESTING=OFF" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_CXX_FLAGS=-std=c++11 -fPIC" "-DCMAKE_INSTALL_PREFIX=/home/travis/build/georust/proj-sys/target/debug/build/proj-sys-03a5fe6428bb060a/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_C_COMPILER=/usr/bin/gcc-9" "-DCMAKE_CXX_COMPILER=/usr/bin/g++-9" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_ASM_COMPILER=/usr/bin/gcc-9"
Note, that -DCMAKE_C_FLAGS and CMAKE_CXX_FLAGS escaped with their values, and not values by itself. I suppose, that these options become in the shell:
cmake -DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64 -DCVMAKE_CXX_FLAGS=-std=c++11 -fPIC
I have tried this command, without quotation marks and cmake just ignored unknown options. So it will not fail.
I suggest you to add cmake option -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON to cmake configuration or VERBOSE=1 to the make options, if you can, and then you can see real compile options passed to gcc in Travis.
You will see somethig like this:
[ 19%] Building C object src/CMakeFiles/proj.dir/wkt1_generated_parser.c.o
cd [some path]/PROJ/cmake_build/src && /usr/bin/cc -DCURL_ENABLED -DMUTEX_pthread
-DPROJ_LIB=\"/usr/local/share/proj\" -DTIFF_ENABLED -I[some path]PROJ/src -I[some
path]/PROJ/include -I[some path]/PROJ/cmake_build/src -O3 -DNDEBUG -
fvisibility=hidden -Wall -Wextra -Wswitch -Wshadow -Wunused-parameter -Wmissing-
declarations -Wformat -Wformat-security -Wmissing-prototypes -std=c99 -o
CMakeFiles/proj.dir/wkt1_generated_parser.c.o -c [some
path]/PROJ/src/wkt1_generated_parser.c
Conclusion:
You can try adding verbosity to see real compile options.
You can try to escape options properly, if it's necessary after first step.
Also I want to note, that SHARED_BUILD_LIBS=ON in my test forbids building static lib, which you want to link. When I set shared libs building setting option to OFF, static lib was built.
I am trying to write a script that uses a library compiled with clang and another library compiled with G++, I get the following error:
ld.lld: error: undefined symbol: myFunction()
Which (according to this Difference between string and char[] types in C++) is apparently due to std::string meaning different things in different versions of G++ and clang.
Does this mean I have to rewrite the Makefile of my target library to use the same version of G++/Clang? Because that seems like an awful amount of effort just to link a prewritten library, and I think I must be missing something here.
More info
I am compiling v8_shell using ninja -C out/debug
defines = -DUSE_UDEV -DUSE_AURA=1 -DUSE_GLIB=1 -DUSE_NSS_CERTS=1 -DUSE_X11=1 -DNO_TCMALLOC -DMEMORY_TOOL_REPLACES_ALLOCATOR -DMEMORY_SANITIZER_INITIAL_SIZE -DADDRESS_SANITIZER -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DCHROMIUM_BUILD -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -DCR_CLANG_REVISION=\"353250-1\" -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DCOMPONENT_BUILD -D_LIBCPP_ABI_UNSTABLE -D_LIBCPP_ABI_VERSION=Cr -DCR_LIBCXX_REVISION=349080 -DCR_LIBCXXABI_REVISION=347903 -D_LIBCPP_ENABLE_NODISCARD -DCR_SYSROOT_HASH=e7c53f04bd88d29d075bfd1f62b073aeb69cbe09 -D_DEBUG -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DWTF_USE_DYNAMIC_ANNOTATIONS=1 -DENABLE_DISASSEMBLER -DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=64 -DENABLE_GDB_JIT_INTERFACE -DENABLE_MINOR_MC -DOBJECT_PRINT -DVERIFY_HEAP -DV8_TRACE_MAPS -DV8_ENABLE_ALLOCATION_TIMEOUT -DV8_ENABLE_FORCE_SLOW_PATH -DV8_INTL_SUPPORT -DENABLE_HANDLE_ZAPPING -DV8_USE_SNAPSHOT -DV8_USE_EXTERNAL_STARTUP_DATA -DV8_CONCURRENT_MARKING -DV8_CHECK_MICROTASKS_SCOPES_CONSISTENCY -DV8_EMBEDDED_BUILTINS -DV8_ENABLE_CHECKS -DV8_DEPRECATION_WARNINGS -DV8_IMMINENT_DEPRECATION_WARNINGS -DV8_TARGET_ARCH_X64 -DDEBUG -DDISABLE_UNTRUSTED_CODE_MITIGATIONS -DUSING_V8_SHARED -DV8_ENABLE_CHECKS -DV8_DEPRECATION_WARNINGS -DV8_IMMINENT_DEPRECATION_WARNINGS -DU_USING_ICU_NAMESPACE=0 -DU_ENABLE_DYLOAD=0 -DUSE_CHROMIUM_ICU=1 -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_FILE -DUCHAR_TYPE=uint16_t -DUSING_V8_BASE_SHARED -DUSING_V8_PLATFORM_SHARED
include_dirs = -I../.. -Igen -I../.. -Igen -I../../include -Igen/include -I../../third_party/icu/source/common -I../../third_party/icu/source/i18n -I../../include
cflags = -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -Wno-builtin-macro-redefined -D__DATE__= -D__TIME__= -D__TIMESTAMP__= -funwind-tables -fPIC -B../../third_party/binutils/Linux_x64/Release/bin -pthread -fcolor-diagnostics -fmerge-all-constants -Xclang -mllvm -Xclang -instcombine-lower-dbg-declare=0 -no-canonical-prefixes -fcomplete-member-pointers -m64 -march=x86-64 -Wall -Werror -Wextra -Wimplicit-fallthrough -Wthread-safety -Wno-missing-field-initializers -Wno-unused-parameter -Wno-c++11-narrowing -Wno-unneeded-internal-declaration -Wno-undefined-var-template -Wno-ignored-pragma-optimize -fno-omit-frame-pointer -g2 -gsplit-dwarf -ggnu-pubnames -gcolumn-info -fsanitize=address -fsanitize-address-use-after-scope -fsanitize-blacklist=../../tools/memory/asan/blacklist.txt -fvisibility=hidden -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare -Wmissing-field-initializers -Wextra-semi -Winconsistent-missing-override -Wunreachable-code -Wshorten-64-to-32 -O2 -fno-ident -fdata-sections -ffunction-sections
cflags_cc = -Wno-undefined-bool-conversion -Wno-tautological-undefined-compare -std=c++14 -fno-exceptions -fno-rtti -nostdinc++ -isystem../../buildtools/third_party/libc++/trunk/include -isystem../../buildtools/third_party/libc++abi/trunk/include --sysroot=../../build/linux/debian_sid_amd64-sysroot -fvisibility-inlines-hidden
label_name = v8_shell
target_out_dir = obj
target_output_name = v8_shell
build obj/v8_shell/shell.o: cxx ../../samples/shell.cc || obj/generate_bytecode_builtins_list.stamp obj/run_torque.stamp obj/v8_dump_build_config.stamp obj/src/inspector/protocol_generated_sources.stamp obj/third_party/icu/icudata.stamp
build ./v8_shell: link obj/v8_shell/shell.o obj/build/config/sanitizers/liboptions_sources.a | ./libv8.so.TOC ./libv8_libbase.so.TOC ./libv8_libplatform.so.TOC ./libicui18n.so.TOC ./libicuuc.so.TOC ./libc++.so.TOC || obj/build/win/default_exe_manifest.stamp obj/v8_dump_build_config.stamp obj/build/config/executable_deps.stamp
ldflags = -pie -Wl,--fatal-warnings -fPIC -Wl,-z,noexecstack -Wl,-z,relro -fuse-ld=lld -Wl,--color-diagnostics -m64 -Werror -Wl,--gdb-index -rdynamic -nostdlib++ --sysroot=../../build/linux/debian_sid_amd64-sysroot -L../../build/linux/debian_sid_amd64-sysroot/usr/local/lib/x86_64-linux-gnu -Wl,-rpath-link=../../build/linux/debian_sid_amd64-sysroot/usr/local/lib/x86_64-linux-gnu -L../../build/linux/debian_sid_amd64-sysroot/lib/x86_64-linux-gnu -Wl,-rpath-link=../../build/linux/debian_sid_amd64-sysroot/lib/x86_64-linux-gnu -L../../build/linux/debian_sid_amd64-sysroot/usr/lib/x86_64-linux-gnu -Wl,-rpath-link=../../build/linux/debian_sid_amd64-sysroot/usr/lib/x86_64-linux-gnu -fsanitize=address -fsanitize-address-use-after-scope -pie -Wl,-rpath-link=. -Wl,--disable-new-dtags -Wl,-rpath=\$$ORIGIN/. -Wl,-rpath-link=. -Wl,-O2 -Wl,--gc-sections -Wl,-u_sanitizer_options_link_helper -fsanitize=address -fsanitize-address-use-after-scope
libs = -L . -ldl -lpthread -lrt
output_extension =
output_dir = .
solibs = ./libtester.so ./libv8.so ./libv8_libbase.so ./libv8_libplatform.so ./libicui18n.so ./libicuuc.so ./libc++.so
Link to code is here:
https://github.com/v8/v8/blob/master/samples/shell.cc
All I have done is add a test library that returns a std::string and this is called from shell.cc.
This test library is compiled with
clang++ -shared -o libtester tester.cpp -fPIC -L . -lpthread
Which yields undefined symbol errors.
I can get it to compile by forcing libc++ that v8 seems to use, but then I get a whole host of core dumps and other errors at runtime, so I don't think thats a legitimate fix.
clang++ -shared -o libtester.so tester.cpp -fPIC -std=c++11 -L . -stdlib=libc++ -lpthread
Example code snippet:
tester.cpp
std::string myFunction() {
std::string newstring;
// do something
return newstring;
}
shell.cc
std::string test = myFunction();
cout << test;
Update please see these posts that document it further C++ Undefined symbol related to std::string in static lib
Can't link libFuzzer.a using clang with libc++
Theres no real answer apart from use libstdc++ rather than libc++ for everything, but thats not really an option to convert an entire project like v8 to libstdc++
The build recipe you use for V8 builds it with -D_LIBCPP_ABI_UNSTABLE -D_LIBCPP_ABI_VERSION=Cr -DCR_LIBCXX_REVISION=349080 -DCR_LIBCXXABI_REVISION=347903. According to libc++ ABI stability, these macros affect the library ABI. Your separate compilation uses just -stdlib=libc++, so it does not enable the incompatible ABI.
You might get better results if you use a different recipe for building V8, something that uses the system C++ standard library with its default (stable) ABI.
This is the solution for compiling v8 with libstdc++.
Type the following command
gn args out/stdc
Add the following arguments to the args file:
is_clang = true
use_custom_libcxx_for_host=false
use_custom_libcxx=false
libcxx_abi_unstable=false
Build with
ninja -C out/stdc
I am on a complex project built in python2.7 that uses the PyDSTool package for analysis of dynamical system. PyDSTool provides two C-based integrators - Radau and Dopri - which I want to use to integrate my system of equations whose source is coded in a bunch of C/C++ files.
I have little control on the package, and when I instantiate the integrator, I can only add headers *.H files, source files (*.C, *.CPP) and pass the directories to include in the search path of the compiler as well as libraries to link to.
Since a consistent part of the code is based on C++11 I am passing to the compiler also the argument -std=C++11.
Eventually, /PyDSTool/Generators/mixins.py launch a setup command (line 185) which in turn runs the command build_ext from distutils to which all the above flags are appended.
For the sake of clarity: the flags that I am appending are:
compile options: '-I/usr/lib64/python2.7/site-packages/numpy/core/include -I/home/maurizio/Dropbox/StabilityAnalysis_tmp -I/usr/local/pydstool/PyDSTool/integrator -I/usr/include/python2_7 -I/usr/include/numpy -I/home/maurizio/Dropbox/Ongoing_Projects/pycustommodules -I/home/maurizio/Dropbox/Ongoing_Projects/c_libraries -I/home/maurizio/Dropbox/Ongoing_Projects/c_libraries/models -I/home/maurizio/Dropbox/Ongoing_Projects/DePitta_PNAS/Software/Stability_Analysis/ -I/usr/lib64/python2.7/site-packages/numpy/core/include -I/usr/include/python2.7 -c'
extra options: '-std=c++11 -w -Wno-return-type -Wall -lpython2.7 -lm -lgsl -lgslcblas -D__DOPRI__'
The resulting compilation command as issued by PyDSTool reads:
error: Command "gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/lib64/python2.7/site-packages/numpy/core/include -I/home/maurizio/Dropbox/StabilityAnalysis_tmp -I/usr/local/pydstool/PyDSTool/integrator -I/usr/include/python2_7 -I/usr/include/numpy -I/home/maurizio/Dropbox/Ongoing_Projects/pycustommodules -I/home/maurizio/Dropbox/Ongoing_Projects/c_libraries -I/home/maurizio/Dropbox/Ongoing_Projects/c_libraries/models -I/home/maurizio/Dropbox/Ongoing_Projects/DePitta_PNAS/Software/Stability_Analysis/ -I/usr/lib64/python2.7/site-packages/numpy/core/include -I/usr/include/python2.7 -c /home/maurizio/Dropbox/StabilityAnalysis_tmp/dop853_temp/ei_network_vf.c -o /home/maurizio/Dropbox/StabilityAnalysis_tmp/dop853_temp/home/maurizio/Dropbox/StabilityAnalysis_tmp/dop853_temp/ei_network_vf.o -std=c++11 -w -Wno-return-type -Wall -lpython2.7 -lm -lgsl -lgslcblas -D__DOPRI__" failed with exit status 1
Once looking into the build.log file automatically generated by PyDSTool, it turns out that the exit status is due to the fact that the compiler does not see the C++ libraries that are in several routines/libs used by my code, e.g.
/usr/include/blitz/blitz.h:45:18: fatal error: string: No such file or directory
#include <string>
^
Compilation Terminated
Now, it is not a problem of my code, because if I compile my code as a standalone in python or through scipy.weave with the same compile and extra options pasted above, it works. It is a problem of making PyDSTool build the code within the integrator. As I am NOT practical with distutils and all gcc options I hope there is some expert here that could provide me with some insight. I suspect in fact that I am missing some options or whatever to pass to the compiler.
Just for the sake of completeness. The issue I pointed out above does not have an easy workaround. PyDSTool C-based integrators (i.e. Radau and Dopri) cannot be compiled with source code for the equations in C++ but only in C. So either you recast your code in plain C or try to edit PyDSTool integrators and recast them in C++. The first option is likely the only one currently possible (at least to some non-experts as who is writing).
I have this standalone C++ code
that I'm trying to wrap in an R
package.
My problem is that I absolutely
want it to be compiled with the
-O3 flag on.
So in the src/Makevars file
I put:
PKG_CPPFLAGS = -I../inst/include
PKG_CXXFLAGS = -O3
CXX_STD = CXX11
and still when I install my package on my
machine, I see:
g++ -std=c++0x -I/usr/share/R/include -DNDEBUG -I../inst/include -O3 -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c mycppfunctions.cpp -o mycppfunctions.o
g++ -std=c++0x -shared -Wl,-Bsymbolic-functions -Wl,-z,relro -o mycppfunctions.so mycppfunctions.o -L/usr/lib/R/lib -lR
(the dreaded -O2 flag appears to the right)
so my question is: how can I overwrite the
cpp flags used when g++ is invoked by R CMD?
Edit:
Recently, in another package, I found a way to do
something similar for a F77 code (also in an R package).
Basically, by adding this to the Makevars:
PKG_FFLAGS = $(FPICFLAGS) $(SHLIB_FFLAGS)
all: $(SHLIB)
otherf77foo.o: otherf77foo.f
$(F77) $(PGK_FFLAGS) -O3 -pipe -g -c -o otherf77foo.o otherf77foo.f
but I don't know how to do the same for a cpp code...
Edit2:
So, doing this is totally possible. Dirk Eddelbuettel question 'b)' from his answer below
guided me to the solution. So, all I had to do was to
place this in the src/Makevars file:
mycppfoo.o: mycppfoo.cpp
g++ -std=c++0x -I/usr/share/R/include -DNDEBUG -I../inst/include -fpic -g -O3 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c mycppfoo.cpp -o mycppfoo.o
g++ -std=c++0x -shared -Wl,-Bsymbolic-functions -Wl,-z,relro -o mycppfoo.so mycppfoo.o -L/usr/lib/R/lib -lR
and my problem was solved!
You can't (as per a comment by Simon Urbanek on r-devel a while back).
But it should not matter as AFAIK the right-most value wins. And R puts its values to the left, and lets you add your values (eg via CXX_FLAGS from, say, ~/.R/Makevars or PKG_CXXFLAGS from the src/Makevars in your package) to the right.
So just override with -O3 and it should be -O3.
For what it is worth, my current values in ~/.R/Makevars are:
CFLAGS += -O3 -Wall -pipe -pedantic -std=gnu99
CXXFLAGS += -O3 -Wall -pipe -Wno-unused -pedantic
and you could of course throw in -mnative or your specific CPU identifier.
Lastly, if you really wanted you could edit /etc/R/Makeconf but you'd have to do that
after each upgrade of the R package. And as I argue here you do not need to as the scheme suggested here should work.
Edit: In response to your edit:
a) The clear recommendation on r-devel (please check the archives) is that you should avoid Makefile logic if you can. IIRC this echoed in the Writing R Extension manual.
b) You declared a rule to build an .o (object) file from an .f (source) file. Did you try doing the same with cpp instead of f?
Lastly, you have not explained exactly why the world is coming to an end if your file is built with -O2 rather than -O3. You should understand that as an author of source, you can't fully control with which compiler options (let alone compiler versions) people will build your package.
newedit: Okay I'm a fool. It solved the problem for Rcpp (which I don't care about), but it doesn't work for the github.com/ohdsi/cyclops.git package that I do care about. That one still gets -O2 stuck right-most. This is ridiculous. Control over command-line parameters might be the single most important piece of this entire operation. R needs a better build system.
edit: Of course after days of trouble, I figure it out right after posting. My problem was that I was using the CXX_STD = CXX11 flag. Apparently with this flag you need to use CXX11FLAGS += .... So if your Makevars file contains CXX11FLAGS += -O0 -Wall it will correctly put this to the right of the -O2 flag if you're using C++11.
No matter what I do I can't get -O0 to show up on the right. I have the following in my ~/.R/Makevars:
CFLAGS += -O0 -Wall
CXXFLAGS += -O0 -Wall
CPPFLAGS += -O0 -Wall
PKG_CFLAGS += -O0 -Wall
PKG_CXXFLAGS += -O0 -Wall
PKG_CPPFLAGS += -O0 -Wall
I have installed Rcpp from source (as a test...I'm not interested in it directly) using
install.packages(getwd(), repos = NULL, type = "source")
and that did correctly use -O0.
With my current configuration, I end up getting three different -O0's to the left and final -O2 on the right. Has anyone else run into this problem?
The software I'm installing is at github.com/ohdsi/cyclops.git, though I'm not sure what that would be important.
My goal is to get rid of some types of compiler warnings. I found out that I can do that by adding compiler flags in my .pro file:
QMAKE_CXXFLAGS += -Wno-unused-variable -Wno-reorder
The problem is that they are added before flags that are generated by Qt build system. I've examined my compiler output:
g++-4.2 -c -pipe -Wno-unused-variable -Wno-reorder -g -gdwarf-2 -arch
x86_64 -Xarch_x86_64 -mmacosx-version-min=10.5 -Wall -W
-DQT_OPENGL_LIB -DQT_GUI_LIB -DQT_CORE_LIB
So as you can see -Wall goes after my flags and discards them. What should I do to add those flags after ?
Don't use QMAKE_CXXFLAGS but rather override QMAKE_CXXFLAGS_WARN_ON with your own warnings:
QMAKE_CXXFLAGS_WARN_ON = -Wno-unused-variable -Wno-reorder