I'm trying to use NetCDF in a Fortran executable that then gets sent out to a lot of machines for parallel computation, so I need it to be statically linked. I'm doing the compiling on my own machine with mpif90, but I've also tried gfortran (Linux, though could also use Windows).
I'd love to build something entirely static, but packaging mpiexec in a static library seems difficult.
Here's a sketch of what I'm doing:
mpif90 -c -o somestuff.o somestuff.f90
mpif90 -c -o thing_that_uses_netCDF.o thing_that_uses_netCDF.f90 \
-I/usr/include -L/usr/lib/x86_64-linux-gnu -lnetcdff
ar cr libintermediate.a thing_that_uses_netCDF.o somestuff.o
mpif90 -c -o downstreamstuff.o downstreamstuff.f90 \
-I./path/to/intermediatelib -L./path/to/intermediatelib -lintermediate \
-I/usr/include -L/usr/lib/x86_64-linux-gnu -lnetcdff
mpif90 -o finalexecutable downstreamstuff.o \
-I./path/to/intermediatelib -L./path/to/intermediatelib -lintermediate \
-I/usr/include -L/usr/lib/x86_64-linux-gnu -lnetcdff
Could someone walk me through linking the netcdff library? When I try it with gfortran if I use the -static flag I get "undefined reference to `__netcdf_MOD..." errors, though it compiles fine without the -static flag.
Thanks.
Related
we are trying to make a c++ shared object library to interface with a static fortran library (compiled with mpif90). There is one fortran wrapper file compiled with gfortran or mpif90 (both fail) and one c++ wrapper file compiled with g++
The compilation command is
g++ -std=c++11 -ftemplate-depth-256 -Wno-inline -fPIC -O3 -pthread -fopenmp -v -I /usr/lib/x86_64-linux-gnu/openmpi/include/ -I /opt/local/nextsim/modules/oasis/include -o /opt/local/nextsim/lib/liboasis.so.1.0 /opt/local/nextsim/objs/./oasis_cpp_interface.o /opt/local/nextsim/objs/./oasis_cpp_interface_ftn.o /docker_io/compile_oa3-mct/lib/libpsmile.MPI1.a /docker_io/compile_oa3-mct/lib/libmct.a /docker_io/compile_oa3-mct/lib/libmpeu.a /docker_io/compile_oa3-mct/lib/libscrip.a -fopenmp -Wl,-rpath,/usr/lib/x86_64-linux-gnu/openmpi/lib/ -L /usr/lib/x86_64-linux-gnu/openmpi/lib/ -lmpi_cxx -lmpi -ldl -lstdc++ -lpthread -L /docker_io/compile_oa3-mct/lib -Wl,-rpath,/usr/lib/x86_64-linux-gnu -L /usr/lib/x86_64-linux-gnu -lnetcdff -shared -Wl,-soname,liboasis.so.1
The error it gives is:
/usr/bin/x86_64-linux-gnu-ld: /docker_io/compile_oa3-mct/lib/libpsmile.MPI1.a(mod_oasis_auxiliary_routines.o): relocation R_X86_64_PC32 against symbol `__mod_oasis_data_MOD_mpi_comm_local' can not be used when making a shared object; recompile with -fPIC
/usr/bin/x86_64-linux-gnu-ld: final link failed: Bad value
As you can see we already compiled with -fPIC (as was the fortran library and the c++ objects). I also tried linking the fortran objects instead of the libraries, but they were also using some other static libraries which gave the same error.
Actually for some reason, our code compiles on one particular server, but not another, and not inside docker (ubuntu) so the problem is a little puzzling.
The -fPIC option is ineffective in your commandline:
g++ -std=c++11 -ftemplate-depth-256 -Wno-inline -fPIC -O3 -pthread -fopenmp -v \
-I /usr/lib/x86_64-linux-gnu/openmpi/include/ -I /opt/local/nextsim/modules/oasis/include \
-o /opt/local/nextsim/lib/liboasis.so.1.0 \
/opt/local/nextsim/objs/./oasis_cpp_interface.o \
/opt/local/nextsim/objs/./oasis_cpp_interface_ftn.o \
/docker_io/compile_oa3-mct/lib/libpsmile.MPI1.a \
/docker_io/compile_oa3-mct/lib/libmct.a \
/docker_io/compile_oa3-mct/lib/libmpeu.a \
/docker_io/compile_oa3-mct/lib/libscrip.a \
-fopenmp -Wl,-rpath,/usr/lib/x86_64-linux-gnu/openmpi/lib/ \
-L /usr/lib/x86_64-linux-gnu/openmpi/lib/ \
-lmpi_cxx -lmpi -ldl -lstdc++ -lpthread -L /docker_io/compile_oa3-mct/lib \
-Wl,-rpath,/usr/lib/x86_64-linux-gnu \
-L /usr/lib/x86_64-linux-gnu -lnetcdff \
-shared -Wl,-soname,liboasis.so.1
because -fPIC is a compilation option and this is a linkage command. No source files
are input. Compilation has already been done. The other compilation options in this commandline:
std=c++11 -ftemplate-depth-256 -Wno-inline -fopenmp
-I /usr/lib/x86_64-linux-gnu/openmpi/include/ -I /opt/local/nextsim/modules/oasis/include \
are also redundant.
The linker says that the object file libpsmile.MPI1.a(mod_oasis_auxiliary_routines.o),
i.e. member mod_oasis_auxiliary_routines.o of the archive libpsmile.MPI1.a, was not
compiled with -fPIC. You say that:
we already compiled with -fPIC (as was the fortran library and the c++ objects).
but it's more likely that -fPIC was not used in the compilation of
the object files in libpsmile.MPI1.a than that the linker is mistaken.
Recompile all the object files input to the linkage, including those within static libraries, ensuring that -fPIC is enabled. All object files that are linked into a shared library must be Position-Independent-Code.
I am new to C++ and learning RTI DDS at the moment by compiling their examples. I am currently using their make files but I want to learn how to compile individual files using gcc directly. The make files first compiles objects and links them together as per below.
g++ -DRTI_UNIX -DRTI_LINUX -DRTI_64BIT -m64 -O2 -o objs/x64Linux3gcc4.8.2/HelloPublisher.o -Isrc -Isrc/idl -I/opt/rti_connext_dds-5.2.3/include -I/opt/rti_connext_dds-5.2.3/include/ndds -c src/HelloPublisher.cpp
g++ -m64 -static-libgcc -Wl,--no-as-needed objs/x64Linux3gcc4.8.2/HelloPublisher.o -o objs/x64Linux3gcc4.8.2/HelloPublisher -L/opt/rti_connext_dds-5.2.3/lib/x64Linux3gcc4.8.2 -lnddscppz -lnddscz -lnddscorez -ldl -lnsl -lm -lpthread -lrt
How can I write a single command using g++/gcc to do both?
The usual way is
g++ -o $prog -DRTI_UNIX $moreflags $file1.cpp $file2.cpp $prog.cpp $libs
You'll have to try a bit with the myriad of arguments you got since order matters.
I am trying to use v8 in a Dart native extension.
The v8 getting started guide says to compile the hello world example like this.
g++ -I. -Iinclude samples/hello-world.cc -o hello-world -Wl,--start-group \
out.gn/x64.release/obj/{libv8_{base,libbase,external_snapshot,libplatform,libsampler},\
third_party/icu/libicu{uc,i18n},src/inspector/libinspector}.a \
-Wl,--end-group -lrt -ldl -pthread -std=c++0x
Dart says to compile native extensions like so:
g++ -fPIC -I{path to SDK include directory} -DDART_SHARED_LIB -c sample_extension.cc
gcc -shared -m32 -Wl,-soname,libsample_extension.so -o libsample_extension.so sample_extension.o
This is the hybrid I came up with
g++ -fPIC -I{path to SDK include directory} -Iinclude -DDART_SHARED_LIB -c sample_extension.cc -std=c++0x
gcc -shared -Wl,-soname,libsample_extension.so -Wl,--start-group out.gn/x64.release/obj/{libv8_{base,libbase,external_snapshot,libplatform,libsampler},third_party/icu/libicu{uc,i18n},src/inspector/libinspector}.a -Wl,--end-group -o libsample_extension.so sample_extension.o -lrt -ldl -pthread -std=c++0x
However, while trying to run my application, I get an error stating that v8 is not linked properly.
dart: symbol lookup error: /mnt/c/Users/zvacu/Documents/Code/Dart/require/libsample_extension.so: undefined symbol: _ZN2v82V828InitializeICUDefaultLocationEPKcS2_
I can manage to link it properly when using the hello world example provided.
Doing a little research it seems like the problem it is on the -shared property on the second command. You need to pay attention with C++ and shared libraries, so check if your library get's all its dependencies by:
ldd /mnt/c/Users/zvacu/Documents/Code/Dart/require/libsample_extension.so
After this you will get a list of all dependencies, then you will need to search if there is anyone missing.
If this does not answer your question, see this related answer:
Undefined symbol when loading a shared library
Somehow my CUDA binary build process has been messed up. All of the .cu files compile nicely to .o files, but when I try to link, I get:
CMakeFiles/tester.dir/tester_intermediate_link.o: In function `__cudaRegisterLinkedBinary_66_tmpxft_00007a5f_00000000_16_cuda_device_runtime_compute_52_cpp1_ii_8b1a5d37':
/tmp/tmpxft_00006b54_00000000-2_tester_intermediate_link.reg.c:7: undefined reference to `__fatbinwrap_66_tmpxft_00007a5f_00000000_16_cuda_device_runtime_compute_52_cpp1_ii_8b1a5d37'
Now, I have not used compute_52 anywhere. My nvcc command-line is:
/usr/local/cuda/bin/nvcc -M -D__CUDACC__ /home/joeuser/src/my_project/src/kernel_specific/elementwise/Add.cu -o /home/joeuser/src/my_project/CMakeFiles/tester.dir/src/kernel_specific/elementwise/tester_generated_Add.cu.o.NVCC-depend -ccbin /usr/bin/gcc-4.9.3 -m64 --std c++11 -D__STRICT_ANSI__ -Xcompiler ,\"-Wall\",\"-g\",\"-g\",\"-O0\" -gencode arch=compute_35,code=compute_35 -g -G --generate-line-info -DNVCC -I/usr/local/cuda/include -I/opt/cub -I/usr/local/cuda/include
and my link line is:
/usr/bin/g++-4.9.3 -Wall -std=c++11 -g some.o files.o here.o blah.o blahblah.o bar.cu.o baz.cu.o -o bin/myapp -rdynamic -Wl,-Bstatic -lcudart_static -Wl,-Bdynamic -lpthread -lrt -ldl /usr/lib/libboost_system.so /usr/lib/libboost_program_options.so -Wl,-Bstatic -lcudart_static -Wl,-Bdynamic -lpthread -lrt -ldl /usr/local/cuda/extras/CUPTI/lib64/libcupti.so -lnvToolsExt -lOpenCL /usr/lib/libboost_system.so /usr/lib/libboost_program_options.so /usr/local/cuda/extras/CUPTI/lib64/libcupti.so -lnvToolsExt -lOpenCL -Wl,-rpath,/usr/lib:/usr/local/cuda/extras/CUPTI/lib64
I'll note I have separate compilation enabled, and do not seem to have skipped my intermediate link phase.
Why is this happening?
CUDA has two compilation modes, relocatable and static.
The relocatable mode is required for some configurations-which we will not get into now.
If you want to compile in relocatable mode -rdc=true, you'll need the Cuda device runtime library.
Which is located in the file cudadevrt.lib.
On some instances, supplying -lcudadevrt as a command line switch to the CUDA linker does the job, but on e.g. MSVC, you'll also need to specify cudadebrt.lib as a link dependency.
Well, I'm not sure why I'm seeing missing references to Compute 5.2 calls, but adding -lcudadevrt to the end of the link command makes the error go away.
I Have a LD_PRELOAD file. On what OS and conditions i should compile this preload to work on most systems (Unix/Linux). The most wanted are FreeBSD, Ubuntu, CenstOS, Solaris.
Thanks!
You need to compile it into a shared library. Here's how I typically compile mine:
libt.so: t.lo
g++ -fPIC -O3 -W -Wall -shared -Wl,-export-dynamic -o $# $^ -lc
t.lo: t.cc
g++ -c -fPIC -O3 -W -Wall $^ -o $#
Then to use it, you add the library to the LD_PRELOAD environment variable before launching that file. If you do it this way from the bash command line:
LD_PRELOAD=libt.so executable_name -and args
then, it will only set it for that command line run, and not affect any other programs you launch afterwards.