How to compile custom cpp files on Google Colab - c++

I'm trying to replicate the result of this github repo using Google Colab since I don't want to install all the requirements on my local machine and to take advantage of the GPU on Google Colab
However, one of the things I need to do (as indicated in the repo's README) is to first compile a cpp makefile. The instruction of the makefile is included below. Obvious I can't follow this instruction since I don't know Google Colab's directories of ncvv, cudalib and tensorflow library
cd latent_3d_points/external
with your editor modify the first three lines of the makefile to point to
your nvcc, cudalib and tensorflow library.
make
Is there a way for me to compile the files included in the makefile (because those functions are needed to run the model) either using the makefile directly or compile each cpp file individually? I included the content of the makefile below to avoid having you to click around in the repo looking for it
nvcc = /usr/local/cuda-8.0/bin/nvcc
cudalib = /usr/local/cuda-8.0/lib64
tensorflow = /orions4-zfs/projects/optas/Virt_Env/tf_1.3/lib/python2.7/site-packages/tensorflow/include
all: tf_approxmatch_so.so tf_approxmatch_g.cu.o tf_nndistance_so.so tf_nndistance_g.cu.o
tf_approxmatch_so.so: tf_approxmatch_g.cu.o tf_approxmatch.cpp
g++ -std=c++11 tf_approxmatch.cpp tf_approxmatch_g.cu.o -o tf_approxmatch_so.so -shared -fPIC -I $(tensorflow) -lcudart -L $(cudalib) -O2 -D_GLIBCXX_USE_CXX11_ABI=0
tf_approxmatch_g.cu.o: tf_approxmatch_g.cu
$(nvcc) -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -c -o tf_approxmatch_g.cu.o tf_approxmatch_g.cu -I $(tensorflow) -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC -O2
tf_nndistance_so.so: tf_nndistance_g.cu.o tf_nndistance.cpp
g++ -std=c++11 tf_nndistance.cpp tf_nndistance_g.cu.o -o tf_nndistance_so.so -shared -fPIC -I $(tensorflow) -lcudart -L $(cudalib) -O2 -D_GLIBCXX_USE_CXX11_ABI=0
tf_nndistance_g.cu.o: tf_nndistance_g.cu
$(nvcc) -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -c -o tf_nndistance_g.cu.o tf_nndistance_g.cu -I $(tensorflow) -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC -O2
clean:
rm tf_approxmatch_so.so
rm tf_nndistance_so.so
rm *.cu.o

You can use the bash like on your pc by adding %%bash in the colab's cells.
Example:
Cell one: write cpp file
%%writefile welcome.cpp
#include <iostream>
int main()
{
std::cout << "Welcome To AI with Ashok's Blog\n";
return 0;
}
Cell two: compile and run
%%bash
g++ welcome.cpp -o welcome
./welcome
You can also open the cpp file in colab's build-in text editor in order to enjoy correct highlights. It opens when you open a text file from the "Files" tab on the left and can be save with "ctr+s" shortcut.

You can install the required version of Cuda in google colab. For eg.
For Cuda 9.2 you can try
!apt-get --purge remove cuda nvidia* libnvidia-*
!dpkg -l | grep cuda- | awk '{print $2}' | xargs -n1 dpkg --purge
!apt-get remove cuda-*
!apt autoremove
!apt-get update
!wget https://developer.nvidia.com/compute/cuda/9.2/Prod/local_installers/cuda-repo-ubuntu1604-9-2-local_9.2.88-1_amd64 -O cuda-repo-ubuntu1604-9-2-local_9.2.88-1_amd64.deb
!dpkg -i cuda-repo-ubuntu1604-9-2-local_9.2.88-1_amd64.deb
!apt-key add /var/cuda-repo-9-2-local/7fa2af80.pub
!apt-get update
!apt-get install cuda-9.2
Similarly, you can find a way to install Cuda 8.2.
For gcc
!apt-get install -qq gcc-5 g++-5 -y
!ln -s /usr/bin/gcc-5
!ln -s /usr/bin/g++-5
!sudo apt-get update
!sudo apt-get upgrade
Then you can compile it or make it by running make, if your installation has a custom make file.
!make

Related

missing function: _ZN4CoreC1Ev when running a emscripten js compiled file

So here is my Makefile file:
webgl:
emcc -s USE_SDL=2 -s USE_SDL_IMAGE=2 -s SDL2_IMAGE_FORMATS='["bmp","png", "jpeg", "jpg"]' -s USE_SDL_TTF=2 -c main.cpp -o build/index.o
emcc -O1 build/index.o -s LLD_REPORT_UNDEFINED -s ERROR_ON_UNDEFINED_SYMBOLS=0 -o build/index.js
Yes, I have been using SDL2 for my emscripten project and compiling it has the big error so I have added a -s LLD_REPORT_UNDEFINED -s ERROR_ON_UNDEFINED_SYMBOLS=0 to compile immediately without errors. After my attempt to build, it works but I can't run because of an error when I typed node index.js and pressing the Enter key.
missing function: _ZN4CoreC1Ev
Aborted(-1)
I searched up _ZN4CoreC1Ev on Google for solution but Your search - _ZN4CoreC1Ev - did not match any documents. it just said. Any solutions?

g++: as fails to determine which assembler to run

Today I wanted to recompile one of my projects. Compiling this project had already worked on my machine, but this time an error occured.
The compiler output goes:
fatal error: as: unknown host architecture (can't determine which assembler to run)`
for the line:
g++ -c -pipe -O2 -Wall -W -D_REENTRANT -DQT_NO_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -DQT_SHARED -I/usr/share/qt4/mkspecs/linux-g++ -I../../.vscode -I/usr/include/qt4/QtCore -I/usr/include/qt4/QtGui -I/usr/include/qt4 -I. -I../../.vscode -I. -o Main.o ../../Main.cpp
I tried to compile some other projects, but realized, that I wouldn't be able to compile anything using any c++ compiler, so I looked it up.
The only fitting thread I found was this one but the solution 'reinstalling binutils' didn't work for me (tried sudo apt-get install --reinstall binutils as well as --reinstall gcc, g++ and build-essential)
One possible reason for this problem that comes to my mind is the iOS-toolchain I installed yesterday - I had to install some different clang versions - but I actually didn't change anything on the system's assembler...
If someone's got an idea; any help would be appreciated :)
Additional info:
Ubuntu 16.04 LTS 64bit
AMD FX(tm)-6300 Six-Core Processor × 6
uname -m
returns x86_64
gcc -march=native -v -E - 2>&1 <<<'' | grep "cc1" | egrep -o -e $'-m(arch|tune)=[^ "\']+'
returns -march=bdver2 and -mtune=bdver2
already tried gcc [...] -march with bdver2 and other architectures
g++ -v -c HelloWorld.cpp gives me: http://pastebin.com/Ks2be0hL
type -a as says:
as is /usr/local/bin/as
as is /usr/bin/as
as --version sadly just show's me the error again, but info as tells me it's binutils-2.26.1-system from 2016-08-07
dpkg -S /usr/bin/as prints: binutils: /usr/bin/as
type -a as says as is /usr/local/bin/as. This is what gcc is running, not /usr/bin/as, because /usr/local/bin/as is found first in your $PATH search order. This is why re-installing packages and so on is having no effect: something else you installed (probably manually) installed a non-standard as.
Have a look at /usr/local/bin/as and figure out where it came from, and what to do with it. For now you can just rename it to as.unknown or something, and then everything will use the normal system assembler (/usr/bin/as).
Remove binutils and reinstall it using the following steps:
Create an installation directory /opt/cross, and make sure you have write permission to .
sudo mkdir -p /opt/cross
sudo chown user /opt/cross
export PATH=/opt/cross/bin:$PATH
Download and install
wget http://mirrors.muzzy.it/gnu/binutils/binutils-2.9.tar.gz
tar xvf binutils-2.9.tar.gz
cd binutils-2.9
linux32 ./configure --prefix=/opt/cross --target=aarch64-linux --disable-multilib
If dosn't work remove --target=aarch64-linux, the --disable-multilib option means that we only want our Binutils installation to work with programs and libraries using the aarch64 instruction set, and not any related instruction sets such as aarch32, run:
linux32 ./configure --prefix=/opt/cross --disable-multilib
linux32 make
linux32 make install

linking pdfium on linux

I'm trying to use the pdfium libraries in linux(debian 64-bit). I managed (finally) to compile the release _x64 version of pdfium and the test programs seem to work. However, I can't seem to to use the libraries in a separate project.
This is my file:
#include <iostream>
#include "fpdfview.h"
#include "fpdftext.h"
#include "fpdfdoc.h"
#include "fpdfedit.h"
main(){
FPDF_InitLibrary();
std::cout << "Hello World!"<<std::endl;
return 0;
}
And this is my Makefile:
CC = g++
CFLAGS = -Wall -g -Wno-unused-variable -Wno-reorder -I/usr/include/pdfium/core/include -I/usr/include/pdfium/fpdfsdk/include -I/usr/include/pdfium/third_party -I/usr/include/pdfium/v8/include
LIBS_pdfium = -static -L/usr/lib/pdfium
LDFLAGS = $(LIBS_pdfium)
Main : Main.o
${CC} ${CFLAGS} Main.o ${LDFLAGS} -o Main
Main.o : Main.cpp
${CC} ${CFLAGS} -c -std=c++11 Main.cpp
clean:
rm *o Main
When I run the makefile the result is:
g++ -Wall -g -Wno-unused-variable -Wno-reorder -I/usr/include/pdfium/core/include -I/usr/include/pdfium/fpdfsdk/include -I/usr/include/pdfium/third_party -I/usr/include/pdfium/v8/include Main.o -static -L/usr/lib/pdfium -o Main
Main.cpp:11: error: undefined reference to 'FPDF_InitLibrary'
collect2: error: ld returned 1 exit status
I've also tried to use the libraries in /home/username/pdfium/out/Release_x64/obj but got the same error
I know that error: undefined reference to FPDF_InitLibrary means that there is a linking error. Therefore I checked the libraries in /home/username/pdfium/out/Release_x64/obj using objdump and one of them contained the InitLibrary symbol. This doesn't seem to make sense...
I don't know if I am referring to the wrong paths in the include or the libraries or if it is something else that is wrong.
I tried to understand the chromiums pdf plugin project makefile since I thought that might help me understand what I am supposed to use but unfortunately it didn't help.
Any ideas for what I am doing wrong?
To compile with PDFium the link line will depend on if you've compiled V8 and/or XFA into your PDFium binary.
With neither of those things enabled you'll need something similar to:
PDF_LIBS="-lpdfium -lfpdfapi -lfxge -lfpdfdoc -lfxcrt -lfx_agg \
-lfxcodec -lfx_lpng -lfx_libopenjpeg -lfx_lcms2 -lfx_freetype -ljpeg \
-lfx_zlib -lfdrm -lpdfwindow -lbigint -lformfiller -ljavascript \
-lfxedit"
PDF_DIR=<path/to/pdfium>
clang -I $PDF_DIR/public -o foo foo.c -L $PDF_DIR/out/Debug -lstdc++ -framework AppKit $PDF_LIBS
public/ is the only directory you should use when working with PDFium for headers. The -framework AppKit is needed on OSX. The PDFium headers are in plain C but you need -lstdc++ as PDFium uses C++ internally and it needs to be able to link in new/delete.
If you're working with V8 you'll need to add in:
-lv8_libbase -lv8_libplatform -lv8_snapshot -licui18n -licuuc -licudata
and if you're using XFA you'll need the V8 includes plus:
-lfpdfxfa -lxfa -lfx_tiff
EDIT
There was recently a pdf_is_complete_lib option added to the PDFium build. Setting that to true in your gn args will create a single libpdfium that can be linked agains. Note, this has only been tested with V8 and XFA disabled.
Args file..
# Build arguments go here.
# See "gn args <out_dir> --list" for available build arguments.
is_debug = false
pdf_is_standalone = true
pdf_use_skia = false
pdf_use_skia_paths = false
pdf_enable_xfa = false
pdf_enable_v8 = false
is_component_build = false
clang_use_chrome_plugins = false
pdf_is_complete_lib = true
use_custom_libcxx = false
Then gn gen your/dir/catalog.
Then ninja -C your/dir/catalog pdfium_all and you take pdfium.a
In Linker
...
g++ -L-I/usr/include/glib-2.0 -o bin/debug/pdfium_test obj/debug/main.o
...
you must have
-pg -s -Wl,--start-group /home/a/repo/pdfium/out/release/obj/libpdfium.a -Wl,--end-group -lpthread -ldl -lpthread
Linking is ok.
I haven't personally built it - because it was too time taking. But I managed to make it work with my golang application using cgo. I used ubuntu 16.04 as my base image in docker. This depends on https://github.com/bblanchon/pdfium-binaries
Following dockerfile downloads the pdfium binary and links to the app you are developing using pkg-config.
FROM ubuntu:16.04
# Specify pdfium version
ARG PdfiumVersion=4026
# Install pkg-config, etc.
RUN apt-get -yqq update && apt-get clean && apt-get install -yqq apt-utils pkg-config tzdata && dpkg-reconfigure -f noninteractive tzdata
# Create .pc file for pkg-config
RUN echo "\n" \
"prefix=/home\n" \
"Name: pdfium\n" \
"Description: pdfium\n" \
"Version: $PdfiumVersion\n" \
"Requires:\n" \
"Libs: -L/home/lib -lpdfium\n" \
"Cflags: -I/home/include\n" > /home/pdfium.pc
# Download and extract pdfium binary
RUN cd /home && wget --quiet https://github.com/bblanchon/pdfium-binaries/releases/download/chromium%2F$PdfiumVersion/pdfium-linux.tgz \
&& tar -xf pdfium-linux.tgz && rm pdfium-linux.tgz
# Setting up paths for pkg-config
ENV LD_LIBRARY_PATH=/home/lib
ENV PKG_CONFIG_PATH=/home/
## COPY YOUR APP TO /app/src/yourApp
# BUILD YOUR APP
WORKDIR /app/src/yourApp
# RUN your app which is linked to pdfium
ENTRYPOINT [“./yourApp"]

How to build Qt 5.1 for QNX target (arm)

new update
I think I should edit the title now.
To make sure I got a clean environment, I
download qt5.1.1 src code from qt-prject.
export QNX_TARGET, QNX_HOST, AND add QNX_HOST into PATH.
then Run the script
./configure -opensource -confirm-license -xplatform qnx-armv7le-qcc -v
so in here, -opensource -confirm-license just avoid the Q&A -v is to show full message.
a lot of error message.
Creating qmake...
make: Nothing to be done for `first'.
Running configuration tests...
Determining architecture... ()
qcc -Vgcc_ntoarmv7le -c -Wno-psabi -lang-c++ -g -Wall -W -fPIE -DQT_NO_CLIPBOARD -I../../mkspecs/qnx-armv7le-qcc -I. -I/opt/qnx650/target/qnx6/usr/include -I/opt/qnx650/target/qnx6/usr/include/freetype2 -o arch.o arch.cpp
Unable to determine architecture!
Could not determine the target architecture!
Turn on verbose messaging (-v) to see the final report.
Determining architecture... ()
g++ -c -pipe -g -Wall -W -fPIE -I../../mkspecs/linux-g++ -I. -o arch.o arch.cpp
g++ -o arch arch.o { test -n "" && DESTDIR="" || DESTDIR=.; } && test $(gdb --version | sed -e 's,[^0-9]\+\([0-9]\)\.\([0-9]\).*,\1\2,;q') -gt 72 && gdb --nx --batch --quiet -ex 'set confirm off' -ex "save gdb-index $DESTDIR" -ex quit 'arch' && test -f arch.gdb-index && objcopy --add-section '.gdb_index=arch.gdb-index' --set-section-flags '.gdb_index=readonly' 'arch' 'arch' && rm -f arch.gdb-index || true
Found architecture in binary
CFG_HOST_ARCH="x86_64"
CFG_HOST_CPUFEATURES=" mmx sse sse2"
System architecture: 'unknown'
Host architecture: 'x86_64'
C++11 auto-detection... ()
qcc -Vgcc_ntoarmv7le -c -Wno-psabi -lang-c++ -O2 -Wc,-std=gnu++0x -Wall -W -fPIE-DQT_NO_CLIPBOARD -I../../../mkspecs/qnx-armv7le-qcc -I. -I/opt/qnx650/target/qnx6/usr/include -I/opt/qnx650/target/qnx6/usr/include/freetype2 -o c++11.o c++11.cpp
C++11 disabled.
floatmath auto-detection... ()
qcc -Vgcc_ntoarmv7le -c -Wno-psabi -lang-c++ -O2 -Wall -W -fPIE -DQT_NO_CLIPBOARD-I../../../mkspecs/qnx-armv7le-qcc -I. -I/opt/qnx650/target/qnx6/usr/include -I/opt/qnx650/target/qnx6/usr/include/freetype2 -o floatmath.o floatmath.cpp
qcc -Vgcc_ntoarmv7le -c -Wno-psabi -lang-c++ -O2 -Wall -W -fPIE -DQT_NO_CLIPBOARD -I../../../mkspecs/qnx-armv7le-qcc -I. -I/opt/qnx650/target/qnx6/usr/include -I/opt/qnx650/target/qnx6/usr/include/freetype2 -o freetype.o freetype.cpp
FreeType disabled.
STL auto-detection... ()
qcc -Vgcc_ntoarmv7le -c -Wno-psabi -lang-c++ -O2 -Wall -W -fPIE -DQT_NO_CLIPBOARD -I../../../mkspecs/qnx-armv7le-qcc -I. -I/opt/qnx650/target/qnx6/usr/include -I/opt/qnx650/target/qnx6/usr/include/freetype2 -o stltest.o stltest.cpp
STL disabled.
STL functionality check failed! Cannot build Qt with this STL library.
Turn on verbose messaging (-v) to /home/pasadeveloper/qt-everywhere-opensourcesrc-5.1.1/qtbase/configure to see the final report.
UPDATE:
I am working on QNX for ARM, target is an arm platform device.
Thing is getting weird. in Env Var, I put
$QNX_CONFIGURATION=/etc/qnx
$QNX_JAVAHOME=/opt/qnx650/_jvm
$QNX_TARGET=/opt/qnx650/target/qnx6
$QNX_HOST=/opt/qnx650/host/linux/x86
but when I do qmake qmake.conf mkspecs/qnx-armv7le-qcc folder
it returns an error message Project ERROR: QNX_TARGET environment variable not set
Have no clue what is going on now.
not just qmake qmake.conf
I try to build qt 5.1.2 at another host, ubuntu 12.04-64bit.
also get the same error message. Project ERROR: QNX_TARGET environment variable not set
I was working at Qt development under linux(FYI Ubuntu 12.04 -64bits), but I need to compile this program to binary for QNX.
I install QNX MOmentics IDE which provide QNX-gcc for me.
but I can't find the qmake-qnx.
Under the QT/gcc_64/mkspecs/qnx-armv7le-qcc, there is a file call qmake.conf. I guess this is where I can generate my qmake-qnx. after I run qmake -o Makefile qmake.conf, there is a Makefile generated.
However, when I run make, error occured.
qcc -Vgcc_ntoarmv7le -lang-c++ -Wl,-rpath-link,/opt/qnx650/target/qnx6/armle-v7/lib -Wl,-rpath-link,/opt/qnx650/target/qnx6/armle-v7/usr/lib -Wl,-O1 -Wl,-O1 -Wl,-rpath,/home/pasadeveloper/Qt5.1.0/5.1.0/gcc_64 -Wl,-rpath,/home/pasadeveloper/Qt5.1.0/5.1.0/gcc_64/lib -o qmake -L/opt/qnx650/target/qnx6/armle-v7/lib -L/opt/qnx650/target/qnx6/armle-v7/usr/lib -lm -L/home/pasadeveloper/Qt5.1.0//5.1.0/gcc_64/lib -lQt5Gui -lQt5Core -lGL -lpthread
cc: no files to process
make: *** [qmake] Error 1
pasadeveloper#ubuntu:~/Qt5.1.0/5.1.0/gcc_64/mkspecs/qnx-armv7le-qcc$
You do not "generate" your qmake-qnx like that. You are supposed to use the native qmake for generating proper makefiles for your target to aid the cross-compilation. Also, running qmake qmake.conf in the relevant mkspecs folder is wrong because that is not a project file as you may think.
When building Qt itself for instance, you should be using the proper mkspecs files for the target in which case, it is the one you also mentioned above if it is built for that particular arm qnx variant, called qnx-armv7le-qcc.
Here is the exact command you need to run after downloading the relevant Qt sources, like 5.1.1:
./configure -opensource -confirm-license -xplatform qnx-armv7le-qcc -v
For this QNX version, the bottom line is, if you do not have SP1 and libscreen, it will not work. The QPA plugin would link against it. This library provides the API to the graphics server on newer QNX variants. You need to talk to your QNX representatives.
Here you can find further information about the topic.
$QNX_TARGET=/opt/qnx650/target/qnx6
is probably not doing what you want. In shell scripts, you don't put a "$" in front of a variable when you are defining the variable, only when you access the variable:
X=hello
echo $X

g++ linking issue with GSL

g++ -o program main.cpp classOne.cpp classTwo.cpp -lgsl -lgslblas -lm
that's how i compile when the GSL-packages are installed.
but now I'm working on a server where i don't have rights to install GSL-Library.
What are my options?
thx
I had to do this regularly, do as following :
On the server, create one directory in your home directory to install the library (let's say mypref) and another one to build the library (let's say tmp). You have two new directories : ~/mypref and ~/tmp.
Download GSL sources in ~/tmp (last version is ftp://ftp.gnu.org/gnu/gsl/gsl-1.14.tar.gz), extract and go in the generated sub-directory (gsl-1.14) :
cd ~/tmp
wget ftp://ftp.gnu.org/gnu/gsl/gsl-1.14.tar.gz
tar -xvzf gsl-1.14.tar.gz
cd gsl-1.14
Launch the configure script specifying ~/mypref as the installation prefix (and maybe other options depending of your server) :
./configure --prefix=${HOME}/mypref
Make :
make
And install :
make install
Now you can remove safely the ~/tmp directory :
cd; rm -rf tmp
Now you can compile your program using :
g++ -o program main.cpp classOne.cpp classTwo.cpp -I${HOME}/mypref/include -lm -L${HOME}/mypref/lib -lgsl -lgslcblas
-I and -L indicate respectively the path for the headers and the library. If your program is meant to be executed in a context where your home directory is not visible, consider static linking :
g++ -o program main.cpp classOne.cpp classTwo.cpp ${HOME}/mypref/lib/libgsl.a ${HOME}/mypref/lib/libgslcblas.a -I${HOME}/mypref/include -lm
The binary produced by the last command is bigger than previously, but entirely independent from GSL and GSLCBLAS.