KLEE for C++ code that uses pthreads - llvm

I am a beginner trying to use KLEE.
I am using the KLEE self contained package to on a C++ program that
uses pthreads.
I have generated a .o file and used KLEE with the following option
klee --libc=uclibc --posix-runtime test.o
But i see i get warning
KLEE: NOTE: Using model:
/home/pgbovine/klee/Release+Asserts/lib/libkleeRuntimePOSIX.bca
KLEE: output directory = "klee-out-4"
KLEE: WARNING: undefined reference to function: klee_get_valuel
KLEE: WARNING: undefined reference to function: pthread_create
KLEE: WARNING: undefined reference to function: pthread_exit
KLEE: WARNING: undefined reference to function: pthread_join
KLEE: WARNING: executable has module level assembly (ignoring)
KLEE: WARNING: calling external: syscall(54, 0, 21505, 571522624)
KLEE: WARNING: calling __user_main with extra arguments.
KLEE: WARNING: calling external: pthread_create(571589384, 0, 563903904, 571574176)
0 klee 0x08965ab8
[pid 1846] +++ killed by SIGSEGV +++
+++ killed by SIGSEGV +++
Segmentation fault
Using klee on a .bc file also gives me the same result.
I am not sure why i get undefined reference to pthread functions. I am
not sure if the libraries for pthreads are being used properly. Is
there a way to ensure this? Using the llvm-ld also does not help.
Below is the llvm-ld command that i used
llvm-ld tests.bc -l=/usr/lib/libpthread.a
I am not sure why there Segmentation fault occurs. The program works
fine when i normally compile the programs with g++ and run the
executable file.
Could someone tell me where i am going wrong?

The issue is that there is no existing pthread support in Klee. Hence, when you call pthread_create(), the Klee will not respond to it(that's why you see KLEE: WARNING: calling external: pthread_create). In this case, Klee will crash due to this failure.

If you want to use the pthread function in KLEE, you can modify the KLEE source code to simulate the execution of multi-thread.
In KLEE, there is a data struct "ExecutionState", and when you create a thread in user code, you can create a ExecutionState in KLEE accordingly, and set "pc" of ExecutionState by thread function. So you can overwrite the pthread functions in KLEE, and intercept the calling of pthread function by user code in the "executeInstruction" function in Executor.cpp.
To simulate the execution of multi-thread, you should modify the searcher of KLEE, which is used to select state to execute from the all the ExecutionState vector.
So it is a hard work.

As of 2010, basic version of KLEE does not support any form of parallelism, including pthread. But Raimondas Sasnauskas (rwth-aachen) has information about dslab's project from EPFL:
https://mailman.cs.umd.edu/pipermail/otter-dev/2010-December/000435.html
The current release of KLEE does not support any kind of
parallelism -- you have to implement/model it on your own.
Nevertheless, people from EPFL have already implemented pthread
support as an extension to KLEE and published a nice paper about
this topic:
http://dslab.epfl.ch/pubs/esd
There is archived link: http://web.archive.org/web/20100516044002/http://dslab.epfl.ch/pubs/esd
"Execution Synthesis: A Technique for Automated Software Debugging", Cristian Zamfir and George Candea. Proc. 5th ACM SIGOPS/EuroSys European Conference on Computer Systems (EuroSys), Paris France, April 2010
In 2013 there was one more posting http://mailman.ic.ac.uk/pipermail/klee-dev/2013-January/000031.html by Cristian Cadar noting that KLEE doesn't support pthreads, with metion that Cloud9 extension of KLEE may handle pthreads:
Currently, KLEE has limited support for C++, and no support for
pthreads. However, there are certain extensions to KLEE that handle
these aspects, e.g., KLOVER for C++ and Cloud9 for pthreads. Take a
look at http://klee.llvm.org/Publications.html for more information.

Related

Tensorflow undefined reference to `tensorflow::TensorShapeBase<tensorflow::TensorShape>::TensorShapeBase(absl::Span<long const>)'

I am using Tensorflow's C++ API to load and run a saved model. When I build my C++ code in GCC using the optimization flag -O2 I get the following error:
undefined reference to `tensorflow::TensorShapeBase<tensorflow::TensorShape>::TensorShapeBase(absl::Span<long const>)'
which I believe is due to the following tensor creation:
Tensor my_tensor(DT_DOUBLE, TensorShape({2, 4}));
However, if I build my C++ code without the compiler flag -O2, the code builds and executes perfectly. I am using Tensorflow 2.5 library that was built from source.
Any suggestions on how to fix the error?
The issue is related to a conflict between C++14 and C++17 when compiling Tensorflow with ABSL.
See this link:
Tensorflow_cc library uses own copy of Absl, and uses
absl::string_view in function signatures. absl::string_view is mapped
to std::string_view if C++17 is detected, and to own implementation if
C++17 is not. That leads to linker errors like this when using Arch
tensorflow_cc library from C++17 code:
The workaround would be to comment out the lines: Using the library
from C++17 after building libraries in C++11 mode (Arch)
One workaround is to comment out this line in tensorflow/include/absl/base/config.h:
#define ABSL_HAVE_STD_STRING_VIEW 1
This will make the library look for the custom absl::string_view implementation even if tensorflow is called from C++17.

What is the meaing of "(.text+0x26c4)" means in fortran linking error message?

I am using gfortran compiler and ld linker in LINUX ubuntu 16.04.
When I am compiling some program written in Fortran 77, I have the following error message:
libdist.a(setup.F.o): In function `setup_':
setup.F:(.text+0x26c4): undefined reference to `mpi_send_'
setup.F:(.text+0x2b3c): undefined reference to `mpi_recv_'
setup.F:(.text+0x7984): undefined reference to `mpi_send_'
setup.F:(.text+0xb107): undefined reference to `mpi_recv_'
I guess it is about the position of error. However, it is difficult to me the find where is the error.
Can I have a better presentation of the position of error? Such as the c/cxx error: "setup.F:15:12: "
It is an address. And is is NOT a Fortran error. It is a linker error. You can get the very same error from a C or C++ code or any other compiled code linked by the same linker.
Generate debugging symbols (compiler option -g or -ggdb or similar - consult Debugging options in the GCC Fortran manual) to get something more meaningful. But if you do that you will realize that it only points you to some location where you do:
call mpi_send(...)
in some subroutine or function called setup().
So, it is not terribly useful in this case. The important thing is that you should link the appropriate MPI library. Normally that is done by calling mpif90 or mpifort or a similar wrapper which is called instead of gfortran. Consult the documentation of your MPI library implementation.

Compiling Octave MXE nsi-installer with --enable-64

I am trying to compile an Octave nsi-installer for Windows with --enable-64 for large array support.
Attempt 1: Using a Linux Mint virtual machine on Windows 10:
./configure shows everything seems to be OK, and after running make, quite a lot completes. However, several minutes into "[build] default-octave", it crashes out. Looking at the log, when trying to link liboctave/.libs/liboctave-3.dll, it says:
libgnu/.libs/libgnu.a(lock.o): In function glthread_recursive_lock_init_multithreaded':
/home/mike/temp/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/libgnu/glthread/lock.c:289: undefined reference to__imp_pthread_mutexattr_init'
/home/mike/temp/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/libgnu/glthread/lock.c:292: undefined reference to __imp_pthread_mutexattr_settype'
/home/mike/temp/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/libgnu/glthread/lock.c:298: undefined reference to__imp_pthread_mutex_init'
/home/mike/temp/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/libgnu/glthread/lock.c:304: undefined reference to __imp_pthread_mutexattr_destroy'
/home/mike/temp/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/libgnu/glthread/lock.c:301: undefined reference to__imp_pthread_mutexattr_destroy'
/home/mike/temp/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/libgnu/glthread/lock.c:295: undefined reference to __imp_pthread_mutexattr_destroy'
libgnu/.libs/libgnu.a(strsignal.o): In functioninit':
/home/mike/temp/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/libgnu/strsignal.c:143: undefined reference to __imp_pthread_key_create'
libgnu/.libs/libgnu.a(strsignal.o): In functionfree_key_mem':
/home/mike/temp/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/libgnu/strsignal.c:170: undefined reference to __imp_pthread_setspecific'
libgnu/.libs/libgnu.a(strsignal.o): In functionstrsignal':
/home/mike/temp/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/libgnu/strsignal.c:101: undefined reference to __imp_pthread_once'
libgnu/.libs/libgnu.a(strsignal.o): In functiongetbuffer':
/home/mike/temp/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/libgnu/strsignal.c:186: undefined reference to __imp_pthread_getspecific'
/home/mike/temp/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/libgnu/strsignal.c:195: undefined reference to__imp_pthread_setspecific'
collect2: error: ld returned 1 exit status
So, the problem is all these undefined references to '__imp_pthread_' functions in lock.c.
When I look at lock.c at those lines, the code shows:
err = pthread_mutexattr_init (&attributes);
So my main question is, why in linking is '__imp_' added to the front of the function?
If I 'nm libpthread.so', I see stuff like:
pthread_mutexattr_init.o:
0000000000000000 T __pthread_mutexattr_init
0000000000000000 T pthread_mutexattr_init
So the functions are in there, but the link doesn't see them because the '__imp_' is attached? I don't understand...
The log shows that '-pthread' is used rather than '-lpthread', which I think is what is recommended. When I look at the top of the log, it shows:
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
Attempt 2: Using a CentOS linux system:
Similar to above, ./configure works but a little ways into "[build] default-octave", it crashes out. Looking at the log, it says:
/u/glaucus-r1/mpjohnso/octave/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/liboctave/array/dim-vector.h: In instantiation of 'dim_vector::dim_vector(octave_idx_type, octave_idx_type, Ints ...) [with Ints = {int}; octave_idx_type = long long int]':
/u/glaucus-r1/mpjohnso/octave/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/libinterp/octave-value/ov-typeinfo.h:203:85: required from here
/u/glaucus-r1/mpjohnso/octave/mxe-octave-3fc30cd416ac/tmp-default-octave/octave-4.1.0+/liboctave/array/dim-vector.h:215:5: error: unable to deduce std::initializer_list<_Tp>&&' from '{r, c, lengths#0}'
for (const auto l: {r, c, lengths...})
^
So, there seems to be some problem with how the '{r, c, lengths...}' line is interpreted... My first try used GCC 4.4.7, but the same thing happened once upgrading to GCC 6.2.0.
Any thoughts for either?

Static linking against the indigo cheminformatics package under Linux x64 complains undefined reference to __dso_handle when Free Pascal is used

Update 1
According to this SO post, Free Pascal may not link against C++ object file, which means static C++ libraries, when libstdc++.so.6.X is used as the C++ lib.
It should be noted that for that specific hello-world program in that SO post, Free Pascal can link against C++ object file using libstdc++.so.5 as the C++ lib. Note libstdc++.so.5 is with GCC 3.3.X.
Update 2
According to the Free Pascal future plans, Linking with C++ code is not until next major version is finished.
Update 3
According to "How to use C code in Free Pascal projects", Free Pascal cannot link in C++ objects directly. They should be placed into shared library to be used from Free Pasacal.
The source code of the indigo cheminformatics package can be compiled into static libraries and provides a plain-C API interface to its C++ core.
The procedure is shown:
cd indigo
module add gcc/4.7.2 # or gcc/4.3.2
module add cmake/2.8.8
python ./build_scripts/indigo-release-libs.py --preset=linux64
Part of the tree structure is shown:
indigo
./dist/indigo-libs-1.1.12-linux64-static.zip
./dist/LICENSE.GPL
./dist/indigo-inchi.h
./dist/indigo-renderer.h
./dist/indigo.h
./dist/static
./dist/static/Linux
./dist/static/Linux/x64
./dist/static/Linux/x64/libcairo.a
./dist/static/Linux/x64/libcommon.a
./dist/static/Linux/x64/libgraph.a
./dist/static/Linux/x64/libinchi.a
./dist/static/Linux/x64/libindigo-inchi-static.a
./dist/static/Linux/x64/libindigo-renderer-static.a
./dist/static/Linux/x64/libindigo-static.a
./dist/static/Linux/x64/liblayout.a
./dist/static/Linux/x64/libmolecule.a
./dist/static/Linux/x64/libpixman.a
./dist/static/Linux/x64/libpng.a
./dist/static/Linux/x64/libreaction.a
./dist/static/Linux/x64/librender2d.a
./dist/static/Linux/x64/libtinyxml.a
./dist/static/Linux/x64/libz.a
To compile .c and static link against the static indigo libraries:
Source code of a sample cTest.c file is shown:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "indigo.h"
int main (void) { printf("%s\n", indigoVersion()); }
, the procedure is shown:
cd indigo/dist/static/Linux/x64
module add gcc/4.7.2 # or gcc/4.3.2
g++ -I../../../ cTest.c -o cTest libindigo-static.a libreaction.a liblayout.a libmolecule.a libgraph.a libcommon.a libz.a libtinyxml.a -lpthread
To compile .pp and static link against the static indigo libraries:
Source code of a sample fpcTest.pp file is shown:
program fpcTest;
{$MODE DELPHI}
{$APPTYPE CONSOLE}
uses
SysUtils;
// {$linklib stdc++}
{$link /usr/lib64/libstdc++.so.6}
{$linklib pthread}
{$linklib libindigo-static}
{$linklib libreaction}
{$linklib liblayout}
{$linklib libmolecule}
{$linklib libgraph}
{$linklib libcommon}
{$linklib libz}
{$linklib libtinyxml}
function indigoVersion (): PChar; cdecl; external 'libindigo-static';
begin
WriteLn(indigoVersion);
end.
, the procedure is shown:
cd indigo/dist/static/Linux/x64
~/fpc-2.6.4/bin/fpc fpc_static.pp
However, it complains "undefined reference to __dso_handle". The Error msg is shown below:
Free Pascal Compiler version 2.6.4 [2014/03/03] for x86_64
Copyright (c) 1993-2014 by Florian Klaempfl and others
Target OS: Linux for x86-64
Compiling fpc_static.pp
fpc_static.pp(4,2) Note: APPTYPE is not supported by the target OS
Linking fpc_static
/usr/bin/ld: warning: link.res contains output sections; did you forget -T?
./libindigo-static.a(option_manager.cpp.o): In function `_GLOBAL__sub_I_option_manager.cpp':
option_manager.cpp:(.text.startup+0x3): undefined reference to `__dso_handle'
./libindigo-static.a(indigo.cpp.o): In function `Indigo::init()':
indigo.cpp:(.text+0x3bf): undefined reference to `__dso_handle'
./libindigo-static.a(indigo.cpp.o): In function `_GLOBAL__sub_I_indigo.cpp':
indigo.cpp:(.text.startup+0x2e): undefined reference to `__dso_handle'
indigo.cpp:(.text.startup+0x5b): undefined reference to `__dso_handle'
./libcommon.a(profiling.cpp.o): In function `_GLOBAL__sub_I_profiling.cpp':
profiling.cpp:(.text.startup+0x22): undefined reference to `__dso_handle'
./libcommon.a(profiling.cpp.o):profiling.cpp:(.text.startup+0x40): more undefined references to `__dso_handle' follow
fpc_static.pp(25,1) Error: Error while linking
fpc_static.pp(25,1) Fatal: There were 1 errors compiling module, stopping
Fatal: Compilation aborted
Error: ~/fpc-2.6.4/bin/ppcx64 returned an error exitcode (normal if you did not specify a source file to be compiled)
Could you help to comment about the reason and the workaround ?
I think in your references you are majorly confusing "pascal calling C++ classes" and "pascal and C++ libraries linked into the same project". update 1 is about the latter, update 2 is about the former, update 3 is old and not really related I guess.
My guess is that the modified FPC startup code that initializes libraries via CTOR and DTOR, ONLY initializes shared libraries, and not static libraries. IOW any C/C++ state linked into the main binary might not be initialized.
It will probably require an exact understanding of CTOR and DTOR internal working, and specially in combination with static linking to figure this out.
To make it clear: the "C++" support in trunk (on which the next FPC version will be based) is about calling (gcc) C++ classes directly. IOW making FPC understand gcc mangling (in some gcc version, since C++ mangling is usually version dependent long term). And while it is in the branch, it is not known yet if it will be considered finished and stable in the next fixes branch.

G++ does not compile my source code with -O0 but it does with -O2 -DNDEBUG, how can I solve it?

I am writing a firmware for an ARM microcontroller. I have built the cross-compiler without syscalls, actually my RTOS (ChibiOS) provides a simple syscalls implementation.
All my code is written in C excepting some parts where I only use C++ for linking with Eigen library (a C++ template library for linear algebra, it is only headers).
If I compile my source example code with -O2 -DNDEBUG (as far as I know with NDEBUG the code won't need assert()) everything compiles fine and the firmware works.
If I compile my source example code with -O0 I have the following:
Linking build/ch.elf
/home/noether/workspace/tool-chains/arm-none-eabi-4.6.2/lib/gcc/arm-none-eabi/4.6.2 /../../../../arm-none-eabi/lib/thumb/cortex-m4/libc.a(lib_a-abort.o): In function `abort':
/home/noether/workspace/tool-chains/summon-arm-toolchain/build/arm-none-eabi/thumb /cortex-m4/newlib/libc/stdlib/../../../../../../../gcc-4.6.2/newlib/libc/stdlib /abort.c:63: undefined reference to `_exit'
/home/noether/workspace/tool-chains/arm-none-eabi-4.6.2/lib/gcc/arm-none-eabi/4.6.2/../../../../arm-none-eabi/lib/thumb/cortex-m4/libc.a(lib_a-signalr.o): In function `_kill_r':
/home/noether/workspace/tool-chains/summon-arm-toolchain/build/arm-none-eabi/thumb/cortex-m4/newlib/libc/reent/../../../../../../../gcc-4.6.2/newlib/libc/reent/signalr.c:61: undefined reference to `_kill'
/home/noether/workspace/tool-chains/arm-none-eabi-4.6.2/lib/gcc/arm-none-eabi/4.6.2/../../../../arm-none-eabi/lib/thumb/cortex-m4/libc.a(lib_a-signalr.o): In function `_getpid_r':
/home/noether/workspace/tool-chains/summon-arm-toolchain/build/arm-none-eabi/thumb/cortex-m4/newlib/libc/reent/../../../../../../../gcc-4.6.2/newlib/libc/reent/signalr.c:96: undefined reference to `_getpid'
collect2: ld returned 1 exit status
make: *** [build/ch.elf] Error 1
It does not matter if I put -DNDEBUG, I have the same output.
I am using those flags as well, -fno-exceptions and fno-rtti.
If I do not use/link the Eigen library (the only C++ stuff), g++ compiles the source fine even with -O0.
Actually, I implemented a simple _kill _getpid and _exit functions, and the code compiles, but the code goes from 13KB to 130KB, and It crashes (maybe I haven't written well those functions).
What I want is to remove this stuff from my code (abort, etc) if I use -O0, as it is done (I guess) with -O2.
Thank you very much. Let me know if you need more information.
These references are almost certainly due to using assert(); that will call abort() on failure, which in turn will try to raise a signal (using the kill() syscall in this implementation) to abort the process. Obviously, this isn't possible if you can't make syscalls.
Building with -DNDEBUG (like you do in the optimised version) will fix this; it causes the assert() macro to generate no code, so there will be no references to abort(). The optimisation level itself shouldn't make any difference.
Alternatively, if you want to preserve your assertions, you could implement your own assert() macro that doesn't require any syscalls.
You could simply implement stubs for those missing syscalls and link them to your code. That will keep the linker quiet, but you might make the stubs do something sensible or useful. exit() might disable interrupts and loop indefinitely or force a reset. _kill() might hook into your RTOS to terminate a thread, or simply call _exit(), and _getpid() might return some dummy value or an RTOS thread ID.