bazel test vs direct execution - c++

I'm using GoogleTest with bazel and when I run bazel test :tokens_test (rule defined below) I have a failing test, but when I run the compiled tests I see the expected result. It is failing because the test can not open the test data file. The directory layout and the BUILD rule for the test I'm running look like this:
tokens/
BUILD
tokens_test.cpp
test_data/
test_input1.txt
cc_test(
name = "tokens_test",
srcs = ["tokens_test.cpp"],
deps = [
'#gtest//:gtest'
'#gtest//:gtest_main',
],
data = ["//tokens/test_data:test_input1.txt"]
)
At this point the test is just a wrapper to open the file and read in the test data which is the bit thats failing.
TEST(Tokenizer, OpenFileTest) {
auto fin = std::ifstream("test_data/test_input1.txt");
std::cout << a.get(); // Outputs -1.
}
When I navigate to the bazel-out location and find my way to the tokens runfiles directory I can see the compiled test excutable.
[jibberish]/__main__/tokens>ls
test_data/
test_input1.txt
tokens_test
And when I run the executable:
[jibberish]/__main__/tokens>./tokens_test
...normal test output...
The correct output!
...more test output...
I'm lost as to where to start looking for the problem. I've tried including a BUILD in the test_data directory with an exports_files rule, using a variety of relatives paths in the BUILD files and my source code and a lot of permutations of those relative paths.

Define your data as
data = ["test_data/test_input1.txt"]
Use absolute path: ./tokens/test_data/test_input1.txt
To find it run your test with a bazel test --sandbox_debug -s. The last command will looks like that one:
SUBCOMMAND: # //:hello_test [action 'Testing //:hello_test', configuration: faff19e6fd939f490ac11578d94024c6b7a032836cde039fd5edd28b838194e8, execution platform: #local_config_platform//:host]
(cd /home/s/.cache/bazel/_bazel_s/fa4c7c7c7db2888182e4f15990b55d58/execroot/com_google_absl_hello_world && \
exec env - \
EXPERIMENTAL_SPLIT_XML_GENERATION=1 \
RUNFILES_DIR=bazel-out/k8-fastbuild/bin/hello_test.runfiles \
RUN_UNDER_RUNFILES=1 \
TEST_BINARY=./hello_test \
TEST_INFRASTRUCTURE_FAILURE_FILE=bazel-out/k8-fastbuild/testlogs/hello_test/test.infrastructure_failure \
TEST_LOGSPLITTER_OUTPUT_FILE=bazel-out/k8-fastbuild/testlogs/hello_test/test.raw_splitlogs/test.splitlogs \
TEST_PREMATURE_EXIT_FILE=bazel-out/k8-fastbuild/testlogs/hello_test/test.exited_prematurely \
TEST_SIZE=medium \
TEST_SRCDIR=bazel-out/k8-fastbuild/bin/hello_test.runfiles \
TZ=UTC \
XML_OUTPUT_FILE=bazel-out/k8-fastbuild/testlogs/hello_test/test.xml \
external/bazel_tools/tools/test/test-setup.sh ./hello_test)
Then you can paste the whole (cd ...) one-liner to reproduce the exact sandbox environment as during the bazel test run. For example you can substitute the last line in that way:
(cd /home/s/.cache/bazel/_bazel_s/fa4c7c7c7db2888182e4f15990b55d58/execroot/com_google_absl_hello_world && \
exec env - \
EXPERIMENTAL_SPLIT_XML_GENERATION=1 \
RUNFILES_DIR=bazel-out/k8-fastbuild/bin/hello_test.runfiles \
RUN_UNDER_RUNFILES=1 \
TEST_BINARY=./hello_test \
TEST_INFRASTRUCTURE_FAILURE_FILE=bazel-out/k8-fastbuild/testlogs/hello_test/test.infrastructure_failure \
TEST_LOGSPLITTER_OUTPUT_FILE=bazel-out/k8-fastbuild/testlogs/hello_test/test.raw_splitlogs/test.splitlogs \
TEST_PREMATURE_EXIT_FILE=bazel-out/k8-fastbuild/testlogs/hello_test/test.exited_prematurely \
TEST_SIZE=medium \
TEST_SRCDIR=bazel-out/k8-fastbuild/bin/hello_test.runfiles \
TZ=UTC \
XML_OUTPUT_FILE=bazel-out/k8-fastbuild/testlogs/hello_test/test.xml \
bash -c 'pwd && ls')
So bash -c 'pwd && ls' will show the current directory path and content.

Related

No rule to make target 'bootstrap', needed by 'distdir-am'. Stop

I'm having the following problem when using automake to 'make dist'. Below is a snap shot of the error being reported by the compiler. I'm using Debian 10.5, with the default compiler version 8.0.3; autoconf 2.69; automake 1.16.1, libtool 2.4.6
######## Problem ########
aperri#debian:~/XerlangCPL2$ make dist
make dist-gzip am__post_remove_distdir='#:'
make[1]: Entering directory '/home/aperri/XerlangCPL2'
make distdir-am
make[2]: Entering directory '/home/aperri/XerlangCPL2'
make[2]: *** No rule to make target 'bootstrap', needed by 'distdir-am'. Stop.
make[2]: Leaving directory '/home/aperri/XerlangCPL2'
make[1]: *** [Makefile:633: distdir] Error 2
make[1]: Leaving directory '/home/aperri/XerlangCPL2'
make: *** [Makefile:710: dist] Error 2
I'm including my configure.ac and makefile.am in this message with the hope that there is a solution to this problem
######## configure.ac ########
# -*- Autoconf -*-
# Process this file with autoconf to produce a configure script.
AC_PREREQ([2.69])
AC_INIT([XerlangCPL], [1.0], [aperri1001#gmail.com])
AC_CONFIG_SRCDIR([src/xmlPROC.cpp])
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_AUX_DIR([build-aux])
AC_CONFIG_MACRO_DIR([m4])
# Enable "automake" to simplify creating Makefiles
AM_INIT_AUTOMAKE([1.16.1 subdir-objects -Wall -Werror])
m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
# Checks for programs.
AC_PROG_CC
AC_PROG_CXXCPP
AC_PROG_CXX
# Used in conjuction with {TARGET}_CPPFLAGS = -DDEBUG in Makefile.am
AM_PROG_CC_C_O
# Checks for libraries.
AX_BOOST_BASE([1.67], [], AC_MSG_ERROR([Could not find a useful version of boost]))
AX_BOOST_FILESYSTEM
AX_BOOST_SYSTEM
AX_BOOST_PROGRAM_OPTIONS
AX_BOOST_REGEX
# AX_BOOST_DATE_TIME
# AX_BOOST_THREAD
# Checks for header files.
# Checks for typedefs, structures, and compiler characteristics.
# Checks for library functions
PKG_CHECK_MODULES(libxml, libxml++-2.6 >= 2.10.0 )
AC_CHECK_PROGS([DOXYGEN], [doxygen])
if test -z "$DOXYGEN";
then AC_MSG_WARN([Doxygen not found - continuing without Doxygen support])
fi
AM_CONDITIONAL([HAVE_DOXYGEN], [test -n "$DOXYGEN"])
AC_CONFIG_FILES([Makefile doc/Doxyfile])
# AM_CONDITIONAL([DEBUG], [test x$debug = xtrue])
# AC_ARG_WITH(debug, [ --with-debug add the debugging module], [AC_DEFINE(WITH_DEBUG,1,0)
# AC_SUBST(WITH_DEBUG,1)
# CXXFLAGS="-O0 -ggdb"])
AC_OUTPUT
echo "
XErlang Compiler ($PACKAGE_NAME) version $PACKAGE_VERSION
Prefix.........: $prefix
Debug Build....: $debug
C++ Compiler...: $CXX $CXXFLAGS $CPPFLAGS
Linker.........: $LD $LDFLAGS $LIBS
"
######## makefile.am ########
ACLOCAL_AMFLAGS = -I m4 --install
bin_PROGRAMS = xerlangCPL
xerlangCPL_SOURCES = \
src/xmlPROC.cpp \
src/xml_structs.h \
src/debug.h \
src/conxsTracker.cpp \
src/conxs_tracker.h \
src/xmlConxsParser.cpp \
src/oven_control
xerlangCPL_LDFLAGS = -DDEBUG \
$(libxml_LIBS) \
$(BOOST_LDFLAGS) \
$(BOOST_SYSTEM_LDFLAGS) \
$(BOOST_FILESYSTEM_LDFLAGS) \
$(BOOST_PROGRAM_OPTIONS_LDFLAGS)
xerlangCPL_CPPFLAGS = $(libxml_CFLAGS) \
$(BOOST_CPPFLAGS)
xerlangCPL_LIBS = $(BOOST_SYSTEM_LIBS) \
$(BOOST_FILESYSTEM_LIBS) \
$(BOOST_PROGRAM_OPTIONS_LIBS)
xerlangCPL_CXXFLAGS = -lboost_filesystem -ldl -lboost_system
# start of Doxygen section
if HAVE_DOXYGEN
doxyfile.stamp:
$(DOXYGEN) $(top_srcdir)/doc/Doxyfile
echo Timestamp > $#
CLEANFILES = $(top_srcdir)/doxyfile.stamp
# all-local: doxyfile.stamp
all-local: doxyfile.stamp
# clean-local:
# rm -rf $(top_srcdir)/
endif
EXTRA_DIST = bootstrap m4/NOTES
The problem could be related to the inclusion of Boost Modules or the lack of them.
The problem is that EXTRA_DIST lists some mysterious file called bootstrap.
Makefile.am provides no instructions for building this file.
Therefore, this file is expected to already exist in the build directory, and it does not.
That's the reason for your make dist failure.
The reason for the missing file is something that you'll have to figure out on your own, the only thing that can be stated here is that its absence is the reason for your build failure.
You could, of course
touch bootstrap
and create an empty file in the build directory, which will be good enough. But you should investigate, in your project, what this file is, and where it should come from, and fix things accordingly.

build PDAL with LAZperf

I'm building PDAL this way in my Ubuntu 18 :
cd /home/magno/install && \
git clone https://github.com/hobu/laz-perf.git && \
cd laz-perf && \
mkdir build && \
cd build && \
cmake .. \
-DEMSCRIPTEN=1 \
-DCMAKE_TOOLCHAIN_FILE=/home/magno/install/emsdk/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake && \
VERBOSE=1 make && \
make install
cd /home/magno/install && \
git clone https://github.com/pgpointcloud/pointcloud && \
cd pointcloud && \
./autogen.sh && \
./configure --with-lazperf=/usr/local/ && \
make && \
make install
cd /home/magno/install && \
git clone https://github.com/PDAL/PDAL.git && \
cd PDAL && \
mkdir build && \
cd build && \
cmake -G Ninja .. && \
ninja && \
ninja install
Running PGUSER=postgres PGPASSWORD=*** PGHOST=localhost PGPORT=5432 ctest can confirm all was fine.
But when I try to check a LAZ file I'm getting this error:
PDAL: readers.las: Can't read compressed file without LASzip or LAZperf decompression library.
This is my pipe file:
{
"pipeline":[
{
"type":"readers.las",
"filename":"airport.laz",
"spatialreference":"EPSG:32616",
"compression":"lazperf"
},
{
"type":"writers.pgpointcloud",
"connection":"dbname=mydb host='localhost' user='postgres' password='****'",
"table":"patchs",
"compression":"lazperf",
"srid":"32616",
"overwrite":"false"
}
]
}
I think lazperf is ok because pgpointcloud doesn't complains with PGUSER=postgres PGPASSWORD=**** PGHOST=localhost make installcheck and tells me :
# PointCloud is now configured for
# -------------- Compiler Info -------------
# C compiler: gcc -g -O2
# SQL preprocessor: /usr/bin/cpp -traditional-cpp -w -P
# -------------- Dependencies --------------
# PostgreSQL config: /usr/bin/pg_config
# PostgreSQL version: PostgreSQL 12.3 (Debian 12.3-1.pgdg100+1) (120)
# Libxml2 config: /usr/bin/xml2-config
# Libxml2 version: 2.9.4
# LazPerf status: /usr/local//include/laz-perf
# CUnit status: enabled
PDAL tests tells me nothing about compression.
How can I build or tell PDAL about my LAZPerf installation?
EDIT pdal info install/PDAL/test/data/las/autzen_trim.las is all ok .
God bless the Google!
Found the solution by reading this, this and this.
Just need to change to cmake -G Ninja -DLazperf_DIR=/usr/local/ -DWITH_LAZPERF=ON ..
and voilĂ :
-- The following OPTIONAL packages have been found:
* Lazperf
* ZSTD
General compression support
* LibXml2
* PkgConfig
* PythonInterp

tesseract command not working and giving file error

I have installed tesseract version 4.0 in ubuntu.
I am able to perform all the actions of tesseract using Tesseract CLI like simple OCR text generation.
I want to train the LSTM.
I read this article and tried to run the following command directly on terminal after isntalling Tesseract from Build.
mkdir -p ~/tesstutorial/engoutput
training/lstmtraining --debug_interval 100 \
--traineddata ~/tesstutorial/engtrain/eng/eng.traineddata \
--net_spec '[1,36,0,1 Ct3,3,16 Mp3,3 Lfys48 Lfx96 Lrx96 Lfx256 O1c111]' \
--model_output ~/tesstutorial/engoutput/base --learning_rate 20e-4 \
--train_listfile ~/tesstutorial/engtrain/eng.training_files.txt \
--eval_listfile ~/tesstutorial/engeval/eng.training_files.txt \
--max_iterations 5000 &>~/tesstutorial/engoutput/basetrain.log
Althoguh it created the engouput directory.
Current path was pointed to SRC directory of tesseract.
Get the following error :
bash: training/lstmtraining: No such file or directory
Running as
Fixed by following code
Create Training Data First
cd ~/tesseract-ocr/src
training/tesstrain.sh \
--fonts_dir /usr/share/fonts/ \
--lang eng \
--linedata_only \
--noextract_font_properties \
--exposures "0" \
--langdata_dir /home/shan/langdata_lstm \
--output_dir /home/shan/tesstutorial/engtrain \
--tessdata_dir /home/shan/tesseract-ocr/tessdata \
--fontlist "Arial"
sudo chmod -R 777 /home/shan/tesstutorial/engtrain
Then LSTM Model
sudo chmod -R 777 /home/shan/tesstutorial/
cd ~/tesseract-ocr/src/
training/lstmtraining --stop_training \
--continue_from ~/tesstutorial/engoutput/base_checkpoint \
--traineddata ~/tesstutorial/engtrain/eng/eng.traineddata \
--model_output ~/tesstutorial/engoutput/eng.traineddata
sudo chmod -R 777 ~/tesstutorial
cd ~/tesseract-ocr/src/
training/lstmtraining --debug_interval 100 \
--traineddata ~/tesstutorial/engtrain/eng/eng.traineddata \
--net_spec '[1,36,0,1 Ct3,3,16 Mp3,3 Lfys48 Lfx96 Lrx96 Lfx256 O1c111]' \
--model_output ~/tesstutorial/engoutput/base --learning_rate 20e-4 \
--train_listfile ~/tesstutorial/engtrain/eng.training_files.txt \
--max_iterations 5000 &>~/tesstutorial/engoutput/basetrain.log

Cannot totally compile and link cpp project as a static library

I have followed this link and adapted its steps to make it work for my project.
My goal is to create a libfile.a to distribute as a static library. The project tree is the following:
project
|
+-src
+- <some cpp and hpp files>
|
+ containers
|
+- <other cpp and hpp files>
I easily created configure.ac file and the Makefile.ams. The tree structure changed this way:
project
|
+- configure.ac
+- Makefile.am
+-src
+- <some cpp and hpp files>
+ Makefile.am
|
+ containers
|
+- <other cpp and hpp files>
Now, when I go: (*)
aclocal; autoreconf --install; autoconf; ./configure
make
.o files are generated for all the .*pp files contained in src, but it fails when it starts generating those targets in src/containers. So the Makefile is not generated properly. What am I doing wrong? Can someone help me?
PS here there are files involved:
# --- configure.ac ---
AC_PREREQ([2.68])
AC_INIT([filea], [1.0], [dev#host.net])
AM_INIT_AUTOMAKE([filea], [1.0])
AC_CONFIG_SRCDIR([src/HashFunctions.cpp])
AC_CONFIG_HEADERS([config.h])
AC_PROG_CXX
AC_PROG_RANLIB
AC_CHECK_HEADERS([stddef.h stdint.h string.h])
AC_HEADER_STDBOOL
AC_C_INLINE
AC_TYPE_SIZE_T
AC_TYPE_UINT16_T
AC_TYPE_UINT32_T
AC_TYPE_UINT8_T
AC_FUNC_MALLOC
AC_FUNC_MKTIME
AC_CHECK_FUNCS([memset])
AC_OUTPUT([Makefile src/Makefile])
# --- Makefile.am ---
AUTOMAKE_OPTIONS = foreign
SUBDIRS = src
# --- src/Makefile.am ---
lib_LIBRARIES = libfile.a
libfile_a_SOURCES = \
ConfigLib.hpp \
ConfigLib.cpp \
HashFunctions.cpp \
HashFunctions.hpp \
Logger.hpp \
Logger.cpp \
Queue.hpp
libfile_a_SOURCES += \
containers/SafeContainer.cpp \
containers/SafeInterger.cpp \
containers/SafeMap.cpp
EDIT 1
as suggested by Brett Hale the commands marked with (*) have been replaced by the following ones:
autoreconf -fvi
Output:
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: running: /usr/bin/autoconf --force
autoreconf: running: /usr/bin/autoheader --force
autoreconf: running: automake --add-missing --copy --force-missing
autoreconf: Leaving directory `.'
When going with:
./configure
make
Still no rules are found to generate targets in the subdirectory.
EDIT 2 I switched to a non-recursive approach (Thanks to Karel Zak's blog) and finally I can makemy lib.
Still I don't know what I was doing wrong following the typical recursive approach; but, finally I made this work, switching to a non-recursive approach (as I wrote in EDIT 2). This article on Karel Zak's Blog helped me a lot!
# -- new configure.ac file --
AC_PREREQ([2.68])
AC_INIT([filea], [1.0], [dev#host.net])
AM_INIT_AUTOMAKE([filea], [1.0])
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_SRCDIR([src/HashFunctions.cpp])
AM_INIT_AUTOMAKE([filea], [1.0])
LT_INIT
AC_CANONICAL_HOST
AC_PROG_LIBTOOL
AC_PROG_GREP
AC_PROG_EGREP
AC_PROG_CXX
...
AC_HEADER_STDBOOL
AC_C_INLINE
AC_TYPE_PID_T
AC_TYPE_SIZE_T
AC_TYPE_SSIZE_T
AC_TYPE_UINT8_T
AC_TYPE_UINT16_T
AC_TYPE_UINT32_T
AC_FUNC_ERROR_AT_LINE
AC_FUNC_FORK
AC_FUNC_MALLOC
AC_FUNC_MKTIME
AC_CHECK_FUNCS([memset socket])
AC_OUTPUT([Makefile]) # the makefile is only one!
# In the subdirectory I have created few
# Makemodule.am included in the "main makefile.am"
# --- makefile.am ---
AUTOMAKE_OPTIONS = foreign
lib_LIBRARIES = filea.a
libfilea_a_SOURCES =
include src/Makemodule.am
include src/containers/Makemodule.am # now filea_a_SOURCES is a
# "global variable"
# -- src/Makemodule.am
libfilea_a_SOURCES += \
src/ConfigLib.hpp \
src/ConfigLib.cpp \
src/HashFunctions.cpp \
src/HashFunctions.hpp \
src/Logger.hpp \
src/Logger.cpp \
src/Queue.hpp
# -- src/containers/Makemodule.am
libfile_a_SOURCES += \
src/containers/SafeContainer.cpp \
src/containers/SafeInterger.cpp \
src/containers/SafeMap.cpp
Notice that now, even if Makemodule.am files are placed at different levels in the dir tree, whenever a filename is typed in one those modules, it has to be preceded by its relative path. This path is relative to Makefile location.

.sym not being generated by -export-symbols in Makefile.am

I am trying to build a shared library by editing Wireshark's Makefile.am to create a "libtshark" ... to do this, I am trying to build a tshark_lib.c which is the same as tshark.c with main() stripped out of it.
So I have edited Makefile.am to include include the following in an attempt to build the shared libtshark library:
lib_LTLIBRARIES = libtshark.la
libtshark_la_LDFLAGS = -version-info 0:1:0 -export-symbols libtshark.sym #LDFLAGS_SHAREDLIB#
libtshark_la_SOURCES = \
$(WIRESHARK_COMMON_SRC) \
$(SHARK_COMMON_CAPTURE_SRC) \
$(TSHARK_TAP_SRC) \
capture_opts.c \
capture_sync.c \
tempfile.c \
tshark-tap-register.c \
tshark_lib.c
# Libraries and plugin flags with which to link tshark.
libtshark_la_LIBADD = \
wiretap/libwiretap.la \
epan/libwireshark.la \
wsutil/libwsutil.la \
#SSL_LIBS# \
$(plugin_ldadd) \
#PCRE_LIBS# \
#GLIB_LIBS# -lm \
#PCAP_LIBS# \
#SOCKET_LIBS# \
#NSL_LIBS# \
#C_ARES_LIBS# \
#ADNS_LIBS# \
#KRB5_LIBS# \
#CORESERVICES_FRAMEWORKS# \
#LIBGCRYPT_LIBS# \
#LIBGNUTLS_LIBS# \
#LIBSMI_LDFLAGS#
#tshark_CFLAGS = $(AM_CLEAN_CFLAGS) $(py_dissectors_dir)
libtshark_la_DEPENDENCIES = \
${top_builddir}/epan/libwireshark.la \
${top_builddir}/wsutil/libwsutil.la \
${top_builddir}/wiretap/libwiretap.la \
libtshark.sym
However, when I try to build the code now, I get the following error:
make[2]: *** No rule to make target `libtshark.sym', needed by `libtshark.la'. Stop.
For some reason, it is not using my 'libtshark_la_LDFLAGS' which contains '-export-symbols libtshark.sym' ... does anyone know what I might be missing?
Here is the entire Makefile.am, which is too long for stackoverflow: Makefile.am
it turns out that the .sym file is pre-generated by wireshark scripts to determine which symbols to export.