I am building a new project using Eliom and having trouble setting up a compilation process for unit tests using OUnit.
I have two files:
Js_Client_Code.eliom - contains all of the client side code
Project.eliom - contains all of the server side code (including opening Js_Client_Code.eliom)
I have set up the files in this way so that I can run unit tests on Js_Client_Code.eliom without using ocsigenserver.
This is my current makefile for the tests:
test: prep unit_tests
prep:
eliomc -infer Js_Client_Code.eliom
js_of_eliom -linkall -a Js_Client_Code.eliom -o file_a.cma
eliomc -a -linkall Js_Client_Code.eliom -o file_b.cma
# Code is here to move the cma files back to the parent directory, since they are written to _client/ and _server/ by default
unit_tests:
ocamlfind ocamlc -thread -syntax camlp4o -package ounit,js_of_ocaml.syntax \
-linkpkg -g -o UnitTests file_a.cma file_b.cma unit_tests.ml
Running make test in the shell produces
File "unit_tests.ml", line 1:
Error: Error while linking file_a.cma(Js_Client_Code):
Reference to undefined global `Eliom_client'
Am I misunderstanding the Eliom/Js_of_ocaml compilation process, or just going about this the wrong way?
You're not linking the eliom libraries at all, so of course it's missing all the external modules.
Honestly, I don't think you'll manage to unit test modules that depends on eliom that way. It would probably be easier to unit test the non-eliom part indivdually, and do end-to-end browser-based tests (there are lot's of framework for that).
If you really want to try, there are explanations on eliom (or ocsigenserver)'s manual to link statically.
Related
I have a big project built by menhir and traditional makefile. First, I wanted to add a mechanism of error handling like this project to my project.
By following the dune of the sample project, I managed to generate .mly, .mli, .ml, .cmi and .cmo of unitActionsParser_e.mly by the following commands:
$ menhir --only-preprocess-u parser_e.mly > unitActionsParser_e.mly
$ menhir --table --external-tokens Parser_e unitActionsParser_e.mly
$ ocamlfind ocamlc -package menhirLib -c unitActionsParser_e.mli
And the incremental API and error handling did work.
Then, I wanted to add error recovery like this project to my project. Then, items state raised an error Error: Unbound value items in my project. Based on the manual and the dune, I guess I need to add --inspection somewhere.
I tried menhir --explain --inspection --table --dump --infer --external-tokens Parser_e unitActionsParser_e.mly, then camlfind ocamlc -package menhirLib -c unitActionsParser_e.mli raised an error Unbound type constructor Parser_e.terminal.
I also tried to directly work on parser_e.mly rather than using unitActionsParser_e by menhir --explain --inspection --table --dump --infer parser_e.mly, but it returned an error Unbound module Utility where Utility is a module in another folder needed by parser_e.mly. After I manually copied utility.cm* to the folder of parser_e.mly, it returned an error Unbound module Sedlexing (here is a fork where we can reproduce the error) (this is probably related to Interaction with build systems of the manual).
Does anyone know what are the correct commands and flags to generate a parser (either UnitActionsParser_e or Parser_e) enabling incremental API and inspection API of menhir?
(* link in discuss.ocaml.org: https://discuss.ocaml.org/t/generate-a-parser-enabling-incremental-api-and-inspection-api/9380 *)
This question is indeed related to the interaction of menhir and build systems. Precisely, the inspection API (--inspection) requires the type information of .mly (including its semantic actions) to be known. I chose to directly work on parse_e.mly rather than unitActionsParser_e.mly, and followed the approach of "Obtaining OCaml type information without calling the OCaml compiler":
menhir --explain --inspection --table --dump --infer-write-query mockfile.ml parser_e.mly to generate mockfile.ml
ocamlfind ocamlc -I lib -package sedlex -package menhirLib -i mockfile.ml > sigfile to generate sigfile. Note that -I lib refers to the directory of external modules, their .cm[io] files should be ready to use.
menhir --explain --inspection --table --dump --infer-read-reply sigfile parser_e.mly to generate especially parser_e.ml, parser_e.mli, .conflicts and automaton.
As a result, parser_e.ml contains more type information and the inspection API is in Parser_e.MenhirInterpreter.
I am using QT Creator to work on a medium-sized project in C++.
The project structure basically looks like this
Project
Group A
Library A1
Group B
Library B1
Library B2
...etc
Test
LibA1_Test
LibB1_Test
LibB2_Test
...etc
The libraries are tested by the executables in the test project. I've managed to compile the tests themselves with gcov enabled, and produce code coverage reports with lcov, but all that they were showing the coverage for were the test cases, not the actual code that I'm testing. I've also tried compiling the static libraries with gcov as well, but when I run the tests against those libraries it does not generate any of the gcov output files.
How could I generate the gcov output files by linking my project libraries against the tests? I want to see if there are any gaps in my unit tests.
From the ld manual
--whole-archive
For each archive mentioned on the command line after the --whole-archive option, include every object file in the archive
in the link, rather than searching the archive for the required object
files.
So link your static-library into your test using --whole-archive, which will result in your test binary having the entire static-library, and give gcov visibility of the entire code
To be honest, you don't really need --whole-archive, it's enough to just add the following to all the staticlibs and Unit-Tests, that you want gcov information from:
# only apply codecoverage when CodeCoverage is set and debug-mode is active
CodeCoverage
{
debug:QMAKE_CXXFLAGS_DEBUG += --coverage # -fprofile-arcs -ftest-coverage
debug:QMAKE_LFLAGS_DEBUG += --coverage # -lgcov
}
--coverage is context-sensitive and the meaning of it depends on whether you pass it to CXXFLAGS or LFLAGS:
https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html#Instrumentation-Options
Also mentioned in https://stackoverflow.com/a/5352713/7820869.
So no need to add -fprofile-arcs or any other mixture of flags.
In your case, add those Flags to A1.pro, B1.pro and B2.pro.
The added scope around the flags just prevent that the codecoverage-flags pollute your build in Release- or Debug-mode. Just pass "CONFIG+=CodeCoverage" to qmake.exe, so it will build with code-coverage:
https://doc.qt.io/archives/qt-4.8/qmake-advanced-usage.html#configuration-and-scopes
After that just check whether after compilation the *.gcno files and after running the Unit-Test's the *.gcda files have been generated in the build-directory and call lcov on them:
https://gcc.gnu.org/onlinedocs/gcc/Gcov-Data-Files.html#Gcov-Data-Files
Additional Info
The stackoverflow, that helped me solving my problems I run into:
Where are the gcov symbols?
in case you are running into linker errors like (undefined reference to __gcov_init and __gcov_merge_add), watch out that you are really adding those flags to every unit:
https://www.spinics.net/lists/gcchelp/msg29518.html
It seems like there are a few answers that kind-of, sort-of make sense, but that I don't know how to carry out. And I haven't found a comprehensive answer.
The First Problem
Google Test should not be an installed library, it should be built with the project. (See the FAQ.) As far as I can tell, this means the Google Test libraries are a dependency of my unit tests, and should be built when I run "make check" within my project for the first time. This should build Google Test libraries in some directory. I don't know how to do this. It mentions some autotools script that's deprecated, and I'm not sure what they're talking about or how to point my build at it properly.
The Second Problem
Assuming the build is successful, how do I write a test that uses my locally-compiled version of Google Test to run tests? I assume that there are a bunch of Makefile.am commands I put in my tests directory. But what are they? And what's an example of a unit test that uses Google Test?
I have solved the problem to my satisfaction! I will move on entirely now. This is basically asking for a tutorial. There are a lot of decisions that must be made, hopefully logically, so that Google Test dovetails nicely into autotools. So I apologize in advance for the long answer, but all the details should be there.
The First Problem
In order to understand the answer, the question needs to be rephrased a little. We are compiling Google Test as a library which our test code will link to. The library will not be installed. The question we want to ask is
"How do we configure autotools to compile Google Test as a library
which our test code can link against?"
In order to do that, we need to download Google Test and place it into our project. I use Github, so I do that by adding a submodule in the root path of my project:
$ git submodule add git#github.com:google/googletest.git
$ git submodule init
$ git submodule update
This downloads googletest into my root of my project:
/:
Makefile.am
configure.ac
src/:
(files for my project)
tests/:
(test files)
googletest/:
googletest/:
include/:
(headers, etc., to be included)
gtest/:
gtest.h
m4/:
(directory for m4 scripts and things)
src/:
(source files for Google Test)
I need to compile per the instructions. I only want the Google Test library to be built upon running make check, so I will use check_LTLIBRARIES. I add the following to my tests Makefile.am in /tests:
check_LTLIBRARIES = libgtest.la
libgtest_la_SOURCES = ../googletest/googletest/src/gtest-all.cc
libgtest_la_CPPFLAGS = -I$(top_srcdir)/googletest/googletest/include -I$(top_srcdir)/googletest/googletest
libgtest_la_LDFLAGS = -pthread
This requires subdir-objects to be enabled in configure.ac. That is accomplished by adding it to the AM_INIT_AUTOMAKE line. I also need to include the makefile in AC_CONFIG_FILES. We also want to use libtool, because we are compiling library files (I'll explain why and how that works in a moment). To use libtool, we add AM_PROG_AR, LT_INIT. We want autoreconf to install m4 macros to /m4, and then we want automake to find them, so we need AC_CONFIG_MACRO_DIRS. My configure.ac has lines updated:
AM_INIT_AUTOMAKE([-Wall -Werror subdir-objects])
...
AM_PROG_AR
LT_INIT
AC_CONFIG_MACRO_DIRS([m4])
...
AC_CONFIG_FILES([Makefile
src/Makefile
tests/Makefile
])
I also need to include the subdirectory and a line pointing to the macros in the /m4 macros directory in my /Makefile.am:
ACLOCAL_AMFLAGS = -I m4
SUBDIRS = src tests
What has this done? Libtool has been enabled with AM_PROG_AR and LT_INIT. The check_LTLIBRARIES means we will use libtool to create what's called a convenience library called libgtest.la. With subdir-objects enabled, it will be built into the /tests directory, but not installed. This means that, whenever we want to update our tests, we don't have to recompile the Google Test library libgtest.la. This will save time when testing and help us iterate faster. Then, we will want to compile our unit tests against it later as we update them. The library will only be compiled upon running make check, saving time by not compiling it if all we want to do is make or make install.
The Second Problem
Now, the second problem needs to be refined: How do you (a) create a test (b) that is linked to the Google Test libraries and thus uses them? The questions are kind of intertwined, so we answer them at once.
Creating a test is just a matter of putting the following code into a gtest.cpp file located at /tests/gtest.cpp:
#include "gtest/gtest.h" // we will add the path to C preprocessor later
TEST(CategoryTest, SpecificTest)
{
ASSERT_EQ(0, 0);
}
int main(int argc, char **argv)
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
This runs only the simple test 0=0. To create a test for your library, you need to read the primer. You'll notice we don't need a header for this (yet). We are linking to the file "gtest/gtest.h", so we'll need to make sure that we tell automake to include a directory that has gtest/gtest.h.
Next, we need to tell automake that we want to build a test and run it. The test is going to build into an executable that we don't want to install. Then automake is going to run that executable. It will report whether that executable says the tests passed or failed.
Automake does that by looking in the makefile for the variable check_PROGRAMS. These are the programs it will compile, but it won't necessarily run them. So we add to /tests/Makefile.am:
check_PROGRAMS = gtest
gtest_SOURCES = gtest.cpp
gtest_LDADD = libgtest.la
gtest_LDFLAGS = -pthread
gtest_CPPFLAGS = -I$(top_srcdir)/googletest/googletest/include -I$(top_srcdir)/googletest/googletest -pthread
The gtest_SOURCES finds the /tests/gtest.cpp file and compiles it. gtest_LDADD links against libgtest.la which will be compiled into the /tests directory. Google wants us to use the gtest_LDFLAGS line to enable pthreads. Finally, we need to include the location where the header "gtest/gtest.h" will be found, and that is the gtest_CPPFLAGS line. Google also wants us to include the /googletest/googletest location, and include the
The state of things: The Google Test library libgtest.la will compile with make into the directory /tests, but not be installed. The binary gtest will only be compiled with make check, but will not be installed.
Next we want to tell automake to actually run the compiled binary gtest and report errors. This is accomplished by adding a line to /tests/Makefile.am:
TESTS = gtest
The final /tests/Makefile.am looks like this:
check_LTLIBRARIES = libgtest.la
libgtest_la_SOURCES = ../googletest/googletest/src/gtest-all.cc
libgtest_la_CPPFLAGS = -I$(top_srcdir)/googletest/googletest/include -I$(top_srcdir)/googletest/googletest -pthread
check_PROGRAMS = gtest demo
gtest_SOURCES = gtest.cpp ../src/fields.cpp
gtest_LDADD = libgtest.la
gtest_LDFLAGS = -pthread
gtest_CPPFLAGS = -I$(top_srcdir)/googletest/googletest/include -I$(top_srcdir)/src
demo_SOURCES = demo.cpp ../src/fields.cpp
demo_CPPFLAGS = -I$(top_srcdir)/src
TESTS = gtest
Now, autoreconf -fiv (note any errors and hopefully fix them) from /, and make check and you should get a test that runs:
build(dev)$ make check
Making check in tests
/Applications/Xcode.app/Contents/Developer/usr/bin/make gtest
make[2]: `gtest' is up to date.
/Applications/Xcode.app/Contents/Developer/usr/bin/make check-TESTS
PASS: gtest
============================================================================
Testsuite summary for IonMotion 0.0.1
============================================================================
# TOTAL: 1
# PASS: 1
# SKIP: 0
# XFAIL: 0
# FAIL: 0
# XPASS: 0
# ERROR: 0
============================================================================
Here is a sample Makefile.am for the unit test project (project name: TestProject). It depends on GTEST and GMOCK:
Makefile.am
#######################################
# The list of executables we are building seperated by spaces
# the 'bin_' indicates that these build products will be installed
# in the $(bindir) directory. For example /usr/bin
#bin_PROGRAMS=exampleProgram
# Because a.out is only a sample program we don't want it to be installed.
# The 'noinst_' prefix indicates that the following targets are not to be
# installed.
noinst_PROGRAMS=utTestProject
#######################################
# Build information for each executable. The variable name is derived
# by use the name of the executable with each non alpha-numeric character is
# replaced by '_'. So a.out becomes a_out and the appropriate suffex added.
# '_SOURCES' for example.
# Sources for the a.out
utTestProject_SOURCES= \
utTestProject.cpp
# Library dependencies
utTestProject_LDADD = \
$(top_srcdir)/../TestProject/build/${host}/libTestProject/.libs/libTestProject.a \
../$(PATH_TO_GTEST)/lib/libgtest.a \
../$(PATH_TO_GMOCK)/lib/libgmock.a
# Compiler options for a.out
utTestProject_CPPFLAGS = \
-std=c++11 \
-I../$(PATH_TO_GTEST)/include \
-I../$(PATH_TO_GMOCK)/include \
-I$(top_srcdir)/include \
-I$(top_srcdir)/..
TESTS = utTestProject
TESTS_ENVIRONMENT = export UT_FOLDER_PATH=$(top_srcdir)/utTestProject; \
export GTEST_OUTPUT="xml";
Compiling gtest:
# Useful vars
SourceVersionedArchiveFolderName="gtest-1.7.0"
#
# Make it
#
pushd .
cd ./${SourceVersionedArchiveFolderName}/make
make gtest.a
if [ $? != 0 ]; then
echo "$0: Make failed"
exit 1
fi
popd
It's worth noting that Googletest doesn't officially maintain its Autotools integration anymore:
Before settling on CMake, we have been providing hand-maintained build
projects/scripts for Visual Studio, Xcode, and Autotools. While we
continue to provide them for convenience, they are not actively
maintained any more. We highly recommend that you follow the
instructions in the above sections to integrate Google Test with your
existing build system.
https://github.com/google/googletest/tree/master/googletest#legacy-build-scripts
It's now recommended to build Googletest with CMake.
Making GoogleTest's source code available to the main build can be
done a few different ways:
Download the GoogleTest source code manually and place it at a known
location. This is the least flexible approach and can make it more
difficult to use with continuous integration systems, etc.
Embed the
GoogleTest source code as a direct copy in the main project's source
tree. This is often the simplest approach, but is also the hardest to
keep up to date. Some organizations may not permit this method.
Add
GoogleTest as a git submodule or equivalent. This may not always be
possible or appropriate. Git submodules, for example, have their own
set of advantages and drawbacks.
Use CMake to download GoogleTest as
part of the build's configure step. This is just a little more
complex, but doesn't have the limitations of the other methods.
https://github.com/google/googletest/tree/master/googletest#incorporating-into-an-existing-cmake-project
I am starting to use GoogleTest. It seems that it needs a main file for running the tests:
Separate test cases across multiple files in google test
But currently in my demo application I already have a main file:
src/
-> MyType.h
-> main.cpp
-> Makefile
Which will eventually be my "production" application. I don't want to clutter that with gtest includes, macros etc.
Should I just create another main.cpp file in another folder e.g.: test/ that will contain all the specific gtest configuration so I would end up with:
src/
-> MyType.h
-> main.cpp
-> Makefile // Makefile for producing production code/binaries
Test/
-> MyTypeTest.h // Unittest for MyType
-> main.cpp // The "Test runner"
-> Makefile // Makefile for producing test executable
EDIT:
Found this based on cmake:
http://www.kaizou.org/2014/11/gtest-cmake/
which seems to be exactly what I am looking for.
The most sensible approach to this is to have a library for your production code and then two executables, one for production and another one for tests:
|-lib/
| |-Makefile
| |-mytype.h
| `-mytype.cpp
|-app/
| |-Makefile
| `-main.cpp
`-test/
|-Makefile
`-mytypetest.cpp
Notice that gtest distribution provides the gtest library and a gtest_main library with the standard main function for your test executable. So unless you need a custom main (rare case) you don't need to provide a main.cpp for your tests and can simply link against gtest_main, e.g. $(CC) mytypetest.cpp -o apptests -lapplib -lgtest_main -lgtest.
The library approach involves slightly more complex Makefiles, but it pays off in compilation time, since not having it implies you need to compile mytype.cpp once for production application and once for test executable.
There are probably a lot of ways to do this, but generally speaking, yes, you should add a test-specific main function to your project. This makes compilation a little bit more complex since you'll have to produce two separate binaries (one for your application and another for your tests) but this is a fairly typical setup.
I'd simply add a test.cpp file with a main and create a test target in my makefile so that I could either make - to build my production code - or make test - to build the tests. In actual projects I use cmake in very similar fashion (I sometimes bundle all common dependencies in a core.a library and then link both main and test against it).
I am trying to use ocamlfind to install a library. I am using OCamlMakeFile. Here is my Makefile:
OCAMLMAKEFILE = OCamlMakeFile
RESULT = owebl
SOURCES = src/utils.ml src/verb.ml src/request.ml src/template.ml src/response.ml src/rule.ml src/handler.ml src/server.ml
PACKS = unix str
all: native-code-library byte-code-library
install: libinstall
uninstall: libuninstall
include $(OCAMLMAKEFILE)
So basically the library is a collection of modules that I want to distribute. A basic usage of the library is (main.ml):
open Response
open Rule
open Verb
open Server
let handler =
Handler.create
(StaticRouteRule.create "/" [GET])
(FileResponse.create
~static_file:(FileResponse.StaticFile "/index.html")
())
let server = Server.create "0.0.0.0" 8080 [handler];;
server#serve
I compile the library just by running "make". It generates three files: "owebl.a, owebl.cma, owebl.cmxa". Then I install the library using ocamlfind:
ocamlfind install owebl META owebl.a owebl.cma owebl.cmxa
Oh, the META file is:
version = "0.1"
name = "OWebl"
description = "A Web Framework for OCaml"
archive(byte) = "owebl.cma"
archive(native) = "owebl.cmxa"
requires = "unix,str"
Everything works until I try to compile the above example that uses the library. I run:
ocamlbuild -use-ocamlfind -pkgs owebl main.native
And I get:
ocamlbuild -use-ocamlfind -pkgs owebl main.native
+ ocamlfind ocamlc -c -package owebl -o main.cmo main.ml
File "main.ml", line 1, characters 5-10:
Error: Unbound module Owebl
Command exited with code 2.
Hint: Recursive traversal of subdirectories was not enabled for this build,
as the working directory does not look like an ocamlbuild project (no
'_tags' or 'myocamlbuild.ml' file). If you have modules in subdirectories,
you should add the option "-r" or create an empty '_tags' file.
To enable recursive traversal for some subdirectories only, you can use the
following '_tags' file:
true: -traverse
<dir1> or <dir2>: traverse
Compilation unsuccessful after building 2 targets (1 cached) in 00:00:00.
make: *** [fileserver] Error 10
I tried adding open Owebl to the beginning of main.ml but I just got "unbound module Owebl" similarly. I'm pretty lost as to why these modules are all unbound. What am I missing?
You should use LIB_PACK_NAME variable if you want to pack your modules under umbrella namespace, like OWebl, otherwise they will be available just as it is, e.g. Response, Rule, etc. Also, you should specify the mli files as well as ml files in SOURCES variable. And finally, you shouldn't forget to install cmi files, as well as .a files. Installing mli files is also considered a good tradition, although is not strictly required. Also, consider using OCamlMakefile installation facilities instead of custom installation script.
I am no longer using OCamlMakefile therefore am not sure, but assuming OCamlMakefile compiles things with -for-pack OWebl option and properly builds a packaged module owebl.cmo, you also need to install owebl.cmi.
OCaml compiler seeks cmi files to check the existence of modules and to get their type information. If not installed, the compiler cannot find it. cma and cmxa files are collections of objects and do not provide type information.