I am interested in generating a C++ header using Apache Avro's code generation tool (i.e. the python script). According to the documentation it should be fairly easy to do, but I don't usually use python, so things look kinda strange to me.
The instructions state:
To generate the code is a two step process:
precompile < imaginary > imaginary.flat
The precompile step converts the schema into an intermediate format that is used by the code generator. This intermediate file is just a text-based representation of the schema, flattened by a depth-first-traverse of the tree structure of the schema types.
python scripts/gen-cppcode.py --input=example.flat --output=example.hh –-namespace=Math
This tells the code generator to read your flattened schema as its input, and generate a C++ header file in example.hh. The optional argument namespace will put the objects in that namespace...
My Issue (no, I can't see a doctor or use a cream for it):
I don't see anything that explains in details how to precompile. The documentation makes it seem like if I just type "precompile" in the command prompt and supply the command line arguments, then things would magically work, but precompile is not a valid Windows command. So what's the proper way to precompile on Windows? If anybody knows how to do it, then PLEASE let me know!
I also tried to run the gen-cppcode.py script, but it gets an error in line 316 (which, I suspect, may be happening because I didn't precompile the schema):
def doEnum(args):
structDef = enumTemplate;
typename = args[1]
structDef = structDef.replace('$name$', typename)
end = False
symbols = '';
firstsymbol = '';
while not end:
line = getNextLine()
if line[0] == 'end': end = True
elif line[0] == 'name':
if symbols== '' :
firstsymbol = line[1]
else :
symbols += ', '
symbols += line[1]
else: print "error" // <-- Syntax Error: invalid syntax
structDef = structDef.replace('$enumsymbols$', symbols);
structDef = structDef.replace('$firstsymbol$', firstsymbol);
addStruct(typename, structDef)
return (typename,typename)
About the only way I figured to do this is to:
Download VirtualBox.
Install Ubuntu (or another distro).
Download Avro.
Install cmake.
Install the C++ compilers (build essential).
Install boost, flex, bison (sudo apt-get install boost flex bison); btw, you will specifically need these boost libraries:
-- regex
-- filesystem
-- system
-- program_options
Build Avro:
$ tar xf avro-cpp-1.5.1.tar.gz
$ cd avro-cpp-1.5.1
$ cmake -G "Unix Makefiles"
$ make -j3
$ build/precompile file.input file.flatoutput
You can now generate a header file (still in the terminal window of the VM):
python scripts/gen-cppcode.py --input=example.flat --output=example.hh
Note that even after you generate the C++ file, you will still be unable to build with it in Windows (even if you have the right dependency includes to the avro-cpp-1.5.1/api. Avro has dependencies on GNU libraries (such as sys/uio.h) and I'm not sure how to specifically resolve them yet.
I found that it is required python version 2 to run gen-cppcode.py
https://www.python.org/downloads/
Related
I've followed every single guide I could possibly find but to be completely honest I have no idea what some installation 'steps' even mean.
I tried installing Cygwin (and MYSY) and running the commands that the guides told me to, but the terminal either doesn't do anything or it gives me the error: 'no such file or directory'. (I changed the folder directory to where my files where)
Also I'm not entirely sure I installed everything correctly because I should've checked some add-ons during the installation right? I did as in the guide but still I maybe missed something...
Could someone please explain to me step by step how to install it saying explicitly all that has to be done, (I'm running windows 7) considering It's the first time I do such thing and I have no idea what ./configure , make and all the other commands even mean...
The following is a simple step by step using only cygwin tools.
To compile C++ we need the g++ compiler; to locate the correct package to be installed the cygwin tool is cygcheck (that is installed by default), with the -p switch it interrogates the database at https://cygwin.com/packages/:
$ cygcheck -p bin/g++
Found 3 matches for bin/g++
gcc-g++-7.3.0-1 - gcc-g++: GNU Compiler Collection (C++)
gcc-g++-7.3.0-2 - gcc-g++: GNU Compiler Collection (C++)
gcc-g++-7.3.0-3 - gcc-g++: GNU Compiler Collection (C++)
so we need the gcc-g++ package.
To install it, we run the cygwin setup, select the Full view, search the gcc-g to filter the thousands of packages and click on skip at the gcc-g++ row
after complety the installation, to verify we have it correctly installed:
$ cygcheck -c gcc-g++
Cygwin Package Information
Package Version Status
gcc-g++ 7.3.0-3 OK
Installing gcc-g++ will pull also the installation of the C compiler package gcc-core.
To compile a gmp program we need the proper header and shared library; that are usually included in a "*-devel" package:
$ cygcheck -p include/gmpxx.h
Found 9 matches for include/gmpxx.h
libgmp-devel-6.1.0-3p1 - libgmp-devel: Library for arbitrary precision arithmetic (development) (installed binaries and support files)
libgmp-devel-6.1.1-1 - libgmp-devel: Library for arbitrary precision arithmetic (development) (installed binaries and support files)
libgmp-devel-6.1.2-1 - libgmp-devel: Library for arbitrary precision arithmetic (development)
mingw64-i686-gmp-6.0.0a-2 - mingw64-i686-gmp: Multiple Precision arithmetic library for Win32 toolchain (installed binaries and support files)
...
All the packages with mingw64 are for cross compiling and we can ignore, so it is libgmp-devel. Verifying that is properly installed
$ cygcheck -c libgmp-devel
Cygwin Package Information
Package Version Status
libgmp-devel 6.1.2-1 OK
And the package content is the header files and the import libraries
$ cygcheck -l libgmp-devel
/usr/include/gmp.h
/usr/include/gmpxx.h
/usr/lib/libgmp.dll.a
/usr/lib/libgmpxx.dll.a
Now we can program one example, I am taking it from
https://gmplib.org/manual/C_002b_002b-Interface-General.html
and written in a file called mpz_add.cpp
You can use whatever editor you want. The important is that the file follows
the Unix line termination standard LF and not the Windows CR+LF (see note below if not)
$ file mpz_add.cpp
mpz_add.cpp: C++ source, ASCII text
$ cat mpz_add.cpp
#include <gmpxx.h>
#include <iostream>
using namespace std;
int main (void)
{
mpz_class a, b, c;
a = 1234;
b = "-5678";
c = a+b;
cout << "sum is " << c << "\n";
cout << "absolute value is " << abs(c) << "\n";
return 0;
}
To compile our example and test it:
$ g++ mpz_add.cpp -lgmpxx -lgmp -o mpz_add
$ ./mpz_add
sum is -4444
absolute value is 4444
We can also verify which library are linked in the program mpz_add, I added some extra comment:
$ cygcheck ./mpz_add
D:\cyg_pub\tmp\gmp\mpz_add.exe
D:\cygwin64\bin\cygwin1.dll <= cygwin library
C:\WINDOWS\system32\KERNEL32.dll <= windows system library
C:\WINDOWS\system32\ntdll.dll ...
C:\WINDOWS\system32\KERNELBASE.dll ...
D:\cygwin64\bin\cyggmp-10.dll <= GMP C library
D:\cygwin64\bin\cyggmpxx-4.dll <= GMP C++ library
D:\cygwin64\bin\cyggcc_s-seh-1.dll <= C library
D:\cygwin64\bin\cygstdc++-6.dll <= C++ library
If the file has the wrong line termination, the best tool is d2u (Dos to Unix)
$ cygcheck -p bin/d2u
Found 6 matches for bin/d2u
...
dos2unix-7.4.0-1 - dos2unix: Line Break Conversion
$ file mpz_add.cpp
mpz_add.cpp: C++ source, ASCII text, with CRLF line terminators
$ d2u mpz_add.cpp
dos2unix: converting file mpz_add.cpp to Unix format...
$ file mpz_add.cpp
mpz_add.cpp: C++ source, ASCII text
As you added also the tag makefile and autotools, the first is in the package make:
$ cygcheck -p bin/make.exe
Found 6 matches for bin/make.exe
..
make-4.2.1-2 - make: The GNU version of the 'make' utility
The second is more complex and you need the packages autoconf automake and libtool,
I would like to know how you can get a Git commit hash and/or other information into the contents of a C++ variable in the compiled binary without having it be part of the source that's tracked by Git.
I need to keep track of firmware release information in a compiled executable running on an embedded processor. Means to identify the particular release of the firmware binary such as meaningful filenames, MD5 checksums or even date/time stamps are not available in this closed environment (i.e., there is no filesystem).
One approach is to have the device's console output produce identifying text, such as 'Release 1.2.3', 'commit hash 1bc123...', or similar. The firmware release information is only of interest to maintenance personnel, so a trained operator could examine the console output. To implement this it could potentially involve manual editing of a version string, which is then compiled into the code and output to the console at program startup.
This is acceptable for major releases where a signoff workflow is used to double-check that the version information is correct. However, this being a manual process it is inherently unreliable. For example, what if the developer forgets to update the version information? - There is now a disconnect between the compiled code and its reported version string.
A workflow where the code is freshly compiled and downloaded each time the user wants to test the hardware is not practical in the situation in question, ie., it is quite onerous to update the firmware.
An automatic way of identifying the version of the code is thus required. In the situation in question, Git is used, and the developers regularly commit their work to feature branches. Displaying the Git commit hash, and perhaps also whether or not there are unstaged changes, would be a way of identifying the status of the source code used to compile the firmware.
The requirement is that I would like my application to have information available to it so that it is able to display:
"Git commit:[01abcdef...etc], branch: experimentalStuffDoNotRelease"
Thus, I would like to automatically include Git information, such as commit hash and branch, in the compiled C and/or C++ code.
The development environment has developers using both Windows and Linux, and uses Eclipse CDT with a relatively unsophisticated workflow of: check out; compile; download to the hardware.
I use a makefile, like so:
GIT_HASH=`git rev-parse HEAD`
COMPILE_TIME=`date -u +'%Y-%m-%d %H:%M:%S UTC'`
GIT_BRANCH=`git branch | grep "^\*" | sed 's/^..//'`
export VERSION_FLAGS=-DGIT_HASH="\"$(GIT_HASH)\"" -DCOMPILE_TIME="\"$(COMPILE_TIME)\"" -DGIT_BRANCH="\"$(GIT_BRANCH)\""
all:
g++ main.cpp $(VERSION_FLAGS)
When the makefile is run, the git hash and the time of compilation are both loaded into macros accessible within the source, like so:
#include <iostream>
int main(){
std::cerr<<"hash="<<GIT_HASH<<", time="<<COMPILE_TIME<<", branch="<<GIT_BRANCH<<std::endl;
}
Which gives output like:
hash=35f531bf1c959626e1b95f2d3e1a7d1e4c58e5ec, time=2017-05-18 04:17:25 UTC, branch=master
In Eclipse CDT, use a pre-build step to generate an include file containing the relevant information, and a conditional inclusion to check that the file was created:
Right-click the project
Select Properties
Expand the C/C++ Build
Select Settings In the Build Steps tab
Enter the following in the Command text box:
git log --pretty=format:'#define GIT_INFO_PRESENT%n static const char* GIT_INFO = "Version Information=[%H,%d]\r\n";' -n 1 > ../src/gitcommit.h
This will, upon build, create a file gitcommit.h that will be included in the source code. To customise it, adjust the string to your needs. (See https://git-scm.com/docs/pretty-formats)
As an example, I produce a debug output at the beginning of the main routine to inform the user of the commit and branch (not strictly needed knowing the commit, but certainly helpful):
Put this in the file, probably at the top
#if __has_include("gitcommit.h")
#include "gitcommit.h"
#else
static const char* GIT_INFO = "Git version information not present.\r\n";
#endif
To display the information somewhere in your code, do similar to this:
printf(GIT_INFO);
Note that I haven't, in this case, made the pre-build step a shell script or Windows/DOS .bat file, as I work often in Linux or Windows.
Note that this isn't tested in Windows.
In both cases, 'git' must be executable from the standard command line.
There is a dependency on provision of __has_include. This was intended to provide simplicity so that a default include file need not be provided.
Note that the gitcommit.h file's path should be discoverable by the compiler.
Usually as part of your build you run some command that generates something like that.
For example, git describe gives you something that you could use:
echo // auto generated version: > version.h
git describe > echo // auto generated version: > version.h
echo -e "#define VERSION " >> version.h
git describe >> version.h
For example x264 use this simple script to generate it:
if [ -d .git ] && command -v git >/dev/null 2>&1 ; then
localver="$(($(git rev-list HEAD | wc -l)))"
if [ "$localver" -gt 1 ] ; then
ver_diff="$(($(git rev-list origin/master..HEAD | wc -l)))"
ver="$((localver-ver_diff))"
echo "#define X264_REV $ver"
echo "#define X264_REV_DIFF $ver_diff"
if [ "$ver_diff" -ne 0 ] ; then
ver="$ver+$ver_diff"
fi
if git status | grep -q "modified:" ; then
ver="${ver}M"
fi
ver="$ver $(git rev-list -n 1 HEAD | cut -c 1-7)"
version=" r$ver"
fi
fi
echo "#define X264_VERSION \"$version\""
echo "#define X264_POINTVER \"0.$ver\""
This will generate something like:
#define X264_VERSION " r2708 86b7198"
#define X264_POINTVER "0.148.2708 86b7198"
Here's simple solution if you need git hash and local modifications flag only:
$ cat update-version-info.sh
#!/bin/sh
version=$(git describe --always --dirty --tags)
echo "#define GIT_VERSION \"$version\"" > git-version.h
$ cat 1.c
#include <stdlib.h>
#include <stdio.h>
#include "git-version.h"
int main() {
#ifdef GIT_VERSION
printf("%s\n", GIT_VERSION);
#endif
return 0;
}
$ ./1
ee4f307-dirty
Based on #Richard's answer and my fixes for qmake/Qt, here is the solution I use:
With Qt 5.14 the following lines in .pro file work for me:
GIT_HASH="\\\"$$system(git -C \""$$_PRO_FILE_PWD_"\" rev-parse --short HEAD)\\\""
GIT_BRANCH="\\\"$$system(git -C \""$$_PRO_FILE_PWD_"\" rev-parse --abbrev-ref HEAD)\\\""
BUILD_TIMESTAMP="\\\"$$system(date -u +\""%Y-%m-%dT%H:%M:%SUTC\"")\\\""
DEFINES += GIT_HASH=$$GIT_HASH GIT_BRANCH=$$GIT_BRANCH BUILD_TIMESTAMP=$$BUILD_TIMESTAMP
In your code you can check the revision like this:
int main(int argc, char *argv[])
{
QStringList args;
for (int i = 0; i < argc; i++)
args << QString(argv[i]);
if (args.contains("-v") || args.contains("--version")) {
std::cout << QString("branch: %1, version: %2, built_at: %3").arg(GIT_BRANCH).arg(GIT_HASH).arg(BUILD_TIMESTAMP).toUtf8().constData() << std::endl;
return 0;
}
// ...
}
A C file or a C++ file can be generated by some program (for example, some shell script on Linux, or some GNU awk script, or your C program running system or popen(3) running some git command).
You just configure your build automation tool (e.g. your Makefile if you use GNU make, or your build.ninja file if you use ninja) appropriately.
Both Bismon and RefPerSys are doing so and keep their git id inside the obtained executable (displaying it when invoked with --version). Note: both are projects I created.
PS a different question is how to configure your IDE to run a particular build automation tool. This is specific to your IDE. But Eclipse CDT FAQ offers an insight.
I am writing a C++ program which has to automatically generate some data to be used by students in an integrated exercise. I have already exported this data to .tex files and would also like the C++ program to be able to compile these tex files automatically.
Usually I would compile tex files from the command line by doing the following:
$ latex file.tex
$ latex file.tex
$ dvipdf file.dvi
So I tried doing the following in my C++ code (directory and filename are both strings):
//Move to the location where I created the files
string mycommand = "cd ";
mycommand += directory;
system(mycommand.c_str());
//Compile latex
mycommand = "latex " + filename + "_main.tex";
system(mycommand.c_str());
system(mycommand.c_str());
//Create .pdf
mycommand = "dvipdf " + filename + "_main.dvi";
system(mycommand.c_str());
Which then produces the following error message on the terminal output:
sh: latex: command not found
sh: latex: command not found
sh: dvipdf: command not found
I have searched this online but I have failed to find a solution for this problem, though I believe it is likely to be something very simple.
I am working on OSX and have the following version of latex installed:
pdfTeX 3.1415926-2.4-1.40.13 (TeX Live 2012)
kpathsea version 6.1.0
All help is greatly appreciated!
First, the path to the programs latex and dvipdf needs to be in your PATH environment variable.
Second, the calls of the shell via system are totally independent (in fact a new instance of the shell is started each time). So if you switch the directory in one, this does not affect the others. Switch instead the current directory of your program via:
chdir(directory.c_str())
You need for this
#include <cunistd>
using namespace std;
at the beginning of the file.
Please note that system calls with command lines depending on input parameters can be easily exploited to run arbitrary commands if the parameters (in your case the filename) are not carefully checked. As you do not have quotes, the program fails if there are e.g. whitespaces in the filename.
Currently I'm trying to start programming on my new Mac. I installed TextWrangler, and chose C++ as my language of choice; since I have some prior knowledge of it, from when I used Windows.
So, I wrote the ever so common "Hello World" program. Although, when I tried to run it, I got an error:
"This file doesn’t appear to contain a valid ‘shebang’ line (application error code: 13304)"
I tried searching the error code to find out how to fix this, but I couldn't find anything.. I have no idea what a 'shebang' line is... Can someone help me out?
You need to compile it with a compiler first. I assume you tried to run the source file like ./source but C++ doesn't work this way.
With some compilers however, you can provide a shebang-line as the first line of the source file (the #! is known as shebang or crunchbang, hence the name), like so:
#!/path/to/compiler
So that the shell knows what application is used to run that sort of file, and when you attempt to run the source file by itself, the compiler will compile and run it for you. That's a compiler-dependent feature though, so I recommend just plain compiling with G++ or whatever Macs use to get an executable, then run that.
While I wouldn't recommend it for regular C++ development, I'm using a simple shell script wrapper for small C++ utilities. Here is a Hello World example:
#if 0 // -- build and run wrapper script for C++ ------------------------------
TMP=$(mktemp -d)
c++ -o ${TMP}/a.out ${0} && ${TMP}/a.out ${#:1} ; RV=${?}
rm -rf ${TMP}
exit ${RV}
#endif // ----------------------------------------------------------------------
#include <iostream>
int main(int argc, char *argv[])
{
std::cout << "Hello world" << std::endl;
return 0;
}
It does appear that you are trying to run the source file directly, however you will need to compile using a C++ compiler, such as that included in the gcc (GNU Compiler Collection) which contains the C++ compiler g++ for the Mac. It is not included with the Mac, you have to download it first:
from http://www.tech-recipes.com/rx/726/mac-os-x-install-gcc-compiler/ : "To install the gcc compiler, download the xcode package from http://connect.apple.com/. You’ll need to register for an Apple Developer Connection account. Once you’ve registered, login and click Download Software and then Developer Tools. Find the Download link next to Xcode Tools (version) – CD Image and click it!"
Once it's installed, if you are going for a quick Hello World, then, from a terminal window in the directory of your source file, you can execute the command g++ HelloWorld.cpp -o HelloWorld. Then you should be able to run it as ./HelloWorld.
Also, if you're coming from a Visual Studio world, you might want to give Mono and MonoDevelop a try. Mono is a free implementation of C# (and other languages), and MonoDevelop is an IDE which is very similar to Visual Studio. MonoDevelop supports C# and other .NET languages, including Visual Basic .NET, as well as C/C++ development. I have not used it extensively, but it does seem to be very similar to VS, so you won't have to learn new everything all in a day. I also have used KDevelop, which I liked a lot while I was using it, although that's been a while now. It has a lot of support for GNU-style development in C/C++, and was very powerful as I recall.
Good luck with your endeavors!
Links:
Mono: http://mono-project.com/Main_Page
MonoDevelop: http://monodevelop.com/
KDevelop: http://kdevelop.org/
shebang is http://en.wikipedia.org/wiki/Shebang_%28Unix%29.
not sure why your program is not running. you will need to compile and link to make an executable.
What I find confusing (/interesting) is C++ program giving "Shebang line" error. Shebang line is a way for the Unix like operating system to specify which program should be used to interpret the rest of the file. The shebang line usually points to the path of the interpreter. C++ is a compiled language and does not have interpreter for it.
To get the real technical details of how shebang lines work, do a man execve and get that man page online here - man execve.
If you're on a mac then doing something like this on the commandline:
g++ -o program program.cpp
Will compile and link your program into an executable called program. Then you can run it like:
./program
The reason you got the 'shebang' error is probably because you tried to run the cpp file like:
./program.cpp
And the shell tries to find an interpreter to run the code in the file. Because this is C++ there is no relevant interpreter but if your file contains Python or Bash then having a line like this
#!/usr/bin/python
at the 1st line in your source file will tell the shell to use the python interpreter
The lines that start with a pattern like this: #!/.../.../.. is called a shebang line. In other words, a shebang is the character sequence consisting of the characters number sign and exclamation mark (#!).In Unix-like operating systems, when a text file with a shebang is used as if it is an executable, the program loader mechanism parses the rest of the file's initial line as an interpreter directive. The loader executes the specified interpreter program, passing to it as an argument the path that was initially used when attempting to run the script, so that the program may use the file as input data.
I am following the hints to this question, but I'm impatient and I would like to run my tests more quickly, without having to wait for the 30+ checks that R CMD check src invokes before checking tests.
what I thought I could do was to add a --standalone option to the doRUnit.R suggested in that R-wiki page so that I could run the unit tests independently of R CMD.
I added these lines to the script:
opt <- list(standalone=NULL)
if(require("getopt", quietly=TRUE)) {
## path to unit tests may be given on command line, in which case
## we also want to move the cwd to this script
opt <- getopt(matrix(c('standalone', 's', 0, "logical"),
ncol=4, byrow=TRUE))
if(!is.null(opt$standalone)) {
## switch the cwd to the dir of this script
args <- commandArgs()
script.name <- substring(args[substring(args, 1, 7)=="--file="], 8, 1000)
if(!is.null(script.name))
setwd(dirname(script.name))
}
}
with this change, the script finds the test.*\.R files, independently from the directory from which I invoke the script.
the remaining problem now is that the doRUnit.R script loads the installed library, it does not source() the files that compose the library.
assuming that I want to load each and every file in the R directory, how would I do that?
assuming you have a better testing schema (satisfying the requirements "quick", "uninstalled"), what is it?
You may have to manually loop over the files in the R directory and source() them, maybe with something like source(dir("/some/Path", pattern="*.R", full.names=TRUE).
But I have the feeling that R CMD INSTALL does a little more. You may be better off working from the installed code. And just running your unit tests directly, as you do and as the wiki page suggests, is already pretty good. So no better scheme from me. But keep us posted.
Edit: Also note that R 2.10.1 gives us new options to accelerate R CMD INSTALL:
2.10.1 NEW FEATURES
R CMD INSTALL has new options --no-R,
--no-libs, --no-data, --no-help, --no-demo, --no-exec, and --no-inst to suppress installation of the specified
part of the package. These are
intended for special purposes (e.g.
building a database of help pages
without fully installing all
packages).
That should help too.
further additions/corrections to the script.
I can now invoke it as doRUnit.R --standalone
or have it invoked by R CMD check
if(!is.null(script.name)) {
setwd(dirname(script.name))
path <- '../inst/RUnit/'
}
.
.
.
if (is.null(opt$standalone)) {
cat("\nRunning unit tests of installed library\n")
library(package=pkg, character.only=TRUE)
} else {
cat("\nRunning unit tests of uninstalled library\n")
source(dir("../R/", pattern=".*\\.R", full.names=TRUE))
}