hkdf.h not found in Crypto++ library - c++

I cannot include the hkdf.h from the crypto++ library into my code.
It says cryptopp/hkdf.h not found. Although I can import other parts from the same library into my code, such as cryptopp/sha.h
I am using the g++ with the -std=c++11 and the -lcryptopp flags
I looked into the docs and it suggested the -lcryptopp flag which I am using. This should not be an issue, am I missing something ?
Edit 1:
Compile command :
g++ -std=c++11 mycode.cpp -lcryptopp
Edit 2:
How do I check the version of my crypto++ library ?
Download command was:
sudo apt-get install libcrypto++-dev
My OS is Ubuntu 16.04

Compile command :
g++ -std=c++11 mycode.cpp -lcryptopp
This does not answer your question. It is just a heads up...
Be careful with g++ -std=c++11 mycode.cpp -lcryptopp. The library and your program have to be built using mostly the same options. I don't believe Debian uses -std=c++11, so you should not use it.
The "use mostly the same options" rule applies to all distros and all C++ libraries; and not just Crypto++ on Debian. You will encounter similar problems if you do the same with the PCRE library on Fedora.
You can check the flags Debian is using to build the library at Debian Package Auto-Building | Crypto++. The flags Debian uses are:
-DHAVE_CONFIG_H -I. -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 \
-fstack-protector-strong -Wformat -Werror=format-security \
-DCRYPTOPP_NO_UNALIGNED_DATA_ACCESS -DNDEBUG -fPIC -DPIC
For your purposes the important flags are -g -O2 -DNDEBUG -fPIC. Those flags are the ones you should use with your program. -DCRYPTOPP_NO_UNALIGNED_DATA_ACCESS is applied in the file config.h so you don't need to pass it on the command line. (Unaligned access was removed recently because it caused too many problems, so you don't even have to worry about -DCRYPTOPP_NO_UNALIGNED_DATA_ACCESS in the future).
You can use -std=c++11 if you like. However, you have to download and build the library yourself from sources to ensure the same flags are used by the library and your program.
If you build the library yourself then see GNUmakefile | Building the Library on the Crypto++ wiki.
If you build the library yourself then do yourself a favor and remove the distro provided version of the library. Something like sudo apt-get remove --purge libcrypto++ libcrypto++-dev libcrypto++-dbg. Otherwise you will inadvertently be mixing and matching them.

Cannot include the hkdf.h from the crypto++ library into my code.
It says cryptopp/hkdf.h not found.
HKDF was added at Crypto++ 5.6.3. Older versions of the library do not have it, like one supplied on Ubuntu 14 or CentOS 5.
It is a header-only implementation using the file hkdf.h. You can download it and drop it in the Crypto++ installation directory. Maybe something like the following for Crypto++ 5.6.3:
# Crypto++ 5.6.3
wget https://raw.githubusercontent.com/weidai11/cryptopp/217cb1f983c6/hkdf.h
sudo cp hkdf.h /usr/include/cryptopp/
The key derivation functions interface changed at Crypto++ 7.0. More correctly, at Crypto++ 7.0, we added a base class to use as the interface (previously there was none). The base class is KeyDerivationFunction, and it allowed us to improve self tests for the key derivation function classes. So maybe something like the following for Crypto++ 7.0:
# Crypto++ 7.0
wget https://raw.githubusercontent.com/weidai11/cryptopp/c8d8caf70074/hkdf.h
sudo cp hkdf.h /usr/include/cryptopp/
To summarize:
Crypto++ 5.6.3: you only need the file that supplies HKDF
Crypto++ 7.0: you need the files that supply both KeyDerivationFunction and HKDF.
Put another way, if you try to use Crypto++ 7.0 HKDF with Crypto++ 5.6.2, then it will never compile because KeyDerivationFunction is missing from the library.
This should not be an issue, am I missing something ?
I think your problem is probably dependent on your distro (or whomever is supplying Crypto++). We could say more if you provided details of the distribution and the library version they supply.
To hazard a guess... Debian and Fedora stay up to date with the Crypto++ releases. Or they have for the last several years. So you are probably not using Debian 8, Ubuntu 17, Fedora 22 or their respective variants.
However, you may be using Debian 7 or earlier, Ubuntu 12 or earlier, or Fedora 21 or earlier. In this case I believe you are using Crypto++ 5.6.2.
I believe Gentoo, the BSDs and some others are behind on the release curve. As far as I know, some distros are still providing Crypto++ 5.6.2.

Related

Rcpp Library Won't Build (Can't find Compiler) on Ubuntu

I have a package that depends on Rcpp and uses two other libraries compiled from sub-directories in src/. The package builds fine on Mac OSX using a clang compiler. However, on an RStudio Ubuntu server, it fails to build. The build's first two steps (creating the static libraries in the sub directories to link in) work fine and I can see sensible build commands like the following taking place:
g++ -Wall -I../../inst/include/ --std=c++11 -lhts -L../htslib/ -lz -lm -c -o someLibFile.o someLibFile.cpp
However, in the very last step of the build process where it tries to build the Rcpp code and bind to the library, for some reason it appears to compleletey fail to put the compiler command in front (g++) and only outputs the second half of the command.
-o mypackage.so RcppExports.o temp.o -lhts -lpbbam -Lpbbam/ -L/htslib/ -Lpbbam/ -L/mnt/software/r/R/3.1.1/usr/lib/R/lib -lR
In contrast, on the Mac it builds just fine, appending clang++ and other flags in front of this final command:
clang++ -std=c++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o pbbamr.so LoadData.o RcppExports.o temp.o -lhts -lpbbam -Lpbbam/ -Lhtslib/ -Lpbbam/ -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
How do I make it use the g++ compiler on Ubuntu at this step? I have a custom Makevars file, but it is there just to build the dependencies in the sub-directory, so I don't know why that would cause any problems (since it works on Mac OSX).
More Information
The compiler seems to be found if I delete my Makevars file. However, the Makevars file I am using is essentially a direct copy of the example given in the R extensions guide with one addition to enable C++11:
CXX_STD = CXX11
.PHONY: all mylibs
all: $(SHLIB)
$(SHLIB): mylibs
mylibs:
(cd subdir; make)
With the line CXX_STD removed, it does stick a compiler in front of the command.
Briefly:
What is your R installation? You should probably run the binaries provided by Michael via CRAN; they are based on my Debian upload; I run these too on a bunch of machines
The reason is that R 'remembers' its compile-time settings via $RHOME/etc/Makefconf. This should just be CXX=g+=.
When you install r-base-dev (from Ubuntu or the newer version from CRAN) you also get the build-essential package as well as all common dependencies. With that things just work.
If however you are doing something special or local, well then you have to deal with your local changes. The basic Ubuntu setup is used by thousands of people and daily jobs--including eg Travis builds for countless GitHub repos.
This is caused by using an outdated/unusual R installation which has poor support for C++11. The best way to resolve his is to upgrade to a more recent version of R, or use a standard R install (sudo apt-get install r-base-dev). A poor work around is described below.
Problems Cause and Bad Work Around
When writing R extension that use C++11, one often sets CXX_STD = CXX11 in the Makevars file or list SystemRequirements: C++11 in the DESCRIPTION file. These will trigger R to use the compiler set by the following flags in the Makeconf file (located at file.path(R.home(), "etc/Makeconf")).
CXX1X
CXX1XFLAGS
CXX1XPICFLAGS
CXX1XSTD
Note that some of these may be set in this file, but not all of them might be there indicating a problem. In the event there is a problem with these settings or they are not set, R appears to use the empty string "" as the compiler/linker for the C++ code, leading to the problem shown above where no compiler argument is given.
If upgrading is not an option and you need to deploy on a known machine, one work around is to manually setup for C++11 by making a more idiosyncratic Makevars file. For example, you could:
Remove the CXX_STD=CXX11 line from the Makevars file.
Remove SystemRequirements: C++11 from the DESCRIPTION file.
Add --std=c++11 and any other requirements needed to PKG_CPPFLAGS, PKG_CFLAGS, PKG_CXXFLAGS or whatever variable is being used to compile your code, to manually set the needed flags (assuming the machine's compiler actually does support C++11).
The above solution is not particularly robust, but can be used as a work around in case the machine cannot be upgraded.
Thanks to #DirkEddelbuettel for not only writing Rcpp but being willing to support it on StackOverflow and help with issues like this.

Linking error when compiling Crypto++ for ARMHF

I'm trying to compile the crypto++ library to run for the armhf architecture. I'm following the method provided in this answer. I tweaked the setenv-embed.sh to match my system's configuration. The output of running . ./setenv-embed.sh is
CPP: /usr/bin/arm-linux-gnueabihf-cpp
CXX: /usr/bin/arm-linux-gnueabihf-g++
AR: /usr/bin/arm-linux-gnueabihf-ar
LD: /usr/bin/arm-linux-gnueabihf-ld
RANLIB: /usr/bin/arm-linux-gnueabihf-gcc-ranlib-4.8
ARM_EMBEDDED_TOOLCHAIN: /usr/bin
ARM_EMBEDDED_CXX_HEADERS: /usr/arm-linux-gnueabihf/include/c++/4.8.2
ARM_EMBEDDED_FLAGS: -march=armv7-a mfloat-abi=hard -mfpu=neon -I/usr/arm-linux-gnueabihf/include/c++/4.8.2 -I/usr/arm-linux-gnueabihf/include/c++/4.8.2/arm-linux-gnueabihf
ARM_EMBEDDED_SYSROOT: /usr/arm-linux-gnueabihf
which indicates that the correct compilers have been found. However, when I build the library using make I run into the following error
/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/../../../../arm-linux-gnueabihf/bin/‌​ld: cannot find /usr/arm-linux-gnueabihf/lib/libc.so.6 inside /usr/arm-linux-gnueabihf
/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/../../../../arm-linux-gnueabihf/bin/‌​ld: cannot find /usr/arm-linux-gnueabihf/lib/libc_nonshared.a inside /usr/arm-linux-gnueabihf
/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/../../../../arm-linux-gnueabihf/bin/‌​ld: cannot find /usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3 inside /usr/arm-linux-gnueabihf
But when I open the location /usr/arm-linux-gnueabihf/lib I can find all the three error files mentioned above ie libc.so.6, libc_nonshared.a and ld-linux-armhf.so.3
I'm trying to compile the library for Beaglebone, if that helps.
Update 1:
The results of running make -f GNUmakefile-cross system after doing a fresh git pull
hassan#hassan-Inspiron-7537:~/cryptopp-armhf$ make -f GNUmakefile-cross system
CXX: /usr/bin/arm-linux-gnueabihf-g++
CXXFLAGS: -DNDEBUG -g2 -Os -Wall -Wextra -DCRYPTOPP_DISABLE_ASM -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb -I/usr/arm-linux-gnueabihf/include/c++/4.8.2 -I/usr/arm-linux-gnueabihf/include/c++/4.8.2/arm-linux-gnueabihf --sysroot=/usr/arm-linux-gnueabihf -Wno-type-limits -Wno-unknown-pragmas
LDLIBS:
GCC_COMPILER: 1
CLANG_COMPILER: 0
INTEL_COMPILER: 0
UNALIGNED_ACCESS:
UNAME: Linux hassan-Inspiron-7537 3.13.0-35-generic #62-Ubuntu SMP Fri Aug 15 01:58:42 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
MACHINE:
SYSTEM:
RELEASE:
make: Nothing to be done for `system'.
The problem is simple. It is in the --sysroot option. The value of this option is /usr/arm-linux-gnueabihf/ and it is used by the linker and the resulting library folder becomes
/usr/arm-linux-gnueabihf/usr/arm-linux-gnueabihf/lib/
I removed the --sysroot option from line 68 in the file GNUmakefile-cross and everything compiled and linked OK.
However, I couldn't run the example on my BeagleBone Black because of mismatch of some shared libraries versions. But this wasn't a real problem for me, because in my application I link crypto++ statically, not dynamically.
Based on Crosswalking's research I think I can explain what is going on. I don't think I agree with the assessment "The problem is simple. It is in the --sysroot option" since the Crypto++ environment script and makefile are doing things as expected.
I think Crosswalking's answer could be how to work around it; but see open questions below. The following is from Crypto++ Issue 134: setenv-embedded.sh and GNUmakefile-cross:
I think this another distro problem, similar to g++-arm-linux-gnueabi
cannot compile a C++ program with
--sysroot.
It might be a Ubuntu problem or a Debian problem if it is coming from
upstream.
When cross-compiling, we expect the following (using ARMHF):
SYSROOT is /usr/arm-linux-gnueabihf
INCLUDEDIR is /usr/arm-linux-gnueabihf/include
LIBDIR is /usr/arm-linux-gnueabihf/lib
BINDIR is /usr/arm-linux-gnueabihf/bin
How LIBDIR morphed into into
/usr/arm-linux-gnueabihf/usr/arm-linux-gnueabihf/lib/ (i.e.,
$SYSROOT/$SYSROOT/lib) is a mystery. But in all fairness, building
GCC is not a trivial task.
You should probably file a bug report with Debian or Ubuntu (or
whomever provides the toolchain).
The open question for me is, since $SYSROOT/lib is messed up, then is $SYSROOT/include messed up, too?
If the include directory is also messed up, then the cross compile is using the host's include files, and not the target include files. That will create hard to diagnose problems later.
If both $SYSROOT/include and $SYSROOT/lib are messed up, then its not enough to simply remove --sysroot. Effectively, this is what has to be done:
# Exported by setenv-embedded
export=ARM_EMBEDDED_SYSROOT=/usr/arm-linux-gnueabihf
# Used by the makefile
-I $ARM_EMBEDDED_SYSROOT/$ARM_EMBEDDED_SYSROOT/include
-L $ARM_EMBEDDED_SYSROOT/$ARM_EMBEDDED_SYSROOT/lib
Which means we should be able to do the following:
# Exported by setenv-embedded
export=ARM_EMBEDDED_SYSROOT=/usr/arm-linux-gnueabihf/usr/arm-linux-gnueabihf
# Used by the makefile
--sysroot="$ARM_EMBEDDED_SYSROOT"
Finally, this looks a lot like Ubuntu's Bug 1375071: g++-arm-linux-gnueabi cannot compile a C++ program with --sysroot. The bug report specifically calls out ... the built-in paths use an extra "/usr/arm-linux-gnueabi".
We need the paths:
A) /usr/arm-linux-gnueabi/include/c++/4.7.3 B)
/usr/arm-linux-gnueabi/include/c++/4.7.3/arm-linux-gnueabi
But the built-in paths tries to use:
C) /usr/arm-linux-gnueabi/usr/arm-linux-gnueabi/include/c++/4.7.3
D)
/usr/arm-linux-gnueabi/usr/arm-linux-gnueabi/include/c++/4.7.3/arm-linux-gnueabi/sf
E)
/usr/arm-linux-gnueabi/usr/arm-linux-gnueabi/include/c++/4.7.3/backward
Notice the built-in paths use an extra "/usr/arm-linux-gnueabi"

Debug symbol bug with MacPorts installed GCC 4.9?

I have recently installed GCC 4.9.2 (port name gcc49) through MacPorts. I am quite happy with its new features such as colorized diagnostics and C++1y/C++14 support improvements, etc.
However, since I started to compile code using GCC 4.9.2, I realized that it is not generating debug symbol directory *.dSYM and gdb says "no debugging symbols found" when I try to debug a program I compiled with -g flag.
Is this a MacPorts specific bug or is there a problem with GCC 4.9?
Thanks
It is not a MacPorts specific issue. MacPorts doesn't really do much to customize the gcc ports.
If you want to create a dSYM bundle and strip your executable, you should just do something like:
gcc-mp-4.9 -g3 -c example.c
gcc-mp-4.9 example.o -o example
dsymutil --out example.dSYM example
strip -S -x example
As a side note, if you want C++11/C++14 support, I suggest you use the clang-3.5 port as that will allow you to use libc++ from the system instead of libstdc++ from MacPorts (and allow you to use system and MacPorts C++ libraries rather than just the STL). Also, lldb is really the preferred debugger for OS X these days.

how to force compilation of Boost to use -fPIC

The team on which I work produces a shared library for use in Python. This library is entirely C++ and we use Boost to expose to python. Because we cannot guarantee that our clients have the Boost libraries installed, we pull in the functionality needed from Boost to the shared object file statically. The final stage in compilation will look familiar to many
g++ -o <output> <objects> -Wl,-Bstatic -lboost_python -lboost_regex ... -Wl,-Bdynamic -shared <other_opts>
We've traditionally used our own build of Boost: 1.47. This version is now quite old and so we wish to update. However, oddly, when I install the necessary objects using yum on my CentOS 7 system, I get the following error from gcc:
relocation R_X86_64_32 against '.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
Well, I thought I'd simply download the latest boost (CentOS 7 installs Boost 1.53) and do my own build. This, after all, has always worked for us. I follow the instructions here but I got the same error. How do I force the use of -fPIC for even the static libraries that it builds?
I believe boost automatically uses -fPIC when compiling a shared library (.so file), but the below command uses -fPIC when compiling a static library (.a file) too.
This worked for me on boost 1.46.1:
sudo ./bjam cxxflags=-fPIC cflags=-fPIC -a ... install
The ... is where you add additional flags like threading=multi or --layout=tagged, and optionally the list of projects to build (for example: --with-regex).
Note: using both cflags and cxxflags is unnecessary, only one is needed. See comments below.
Reference links:
https://cmake.org/Wiki/TubeTK/Build_Instructions#Boost_.28optional.29
http://lists.boost.org/boost-users/2010/07/60682.php
Just for convenience, I combined previous answer and comments to it:
sudo ./bjam cxxflags=-fPIC -a --with-system install
--with-system is not necessary, but it's a place where you can add other boost compile options
It works for me at CentOS 7 with boost 1.67
Another solution:
./bootstrap.sh
./b2 cxxflags=-fPIC cflags=-fPIC

How to use OpenSSL with GCC?

I'm trying to use openssl in a gcc program but it isn't working.
g++ server.cpp /usr/lib/libssl.a -o server
gives an error message, as does anything with the -l option. What must I type on the command line to link with openssl? The file /usr/lib/libssl.a exists, but nevertheless I still get the linker error no such function MD5() exists.
Without knowing the exact errors you are seeing, it is difficult to provide an exact solution. Here is my best attempt.
From the information you provided, it sounds as though the linker is failing because it cannot find a reference to the md5 function in libssl.a. I believe this function is actually in libcrypto so you may need to specify this library as well.
g++ server.cpp -L/usr/lib -lssl -lcrypto -o server
You or others may find this article developerWorks article helpful.
Secure programming with the OpenSSL API
https://developer.ibm.com/technologies/linux/tutorials/l-openssl
It describes most things you need to know to get off the ground with OpenSSL and C/C++. If you find you are following most of the same steps, it might help you see what needs doing.
Good luck.
update
revised link: https://developer.ibm.com/technologies/linux/tutorials/l-openssl
Which has been shuffled around
original link: http://www.ibm.com/developerworks/linux/library/l-openssl.html
Which now goes to a digest page including the article.
Note: keeping both links because they be used to find new discoveries.
In Eclipse IDE select Your Project property --> c/c++ Build --> Settings gcc c linker(from tools settings)--> add to Library Search Path (-L)
/usr/lib -lssl -lcrypto
The location of the library is not fixed. In my case (Ubuntu 18.04), the .a files are located in /usr/lib/x86_64-linux-gnu/. So here are the complete steps:
1) install the library,
sudo apt install libss-dev
2) check the installed files,
dpkg-query -L libssl-dev
3) change the gcc flags -L(library directory) -l(library name), e.g.,
gcc XXX.c XXXXX.c -L/usr/lib/x86_64-linux-gnu/ -lcrypto -lssl
On top of the accepted answers, I could not make compile the OpenSSL example for AES-CCM:
https://github.com/openssl/openssl/blob/master/demos/evp/aesccm.c
To make it work I needed to add two more things:
The Dinamic Linking Library : -ldl
The PThread library to use POSIX threading support: -pthread (Adding directly the library with -lpthread is not recommended )