How to load checkpoint and inference with C++ for tensorflow? - c++

I have Tensorflow with python api and got these checkpoint model file:
model.ckpt-17763.data-00000-of-00001
model.ckpt-17763.index
model.ckpt-17763.meta
But I want a C/C++ shared library (.so file) when integrating into production. So I need to load these model file and inference with C++ code and compile to a shared library. Is there some tutorial or sample for doing this?

You can write c++ code to load and use your graph with the instructions given here.
You can use the files here to make a Cmake project with tensorflow outside the TF repository, and compile your library.
However, you'll still need the .ckpt files next to your .so, I don't know how to intergate them inside it.
There are a lot of questions about that on S.O., and a few tutorials (see the two cited in this answer) but since tensorflow is evolving fast, they quickly become outdated, and it's always a bit of struggle to get it to work properly (totally feasible, and getting easier, though).

Related

How to create a portable executable with allegro 4.4.2

So I am creating a C++ program using the allegro version 4.4.2 library and I need to be able to produce an executable package which I can submit to a course instructor and have run on his computer without issues. I am developing the program using DevC++. If I am understanding this correctly so far, I need to link to a static library for allegro and then the executable should include the library info and thus the library itself is not necessary to have included in the executable package. I have also seen some recommendations regarding third party tools that produce executable packages, but these seem to be more general (not necessarily allegro compatible?) and I am hoping to avoid downloading more third party software.
I have tried linking to the static library liballegro-4.4.2-monolith-static-mt.a but for some reason when I do this I get a whole slew of undefined reference compile time errors. I have read that it is necessary to have #define ALLEGRO_STATICLINK included in the project, I have this statement at the top of my main source file but I am not sure if I am meant to define this somewhere in the project options instead?
Any guidance or link to resources which will help me solve my problem would be extremely appreciated! I am just getting into the meat of programming with C++ beginning to learn object oriented programming methods, I still occasionally struggle with the basics so thank you in advance for your help!
Backstory (if interested/relevant?): I am taking a Game AI programming course for which we need to create some example programs to demonstrate AI algorithms. The course specifies Java but I am most familiar with C++ and the course instructor says this is fine but I must be able to submit an executable to him which he can just run on his computer without issues. To best achieve what the course asks I feel the allegro library (which I already have installed) will be of great help, mostly for drawing graphics to the screen and such.
So I just ended up using a dynamically linked version of my program and including the appropriate .dll files which turned out to be a lot less work than I thought it would be.
Basically I just had to ensure the allegro-4.4.2-md.dll from the bin folder was included in the folder with the executable. I then sent off the package (the .exe compiled file and the needed supporting files such as bitmaps sound files etc) to a few friends and had them try to run it. They then gave me the warnings from their computers along the lines of "Test.exe cannot be run ****.dll cannot be found".
I searched the exact names of the dll files they were receiving warnings for in the folder with my game and voila it now seems to execute on whatever PC it is sent to.

Using tensorflow in C++ on Windows

I know there are ways of using Tensorflow in C++ they even have a documentation for it but I can seem to be able to get the library for it. I've checked the build from source instructions but it seems to builds a pip package rather than a library I can link to my project. I also found a tutorial but when I tried it out I ran out of memory and my computer crashed. My question is, how can I actually get the C++ library to work on my project? I do have these requirements, I have to work on windows with Visual Studio in C++. What I would love to is if I could get a pre-compiled DLL that I could just link but I haven't found such a thing and I'm open to other alternatives.
I can't comment so I am writing this as an answer.
If you don't mind using Keras, you could use the package frugally deep. I haven't seen a library myself either, but I came across frugally deep and it seemed easy to implement. I am currently trying to use it, so I cannot guarantee it will work.
You could check out neural2D from here:
https://github.com/davidrmiller/neural2d
It is a neural network implementation without any dependent libraries (all written from scratch).
I would say that the best option is to use cppflow, an easy wrapper that I created to use Tensorflow from C++ easily.
You won't need to install anything, just download the TF C API, and place it somewhere in your computer. You can take a look to the docs to see how to do it, and how to use the library.
The answer seems to be that it is hard :-(
Try this to start. You can follow the latest instructions for building from source on Windows up to the point of building the pip package. But don't do that - do this/these instead:
bazel -config=opt //tensorflow:tensorflow.dll
bazel -config=opt //tensorflow:tensorflow.lib
bazel -config=opt tensorflow:install_headers
That much seems to work fine. The problems really start when you try to use Any of the header files - you will probably get compilation errors, at least with TF version >= 2.0. I have tried:
Build the label_image example (instructions in the readme.md file)
It builds and runs fine on Windows, meaning all the headers and source are there somewhere
Try incorporating that source into Windows console executable: runs into compiler errors due to conflicts with std::min & std::max, probably due to Windows SDK.
Include c_api.h in a Windows console application: won't compile.
Include TF-Lite header files: won't compile.
There is little point investing the lengthy compile time in the first two bazel commands if you can't get the headers to compile :-(
You may have time to invest in resolving these errors; I don't. At this stage Tensorflow lacks sufficient support for Windows C++ to rely on it, particularly in a commercial setting. I suggest exploring these options instead:
If TF-Lite is an option, watch this
Windows ML/Direct ML (requires conversion of TF models to ONNX format)
CPPFlow
Frugally Deep
Keras2CPP
UPDATE: having explored the list above, I eventually found the following worked best in my context (real-time continuous item recognition):
convert models to ONNX format (use tf2onnx or keras2onnx
use Microsoft's ONNX runtime
Even though Microsoft recommends using DirectML where milliseconds matter, the performance of ONNX runtime using DirectML as an execution provider means we can run a 224x224 RGB image through our Intel GPU in around 20ms, which is quick enough for us. But it was still hard finding our way to this answer

Making predictions with TensorFlow trained model with existing C++ program (not Bazel)

I have a big C++ program built with Automake and it would be a very big hassle (practically nearly impossible given my time constraints) to convert it to use the Bazel build system. Is there any way I can use a TensorFlow trained model (deep convolutional net) within my program to make predictions (I don't need to do learning within the program right now, but it would be really cool if that can also be done)? Like using TensorFlow as a library?
Thanks!
TensorFlow has a pretty narrow C API exported in the c_api.h header file. The library file can be produced by building the libtensorflow.so target. The C API has everything you require to load a pre-trained model and run inference on it to make predictions. You don't need to convert your build system to Bazel. All you need is to use Bazel to build //tensorflow:libtensorflow.so target, and copy the libtensorflow.so and c_api.h to where you see fit.

Image Library (IMGLIB) on Gumstix?

I need to run AprilTags C++ algorithm on gumstix using DSP image library. Without DSP, Algorithm is eating up 85% cpu with 4fps. I am trying to get TI Image Library IMGLIB on Gumstix with Yocto Project. I could not find any resources or hints how to do it.
It would be great if I can get some inputs on how to proceed with getting imglib on gumstix using yocto project/open-embedded
Should I write a recipe with bitbake or is there any other way to get IMGLIB on Gumstix?
Try to bitbake the TI Image Library IMGLIB, and you should obtain
a library and modules DSPlink, CMem, etc which should be used to
communicate with the Gumstix DSP. In your application you link
to the library (e.g. specify its path in the makefile) and add
the include header (.h) in your application source code. That
is the theory anyway. I have bitbaked it and trying to use IMGLIB
now. Can report some progress. What OS/kernel you are using ?
Brahim

Activate Run the libLAS C++ library within R

I would like to use the libLAS C/C++ library functions within R to import, analyse, export terrestrial lidar data. libLAS is a C/C++ library for reading and writing the very common LAS LiDAR format ( http://liblas.org/index.html ).
Would it be possible to use the Rcpp package to run this library (or other packages)? http://dirk.eddelbuettel.com/code/rcpp.html
Or should I compile and install it in order to use it following the compilation instructions http://liblas.org/compilation.html ? I am working on a MacOSx 10.6.5. As such I could also use it within Open Source GIS GRASS as described in the following wiki http://grass.osgeo.org/wiki/LIDAR#Micro-tutorial_for_LAS_data_import .
All advice is welcome related to reading and processing LIDAR data with R/GRASS.
Thanks,
Jan
For the question
Would it be possible to use the Rcpp
package to run this library (or other
packages)?
the answer is whopping Yup! as using it for glueing R to a given C/C++ library was pretty much the reason Rcpp was written for. Come and see the documentation and/or the rcpp-devel list for examples. There is some exciting new stuff happening with Rcpp modules but you can also get going the old-fashioned way of writing your wrapper. Rcpp makes mapping and R and C++ types (in both directions) a lot easier.