I am trying to get a TensorFlow Lite example to run on a machine with an ARM Cortex-A72 processor. Unfortunately, I wasn't able to deploy a test model due to the lack of examples on how to use the C++ API. I will try to explain what I have achieved so far.
Create the tflite model
I have created a simple linear regression model and converted it, which should approximate the function f(x) = 2x - 1. I got this code snippet from some tutorial, but I am unable to find it anymore.
import tensorflow as tf
import numpy as np
from tensorflow import keras
from tensorflow.contrib import lite
model = keras.Sequential([keras.layers.Dense(units=1, input_shape=[1])])
model.compile(optimizer='sgd', loss='mean_squared_error')
xs = np.array([ -1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)
ys = np.array([ -3.0, -1.0, 1.0, 3.0, 5.0, 7.0], dtype=float)
model.fit(xs, ys, epochs=500)
print(model.predict([10.0]))
keras_file = 'linear.h5'
keras.models.save_model(model, keras_file)
converter = lite.TocoConverter.from_keras_model_file(keras_file)
tflite_model = converter.convert()
open('linear.tflite', 'wb').write(tflite_model)
This creates a binary called linear.tflite, which I should be able to load.
Compile TensorFlow Lite for my machine
TensorFlow Lite comes with a script for the compilation on machines with the aarch64 architecture. I followed the guide here to do this, even though I had to modify the Makefile slightly. Note that I compiled this natively on my target system. This created a static library called libtensorflow-lite.a.
Problem: Inference
I tried to follow the tutorial on the site here, and simply pasted the the code snippets from loading and running the model together, e.g.
class FlatBufferModel {
// Build a model based on a file. Return a nullptr in case of failure.
static std::unique_ptr<FlatBufferModel> BuildFromFile(
const char* filename,
ErrorReporter* error_reporter);
// Build a model based on a pre-loaded flatbuffer. The caller retains
// ownership of the buffer and should keep it alive until the returned object
// is destroyed. Return a nullptr in case of failure.
static std::unique_ptr<FlatBufferModel> BuildFromBuffer(
const char* buffer,
size_t buffer_size,
ErrorReporter* error_reporter);
};
tflite::FlatBufferModel model("./linear.tflite");
tflite::ops::builtin::BuiltinOpResolver resolver;
std::unique_ptr<tflite::Interpreter> interpreter;
tflite::InterpreterBuilder(*model, resolver)(&interpreter);
// Resize input tensors, if desired.
interpreter->AllocateTensors();
float* input = interpreter->typed_input_tensor<float>(0);
// Fill `input`.
interpreter->Invoke();
float* output = interpreter->typed_output_tensor<float>(0);
When trying to compile this via
g++ demo.cpp libtensorflow-lite.a
I get a load of errors. Log:
root#localhost:/inference# g++ demo.cpp libtensorflow-lite.a
demo.cpp:3:15: error: ‘unique_ptr’ in namespace ‘std’ does not name a template type
static std::unique_ptr<FlatBufferModel> BuildFromFile(
^~~~~~~~~~
demo.cpp:10:15: error: ‘unique_ptr’ in namespace ‘std’ does not name a template type
static std::unique_ptr<FlatBufferModel> BuildFromBuffer(
^~~~~~~~~~
demo.cpp:16:1: error: ‘tflite’ does not name a type
tflite::FlatBufferModel model("./linear.tflite");
^~~~~~
demo.cpp:18:1: error: ‘tflite’ does not name a type
tflite::ops::builtin::BuiltinOpResolver resolver;
^~~~~~
demo.cpp:19:6: error: ‘unique_ptr’ in namespace ‘std’ does not name a template type
std::unique_ptr<tflite::Interpreter> interpreter;
^~~~~~~~~~
demo.cpp:20:1: error: ‘tflite’ does not name a type
tflite::InterpreterBuilder(*model, resolver)(&interpreter);
^~~~~~
demo.cpp:23:1: error: ‘interpreter’ does not name a type
interpreter->AllocateTensors();
^~~~~~~~~~~
demo.cpp:25:16: error: ‘interpreter’ was not declared in this scope
float* input = interpreter->typed_input_tensor<float>(0);
^~~~~~~~~~~
demo.cpp:25:48: error: expected primary-expression before ‘float’
float* input = interpreter->typed_input_tensor<float>(0);
^~~~~
demo.cpp:28:1: error: ‘interpreter’ does not name a type
interpreter->Invoke();
^~~~~~~~~~~
demo.cpp:30:17: error: ‘interpreter’ was not declared in this scope
float* output = interpreter->typed_output_tensor<float>(0);
^~~~~~~~~~~
demo.cpp:30:50: error: expected primary-expression before ‘float’
float* output = interpreter->typed_output_tensor<float>(0);
I am relatively new to C++, so I may be missing something obvious here. It seems, however, that other people have trouble with the C++ API as well (look at this GitHub issue). Has anybody also stumbled across this and got it to run?
The most important aspects for me to cover would be:
1.) Where and how do I define the signature, so that the model knows what to treat as inputs and outputs?
2.) Which headers do I have to include?
Thanks!
EDIT
Thanks to #Alex Cohn, the linker was able to find the correct headers. I also realized that I probably do not need to redefine the flatbuffers class, so I ended up with this code (minor change is marked):
#include "tensorflow/lite/interpreter.h"
#include "tensorflow/lite/kernels/register.h"
#include "tensorflow/lite/model.h"
#include "tensorflow/lite/tools/gen_op_registration.h"
auto model = tflite::FlatBufferModel::BuildFromFile("linear.tflite"); //CHANGED
tflite::ops::builtin::BuiltinOpResolver resolver;
std::unique_ptr<tflite::Interpreter> interpreter;
tflite::InterpreterBuilder(*model, resolver)(&interpreter);
// Resize input tensors, if desired.
interpreter->AllocateTensors();
float* input = interpreter->typed_input_tensor<float>(0);
// Fill `input`.
interpreter->Invoke();
float* output = interpreter->typed_output_tensor<float>(0);
This reduces the number of errors greatly, but I am not sure how to resolve the rest:
root#localhost:/inference# g++ demo.cpp -I/tensorflow
demo.cpp:10:34: error: expected ‘)’ before ‘,’ token
tflite::InterpreterBuilder(*model, resolver)(&interpreter);
^
demo.cpp:10:44: error: expected initializer before ‘)’ token
tflite::InterpreterBuilder(*model, resolver)(&interpreter);
^
demo.cpp:13:1: error: ‘interpreter’ does not name a type
interpreter->AllocateTensors();
^~~~~~~~~~~
demo.cpp:18:1: error: ‘interpreter’ does not name a type
interpreter->Invoke();
^~~~~~~~~~~
How do I have to tackle these? It seems that I have to define my own resolver, but I have no clue on how to do that.
I finally got it to run. Considering my directory structure looks like this:
/(root)
/tensorflow
# whole tf repo
/demo
demo.cpp
linear.tflite
libtensorflow-lite.a
I changed demo.cpp to
#include <stdio.h>
#include "tensorflow/lite/interpreter.h"
#include "tensorflow/lite/kernels/register.h"
#include "tensorflow/lite/model.h"
#include "tensorflow/lite/tools/gen_op_registration.h"
int main(){
std::unique_ptr<tflite::FlatBufferModel> model = tflite::FlatBufferModel::BuildFromFile("linear.tflite");
if(!model){
printf("Failed to mmap model\n");
exit(0);
}
tflite::ops::builtin::BuiltinOpResolver resolver;
std::unique_ptr<tflite::Interpreter> interpreter;
tflite::InterpreterBuilder(*model.get(), resolver)(&interpreter);
// Resize input tensors, if desired.
interpreter->AllocateTensors();
float* input = interpreter->typed_input_tensor<float>(0);
// Dummy input for testing
*input = 2.0;
interpreter->Invoke();
float* output = interpreter->typed_output_tensor<float>(0);
printf("Result is: %f\n", *output);
return 0;
}
Also, I had to adapt my compile command (I had to install flatbuffers manually to make it work). What worked for me was:
g++ demo.cpp -I/tensorflow -L/demo -ltensorflow-lite -lrt -ldl -pthread -lflatbuffers -o demo
Thanks to #AlexCohn for getting me on the right track!
Here is the minimal set of includes:
#include "tensorflow/lite/interpreter.h"
#include "tensorflow/lite/kernels/register.h"
#include "tensorflow/lite/model.h"
#include "tensorflow/lite/tools/gen_op_registration.h"
These will include other headers, e.g. <memory> which defines std::unique_ptr.
I'm facing NA/NaN errors with my TMB code. Searching on the internet, I found that including fenv.h file into my code can give me a clue about the source of errors.
Since this example nan error does not work on Windows, I decided to follow an advice to run this example in Virtual TMB (Because my old desktop cannot afford to run more than one operating system).
But the new problem appeared. I get unused variable errors after running the code.
This code is copied and pasted from the link above.
cpp code
// Illustrates how to make the debugger catch a floating point error.
#include <TMB.hpp>
#include <fenv.h> // Extra line needed
template<class Type>
Type objective_function<Type>::operator() ()
{
feenableexcept(FE_INVALID | FE_OVERFLOW | FE_DIVBYZERO | FE_UNDERFLOW); // Extra line needed
DATA_SCALAR(lambda);
PARAMETER(x);
Type f;
f = sqrt(-1.); // FE_INVALID ( sqrt(-1.) returns NaN )
//f = 1./0.; // FE_DIVBYZERO ( division by zero )
//f = exp(100000.); // FE_OVERFLOW ( exp(100000.) returns Inf ) [Does not work on all platforms]
//f = exp(-100000.); // FE_UNDERFLOW ( exp(-100000.) returns 0 )
return f;
}
and here is the corresponding r code for Linux
data <- list(lambda = 25)
parameters <- list(x=1)
require(TMB)
compile('nan.cpp','-fno-gnu-unique -O0 -Wall')
dyn.load(dynlib('nan'))
model <- MakeADFun(data, parameters)
fit <- nlminb(model$par, model$fn, model$gr)
rep <- sdreport(model)
print(rep)
I attached my sessionInfo and the warning messages via image files.
Error
nan.cpp: In instantiation of 'Type objective_function<Type>::operator()() [with Type =double]':
/home/tmb/r/library/TMB/include/tmb_core.hpp:1416:16: required from here
nan.cpp:7:15:warning:unused variable 'lambda' [-Wunused -variable]
DATA_SCALAR(lambda);
^
/home/tmb/r/library/TMB/include/tmb_core.hpp:199:32: note: in definition of macro 'DATA_SCALAR'
#define DATA_SCALAR(name) Type name(asVector<Type>( \
nan.cpp:8:13: warning: unused variable 'x' [-Wunused-variable]
PARAMETER(x);
^
/home/tmb/r/library/TMB/include/tmb_core.hpp:182:30: note: in definition of macro 'PARAMETER'
How can I solve this problem?
Thanks!
Warning message:
Session Info:
I have written a library called "myprotocol" in c++ using QT creator as IDE and the normal make command to compile, I have tested the library in a a seperate project as a testbed and the library works perfectly.
Later I try to use the library in another project which uses b2 tool to compile its sources. I am able to link against the myprotocol library but I get strange output errors saying that I have problem in the original "myprotocol" library project, which is not correct.
Here are the errors:
/home/HA/myprotocol/archive.hpp:371:25: error: declaration of ‘operator&’ as non-function
/home/HA/myprotocol/archive.hpp:371:23: error: expected ‘;’ at end of member declaration
/home/HA/myprotocol/archive.hpp:371:41: error: expected ‘)’ before ‘&’ token
This is the the line which contains the error:
iarchive& operator&(CDBRequest& buf);
This is the definition of the CDBRequest:
class CDBRequest
{
public:
CDBRequest() : _cdb_request(0)
{}
CDBRequest(uint16 value)
{
_cdb_request = uint32(value) << 16;
}
CDBRequest& operator=(uint32 cdb_request)
{
_cdb_request = cdb_request;
return *this;
}
void set_value(uint16 value)
{
_cdb_request = uint32(value) << 16;
}
operator uint32&() { return _cdb_request; }
private:
uint32 _cdb_request;
};
This is the command I use to compile my project in b2:
b2 variant=release link=static toolset=gcc include=/home/HA/myprotocol/ linkflags="-L /home/HA/myprotocol/ -lmyprotocol"
Can anyone tell me what might be the problem?
I found the source of the problem:
In the library project I have a header file called "types.hpp", later when I test it in the testbed project, I only have one cpp file.
But eventually when I test it in the root project, the root project has also a header file named "types.hpp" so when I changed the name of the file everything went smoothly.
I think the problem also is because of the #ifdef for both files since they have also the same name.
I decided to use FMOD for sound playback in my project, but I'm getting lots of compiler errors which I am unsure of how to fix.
The header file of the class using FMOD looks more or less like this:
#ifndef PROJECTNAME_SOUNDMANAGER_H_
#define PROJECTNAME_SOUNDMANAGER_H_
#include <iostream>
#include <fmod.h>
#include <fmod.hpp>
#include <fmod_errors.h>
class SoundManager {
public:
static SoundManager &instance();
void play(char *data, size_t size, bool loop=false);
void stopAll();
private:
void ERRCHECK(FMOD_RESULT result);
SoundManager() : mSystem(nullptr) {
initFMOD();
}
SoundManager(const SoundManager &other);
SoundManager &operator=(const SoundManager &other);
void initFMOD();
FMOD::System *mSystem;
FMOD::Sound *mSound;
FMOD::Channel *mSoundChannel;
};
#endif // PROJECTNAME_SOUNDMANAGER_H_
And here are some of the compilation errors:
...../api/inc/fmod.h:1054:33: error: expected ')' before '*' token
...../api/inc/fmod.h:1056:33: error: expected ')' before '*' token
...../api/inc/fmod.h:1058:33: error: expected ')' before '*' token
...../api/inc/fmod.h:1059:33: error: expected ')' before '*' token
.....
...../api/inc/fmod.h:1465:5: error: 'FMOD_SOUND_PCMREADCALLBACK' does not name a type
...../api/inc/fmod.h:1466:5: error: 'FMOD_SOUND_PCMSETPOSCALLBACK' does not name a type
...../api/inc/fmod.h:1467:5: error: 'FMOD_SOUND_NONBLOCKCALLBACK' does not name a type
...../api/inc/fmod.h:1473:5: error: 'FMOD_FILE_OPENCALLBACK' does not name a type
.....
...../api/inc/fmod.h:1828:19: error: expected initializer before 'FMOD_Memory_GetStats'
...../api/inc/fmod.h:1829:19: error: expected initializer before 'FMOD_Debug_SetLevel'
...../api/inc/fmod.h:1830:19: error: expected initializer before 'FMOD_Debug_GetLevel'
...../api/inc/fmod.h:1831:19: error: expected initializer before 'FMOD_File_SetDiskBusy'
.....
...../api/inc/fmod.hpp:59:21: error: expected ';' at end of member declaration
...../api/inc/fmod.hpp:59:51: error: ISO C++ forbids declaration of 'release' with no type [-fpermissive]
...../api/inc/fmod.hpp:62:21: error: expected ';' at end of member declaration
...../api/inc/fmod.hpp:62:21: error: declaration of 'FMOD_RESULT FMOD::System::_stdcall'
...../api/inc/fmod.hpp:59:21: error: conflicts with previous declaration 'FMOD_RESULT FMOD::System::_stdcall'
...../api/inc/fmod.hpp:62:73: error: ISO C++ forbids declaration of 'setOutput' with no type [-fpermissive]
...../api/inc/fmod.hpp:63:21: error: expected ';' at end of member declaration
...../api/inc/fmod.hpp:63:21: error: declaration of 'FMOD_RESULT FMOD::System::_stdcall'
...../api/inc/fmod.hpp:59:21: error: conflicts with previous declaration 'FMOD_RESULT FMOD::System::_stdcall'
.....
If it makes any difference, I'm compiling with -std=c++0x.
I've tried searching but I wasn't able to find anything that helps me with these errors.
Please note that I'm using FMOD Ex 4.44.06.
EDIT: I seem to have found the problem. When I make a minimal example and compile it without -std=c++0x, everything compiles fine. However, if I add that flag, I get the same errors as with this project. Is there no way to make FMOD play nice with C++11?
My guess is that there's something defined as a macro or something not defined as a macro. Now, your task is to provide a minimal example. This can mean manually deleting large pieces of code or copying code from the header files. Do that until you can provide the offending code in a few lines. I guess that doing so, you will find the problem yourself.
There are a few things I noticed with the little code you provided:
fmod() is actually a function and I could imagine a few compilers providing this as a macro, which in turn conflicts with #include, but that doesn't seem to be your problem.
You include both fmod.h and fmod.hpp, which looks suspicious.
void ERRCHECK(FMOD_RESULT result); looks like a mix between function and macro.
play() should probably take a const char* data.
Under MSYS2 and GCC v5.4.0 I was facing the same problem.
The solution was add the compile flag -D__CYGWIN32__.
This is due the following in fmod.h:
#if defined(__CYGWIN32__)
#define F_CDECL __cdecl
#define F_STDCALL __stdcall
#define F_DECLSPEC __declspec
#define F_DLLEXPORT ( dllexport )
#elif defined(WIN32) || defined(_WIN32) || defined(__WIN32__) || defined(_WIN64)
#define F_CDECL _cdecl
#define F_STDCALL _stdcall
#define F_DECLSPEC __declspec
#define F_DLLEXPORT ( dllexport )
...
I'm trying to write a simple program to calculate numerical approximations with Euler's method and every complier I've used hasn't printed anything. Codeblocks is running an error but I think that is because the compiler isn't set up right. xCode will build it but nothing happens. When I run g++ Euler.cpp I get:
Euler.cpp:1: error: expected constructor, destructor, or type conversion before ‘<’ token
Euler.cpp: In function ‘int main()’:
Euler.cpp:13: error: ‘cin’ was not declared in this scope
Euler.cpp:19: error: ‘cout’ was not declared in this scope
Euler.cpp:19: error: ‘endl’ was not declared in this scope
I usually never have problems with simple c++ programs and fear it's something very obvious.
//
// Euler.cpp
// Numerical Approximations (Euler's Method)
//
// Created by XXXXXXXXXXXX on 6/18/12.
// Copyright (c) 2012 University of Kansas Department of Mathematics. All rights reserved.
//
#include <iostream>
using namespace std;
int main ()
{
int N=4;
//cout<<"Number of steps (N):";
//cin>>t;
float h=0.1;
//cout<<endl<<" Step size (h):";
cin>>h;
float y0=1;
//cout<<endl<<"y(0)=";
//cin>>y0;
cout<<"test!"<<endl;
float data[N][4];
int n=0;
data[0][2] = y0;
while (n<N){
data[n][0]=n;
if(n>0){
data[n][2]=data[n-1][3];
}
data[n][1]=h*n;
data[n][3] = data[n][2] + ((3 + data[n][1] - data[n][2])*h);
n++;
cout<<"n="<<n<<". tn="<<data[n][1]<<". y(n)="<<data[n][2]<<". y(n+1)="<<data[n][3] <<"."<<endl;
}
return 0;
}
It's probably something obvious but I don't see it.
It's not finding the iostream header. Do you see an error message reading something like "couldn't find header iostream" ?