Occasional SEGFAULT in my c++ code with flatbuffers - c++

I'm working on my C++ project with flatbuffers. I started with google's online example and wrote a google test. However, this test sometimes failed with SEGFAULT.
Following are the code snippets.
// moster.fbs
namespace MyGame.Sample;
enum Color:byte { Red = 0, Green, Blue = 2 }
union Equipment { Weapon } // Optionally add more tables.
struct Vec3 {
x:float;
y:float;
z:float;
}
table Monster {
pos:Vec3; // Struct.
mana:short = 150;
hp:short = 100;
name:string;
friendly:bool = false (deprecated);
inventory:[ubyte]; // Vector of scalars.
color:Color = Blue; // Enum.
weapons:[Weapon]; // Vector of tables.
equipped:Equipment; // Union.
path:[Vec3]; // Vector of structs.
}
table Weapon {
name:string;
damage:short;
}
root_type Monster;
// test.cpp
#include <fstream>
#include <string>
#include "gtest/gtest.h"
#include "monster_generated.h"
using namespace MyGame::Sample;
void WriteMonsterToFile(const std::string& filename)
{
// Build up a serialized buffer algorithmically:
flatbuffers::FlatBufferBuilder builder;
// First, lets serialize some weapons for the Monster: A 'sword' and an 'axe'.
auto weapon_one_name = builder.CreateString("Sword");
short weapon_one_damage = 3;
auto weapon_two_name = builder.CreateString("Axe");
short weapon_two_damage = 5;
// Use the `CreateWeapon` shortcut to create Weapons with all fields set.
auto sword = CreateWeapon(builder, weapon_one_name, weapon_one_damage);
auto axe = CreateWeapon(builder, weapon_two_name, weapon_two_damage);
// Create a FlatBuffer's `vector` from the `std::vector`.
std::vector<flatbuffers::Offset<Weapon>> weapons_vector;
weapons_vector.push_back(sword);
weapons_vector.push_back(axe);
auto weapons = builder.CreateVector(weapons_vector);
// Second, serialize the rest of the objects needed by the Monster.
auto position = Vec3(1.0f, 2.0f, 3.0f);
auto name = builder.CreateString("MyMonster");
unsigned char inv_data[] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9};
auto inventory = builder.CreateVector(inv_data, 10);
// Shortcut for creating monster with all fields set:
auto orc = CreateMonster(builder, &position, 150, 80, name, inventory,
Color_Red, weapons, Equipment_Weapon, axe.Union());
builder.Finish(orc); // Serialize the root of the object.
// We now have a FlatBuffer we can store on disk or send over a network.
std::ofstream outfile(filename, std::ios::binary);
outfile.write((char*)builder.GetBufferPointer(), builder.GetSize());
outfile.flush();
outfile.close();
}
TEST(FlatbuffersTest, Monster)
{
WriteMonsterToFile("monster.bin");
// ** file/network code goes here :) **
std::ifstream infile;
infile.open("monster.bin", std::ios::binary | std::ios::in);
infile.seekg(0, std::ios::end);
int length = infile.tellg();
infile.seekg(0, std::ios::beg);
char* data = new char[length];
infile.read(data, length);
infile.close();
// Get access to the root:
auto monster = GetMonster(data);
delete[] data;
// Get and test some scalar types from the FlatBuffer.
EXPECT_EQ(monster->hp(), 80);
EXPECT_EQ(monster->mana(), 150);
EXPECT_EQ(monster->name()->str(), "MyMonster");
// Get and test a field of the FlatBuffer's `struct`.
auto pos = monster->pos();
EXPECT_EQ(pos->z(), 3.0f);
// Get a test an element from the `inventory` FlatBuffer's `vector`.
auto inv = monster->inventory();
EXPECT_EQ(inv->Get(9), 9);
// Get and test the `weapons` FlatBuffers's `vector`.
std::string expected_weapon_names[] = {"Sword", "Axe"};
short expected_weapon_damages[] = {3, 5};
auto weps = monster->weapons();
for (unsigned int i = 0; i < weps->size(); i++) {
EXPECT_EQ(weps->Get(i)->name()->str(), expected_weapon_names[i]);
EXPECT_EQ(weps->Get(i)->damage(), expected_weapon_damages[i]);
}
// Get and test the `Equipment` union (`equipped` field).
EXPECT_EQ(monster->equipped_type(), Equipment_Weapon);
auto equipped = static_cast<const Weapon*>(monster->equipped());
EXPECT_EQ(equipped->name()->str(), "Axe");
EXPECT_EQ(equipped->damage(), 5);
}
test output:
$ ctest --rerun-failed --output-on-failure
Test project /root/cpcos/build_linux64
Start 3: FlatbuffersTest.Monster
1/1 Test #3: FlatbuffersTest.Monster ..........***Exception: SegFault 0.01 sec
Running main() from /home/conan/w/BuildSingleReference/.conan/data/gtest/1.11.0/_/_/build/7320405f83ec32d8556b524cdda87ee295bb7b84/source_subfolder/googletest/src/gtest_main.cc
Note: Google Test filter = FlatbuffersTest.Monster
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from FlatbuffersTest
[ RUN ] FlatbuffersTest.Monster
/root/cpcos/tests/flatbuffers_gtest.cpp:71: Failure
Expected equality of these values:
monster->hp()
Which is: 100
80
0% tests passed, 1 tests failed out of 1
Total Test time (real) = 0.04 sec
The following tests FAILED:
3 - FlatbuffersTest.Monster (SEGFAULT)
Errors while running CTest
I compiled the code and ran it many times, strangely, it sometimes PASS while sometimes failed with SEGFAULT.
What did I do wrong?
Thanks.

Related

Creating an Array C++

I have two int values:
v_y
v_x
That I would like to convert to a size 2 char array then write to my serial port.
I currently have this code that isn't working:
void Array2 (char charArray[], int sizeOfArray);
........
}
{
char one[] = { 'v_x', 'v_y' };
Array2(one, 2);
Serial::WriteData(one, 2);
}
}
I currently get two errors:
a nonstatic member reference must be relative to a specific object
and
'Serial::WriteData':illegal call of non-static member function.
Any help, hint or idea on what I'm doing wrong would be great!
Edit: I'm using this code to communicate with my serialport CODE
// data.h or similar, you need this for both the arduino and desktop machine
struct Data {
double v_x;
double v_y;
}
On your arduino: To write
Data data = {
.v_x = 1.0, // Example put your values here
.v_y = 2.0,
};
// This should write from your Arduino to the computer
Serial::WriteData((const char*)data, sizeof(data));
On your computer
Data data;
// This reads from the serial port, and put the data in the struct
auto len = SP->ReadData((const char*)data, sizeof(data));
// check that data is the right size
// use the data from data.v_x and data.v_y
auto v_x = data.v_x;
auto v_y = data.v_y; // etc

Pass Byte Array as std::vector<char> from Node.js to C++ Addon

I have some constraints where the addon is built with nan.h and v8 (not the new node-addon-api).
The end function is a part of a library. It accepts std::vector<char> that represents the bytes of an image.
I tried creating an image buffer from Node.js:
const img = fs.readFileSync('./myImage.png');
myAddonFunction(Buffer.from(img));
I am not really sure how to continue from here. I tried creating a new vector with a buffer, like so:
std::vector<char> buffer(data);
But it seems like I need to give it a size, which I am unsure how to get. Regardless, even when I use the initial buffer size (from Node.js), the image fails to go through.
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
[1] 16021 abort (core dumped)
However, when I read the image directly from C++, it all works fine:
std::ifstream ifs ("./myImage.png", std::ios::binary|std::ios::ate);
std::ifstream::pos_type pos = ifs.tellg();
std::vector<char> buffer(pos);
ifs.seekg(0, std::ios::beg);
ifs.read(&buffer[0], pos);
// further below, I pass "buffer" to the function and it works just fine.
But of course, I need the image to come from Node.js. Maybe Buffer is not what I am looking for?
Here is an example based on N-API; I would also encourage you to take a look similar implementation based on node-addon-api (it is an easy to use C++ wrapper on top of N-API)
https://github.com/nodejs/node-addon-examples/tree/master/array_buffer_to_native/node-addon-api
#include <assert.h>
#include "addon_api.h"
#include "stdio.h"
napi_value CArrayBuffSum(napi_env env, napi_callback_info info)
{
napi_status status;
const size_t MaxArgExpected = 1;
napi_value args[MaxArgExpected];
size_t argc = sizeof(args) / sizeof(napi_value);
status = napi_get_cb_info(env, info, &argc, args, nullptr, nullptr);
assert(status == napi_ok);
if (argc < 1)
napi_throw_error(env, "EINVAL", "Too few arguments");
napi_value buff = args[0];
napi_valuetype valuetype;
status = napi_typeof(env, buff, &valuetype);
assert(status == napi_ok);
if (valuetype == napi_object)
{
bool isArrayBuff = 0;
status = napi_is_arraybuffer(env, buff, &isArrayBuff);
assert(status == napi_ok);
if (isArrayBuff != true)
napi_throw_error(env, "EINVAL", "Expected an ArrayBuffer");
}
int32_t *buff_data = NULL;
size_t byte_length = 0;
int32_t sum = 0;
napi_get_arraybuffer_info(env, buff, (void **)&buff_data, &byte_length);
assert(status == napi_ok);
printf("\nC: Int32Array size = %d, (ie: bytes=%d)",
(int)(byte_length / sizeof(int32_t)), (int)byte_length);
for (int i = 0; i < byte_length / sizeof(int32_t); ++i)
{
sum += *(buff_data + i);
printf("\nC: Int32ArrayBuff[%d] = %d", i, *(buff_data + i));
}
napi_value rcValue;
napi_create_int32(env, sum, &rcValue);
return (rcValue);
}
The JavaScript code to call the addon
'use strict'
const myaddon = require('bindings')('mync1');
function test1() {
const array = new Int32Array(10);
for (let i = 0; i < 10; ++i)
array[i] = i * 5;
const sum = myaddon.ArrayBuffSum(array.buffer);
console.log();
console.log(`js: Sum of the array = ${sum}`);
}
test1();
The Output of the code execution:
C: Int32Array size = 10, (ie: bytes=40)
C: Int32ArrayBuff[0] = 0
C: Int32ArrayBuff[1] = 5
C: Int32ArrayBuff[2] = 10
C: Int32ArrayBuff[3] = 15
C: Int32ArrayBuff[4] = 20
C: Int32ArrayBuff[5] = 25
C: Int32ArrayBuff[6] = 30
C: Int32ArrayBuff[7] = 35
C: Int32ArrayBuff[8] = 40
C: Int32ArrayBuff[9] = 45
js: Sum of the array = 225

How to read a HDF5 scalar attribute whose value is an array of variable length char* (i.e. c_strings?)

I have successfully created a scalar valued attribute whose value is a variable length array of const char*. I do not understand how to read this attribute however!
This is code I used to create the attribute:
void create_attribute_with_vector_of_strings_as_value()
{
using namespace H5;
// Create some test strings.
std::vector<std::string> strings;
for (int iii = 0; iii < 10; iii++)
{
strings.push_back("this is " + boost::lexical_cast<std::string>(iii));
}
// Part 1: grab pointers to the chars
std::vector<const char*> chars;
for (auto si = strings.begin(); si != strings.end(); ++si)
{
std::string &s = (*si);
chars.push_back(s.c_str());
}
BOOST_TEST_MESSAGE("Size of char* array is: " << chars.size());
// Part 2: create the variable length type
hvl_t hdf_buffer;
hdf_buffer.p = chars.data();
hdf_buffer.len = chars.size();
// Part 3: create the type
auto s_type = H5::StrType(H5::PredType::C_S1, H5T_VARIABLE);
auto svec_type = H5::VarLenType(&s_type);
try
{
// Open an existing file and dataset.
H5File file(m_file_name.c_str(), H5F_ACC_RDWR);
// Part 4: write the output to a scalar attribute
DataSet dataset = file.openDataSet(m_dataset_name.c_str());
std::string filter_names = "multi_filters";
Attribute attribute = dataset.createAttribute( filter_names.c_str(), svec_type, H5S_SCALAR);
attribute.write(svec_type, &hdf_buffer);
file.close();
}
Here is the dataset with attribute as seen from h5dump:
HDF5 "d:\tmp\hdf5_tutorial\h5tutr_dset.h5" {
GROUP "/" {
DATASET "dset" {
DATATYPE H5T_STD_I32BE
DATASPACE SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
DATA {
(0,0): 1, 7, 13, 19, 25, 31,
(1,0): 2, 8, 14, 20, 26, 32,
(2,0): 3, 9, 15, 21, 27, 33,
(3,0): 4, 10, 16, 22, 28, 34
}
ATTRIBUTE "multi_filters" {
DATATYPE H5T_VLEN { H5T_STRING {
STRSIZE H5T_VARIABLE;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}}
DATASPACE SCALAR
DATA {
(0): ("this is 0", "this is 1", "this is 2", "this is 3", "this is 4", "this is 5", "this is 6", "this is 7", "this is 8", "this is 9")
}
}
}
}
}
I do not understand how to read this data. The code I've experimented with so far is below. It compiles, but I've hardwired the array-size to the known length and the variable-length cstrings are empty? Does anyone have any suggestions as to where I'm going wrong? In particular, how do I query for the length of the array of const char* and how do I read the actual const char* cstrings contained in the array?
void read_attribute_with_vector_of_strings_as_value()
{
using namespace H5;
std::vector<std::string> strings;
try
{
// Open an existing file and dataset readonly
H5File file(m_file_name.c_str(), H5F_ACC_RDONLY);
// Part 4: Open the dataset
DataSet dataset = file.openDataSet(m_dataset_name.c_str());
// Atribute_name
std::string filter_names = "multi_filters";
Attribute attribute = dataset.openAttribute(filter_names.c_str());
size_t sz = attribute.getInMemDataSize();
size_t sz_1 = attribute.getStorageSize();
auto t1 = attribute.getDataType();
VarLenType t2 = attribute.getVarLenType();
H5T_class_t type_class = attribute.getTypeClass();
if (type_class == H5T_STRING)
BOOST_TEST_MESSAGE("H5T_STRING");
int length = 10;
std::vector<char*> tmp_vec(length);
auto s_type = H5::StrType(H5::PredType::C_S1, H5T_VARIABLE);
auto svec_type = H5::VarLenType(&s_type);
hvl_t hdf_buffer;
hdf_buffer.p = tmp_vec.data();
hdf_buffer.len = length;
attribute.read(svec_type, &hdf_buffer);
//attribute.read(s_type, &hdf_buffer);
//attribute.read(tmp_vec.data(), s_type);
for(size_t x = 0; x < tmp_vec.size(); ++x)
{
fprintf(stdout, "GOT STRING [%s]\n", tmp_vec[x] );
strings[x] = tmp_vec[x];
}
file.close();
}
If you are not required to use specific technologies to implement what you have in mind, you may consider HDFql (http://www.hdfql.com) which is a high-level language to manage HDF files easily (think SQL). That way you can be alleviated from all the low-level details of manipulating HDF files that you describe. Using HDFql in C++, reading an array of variable-length char is done like this:
// include HDFql C++ header file (make sure it can be found by the C++ compiler)
#include <iostream>
#include "HDFql.hpp"
int main(int argc, char *argv[])
{
// create an HDF file named "example.h5" and use (i.e. open) it
HDFql::execute("CREATE AND USE FILE example.h5");
// create an attribute named "multi_filters" of type varchar of one dimension (size 5)
HDFql::execute("CREATE ATTRIBUTE multi_filters AS VARCHAR(5)");
// insert (i.e. write) values "Red", "Green", "Blue", "Orange" and "Yellow" into attribute "multi_filters"
HDFql::execute("INSERT INTO multi_filters VALUES(Red, Green, Blue, Orange, Yellow)");
// select (i.e. read) attribute "multi_filters" into HDFql default cursor
HDFql::execute("SELECT FROM multi_filters");
// display content of HDFql default cursor
while(HDFql::cursorNext() == HDFql::Success)
{
std::cout << "Color " << HDFql::cursorGetChar() << " has a size of " << HDFql::cursorGetSize() << std::endl;
}
return 0;
}

Google Tests ValuesIn with a global vector

I'm trying to run a bunch of google tests based off configuration files in a directory. This way I can just add a new file and run the tests without having to add it to my parametrized test and recompile. Here is my code:
typedef std::pair<QString, int> TestParam;
QString generatedLogic;
std::vector<TestParam> badExpressionTests;
class LogicBuilderTest : public ::testing::TestWithParam<TestParam> {};
GTEST_API_ int main(int argc, char **argv) {
testing::InitGoogleTest(&argc, argv);
::generatedLogic = "/home/mitydsp/trunk/System/logicExecutionEngine/engine/include/GeneratedLogic.h";
QString dir = "/home/mitydsp/trunk/System/logicExecutionEngine/unittest/expressions/bad/";
QDirIterator it(dir, QStringList() << "*.txt", QDir::Files, QDirIterator::Subdirectories);
while (it.hasNext()) {
QString path = it.next();
QStringList nameParts = it.fileName().split("_");
int exitCode = nameParts[0].toInt();
::badExpressionTests.push_back(TestParam(path, exitCode));
}
std::cout << "Size of vector: " << badExpressionTests.size() << std::endl;
return RUN_ALL_TESTS();
}
/**
* Run parameterized test
*/
TEST_P(LogicBuilderTest, TestExitWithCode)
{
::testing::FLAGS_gtest_death_test_style = "threadsafe";
// Simulate fake main()
char arg0[] = "logicTest";
char* argv[] = { &arg0[0], NULL };
int argc = (int)(sizeof(argv) / sizeof(argv[0])) - 1;
EXPECT_EXIT({
// Need to run Qt Application because logic builder uses async QTimer::singleShot()
QCoreApplication app(argc, argv);
// Create a logic builder instance
LogicBuilder logicBuilder(
(int) ParameterStoreInterface::DEFAULT_PARAM_STORE_VALUES_KEY,
(int) ParameterStoreInterface::DEFAULT_PARAM_STORE_NAMES_KEY,
(int) ParameterStoreInterface::DEFAULT_PARAM_STORE_LOCK_KEY,
QString(GetParam().first),
QStringList(""),
QStringList(generatedLogic),
QString(LogicBuilder::DEFAULT_INTERMEDIATE_DIR),
QString(LogicBuilder::DEFAULT_OUTPUT_SRC),
QString(LogicBuilder::DEFAULT_OUTPUT_LIB),
true );
app.exec();
}, ::testing::ExitedWithCode(GetParam().second), "");
}
INSTANTIATE_TEST_CASE_P(TestBadExpressions, LogicBuilderTest,
::testing::ValuesIn(::badExpressionTests));
When I run this, it shows that 0 tests are being ran even though the vector says its size is two. How come these parametrized tests are not being ran?
Size of vector: 2
[==========] Running 0 tests from 0 test cases.
[==========] 0 tests from 0 test cases ran. (0 ms total)
[ PASSED ] 0 tests.
Originally I was running the tests by manually defining the configuration files but I don't like this:
INSTANTIATE_TEST_CASE_P(TestBadExpressions, LogicBuilderTest,
::testing::Values(
TestParam("unittest/expressions/bad/17_no_lhs.txt", LogicBuilder::LB_VARIABLE_NO_LHS),
TestParam("unittest/expressions/bad/25_incomplete_rhs.txt", LogicBuilder::LB_PARSE_ERROR)
));
I've spent the last hour trying to figure this out and then as soon as I post I come up with the solution. I'll leave this here because it may help someone in the future:
Instead of trying to create the list of files in main, pass the ValuesIn a function which loads the files and returns a vector.
std::vector<TestParam> GetFilesInDir()
{
std::vector<TestParam> values;
QString dir = "/home/mitydsp/trunk/System/logicExecutionEngine/unittest/expressions/bad/";
QDirIterator it(dir, QStringList() << "*.txt", QDir::Files, QDirIterator::Subdirectories);
while (it.hasNext()) {
QString path = it.next();
QStringList nameParts = it.fileName().split("_");
int exitCode = nameParts[0].toInt();
values.push_back(TestParam(path, exitCode));
}
return values;
}
INSTANTIATE_TEST_CASE_P(TestBadExpressions, LogicBuilderTest,
::testing::ValuesIn(GetFilesInDir()));
Thanks to Maksim Solovjov for reminding me that INSTANTIATE_TEST_CASE_P macro was being executed before main.
we spent some hours on this issue and thought this snippet will be helpful.
#include <iostream>
#include <string>
#include <vector>
using namespace std;
#include "gtest/gtest.h"
GTEST_API_ main( int argc, char* argv[] ) {
testing::InitGoogleTest( &argc, argv );
return RUN_ALL_TESTS();
}
struct test_parms_s {
string name; // Simple name of this test
string description; // Description of this test
void* function_parms; // Pointer to test function specific data
};
std::vector<test_parms_s*> BuildTestData() {
test_parms_s* pTestDataA = new test_parms_s();
test_parms_s* pTestDataB = new test_parms_s();
test_parms_s* pTestDataC = new test_parms_s();
pTestDataA->name.assign("testA");
pTestDataA->description.assign("testA_desc");
pTestDataA->function_parms = NULL;
pTestDataB->name.assign("testB");
pTestDataB->description.assign("testB_desc");
pTestDataB->function_parms = NULL;
pTestDataC->name.assign("testC");
pTestDataC->description.assign("testC_desc");
pTestDataC->function_parms = NULL;
std::vector<test_parms_s*> values;
values.push_back(pTestDataA);
values.push_back(pTestDataB);
values.push_back(pTestDataC);
cout << "BuildTestData" << endl;
return values;
}
//------------------------------------------------------------------------
class Testy {
private:
string testname_;
public:
Testy( string testname );
~Testy();
string GetTestName();
void SetTestName( string testname );
};
Testy::Testy( string testname ) {
testname_.assign(testname);
}
Testy::~Testy() {}
string Testy::GetTestName() { return testname_; }
void Testy::SetTestName( string testname ) { testname_ = testname; }
//------------------------------------------------
class TestFixture : public ::testing::TestWithParam<test_parms_s*> {
protected:
Testy* testy;
virtual void SetUp() {
testy = new Testy( "Test");
}
virtual void TearDown() {
delete( testy );
}
};
//------------------------------------------------
TEST_P( TestFixture, ParamTest ) {
test_parms_s* pTestParms = GetParam();
cout << pTestParms->name << endl;
}
//------------------------------------------------
INSTANTIATE_TEST_CASE_P( TestFixture_instance, TestFixture,
::testing::ValuesIn(BuildTestData()));
//========================================================================
the output will be
BuildTestData
[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from TestFixture_instance/TestFixture
[ RUN ] TestFixture_instance/TestFixture.ParamTest/0
testA
[ OK ] TestFixture_instance/TestFixture.ParamTest/0 (0 ms)
[ RUN ] TestFixture_instance/TestFixture.ParamTest/1
testB
[ OK ] TestFixture_instance/TestFixture.ParamTest/1 (0 ms)
[ RUN ] TestFixture_instance/TestFixture.ParamTest/2
testC
[ OK ] TestFixture_instance/TestFixture.ParamTest/2 (0 ms)
[----------] 3 tests from TestFixture_instance/TestFixture (0 ms total)
[----------] Global test environment tear-down
[==========] 3 tests from 1 test case ran. (0 ms total)
[ PASSED ] 3 tests.
I'm not an expert in manually instantiating Google Tests, but the way you did it can't possibly work: you call INSTANTIATE_TEST_CASE_P in the static context, so it is being evaluated before main is ever called.
You could try moving instantiation just before RUN_ALL_TESTS; however, I don't know what that macro does and this might be illegal. In that case, I think that you can't create your tests with INSTANTIATE_TEST_CASE_P in the dynamic way that you want.

How to train in Matlab a model, save it to disk, and load in C++ program?

I am using libsvm version 3.16. I have done some training in Matlab, and created a model. Now I would like to save this model to disk and load this model in my C++ program. So far I have found the following alternatives:
This answer explains how to save a model from C++, which is based on this website. Not exactly what I need, but could be adapted. (This requires development time).
I could find the best training parameters (kernel,C) in Matlab and re-train everything in C++. (Will require doing the training in C++ each time I change a parameter. It's not scalable).
Thus, both of these options are not satisfactory,
Does anyone have an idea?
My solution was to retrain in C++ because I couldn't find a nice way to directly save the model. Here's my code. You'll need to adapt it and clean it up a bit. The biggest change you'll have to make it not hard coding the svm_parameter values like I did. You'll also have to replace FilePath with std::string. I'm copying, pasting and making small edits here in SO so the formatting won't e perfect:
Used like this:
auto targetsPath = FilePath("targets.txt");
auto observationsPath = FilePath("observations.txt");
auto targetsMat = MatlabMatrixFileReader::Read(targetsPath, ',');
auto observationsMat = MatlabMatrixFileReader::Read(observationsPath, ',');
auto v = MiscVector::ConvertVecOfVecToVec(targetsMat);
auto model = SupportVectorRegressionModel{ observationsMat, v };
std::vector<double> observation{ { // 32 feature observation
0.883575729725847,0.919446119013878,0.95359403450317,
0.968233630936732,0.91891307107125,0.887897763183844,
0.937588566544751,0.920582702918882,0.888864454119387,
0.890066735260163,0.87911085669864,0.903745573664995,
0.861069296586979,0.838606194934074,0.856376230548304,
0.863011311537075,0.807688936997926,0.740434984165146,
0.738498042748759,0.736410940165691,0.697228384912424,
0.608527698289016,0.632994967880269,0.66935784966765,
0.647761430696238,0.745961037635717,0.560761134660957,
0.545498063585615,0.590854855113663,0.486827902942118,
0.187128866890822,- 0.0746523069562551
} };
double prediction = model.Predict(observation);
miscvector.h
static vector<double> ConvertVecOfVecToVec(const vector<vector<double>> &mat)
{
vector<double> targetsVec;
targetsVec.reserve(mat.size());
for (size_t i = 0; i < mat.size(); i++)
{
targetsVec.push_back(mat[i][0]);
}
return targetsVec;
}
libsvmtargetobjectconvertor.h
#pragma once
#include "machinelearning.h"
struct svm_node;
class LibSvmTargetObservationConvertor
{
public:
svm_node ** LibSvmTargetObservationConvertor::ConvertObservations(const vector<MlObservation> &observations, size_t numFeatures) const
{
svm_node **svmObservations = (svm_node **)malloc(sizeof(svm_node *) * observations.size());
for (size_t rowI = 0; rowI < observations.size(); rowI++)
{
svm_node *row = (svm_node *)malloc(sizeof(svm_node) * numFeatures);
for (size_t colI = 0; colI < numFeatures; colI++)
{
row[colI].index = colI;
row[colI].value = observations[rowI][colI];
}
row[numFeatures].index = -1; // apparently needed
svmObservations[rowI] = row;
}
return svmObservations;
}
svm_node* LibSvmTargetObservationConvertor::ConvertMatToSvmNode(const MlObservation &observation) const
{
size_t numFeatures = observation.size();
svm_node *obsNode = (svm_node *)malloc(sizeof(svm_node) * numFeatures);
for (size_t rowI = 0; rowI < numFeatures; rowI++)
{
obsNode[rowI].index = rowI;
obsNode[rowI].value = observation[rowI];
}
obsNode[numFeatures].index = -1; // apparently needed
return obsNode;
}
};
machinelearning.h
#pragma once
#include <vector>
using std::vector;
using MlObservation = vector<double>;
using MlTarget = double;
//machinelearningmodel.h
#pragma once
#include <vector>
#include "machinelearning.h"
class MachineLearningModel
{
public:
virtual ~MachineLearningModel() {}
virtual double Predict(const MlObservation &observation) const = 0;
};
matlabmatrixfilereader.h
#pragma once
#include <vector>
using std::vector;
class FilePath;
// Matrix created with command:
// dlmwrite('my_matrix.txt', somematrix, 'delimiter', ',', 'precision', 15);
// In these files, each row is a matrix row. Commas separate elements on a row.
// There is no space at the end of a row. There is a blank line at the bottom of the file.
// File format:
// 0.4,0.7,0.8
// 0.9,0.3,0.5
// etc.
static class MatlabMatrixFileReader
{
public:
static vector<vector<double>> Read(const FilePath &asciiFilePath, char delimiter)
{
vector<vector<double>> values;
vector<double> valueline;
std::ifstream fin(asciiFilePath.Path());
string item, line;
while (getline(fin, line))
{
std::istringstream in(line);
while (getline(in, item, delimiter))
{
valueline.push_back(atof(item.c_str()));
}
values.push_back(valueline);
valueline.clear();
}
fin.close();
return values;
}
};
supportvectorregressionmodel.h
#pragma once
#include <vector>
using std::vector;
#include "machinelearningmodel.h"
#include "svm.h" // libsvm
class FilePath;
class SupportVectorRegressionModel : public MachineLearningModel
{
public:
SupportVectorRegressionModel::~SupportVectorRegressionModel()
{
svm_free_model_content(model_);
svm_destroy_param(&param_);
svm_free_and_destroy_model(&model_);
}
SupportVectorRegressionModel::SupportVectorRegressionModel(const vector<MlObservation>& observations, const vector<MlTarget>& targets)
{
// assumes all observations have same number of features
size_t numFeatures = observations[0].size();
//setup targets
//auto v = ConvertVecOfVecToVec(targetsMat);
double *targetsPtr = const_cast<double *>(&targets[0]); // why aren't the targets const?
LibSvmTargetObservationConvertor conv;
svm_node **observationsPtr = conv.ConvertObservations(observations, numFeatures);
// setup observations
//svm_node **observations = BuildObservations(observationsMat, numFeatures);
// setup problem
svm_problem problem;
problem.l = targets.size();
problem.y = targetsPtr;
problem.x = observationsPtr;
// specific to out training sets
// TODO: This is hard coded.
// Bust out these values for use in constructor
param_.C = 0.4; // cost
param_.svm_type = 4; // SVR
param_.kernel_type = 2; // radial
param_.nu = 0.6; // SVR nu
// These values are the defaults used in the Matlab version
// as found in svm_model_matlab.c
param_.gamma = 1.0 / (double)numFeatures;
param_.coef0 = 0;
param_.cache_size = 100; // in MB
param_.shrinking = 1;
param_.probability = 0;
param_.degree = 3;
param_.eps = 1e-3;
param_.p = 0.1;
param_.shrinking = 1;
param_.probability = 0;
param_.nr_weight = 0;
param_.weight_label = NULL;
param_.weight = NULL;
// suppress command line output
svm_set_print_string_function([](auto c) {});
model_ = svm_train(&problem, &param_);
}
double SupportVectorRegressionModel::Predict(const vector<double>& observation) const
{
LibSvmTargetObservationConvertor conv;
svm_node *obsNode = conv.ConvertMatToSvmNode(observation);
double prediction = svm_predict(model_, obsNode);
return prediction;
}
SupportVectorRegressionModel::SupportVectorRegressionModel(const FilePath & modelFile)
{
model_ = svm_load_model(modelFile.Path().c_str());
}
private:
svm_model *model_;
svm_parameter param_;
};
Option 1 is actually pretty reasonable. If you save the model in libsvm's C format through matlab, then it is straightforward to work with the model in C/C++ using functions provided by libsvm. Trying to work with matlab-formatted data in C++ will probably be much more difficult.
The main function in "svm-predict.c" (located in the root directory of the libsvm package) probably has most of what you need:
if((model=svm_load_model(argv[i+1]))==0)
{
fprintf(stderr,"can't open model file %s\n",argv[i+1]);
exit(1);
}
To predict a label for example x using the model, you can run
int predict_label = svm_predict(model,x);
The trickiest part of this will be to transfer your data into the libsvm format (unless your data is in the libsvm text file format, in which case you can just use the predict function in "svm-predict.c").
A libsvm vector, x, is an array of struct svm_node that represents a sparse array of data. Each svm_node has an index and a value, and the vector must be terminated by an index that is set to -1. For instance, to encode the vector [0,1,0,5], you could do the following:
struct svm_node *x = (struct svm_node *) malloc(3*sizeof(struct svm_node));
x[0].index=2; //NOTE: libsvm indices start at 1
x[0].value=1.0;
x[1].index=4;
x[1].value=5.0;
x[2].index=-1;
For SVM types other than the classifier (C_SVC), look at the predict function in "svm-predict.c".