Basically what I'm trying to do is get the x and y coordinates from a ProtoBuf message that is similar to the SHPObject type. I know that with SHPObject I can do things like double* x = obj->padfX and double* y = obj->padfY. However, I'm not entirely sure how to extract this same information from my ProtoBuf (code is shown below). Thank you for your help!
This what I have tried so far:
myProject::protobuf::NewShape _NewShape;
auto obj = _NewShape.shape(0);
double* x = obj.polygon(0).point(0);
Gives the error:
cannot initialize a variable of the type 'double' with an rvalue of type 'unsigned int'
And, then I tried this which compiles but doesn't do anything (does not give me the desired output):
double x_coordinate = obj.polygon(0).point(0);
double *x_ptr = &x_coordinate;
Here's my ProtoBuf file:
newShape.proto
syntax = "proto2";
package myProject.protobuf;
message NewShape {
message Polygon
{
enum PolygonType {
POLY_TYPE_OUTER = 1;
POLY_TYPE_INNER = 2;
};
optional PolygonType type = 1 [default = POLY_TYPE_OUTER];
// x, y coordinates
repeated uint32 point = 2 [packed = true];
}
message Shape
{
repeated Polygon polygon = 1;
}
repeated Shape shape = 2;
}
Given your format, you can access points from a properly populated and deserialized object like this:
// newShape is the deserialized object here
const auto s0p0p0 = newShape.shape(0).polygon(0).point(0);
const auto s0p0p1 = newShape.shape(0).polygon(0).point(1);
Similarly, shape(1) would give you the second shape to access it points in the polygon object. You should check the *_size() methods before accessing an index to ensure valid access e.g. newShape.shape_size(), shape.polygon_size() and polygon.point_size().
If you intend to modify the message, you can use mutable_* methods to get the pointers to respective objects and then you can change those.
For example (change points of first polygon of first shape):
auto p = newShape2.mutable_shape(0);
p->mutable_polygon(0)->set_point(0, 123);
p->mutable_polygon(0)->set_point(1, 345);
Here's a complete working example of serialization and deserialization:
#include <iostream>
#include "newShape.pb.h"
int main()
{
// Serailization
::myProject::protobuf::NewShape newShape1;
auto shape1 = newShape1.add_shape();
auto polygon1 = shape1->add_polygon();
polygon1->add_point(1);
polygon1->add_point(2);
auto shape2 = newShape1.add_shape();
auto polygon2 = shape2->add_polygon();
polygon2->add_point(3);
polygon2->add_point(4);
const auto serializedData = newShape1.SerializeAsString();
std::cout << "Serialized Data Size: " << serializedData.size() << "\n\n";
// Send/Store Serialized Data
// Deserialization
::myProject::protobuf::NewShape newShape2;
if ( !newShape2.ParseFromString( serializedData ) )
{
std::cerr << "Deserialization failed!\n";
return -1;
}
std::cout << "Deserialized Data Size: " << newShape2.ByteSize() << "\n\n";
std::cout << "NewShape [Shapes: " << newShape2.shape_size() << "]\n";
for ( int i {0}; i < newShape2.shape_size(); ++i )
{
std::cout << " Shape # " << i << '\n';
const auto& shape = newShape2.shape( i );
for ( int j {0}; j < shape.polygon_size(); ++j )
{
std::cout << " Polygon # " << j << '\n';
const auto& polygon = shape.polygon( j );
for ( int k {0}; k < polygon.point_size(); ++k )
{
const auto& point = polygon.point( k );
std::cout << " Point # " << k << ": " << point << '\n';
}
}
}
return 0;
}
Output:
Serialized Data Size: 16
Deserialized Data Size: 16
NewShape [Shapes: 2]
Shape # 0
Polygon # 0
Point # 0: 1
Point # 1: 2
Shape # 1
Polygon # 0
Point # 0: 3
Point # 1: 4
i'm trying to read from a binary file to a char array. When printing array entries, an arbitrary number (newline) and the desired number are being printed. I really cant get my head around this.
The first few bytes of the file are:
00 00 08 03 00 00 EA 60 00 00 00 1C 00 00 00 1C 00 00
My Code:
void MNISTreader::loadImagesAndLabelsToMemory(std::string imagesPath,
std::string labelsPath) {
std::ifstream is(imagesPath.c_str());
char *data = new char[12];
is.read(data, 12);
std::cout << std::hex << (int)data[2] << std::endl;
delete [] data;
is.close();
}
E.g it prints:
ffffff9b
8
8 is correct. The preceding number changes from execution to execution. And where does this newline come from?
You asked about reading data from a binary file and saving it into a char[] and you showed us this code that you submitted for your question:
void MNISTreader::loadImagesAndLabelsToMemory(std::string imagesPath,
std::string labelsPath) {
std::ifstream is(imagesPath.c_str());
char *data = new char[12];
is.read(data, 12);
std::cout << std::hex << (int)data[2] << std::endl;
delete [] data;
is.close();
}
And you wanted to know:
The preceding number changes from execution to execution. And where does this newline come from?
Before you can actually answer that question you need to know the binary file. That is what is the structure of the file internally. When you are reading data from a binary you have to remember that some program had written data to that file and that data was written in a structured format. It is this format that is unique to each family or file type of binary that is important. Most binaries will usually follow a common pattern such that they would container a header then maybe even sub headers then either clusters, or packets or chunks, etc. or even raw data after the header while some binaries may just be purely raw data. You have to know how the file is structured in memory.
What is the structure of the data?
Is the data type for the first entry into the file a char = 1 byte, int = 4 bytes (32bit system) 8 bytes (64bit system), float = 4bytes, double = 8bytes, etc.
According to your code you have an array of char with a size of 12 and knowing that a char is 1 byte in memory you are asking for 12 bytes. Now the problem here is that you are pulling off 12 consecutive individual bytes in a row and by not knowing the file structure how can you determine if the first byte was an actual char written or an unsigned char, or a int?
Consider these two different binary file structures that are created by C++ structs that contains all the needed data and both are written out to a file in a binary format.
A Generic Header Structure that both File Structures will use.
struct Header {
// Size of Header
std::string filepath;
std::string filename;
unsigned int pathSize;
unsigned int filenameSize;
unsigned int headerSize;
unsigned int dataSizeInBytes;
};
FileA Unique Structure For File A
struct DataA {
float width;
float length;
float height;
float dummy;
}
FileB Unique Structure For File B
struct DataB {
double length;
double width;
}
A File in memory in general would be something like this:
First Few Bytes are the path and file name and stored sizes
This can vary from file to file depending on how many characters
are used for both the file path and file name.
After the strings we do know that the next 4 data types are unsigned
so we know that on a 32bit system it will be 4bytes x 4 = 16 total bytes
For a 64bit system it will be 8bytes x 4 = 32 total bytes.
If we know the system architecture then we can get past this easily enough.
Of these 4 unsigned(s) the first two are for the length of the path and filename. Now these could be the first two read in from the file and not the actual paths. The order of these could be reversed.
It is the next 2 unsigned(s) that are of importance
The next is the full size of the header and can be used to read in and skip over the header.
The next one tells you the size of the data to be pulled in now these could be in chunks with a count of how many chunks because it could be a series of the same data structures but for simplicity I left out chunks and counts and using a single instance structure.
It is here were we can then extract the amount of data in bytes by how many bytes to extract.
Lets consider the two different binary files where we are already past all the header information and we are reading in the bytes to parse. We get to the size of the data in bytes and for FileA we have 4 floats = 16bytes and for FileB we have 2 doubles = 16bytes. So now we know how to call the method to read in x amount of data for a y type of data. Since y is now a type and x is amount of we can say this: y(x) As if y is a built in type and x is a numerical initializer for the default built in type of constructor for this built in type either it be an int, float, double, char, etc.
Now let's say we were reading in either one of these two files but didn't know the data structure and how its information was previously stored to the file and we are seeing by the header that the data size is 16 bytes in memory but we didn't know if it was being stored as either 4 floats = 16 bytes or 2 doubles = 16 bytes. Both structures are 16 bytes but have a different amount of different data types.
The summation of this is that without knowing the file's data structure and knowing how to parse the binary does become an X/Y Problem
Now let's assume that you do know the file structure to try and answer your question from above you can try this little program and to check out some results:
#include <string>
#include <iostream>
int main() {
// Using Two Strings
std::string imagesPath("ImagesPath\\");
std::string labelsPath("LabelsPath\\");
// Concat of Two Strings
std::string full = imagesPath + labelsPath;
// Display Of Both
std::cout << full << std::endl;
// Data Type Pointers
char* cData = nullptr;
cData = new char[12];
unsigned char* ucData = nullptr;
ucData = new unsigned char[12];
// Loop To Set Both Pointers To The String
unsigned n = 0;
for (; n < 12; ++n) {
cData[n] = full.at(n);
ucData[n] = full.at(n);
}
// Display Of Both Strings By Character and Unsigned Character
n = 0;
for (; n < 12; ++n) {
std::cout << cData[n];
}
std::cout << std::endl;
n = 0;
for (; n < 12; ++n) {
std::cout << ucData[n];
}
std::cout << std::endl;
// Both Yeilds Same Result
// Okay lets clear out the memory of these pointers and then reuse them.
delete[] cData;
delete[] ucData;
cData = nullptr;
ucData = nullptr;
// Create Two Data Structurs 1 For Each Different File
struct A {
float length;
float width;
float height;
float padding;
};
struct B {
double length;
double width;
};
// Constants For Our Data Structure Sizes
const unsigned sizeOfA = sizeof(A);
const unsigned sizeOfB = sizeof(B);
// Create And Populate An Instance Of Each
A a;
a.length = 3.0f;
a.width = 3.0f;
a.height = 3.0f;
a.padding = 0.0f;
B b;
b.length = 5.0;
b.width = 5.0;
// Lets First Use The `Char[]` Method for each struct and print them
// but we need 16 bytes instead of `12` from your problem
char *aData = nullptr; // FileA
char *bData = nullptr; // FileB
aData = new char[16];
bData = new char[16];
// Since A has 4 floats we know that each float is 4 and 16 / 4 = 4
aData[0] = a.length;
aData[4] = a.width;
aData[8] = a.height;
aData[12] = a.padding;
// Print Out Result but by individual bytes without casting for A
// Don't worry about the compiler warnings and build and run with the
// warning and compare the differences in what is shown on the screen
// between A & B.
n = 0;
for (; n < 16; ++n) {
std::cout << aData[n] << " ";
}
std::cout << std::endl;
// Since B has 2 doubles weknow that each double is 8 and 16 / 8 = 2
bData[0] = b.length;
bData[8] = b.width;
// Print out Result but by individual bytes without casting for B
n = 0;
for (; n < 16; ++n) {
std::cout << bData[n] << " ";
}
std::cout << std::endl;
// Let's Print Out Both Again But By Casting To Their Approriate Types
n = 0;
for (; n < 4; ++n) {
std::cout << reinterpret_cast<float*>(aData[n]) << " ";
}
std::cout << std::endl;
n = 0;
for (; n < 2; ++n) {
std::cout << reinterpret_cast<double*>(bData[n]) << " ";
}
std::cout << std::endl;
// Clean Up Memory
delete[] aData;
delete[] bData;
aData = nullptr;
bData = nullptr;
// Even By Knowing The Appropriate Sizes We Can See A Difference
// In The Stored Data Types. We Can Now Do The Same As Above
// But With Unsigned Char & See If It Makes A Difference.
unsigned char *ucAData = nullptr;
unsigned char *ucBData = nullptr;
ucAData = new unsigned char[16];
ucBData = new unsigned char[16];
// Since A has 4 floats we know that each float is 4 and 16 / 4 = 4
ucAData[0] = a.length;
ucAData[4] = a.width;
ucAData[8] = a.height;
ucAData[12] = a.padding;
// Print Out Result but by individual bytes without casting for A
// Don't worry about the compiler warnings and build and run with the
// warning and compare the differences in what is shown on the screen
// between A & B.
n = 0;
for (; n < 16; ++n) {
std::cout << ucAData[n] << " ";
}
std::cout << std::endl;
// Since B has 2 doubles weknow that each double is 8 and 16 / 8 = 2
ucBData[0] = b.length;
ucBData[8] = b.width;
// Print out Result but by individual bytes without casting for B
n = 0;
for (; n < 16; ++n) {
std::cout << ucBData[n] << " ";
}
std::cout << std::endl;
// Let's Print Out Both Again But By Casting To Their Approriate Types
n = 0;
for (; n < 4; ++n) {
std::cout << reinterpret_cast<float*>(ucAData[n]) << " ";
}
std::cout << std::endl;
n = 0;
for (; n < 2; ++n) {
std::cout << reinterpret_cast<double*>(ucBData[n]) << " ";
}
std::cout << std::endl;
// Clean Up Memory
delete[] ucAData;
delete[] ucBData;
ucAData = nullptr;
ucBData = nullptr;
// So Even Changing From `char` to an `unsigned char` doesn't help here even
// with reinterpret casting. Because These 2 Files Are Different From One Another.
// They have a unique signature. Now a family of files where a specific application
// saves its data to a binary will all follow the same structure. Without knowing
// the structure of the binary file and knowing how much data to pull in and the big key
// word here is `what type` of data you are reading in and by how much. This becomes an (X/Y) Problem.
// This is the hard part about parsing binaries, you need to know the file structure.
char c = ' ';
std::cin.get(c);
return 0;
}
After running the short program above don't worry about what each value being displayed to the screen is; just look at the patterns that are there for the comparison of the two different file structures. This is just to show that a struct of floats that is 16 bytes wide is not the same as a struct of doubles that is also 16 bytes wide. So when we go back to your problem and you are reading in 12 individual consecutive bytes the question then becomes what does these first 12 bytes represent? Is it 3 ints or 3 unsigned ints if on 32bit machine or 2 ints or 2 unsigned ints on a 64bit machine, or 3 floats, or is a combination such as 2 doubles and 1 float? What is the current data structure of the binary file you are reading in?
Edit In my little program that I wrote; I did forget to try or add in the << std::hex << in the print out statements they can be added in as well were each printing of the index pointers are used but there is no need to do so because the output to the display is the same exact thing as this only shows or expresses visually the difference of the two data structures in memory and what their patterns look like.
I'm screwing around with opengl in c++, this is my first time really writing anything in c++, so I'm having troubles with pointers and arrays. I'd like to make a function that makes a strip of terrain (represented by an array of vertices, each vertex being an array of 3 floats) and returns it. This 2d array (unknown sized array of 3 sized arrays) can then be rendered with gl triangles. I'm not having any trouble with rendering it, but when my code runs currently, when I log the values of the vertices in the array local to the function, everything is fine, but in the main source, the values aren't logged correctly.
tl;dr:
How can I generate an array of arrays in a c++ function.
Here's my function right now:
float** makeTerrainStrip(float tile, unsigned int tiles)
{
float** strip = new float*[(tiles + 1) * 2];
for (unsigned int i = 0; i < tiles; i++) {
float x = i * tile;
float y = 0;
float z1 = 0;
float z2 = tile;
float v1[3] = {x, y, z1};
float v2[3] = {x, y, z2};
strip[i * 2] = v1;
strip[i * 2 + 1] = v2;
}
std::cout << strip[0][0] << std::endl;
std::cout << strip[0][1] << std::endl;
std::cout << strip[0][2] << std::endl;
return strip;
}
and it gets called from the main source here:
strip = makeTerrainStrip(1, 5); // strip is a global defined earlier
std::cout << strip[0][0] << std::endl;
std::cout << strip[0][1] << std::endl;
std::cout << strip[0][2] << std::endl;
Here's the output:
// in the function
4
0
0
// in the main source
4
1.26749e+029
7987.1
How can I use XGBOOST https://github.com/dmlc/xgboost/ library in c++? I have founded Python and Java API, but I can't found API for c++
I ended up using the C API, see below an example:
// create the train data
int cols=3,rows=5;
float train[rows][cols];
for (int i=0;i<rows;i++)
for (int j=0;j<cols;j++)
train[i][j] = (i+1) * (j+1);
float train_labels[rows];
for (int i=0;i<rows;i++)
train_labels[i] = 1+i*i*i;
// convert to DMatrix
DMatrixHandle h_train[1];
XGDMatrixCreateFromMat((float *) train, rows, cols, -1, &h_train[0]);
// load the labels
XGDMatrixSetFloatInfo(h_train[0], "label", train_labels, rows);
// read back the labels, just a sanity check
bst_ulong bst_result;
const float *out_floats;
XGDMatrixGetFloatInfo(h_train[0], "label" , &bst_result, &out_floats);
for (unsigned int i=0;i<bst_result;i++)
std::cout << "label[" << i << "]=" << out_floats[i] << std::endl;
// create the booster and load some parameters
BoosterHandle h_booster;
XGBoosterCreate(h_train, 1, &h_booster);
XGBoosterSetParam(h_booster, "booster", "gbtree");
XGBoosterSetParam(h_booster, "objective", "reg:linear");
XGBoosterSetParam(h_booster, "max_depth", "5");
XGBoosterSetParam(h_booster, "eta", "0.1");
XGBoosterSetParam(h_booster, "min_child_weight", "1");
XGBoosterSetParam(h_booster, "subsample", "0.5");
XGBoosterSetParam(h_booster, "colsample_bytree", "1");
XGBoosterSetParam(h_booster, "num_parallel_tree", "1");
// perform 200 learning iterations
for (int iter=0; iter<200; iter++)
XGBoosterUpdateOneIter(h_booster, iter, h_train[0]);
// predict
const int sample_rows = 5;
float test[sample_rows][cols];
for (int i=0;i<sample_rows;i++)
for (int j=0;j<cols;j++)
test[i][j] = (i+1) * (j+1);
DMatrixHandle h_test;
XGDMatrixCreateFromMat((float *) test, sample_rows, cols, -1, &h_test);
bst_ulong out_len;
const float *f;
XGBoosterPredict(h_booster, h_test, 0,0,&out_len,&f);
for (unsigned int i=0;i<out_len;i++)
std::cout << "prediction[" << i << "]=" << f[i] << std::endl;
// free xgboost internal structures
XGDMatrixFree(h_train[0]);
XGDMatrixFree(h_test);
XGBoosterFree(h_booster);
Use XGBoost C API.
BoosterHandle booster;
const char *model_path = "/path/of/model";
// create booster handle first
XGBoosterCreate(NULL, 0, &booster);
// by default, the seed will be set 0
XGBoosterSetParam(booster, "seed", "0");
// load model
XGBoosterLoadModel(booster, model_path);
const int feat_size = 100;
const int num_row = 1;
float feat[num_row][feat_size];
// create some fake data for predicting
for (int i = 0; i < num_row; ++i) {
for(int j = 0; j < feat_size; ++j) {
feat[i][j] = (i + 1) * (j + 1)
}
}
// convert 2d array to DMatrix
DMatrixHandle dtest;
XGDMatrixCreateFromMat(reinterpret_cast<float*>(feat),
num_row, feat_size, NAN, &dtest);
// predict
bst_ulong out_len;
const float *f;
XGBoosterPredict(booster, dtest, 0, 0, &out_len, &f);
assert(out_len == num_row);
std::cout << f[0] << std::endl;
// free memory
XGDMatrixFree(dtest);
XGBoosterFree(booster);
Note when you want to load an existing model(like above code shows), you have to ensure the data format in training is the same as in predicting. So, if you predict with XGBoosterPredict, which accepts a dense matrix as parameter, you have to use dense matrix in training.
Training with libsvm format and predict with dense matrix may cause wrong predictions, as XGBoost FAQ says:
“Sparse” elements are treated as if they were “missing” by the tree booster, and as zeros by the linear booster. For tree models, it is important to use consistent data formats during training and scoring.
Here is what you need：https://github.com/EmbolismSoil/xgboostpp
#include "xgboostpp.h"
#include <algorithm>
#include <iostream>
int main(int argc, const char* argv[])
{
auto nsamples = 2;
auto xgb = XGBoostPP(argv[1], 3); //特征列有4列, label有3个, iris例子中分别为三种类型的花，回归任何的话，这里nlabel=1即可
//result = array([[9.9658281e-01, 2.4966884e-03, 9.2058454e-04],
// [9.9608469e-01, 2.4954407e-03, 1.4198524e-03]], dtype=float32)
XGBoostPP::Matrix features(2, 4);
features <<
5.1, 3.5, 1.4, 0.2,
4.9, 3.0, 1.4, 0.2;
XGBoostPP::Matrix y;
auto ret = xgb.predict(features, y);
if (ret != 0){
std::cout << "predict error" << std::endl;
}
std::cout << "intput : \n" << features << std::endl << "output: \n" << y << std::endl;
}
In case training in Python is okay and you only need to run the prediction in C++, there is a nice tool for generating static if/else-code from a trained model:
https://github.com/popcorn/xgb2cpp
I ended up using this after spending a day trying to load and use a xgboost model in C++ without success. The code generated by xgb2cpp was working instantly and also has the nice benefit that it does not have any dependencies.
There is no example I am aware of. there is a c_api.h file that contains a C/C++ api for the package, and you'll have to find your way using it. I've just did that. Took me a few hours reading the code and trying few things out. But eventually I managed to create a working C++ example of xgboost.
To solve this problem we runs the xgboost program from C++ source code.
I am trying to get the value of an element in an array in order to use it in an if statement but unfortunately the following code is not working for me. The cout of comp is not matching the first element of the array C. I'm new to OpenCV so any help is appreciated.
Mat A = (Mat_<double>(2,1) << u, v);
Mat B = (Mat_<double>(2,6) << -1/Z, 0 , x/Z , x*y , -(x*x+1),y,
0 ,-1/Z, y/Z ,y*y+1, -x*y ,-x);
Mat pinvB = B.inv(DECOMP_SVD);
Mat C=pinvB*A; // 6x1 Array
float comp = C.at<float>(0,0);
cout << "comp " << comp << endl; //This value does not match C[0,0]
cout << "C " << C << endl;
if (comp < 0.0001){
//process
}
Your Mat_<double> instances internally store doubles. When you do this:
float comp = C.at<float>(0,0);
you are trying to use some of the bits that form a double, and interpret them as a float. Floating point representation means that half of the bits of a double don't translate into a meaningful float (assuming a platform where float has half the size of a double, which is quite common). So, call C.at<double> instead.
Actually, if you use the template version of cv::Mat_<_Tp>, you can access pixel value by Mat_<_Tp>::operator ()(int y, int x)
cv::Mat_<double> M(3, 3);
for (int i = 0;i < 3; ++i) {
for (int j = 0;j < 3; ++j) {
std::cout<<M(i, j)<<std::endl;
}
}
so that later if you change the template argument from double to float, you don't need to modify each at().