org.proto
message Optimize {
required int element_size = 1;
required string element_name = 2;
}
message nodes {
repeated Optimize = 1;
}
Have this decoding function:
DecodeNodeMsg(char *msg, int size)
{
Org::nodes node;
int element_size;
string element_name;
int NumofElem = 0;
node.ParseFromArray((void *)msg, size);
for (int i = 0; i < nodes.Optimize().size(); i++)
{
element_size = nodes.Optimize(i).element_size();
element_name = nodes.Optimize(i).element_name();
cout << "size" << element_size << endl;
cout << "name" << element_name << endl;
NumofElem++;
}
cout << "number of" << NumofElem << endl;
}
I am encoding a nodes message with three Optimize messages in it. and calling this decode function. Encoding part is an old code which is working fine from long time. So i dont suspect the encoding function.
In the decode function, I see that the NumofElem is correctly printed as three. However, i see that both element_size & element_name are just garbage. Integer has some junk value and string has a binary data.
I am having this issue only if this repeated fields. If fields are required/optional fields, then i dont have this issue.
Could some one has similar issue... ? if so any clues on how to fix this ?
Thanks,
Kiran
I don't see where you are actually decoding a message. I see you creating a new node object, but then calling Org::nodes() which looks wrong. I think you need to access the Optimize elements like this:
for (int i = 0; i < node->optimize_size(); i++)
{
element_size = node->optimize(i).element_size();
element_name = node->optimize(i).element_name();
cout << "size" << element_size << endl;
cout << "name" << element_name << endl;
NumofElem++;
}
But again I think your nodes object needs to be decoded from something. The mutable methods allow you to set data. There's also the ParseFrom methods. Also in my proto files I number the elements in a message. See https://developers.google.com/protocol-buffers/docs/overview
message nodes {
repeated Optimize = 1;
}
The function deserializes the buffer into the local variable node, but the loop references nodes. I'd also validate the return value from ParseFromArray.
Related
First off, I know there are posts with similar problems, but I cannot find the solution to mine in any of them.
This is for a programming assignment using binary and text files to store "Corporate sales data." (Division name, quarter, and sales), and then to search for specified records inside the binary data file and display them.
Here are the important parts of my code:
#include stuff
...
// Struct to hold division data
struct DIVISION_DATA_S
{
string divisionName;
int quarter;
double sales;
};
int main()
{
...
// Open the data file
fstream dataFile;
dataFile.open(dataFilePath, ios::in | ios::out | ios::binary);
... Get data from user, store in an instance of my struct ...
// Dump struct into binary file
dataFile.write(reinterpret_cast<char *>(&divisionData), sizeof(divisionData));
// Cycle through the targets file and display the record from divisiondata.dat for each entry
while(targetsFile >> targetDivisionName)
{
int targetQuarter; // Target quarter
string targetQuarterStr;
targetsFile.ignore(); // Ignore the residual '\n' from the ">>" read
getline(targetsFile, targetQuarterStr);
targetQuarter = atoi(targetQuarterStr.c_str()); // Parses into an int
cout << "Target: " << targetDivisionName << " " << targetQuarter << endl;
// Linear search the data file for the required name and quarter to find sales amount
double salesOfTarget;
bool isFound = false;
while (!isFound && !dataFile.eof())
{
cout << "Found division data: " << targetDivisionName << " " << targetQuarter << endl;
DIVISION_DATA_S divisionData;
// Read an object from the file, cast as DIVISION_DATA_S
dataFile.read(reinterpret_cast<char *>(&divisionData), sizeof(divisionData));
cout << "Successfully read data for " << targetDivisionName << " " << targetQuarter << endl
<< "Name: " << divisionData.divisionName << ", Q: " << divisionData.quarter << ", "
<< "Sales: " << divisionData.sales << endl;
// Test for a match of both fields
if (divisionData.divisionName == targetDivisionName && divisionData.quarter == targetQuarter)
{
isFound = true;
cout << "Match!" << endl;
salesOfTarget = divisionData.sales;
}
}
if (!isFound) // Error message if record is not found in data file
{
cout << "\nError. Could not find record for " << targetDivisionName
<< " division, quarter " << targetQuarter << endl;
}
else
{
// Display the corresponding record
cout << "Division: " << targetDivisionName << ", Quarter: " << targetQuarter
<< "Sales: " << salesOfTarget << endl;
totalSales += salesOfTarget; // Add current sales to the sales accumulator
numberOfSalesFound++; // Increment total number of sales found
}
}
Sorry for the lack of indent for the while loop, copy/paste kind of messed it up.
My problem appears when attempting to access information read from the binary file. For instance, when it tries to execute the cout statement I added for debugging, it gives me this error:
Unhandled exception at 0x0FED70B6 (msvcp140d.dll) in CorporateSalesData.exe: 0xC0000005: Access violation reading location 0x310A0D68.
Now, from what I have read, it seems that this means something is trying to read from the very low regions of memory, AKA something somewhere has something to do with a null pointer, but I can't imagine how that would appear. This whole read operation is copied exactly from my textbook, and I have no idea what a reinterpret_chast is, let alone how it works or how to fix errors with it. Please help?
EDIT: Thanks for all the help. To avoid complications or using something I don't fully understand, I'm gonna go with switching to a c-string for the divisionName.
dataFile.write(reinterpret_cast<char *>(&divisionData), sizeof(divisionData));
Works only if you have POD types. It does not work when you have a std::string in there. You'll need to use something along the lines of:
// Write the size of the string.
std::string::size_type size = divisionDat.divisionName.size();
dataFile.write(reinterpret_cast<char*>(&size), sizeof(size));
// Now write the string.
dataFile.write(reinterpret_cast<char*>(divisionDat.divisionName.c_str()), size);
// Write the quarter and the sales.
dataFile.write(reinterpret_cast<char*>(&divisionDat.quarter), sizeof(divisionDat.quarter));
dataFile.write(reinterpret_cast<char*>(&divisionDat.sales), sizeof(divisionDat.sales));
Change the read calls to match the write calls.
// Dump struct into binary file
dataFile.write(reinterpret_cast<char *>(&divisionData), sizeof(divisionData));
/*...*/
// Read an object from the file, cast as DIVISION_DATA_S
dataFile.read(reinterpret_cast<char *>(&divisionData), sizeof(divisionData));
This will categorically not work under any circumstances.
std::string uses heap-allocated pointers to store any string data it contains. What you're writing to the file is not the contents of the string, but simply the address where the string's data is located (along with some meta-data). If you arbitrarily read those pointers and treat them as memory (like you are in the cout statement) you'll reference deleted memory.
You have two options.
If all you want is a struct that can be easily serialized, then simply convert it like so:
// Struct to hold division data
struct DIVISION_DATA_S
{
char divisionName[500];
int quarter;
double sales;
};
Of course, with this style, you're limited to interacting with the name as a c-string, and also are limited to 500 characters.
The other option is to properly serialize this object.
// Struct to hold division data
struct DIVISION_DATA_S
{
string divisionName;
int quarter;
double sales;
string serialize() const { //Could also have the signature be std::vector<char>, but this will make writing with it easier.
string output;
std::array<char, 8> size_array;
size_t size_of_string = divisionName.size();
for(char & c : size_array) {
c = size_of_string & 0xFF;
size_of_string >>= 8;
}
output.insert(output.end(), size_array.begin(), size_array.end());
output.insert(output.end(), divisionName.begin(), divisionName.end());
int temp_quarter = quarter;
for(char & c : size_array) {
c = temp_quarter & 0xFF;
temp_quarter >>= 8;
}
output.insert(output.end(), size_array.begin(), size_array.begin() + sizeof(int));
size_t temp_sales = reinterpret_cast<size_t>(sales);
for(char & c : size_array) {
c = temp_sales & 0xFF;
temp_sales >>= 8;
}
output.insert(output.end(), size_array.begin(), size_array.end());
return output;
}
size_t unserialize(const string & input) {
size_t size_of_string = 0;
for(int i = 7; i >= 0; i--) {
size_of_string <<= 8;
size_of_string += unsigned char(input[i]);
}
divisionName = input.substr(7, 7 + size_of_string);
quarter = 0;
for(int i = 10 + size_of_string; i >= 7 + size_of_string; i--) {
quarter <<= 8;
quarter += unsigned char(input[i]);
}
size_t temp_sales = 0;
for(int i = 18 + size_of_string; i >= 11 + size_of_string; i--) {
temp_sales <<= 8;
temp_sales += unsigned char(input[i]);
}
sales = reinterpret_cast<double>(temp_sales);
return 8 + size_of_string + 4 + 8;
}
};
Writing to files is pretty easy:
dataFile << divisionData.serialize();
Reading can be a little harder:
stringstream ss;
ss << dataFile.rdbuf();
string file_data = ss.str();
size_t size = divisionData.unserialize(file_data);
file_data = file_data.substr(size);
size = divisionData.unserialize(file_data);
/*...*/
By the way, I haven't checked my code for syntax or completeness. This example is meant to serve as a reference for the kind of code that you'd need to write to properly serialize/unserialize complex objects. I believe it to be correct, but I wouldn't just throw it in untested.
welcome to the world of serialization. You are trying to 'bit blit' your structure into a file. This only works for very simple types (int, float, char[xxx]) where the data is actually inline. And even when it does work you are stuck with reloading the data into the same type of machine (same word size, same endianness).
What you need to do is serilaze the data and then deserialize it back. You can invent ways of doing that yourself or you can use one on many standards. There are 2 basic types - binary (efficient, not human readable) and text (less efficient but human readable)
text
json
yaml
xml
csv
binary
protobuf
boost has a serialization library http://www.boost.org/doc/libs/1_61_0/libs/serialization/doc/
Also you might like to look here
https://isocpp.org/wiki/faq/serialization
I wrote a function within my code that should create some sort of matrices. It is fine when the size is small, but when it gets bigger, it crashes at the middle of this function without giving any information. I did that with both debug and release mode and same thing happened. Any idea on what can be wrong? Someone suggested me it could be buffer overrun.
In this function when kl.mechelms get bigger than a certain number, it crashes. The following code uses a function and gets a 3 by 3 matrix and stores it in kl.scoff which size is [3][3][kl.mechelms][kl.mechelms]. The problem happens when kl.mechelms are like bigger than 7000, but we need far more than that for our code.
Could the function Calc_3D which I use within this part cause the problem? I think it shouldn't since it just reads some values.
for (int z = 0;z<3;z++) {
for (int q = 0;q<3;q++) {
kl.scofsarr[z][q] = new double *[kl.mechelms];
}
}
for (int i = 0;i<kl.mechelms;i++) {
cout << "element " << i << "of " << kl.mechelms << endl;
kl.comments << "element " << i << "of " << kl.mechelms << endl;
for (int z = 0;z<3;z++) {
for (int q = 0;q<3;q++) {
kl.scofsarr[z][q][i] = new double[kl.mechelms];
}
}
for (int j = 0;j<kl.mechelms;j++) {
Calc_3D(i,j, kl.elmx[j], kl.elmy[j], kl.elmz[j], kl.anglemat[j], kl.dip[j], kl.elmx[i],kl.elmy[i],kl.elmz[i],
kl.elma[i],kl.rectza,kl.anglemat[i],kl.dip[i], kl.G, kl.v, kl.scofs, kl.rdepth);
for (int z = 0;z<3;z++) {
for (int q = 0;q<3;q++) {
kl.scofsarr[z][q][i][j] = kl.scofs[z][q];
}
}
}
}
I basically want to pass this array of data I'm reading to diffent functions and eventually plot it.
The array contains a 32 bit word of '1's and '0's, I then want to add these individual bits together to see where my data spikes. So in other words if I add "0100" to "0110" I get "0210" - which is probably easier done with separate bins and plotting.
At the moment I'm just getting garbage out.
void binary(int convert, int* dat) {
bitset<32> bits(convert);
//cout << bits.to_string() << endl;
char data[32];
for(unsigned i = 0; i < 32; ++i) {
data[i] = bits[i];
}
for(unsigned i = 32; i; --i) {
dat[i] = (int(data[i-1]))+dat[i];
}
}
void SerDi() {
int dat[32];
cout << " Reading data from memory..." << endl;
ValVector< uint32_t> data=hw.getNode("SerDi.RAM").readBlock(8);
hw.dispatch();
cout << data[0]<<endl;
cout << data[1]<<endl;
for (unsigned i = 2; i < 7; i++) {
binary(data[i], dat);
}
cout << dat[7] << endl;
graph(dat); //passes the array to a place where I can plot the graph
}
You have
int dat[32];
But in convert you have i = 32 and dat[i] That will access something outside of the array and bad things will happen.
Also that is not initialized. Add a memset/loop somewhere to make dat initially 0.
Im allocating 25,000,000,000 units of these :
struct test
{
public:
test();
void insert(unsigned int var1, unsigned int var2, unsigned long var3);
bool isEmpty() const;
unsigned char * data;
};
test::test()
{
data = NULL;
}
bool test::isEmpty() const
{
return (data == NULL);
}
unsigned long index = 25000000000;
like this :
test *something = new (nothrow) test[index];
if (!something)
{
cout << "Error allocating memory for [something]" << endl;
return ;
}
cout << "DONE !" << endl;
then i even made sure its data initialized to NULL.
unsigned long j=0;
while (j<index)
{
something[j].data = NULL;
j++;
}
And its all good, except when i iterate through the something[], like this :
test *chk;
unsigned long empty = 0;
unsigned long not_empty = 0;
unsigned long i=0;
for (i=0; i< index; i++ )
{
chk = &something[i];
if (!chk->isEmpty())
{
not_empty++;
} else
empty++;
}
cout << "Empty [" << empty << "]" << endl;
cout << "Not Empty [" << not_empty << "]" << endl;
cout << "Total [" << empty + not_empty << "]" << endl;
(LAST UPDATE)
It turned out to be hardware problem - memory. Some of sticks were not working properly together with others. Thanks everyone for suggestions and too bad hardware path isnt in answers o/
(UPDATE)
I get constant number of initialized elements that are not NULL (not_empty). Still have no idea why.
Of course, empty + not_empty = index.
If i comment out data = NULL; in constructor function, i get proper empty/not_empty numbers if i loop through array right after allocating it. but still, after forcing pointers to NULL, i get same constant not_empty value, after looping through array again.
I also had this reworked to plain pointers, using malloc(), and it made no difference, except that it was first the time i noticed pointers to be correct (empty) until i set them.
I did some tests with smaller [index] value (5,500,000,000), and i could not replicate the behavior.
Im using google`s ltcmalloc and allocated memory size suggests, it is using correct [index] size;
Rather than 25000000000, write 25000000000ul. I have suspect your number becomes a 32-bit integer, thus the j < 25000000000 is always true as int(25000000000) = -769803776.
I'm working on a project that deals with creating two strings, a username and a password. The two elements make an object of an Account. In the main, there is an Array of Accounts that is initialized at 10.
I have a Save & Quit option, which saves the Username on one line and the Password on the next in the same file. A pair of lines signifies another account.
My question is, how do you properly save the data from the Array of Accounts, then load the data from the previous Array of Accounts?
I get a std::bad_alloc memory error every time I try the loadAccounts() function. I've several different methods, but to no avail.
So far I've come up with this for saving the array (works just as it should so far) :
void saveAccounts(Account accs [], int numIndexes)
{
std::ofstream savefile("savedata.sav", std::ofstream::binary); // By re-initializing the file, the old contents are overwritten.
for (int i = 0; i < numIndexes; i++)
{
savefile << accs[i].getUsername() << endl;
savefile << accs[i].getPassword() << endl;
}
savefile.close();
}
As for my loading function I have :
Account* loadAccounts() // Load the data from the file to later print to make sure it works correctly.
{
cout << "LOADING ACCOUNTS!" << endl;
std::ifstream loadfile("savedata.sav", std::ifstream::binary);
Account * acc_arr; // The "Array" to be returned.
Account tmp_arr [10]; // The array to help the returned "Array."
acc_arr = tmp_arr; // Allowing the "Array" to be used and returned because of the actual array.
if (loadfile.is_open())
{
int i = 0;
while (loadfile.good())
{
cout << "Loadfile is good and creating Account " << i+1 << "." << endl; // For my own benefit to make sure the data being read is good and actually entering the loop.
std::string user;
std::getline(loadfile, user);
std::string pass;
std::getline(loadfile, pass);
Account tmpAcc(user, pass);
tmp_arr[i] = tmpAcc;
++i;
}
Account endAcc = Account(); // The default constructor sets Username to "NULL."
tmp_arr[i] = endAcc;
}
loadfile.close();
cout << "ACCOUNTS LOADED SUCCESSFUL!" << endl;
return acc_arr;
}
I've gathered that I can return an array by using a pointer and an actual array to do that same, since an array can't actually be returned.
I try to use the returned array here, which I'm trying to "copy" over the loaded array to the Array that will actually be printed. Later, I'll print the array (acc_arr) to ensure that the loaded array was loaded successfully :
else if (selection == 'l' || selection == 'L')
{
Account * tmp_acc_arr = new Account [10];
tmp_acc_arr = loadAccounts();
_getch();
for (size_t i = 0; i < size_t(10); i++)
{
if (tmp_acc_arr[i].getUsername() == "NULL")
{
break;
}
acc_arr[i] = tmp_acc_arr[i];
cout << "Added Account " << i << " succesfully." << endl;
}
}
The error is caused by this last block of code. I've checked to make sure the data copied correctly by using
EDIT: Awkward... by using an if statement to make sure the data within the tmp_acc_arr actually has data stored once it was returned and initialized in the main.
tmp_arr ist local in loadAccounts and on the stack. It will be invalid once loadAccounts() returns. Your return value is an invalid stack-pointer.
You could hand your pointer tmp_acc_arr to the function as an argument and fill it with the values from your file.
You should also check for overflow or better use STL containers like std::vector.
edit
void loadAccounts(Account * acc_memory, std::allocator<Account> alloc, size_t acc_array_size) // Load the data from the file to later print to make sure it works correctly.
{
Account *end_of_construction = acc_memory;
try
{
cout << "LOADING ACCOUNTS!" << endl;
std::ifstream loadfile("savedata.sav", std::ifstream::binary);
if (loadfile.is_open() && loadfile.good())
{
size_t i = 0;
for (size_t i=0; i<acc_array_size; ++i)
{
if (loadfile.good())
{
cout << "Loadfile is good and creating Account " << i+1 << "." << endl; // For my own benefit to make sure the data being read is good and actually entering the loop.
std::string user, pass;
std::getline(loadfile, user);
if (loadfile.good())
{
std::getline(loadfile, pass);
alloc.construct(end_of_construction++, user, pass);
}
else alloc.construct(end_of_construction++);
}
else alloc.construct(end_of_construction++);
}
}
loadfile.close();
cout << "ACCOUNTS LOADED SUCCESSFUL!" << endl;
}
catch (...)
{
size_t num_constructed = end_of_construction-acc_memory;
for (size_t i=0; i<num_constructed; ++i) alloc.destroy(acc_memory + i);
throw;
}
}
Used like
size_t const num_elements = 10;
std::allocator<Account> acc_alloc;
Account * tmp_acc_arr = acc_alloc.allocate(num_elements);
loadAccounts(tmp_acc_arr, acc_alloc, num_elements);
// do stuff
for (size_t i=0; i<num_elements; ++i) acc_alloc.destroy(tmp_acc_arr + i);
acc_alloc.deallocate(tmp_acc_arr, num_elements);
You return a pointer that points to an array that is destroyed as soon as you exit the function. Using that pointer leads to undefined behavior, that is BAD.
As you observed, array is not a valid return type, so to return it actually you shall put it inside a struct, and return the struct.