Consider this scenario.
I'm creating a struct(flatbuffers::Table) using CreateXXX generated code. This creates the struct on the FlatBuffer buffer and gives me the offset.
Then I can get the memory block with GetBufferPointer() and transfer it.
Reversely, if I get a memory block, I can use GetXXX to get my struct(flatbuffers::Table) from that.
But after I get this struct, if I need to serialize it again, how can I do it? After the serialization, I should be able to transfer that data and do GetXXX on that data like before.
flatbuffers::Parser parser;
parser.Parse(schema.c_str());
parser.SetRootType("license");
parser.Parse(j.c_str());
auto* buf = parser.builder_.GetBufferPointer();
auto li = flatbuffers::GetRoot<license>(buf);
std::cout << "ID: " << li->id()->c_str() << " Rand: " << li->rand()->c_str() << " Secret: " << li->secret()->c_str() << std::endl;
uint8_t* buf2 = ????????????
auto li2 = flatbuffers::GetRoot<license>(buf2);
std::cout << "ID: " << li2->id()->c_str() << " Rand: " << li2->rand()->c_str() << " Secret: " << li2->secret()->c_str() << std::endl;
The obvious answer is that you keep the original buffer pointer (and size) around. Then you can "re-serialize" it by just writing out the existing buffer.
There is a function GetBufferStartFromRootPointer if you really must use just the root (li in your example).
// Update the server status xml
string filelocation ("/var/www/html/index.xml");
string firstline ("<server>\n");
string secondline ("\t<current>" + msg.getCount() + "</current>\n");
string thirdline ("\t<highest>" + "--" + "</highest>\n");
string fourthline ("\t<status>Online</status>\n")
string finalline ("</server>");
fstream file;
file.open(filelocation);
file.write(firstline + secondline + thirdline + fourthline + finalline);
string updateFlush ("Server Status updated.");
printf("%s\n", updateFlush);
file.close();
Note that msg.getCount() is a function in the same file to get player count from the central server.
Gives out errors about an operands const char*. Something to do with + or -
Thanks
Take a look at the line
string secondline ("\t<current>" + msg.getCount() + "</current>\n");
"\t<current>" is a const char *
msg.getCount() looks like an int or size_t
</current>\n again is a const char *
Adding a const char * to an int or size_t creates a new const char * pointing to a different address.
The same happens in the line
string thirdline ("\t<highest>" + "--" + "</highest>\n");
Here you are adding pointers together. The result is a pointer pointing to a more or less random address.
And in these two lines:
string updateFlush ("Server Status updated.");
printf("%s\n", updateFlush);
You are creating a C++ string-object and trying to print it using a C print function with a format string that requires a char *.
You are mixing C and C++ or stream based I/O with conventional I/O.
In current C++ you should do it this way:
string filelocation ("/var/www/html/index.xml");
fstream file;
file.open(filelocation);
file
<< "<server>\n"
<< "\t<current>" << msg.getCount() << "</current>\n"
<< "\t<highest>" << "--" << "</highest>\n"
<< "\t<status>Online</status>\n"
<< "</server>";
string updateFlush ("Server Status updated.");
cout << updateFlush << std::endl;
file.close();
Or even more readable:
auto file = std::ofstream("/var/www/html/index.xml");
file
<< "<server>" << std::endl
<< "\t<current>" << msg.getCount() << "</current>" << std::endl
<< "\t<highest>" << "--" << "</highest>" << std::endl
<< "\t<status>Online</status>" << std::endl
<< "</server>";
file.close();
std::cout << "Server status updated." << std::endl;
If operating with streams use std::endl to output a newline. It outputs the correct newline for the operation system (CRLF or LF or whatever) and it flushes the stream.
To use std::cout you have to include <iostream> and for std::ofstream include <fstream>.
If you like it short, you could even do this:
std::ofstream("/var/www/html/index.xml")
<< "<server>" << std::endl
<< "\t<current>" << msg.getCount() << "</current>" << std::endl
<< "\t<highest>" << "--" << "</highest>" << std::endl
<< "\t<status>Online</status>" << std::endl
<< "</server>";
std::cout << "Server status updated." << std::endl;
I'm trying to write a program that reads in an OpenGL shader from a .txt file. I've actually already done this a few days ago, this was the code I used:
char vShaderData[2000];
char fShaderData[2000];
void readShaders() {
std::ifstream vShaderF;
std::ifstream fShaderF;
vShaderF.open("shaders//vertexShader.txt");
fShaderF.open("shaders//fragShader.txt");
if (vShaderF.is_open() && fShaderF.is_open()) std::cout << m << "Shader read success" << std::endl;
else std::cout << "Shader read fail" << std::endl;
std::cout << m << "vertex shader: " << std::endl;
vShaderF.read(vShaderData, 2000);
for (int i = 0; i < 2000; i++) {
std::cout << vShaderData[i];
}
std::cout << std::endl << std::endl;
std::cout << m << "frag shader: " << std::endl;
fShaderF.read(fShaderData, 2000);
for (int i = 0; i < 2000; i++) {
std::cout << fShaderData[i];
}
std::cout << std::endl;
vShaderF.close();
fShaderF.close();
}
This worked great. my shader file was not actually not 2000 in length, but the read() call seemed to store the extra characters as whitespace into the char array which is what I wanted.
Now having restructured my code a little bit in a newer program, my reader now looks like this:
std::ifstream shaderFile;
shaderFile.open(path);
if (shaderFile.is_open()) cout << "Shader at: " << path << ", initalized" << endl;
char data[2000];
shaderFile.read(data, 2000);
for (int i = 0; i < 2000; i++) std::cout << data[i];
The actual text portion still reads correct. However, now the extra space in the char array is stored with this instead of whitespace:
In case the image won't show, it is basically just a reapeating pattern of these two characters [|[|[|....
Why is this happening and how can I fix it?
NOTE: I'm using the same shader file, same computer, same IDE, same everything. The old one still works.
When using std::istream:read() it will not set the parts of the buffer to spaces which were not read. The memory will be left untouched. If you want to get spaces into an unread area of the buffer, you'll need to put the spaces there yourself. If the program indeed had spaces in the buffer it was because the buffer somehow already contained spaces by chance.
You can use std::istream::gcount() to determine how many characters were read.
If you want the arrays to contain predefined data, you'll have to initialize it with such predefined data. If the stream reads fewer data than the array size, you will have the padding you want.
The project I'm working on, as a custom file format consisting of the header of a few different variables, followed by the pixel data. My colleagues have developed a GUI, where processing, writing reading and displaying this type of file format works fine.
But my problem is, while I have assisted in writing the code for writing data to disk, I cannot myself read this kind of file and get satisfactorily values back. I am able to read the first variable back (char array) but not the following value(s).
So the file format matches the following structure:
typedef struct {
char hxtLabel[8];
u64 hxtVersion;
int motorPositions[9];
int filePrefixLength;
char filePrefix[100];
..
} HxtBuffer;
In the code, I create an object of the above structure and then set these example values:
setLabel("MY_LABEL");
setFormatVersion(3);
setMotorPosition( 2109, 5438, 8767, 1234, 1022, 1033, 1044, 1055, 1066);
setFilePrefixLength(7);
setFilePrefix( string("prefix_"));
setDataTimeStamp( string("000000_000000"));
My code for opening the file:
// Open data file, binary mode, reading
ifstream datFile(aFileName.c_str(), ios::in | ios::binary);
if (!datFile.is_open()) {
cout << "readFile() ERROR: Failed to open file " << aFileName << endl;
return false;
}
// How large is the file?
datFile.seekg(0, datFile.end);
int length = datFile.tellg();
datFile.seekg(0, datFile.beg);
cout << "readFile() file " << setw(70) << aFileName << " is: " << setw(15) << length << " long\n";
// Allocate memory for buffer:
char * buffer = new char[length];
// Read data as one block:
datFile.read(buffer, length);
datFile.close();
/// Looking at the start of the buffer, I should be seeing "MY_LABEL"?
cout << "buffer: " << buffer << " " << *(buffer) << endl;
int* mSSX = reinterpret_cast<int*>(*(buffer+8));
int* mSSY = reinterpret_cast<int*>(&buffer+9);
int* mSSZ = reinterpret_cast<int*>(&buffer+10);
int* mSSROT = reinterpret_cast<int*>(&buffer+11);
int* mTimer = reinterpret_cast<int*>(&buffer+12);
int* mGALX = reinterpret_cast<int*>(&buffer+13);
int* mGALY = reinterpret_cast<int*>(&buffer+14);
int* mGALZ = reinterpret_cast<int*>(&buffer+15);
int* mGALROT = reinterpret_cast<int*>(&buffer+16);
int* filePrefixLength = reinterpret_cast<int*>(&buffer+17);
std::string filePrefix; std::string dataTimeStamp;
// Read file prefix character by character into stringstream object
std::stringstream ss;
char* cPointer = (char *)(buffer+18);
int k;
for(k = 0; k < *filePrefixLength; k++)
{
//read string
char c;
c = *cPointer;
ss << c;
cPointer++;
}
filePrefix = ss.str();
// Read timestamp character by character into stringstream object
std::stringstream timeStampStream;
/// Need not increment cPointer, already pointing # 1st char of timeStamp
for (int l= 0; l < 13; l++)
{
char c;
c = * cPointer;
timeStampStream << c;
}
dataTimeStamp = timeStampStream.str();
cout << 25 << endl;
cout << " mSSX: " << mSSX << " mSSY: " << mSSY << " mSSZ: " << mSSZ;
cout << " mSSROT: " << mSSROT << " mTimer: " << mTimer << " mGALX: " << mGALX;
cout << " mGALY: " << mGALY << " mGALZ: " << mGALZ << " mGALROT: " << mGALROT;
Finally, what I see is here below. I added the 25 just to double check that not everything was coming out in hexadecimal. As you can see, I am able to see the label "MY_LABEL" as expected. But the 9 motorPositions all come out looking suspiciously like addresses are not values. The file prefix and the data timestamp (which should be strings, or at least characters), are just empty.
buffer: MY_LABEL M
25
mSSX: 0000000000000003 mSSY: 00000000001BF618 mSSZ: 00000000001BF620 mSSROT: 00000000001BF628 mTimer: 00000000001BF630 mGALX: 00000000001BF638 mGALY: 00000000001BF640 mGALZ: 00000000001BF648 mGALROT: 00000000001BF650filePrefix: dataTimeStamp:
I'm sure the solution can't be too complicated, but I reached a stage where I had this just spinning and I cannot make sense of things.
Many thanks for reading this somewhat long post.
-- Edit--
I might hit the maximum length allowed for a post, but just in case I thought I shall post the code that generates the data that I'm trying to read back:
bool writePixelOutput(string aOutputPixelFileName) {
// Write pixel histograms out to binary file
ofstream pixelFile;
pixelFile.open(aOutputPixelFileName.c_str(), ios::binary | ios::out | ios::trunc);
if (!pixelFile.is_open()) {
LOG(gLogConfig, logERROR) << "Failed to open output file " << aOutputPixelFileName;
return false;
}
// Write binary file header
string label("MY_LABEL");
pixelFile.write(label.c_str(), label.length());
pixelFile.write((const char*)&mFormatVersion, sizeof(u64));
// Include File Prefix/Motor Positions/Data Time Stamp - if format version > 1
if (mFormatVersion > 1)
{
pixelFile.write((const char*)&mSSX, sizeof(mSSX));
pixelFile.write((const char*)&mSSY, sizeof(mSSY));
pixelFile.write((const char*)&mSSZ, sizeof(mSSZ));
pixelFile.write((const char*)&mSSROT, sizeof(mSSROT));
pixelFile.write((const char*)&mTimer, sizeof(mTimer));
pixelFile.write((const char*)&mGALX, sizeof(mGALX));
pixelFile.write((const char*)&mGALY, sizeof(mGALY));
pixelFile.write((const char*)&mGALZ, sizeof(mGALZ));
pixelFile.write((const char*)&mGALROT, sizeof(mGALROT));
// Determine length of mFilePrefix string
int filePrefixSize = (int)mFilePrefix.size();
// Write prefix length, followed by prefix itself
pixelFile.write((const char*)&filePrefixSize, sizeof(filePrefixSize));
size_t prefixLen = 0;
if (mFormatVersion == 2) prefixLen = mFilePrefix.size();
else prefixLen = 100;
pixelFile.write(mFilePrefix.c_str(), prefixLen);
pixelFile.write(mDataTimeStamp.c_str(), mDataTimeStamp.size());
}
// Continue writing header information that is common to both format versions
pixelFile.write((const char*)&mRows, sizeof(mRows));
pixelFile.write((const char*)&mCols, sizeof(mCols));
pixelFile.write((const char*)&mHistoBins, sizeof(mHistoBins));
// Write the actual data - taken out for briefy sake
// ..
pixelFile.close();
LOG(gLogConfig, logINFO) << "Written output histogram binary file " << aOutputPixelFileName;
return true;
}
-- Edit 2 (11:32 09/12/2015) --
Thank you for all the help, I'm closer to solving the issue now. Going with the answer from muelleth, I try:
/// Read into char buffer
char * buffer = new char[length];
datFile.read(buffer, length);// length determined by ifstream.seekg()
/// Let's try HxtBuffer
HxtBuffer *input = new HxtBuffer;
cout << "sizeof HxtBuffer: " << sizeof *input << endl;
memcpy(input, buffer, length);
I can then display the different struct variables:
qDebug() << "Slice BUFFER label " << QString::fromStdString(input->hxtLabel);
qDebug() << "Slice BUFFER version " << QString::number(input->hxtVersion);
qDebug() << "Slice BUFFER hxtPrefixLength " << QString::number(input->filePrefixLength);
for (int i = 0; i < 9; i++)
{
qDebug() << i << QString::number(input->motorPositions[i]);
}
qDebug() << "Slice BUFFER filePrefix " << QString::fromStdString(input->filePrefix);
qDebug() << "Slice BUFFER dataTimeStamp " << QString::fromStdString(input->dataTimeStamp);
qDebug() << "Slice BUFFER nRows " << QString::number(input->nRows);
qDebug() << "Slice BUFFER nCols " << QString::number(input->nCols);
qDebug() << "Slice BUFFER nBins " << QString::number(input->nBins);
The output is then mostly as expected:
Slice BUFFER label "MY_LABEL"
Slice BUFFER version "3"
Slice BUFFER hxtPrefixLength "2"
0 "2109"
1 "5438"
...
7 "1055"
8 "1066"
Slice BUFFER filePrefix "-1"
Slice BUFFER dataTimeStamp "000000_000000P"
Slice BUFFER nRows "20480"
Slice BUFFER nCols "256000"
Slice BUFFER nBins "0"
EXCEPT, dataTimeStamp, which is 13 chars long, displays instead 14 chars. The 3 variables that follow: nRows, nCols and nBins are then incorrect. (Should be nRows=80, nCols=80, nBins=1000). My guess is that the bits belonging to the 14th char of dataTimeStamp should be read along with nRows, and so cascade on to produce the correct nCols and nBins.
I have separately verified (not shown here) using qDebug that what I'm writing into the file, really are the values I expect, and their individual sizes.
I personally would try to read exactly the number of bytes your struct is from the file, i.e. something like
int length = sizeof(HxtBuffer);
and then simply use memcpy to assign a local structure from the read buffer:
HxtBuffer input;
memcpy(&input, buffer, length);
You can then access your data e.g. like:
std::cout << "Data: " << input.hxtLabel << std::endl;
Why do you read to buffer, instead of using the structure for reading?
HxtBuffer data;
datFile.read(reinterpret_cast<char *>(&data), sizeof data);
if(datFile && datFile.gcount()!=sizeof data)
throw io_exception();
// Can use data.
If you want to read to a chracter buffer, than your way of getting the data is just wrong. You probably want to do something like this.
char *buf_offset=buffer+8+sizeof(u64); // Skip label (8 chars) and version (int64)
int mSSX = *reinterpret_cast<int*>(buf_offset);
buf_offset+=sizeof(int);
int mSSY = *reinterpret_cast<int*>(buf_offset);
buf_offset+=sizeof(int);
int mSSZ = *reinterpret_cast<int*>(buf_offset);
/* etc. */
Or, a little better (provided you don't change the contents of the buffer).
int *ptr_motors=reinterpret_cast<int *>(buffer+8+sizeof(u64));
int &mSSX = ptr_motors[0];
int &mSSY = ptr_motors[1];
int &mSSZ = ptr_motors[2];
/* etc. */
Notice that I don't declare mSSX, mSSY etc. as pointers. Your code was printing them as addresses because you told the compiler that they were addresses (pointers).
I have a function that iterates through a const char * and uses the character to add objects to an instance of std::map if it is one of series of recognized characters.
#define CHARSEQ const char*
void compile(CHARSEQ s) throw (BFCompilationError)
{
std::cout << "#Receive call " << s << std::endl;
for(int i = 0; s[i] != '\0'; i++)
{
if (std::string("<>-+.,[]").find_first_of(s[i]) == std::string::npos)
{
throw BFCompilationError("Unknown operator",*s,i);
}
std::cout << "#Compiling: " << s[i] << std::endl;
std::cout << "#address s " << (void*)s << std::endl;
std::cout << "#var s " << s << std::endl;
controlstack.top().push_back(opmap[s[i]]);
}
}
The character sequence passed is "++++++++++."
For the first three iterations, the print statements display the expected values of '+', '+', and '+', and the value of s continues to be "+++++++++++.". However, on the fourth iteration, s becomes mangled, producing bizarre values such as 'Ð', 'öê', 'cR ', 'œk' and many other character sequences. If the line that throws the exception is removed and the loop is allowed to continue, the value of s does not change after again.
Other functions have access to s but since this is not a multithreaded program I don't see why that would matter. I am not so much confused about why s is changing but why it only changes on the fourth iteration.
I have searched SO and the only post that seems at all relevant is this one but it still doesn't answer my question. (Research has been difficult because searching "const char* changing value" or similar terms just comes up with hundreds of posts about what part of is is const).
Lastly, I know I should probably be using std::string, which I will if no answers come forth, but I would still like to understand this behavior.
EDIT:
Here is the code that calls this function.
CHARSEQ text = load(s);
std::cout << "#Receive load " << text << std::endl;
try
{
compile(text);
}
catch(BFCompilationError& err)
{
std::cerr << "\nError in bf code: caught BFCompilationError #" << err.getIndex() << " in file " << s << ":\n";
std::cerr << text << '\n';
for(int i = 0; i < err.getIndex(); i++)
{
std::cerr << " ";
}
std::cerr << "^\n";
std::cerr << err.what() << err.getProblemChar() << std::endl;
return 1;
}
Where load is:
CHARSEQ load(CHARSEQ fname)
{
std::ifstream infile (fname);
std::string data(""), line;
if (infile.is_open())
{
while(infile.good())
{
std::getline(infile,line);
std::cout << "#loading: "<< line << '\n';
data += line;
}
infile.close();
}
else
{
std::cerr << "Error: unable to open file: " << fname << std::endl;
}
return std::trim(data).c_str();
}
and the file fname is ++++++++++. spread such that there is one character per line.
EDIT 2:
Here is an example of console output:
#loading: +
#loading: +
#loading: +
#loading: +
#loading: +
#loading: +
#loading: +
#loading: +
#loading: +
#loading: +
#loading: .
#Receive load ++++++++++.
#Receive call ++++++++++.
#Compiling: +
#address s 0x7513e4
#var s ++++++++++.
#Compiling: +
#address s 0x7513e4
#var s ++++++++++.
#Compiling: +
#address s 0x7513e4
#var s ++++++++++.
#Compiling:
#address s 0x7513e4
#var s ßu
Error in bf code: caught BFCompilationError #4 in file bf_src/Hello.txt:
ßu
^
Unknown operatorß
Your load function is flawed. The const char* pointer returned by c_str() is valid only until the underlying std::string object exists. But data is a local variable in load and is cleared after return. Its buffer is not overwritten by zeroes but left as it were as free memory. Therefore printing out the value immediately after returning is likely to work but your program may put new values there and the value pointed by your pointer will change.
I suggest to use std::string as the return value of load as a workaround.