A bit of confusion here: I'm trying to do this:
QBuffer _ourLogMessageBuffer;
QByteArray theLogMessage;
...
qDebug() << "Writing " << theLogMessage.size() << " bytes to buffer\n";
qint64 numberOfBytes - _ourLogMessagesBuffer.write(theLogMessage);
qDebug() << "Wrote " << numberOfBytes << " bytes to buffer\n";
qDebug() << "Buffer has " << _ourLogMessagesBuffer.bytesAvailable()
<< " bytes available to read (after write)\n";
This outputs the following:
Writing 196 bytes to buffer
Wrote 196 bytes to buffer
Buffer has 0 bytes available to read (after write)
That last line really confuses me. I thought the return value from the .write() method was supposed to say how many bytes were written? Why would they not be available?
And, later, I attempt the following:
qDebug() << "Buffer has " << _ourLogMessagesBuffer.bytesAvailable()
<< " bytes available to read (before read)\n";
char logMessageBytes[565];
qint64 numberOfBytes = _ourLogMessagesBuffer.read(logMessageBytes, 565);
qDebug() << "Read " << numberOfBytes << " bytes from buffer\n";
Considering the previous bytesAvailable result, the output of these calls aren't too surprising. They output the following:
Buffer has 0 bytes available to read (before read)
Read 0 bytes from buffer
So I feel like I'm missing a step, and that you have to do something between writing and the data being available to read. Perhaps some sort of seek or something? But I seem to be missing where it says that in the documentation.
Any tips would be appreciated. Thank you!
You need to seek back to the position you want to read from:
_ourLogMessagesBuffer.seek(0);
Then you will be able to see an appropriate amount of bytesAvailable. If you think about as a (physical) pointer to a position on a tape, it makes sense. As you write, the pointer moves to the end where it can write more data. Any tape ahead of the pointer is "blank"; there's nothing to read (for a "blank" tape, a new or empty buffer).
When just writing, the position is automatically updated for you. But if you want to read data you already wrote, you need to tell it to go back.
An exception to this is with, say, a file format. If we are modifying an existing file, we could update a fixed-length timestamp in one part, then immediately read a couple bytes denoting the length of an "author" string, and then read that string in. For that, we would not need a seek as all the data is contiguous, and the write and read functions handle moving the position within the file (buffer) automatically.
If you have non-contiguous reads/writes, you need to seek. Otherwise, it can't read your mind on where you want to read from.
Related
I've just noticed something when using QNetworkReply that I was unable to find the slightest hint in the Qt documentation for QIODevice::readAll() (which the QNetworkReply inherits this method from).
Here is what the documentation states:
Reads all remaining data from the device, and returns it as a byte
array.
This function has no way of reporting errors; returning an empty
QByteArray can mean either that no data was currently available for
reading, or that an error occurred.
Let's say I have the following connection:
connect(this->reply, &QIODevice::readyRead, this, &MyApp::readyReadRequest);
Ths readyReadRequest() slot looks like this:
void MyApp::readyReadRequest()
{
LOG(INFO) << "Received data from \"" << this->url.toString() << "\"";
LOG(INFO) << "Data contents:\n" << QString(this->reply->readAll());
this->bufferReply = this->reply->readAll();
}
The surprise came after I called this->bufferReply (which a QByteArray class member of MyApp). I passed it to a QXmlStreamReader and did:
while (!reader.atEnd())
{
LOG(DEBUG) << "Reading next XML element";
reader.readNext();
LOG(DEBUG) << reader.tokenString();
}
if (reader.hasError())
{
LOG(ERROR) << "Encountered error while parsing XML data:" << reader.errorString();
}
Imagine my surprise when I got the following output:
2017-10-17 16:12:18,591 DEBUG [default] [void MyApp::processReply()][...] Reading next XML element
2017-10-17 16:12:18,591 DEBUG [default] [void MyApp::processReply()] [...] Invalid
2017-10-17 16:12:18,591 ERROR [default] Encountered error while parsing XML data: Premature end of document
Through debugging I got that my bufferReply at this point is empty. I looked in the docs again but couldn't find anything that hints removing the data from the device (in my case the network reply) after reading it all.
Removing the line where I print the byte array or simply moving it after this->bufferReply = this->reply->readAll(); and then printing the contents of the class member fixed the issue:
void MyApp::readyReadRequest()
{
LOG(INFO) << "Received data from \"" << this->url.toString() << "\"";
this->bufferReply = this->reply->readAll();
LOG(INFO) << "Data contents:\n" << QString(this->bufferReply);
}
However I would like to know if I'm doing something wrong or is the documentation indeed incomplete.
Since readAll() doesn't report errors or that data is not available at the given point in time returning an empty byte array is the only thing that hints towards the fact that something didn't work as intended.
Yes. When you call QIODevice::readAll() 2 times, it is normal that the 2nd time you get nothing. Everything has been read, there is nothing more to be read.
This behavior is standard in IO read functions: each call to a read() function returns the next piece of data. Since readAll() reads to the end, further calls return nothing.
However, this does not necessarily means that the data has been flushed. For instance when you read a file, it just moves a "cursor" around and you can go back to the start of the file with QIODevice::seek(0). For QNetworkReply, I'd guess that the data is just discarded.
The following code works on bidirectional streams and finds the record id from file and then replaces contents for that record from the file. But before overwriting the content, it shifts the put pointer to the position of the get pointer. Through tellp()and tellg() it is found that they both were already at the same position before shifting. But on removing the seekp() line the code does not overwrite the data.
Contents in data.txt:
123 408-555-0394
124 415-555-3422
263 585-555-3490
100 650-555-3434
Code:
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main()
{
int inID = 263;
const string& inNewNumber = "777-666-3333";
fstream ioData("data.txt");
// Loop until the end of file
while (ioData.good()) {
int id;
string number;
// Read the next ID.
ioData >> id;
// Check to see if the current record is the one being changed.
if (id == inID) {
cout << "get pointer position " << ioData.tellg() << endl; //Displays 39
cout << "put pointer position " << ioData.tellp() << endl; //Displays 39
ioData.seekp(ioData.tellg()); //Commenting this line stops code from working
ioData << " " << inNewNumber;
break;
}
// Read the current number to advance the stream.
ioData >> number;
}
return 0;
}
Question:
What is the need of using seekp() to shift the position of the put pointer if it is already there, as the get and put pointers move together?
The question linked by #Revolver_Ocelot in the comments gives relevant information. The most important part is that you have to either flush or seek between read and write access. I therefore modified your code in the following way:
if (id == inID) {
cout << "get pointer position " << ioData.tellg() << endl; //Displays 39
cout << "put pointer position " << ioData.tellp() << endl; //Displays 39
ioData.flush();
cout << "get pointer position " << ioData.tellg() << endl;
cout << "put pointer position " << ioData.tellp() << endl;
ioData.seekp(ioData.tellg()); //Commenting this line stops code from working
ioData << " " << inNewNumber;
break;
}
This gives the following interesting output:
get pointer position 39
put pointer position 39
get pointer position 72
put pointer position 72
(Calling flush() doesn't actually resolve the problem. I just added it to your code in order to show you that it modifies the file pointer.)
My assumption on your original code is the following: If you write to your file after reading from it first, without calling seekp() in between, then the file pointer gets modified by the write command before the data is actually written to the file. I assume that the write command performs some kind of flushing and that this modifies the file pointer in a similar way as the flush() command that I added to your code.
When I ran the code above on my PC, the flush() command moved the file pointer to position 72. If we remove the seekp() command from your original code, I think that the write command will also move the file pointer to position 72 (or maybe another invalid position) before actually writing to the file. In this case writing fails, because position 72 is behind the end of the file.
Consequently, ioData.seekp(ioData.tellg()); is needed to ensure that the file pointer is set to the correct file position, because it can change when you switch between reading from and writing to your file without calling seekp().
The last paragraph of this answer gives some similar explanation.
It is because it's a rule of c++ bidirectional streams that if someone wants to shift from input operation to output operation. Then one must use seek() function to make such shift.
This functionality is borrowed from the core of c language as whenever someone uses a bidirectional stream then programmer may be working with two different buffers in which one buffer may be for input and another for output. Now synchronizing both the buffers would be a performance inefficient solution. As most of the time programmer may not need to use both the input and output functionality and program would be maintaining both the buffers for the programmer for no good reason.
So as an alternative to this, another solution was implemented to let programmer explicitly perform the flushing and other management by invoking seek() function.
Which means that seek() function that we often use does not simply repositions the file pointer but also updates the buffers and stream also.
See also
why fseek or fflush is always required between reading and writing in the read/write "+" modes
When I read in information via fstream, it has ocurred twice in two different programs, that the input given to my program isn't stable, even if a given file doesn't change.
In my most recent program, which is concerned with audio-reading. I'm doing a simple check on the first four letters in the file. These letters are supposed to be RIFF, which they also are - I checked.
So, in order to check the format of a given binary file, I buffer the first four letters and see if they are equal to 'RIFF'.
char buffer[4];
std::ifstream in(fn,std::ios::binary);
in.read(buffer,4);
if(buffer!="RIFF"){//Always wrong atm
std::cout << "INVALID WAV FILE: " << buffer << std::endl;
}
When I first made the program, I recall this working properly. Now though, I get an error via my own cout:
INVALID WAV FILE: RIFFýfK
Does anyone have any idea as to what has gone wrong? Perhaps a way to make fstream more consistent?
You're reading 4 characters but not adding a zero terminator, furthermore your comparison is wrong since you're not comparing strings equality, you should rather do:
char buffer[5];
std::ifstream in(fn, std::ios::binary);
in.read(buffer, 4);
buffer[4] = '\0'; // Add a zero-terminator at the end
if (strcmp(buffer,"RIFF")) { // If buffer isn't {'R','I','F','F','\0'}..
std::cout << "INVALID WAV FILE: " << buffer << std::endl;
}
this is my first Post as I came along with using the search function so far. But now I wasted a whole day on the following issue:
I recorded a 12bit (written as 16bit) grayscale video and wrote it directly into a binary stream-file (no headers or such).
Now the task is to read the file and output every frame as a 16bit pgm.
The following snipped illustrates what i tried. The output is a valid pgm, but whith "white noise on it".
...
imageBufferShort = new short[imageWidth*imageHeight* sizeof(short)];
...
streamFileHandle.read(reinterpret_cast<char*>(imageBufferShort),2*imageWidth*imageHeight); //double amount because 8bit chars!
// As .read only takes chars, I thought, that I just read the double amount of char-bytes and when it is interpreted as short (=16bit) everything is ok?!?
...now the pgm output:
std::ofstream f_hnd(fileName,std::ios_base::out |std::ios_base::binary |std::ios_base::trunc);
// write simple header
f_hnd.write("P5\n",3);
f_hnd << imageWidth << " " << imageHeight << "\n4095\n"; //4095 should tell the pgm to use 2 bytes for each pixel
f_hnd.write(reinterpret_cast<char*>(imageBufferShort),2*imageWidth*imageHeight);
f_hnd.close();
Again, the file is produced and viewable but contains rubbish. Is the initial guess ok? read 2 "chars" and handle them as one "short"? I also trieb a whitespace after every line but this changes nothing, so I decided to post this shorter code.
Thanks for any help!
As #Domi and #JoeZ pointed out: Your endianness is probably screwed up. Meaning, the order of your bytes is wrong.
To fix your problem, you will have to iterate over every pixel and swap it's bytes before writing it back to file.
Problem Solved. Thank you all very much. The Endianess was indeed the Problem. The solution is given below:
f_hnd << "P5" << " " << imDimensions.GetWidth() << " " << imDimensions.GetHeight() << " " << "4095\n";
// convert imageBufferShort to Big-Endian format
unsigned short imageBufferShortBigEndian[imDimensions.GetWidth()*imDimensions.GetHeight()];
for (int k=0 ; k<imDimensions.GetWidth()*imDimensions.GetHeight() ; k++)
{
imageBufferShortBigEndian[k] = ( (imageBufferShort[k] << 8) | (imageBufferShort[k] >> 8) );
}
f_hnd.write(reinterpret_cast<char*>(imageBufferShortBigEndian),2*imDimensions.GetWidth()*imDimensions.GetHeight());
f_hnd.close();
imageBufferShort has also to an unsigned short-array. If signed types are used, the bitshift-conversion gets slightly more difficult.
Thanks again!
I am learning C++ and I found something that I can't understand:
Output buffers can be explicitly flushed to force the buffer to be
written. By default, reading cin flushes cout; cout is also flushed
when the program ends normally.
So flushing the buffer (for example an output buffer): does this clear the buffer by deleting everything in it or does it clear the buffer by outputting everything in it? Or does flushing the buffer mean something completely different?
Consider writing to a file. This is an expensive operation. If in your code you write one byte at a time, then each write of a byte is going to be very costly. So a common way to improve performance is to store the data that you are writing in a temporary buffer. Only when there is a lot of data is the buffer written to the file. By postponing the writes, and writing a large block in one go, performance is improved.
With this in mind, flushing the buffer is the act of transferring the data from the buffer to the file.
Does this clear the buffer by deleting everything in it or does it clear the buffer by outputting everything in it?
The latter.
You've quoted the answer:
Output buffers can be explicitly flushed to force the buffer to be written.
That is, you may need to "flush" the output to cause it to be written to the underlying stream (which may be a file, or in the examples listed, a terminal).
Generally, stdout/cout is line-buffered: the output doesn't get sent to the OS until you write a newline or explicitly flush the buffer. The advantage is that something like std::cout << "Mouse moved (" << p.x << ", " << p.y << ")" << endl causes only one write to the underlying "file" instead of six, which is much better for performance. The disadvantage is that a code like:
for (int i = 0; i < 5; i++) {
std::cout << ".";
sleep(1); // or something similar
}
std::cout << "\n";
will output ..... at once (for exact sleep implementation, see this question). In such cases, you will want an additional << std::flush to ensure that the output gets displayed.
Reading cin flushes cout so you don't need an explicit flush to do this:
std::string colour;
std::cout << "Enter your favourite colour: ";
std::cin >> colour;
Clear the buffer by outputting everything.