In a first time i want to thanks HostileFork to help me to explain my problem.
Thanks you !
i'm trying to build a client and a server who send their data through a binary protocol.
my problem is i want to send a class from a QT client to a Boost Server. My header(one integer who is the size of my class) is writting on the socket. When i want to read the header on the server side, i can't get the good integer(instead of that i have an big number like -13050660). I think that the problem come to the deserialization on the server but i am not sure.
This is the technique that my Qt client code uses to write the number 10 to onto a socket:
QByteArray paquet;
QDataStream out(&paquet, QIODevice::WriteOnly);
out << (quint32) 0;
out.device()->seek(0);
out << (quint32) (10);
cout << "Writing " << sizeof(quint32) << " bytes to socket." << endl;
Then I try to read it on a server process, which uses boost's async_read():
this->Iheader.resize(size, '\0'); // Iheader is a vector of char
async_read(
this->socket,
buffer(this->Iheader),
bind(
&Client::endRead,
cli,
placeholders::error,
placeholders::bytes_transferred)
);
Here's the function that operates on the string result:
#ifdef WIN32
#define MYINT INT32
#include <Windows.h>
#else
#define MYINT int
#endif
void Client::endRead(const error_code& error, size_t nbytes)
{
if (!error && nbytes == sizeof(MYINT)) {
cout << "Read " << sizeof(MYINT) << " bytes from a socket." << endl;
istringstream stream(this->connection->getIheader(nbytes));
stream >> this->Isize;
cout << "Integer value read was " << this->Isize << endl;
} else {
cout << "Could not read " << sizeof(MYINT) << " bytes." << endl;
}
}
I do get the 32-bit signed integer (4 bytes), but it is not ten, instead it is something like -1163005939. Anyone have and ideas why this is not working?
The server and the client are both on launching on Windows7 pro, 64-bit.
You're welcome...and thanks for following my suggestions on editing the question, and doing the requisite effort to pinpoint the problem more clearly. So now I can tell you what's wrong. :)
The behavior of the << and >> are different on QDataStream than on C++ standard IOstreams. In the world of classes like std::stringstream these operators are called "inserters"/"extractors" and are intended for dealing with information formatted as text. If you want to read a certain number of bytes into a memory address, what you'll want is:
http://www.cplusplus.com/reference/iostream/istream/read/
(Note that if you wish to read binary data out of something that is not a stringstream, you need to be using ios::binary to keep it from messing with line ending conversions)
QDataStream doesn't follow that convention...it's a good helper for binary data. Nothing wrong with that...since abstractly speaking the << and >> operators are available in the language to be overloaded to do whatever you want within your own class hierarchies. Qt was free to define its own semantics for its own streams, and they did.
Do heed the advice given by #vitakot about (if possible) using the same methodology for both input and output. Also heed my warning about byte-ordering issues that start to come up if you aren't careful.
(Good news is that if you are using QDataStream it finesses this issue by taking care of it for you.)
Be aware that in your code as written, your stringstream is making a copy of the buffer in order to read from it. I'm not experienced with boost::asio or the best practices of async_read, but I'm sure there are better ways you might dig around and find.
HostileFork is right, from the information we have it is not possible to isolate a bug in your code.
However, I would suggest you to use boost serialization in your Qt client as well. There is no reason to not combine Boost and Qt libraries. Otherwise you will have to deal with a lot of troubles when sending more complicated classes over the network...
Related
I think I’m a kind of at a loss here. I trying such a simple thing, I can’t believe that there is nothing build-in Qt (using Qt 5.6.2). I try to convert the data inside a QByteArray from big endian to little endian. Always starts with the same test QByteArray like this.
QByteArray value;
value.append(0x01);
value.append(0x02);
value.append(0x03);
qDebug() << "Original value is: " << value.toHex(); // “010203” like expected
What I need is the little endian, which means the output should be “030201”. Is there any build in thing so far in the Qt Framework? I don’t find one. What I tried so far
// Try with build in QtEndian stuff
QByteArray test = qToLittleEndian(value);
qDebug() << "Test value is: " << test.toHex(); // 010203
// Try via QDataStream
QByteArray data;
QDataStream out(&data, QIODevice::ReadWrite);
out.setByteOrder(QDataStream::LittleEndian);
out << value;
qDebug() << "Changed value is: " << data.toHex(); // "03000000010203"
Any good idea? Or do I really need to shift hands by hand? Found nothing helpfull on SO or on google or maybe ask the wrong question...
It sounds like you want to reverse the array, rather than manipulate the endianness of any of the multi-byte types inside the array. The standard library has a solution for this:
QByteArray arr;
std::reverse(arr.begin(), arr.end());
// arr is now in reversed order
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 6 years ago.
Improve this question
I am trying to add data into a text file. However, i see for some reason it produces garbage data. I also notice, it will input the correct data once, but then it follow with garbage data.
void TextFileLogger::log(std::string msg){
using namespace std;
//ofstream output_file("students.data", ios::binary);
std::ofstream logFile;
// creating, opening and writing/appending data to a file
char filename[] = "log.txt";
logFile.open(filename, ios::binary | ios::app |ios::out);
if (logFile.fail())
{
std::cout << "The " << filename << " file could not be created/opened!" << std::endl;
// 0-normal, non zero - some errors
}
else
{
if (!logFile.write((char*)&msg, sizeof(msg)))
{
cout << "Could not write file" << endl;
}
else
{
streamsize bytesWritten = logFile.tellp();
if (bytesWritten != sizeof(msg))
{
cout << "Could not write expected number of bytes" << endl;
}
else
{
logFile << msg << std::endl;
cout << "file written OK" << endl;
}
}
}
}
That one is fun!
(char*)&msg does not do what you expect: std::string is mainly a pointer to a dynamically-allocated buffer which contains the actual data. When you take the std::string's address and try to read what's inside, you get a view of its innards, not its data. Using a C++ static_cast here would have spared you the trouble by telling you that the conversion makes no sense. sizeof(msg) similarly returns the size of the std::string, not the length of its data.
So, your solution is: use msg.data() and msg.size(), it's exactly what they're designed for.
But... why would it (sometimes) output your string, and a bunch of garbage? Well, std::strings typically use SSO (Small String Optimization). The std::string actually contains a small buffer, to store short enough strings without dynamic allocation. When you inspect the whole std::string object, you see this buffer pass by.
You are writing the contents of the whole std::string object, with all the member variables that it contains internally.
You either want:
logFile << msg;
or if you really want to use write():
logFile.write( msg.c_str(), msg.length());
And, I wonder: Why do create/open the file in binary mode, when you write strings afterwards?
And finally, you write the data twice, the second time in your last else clause.
The problem is with this line:
if (!logFile.write((char*)&msg, sizeof(msg)))
It should be this:
if (!logFile.write(msg.c_str(), msg.length()))
Since you are passing a std::string into the function, you should take advantage of the functions it provides (c_str() and length()) instead of trying to cast it to a char* (this always gets messy, plus you are casting away the const, which is also typically bad).
This:
if (!logFile.write((char*)&msg, sizeof(msg)))
is wrong in so many ways. msg is not an array of char, it's a std::string - lying to the compiler by using a cast is always a bad thing to do. And the size of a string is not the size of the characters it contains. Why the heck are you not using the obvios:
logfile << msg << std::endl;
Replace sizeof(msg) with msg.size(), sizeof() is not doing what you think!
Also (char*)&msg does not do whatever you think, use msg.data() instead.
logFile.write((char*)&msg, sizeof(msg));
should be rewritten to:
logFile.write(msg.data(), msg.size());
or, even better, because ofstream overrides operator<< for std::string:
logfile << msg;
I have been having a very hard time writing to a binary file and reading back. I am basically writing records of this format
1234|ABCD|efgh|IJKL|ABC
Before writing this record, I would write the length of this entire record ( using string.size()) and then I write the record to the binary file using ofstream as follows:
int size;
ofstream studentfile;
studentfile.open( filename.c_str(),ios::out|ios::binary );
studentfile.write((char*)&size,sizeof(int));
studentfile.write(data.c_str(),(data.size()*(sizeof(char))));
cout << "Added " << data << " to " << filename << endl;
studentfile.close();
And I read this data at some other place
ifstream ifile11;
int x;
std::string y;
ifile11.open("student.db", ios::in |ios::binary);
ifile11.read((char*)&x,sizeof(int));
ifile11.read((char*)&y,x);
cout << "X " << x << " Y " << y << endl;
first I read the length of the record into the variable x, and then read the record into string y. The problem is, the output shows x as being '0' and 'y' is empty.
I am not able figure this out. Someone who can look into this problem and provide some insight will be thanked very much.
Thank you
You can't read a string that way, as a std::string is really only a pointer and a size member. (Try doing std::string s; sizeof(s), the size will be constant no matter what you set the string to.)
Instead read it into a temporary buffer, and then convert that buffer into a string:
int length;
ifile11.read(reinterpret_cast<char*>(&length), sizeof(length));
char* temp_buffer = new char[length];
ifile11.read(temp_buffer, length);
std::string str(temp_buffer, length);
delete [] temp_buffer;
I know I am answering my own question, but I strictly feel this information is going to help everyone. For most part, Joachim's answer is correct and works. However, there are two main issues behind my problem :
1. The Dev-C++ compiler was having a hard time reading binary files.
2. Not passing strings properly while writing to the binary file, and also reading from the file. For the reading part, Joachim's answer fixed it all.
The Dev-C++ IDE didn't help me. It wrongly read data from the binary file, and it did it without me even making use of a temp_buffer. Visual C++ 2010 Express has correctly identified this error, and threw run-time exceptions and kept me from being misled.
As soon as I took all my code into a new VC++ project, it appropriately provided me with error messages, so that I could fix it all.
So, please do not use Dev-C++ unless you want to run into real troubles like thiis. Also, when trying to read strings, Joachim's answer would be the ideal way.
This question already has answers here:
How can I compose output streams, so output goes multiple places at once?
(3 answers)
Closed 4 months ago.
I have to send the same string (e.g. a log message) to multiple streams.
Which of the following solutions is the most efficient?
Rebuild the same string for each stream and send it to the stream itself.
outstr1 << "abc" << 123 << 1.23 << "def" << endl;
outstr2 << "abc" << 123 << 1.23 << "def" << endl;
outstr3 << "abc" << 123 << 1.23 << "def" << endl;
Build the string once with string's operators, and send it to all the streams.
std::string str = "abc" + std::to_string(123) + std::to_string(1.23) + "def";
outstr1 << str;
outstr2 << str;
outstr3 << str;
Build the string once with a stream, and send it to all the streams:
std::stringstream sstm;
sstm << "abc" << 123 << 1.23 << "def" << endl;
std::string str = sstm.str();
outstr1 << str;
outstr2 << str;
outstr3 << str;
Some or all of these output streams could be on a RAM disk.
Any other ways to do the same thing?
I would use a tee output stream. Do something like (pseudocode):
allstreams = tee(outstr1, outstr2, outstr3);
allstreams << "abc" << 123 << 1.23 << "def" << endl;
There doesn't seem to be anything in the standard c++ library to do this, but Boost has one.
See also the answers to How can I compose output streams, so output goes multiple places at once?
Although it is unlikely that you would see much difference either way1, option #3 sounds the most plausible: unlike the first option, it does not convert ints to strings multiple times; unlike the second option, it does not allocate and delete multiple string objects for its intermediate results2. It also looks cleanest from the readability point of view: no code is duplicated, and output looks like an output, not like a concatenation.
1 Insert a mandatory disclaimer about optimization before profiling being evil here.
2 Small String Optimization may help on systems where it is supported (thanks, Prætorian), but the constructor and destructor calls for the intermediate objects are not going away.
The "proper" way to do something like this is to have a stream buffer writing to multiple destinations and use this stream buffer via a std::ostream. This way the code looks as if it writing just once but the characters are sent multiple times. Searching for "teebuf Dietmar" will find a few variations on the same theme.
To also comment on your question: Which one of the three alternatives is the fastest depends on the exact expressions you have:
needs to evaluate the involved expressions and perform the conversions three times. Depending on what you actually do this may still be fairly fast.
actually creates and destroys multiple streams and does multiple allocations for std::string. I'd expect this to be slowest.
still creates a stream (which should actually be a std::ostringstream) and allocates some memory. Out of your options I'd expect it to be fastest.
Using a teebuf is probably fastest, at least, when it does some buffering but uses only fixed suze arrays, both for the buffer and the array of stream buffer pointers. Note, that you'll need to override sync() to deal with the buffer in a timely manner, though.
To determine the actual performance you'll need to measure!
I try to send a class' size from a Qt client to a Boost server (I made both).
This is the class I want to serialize
class Commande
{
public:
std::string login;
std::string mdp;
std::string IP;
std::string to;
std::string from;
bool rep;
int nbCmd;
};
This is the function I use to serialize and send the Commande size and the object.
_socket is a QTcpSocket
void BNetwork::sendData(void)
{
QByteArray paquet;
QDataStream out(&paquet, QIODevice::WriteOnly);
Commande cmd;
cmd.setCmd(1);
cmd.setFrom("Paris");
cmd.setIP("127.0.0.1");
cmd.setLogin("test1");
cmd.setMdp("mdp1");
cmd.setRep(0);
cmd.setTo("maryline");
out << (quint32) 0;
out << cmd;
out.device()->seek(0);
out << (paquet.size() - (int)sizeof(quint32));
this->_socket.write(paquet);
}
QDataStream &operator<<(QDataStream &out, Commande &cmd)
{
QString test(cmd.from.c_str());
out << (quint32)cmd.nbCmd;
out << test;
test = cmd.IP.c_str();
out << test;
test = cmd.mdp.c_str();
out << test;
out << (quint32)cmd.rep;
test = cmd.to.c_str();
out << test;
return out;
}
This is the function I use to convert the header size I received from the QT client.
std::string save = this->connection->getIheader(); \\ this is the string i read on the socket
std::istringstream stream(save);
std::cout << save << std::endl;
if (!(stream >> std::dec >> this->Isize))
throw my_exception("error in endRead()");
For an unknown reason, save contains "1000", and when I try to convert this string into an integer, it's not working so I think the value I get isn't correct.
The client and the server are running on windows 7 64bits.
Do you have a solution to my problem?
First, confirm that when you use out.device()->seek(0), you cause the next writes to prepend as you expect, rather than overwriting the data you've already written, as I expect.
Then, consult this answer to a question that very likely has the same or similar problem.
Let me know in a comment if you need more help.
ETA:QStringstores data as 16 bitQChar, in order to support unicode. std::stringandstd::istringstreamare reading 8 bitchar. See also this answer. The QChars serialized by QT are likely to cause you trouble down the line. Note also that any 0 (null) char that happens to be in the char sequence returned by getIheader() will terminate std::string's assignment operator.
I recommend replacing thestd::istringstreamwith aQDataStream, and using that class to read out your data to exactly the types you originally wrote. You can cast from QT types to c++ native types after you've put serialization behind you. Otherwise, you'll have to take lots of care to figure out what QT is doing under the hood & match it by your own effort.