I'm having hard time understanding how to work with 0MZ properly. When I'm trying to send a message initialized with size larger than 29, something goes wrong. My code is very simple:
zmq::context_t context (1);
zmq::socket_t req(context,ZMQ_REQ);
req.connect("tcp://localhost:6969");
int msgSize = 100;
zmq::message_t test(msgSize);
snprintf((char*)test.data(),msgSize,"short message");
cout << static_cast<char*>(test.data())<< endl; // this is always fine - 'short message'
so far so good, but after sending this message, if msgSize>29, i can't get the same result again
req.send(test);
cout << static_cast<char*>(test.data())<< endl; // now it's gibberish, like '&?+#'
what's even more puzzling, if my server receives the message it also looks like '&?+#' there, but if it is sending it back simply with PUB socket, I can read it again in my client:
zmq::message_t reply;
req.recv(&reply);
cout << static_cast<char*>(test.data())<< endl; - 'my message' again!
I understand, that there is some 29 bytes limit on short messages, but how can i get around it, without dealing with multipart messages? I literally need like 40 chars....
If the message is > 30 bytes, the memory once occupied by 'test', but then freed, must be being reused by the reply data (obviously by serendipity). Thus, when you look at 'test' again it magically appears to be what you think it should be. This theory should be very simple for you to verify in the debugger by looking at addresses.
Whatever, as Hristo said, sending a message frees it's original contents and shouldn't be used again.
ZeroMQ has an optimisation for small messages where the payload doesn't need to be separately allocated. Again, the fact that you can still see the contents you expect after sending a message is just an artefact; you cannot rely on it.
If you have a requirement to retain the contents of messages after they're sent, take a look at zmq_send_const(), which is new with ZMQ 4.0. I don't know if any bindings make use of it.
As it turned out, I had an error generating piece of code within my server app, after receiving a message I did instant ping pong style reply, like:
zmq::message_t msg(msgSize);
REC.recv(&msg);
//pong
REC.send(msg);
And as above answer points out, sending a message frees it's original contents, leaving me with unwanted gibberish of random bytes.
Related
I have a UDP client that is sending messages to a server, at a specified rate. The rate needs to be constant, so I decided to try to do my receiving of replies in a separate thread to avoid blocking or delaying on recvfrom(). Is it at all possible to 'wait' for a full message before receiving? What would be the best strategy to go about doing this?
while (true)
{
//std::this_thread::sleep_for(std::chrono::milliseconds(5000));
if (recvfrom(threadSock, ReceiveBuf, BufLength, 0, 0, 0) == SOCKET_ERROR)
{
printf("Thread Receive failed with error %ld\n", GetLastError());
break;
}
else
{
printf("Reply received: %s\n\n", ReceiveBuf);
}
memset(ReceiveBuf, '\0', BufLength);
}
Above is my receiving code. Currently, only the first 8 characters of a reply are being read into the buffer (the buffer is 512 bytes).
How can I wait for a full message (bearing in mind the message lengths are variable).
Is this even possible? Perhaps there is a better approach.
Thanks in advance.
EDIT: I should clarify the prints are for testing only. They won't be in the final result, as printing from a thread gives weird inline prints.
According to MSDN:
The recvfrom function receives a datagram and stores the source address.
For message-oriented sockets, data is extracted from the first enqueued message, up to the size of the buffer specified. If the datagram or message is larger than the buffer specified, the buffer is filled with the first part of the datagram, and recvfrom generates the error WSAEMSGSIZE. For unreliable protocols (for example, UDP) the excess data is lost. For UDP if the packet received contains no data (empty), the return value from the recvfrom function function is zero.
Thus, you can't receive a part of the incoming message, the receive returns only when the OS can process and return an enqueued datagram.
In the interest of completeness, and the small chance anyone suffering from similar confusion finds this, solution follows:
Yes, it was a silly question, I should've realised recvfrom waits for a full datagram. The problem was with my server.
It was an issue of the server not sending the full data. I'm not sure as to the exact cause, but to fix it I converted the char* my reply was being stored to (and printing correctly) to a char[], which, when sent, worked fine.
I have some code where in, there is a TCP connection between the client and the server. I need to send some xml data to the server and receive a response from it. I am trying to do it like this:
char request[MAX];
sprintf(request, "<attestationRequest><majorVersion>%d</majorVersion><minorVersion>%d</minorVersion></attestationRequest>", major, minor);
write(sockfd,request, length);
while(recv(sockfd, response, MAX, 0) >= 0)
{
cout << "response " << response;
//do something;
}
But the data received is (expected response + request).
If I try to fill the data using QString, the data received is proper.
QString request = QString("<attestationRequest>"
"<majorVersion>%1</majorVersion>"
"<minorVersion>%2</minorVersion>"
"</attestationRequest>")
.arg(major)
.arg(minor)
The rest of the code is same for both the cases. I receive more than 11000 bytes of data when I use sprintf, but 9000 bytes in case of QString. I am not able to understand if I am missing something.
IMO both solution are bad.
It is safer and better to use QXmlStreamWriter. Manual manipulation on text will in most cases end with incorrect encoding or missing escape sequence or other xml format violation.
Other problem is that you didn't gave any data how you perform communication. It is highly probable that there is an error causing your problem.
Most likely your problem is, that UTF-16 data is sent through the connection (that is QString.)
Maybe if you tried the w_char-versions of those commands, it might succeed.
I am trying to send a large amount of data around 50KByte or above over a TCP socket using the following command in C++:
boost::asio::async_write(sock, boost::asio::buffer(sbuff, slen),
boost::bind((&send_handler), placeholders::error));
Where sbuff is a pointer to the data to be transmitted, and slen is the length of the data.
Sometimes the operation successes and sometimes I get an error with Operation cancelled
Here is the code part for the receiver, waiting for a specific amount of the data to be received.
boost::asio::async_read(_sock,
boost::asio::buffer(rbuf, rlen),
boost::bind(&session::handle_read_payload,
this,
placeholders::bytes_transferred,
placeholders::error));
void session::handle_read_payload(buffer<uint8> &buff, size_t rbytes, const boost::system::error_code &e)
Where rlen is the number of the bytes to wait to receive. And rbuf is a pointer to where I store the received bytes.
I checked the flow of the TCP packets between the two machines using Wireshark and I found that suddenly the receiver sends back a packet with FIN flag set to the sender, which terminates the connection.
So can anyone tell me what might be the root of the problem? IS there any problem with my code?
Does it matter if I call _acceptor.listen(); before async_accept. Because when I tested without _acceptor.listen(); , it works perfectly. So what would be the difference?
From the discussion in the comments to the question, it sounds very much like there is a disagreement between the sender and the receiver about the size of the message being sent.
The receiver receives what it thinks is a complete message then closes the socket while the sender still thinks there is more data that the receiver has not accepted.
To diagnose the problem, I suggest that you display slen on the sender side, and display rlen on the receiver side before issuing the respective read/write requests (by display I mean write to a log or to std::cerr or whatever other approach works for your application.) If the two numbers are not equal you know where to look for the underlying cause of the problem. If they are equal -- then more investigation will be needed.
Now the server (implemented with java) will send some stream data to me, my code is like below:
connect(socket, SIGNAL(readyRead()), this, SLOT(read_from_server()));
in the read_from_server():
{
while (socket->bytesAvailable())
{
QString temp = socket->readAll();
}
}
but I find that even the server sent me a string with only several characters, the data is truncated, and my function is called twice, thus temp is the never complete data that I want.
If server send me a longer string, my function may be called three or more times, making me diffficult to know at which time the data is transfered completely.
So anyone can tell me how to completely receive the data easily, without so many steps of bothering? I'm sorry if this is duplicated with some questions else, I couldn't get their answers work for me. Many thanks!
What you're seeing is normal for client-server communication. Data is sent in packets and the readyRead signal is informing your program that there is data available, but has no concept of what or how much data there is, so you have to handle this.
To read the data correctly, you will need a buffer, as mentioned by #ratchetfreak, to append the bytes as they're read from the stream. It is important that you know the format of the data being sent, in order to know when you have a complete message. I have previously used at least two methods to do this: -
1) Ensure that sent messages begin with the size, in bytes, of the message being sent. On receiving data, you start by reading the size and keep appending to your buffer until it totals the size to expect.
2) Send all data in a known format, such as JSON or XML, which can be checked for the end of the message. For example, in the case of JSON, all packets will begin with an opening brace '{' and end with a closing brace '}', so you could count braces and match up the data, or use QJsonDocument::fromRawData to verify that the data is complete.
Having used both of these methods, I recommend using the first; include the size of a message that is being sent.
you can use a buffer field to hold the unfinished data temporarily and handle packets as they complete:
{
while (socket->bytesAvailable())
{
buffer.append(socket->readAll());
int packetSize = getPacketSize(buffer);
while(packetSize>0)
{
handlePacket(buffer.left(packetSize);
buffer.remove(0,packetSize);
packetSize = getPacketSize(buffer);
}
}
}
If all of the data has not yet arrived then your while loop will exit prematurely. You need to use a message format that will let the receiving code determine when the complete message has been received. For example, the message could begin with a length element, or if you are dealing with text the message could end with some character used as a terminator.
Problem is that during tcp data transfer data are send in undefined chunks. If you are trying to read defined block size you have to know in advance expected chunk size ore have a way to determinate when your block ends (something like zero terminated c-string).
Check if this answer doesn't help you (there is a trick to wait for expected data block).
I'd like to make a chatting program using win socket in c/c++. (I am totally newbie.)
The first question is about how to check if the client receives packets from server.
For instance, a server sends "aaaa" to a client.
And if the client doesn't receive packet "aaaa", the server should re-send the packet again.(I think). However, I don't know how to check it out.
Here is my thought blow.
First case.
Server --- "aaaa" ---> Client.
Server will be checking a sort of time waiting confirm msg from the client.
Client --- "I received it" ---> Server.
Server won't re-send the packet.
The other case.
Server --- "aaaa" ---> Client.
Server is waiting for client msg until time out
Server --- "aaaa" ---> Client again.
But these are probably inappropriate.
Look at second case. Server is waiting a msg from client for a while.
And if time's out, server will re-send a packet again.
In this case, client might receive the packet twice.
Second question is how to send unlimited size packet.
A book says packet should have a type, size, and msg.
Following it, I can only send msg with the certain size.
But i want to send msg like 1Mbytes or more.(unlimited)
How to do that?
Anyone have any good link or explain correct logic to me as easy as possible.
Thanks.
Use TCP. Think "messages" at the application level, not packets.
TCP already handles network-level packet data, error checking & resending lost packets. It presents this to the application as a "stream" of bytes, but without necessarily guaranteed delivery (since either end can be forcibly disconnected).
So at the application level, you need to handle Message Receipts & buffering -- with a re-connecting client able to request previous messages, which they hadn't (yet) correctly received.
Here are some data structures:
class or struct Message {
int type; // const MESSAGE.
int messageNumber; // sequentially incrementing.
int size; // 4 bytes, probably signed; allows up to 2GB data.
byte[] data;
}
class or struct Receipt {
int type; // const RECEIPT.
int messageNumber; // last #, successfully received.
}
You may also want a Connect/ Hello and perhaps a Disconnect/ Goodbye handshake.
class Connect {
int type; // const CONNECT.
int lastReceivedMsgNo; // last #, successfully received.
// plus, who they are?
short nameLen;
char[] name;
}
etc.
If you can be really simple & don't need to buffer/ re-send messages to re-connecting clients, it's even simpler.
You could also adopt a "uniform message structure" which had TYPE and SIZE (4-byte int) as the first two fields of every message or handshake. This might help standardize your routines for handling these, at the expense of some redundancy (eg in 'name' field-sizes).
For first part, have a look over TCP.
It provides a ordered and reliable packet transfer. Plus you can have lot of customizations in it by implementing it yourself using UDP.
Broadly, what it does is,
Server:
1. Numbers each packet and sends it
2. Waits for acknowledge of a specific packet number. And then re-transmits the lost packets.
Client:
1. Receives a packet and maintains a buffer (sliding window)
2. It keeps on collecting packets in buffer until the buffer overflows or a wrong sequenced packet arrives. As soon as it happens, the packets with right sequence are 'delivered', and the sequence number of last correct packet is send with acknowledgement.
For second part:
I would use HTTP for it.
With some modifications. Like you should have some very unique indicator to tell client that transmission is complete now, etc