I have a problem concerning boost asio libraries. I successfully tried to create a socket between a client and a server, this involves creation of resolvers in order to specify ip and port to the server (the server only requires port) and other objects, but, most importantly, it is necessary to use write and read_some as functions to read and write from/in the socket.
I would really appreciate to use a stream, and this is possible in boost asio, but that's strange...
In almost all examples using streams, to create a server it is necessary to provide port, ok, let's talk about the client... client side, it is necessary to use the iostream constructor to specify coordinates for connecting the stream, here's the code:
tcp::iostream() s(argv[1], "daytime");
Well, I don't really understand what is passed in the first parameter and really don't know what daytime might ever represent...
Basically, here, I'm telling: "Hey stream, you must connect to this server..." but how can I specify ip and port of that server?
Note that, on the opposite, everything is almost clear server side:
boost::asio::io_service io_s;
tcp::acceptor acc(io_s, tcp::endpoint(tcp::v4(), 1950));
for (;;) {
tcp::iostream stream;
acc.accept(*stream.rdbuf());
stream << "Message" << std::endl;
}
Using this model, I would like to use
stream << mymessage_to_send << std::endl;
stream >> a_string_containing_my_message;
in order to send and receive.
How can I do this?
Thank you very much.
The boost asio sample code you quoted:
tcp::iostream s(argv[1], "daytime");
uses "daytime" as a lookup into the services table (usually in /etc/services on a linux system), which would identify that the port for the daytime service is 13.
If you want to connect to a port that is not one of the well known services, you can do so with something like:
tcp::iostream s("localhost", "57002");
Note that the port number is supplied as a string, not as an unsigned short integer as one might be tempted to try.
Of course, "localhost" can be replaced with an IP address "127.0.0.1"
Let's solve all 3 issues here:
Creating the iostream around the socket client side.
This is really simple:
boost::asio::ip::tcp::iostream socketStream;
socketStream.connect( hostname, std::to_string( port ) );
You have to check the state of the stream to see if it connected successfully.
Creating the iostream around the socket server side.
Assuming you have your acceptor object and it is bound and listening..
boost::asio::ip::tcp::iostream connectionSocketStream; // from the connection object
acceptor.accept( *connectionSocketStream.rdbuf() );
// or
acceptor.async_accept( *connectionSocketStream.rdbuf(), callback );
where callback is a function that takes an error code.
Streaming the objects
Now for the streaming itself and here your issue is that when you stream out the string "Message" the client side will need to know where this message begins and ends, and the regular iostream won't write anything to specify this. This is a flaw in iostream itself really.
The answer therefore is to use a boost archive, and you can use a text or binary archive as long as you use the same both ends. It even doesn't matter if one side is using 64-bit big-endian and the other side 32-bit little endian or any other mix.
Using binary archive you would send a message this way:
boost::archive::binary_oarchive oarch( socketStream, boost::archive::no_header );
oarch << "Message";
Remember to flush the stream (socketStream, not oarch) when you have completed sending all you wish to send at this point.
and receive a message
boost::archive::binary_iarchive iarch( socketStream, boost::archive::no_header );
iarch >> message;
You would potentially create one archive and use it throughout, especially for outbound. For inbound you may have issues if you get a streaming error as it will break your archive.
You can use a text archive instead of a binary one.
The boost archive will automatically put in header information so it knows when an object is complete and will only return to you once it has a complete object or something has broken.
Note: primitive types, eg std::string and even vector< int > etc. are automatically handled by an archive. Your own classes will need special overloads as to how to stream them. You should read boost::archive documentation.
Note: You can connect the archive object to the stream before the stream has been opened. The archive works around the streambuf object which does not change dependent on the stream opening successfully.
Creating without no_header would be an issue though as the archives immediately try to use the stream on construction (to read or write their header)
I've written a client/server system using Boost.Asio. The source is available on GitHub: Client.cpp and Server.cpp. Using Boost.Serialization together with Boost.Asio allows me to send arbitrary datastructures over the wire. I must say it is quite impressive!
Related
first of all a little background on my situation:
- Qt/C++ UI desktop application
- embedded device (Stm32l4xx family) +ATWINC1500 wifi module
I'm developing the gui application in order to send commands and files to the emdedded device via sockets.
For simple commands I've done all successfully, but for sending files (text files in GCODE format) I am stuck with some issues.
The embedded device has already a socket management(not written by me, so I have not the possibility to modify the way sockets are managed, coming from third party company), and the reception of that type of files is managed in a way that the API waits for every single line of the file being sent, and then wrotes it into a reserved portion of the flash.
My problem is that when I send file from qt Application(by reading each line and and calling write() on the line, in reality my socket sends an entire chunk of the file, like 50 lines, resulting in my device not managing the file reception.
My sending code is this:
void sendGCODE(const QString fileName)
{
QFile *file = new QFile(fileName,this);
bool result = true;
if (file->open(QIODevice::ReadOnly))
{
while (!file->atEnd())
{
QByteArray bytes(file->readLine());
result = communicationSocket->write(bytes);
communicationSocket->flush();
if(result)
{
console->append("-> GCODE line sent:"+ QString(bytes));
}
else
{
console->append("-> Error sending GCODE line!");
}
}
file->close();
}
}
Have anyone of you guys any hints on what I am doing wrong?
I've already searched and someone suggests on other topic that for this purpose it should be better to use UDP instead of TCP sockets, but unfortunately I cannot touch the embedded-device-side code.
thank you all!
EDIT
After suggestions from comments, I've sniffed tcp packets and the packets are sent correctly(i.e. each packet contains a single line). BUT... at the receiver(device), I understood that there is something regarding memory which is not well managed. an example:
sender sends the line "G1 X470.492 Y599.623 F1000" ; receiver receives correctly the string "G1 X470.492 Y599.623 F1000"
next, if the line length is less than the previous sent, i.e. sending "G1 Z5", the receiver receives: "G1 Z5\n\n.492 Y599.623 F1000", so it is clear that the buffer used to store the data packet is not re-initialized from previous packet content, and the new part overwrites the previous values where the remaining part is from the previous packet
I'm trying to figure out how I could reset that part of memory.
This is all wrong. TCP is not a message-oriented protocol. There is no way to ensure that the TCP packets contain any particular amount of data. The receiver code on the device mustn't expect that either - you perhaps misunderstood the receiver's code, or are otherwise doing something wrong (or the vendor is). What the receiver must do is wait for a packet, add the packet's data to a buffer, then extract and process as many complete lines as it can, then move the remaining data to the beginning of the buffer. And repeat that on every packet.
Thus you're looking for the wrong problem at the wrong place, unless your device never ever had a chance of working. If that device works OK with other software, then your "packetized" TCP assumption doesn't hold any water.
Here's how to proceed:
If the device is commercially available and has been tested to work, then you're looking in the wrong place.
If the device is a new product and still in development, then someone somewhere did something particularly stupid and you either need to fix that stupidity, or have the vendor fix it, or hire a consultant to fix it. But just to be completely clear: that's not how TCP works, and you cannot just accept that "it's how it is".
First of all, I know there are several other threads on the same theme, but I was unable to find anything in those that could help me so I'll try to be very specific with my situation.
I have set up a simple UDP Client / UDP Server pair that is responsible to send data between several parallel simulations. That is, every instance of the simulator is running in a separate thread and send data on a UDP socket. In the master thread the server is running and routes the messages between the simulations.
The (for this problem) important parts of the server code looks like this:
UDPServer::UDPServer(boost::asio::io_service &m_io_service) :
m_socket(m_io_service, udp::endpoint(udp::v4(), PORT_NUMBER)),
m_endpoint(boost::asio::ip::address::from_string("127.0.0.1"), PORT_NUMBER)
{
this->start_receive();
};
void UDPServer::start_receive() {
// Set SO_REUSABLE to true
boost::asio::socket_base::reuse_address option(true);
this->m_socket.set_option(option);
// Specify what happens when a message is received (it should call the handle_receive function)
this->m_socket.async_receive_from( boost::asio::buffer(this->recv_buffer),
this->m_endpoint,
boost::bind(&UDPServer::handle_receive, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred));
};
This works fine on my windows workstation.
The thing is; I want to be able to run this on a linux cluster, which is why I compiled it and tried to run it on a cluster node. The code compiled without a hitch, but when I try to run it I get the error
bind: address already in use
I use a port number above 1024, and have verified that it is not in use by another program. And as is seen above, I also set the reuse_address option, so I really don't know what else could be wrong.
To portably use SO_REUSEADDR you need to set the option before binding the socket to the wildcard address:
UDPServer::UDPServer(boost::asio::io_service &m_io_service) :
m_socket(m_io_service, udp::v4()),
m_endpoint()
{
boost::asio::socket_base::reuse_address option(true);
this->m_socket.set_option(option);
this->m_socket.bind(udp::endpoint(udp::v4(), PORT_NUMBER));
this->start_receive();
}
In your original code, the constructor that takes an endpoint constructs, opens and binds the socket in a single line - it's concise but not very flexible. Here we're constructing and opening the socket in the constructor call, and then binding it later after we set the option.
As an aside, there's not much point initialising m_endpoint if you're just going to use it as the out argument of async_receive_from anyway.
Try running the following command on Linux to see if the port is already being used by another program.
netstat -antup | grep 1024
If you are getting "address already in use" then it is definitely being used by some other program. If the above command yields some result, then kill the process id that is reported in the command. If this does not work, try changing the port number to some other arbitrary port and check if the problem persists.
I have a UDP server using the following code:
void initialize()
{
connect(&_udpSocket, SIGNAL(readyRead()), this, SLOT(onUdpDatagram()));
_udpSocket.bind(QHostAddress::Any, 28283);
}
void onUdpDatagram()
{
qDebug() << "udp packet received!";
_udpSocket.write("Hello");
}
Unfortunately when a UDP packet is received, I have the following error in the log:
QIODevice::write: device not open
How can I make the UDP socket writable? I tried to create another socket for the answer that connect to the sender address and port but the sending won't use the 28283 port anymore...
Any idea?
For info: I'm using Qt 5.2.1 on MacOS 10.9
UDP is not a connection-based protocol. You don't get a separate socket for each peer, instead there's one socket for all communication on a single port.
Therefore, there's some extra effort needed to reply to an incoming UDP packet. You need to retrieve the sender address from the datagram you received, and send back to that same address. In the sockets API this is done by using recvfrom and sendto functions instead of recv (or read) and send (or write) -- the latter are designed for connected sockets like you use with TCP.
You didn't show the declaration (really, the type) for your _udpSocket variable, so I'm assuming that you are using a QUdpSocket. In that case, it looks like you will want to use the readDatagram and writeDatagram functions, which like recvfrom and sendto, have an additional parameter for the peer address (actually, it's a pair, one for the IP address, one for the port).
Here's what the Qt documentation says about that:
The most common way to use this class is to bind to an address and port using bind(), then call writeDatagram() and readDatagram() to transfer data. If you want to use the standard QIODevice functions read(), readLine(), write(), etc., you must first connect the socket directly to a peer by calling connectToHost().
Coincidentally, this warning was introduced by me in Qt upstream:
QIODevice::write: device not open
It should be pretty clear unlike before the introduction of this, namely: you have forgotten to connect to the host with your udp socket. You cannot expect it to write and/or read if it is not even open and/or connected. See the documentation for details:
If you want to use the standard QIODevice functions read(), readLine(), write(), etc., you must first connect the socket directly to a peer by calling connectToHost().
You have to do something like this somewhere in your code:
_udpSocket.connectToHost(myHostAddress, 28283, ReadWrite, AnyIPProtocol);
The last two parameters can be skipped as they are the default. As you can read from the documentation, this method call will open the socket for you, too, which is necessary to get done for QIODevice read and write operations.
That being said, you really should not neglect error checking in your code as it currently seems to stand. It will be difficult to find the issues this way.
Also, it is ice on the cake, but I would encourage you to start using the "new" signal-slot syntax, which is not so new, but much more modern and handier:
void initialize()
{
connect(&_udpSocket, &QUdpSocket::connected, [&_udpSocket]() {
connect(&_udpSocket, &QUdpSocket::readyRead, [&_udpSocket]() {
qDebug() << "udp packet received!";
if (_udpSocket.write("Hello") != 6)
qDebug() << "Failed to write:" << _udpSocket.errorString();
});
});
connect(&_udpSocket, &QUdpSocket::error, [&_udpSocket]() {
qDebug() << "Error occured:" << _udpSocket.errorString();
});
_udpSocket.connectToHost(myHostAddress, 28283, ReadWrite, AnyIPProtocol);
}
I have just started using the Poco library. I am having issues getting two computers to communicate using Poco's DatagramSocket objects. Specifically, the receiveBytes function does not seem to return (despite running Wireshark and seeing that the UDP packets I am sending ARE arriving at the destination machine). I assume I am omitting something simple and this is all due to a dumb mistake on my part. I have compiled Poco 1.4.3p1 on Windows 7 using Visual Studio Express 2010. Below are code snippets showing how I am trying to use Poco. Any advise would be appreciated.
Sending
#include "Poco\Net\DatagramSocket.h"
#include "Serializer.h" //A library used for serializing data
int main()
{
Poco::Net::SocketAddress remoteAddr("192.168.1.140", 5678); //The IP address of the remote (receiving) machine
Poco::Net::DatagramSocket mSock; //We make our socket (its not connected currently)
mSock.connect(remoteAddr); //Sends/Receives are restricted to the inputted IPAddress and port
unsigned char float_bytes[4];
FloatToBin(1234.5678, float_bytes); //Serializing the float and storing it in float_bytes
mSock.sendBytes((void*)float_bytes, 4); //Bytes AWAY!
return 0;
}
Receiving (where I am having issues)
#include "Poco\Net\DatagramSocket.h"
#include "Poco\Net\SocketAddress.h"
#include "Serializer.h"
#include <iostream>
int main()
{
Poco::Net::SocketAddress remoteAddr("192.168.1.116", 5678); //The IP address of the remote (sending) machine
Poco::Net::DatagramSocket mSock; //We make our socket (its not connected currently)
mSock.connect(remoteAddr); //Sends/Receives are restricted to the inputted IPAddress and port
//Now lets try to get some datas
std::cout << "Waiting for float" << std::endl;
unsigned char float_bytes[4];
mSock.receiveBytes((void*)float_bytes, 4); //The code is stuck here waiting for a packet. It never returns...
//Finally, lets convert it to a float and print to the screen
float net_float;
BinToFloat(float_bytes, &net_float); //Converting the binary data to a float and storing it in net_float
std::cout << net_float << std::endl;
system("PAUSE");
return 0;
}
Thank you for your time.
The POCO sockets are modeled on the Berkeley sockets. You should read a basic tutorial on the Berkeley socket API, this will make it easier to understand the POCO OOP socket abstractions.
You cannot connect() on both client and server. You connect() on the client only. With UDP, connect() is optional, and can be skipped (then you have to use sendTo() instead of SendBytes()).
On the server, either you bind() on the wildcard IP address (meaning: will then receive on all the available network interfaces on the host), or to a specific IP address (meaning: will then receive only on that IP address).
Looking at your receiver/server code, it seems you want to filter on the address of the remote client. You cannot do it with connect(), you have to read with receiveFrom(buffer, length, address) and then filter yourself on "address".
Security-wise, be careful with the assumptions you make with the source address of the UDP packets you receive. Spoofing a UDP packet is trivial. Said in another way: do not make authentication or authorization decisions based on an IP address (or on anything not secured by proper cryptography).
The POCO presentation http://pocoproject.org/slides/200-Network.pdf explains, with code snippets, how to do network programming with POCO. See slides 15, 16 for DatagramSocket. Note that on slide 15 there is a typo, replace msg.data(), msg.size() with syslogMsg.data(), syslogMsg.size() to compile :-)
Have a look also at the "poco/net/samples" directory for short examples that show also the best practices in using POCO.
I am trying to send C++ ojbects through a tcp connection:
My objects are all serializable, using boost serialization.
The TCP server/client is made with boost asio.
Basically I would like to send message like that would contain the message type (the type of the object being sent) and the data itself (the serialized object) and the size of the data so I can process the buffer (the size can vary for objects of the same type, as it is not POD).
I am a bit stuck, because I don't know how I can send this. I don't understand what are the steps to convert the data to a char buffer, and adding the extra information (message type & size) at the beginning of the buffer, and then giving this buffer to the send function of the tcp connection, all that with doing as few copies as possible.
Thanks.
-
Here you can find a good example on how to use boost::serialization together with boost::asio.
Something like this is the core of what you need:
std::ostringstream archive_stream;
boost::archive::text_oarchive archive(archive_stream);
archive << YOUR_DATA;
outbound_data_ = archive_stream.str();
boost::asio::async_write(socket_, boost::asio::buffer(outbound_data_), handler);