Streaming images over QTcpSocket on Windows 7 - c++

Question

I am trying to create two test applications based on QTcpSocket, a server that streams (uncompressed) images and a client that receives them.
My code is mostly taken from Qts Fortune Client Example and Fortune Server Example. I have cut out all gui. Instead of opening a connection, sending a fortune and then immediately closing it, my server keeps it open and continuously streams my image. The client just reads the images from the QTcpSocket and then discards them.
The image I'm sending is 800x600 RGB (=1440000 bytes), I am sending it as often as I am allowed. The image is read from file each time before it is sent and I am not using any compression.
The server seems to be sending images just as it should. BUT the client receives them too slowly, 1-4 frames per second, and it seems to not be receiving any data at times, which in turn causes my server to use alot of memory (because the client isn't reading as fast as the server is writing).
I've tried running my server and client on different machines and both on one machine, both setups produce the same problem.
When running my applications on a Linux machine, the client receives images at a much higher rate (think it was 14 frames per second). The client seems to be able to read as fast as the server writes.
Can anyone help shed some light on this problem?
How can I speed up the data transfere? (without using compression)
How can I get the client to read as fast as the server is writing?
How can I get this to be stable on my Windows machine? (no sudden pauses...) Clicking in the console window of the client sometimes seem to "wake" the application btw. ^^
Here is my code:
Server:
main.cpp
#include <iostream>
#include <QCoreApplication>
#include "Server.h"
int main(int argc, char *argv[]){
QCoreApplication app(argc, argv);
QString ipaddress = QString(argv[1]);
int port = atoi(argv[2]);
Server* server = new Server(ipaddress, port);
int retVal = app.exec();
return retVal;
}
Server.h
#ifndef SERVER_H_
#define SERVER_H_
#include <QObject>
QT_BEGIN_NAMESPACE
class QTcpServer;
class QNetworkSession;
class QTcpSocket;
class QTimer;
QT_END_NAMESPACE
class Server : public QObject
{
Q_OBJECT
public:
Server(QString ipAddress, int port, QObject *parent = 0);
private slots:
void newConnectionSlot();
void sendSlot();
private:
QTcpServer *mTcpServer;
QTcpSocket *mTcpSocket;
QTimer *mSendTimer;
};
#endif /* SERVER_H_ */
Server.cpp
#include "Server.h"
#include <iostream>
#include <QTcpServer>
#include <QTcpSocket>
#include <QTimer>
#include <QDateTime>
#include <QSettings>
#include <highgui.h>
Server::Server(QString ipAddress, int port, QObject *parent) :
QObject(parent), mTcpServer(0), mTcpSocket(0)
{
mTcpServer = new QTcpServer(this);
connect(mTcpServer, SIGNAL(newConnection()), this, SLOT(newConnectionSlot()));
if (!mTcpServer->listen(QHostAddress(ipAddress), port)) {
std::cout << "Unable to start the server: " << mTcpServer->errorString().toStdString() << std::endl;
return;
}
std::cout << "The server is running on\n\nIP: "<< ipAddress.toStdString()
<< "\nport: " << mTcpServer->serverPort() << "\n\nRun the Client now.\n" << std::endl;
}
void Server::newConnectionSlot()
{
mTcpSocket = mTcpServer->nextPendingConnection();
connect(mTcpSocket, SIGNAL(disconnected()),
mTcpSocket, SLOT(deleteLater()));
// setup timer to send data at a given interval
mSendTimer = new QTimer(this);
connect(mSendTimer, SIGNAL(timeout()),
this, SLOT(sendSlot()));
mSendTimer->start(40);
}
void Server::sendSlot()
{
if(!mTcpSocket)
return;
//know that the image is this big
int width = 800;
int height = 600;
int nChannels = 3;
int depth = 8;
qint64 blockSize = 1440000; //in bytes
qint64 imagesInQue = mTcpSocket->bytesToWrite()/blockSize;
int maxPendingImages = 25;
if(imagesInQue > maxPendingImages)
{
std::cout << "Dumping." << std::endl;
return;
}
//load image
IplImage* img = cvLoadImage("pic1_24bit.bmp");
if(!img)
std::cout << "Error loading image " << std::endl;;
//send data
quint64 written = mTcpSocket->write(img->imageData, img->imageSize);
//clean up
cvReleaseImage( &img );
}
Client:
main.cpp
#include <iostream>
#include <QCoreApplication>
#include "Client.h"
int main(int argc, char *argv[]){
QCoreApplication app(argc, argv);
QString ipaddress = QString(argv[1]);
int port = atoi(argv[2]);
Client* client = new Client(ipaddress, port);
int retVal = app.exec();
}
Client.h
#ifndef CLIENT_H_
#define CLIENT_H_
#include <QObject>
#include <QAbstractSocket>
QT_BEGIN_NAMESPACE
class QTcpSocket;
QT_END_NAMESPACE
class Client : public QObject
{
Q_OBJECT
public:
Client(QString ipAddress, int port, QObject *parent=0);
private slots:
void readSlot();
void displayErrorSlot(QAbstractSocket::SocketError);
private:
QTcpSocket *mTcpSocket;
QString mIpAddress;
int mPort;
};
#endif /* CLIENT_H_ */
Client.cpp
#include "Client.h"
#include <iostream>
#include <QTcpSocket>
#include <QSettings>
#include <QDateTime>
Client::Client(QString ipAddress, int port, QObject *parent):
QObject(parent), mTcpSocket(0), mIpAddress(ipAddress), mPort(port)
{
mTcpSocket = new QTcpSocket(this);
connect(mTcpSocket, SIGNAL(readyRead()),
this, SLOT(readSlot()));
connect(mTcpSocket, SIGNAL(error(QAbstractSocket::SocketError)),
this, SLOT(displayErrorSlot(QAbstractSocket::SocketError)));
std::cout << "Connecting to ip: " << mIpAddress.toStdString() << " port: " << mPort << std::endl;
mTcpSocket->connectToHost(mIpAddress, mPort);
}
void Client::readSlot()
{
static qint64 starttime = QDateTime::currentMSecsSinceEpoch();
static int frames = 0;
//know that the image is this big
int width = 800;
int height = 600;
int nChannels = 3;
int depth = 8;
qint64 blockSize = 1440000; //in bytes
if (mTcpSocket->bytesAvailable() < blockSize)
{
return;
}
frames++;
char* data = (char*) malloc(blockSize+100);
qint64 bytesRead = mTcpSocket->read(data, blockSize);
free(data);
//FPS
if(frames % 100 == 0){
float fps = frames/(QDateTime::currentMSecsSinceEpoch() - starttime);
std::cout << "FPS: " << fps << std::endl;
}
}
void Client::displayErrorSlot(QAbstractSocket::SocketError socketError)
{
switch (socketError) {
case QAbstractSocket::RemoteHostClosedError:
break;
case QAbstractSocket::HostNotFoundError:
std::cout << "The host was not found. Please check the "
"host name and port settings."<< std::endl;
break;
case QAbstractSocket::ConnectionRefusedError:
std::cout << "The connection was refused by the peer. "
"Make sure the fortune server is running, "
"and check that the host name and port "
"settings are correct."<< std::endl;
break;
default:
std::cout << "The following error occurred: " << mTcpSocket->errorString().toStdString() << std::endl;
break;
}
}

Answer 1

You are miscounting your frames. You must increment the frame counter after the mTcpSocket->bytesAvailable() < blockSize test! That's the source of your non-problem: it works OK, but you miscount the frames. That's also why it "works better" on Linux: the network stack there is buffering the data differently, giving you less signals.
You must limit the amount of memory on the wire (buffered) on the server end. Otherwise, as you correctly noticed, you'll run out of memory. See my other answer for an example of how to do it.
You're never freeing the memory you malloc() in the receiver. Be aware that you should expect to consume 35 megabytes of RAM per second if the server really sends at 40ms intervals. The receiver may well get bogged down very quickly by that memory leak. That's probably the source of the trouble.
What is that QDataStream for? You're not using a datastream to send the image, and there's no use for it on the receiving end either.