I've got a server that uses a two thread system to manage between 100 and 200 concurrent connections. It uses TCP sockets, as packet delivery guarantee is important (it's a communication system where missed remote API calls could FUBAR a client).
I've implemented a custom protocol layer to separate incoming bytes into packets and dispatch them properly (the library is included below). I realize the issues of using MSG_PEEK, but to my knowledge, it is the only system that will fulfill the needs of the library implementation. I am open to suggestions, especially if it could be part of the problem.
Basically, the problem is that, randomly, the server will drop the client's socket due to a lack of incoming packets for more than 20 seconds, despite the client successfully sending a keepalive packet every 4. I can verify that the server itself didn't go offline and that the connection of the users (including myself) experiencing the problem is stable.
The library for sending/receiving is here:
short ncsocket::send(wstring command, wstring data) {
wstringstream ss;
int datalen = ((int)command.length() * 2) + ((int)data.length() * 2) + 12;
ss << zero_pad_int(datalen) << L"|" << command << L"|" << data;
int tosend = datalen;
short __rc = 0;
do{
int res = ::send(this->sock, (const char*)ss.str().c_str(), datalen, NULL);
if (res != SOCKET_ERROR)
tosend -= res;
else
return FALSE;
__rc++;
Sleep(10);
} while (tosend != 0 && __rc < 10);
if (tosend == 0)
return TRUE;
return FALSE;
}
short ncsocket::recv(netcommand& nc) {
vector<wchar_t> buffer(BUFFER_SIZE);
int recvd = ::recv(this->sock, (char*)buffer.data(), BUFFER_SIZE, MSG_PEEK);
if (recvd > 0) {
if (recvd > 8) {
wchar_t* lenstr = new wchar_t[4];
memcpy(lenstr, buffer.data(), 8);
int fulllen = _wtoi(lenstr);
delete lenstr;
if (fulllen > 0) {
if (recvd >= fulllen) {
buffer.resize(fulllen / 2);
recvd = ::recv(this->sock, (char*)buffer.data(), fulllen, NULL);
if (recvd >= fulllen) {
buffer.resize(buffer.size() + 2);
buffer.push_back((char)L'\0');
vector<wstring> data = parsewstring(L"|", buffer.data(), 2);
if (data.size() == 3) {
nc.command = data[1];
nc.payload = data[2];
return TRUE;
}
else
return FALSE;
}
else
return FALSE;
}
else
return FALSE;
}
else {
::recv(this->sock, (char*)buffer.data(), BUFFER_SIZE, NULL);
return FALSE;
}
}
else
return FALSE;
}
else
return FALSE;
}
This is the code for determining if too much time has passed:
if ((int)difftime(time(0), regusrs[i].last_recvd) > SERVER_TIMEOUT) {
regusrs[i].sock.end();
regusrs[i].is_valid = FALSE;
send_to_all(L"removeuser", regusrs[i].server_user_id);
wstringstream log_entry;
log_entry << regusrs[i].firstname << L" " << regusrs[i].lastname << L" (suid:" << regusrs[i].server_user_id << L",p:" << regusrs[i].parent << L",pid:" << regusrs[i].parentid << L") was disconnected due to idle";
write_to_log_file(server_log, log_entry.str());
}
The "regusrs[i]" is using the currently iterated member of a vector I use to story socket descriptors and user information. The 'is_valid' check is there to tell if the associated user is an actual user - this is done to prevent the system from having to deallocate the member of the vector - it just returns it to the pool of available slots. No thread access/out-of-range issues that way.
Anyway, I started to wonder if it was the server itself was the problem. I'm testing on another server currently, but I wanted to see if another set of eyes could stop something out of place or cue me in on a concept with sockets and extended keepalives that I'm not aware of.
Thanks in advance!
I think I see what you're doing with MSG_PEEK, where you wait until it looks like you have enough data to read a full packet. However, I would be suspicious of this. (It's hard to determine the dynamic behaviour of your system just by looking at this small part of the source and not the whole thing.)
To avoid use of MSG_PEEK, follow these two principles:
When you get a notification that data is ready (I assume you're using select), then read all the waiting data from recv(). You may use more than one recv() call, so you can handle the incoming data in pieces.
If you read only a partial packet (length or payload), then save it somewhere for the next time you get a read notification. Put the packets and payloads back together yourself, don't leave them in the socket buffer.
As an aside, the use of new/memcpy/wtoi/delete is woefully inefficient. You don't need to allocate memory at all, you can use a local variable. And then you don't even need the memcpy at all, just a cast.
I presume you already assume that your packets can be no longer than 999 bytes in length.
I'm trying to send some hex string to a serial port, and directly read its answer in c++. If there is no answer it should timeout after a given period of time.
What is the simplest implementation of such a task?
Do i need to use stuff like boost?
To be clear: I'm searching for the most simple way to achieve this.
Sorry if my question is dumb, but im new to this topic, thanks in advance!
EDIT: Sorry that I forgot to mention, it should run on Linux.
this should be similar to your declaration:
Setup(CSerial::EBaud9600,CSerial::EData8,CSerial::EParNone,CSerial::EStop1);
Setup(CSerial::EBaudrate(9600),
CSerial::EDataBits(8),
CSerial::EParity(NOPARITY),
CSerial::EStopBits(ONESTOPBIT));
Reading data:
// Read data, until there is nothing left
DWORD dwBytesRead = 0;
BYTE abBuffer[100];
do
{
// Read data from the COM-port
serial.Read(abBuffer,sizeof(abBuffer),&dwBytesRead);
if (dwBytesRead > 0)
{
// TODO: Process the data
}
}
while (dwBytesRead == sizeof(abBuffer));
More details can be found here on code project: http://www.codeproject.com/Articles/992/Serial-library-for-C
Please note: I am used to programming serial ports in c#, and so (to the best of my knowledge) believe this would work for you. (I would also like to point outall serial port communication is send actually through as hex, but may be read via the buffer as a decimal value ( please refer to http://www.asciitable.com/ for converstion, or use something similar to UTF8 encoding)
EDIT - As per comment;
Please refer to : http://msdn.microsoft.com/en-gb/library/windows/hardware/hh439614(v=vs.85).aspx for details on serial port read/write timeouts
Write timeout will be similar to;
[BrowsableAttribute(true)]
public:
property int WriteTimeout {
int get ();
void set (int value);
}
Which allows you to get or set the timeout attribute
Full Program
public:
static void Main()
{
String^ name;
String^ message;
StringComparer^ stringComparer = StringComparer::OrdinalIgnoreCase;
Thread^ readThread = gcnew Thread(gcnew ThreadStart(PortChat::Read));
// Create a new SerialPort object with default settings.
_serialPort = gcnew SerialPort();
// Allow the user to set the appropriate properties.
_serialPort->PortName = SetPortName(_serialPort->PortName);
_serialPort->BaudRate = SetPortBaudRate(_serialPort->BaudRate);
_serialPort->Parity = SetPortParity(_serialPort->Parity);
_serialPort->DataBits = SetPortDataBits(_serialPort->DataBits);
_serialPort->StopBits = SetPortStopBits(_serialPort->StopBits);
_serialPort->Handshake = SetPortHandshake(_serialPort->Handshake);
// Set the read/write timeouts
_serialPort->ReadTimeout = 500;
_serialPort->WriteTimeout = 500;
_serialPort->Open();
_continue = true;
readThread->Start();
Console::Write("Name: ");
name = Console::ReadLine();
Console::WriteLine("Type QUIT to exit");
while (_continue)
{
message = Console::ReadLine();
if (stringComparer->Equals("quit", message))
{
_continue = false;
}
else
{
_serialPort->WriteLine(
String::Format("<{0}>: {1}", name, message) );
}
}
readThread->Join();
_serialPort->Close();
}
static void Read()
{
while (_continue)
{
try
{
String^ message = _serialPort->ReadLine();
Console::WriteLine(message);
}
catch (TimeoutException ^) { }
}
}
EDIT 2
Please refer to Linux Serial Port: Blocking Read with Timeout where this has been declared in the question and some of the answers may prove useful to you as well! :)
I recently came for the same issue and I found a good solution using select()
Read the documentation here: manpages.courirer-mta.org/htmlman2/select2.html
In your case you need to setup the timeout, which is the 5th argument of select:
struct timeval timeout;
timeout.tv_sec = 0 ; // seconds
timeout.tv_usec = 1000 ; // microseconds
Hope this helps.
I have written a client/server application where the server spawns multiple threads depending upon the request from client.
These threads are expected to send some data to the client(string).
The problem is, data gets overwritten on the client side. How do I tackle this issue ?
I have already read some other threads on similar issue but unable to find the exact solution.
Here is my client code to receive data.
while(1)
{
char buff[MAX_BUFF];
int bytes_read = read(sd,buff,MAX_BUFF);
if(bytes_read == 0)
{
break;
}
else if(bytes_read > 0)
{
if(buff[bytes_read-1]=='$')
{
buff[bytes_read-1]='\0';
cout<<buff;
}
else
{
cout<<buff;
}
}
}
Server Thread code :
void send_data(int sd,char *data)
{
write(sd,data,strlen(data));
cout<<data;
}
void *calcWordCount(void *arg)
{
tdata *tmp = (tdata *)arg;
string line = tmp->line;
string s = tmp->arg;
int sd = tmp->sd_c;
int line_no = tmp->line_no;
int startpos = 0;
int finds = 0;
while ((startpos = line.find(s, startpos)) != std::string::npos)
{
++finds;
startpos+=1;
pthread_mutex_lock(&myMux);
tcount++;
pthread_mutex_unlock(&myMux);
}
pthread_mutex_lock(&mapMux);
int t=wcount[s];
wcount[s]=t+finds;
pthread_mutex_unlock(&mapMux);
char buff[MAX_BUFF];
sprintf(buff,"%s",s.c_str());
sprintf(buff+strlen(buff),"%s"," occured ");
sprintf(buff+strlen(buff),"%d",finds);
sprintf(buff+strlen(buff),"%s"," times on line ");
sprintf(buff+strlen(buff),"%d",line_no);
sprintf(buff+strlen(buff),"\n",strlen("\n"));
send_data(sd,buff);
delete (tdata*)arg;
}
On the server side make sure the shared resource (the socket, along with its associated internal buffer) is protected against the concurrent access.
Define and implement an application level protocol used by the server to make it possible for the client to distinguish what the different threads sent.
As an additional note: One cannot rely on read()/write() reading/writing as much bytes as those two functions were told to read/write. It is an essential necessity to check their return value to learn how much bytes those functions actually read/wrote and loop around them until all data that was intended to be read/written had been read/written.
You should put some mutex to your socket.
When a thread use the socket it should block the socket.
Some mutex example.
I can't help you more without the server code. Because the problem is probably in the server.
First time poster long time reader.
I've been playing round with reading in data from a bluetooth GPS unit.
I can connect to it using hyperterm and see the data
The following log is from the hyperterm
$GPRMC,195307.109,A,5208.2241,N,00027.7689,W,000.0,345.8,310712,,,A*7E
$GPVTG,345.8,T,,M,000.0,N,000.0,K,A*07
$GPGGA,195308.109,5208.2242,N,00027.7688,W,1,04,2.1,58.9,M,47.3,M,,0000*7E
$GPGSA,A,3,19,03,11,22,,,,,,,,,5.5,2.1,5.0*3F
$GPRMC,195308.109,A,5208.2242,N,00027.7688,W,000.0,345.8,310712,,,A*73
$GPVTG,345.8,T,,M,000.0,N,000.0,K,A*07
$GPGGA,195309.109,5208.2243,N,00027.7688,W,1,04,2.1,58.9,M,47.3,M,,0000*7E
END LOG
The following log is from my C++ program
$GPGSV,3,3,12,14,20,105,16,28,18,323,,08,07,288,,16,01,178,*7A
$GPRMC,195,3,2ÿþÿÿÿL.š945.109,A,5208.2386,N,00027.7592,W,000.0,169.5,8,323,,08,07,288,,16,01,178,*7A
$GPRMC,195,3,2ÿþÿÿÿL.š310712,,,A*70
$GPVTG,169.5,T,,M,000.0,N,000.0,K,A*06
8,07,288,,16,01,178,*7A
$GPRMC,195,3,2ÿþÿÿÿL.š310712,,,A*70
$GPVTG,169.5,T,,M,000.0,N,000.0,K,A*06
8,07,288,,16,01,178,*7A
$GPRMC,195,3,2ÿþÿÿÿL.š$GPGGA,195946.109,5208.2386,N,00027.7592,W,1.0,K,A*06
8,07,288,,16,01,178,*7A
END LOG
THE PROBLEM
I've left the line feeds as they come, the C++ output has extra line feeds, not sure why?
The C++ log also has some funky chars...?
The Code
for (int n=0;n<100;n++) {
char INBUFFER[100];
cv::waitKey(1000);
bStatus = ReadFile(comport, // Handle
&INBUFFER, // Incoming data
100, // Number of bytes to read
&bytes_read, // Number of bytes read
NULL);
cout << "bStatus " << bStatus << endl;
if (bStatus != 0)
{
// error processing code goes here
}
LogFile << INBUFFER;
}
I'm using settings...
comSettings.BaudRate = 2400;
comSettings.StopBits = ONESTOPBIT;
comSettings.ByteSize = 8;
comSettings.Parity = NOPARITY;
comSettings.fParity = FALSE;
...which as far as I can tell are the same as the settings used by hyperterm.
Any hints on what I'm doing wrong?
cheers!
UPDATE
So after updating to use bytes_read and account for the extra LF at the end of NMEA data I now have...
if (bytes_read!=0) {
for (int i=0; i < bytes_read; i++) {
LogFile << INBUFFER[i];
}
}
Which appears to have fixed things!
$GPGGA,215057.026,5208.2189,N,00027.7349,W,1,04,6.8,244.6,M,47.3,M,,0000*41
$GPGSA,A,3,32,11,01,19,,,,,,,,,9.7,6.8,7.0*3D
$GPRMC,215057.026,A,5208.2189,N,00027.7349,W,002.0,208.7,310712,,,A*74
$GPVTG,208.7,T,,M,002.0,N,003.8,K,A*09
$GPGGA,215058.026,5208.2166,N,00027.7333,W,1,04,6.8,243.1,M,47.3,M,,0000*42
Thanks folks, your help was much appreciated.
You have a bytes_read var, but you don't do anything with it? Seems to me that you're dumping the entire INBUFFER to the file, no matter how many/few bytes are actually loaded into it?