pcap_next_ex() never sets the pointer to the raw packet? - c++

I tried to read in the raw packet with libpcap (1.4.0 on CentOS 6).
However, for some reasons, rawPacket is always NULL after pcap_next_ex().
However, pcap_next_ex() does return 1 though it could mean timeout expired (where the timeout is set by the way?).
First, I thought that the filter string I passed into pcap_compile() was wrong. But I tried to copy and paste the same string to tcpdump, it worked fine -- I see expected packets being captured.
struct pcap_pkthdr *pHeader;
const u_char* rawPacket = NULL;
int rc = 0;
while (1) {
rc = pcap_next_ex(pDevice, &pHeader, &rawPacket);
if (-1 != rc && NULL != rawPacket) {
// process
struct ether_header* eptr = (struct ether_header *) rawPacket;
if (ntohs (eptr->ether_type) == ETHERTYPE_IP) {
printf("Ethernet type hex:%x dec:%d is an IP packet\n",
ntohs(eptr->ether_type),
ntohs(eptr->ether_type));
}
}
}
Any idea?
Thanks in advance.

Actually, what the pcap_next_ex() man page says is
pcap_next_ex() returns 1 if the packet was read without problems, 0 if
packets are being read from a live capture, and the timeout expired, -1
if an error occurred while reading the packet, and -2 if packets are
being read from a ``savefile'', and there are no more packets to read
from the savefile. If -1 is returned, pcap_geterr() or pcap_perror()
may be called with p as an argument to fetch or display the error text.
I need to edit it to remove the comment between "live capture" and "and the timeout expired", because what that means is that pcap_next_ex() returns:
1, if a packet was read or captured, in which case the pointer should be set to the raw packet;
0, if this is a live capture and the timeout (as specified in pcap_open_live() or, if you used pcap_create() and pcap_activate(), pcap_set_timeout()) expired while waiting for a packet, in which case no packet was read and the pointer will be set to NULL;
-1, if an error occurred while reading or capturing, in which case no packet was read and the pointer will be set to NULL;
-2, if this is a file being read and there are no more packets left to be read, in which case no packet was read and the pointer will be set to NULL.
So what you should do, after the pcap_next_ex() call, is:
if (1 == rc) {
// process
struct ether_header* eptr = (struct ether_header *) rawPacket;
if (ntohs (eptr->ether_type) == ETHERTYPE_IP) {
printf("Ethernet type hex:%x dec:%d is an IP packet\n",
ntohs(eptr->ether_type),
ntohs(eptr->ether_type));
}
} else if (0 == rc) {
// do nothing here - this isn't an error, but you have no packet
;
} else if (-1 == rc) {
// error
fprintf(stderr, "Error capturing or reading: %s\n", pcap_geterr(pDevice));
// quit trying to read or capture packets here
} else if (-2 == rc) {
// end of file if you're reading from a file
// this isn't an error, but there are no more packets to read
// so quit trying to read packets here
}

Related

Strange IOCP behaviour when communicating with browsers

I'm writing IOCP server for video streaming from desktop client to browser.
Both sides uses WebSocket protocol to unify server's achitecture (and because there is no other way for browsers to perform a full-duplex exchange).
The working thread starts like this:
unsigned int __stdcall WorkerThread(void * param){
int ThreadId = (int)param;
OVERLAPPED *overlapped = nullptr;
IO_Context *ctx = nullptr;
Client *client = nullptr;
DWORD transfered = 0;
BOOL QCS = 0;
while(WAIT_OBJECT_0 != WaitForSingleObject(EventShutdown, 0)){
QCS = GetQueuedCompletionStatus(hIOCP, &transfered, (PULONG_PTR)&client, &overlapped, INFINITE);
if(!client){
if( Debug ) printf("No client\n");
break;
}
ctx = (IO_Context *)overlapped;
if(!QCS || (QCS && !transfered)){
printf("Error %d\n", WSAGetLastError());
DeleteClient(client);
continue;
}
switch(auto opcode = client->ProcessCurrentEvent(ctx, transfered)){
// Client owed to receive some data
case OPCODE_RECV_DEBT:{
if((SOCKET_ERROR == client->Recv()) && (WSA_IO_PENDING != WSAGetLastError())) DeleteClient(client);
break;
}
// Client received all data or the beginning of new message
case OPCODE_RECV_DONE:{
std::string message;
client->GetInput(message);
// Analizing the first byte of WebSocket frame
switch( opcode = message[0] & 0xFF ){
// HTTP_HANDSHAKE is 'G' - from GET HTTP...
case HTTP_HANDSHAKE:{
message = websocket::handshake(message);
while(!client->SetSend(message)) Sleep(1); // Set outgoing data
if((SOCKET_ERROR == client->Send()) && (WSA_IO_PENDING != WSAGetLastError())) DeleteClient(client);
break;
}
// Browser sent a closing frame (0x88) - performing clean WebSocket closure
case FIN_CLOSE:{
websocket::frame frame;
frame.parse(message);
frame.masked = false;
if( frame.pl_len == 0 ){
unsigned short reason = 1000;
frame.payload.resize(sizeof(reason));
frame.payload[0] = (reason >> 8) & 0xFF;
frame.payload[1] = reason & 0xFF;
}
frame.pack(message);
while(!client->SetSend(message)) Sleep(1);
if((SOCKET_ERROR == client->Send()) && (WSA_IO_PENDING != WSAGetLastError())) DeleteClient(client);
shutdown(client->Socket(), SD_SEND);
break;
}
IO context struct:
struct IO_Context{
OVERLAPPED overlapped;
WSABUF data;
char buffer[IO_BUFFER_LENGTH];
unsigned char opcode;
unsigned long long debt;
std::string message;
IO_Context(){
debt = 0;
opcode = 0;
data.buf = buffer;
data.len = IO_BUFFER_LENGTH;
overlapped.Offset = overlapped.OffsetHigh = 0;
overlapped.Internal = overlapped.InternalHigh = 0;
overlapped.Pointer = nullptr;
overlapped.hEvent = nullptr;
}
~IO_Context(){ while(!HasOverlappedIoCompleted(&overlapped)) Sleep(1); }
};
Client Send function:
int Client::Send(){
int var_buf = O.message.size();
// "O" is IO_Context for Output
O.data.len = (var_buf>IO_BUFFER_LENGTH)?IO_BUFFER_LENGTH:var_buf;
var_buf = O.data.len;
while(var_buf > 0) O.data.buf[var_buf] = O.message[--var_buf];
O.message.erase(0, O.data.len);
return WSASend(connection, &O.data, 1, nullptr, 0, &O.overlapped, nullptr);
}
When the desktop client disconnects (it uses just closesocket() to do it, no shutdown()) the GetQueuedCompletionStatus returns TRUE and sets transfered to 0 - in this case WSAGetLastError() returns 64 (The specified network name is no longer available), and it has sense - client disconnected (line with if(!QCS || (QCS && !transfered))). But when the browser disconnects, the error codes confuse me... It can be 0, 997 (pending operation), 87 (invalid parameter)... and no codes related to end of connection.
Why do IOCP select this events? How can it select a pending operation? Why the error is 0 when 0 bytes transferred? Also it leads to endless trying to delete an object associated with the overlapped structure, because the destructor calls ~IO_Context(){ while(!HasOverlappedIoCompleted(&overlapped)) Sleep(1); } for secure deleting. In DeleteClient call the socket is closing with closesocket(), but, as you can see, I'm posting a shutdown(client->Socket(), SD_SEND); call before it (in FIN_CLOSE section).
I understand that there are two sides of a connection and closing it on a server side does not mean that an other side will close it too. But I need to create a stabile server, immune to bad and half opened connections. For example, the user of web application can rapidly press F5 to reload page few times (yeah, some dudes do so :) ) - the connection will reopen few times, and the server must not lag or crash due to this actions.
How to handle this "bad" events in IOCP?
you have many wrong code here.
while(WAIT_OBJECT_0 != WaitForSingleObject(EventShutdown, 0)){
QCS = GetQueuedCompletionStatus(hIOCP, &transfered, (PULONG_PTR)&client, &overlapped, INFINITE);
this is not efficient and wrong code for stop WorkerThread. at first you do excess call WaitForSingleObject, use excess EventShutdown and main this anyway fail todo shutdown. if your code wait for packet inside GetQueuedCompletionStatus that you say EventShutdown - not break GetQueuedCompletionStatus call - you continue infinite wait here. correct way for shutdown - PostQueuedCompletionStatus(hIOCP, 0, 0, 0) instead call SetEvent(EventShutdown) and if worked thread view client == 0 - he break loop. and usually you need have multiple WorkerThread (not single). and multiple calls PostQueuedCompletionStatus(hIOCP, 0, 0, 0) - exactly count of working threads. also you need synchronize this calls with io - do this only after all io already complete and no new io packets will be queued to iocp. so "null packets" must be the last queued to port
if(!QCS || (QCS && !transfered)){
printf("Error %d\n", WSAGetLastError());
DeleteClient(client);
continue;
}
if !QCS - the value in client not initialized, you simply can not use it and call DeleteClient(client); is wrong under this condition
when object (client) used from several thread - who must delete it ? what be if one thread delete object, when another still use it ? correct solution will be if you use reference counting on such object (client). and based on your code - you have single client per hIOCP ? because you retriever pointer for client as completion key for hIOCP which is single for all I/O operation on sockets bind to the hIOCP. all this is wrong design.
you need store pointer to client in IO_Context. and add reference to client in IO_Context and release client in IO_Context destructor.
class IO_Context : public OVERLAPPED {
Client *client;
ULONG opcode;
// ...
public:
IO_Context(Client *client, ULONG opcode) : client(client), opcode(opcode) {
client->AddRef();
}
~IO_Context() {
client->Release();
}
void OnIoComplete(ULONG transfered) {
OnIoComplete(RtlNtStatusToDosError(Internal), transfered);
}
void OnIoComplete(ULONG error, ULONG transfered) {
client->OnIoComplete(opcode, error, transfered);
delete this;
}
void CheckIoError(ULONG error) {
switch(error) {
case NOERROR:
case ERROR_IO_PENDING:
break;
default:
OnIoComplete(error, 0);
}
}
};
then are you have single IO_Context ? if yes, this is fatal error. the IO_Context must be unique for every I/O operation.
if (IO_Context* ctx = new IO_Context(client, op))
{
ctx->CheckIoError(WSAxxx(ctx) == 0 ? NOERROR : WSAGetLastError());
}
and from worked threads
ULONG WINAPI WorkerThread(void * param)
{
ULONG_PTR key;
OVERLAPPED *overlapped;
ULONG transfered;
while(GetQueuedCompletionStatus(hIOCP, &transfered, &key, &overlapped, INFINITE)) {
switch (key){
case '_io_':
static_cast<IO_Context*>(overlapped)->OnIoComplete(transfered);
continue;
case 'stop':
// ...
return 0;
default: __debugbreak();
}
}
__debugbreak();
return GetLastError();
}
the code like while(!HasOverlappedIoCompleted(&overlapped)) Sleep(1); is always wrong. absolute and always. never write such code.
ctx = (IO_Context *)overlapped; despite in your concrete case this give correct result, not nice and can be break if you change definition of IO_Context. you can use CONTAINING_RECORD(overlapped, IO_Context, overlapped) if you use struct IO_Context{
OVERLAPPED overlapped; } but better use class IO_Context : public OVERLAPPED and static_cast<IO_Context*>(overlapped)
now about Why do IOCP select this events? How to handle this "bad" events in IOCP?
the IOCP nothing select. he simply signaling when I/O complete. all. which specific wsa errors you got on different network operation absolute independent from use IOCP or any other completion mechanism.
on graceful disconnect is normal when error code is 0 and 0 bytes transferred in recv operation. you need permanent have recv request active after connection done, and if recv complete with 0 bytes transferred this mean that disconnect happens

UDP datagram is being split on space character

I am trying to get my head into SDLnet and I am encountering a problem where any UDP packets that I send from the client to the server are being broken up on the space character. I can't see any reason for this happening as I am not explicitly programming this behaviour in - I am literally just sending across a string.
The source code I am using is part of an online example on The Game Programming Wiki
Server
printf("Fill the buffer\n>");
scanf("%s", (char *)p->data);
p->address.host = srvadd.host; /* Set the destination host */
p->address.port = srvadd.port; /* And destination port */
p->len = strlen((char *)p->data) + 1;
SDLNet_UDP_Send(sd, -1, p); /* This sets the p->channel */
/* Quit if packet contains "quit" */
if (!strcmp((char *)p->data, "quit"))
quit = 1;
Client
/* Wait a packet. UDP_Recv returns != 0 if a packet is coming */
if (SDLNet_UDP_Recv(sd, p))
{
printf("UDP Packet incoming\n");
printf("\tChan: %d\n", p->channel);
printf("\tData: %s\n", (char *)p->data);
printf("\tLen: %d\n", p->len);
printf("\tMaxlen: %d\n", p->maxlen);
printf("\tStatus: %d\n", p->status);
printf("\tAddress: %x %x\n", p->address.host, p->address.port);
/* Quit if packet contains "quit" */
if (strcmp((char *)p->data, "quit") == 0)
quit = 1;
}
Output
The output looks like this image.
The operating system I am running on is Windows 7 64-bit and I'm wondering if this could be something OS-related.
This is not the fault of UDP, it's go to do with the char* being split up when using scanf. ( I'm not a 100% sure about the details here. ) But as a general rule, in C, you shouldn't use scanf
Since you are using C++ ( at least according to the tags), you should do this the C++ way :
std::string msg = "";
std::cout << "Type a message and hit enter\n";
// Let user type a message
std::cin.ignore();
std::getline(std::cin, msg );
// UDPpacket uses Uint8, whereas msg.c_str() gives us a char*
// This simply copies the integer value of the chars into packet->data
memcpy(packet->data, msg.c_str(), msg.length() );
packet->len = msg.length();
Note :
The std::cin.ignore(); is there to make sure we stop and wait for the user to type in the message.

Corruption of data in memcpy

I'm currently working on a project using sockets via WinSock and have come across a peculiar problem. I'll attach the code before I start explaining.
#include "Connection.h"
Connection::Connection(SOCKET sock, int socketType)
: m_sock(sock), m_recvCount(0), m_sendCount(0), m_socketType(socketType)
{
printf("Succesfully created connection\n");
}
Connection::~Connection(void)
{
printf("Closing socket %d", m_sock);
closesocket(m_sock);
}
void Connection::ProcessMessage(const NetMessage *message){
printf("Got network message: type %d, data %s\n", message->type, message->data);
}
bool Connection::ReadSocket(){
// Call this when the socket is ready to read.
// Returns true if the socket should be closed.
// used to store count between the sockets
int count = 0;
if(m_socketType == SOCK_STREAM){
// attempt to read a TCP socket message
// Receive as much data from the client as will fit in the buffer.
count = recv(m_sock, &m_recvBuf[m_recvCount], sizeof(m_recvBuf) - m_recvCount, 0);
}
else if(m_socketType == SOCK_DGRAM){
// attempt to read UDP socket message
// temporarily stores details of the address which sent the message
// since UDP doesn't worry about whether it's connected to the
// sender or not
sockaddr_in fromAddr;
int fromAddrSize = sizeof(fromAddr);
count = recvfrom(m_sock, &m_recvBuf[m_recvCount], sizeof(m_recvBuf) - m_recvCount, 0, (sockaddr*) &fromAddr, &fromAddrSize);
}
else{
printf("Unknown socket type %d\n", m_socketType);
return true;
}
if (count <= 0)
{
printf("Tried to receive on socket %d and got %d bytes\n", m_sock, count);
printf("Client connection closed or broken\n");
return true;
}
// if we get to this point we have essentially received a complete message
// and must process it
printf("Received %d bytes from the client (total %d)\n", count, m_recvCount);
m_recvCount += count;
// Have we received a complete message?
// if so, process it
if (m_recvCount == sizeof NetMessage)
{
ProcessMessage((const NetMessage *) m_recvBuf);
m_recvCount = 0;
}
return false;
}
bool Connection::WriteSocket(){
// Sends the data in the send buffer through the socket
int count;
if(m_socketType == SOCK_STREAM){
// attempt to read TCP socket message
count = send(m_sock, m_sendBuf, m_sendCount, 0);
}
else if(m_socketType == SOCK_DGRAM){
// attempt to read UDP socket message
count = sendto(m_sock, m_sendBuf, m_sendCount, 0, 0, 0);
}
else{
// unhandled type of socket, kill server
printf("Unknown socket type %d", m_socketType);
return true;
}
if (count <= 0)
{
// we have received an error from the socket
printf("Client connection closed or broken\n");
return true;
}
m_sendCount -= count;
printf("Sent %d bytes to the client (%d left)\n", count, m_sendCount);
printf("Data: %s", m_sendBuf);
// Remove the sent data from the start of the buffer.
memmove(m_sendBuf, &m_sendBuf[count], m_sendCount);
return false;
}
bool Connection::WantWrite(){
if(m_sendCount > 0){
return true;
}
return false;
}
bool Connection::WantRead(){
return true;
}
bool Connection::SetMessage(const NetMessage *message){
// store contents of the message in the send buffer
// to allow us to send later
if (m_sendCount + sizeof(NetMessage) > sizeof(m_sendBuf))
{
return true;
}
memcpy(&m_sendBuf, message, sizeof(message));
m_sendCount += sizeof(NetMessage);
return false;
}
and the protocol
/* Definitions for the network protocol that the client and server use to communicate */
#ifndef PROTOCOL_H
#define PROTOCOL_H
// Message types.
enum MessageType
{
MT_UNKNOWN = 0,
MT_WELCOME = 1,
MT_KEYPRESS = 2,
MT_CHATMESSAGE = 3
};
// The message structure.
// This is a "plain old data" type, so we can send it over the network.
// (In a real program, we would want this structure to be packed.)
struct NetMessage
{
MessageType type;
char* data;
NetMessage()
: type(MT_UNKNOWN)
{
}
};
#endif
Essentially the protocol holds the definition of the messages that the client and server throw around to each other. The problem I am having is that, in connection.cpp line 132 (memcpy), the message becomes garbled in sendBuf.
http://imgur.com/MekQfgm,9ShRtHi
The image above shows exactly what is happening. As said in protocol.h the struct is a POD so when I do memcpy it should transfer the number of bytes as is held in the struct (so for example the message type should be 1 byte, followed by 7 or 8 bytes of data, in the example).
Can anyone shed some light on this? It's driving me crazy.
The line you wrote will copy 4 bytes (sizeof(pointer)) on 32bit systems:
memcpy(&m_sendBuf, message, sizeof(message));
what you probably meant is:
memcpy(&m_sendBuf, message, sizeof(NetMessage));
Edit:
In addition, as a commenter remarked, your data type is NOT a POD. It holds a pointer. You transfer that pointer. At the target system, it will point to the same place in RAM, but there will not be anything there. You need to actually make your datatype a POD by using an array or you need to find a way to transfer the data pointed to. You can achieve this by transfering the type, a length and a number of characters. That means that your receiver can NOT rely on messages being of fixed size.

Berkeley Socket Send returning 0 on successful non-blocking send

I am writing a non-blocking chat server, so far the server works fine, but I can't figure out how to correct for partial sends if they happen. The send(int, char*, int); function always returns 0 on a success and -1 on a failed send. Every doc/man page I have read says it should return the number of bytes actually feed to the network buffer. I have checked to be sure that I can send to the server and recv back the data repeatedly without problem.
This is the function I use to call the send. I both tried to print the return data to the console first, then tried line breaking on the return ReturnValue; while debugging. Same result, ReturnValue is always 0 or -1;
int Connection::Transmit(string MessageToSend)
{
// check for send attempts on a closed socket
// return if it happens.
if(this->Socket_Filedescriptor == -1)
return -1;
// Send a message to the client on the other end
// note, the last parameter is a flag bit which
// is used for declaring out of bound data transmissions.
ReturnValue = send(Socket_Filedescriptor,
MessageToSend.c_str(),
MessageToSend.length(),
0);
return ReturnValue;
}
Why don't you try to send in a loop? For instance:
int Connection::Transmit(string MessageToSend)
{
// check for send attempts on a closed socket
// return if it happens.
if(this->Socket_Filedescriptor == -1)
return -1;
int expected = MessageToSend.length();
int sent = 0;
// Send a message to the client on the other end
// note, the last parameter is a flag bit which
// is used for declaring out of bound data transmissions.
while(sent < expected) {
ReturnValue = send(Socket_Filedescriptor,
MessageToSend.c_str() + sent, // Send from correct location
MessageToSend.length() - sent, // Update how much remains
0);
if(ReturnValue == -1)
return -1; // Error occurred
sent += ReturnValue;
}
return sent;
}
This way your code will continually try to send all the data until either an error occurs, or all data is sent successfully.

Using select() and fgets() to access information from a serial port

This is a followup to this question: How to wait for input from the serial port in the middle of a program
I am writing a program to control an Iridium modem that needs to wait for a response from the serial port in the middle of the program in order to verify that the correct response was given. In order to accomplish this, a user recommended I use the select() command to wait for this input.
However, I have run into some difficulty with this approach. Initially, select() would return the value indicated a timeout on the response every time (even though the modem was sending back the correct responses, which I verified with another program running at the same time). Now, the program stops after one iteration, even with the correct response sent back from the modem.
//setting the file descriptor to the port
int fd = open(portName.c_str(), O_RDWR | O_NOCTTY | O_NDELAY);
if (fd == -1)
{
/*
* Could not open the port.
*/
perror("open_port: Unable to open /dev/ttyS0 - ");
}
else
fcntl(fd, F_SETFL, 0);
FILE *out = fopen(portName.c_str(), "w");//sets the serial port
FILE *in = fopen(portName.c_str(), "r");
fd_set fds;
FD_ZERO(&fds);
FD_SET(fd, &fds);
struct timeval timeout = { 10, 0 }; /* 10 seconds */
//int ret = select(fd+1, &fds, NULL, NULL, &timeout);
/* ret == 0 means timeout, ret == 1 means descriptor is ready for reading,
ret == -1 means error (check errno) */
char buf[100];
int i =0;
while(i<(sizeof(messageArray)/sizeof(messageArray[0])))
{
//creates a string with the AT command that writes to the module
std::string line1("AT+SBDWT=");
line1+=convertInt( messageArray[i].numChar);
line1+=" ";
line1+=convertInt(messageArray[i].packetNumber);
line1+=" ";
line1+=messageArray[i].data;
line1+=std::string("\r\n");
//creates a string with the AT command that initiates the SBD session
std::string line2("AT+SBDI");
line2+=std::string("\r\n");
fputs(line1.c_str(), out); //sends to serial port
//usleep(7000000);
int ret =select(fd+1, &fds, NULL, NULL, &timeout);
/* ret == 0 means timeout, ret == 1 means descriptor is ready for reading,
ret == -1 means error (check errno) */
if (ret ==1){
fgets (buf ,sizeof(buf), in);
//add code to check if response is correct
}
else if(ret == 0) {
perror("timeout error ");
}
else if (ret ==-1) {
perror("some other error");
}
fputs(line2.c_str(), out); //sends to serial port
//usleep(7000000); //Pauses between the addition of each packet.
int ret2 = select(fd+1, &fds, NULL, NULL, &timeout);
/* ret == 0 means timeout, ret == 1 means descriptor is ready for reading,
ret == -1 means error (check errno) */
if(ret2 == 0) {
perror("timeout error ");
}
else if (ret2 ==-1) {
perror("some other error");
}
i++;
}
You aren't using the same file handle for read/write/select, which is somewhat strange.
You are not resetting your fd_sets, which are modified by select and would have all of your fds unset in the case of a timeout, making the next call timeout by default (as you are asking for no fds).
you are also using buffered IO, which is bound to create headaches in this case. eg. fgets waits for either EOF (which won't occur), or a newline, reading all the while. It will block until it gets its newline, so may keep you hanging indefinitely if that never occurs.
It may also read more than it needs into the buffer, messing up your select read signal (you have data in the buffer, but select will time out, since there's nothing to read on the filehandle).
Bottom line is this:
use FD_SET in the loop to set/reset your fd sets, also reset your timeout, as select may modify it.
use a single handle for read/write/select, instead of multiple handles, eg. open file with fopen(..., "w+") or open(..., O_RDWR)
if still using fopen, try disabling buffering using setvbuf with the _IONBF buffering option.
otherwise, use open/read/write instead of fopen etc.
I will note that part of this was mentioned in this answer to your previous question.
You should perhaps use fflush() on your output file stream.