I'm trying to monitor a unix socket ("/tmp/mysocket").
I can do this fine in Node.js: it raises an event when 1) a socket is bound, 2) someone connects to the socket, 3) data is sent to the socket and 4) the socket is disconnected.
I'm trying to now do this in C/C++: I want to monitor "/tmp/mysocket" for the above events. I've looked at libevent (which I'd preferably like to use), but see that it requires an IP:port. Is there any way to monitor unix sockets?
Or can anyone suggest another C/C++ solution?
You could monitor a UNIX domain socket just like a regular file, since it can be operated like a file, e.g. in libev,
struct sockaddr_un address;
memset(&address, 0, sizeof(address));
address.sun_family = AF_LOCAL;
strcpy(address.sun_path, "/tmp/mysocket");
bind(socket, (struct sockaddr*)(&address), sizeof(address));
listen(socket, 5);
// now listen if someone has connected to the socket.
// we use 'ev_io' since the 'socket' can be treated as a file descriptor.
struct ev_io* io = malloc(sizeof(ev_io));
ev_io_init(io, accept_cb, socket, EV_READ);
ev_io_start(loop, io);
...
void accept_cb(struct ev_loop* loop, struct ev_io* io, int r)
{
// someone has connected. we accept the child.
struct sockaddr_un client_address;
socklen_t client_address_len = sizeof(client_address);
int client_fd = accept(socket, (sockaddr*)(&client_address),
&client_address_len);
// 'read' / 'recv' from client_fd here.
// or use another 'ev_io' for async read.
}
libevent should be similar.
Related
I have a simple socket server set up using sys/socket and OpenSSL. For each connection, the client is required to send a message to the server, receive a response and then reply to that response.
I can't find any clear mechanism for making these sockets non-blocking? The system has to be able to handle multiple sockets concurrently...
My server code for listening for connections:
while(1)
{
struct sockaddr_in addr;
uint len = sizeof(addr);
SSL *ssl;
int client = accept(sock, (struct sockaddr*)&addr, &len);
if (client > 0)
{
std::cout<<"Client accepted..."<<std::endl;
}
else
{
perror("Unable to accept");
exit(EXIT_FAILURE);
}
ssl = SSL_new(ctx);
SSL_set_fd(ssl, client);
if (SSL_accept(ssl) <= 0)
{
std::cout<<"ERROR"<<std::endl;
}
else
{
char buff[1024];
SSL_read(ssl, buff, 1024);
std::cout<<buff<<std::endl;
std::string reply="Thanks from the server";
char buff_response[1024];
reply.copy(buff_response, 1024);
const void *buf=&buff_response;
SSL_write(ssl, buf, 1024);
char another_buff[1024];
SSL_read(ssl,another_buff,1024);
std::cout<<another_buff<<std::endl;
}
}
I've looked into 'select()', however this doesn't seem to allow concurrency as such, but allows the system to know when a socket is freed?
Does anyone have any experience in solving this basic problem?
First, with server code, it's important to differentiate between concurrency and parallelism. A reasonable server will typically handle many more connections concurrently than its number of cores. Consequently, it's important to make the code concurrent in the sense that it can (efficiently) handle many concurrent connections, in a way that does not rely on parallelism (in the sense of having each connection handled by a thread).
In this sense, select is actually a reasonable choice for concurrency, and gives you the effect of being non-blocking.
When your system handles multiple sockets concurrently, select indicates on which socket(s) you can perform operations such as send and recv without their blocking when you do so. If you use select well you won't have cases where your thread is idling, waiting indefinitely for some operation to proceed, while other sockets are ready.
The minimal example from gnu.org shows a reasonably efficient server which it seems you can adapt to your needs.
fd_set active_fd_set, read_fd_set;
FD_ZERO (&active_fd_set);
FD_ZERO (&read_fd_set);
// Use FD_SET to add sockets according to what you want to do with them
/* This call (checking to see who can be read) is the
* only thing that blocks. But if it does, no socket is ready for reading. */
if (select (FD_SETSIZE, &read_fd_set, NULL, NULL, NULL) < 0) {
// Handle error;
for (i = 0; i < FD_SETSIZE; ++i)
if (FD_ISSET (i, &read_fd_set))
// Here you can read without its blocking.
I am learning about IOCP and have decided to write my own wrapper class based on the following article:
http://www.codeproject.com/Articles/13382/A-simple-application-using-I-O-Completion-Ports-an
My project is a C++ TCP server using IOCP. The client uses send() and recv() to send and receive data which I cannot change (from what I've been told this shouldn't cause any problem, but I am mentioning it just in case). It also creates a socket using socket() (and not WSASocket()).
Everything seems to be working fine (no error with CreateIoCompletionPort, I can add a socket descriptor to the existing completion port without any error. I've checked everything by adding a call to WSAGetLastError() after each of these functions).
(Before anything, please don't mind the inconsistent coding style. I like to make stuff work first and then clean it all up.)
socket_ = WSASocket(AF_INET, SOCK_STREAM, IPPROTO_IP, NULL, 0, WSA_FLAG_OVERLAPPED);
setsockopt(socket_, IPPROTO_IP, SO_DEBUG | TCP_NODELAY, sockopt, 4);
ioctlsocket(socket_, FIONBIO, &ulSockMode_);
sin_.sin_family = AF_INET;
sin_.sin_port = htons((uint16_t)uiPort_));
hAccept_[0] = WSACreateEvent(); //only 1 event, I'm using an array for convenience
if (hAccept_ == WSA_INVALID_EVENT)
{
//this is never executed
}
WSAEventSelect(socket_, hAccept_[0], FD_ACCEPT);
After an incoming connection is detected (I use WSAWaitForMultipleEevents and WSAEnumNetworkEvents which work don't trigger any error), I use the following code to accept the client (and this is where the problems start):
SOCKET sock_client{ INVALID_SOCKET };
int32_t len_si{ sizeof(SOCKADDR_IN) };
//sock_client = accept(socket_, reinterpret_cast<SOCKADDR*>(pSockAddr), &len_si); // this works fine
//sock_client = sock_client = WSAAccept(socket_, reinterpret_cast<SOCKADDR*>(pSockAddr), &len_si, NULL, 0);//works fine too
char buf[2 * (sizeof(SOCKADDR_IN) + 16)];
WSAOVERLAPPED wsaovl;
uint32_t bytes{ 0 };
BOOL b = AcceptEx(socket_, sock_client, (PVOID)buf, 0, sizeof(SOCKADDR_IN) + 16, sizeof(SOCKADDR_IN) + 16, reinterpret_cast<LPDWORD>(&bytes), &wsaovl); //this fails, returns 0
int32_t test = WSAGetLastError(); // this returns 6 (WSA_INVALID_HANDLE)
I have no idea why it works with accept() and WSAAccept(), however it doesn't with AcceptEx().
If I use accept() though, after accepting the client I need to call WSARecv() immediately. I'm not sending anything back to the client just yet but I read that it needs to be called before GetQueuedCompletionStatus() in the worker thread:
WSABUF* buf = new WSABUF;
OVERLAPPED* ovl = new OVERLAPPED;
int32_t flags{ 0 };
int32_t bytes{ 0 };
int32_t bytes_recv = WSARecv(client_socket, buf, 1, &flags, &bytes, ovl, NULL); // this returns -1
int32_t err = WSAGetLastError(); // this returns 6 (WSA_INVALID_HANDLE)
And since this doesn't work, the GetQueuedCompletionStatus() routine in my worker thread keeps on hanging (or at least, I assume this is the reason)
Is there anything wrong with what I'm doing? I've been trying to search around and fix it since yesterday night, I know it's not a lot of time but I really don't see what I'm not doing correctly.
UPDATE:
I have changed the way I initialize my socket for AcceptEx().
SOCKET sock_client = WSASocket(AF_INET, SOCK_STREAM, IPPROTO_IP, NULL, 0, WSA_FLAG_OVERLAPPED);
and
WSAOVERLAPPED wsaovl = {};
AcceptEx() still returns false, however the error returned by WSAGetLastError() is now 997 (WSA_IO_PENDING). I'm not sure what I/O operation is pending exactly and how I would go about fixing it.
I had stumbled upon similar hurdle when I was learning I/O Completion Ports (IOCP)...
I think the problem is that, in the scheme of IOCP socket model, the most complicated part is the beginning phase of 'socket acceptance'. That is why most tutorial skip over it and begin the discussion on how to handle send/recv, instead.
If you want to develop sufficient understanding of IOCP so that you could implement a production software then my advice to you is to study it until you completely grasp it (this answer below is not enough). One document that I would recommend is chapter 5 of 'Network Programming for Microsoft Windows - 2nd edition'. The book may be old but valid for IOCP. Also, the article 'Windows via C/C++: Synchronous and Asynchronous Device I/O' touches some aspects of IOCP, though not enough information to do production software.
I will try to explain as best as I can, however, I must warn you that this may not be enough. Here it goes...
So, the part you are missing is "How to do 'socket acceptance' in an IOCP socket model".
First off, lets examine the typical Winsock (non-IOCP) sequence of calls on server;
// (1) Create listen socket on server.
WSASocket()
// (2) Bind an address to your listen socket.
bind()
// (3) Associate the listen socket with an event object on FD_ACCEPT event.
WSAEventSelect(,, FD_ACCEPT )
// (4) Put socket in listen state - now, Windows listening for new
// connection requests. If new requests comes, the associated
// event object will be set.
listen()
// (5) Wait on the event object associated on listen socket. This
// will get signaled when a new connection request comes.
WaitForSingleObject() {
// (6) A network activity has occurred. Verify that FD_ACCEPT has
// raised the event object. This also resets the event object
// so WaitForSingleObject() does not loop non-stop.
WSAEnumNetworkEvents()
// (7) Understanding this part is important. The WSAAccept() doesn't
// just accept connection, it first creates a new socket and
// then associates it with the newly accepted connection.
WSAAccept()
}
The step (7) is ok for non-IOCP based models. However, when looking it from the performance point of view - socket creation is expensive. And it slows down the connection acceptance process.
In IOCP model, sockets are created in advance for new incoming connection requests. Not only sockets are created in advance they are associated with the listen socket even before the connection request comes. To achieve this Microsoft has provided extension functions. Two such functions that are required for IOCP model are AcceptEx() & GetAcceptExSockaddrs().
Note: When using these extension functions they need to be loaded at runtime in order to avoid performance penalty. This can be achieved using WSAIoctl(). For further read refer to the MSDN documentation on AcceptEx().
Caveat: AcceptEx() can be used to set the new socket to receive some data as part of connection-acceptance process. This feature needs to be disabled as it makes application susceptible to DoS attack i.e., a connection request is issued but no data is sent. The receiving application will wait on that socket indefinitely. To avoid that just pass 0 value for its 'dwReceiveDataLength' parameter.
How to setup connection-acceptance code for IOCP model?
One way to do this is;
// (1) Create IO completion port
CreateIoCompletionPort()
// (2) Have a method that creates worker threads say 'CreateWorkerThreads()'.
// This assign same method (say WorkerThread_Func()) to all worker threads.
// In the WorkerThread_Func() threads are blocked on call to
// GetQueuedCompletionStatus().
CreateWorkerThreads()
// (3) Create listen socket.
WSASocket()
// (4) Associate listen socket to IO Completion Port created earlier.
CreateIoCompletionPort()
// (5) Bind an address to your listen socket.
bind()
// (6) Put socket in listen state - now, Windows listening for new
// connection requests. If a new request comes, GetQueuedCompletionStatus()
// will release a thread.
listen()
// (7) Create sockets in advance and call AcceptEx on each of
// these sockets. If a new connection requests comes
// Windows will pick one of these sockets and associate the
// connection with it.
//
// As an example, below loop will create 1000 sockets.
GUID GuidAcceptEx = WSAID_ACCEPTEX;
DWORD dwBytes;
LPFN_ACCEPTEX lpfnAcceptEx;
// First, load extension method.
int retCode = WSAIoctl(listenSocket,
SIO_GET_EXTENSION_FUNCTION_POINTER,
&GuidAcceptEx,
sizeof(GuidAcceptEx),
&lpfnAcceptEx,
sizeof(lpfnAcceptEx),
&dwBytes,
NULL,
NULL
);
for( /* loop for 1000 times */ ) {
SOCKET preemptiveSocket = WSASocket(, , , , , WSA_FLAG_OVERLAPPED);
lpfnAcceptEx(listenSocket, preemptiveSocket,,,,,,);
}
This essentially prepare your application to accept sockets in IOCP way. When a new connection requests comes one of the worker threads, that are waiting on GetQueuedCompletionStatus(), will be released and handed over the pointer to the data structure. This will have the socket that was created by lpfnAcceptEx().
Is the process complete? Not yet. The socket accepted through AcceptEx() call does not inherit properties of listenSocket. To do that you need to call;
setsockopt( acceptSocket, SOL_SOCKET, SO_UPDATE_ACCEPT_CONTEXT,
(char*)&listenSocket, sizeof(listenSocket) );
Now, acceptSocket is good to use for WSASend / WSARecv!
Something is missing! I skipped the part about how does a worker thread gets the acceptedSocket from GetQueuedCompletionStatus() ?
The answer is, by passing your specially crafted structure to lpfnAcceptEx(). When the GetQueuedCompletionStatus() returns it will have this data structure containing the socket that you will have passed.
How to make such a structure? By creating a structure having 'WSAOVERLAPPED' as its first member. You can have any members of your own after the first member. For example, my structure looked like;
typedef struct _WSAOVERLAPPEDPLUS
{
WSAOVERLAPPED ProviderOverlapped; // 'WSAOVERLAPPED' has to be the first member.
SOCKET client; // Use this to pass preemptive socket.
SOCKET listenSocket; // Use this to pass the listenSocket.
DWORD dwBytes;
SOCKET_OPERATION operation; // Enum to assist in knowing what socket operation ...
} WSAOVERLAPPEDPLUS, *LPWSAOVERLAPPEDPLUS;
...
typedef enum SOCKET_OPERATION {
UNINITIALIZED_ENUM, // To protect against memory leaks and uninitialized buffers.
OP_ACCEPTEX,
OP_RECEIVE,
OP_SEND
};
...
//
// So the previously mentioned for() loop will become;
//
for( /* loop for 1000 times */ ) {
SOCKET preemptiveSocket = WSASocket(, , , , , WSA_FLAG_OVERLAPPED);
LPWSAOVERLAPPEDPLUS pOl = new WSAOVERLAPPEDPLUS();
// Initialize our "extended" overlapped structure
memset(pOl, 0, sizeof(WSAOVERLAPPEDPLUS));
pOl->operation = OP_ACCEPTEX;
pOl->client = preemptiveSocket;
pOl->listenSocket = listenSocket;
int buflen = (sizeof(SOCKADDR_IN) + 16) * 2;
char* pBuf = new char[buflen];
memset(pBuf, 0, buflen);
m_lpfnAcceptEx(listenSocket,
preemptiveSocket,
pBuf,
0, // Passed 0 to avoid reading data on accept which in turn
// avoids DDoS attack i.e., connection attempt without data will
// cause AcceptEx to wait indefinitely.
sizeof(SOCKADDR_IN) + 16,
sizeof(SOCKADDR_IN) + 16,
&pOl->dwBytes,
&pOl->ProviderOverlapped
);
}
... and in the worker thread when GetQueuedCompletionStatus() returns;
while (TRUE)
{
bOk = ::GetQueuedCompletionStatus(hCompPort, &bytes_transferred, &completion_key, &pOverlapped, INFINITE);
if (bOk) {
// Process a successfully completed I/O request
if (completion_key == 0) {
// Safe way to extract the customized structure from pointer
// is to use 'CONTAINING_RECORD'. Read more on 'CONTAINING_RECORD'.
WSAOVERLAPPEDPLUS *pOl = CONTAINING_RECORD(pOverlapped, WSAOVERLAPPEDPLUS, ProviderOverlapped);
if (pOl->operation == OP_ACCEPTEX) {
// Before doing any WSASend/WSARecv, inherit the
// listen socket properties by calling 'setsockopt()'
// as explained earlier.
// The listenSocket and the preemptive socket are available
// in the 'pOl->listenSocket' & 'pOl->client', respectively.
}
delete pOl;
}
}
else {
// Handle error ...
}
I hope this gave you idea on how AcceptEx() is utilized with IOCP.
I'm using winsocks and I am coding an IDS/Honeypot, this is just a small section of it, because at the moment I want the server to listen on multiple sockets (7) and accept the connections, but I've tried to dynamically create the sockets with an array (and the listener etc) but I am still having trouble - I've tried it multiple ways but so far, all I've managed to do is get it working successfully on ONE socket, and LISTEN to all sockets, but not accept them.
So, this was my last attempt but not sure, maybe I need to use threads or declare the sockets differently?
So far, in this small test code, I want:
Initialize server
listen on all 7 ports (1111,2222 ...etc)
Accept an incoming connection on ANY of them
display both messages on client/server
drop the connection
and continue
It's a little sloppy I know, but here is the code so far and I think you can see where I am going with it:
#include <iostream>
#include <winsock2.h>
#include <string>
#pragma comment(lib, "ws2_32.lib")
int main()
{
std::cout<<"Honeypot server [test #1] by Dreamwalker"<<std::endl;
WSADATA wsa;
SOCKET s[7] , new_socket[7];
struct sockaddr_in server , client;
int c, port[7] = {1111,2222,3333,4444,5555,6666,7777};
char *message;
std::cout<<"\nInitialising Winsock and other components...";
if (WSAStartup(MAKEWORD(2,2),&wsa) != 0)
{
std::cout<<"Failed. Error Code :"<<WSAGetLastError()<<std::endl;
return 1;
}
//!IMPORTANT: create multiple new sockets on different ports
int i = 0;
for( i = 0; i < 7; i++)
{
//Create socket
if((s[i] = socket(AF_INET , SOCK_STREAM , 0 )) == INVALID_SOCKET)
{
std::cout<<"Could not create socket : "<< WSAGetLastError()<<std::endl;
}
//Prepare the sockaddr_in structure
server.sin_family = AF_INET;
server.sin_addr.s_addr = INADDR_ANY;
server.sin_port = htons( port[i] );
//Bind
if( bind(s[i] ,(struct sockaddr *)&server , sizeof(server)) == SOCKET_ERROR)
{
std::cout<<"Bind failed with error code : "<< WSAGetLastError()<<std::endl;
}
/*!ALL CREATION CHECKING DONE, now create multiple sockets on the server
and listen for connections*/
c = sizeof(struct sockaddr_in);
listen(s[i] , SOMAXCONN);
}
///ALL INITIALIZED
std::cout<<"DONE!"<<std::endl;
//Listen/accept incoming connections
std::cout<<"Now listening for connections"<<std::endl;
new_socket[i] = accept(s[i] , (struct sockaddr *)&client, &c);
if (new_socket[i] == INVALID_SOCKET)
{
std::cout<<"accept failed with error code : "<< WSAGetLastError()<<std::endl;
}
//Accepted connection
else{
std::cout<<"Someone has connected to this machine!"<<std::endl;
message = "Hello Client , I have received your connection.\n";
send(new_socket[i] , message , strlen(message) , 0);
closesocket(s[i]);
}
std::cout<<"FINISHED"<<std::endl;
WSACleanup();
getchar();
return 0;
}
And now it's throwing a runtime error as well:
WSAENOTSOCK
10038
Socket operation on nonsocket.
An operation was attempted on something that is not a socket. Either the socket handle parameter did not reference a valid socket,
or for select, a member of an fd_set was not valid.
Which (including debugging) indicates that the socket isn't declared properly when creating on an array, advice?
You code to create/bind/listen is all good. Then:
new_socket[i] = accept(s[i] , (struct sockaddr *)&client, &c);
Firstly, by the time this runs you're outside the loop, and i is 7 which is past the end of the array of sockets, which is why you get the not-a-socket error.
Secondly, accept() is a blocking call, so you can't just call accept() on all the sockets from the same thread the way you did for listen. You need to either have a separate thread block in accept() for each of the ports, or find out which one has a client connection attempt in progress using e.g. select (or epoll - does Windows have that?), then accept() a client on that specific socket (but then you've still got to either create a thread to handle the client read/recvs and write/sends or use select/epoll to find out when there's input ready to read, or more space in output buffers for transmission). There's also a race condition to be wary of if you use select/epoll - a listening socket might signal readiness for accepting a client connection, but by the time you call accept() that connection attempt's failed and forgotten, then if the listening socket hasn't been set to non-blocking mode it'll hang there waiting for another client to connect to that specific socket. IMHO, this is a case where threading is actually easier.
I think it's more "Windowsy" to use IO Completion Ports (you might want to Google), but AFAIK they're totally unportable. Winsock's not an exact match for BSD sockets, but the porting or dual-support effort's small.
I used two sockets in the same process, one acting as multicast server and the other acting as multicast client. I am trying to combine both server and client functionality in a single socket. Could not succeed till now. Did any one tried making a single socket as multicast server and client. Is it possible? If not any reference that states the limitation would be appreciated. Thank a lot.
It is possible, at least on windows. Try following example(without error checking):
SOCKET the_socket = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
sockaddr_in service;
service.sin_family = AF_INET;
service.sin_addr.s_addr = inet_addr("192.168.1.117");
service.sin_port = htons(56565);
int ret = bind(the_socket, (SOCKADDR *) &service, sizeof (service));
unsigned int grpaddr = inet_addr("224.4.4.44");
unsigned int iaddr = inet_addr("192.168.1.117");
//You should have the equivalent function in your existing client
//to join the multicast group
ret = join_source_group(the_socket, grpaddr, service.sin_addr.s_addr, iaddr);
std::string data = "AA";
sockaddr_in group;
group.sin_family = AF_INET;
group.sin_addr.s_addr = inet_addr("224.4.4.44");
group.sin_port = htons(56565);
ret = sendto(the_socket, data.c_str(), data.length(), 0, (const sockaddr*)&group, sizeof(group));
char recvdata[3] = {0};
ret = recv(the_socket, recvdata, 3, 0);
leave_source_group(the_socket, grpaddr, service.sin_addr.s_addr, iaddr);
"Server" and "Client" are not terms which really fit very comfortably into non connection-based sockets. Particularly with multicasting.
So a "Multicast receiver" and "Multicast sender" might be more appropriate.
You can of course, use the same socket to do both at once. It is not mandatory to join a group to send messages to it, but if you do, you may receive your own messages (in some OSs this is optional, for instancce using Linux's setsockopt IP_MULTICAST_LOOP).
"Server" and "client" are really concepts which work at a higher layer than multicasting. I hope that you can explain what you are trying to do better, maybe in another question.
Whether a multicast sender or receiver is the client or server, depends entirely on the application.
I am writing a small program that tests an UDP network service. The implementation of the service is allowed to create a new socket for the session and respond to the client from there, at which point the client is then required to talk to this address (similar to TFTP).
Minimal client sans error checking looks like this:
int fd = socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP);
sockaddr_in destaddr = { ... };
MSGBUF msg[] = { ... };
DWORD sent;
WSASendTo(fd, msg, sizeof msg / sizeof *msg, &sent, 0, (sockaddr *)sa, sizeof sa, 0, 0);
char buffer[4096];
MSGBUF rcvmsg = { sizeof buffer, buffer };
DWORD received;
sockaddr_storage sa;
socklen_t sa_len = sizeof sa;
DWORD flags = 0;
WSARecvFrom(fd, &rcvmsg, 1, &received, &flags, (sockaddr *)&sa, &sa_len, 0, 0);
The client works fine if the server responds from the same address and port that the initial message was sent to, however replies from another port are silently discarded and the client hangs in WSARecvFrom.
Explicitly binding the socket to { AF_INET, INADDR_ANY, 0 } to force assignment of a local port, or invoking listen(fd, 5); makes no difference, as expected.
Is there anything in WSASendTo that implicitly connects an UDP socket, and if so, what should I do to avoid this?
UDP doesn't have connections. Datagrams are sent to and from ports; it's one-way communication.
It sounds to me like your server is letting itself be assigned a temporary port (i.e. passing 0 as the port in sockaddr_in), instead of using a specific port. This won't work.
Since UDP has no concept of a connection, each time you send data, it could be sent from a different port; the first send doesn't reserve the port that it was given, it just sends a datagram from it and then lets it go.
Your server should be binding itself to a specific port.
Meh, it was a firewall issue. Adding the application to the list of programs allowed to receive incoming traffic fixed the issue.