I'am trying to do an integration via HTTP socket. I'am using python to create the socket client and send data to a socket server created in C.
As you can see in the following images, the integration documentation gives an example in C that shows how I must send the data to the server:
Integration documentation example:
1- define record / structure types for the message header and for each message format
2- Declare / Create a client socket object
3- Open the socket component in non blocking mode
4- declare a variable of the data structure type relevant to the API function you wish to call – then fill it with the correct data (including header). Copy the structure data to a byte array and send it through the socket
I've tried to do that using the ctypes module from python:
class SPMSifHdr(ctypes.Structure):
_fields_ = [
('ui32Synch1', ctypes.c_uint32),
('ui32Synch2', ctypes.c_uint32),
('ui16Version', ctypes.c_uint16),
('ui32Cmd', ctypes.c_uint32),
('ui32BodySize', ctypes.c_uint32)
]
class SPMSifRegisterMsg(ctypes.Structure):
_fields_ = [
('hdr1', SPMSifHdr),
('szLisence', ctypes.c_char*20),
('szApplName', ctypes.c_char*20),
('nRet', ctypes.c_int)
]
body_len = ctypes.sizeof(SPMSifRegisterMsg)
header = SPMSifHdr(ui32Synch1=0x55555555, ui32Synch2=0xaaaaaaaa, ui16Version=1, ui32Cmd=1, ui32BodySize=body_len)
body = SPMSifRegisterMsg(hdr1=header, szLisence='12345', szApplName='MyPmsTest', nRet=1)
socket_connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# config is a dict with the socket server connection params
socket_connection.connect((config.get('ip'), int(config.get('port'))))
socket_connection.sendall(bytearray(body))
socket_connection.recv(1024)
When I call the socket recv function it never receive anything, so I have used a windows tool to check the data that I sent and as you can see in the next image it seems any data is sent:
Socket sniff
I've tried to send even a simple "Hello! world" string and the result is always the same.
The socket connection is open. I know it because I can see how many connections are open from the server panel.
What am I doing wrong?
The error was that the SocketSniff program only shows the sent data if the server return a response. In it case the server did never return nothing because some bytes were missing.
I found it creating my own socket echo server and checking that the data I was sending were uncomplete.
Mystery solved. :D
Related
I have inherited two applications, one Test Harness (a client) running on a Windows 7 PC and one server application running on a Windows 10 PC. I am attempting to communicate between the two using TCP/IP sockets. The Client sends requests (for data in the form of XML) to the Server and the Server then sends the requested data (also XML) back to the client.
The set up is as shown below:
Client Server
-------------------- --------------------
| | Sends Requests | |
| Client Socket | -----------------> | Server Socket |
| | <----------------- | |
| | Sends Data | |
-------------------- --------------------
This process always works on an initial connection (i.e. freshly launched client and server applications). The client has the ability to disconnect from the server, which triggers cleanup of sockets. Upon reconnection, I almost always (it does not always happen, but does most of the time) receive the following error:
"Receive() - The socket is marked as nonblocking and the receive operation would block"
This error is displayed at the client and the socket in question is an asynchronous, non-blocking socket.
The line which causes this SOCKET_ERROR is:
numBytesReceived = theSocket->Receive(theReceiveBuffer, 10000));
where:
- numBytesReceived is an integer (int)
- theSocket is a pointer to a class called CClientSocket which is a specialisation of CASyncSocket, which is part of the MFC C++ Library. This defines the socket object which is embedded within the client. It is an asynchonous, non-blocking socket.
- Receive() is a virtual function within the CASyncSocket object
- theReceiveBuffer is a char array (10000 elements)
In executing the line descirbed above, SOCKET_ERROR is returned from the function and calling theSocket->GetLastError() returns WSAEWOULDBLOCK.
SocketTools highlights that
When a non-blocking (asynchronous) socket attempts to perform an operation that cannot be performed immediately, error 10035 will be returned. This error is not fatal, and should be considered advisory by the application. This error code corresponds to the Windows Sockets error WSAEWOULDBLOCK.
When reading data from a non-blocking socket, this error will be returned if there is no more data available to be read at that time. In this case, the application should wait for the OnRead event to fire which indicates that more data has become available to read. The IsReadable property can be used to determine if there is data that can be read from the socket.
When writing data to a non-blocking socket, this error will be returned if the local socket buffers are filled while waiting for the remote host to read some of the data. When buffer space becomes available, the OnWrite event will fire which indicates that more data can be written. The IsWritable property can be used to determine if data can be written to the socket.
It is important to note that the application will not know how much data can be sent in a single write operation, so it is possible that if the client attempts to send too much data too quickly, this error may be returned multiple times. If this error occurs frequently when sending data it may indicate high network latency or the inability for the remote host to read the data fast enough.
I am consistently getting this error and failing to receive anything on the socket.
Using Wireshark, the following communications occur with the source, destinaton and TCP Bit Flags presented here:
Event: Connect Test Harness to Server via TCP/IP
Client --> Server: SYN
Server --> Client: SYN, ACK
Client --> Server: ACK
This appears to be correct and represents the Three-Way Handshake of connecting.
SocketSniff confirms that a Socket is closed on the client side. It was not possible to get SocketSniff to work with the Windows 10 Server application.
Event: Send a Request for Data from the Test Harness
Client --> Server: PSH, ACK
Server --> Client: PSH, ACK
Client --> Server: ACK
Both request data and received data is confirmed to be exchanged successfully
Event: Disconnect Test Harness from Server
Client --> Server: FIN, ACK
Server --> Client: ACK
Server --> Client: FIN, ACK
Client --> Server: ACK
This appears to be correct and represents the Four-Way handshake of connection closure.
SocketSniff confirms that a Socket is closed on the client side. It was not possible to get SocketSniff to work with the Windows 10 Server application.
Event: Reconnect Test Harness to Server via TCP/IP
Client --> Server: SYN
Server --> Client: SYN, ACK
Client --> Server: ACK
This appears to be correct and represents the Three-Way Handshake of connecting.
SocketSniff confirms that a new Socket is opened on the client side. It was not possible to get SocketSniff to work with the Windows 10 Server application.
Event: Send a Request for Data from the Test Harness
Client --> Server: PSH, ACK
Server --> Client: ACK
We see no data being pushed (PSH) back to the client, yet we do see an acknowledgement.
Has anyone got any ideas what may be going on here? I understand it would be difficult for you to diagnose without seeing the source code, however I was hoping others may have had experience with this error and could point me down the specific route to investigate.
More Info:
The Server initialises a listening thread and binds to 0.0.0.0:49720. The 'WSAStartup()', 'bind()' and 'listen()' functions all return '0', indicating success. This thread persists throughout the lifetime of the server application.
The Server initialises two threads, a read and a write thread. The read thread is responsible for reading request data off its socket and is initialised as follows with a class called Connection:
HANDLE theConnectionReadThread
= CreateThread(NULL, // Security Attributes
0, // Default Stacksize
Connection::connectionReadThreadHandler, // Callback
(LPVOID)this, // Parameter to pass to thread
CREATE_SUSPENDED, // Don't start yet
NULL); // Don't Save Thread ID
The write thread is initialised in a similar way.
In each case, the CreateThread() function returns a suitable HANDLE, e.g.
theConnectionReadThread = 00000570
theConnectionWriteThread = 00000574
The threads actually get started within the following function:
void Connection::startThreads()
{
ResumeThread(theConnectionReadThread);
ResumeThread(theConnectionWriteThread);
}
And this function is called from within another class called ConnectionManager which manages all the possible connections to the server. In this case, I am only concerned with a single connection, for simplicity.
Adding text output to the server application reveals that I can successfully connect/disconnect the client and server several times before the faulty behaviour is observed. For example, Within the connectionReadThreadHandler() and connectionWriteThreadHandler() functions, I am outputing text to a log file as soon as they execute.
When correct behaviour is observed, the following lines are output to the log file:
Connection::ResumeThread(theConnectionReadThread) returned 1
Connection::ResumeThread(theConnectionWriteThread) returned 1
ConnectionReadThreadHandler() Beginning
ConnectionWriteThreadHandler() Beginning
When faulty behaviour is observed, the following lines are output to the log file:
Connection::ResumeThread(theConnectionReadThread) returned 1
Connection::ResumeThread(theConnectionWriteThread) returned 1
The callback functions do not appear to being invoked.
It is at this point that the error is displayed on the client indicating that:
"Receive() - The socket is marked as nonblocking and the receive operation would block"
On the Client side, I've got a class called CClientDoc, which contains the client side socket code. It first initialises theSocket which is the socket object which is embedded within a client:
private:
CClientSocket* theSocket = new CClientSocket;
When a connection is initialised between client and server, this class calls a function called CreateSocket() part of which is included below, along with ancillary functions which it calls:
void CClientDoc::CreateSocket()
{
AfxSocketInit();
int lastError;
theSocket->Init(this);
if (theSocket->Create()) // Calls CAyncSocket::Create() (part of afxsock.h)
{
theErrorMessage = "Socket Creation Successful"; // this is a CString
theSocket->SetSocketStatus(WAITING);
}
else
{
// We don't fall in here
}
}
void CClientDoc::Init(CClientDoc* pDoc)
{
pClient = pDoc; // pClient is a pointer to a CClientDoc
}
void CClientDoc::SetSocketStatus(SOCKET_STATUS sock_stat)
{
theSocketStatus = sock_stat; // theSocketStatus is a private member of CClientSocket of type SOCKET_STATUS
}
Immediately after CreateSocket(), SetupSocket() is called which is also provided here:
void CClientDoc::SetupSocket()
{
theSocket->AsyncSelect(); // Function within afxsock.h
}
Upon disconnection of the client from the server,
void CClientDoc::OnClienDisconnect()
{
theSocket->ShutDown(2); // Inline function within afxsock.inl
delete theSocket;
theSocket = new CClientSocket;
CreateSocket();
SetupSocket();
}
So we delete the current socket and then create a new one, ready for use, which appears to work as expected.
The error is being written on the Client within the DoReceive() function. This function calls the socket to attempt to read in a message.
CClientDoc::DoReceive()
{
int lastError;
switch (numBytesReceived = theSocket->Receive(theReceiveBuffer, 10000))
{
case 0:
// We don't fall in here
break;
case SOCKET_ERROR: // We come in here when the faulty behaviour occurs
if (lastError = theSocket->GetLastError() == WSAEWOULDBLOCK)
{
theErrorMessage = "Receive() - The socket is marked as nonblocking and the receive operation would block";
}
else
{
// We don't fall in here
}
break;
default:
// When connection works, we come in here
break;
}
}
Hopefully the addition of some of the code proves insightful. I should be able to add a bit more if needed.
Thanks
The WSAEWOULDBLOCK error DOES NOT mean the socket is marked as blocking. It means the socket is marked as non-blocking and there is NO DATA TO READ at that time.
WSAEWOULDBLOCK means the socket WOULD HAVE blocked the calling thread waiting for data if the socket HAD BEEN marked as blocking.
To know when a non-blocking socket has data waiting to be read, use Winsock's select() function, or the CClientSocket::AsyncSelect() method to request FD_READ notifications, or other equivalent. Don't try to read until there is something to read.
In your analysis, you see the client sending data to the server, but the server is not sending data to the client. So you clearly have a logic bug in your code somewhere, you need to find and fix it. Either the client is not terminating its request correctly, or the server is not receiving/processing/replying to it correctly. But since you did not show your actual code, we can't tell you what is actually wrong with it.
I'm trying to create a UDP socket with boost to send and receive data. Here's the relevant code:
// Create IO service
mIoService = std::make_unique<boost::asio::io_service>();
// Create local endpoint on random port
mLocalEndpoint = std::make_unique<udp::endpoint>(udp::v4(), 0);
// Create socket
mSocket = std::make_unique<udp::socket>(*mIoService, *mLocalEndpoint);
mSocket->open(mLocalEndpoint->protocol());
This gives me the following exception:
open: Already open
I have fiddled around with this code for quite some time now, but the result stays the same. What am I missing here?
It looks like you try to create a UDP end point on port 0. I'd assume that it may be already used, so you can try to change a port:
mLocalEndpoint = std::make_unique<udp::endpoint>(udp::v4(), 40000);
I'm beginner to Networking. We have sender and receiver application. I have captured the packets sent by sender to receiver using WinDump.
I'm writing a python sender application which will fuzz the packets sent by sender to receiver.
I just want to confirm, can I directly put the packet data obtained using WinDump to socket send() method.
Say, "arp who-has host1 tell host2" is the packet obtained by WinDump
Can I write,
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
s.send(arp who-has host1 tell host2)
In general pyhton network programming you can not pass as the arguments using send method.you just only pass the string as arguments as variable.
Basically we are using the send() and sendall() method to transmits the TCP messages.
I have mentioned the below desciption for both the method.
socket send() method :
The socket's send() method is not guaranteed to send all of the data you pass it. Instead, it returns the number of bytes that were actually sent and expects your application to handle retransmission of the unsent portion
from socket import socket
sock = socket()
sock.connect(('1.2.3.4', 1234))
sock.send('My Name is Dasadiya Chaitanya !!!\n'')
sock.close()
socket sendall() method :
which is same work as the send() method but python provides a convenience method called sendall() that makes sure all of your data is sent before returning and also provide the guarantee to send all the data to the receiver.
from socket import socket
sock = socket()
sock.connect(('1.2.3.4', 1234))
sock.sendall('My Name is Dasadiya Chaitanya !!!\n')
sock.close()
I hope this should helpful for you :)
Is it right method client send data using the same connection accepted by server?.
The situation is like this, I have blue tooth server running on my PC and on the other side I have android phone with client and server. From android side the client start connection. I am using blue-tooth chat example from android samples.
And the server-client on android look like
BluetoothSocket socket;
InputStream tmpIn = null;
OutputStream tmpOut = null;
// Get the BluetoothSocket input and output streams
tmpIn = socket.getInputStream();
tmpOut = socket.getOutputStream();
And in the PC side I am using Bluez libraries to implement server and client.
The code includes blue tooth receive thread and a main thread, whenever the server accept a connection from android phone I just assign the socket value to a global variable and whenever the client need to send data it send using the same socket ,
Server:-
int GLOBAL_CLIENT;
void* recive_bluetooth_trd(void*)
{
...............................
..............................
client = accept(s, (struct sockaddr *)&rem_addr, &opt);
GLOBAL_CLIENT=client;
while(1){
bytes_read = read(client, buf, sizeof(buf));
....................
...................
}
Client:-
void clinet(char *msg, int length){
........................
int bytes_write=write(GLOBAL_CLIENT,message, length);
..........................
}
My question is, Is it a right method ? The problem is that some times the client send data successfully from PC but not receiving on android side.
The biggest problem I see is that you won't ever leave your while(1) loop, even when the client disconnects. Read will return immediately forever with 0 bytes read (check for a return code of <= 0), trying to signal that the socket is disconnected. Your code will go into a tight infinite loop and use up all the CPU resources it can get its single-threaded hands on.
You need to make sure you ALWAYS check your socket and IO return codes and handle the errors correctly. The error handling for sockets is usually about 3x the actual socket code.
Unless of course the .......... stuff is the important bits. Always tough to tell when people hide code relevant to the question they are asking.
Seems correct to me, but after read you have to NUL ('\0') terminate your buffer if you are treating with strings:
buf[bytes_read] = '\0';
I am writing an XMLRPC client in c++ that is intended to talk to a python XMLRPC server.
Unfortunately, at this time, the python XMLRPC server is only capable of fielding one request on a connection, then it shuts down, I discovered this thanks to mhawke's response to my previous query about a related subject
Because of this, I have to create a new socket connection to my python server every time I want to make an XMLRPC request. This means the creation and deletion of a lot of sockets. Everything works fine, until I approach ~4000 requests. At this point I get socket error 10048, Socket in use.
I've tried sleeping the thread to let winsock fix its file descriptors, a trick that worked when a python client of mine had an identical issue, to no avail.
I've tried the following
int err = setsockopt(s_,SOL_SOCKET,SO_REUSEADDR,(char*)TRUE,sizeof(BOOL));
with no success.
I'm using winsock 2.0, so WSADATA::iMaxSockets shouldn't come into play, and either way, I checked and its set to 0 (I assume that means infinity)
4000 requests doesn't seem like an outlandish number of requests to make during the run of an application. Is there some way to use SO_KEEPALIVE on the client side while the server continually closes and reopens?
Am I totally missing something?
The problem is being caused by sockets hanging around in the TIME_WAIT state which is entered once you close the client's socket. By default the socket will remain in this state for 4 minutes before it is available for reuse. Your client (possibly helped by other processes) is consuming them all within a 4 minute period. See this answer for a good explanation and a possible non-code solution.
Windows dynamically allocates port numbers in the range 1024-5000 (3977 ports) when you do not explicitly bind the socket address. This Python code demonstrates the problem:
import socket
sockets = []
while True:
s = socket.socket()
s.connect(('some_host', 80))
sockets.append(s.getsockname())
s.close()
print len(sockets)
sockets.sort()
print "Lowest port: ", sockets[0][1], " Highest port: ", sockets[-1][1]
# on Windows you should see something like this...
3960
Lowest port: 1025 Highest port: 5000
If you try to run this immeditaely again, it should fail very quickly since all dynamic ports are in the TIME_WAIT state.
There are a few ways around this:
Manage your own port assignments and
use bind() to explicitly bind your
client socket to a specific port
that you increment each time your
create a socket. You'll still have
to handle the case where a port is
already in use, but you will not be
limited to dynamic ports. e.g.
port = 5000
while True:
s = socket.socket()
s.bind(('your_host', port))
s.connect(('some_host', 80))
s.close()
port += 1
Fiddle with the SO_LINGER socket
option. I have found that this
sometimes works in Windows (although
not exactly sure why):
s.setsockopt(socket.SOL_SOCKET,
socket.SO_LINGER, 1)
I don't know if this will help in
your particular application,
however, it is possible to send
multiple XMLRPC requests over the
same connection using the
multicall method. Basically
this allows you to accumulate
several requests and then send them
all at once. You will not get any
responses until you actually send
the accumulated requests, so you can
essentially think of this as batch
processing - does this fit in with
your application design?
Update:
I tossed this into the code and it seems to be working now.
if(::connect(s_, (sockaddr *) &addr, sizeof(sockaddr)))
{
int err = WSAGetLastError();
if(err == 10048) //if socket in user error, force kill and reopen socket
{
closesocket(s_);
WSACleanup();
WSADATA info;
WSAStartup(MAKEWORD(2,0), &info);
s_ = socket(AF_INET,SOCK_STREAM,0);
setsockopt(s_,SOL_SOCKET,SO_REUSEADDR,(char*)&x,sizeof(BOOL));
}
}
Basically, if you encounter the 10048 error (socket in use), you can simply close the socket, call cleanup, and restart WSA, the reset the socket and its sockopt
(the last sockopt may not be necessary)
i must have been missing the WSACleanup/WSAStartup calls before, because closesocket() and socket() were definitely being called
this error only occurs once every 4000ish calls.
I am curious as to why this may be, even though this seems to fix it.
If anyone has any input on the subject i would be very curious to hear it
Do you close the sockets after using it?