I have written a program making an http POST request to google-analytics using the following code:
Poco::Net::HTTPClientSession session("www.google-analytics.com");
Poco::Net::HTTPRequest req(Poco::Net::HTTPRequest::HTTP_POST, "/collect",Poco::Net::HTTPMessage::HTTP_1_1);
Poco::URI uri("");
uri.addQueryParameter("v", "1");
////// more parameters of the uri //////
req.setContentLength(uri.getRawQuery().length());
session.sendRequest(req) << uri.getRawQuery();
A user of the program has reported a 20 seconds freeze of the program every time a request is made. I have not been able to replicate the problem on any other computer, nor understand where it could come from.
He uses an ethernet connection to a modem TP-Link 300Mbps Wireless N Gigabit ADSL2+Modem Router TD-W8970. When the modem is turned off, the lag does not occur (my code throws an exception when sendRequest fails).
Any help/suggestion on the problem would be much appreciated!
Thanks.
Related
When working with Binance REST API using WinHTTP, I noticed that requests sometimes are failed with the error
12030 the connection with the server was terminated abnormally
Each request is made like this: (the complete example is here)
WinHttpOpen(...)
WinHttpConnect(...)
{... making request itself; }
if (hRequest) WinHttpCloseHandle(hRequest);
if (hConnect) WinHttpCloseHandle(hConnect);
if (hSession) WinHttpCloseHandle(hSession);
However, despite I close handles, the connection is reused (which I see in the HttpAnalyzer).
I suppose that this is the issue: the server starts dropping connection after N requests or seconds, accoring to server's keep-alive settings; At the same moment my client starts new request which failed with 12030.
Moreover. inspesting successful requests headers, I noticed that
Connection: Close
is passed from the server from time to time; I'd like to reconnect in such cases.
The question is: how do I prevent WInHTTP from trying to reuse connection which is about to be dropped by server?
Changing the registry "TcpTimedWaitDelay" parameter would not be efficient because request count per time may vary.
The same quesstion is actual for Delphi TNetHTTPClient (which uses WinHTTP) and Indy TIdHTTP.
As for Indy, releasing of the TIdHTTP object does not close the connection; however in this case we have source codes. The following patch in the IdSSLOpenSSL.pas solves it:
If SSL_shutdown(fSSL) = 0
then SSL_shutdown(fSSL);
// SSL_shutdown(fSSL); <= was in original Indy10 IdSSLOpenSSL
EDIT5:
I eventually fixed this issue by more or less throwing away half my code. Rather than sending data to a ruby server using HTTP, I'm now using MQTT to a broker to a NodeJS server. The NodeJS part here isn't important but to anyone else having this issue I STRONGLY recommend sending all IoT data using MQTT, and that's what solved my issue.
I'm currently trying to send data collected from sensors on an Arduino WiFi rev2, to my rails server hosted on Heroku. I do this by sending my data in a JSON format. My problem is that while my methods seem to work initially, with the first few POST requests being received and processed fine, after 2-3 requests the arduino hangs, and I receive status code: -2. I'm using the ArduinoHttpClient library.
I've tried using a local server, which has the same problem, aswell as sending the POST request via both curl and postman. Both curl and postman seem to work as expected, so I imagine the issue is with the arduino code although I can't be sure.
client.beginRequest();
client.post("/input");
client.sendHeader("Content-Type", "application/json");
client.sendHeader("Content-Length", postData.length());
client.beginBody();
client.println(postData);
client.endRequest();
LED(0,128,0);
Serial.println("Gone");
int statusCode = client.responseStatusCode();
String response = client.responseBody();
Serial.print("Status code: ");
Serial.println(statusCode);
Serial.print("Response: ");
Serial.println(response);
When this code fails, the arduino will hang for about 20-40 seconds and I will receive 'status code -3' from the Serial. However I have also received status code -2 and -4 in the past.
When it does succeed I receive the following: "Status code: 204" which is what I would expect.
EDIT:
I've since tried posting to requestcatcher.com, and the problem persisted. I'm therefore fairly confident this is an arduino problem, I also received the following output:
POST /input HTTP/1.1
Host: arduino.requestcatcher.com
Connection: close
Connection: close
Content-Length: 88
Content-Type: application/json
User-Agent: Arduino/2.2.0
{"inputs":[{"input_id":"1","value":1.778002}{"input_id":"2","value":18.037}],"id":"13"}
EDIT 2:
I accidentally discovered that the POST requests go through fine if the "Content-Length:" Header is omitted. Obviously no JSON actually gets sent so this does not fix my issue but it is likely that this header or the JSON itself is the issue.
EDIT 3:
Regardless of server I receive either status code -4, or -3, even on request catcher.
EDIT 4:
After various adjustments, code now looks as below. This seems to have helped a little and it fails less often but still does fail. I'm beginning to wonder if this is a problem with ArduinoHttpClient.
String postData = "";
serializeJson(doc, postData);
serializeJson(doc, Serial);
Serial.println(postData)
client.post("/input", "application/json", postData.c_str());
LED(0,128,0);
Serial.println("Gone");
int statusCode = client.responseStatusCode();
Serial.print("Status code: ");
Serial.println(statusCode);
client.stop();
doc.clear();
lastCycle = millis();
Try replacing
client.beginRequest();
client.post("/input");
client.sendHeader("Content-Type", "application/json");
client.sendHeader("Content-Length", postData.length());
client.beginBody();
client.println(postData);
client.endRequest();
with just
String contentType = "application/json";
client.post("/input", contentType, postData);
or
client.post("/input", "application/json", postData.c_str());
You don't need to explicitly specify the request headers - or call beginRequest(), etc. - when using the post() method(s) in that library.
I'm working on Boost Asio and Boost Beast for simple RESTful server. For normal HTTP and TCP socket, it works perfectly. I put it under load test with JMeter, everything works fine.
I tried to add the SSL socket. I set the 'ssl::context' and also called the 'async_handshake()' - additional steps for SSL compared to normal socket. It works for the first time only. Client can connected with me (server) and I also able to receive the data via 'boost::beast::http::async_read()'.
Because this is RESTful, so the connection will drop after the request & respond. I call 'SSL_Socket.shutdown()' and follow by 'SSL_Socket.lowest_layer().close()' to close the SSL socket.
When the next incoming request, the client able to connect with me (server). I called 'SSL_Socket.async_handshake()' and then follow by 'boost::beast::http::async_read()'. But this time I not able to receive any data. But the connection is successfully established.
Anyone has any clue what i missed?
Thank you very much!
If you want to reuse the stream instance, you need to manipulate SSL_Socket.native_handle() with openssl lib function. After ssl shutdown, use SSL_clear() before start a new ssl handshake.
please read(pay attention to warnings) link for detail
SSL_clear() resets the SSL object to allow for another connection. The reset operation however keeps several settings of the last sessions (some of these settings were made automatically during the last handshake)
.........
WARNINGS
SSL_clear() resets the SSL object to allow for another connection. The reset operation however keeps several settings of the last sessions (some of these settings were made automatically during the last handshake). It only makes sense for a new connection with the exact same peer that shares these settings, and may fail if that peer changes its settings between connections. Use the sequence SSL_get_session(3); SSL_new(3); SSL_set_session(3); SSL_free(3) instead to avoid such failures (or simply SSL_free(3); SSL_new(3) if session reuse is not desired).
In regard to ssl shutdown issue, link explain how boost asio ssl shutdown work.
In Boost.Asio, the shutdown() operation is considered complete upon error or if the party has sent and received a close_notify message.
If you look at boost.asio (1.68) source code boost\asio\ssl\detail\impl\engine.ipp, it shows how boost.asio do ssl shutdown and stream_truncated happens when there is data to be read or ssl shutdown expected from peer not received.
int engine::do_shutdown(void*, std::size_t)
{
int result = ::SSL_shutdown(ssl_);
if (result == 0)
result = ::SSL_shutdown(ssl_);
return result;
}
const boost::system::error_code& engine::map_error_code(
boost::system::error_code& ec) const
......
// If there's data yet to be read, it's an error.
if (BIO_wpending(ext_bio_))
{
ec = boost::asio::ssl::error::stream_truncated;
return ec;
}
......
// Otherwise, the peer should have negotiated a proper shutdown.
if ((::SSL_get_shutdown(ssl_) & SSL_RECEIVED_SHUTDOWN) == 0)
{
ec = boost::asio::ssl::error::stream_truncated;
}
}
Also you can see boost.asio ssl shutdown routine may call openssl SSL_shutdown() twice if first return 0, openssl document allows it but advice call SSL_read() to do a bidirectional shutdown if first SSL_shutdown() returns 0.
Read link for details.
I had a similar issue, the 2nd time onward my asynchonous accept always failed with session id uninitialized.
I solved this problem calling SSL_CTX_set_session_id_context on context or
setting context cache mode with SSL_SESS_CACHE_OFF and SSL_OP_NO_TICKET on context options.
This is my cents to someone else's problem.
I managed to resolve the problem by switching 'ssl::stream' socket to 'boost::optional' and then added 'SSL_Socket.emplace(io_context, oSSLContext)' each time the socket is shutdown and closed.
Big credit to sehe at 'Can't implement boost::asio::ssl::stream<boost::asio::ip::tcp::socket> reconnect to server'. His statement "the purest solution would be to not reuse the stream/socket objects" rocks! Save my time.
Thanks.
Ref: I have a RTSP media server that sends video via TCP streaming.I am using rtsp-sink(RTSPStreamer) DirectShow.Net filter based RTSP server which is developed in C++.Where as The Wrapper app is developed using C#.
The problem i am facing is the moment RTSP server start streaming,it affects the system level internet connection & drops internet connection speed by 90 percent.
I wanted to get your input on how it would be possible? (if at all).Because it impacts on system level internet connection not the very app level.
For Example:- My normal internet connection speed is 25 mbps. It suddenly drops to 2 mbps , whenever the RTSP streaming started in the app's server tab.
Sometimes even disables the internet connection in the system(Computer) where the app is running.
I'm asking you because I consider you an expert so please bear with me on this "maybe wild" question and thanks ahead.
...of all the things I've lost, I miss my mind the most.
Code Snippet of RTSPSender.CPP
//////////////////////////////////////////////////////
// CStreamingServer
//////////////////////////////////////////////////////
UsageEnvironment* CStreamingServer::s_pUsageEnvironment = NULL;
CHAR CStreamingServer::s_szDefaultBroadCastIP[] = "239.255.42.42";
//////////////////////////////////////////////////////
CStreamingServer::CStreamingServer(HANDLE hQuit)
: BasicTaskScheduler(10000)
, m_hQuit(hQuit)
, m_Streams(NAME("Streams"))
, m_bStarting(FALSE)
, m_bSessionReady(FALSE)
, m_pszURL(NULL)
, m_rtBufferingTime(UNITS * 2)
{
s_pUsageEnvironment = BasicUsageEnvironment::createNew(*this);
rtspServer = NULL;
strcpy_s(m_szAddress,"");
strcpy_s(m_szStreamName,"stream");
strcpy_s(m_szInfo,"media");
strcpy_s(m_szDescription,"Session streamed by \"RTSP Streamer DirectShow Filter\"");
m_nRTPPort = 6666;
m_nRTSPPort = 8554;
m_nTTL = 1;
m_bIsSSM = FALSE;
}
Edited:WireShark logs:
WireShark logs at the time of RTSP streaming started
The following code is inside a thread and reads input data coming over usb. Approximately every 80 readings it misses one of the packets coming from an stm32 board. The board is programmed to send data packets to the android tablet every one second.
// Non Working Code
while(running){
int resp = bulktransfer(mInEp,mBuf,mBuf.lenght,1000);
if(resp>0){
dispatchMessage(mBuf);
}else if(resp<0)
showsBufferEmptyMessage();
}
I was looking the Missile Launcher example in android an other libraries on the internet and they put a delay of 50ms between each poll. Doing this it solves the missing package problem.
//Working code
while(running){
int resp = bulktransfer(mInEp,mBuf,mBuf.lenght,1000);
if(resp>0){
dispatchMessage(mBuf);
}else if(resp<0)
showsBufferEmptyMessage();
try{
Thread.sleep(50);
}catch(Exception e){}
}
Does anyone knows the reason why the delay works. Most of the libraries on github has this delay an as I mention before the google example too.
I am putting down my results regarding this problem. After all seems that the UsbConnection.bulkTransfer(...) method has some bug when called continuously. The solution was to use the asynchronous API, UsbRequest class. Using this method I could read from the input endpoint without delay and no data was lost during the whole stress test. So the direction to take is asynchronous UsbRequest instead of synchronously bulktransfer.