I developed a program that makes parameters tracking. I want to inform a server with a http message over udp when a parameter value changes.
I want to use libcurl for that. Does libcurl able to send a http message over UDP?
No it doesn't. For HTTP, libcurl only supports TCP or Unix domain socket. It could possibly be something to add in a future.
(libcurl supports UDP transfers for a few other protocols.)
HTTP could in theory be used over a different protocol than TCP (this case is mentioned in RFC2616), but it needs to be a reliable protocol, that is a protocol which provides guaranteed delivery, must keep the order of messages and must detect duplicates. Plain UDP does not provide these things and thus it is not possible to use HTTP over plain UDP and thus libcurl does not provide it.
If you are interested in a protocol with a syntax close to HTTP, but with support of UDP, then have a look at SIP which is used for VoIP.
Related
Okay, so a little context:
I have an app running on an embedded system that sends a few different requests over HTTP (using libcurl in C++) at the following intervals:
5 minutes
15 minutes
1 hour
24 hours
My goal: Reduce data consumption (runs over cellular)
We have both client and server side TLS authentication, so the handshake is costly. The idea is that we use persistent connections (at least for the shorter interval files) to avoid doing the handshake every time.
Unfortunately, after much tinkering I've figured out that the server is closing the connection before the intervals pass. Maybe this is something we can extend? I'll have to talk to the server side guys.
I was under the impression that was the reason the "TCP keep-alive" packets existed, but supposedly those "check the connection" not "keep it open" like the name suggests.
My idea is this:
Have my app send a packet (as small as possible) every 2 minutes or so (however long the timeout is) to "nudge" the connection into staying open.
My questions are:
Does that make any sense?
I don't suppose there is an easy way to do this in libcurl is there?
If so, how small could we get the request?
Is there an even easier way to do it? My only issue here is that all the connection stuff "lives" in libcurl.
Thanks!
It would be easier to give a more precise answer if you gave a little more detail on your application architecture. For example, is it a RESTful API? Is the use of HTTP absolutely mandatory? If so, what HTTP server are you using (nginx, apache, ...)? Could you consider websockets as an alternative to plain HTTP?
If you are at liberty to use something other than regular HTTP or HTTPs - and to use something other than libcurl on the client side - you would have more options.
If, on the other hand, if you are constrained to both
use HTTP (rather than a raw TCP connection or websockets), and
use libcurl
then I think your task is a good bit more difficult - but maybe still possible.
One of your first challenges is that the typical timeouts for a HTTP connection are quite low (as low as a few seconds for Apache 2). If you can configure the server you can increase this.
I was under the impression that was the reason the "TCP keep-alive" packets existed, but supposedly those "check the connection" not "keep it open" like the name suggests.
Your terminology is ambiguous here. Are you referring to TCP keep-alive packets or persistent HTTP connections? These don't necessarily have anything to do with each other. The former is an optional mechanism in TCP (which is disabled by default). The latter is an application-layer concept which is specific to HTTP - and may be used regardless of whether keep-alive packets are being used at the transport layer.
My only issue here is that all the connection stuff "lives" in libcurl.
The problem with using libcurl is that it first and foremost a transfer library. I don't think it is tailored for long-running, persistent TCP connections. Nonetheless, according to Daniel Stenberg (the author of libcurl), the library will automatically try to reuse existing connections where possible - as long as you re-use the same easy handle.
If so, how small could we get the request?
Assuming you are using a 'ping' endpoint on your server - which accepts no data and returns a 204 (success but no content) response, then the overhead - in the application layer - would be the size of the HTTP request headers + the size of the HTTP response headers. Maybe you could get it down to 200-300 bytes, or thereabouts.
Alternatives to (plain) HTTP
If you are using a RESTful API, this paradigm sort of goes against the idea of a persistent TCP connection - although I can not think of any reason why it would not work.
You might consider websockets as an alternative, but - again - libcurl is not ideal for this. Although I know very little about websockets, I believe they would offer some advantages.
Compared to plain HTTP, websockets offer:
significantly less overhead than HTTP per message;
the connection is automatically persistent: there is no need to send extra 'keep alive' messages to keep it open;
Compared to a raw TCP connection, the benefits of websockets are that:
you don't have to open a custom port on your server;
it automatically handles the TLS/SSL stuff for you.
(Someone who knows more about websockets is welcome to correct me on some of the above points - particularly regarding TLS/SSL and keep alive messages.)
Alternatives to libcurl
An alternative to libcurl which might be useful here is the Mongoose networking library. It would provide you with a few different alternatives:
use a plain TCP connection (and a custom application layer protocol),
use a TCP connection and handle the HTTP requests yourself manually,
use websockets - which it has very good support for (both as server and client).
Mongoose allows you to enable SSL for all of these options also.
Libcurl allows to get information how much bytes application level protocol (HTTP, FTP, etc) sent and received. However, is there any way to get amount of bytes that underlying socket sent and received? I am about all data, including, for example, bytes that socket used to establish SSL connection. So, I am basically searching a way to get the same information from libcurl that Apache HTTP client gives in HttpConnectionMetrics.getSentBytesCount() and HttpConnectionMetrics.getReceivedBytesCount().
One idea is to access socket that Libcurl uses directly from C++; however, how to get this socket total sent/received bytes count?
Use CURLOPT_DEBUGFUNCTION and just add up the different parts, as it shows the socket level amounts. This will thus give you an exact number for all protocols speaking plain-text, those not using SSL
However - that won't necessarily give you the counters for the SSL level stuff though in case of HTTPS/FTPS etc, as libcurl doesn't always expose that. It depends on what particular TLS backend it was built to use. The OpenSSL backend should be fine and it will tell you about incoming and outgoing TLS data too (using the same debug callback).
so i have been searching different ways to create client and server program (using visual studios in c++) and i came across RPC (Remote Procedure Call). But i noticed that this also uses a tcp/ip or udp connection.
so what the difference from using RPC to just a basic tcp/ip or udp connection to connect the client and server?
the code is completely different for example in RCP to use tcp:
reinterpret_cast<unsigned char*>("ncacn_ip_tcp"), // Use TCP/IP protocol.
reinterpret_cast<unsigned char*>("localhost"), // TCP/IP network address to use.
reinterpret_cast<unsigned char*>("4747"), // TCP/IP port to use.
but in other programs (eg using WinSock) it requires a lot more code
is one better than the other?
TCP/IP and UDP are protocols for data transfer (getting data packets from here to there). The former is connection-oriented and reliable, whereas UDP is connectionless and unreliable. Neither of them care what the data actually is (a file, a webpage, a video, etc).
TCP/IP creates a connection between two endpoints and then whatever data is sent at one end is received at the other.
UDP does not have a notion of connections, it's for a one-shot "send", without guarantees about delivery.
Several higher-level network protocols are built on top of TCP/IP and UDP. For instance, HTTP (that lets you view this webpage).
RPC is a higher-level protocol, which allows one computer to execute code on another computer. The lines of code you quoted are merely setting the configuration parameters for that RPC implementation.
Summary: RPC needs a network transfer protocol (like TCP/IP) to do its job, but RPC is a higher-level protocol and fulfills a different purpose than merely sending unstructured data from one computer to another.
You are talking about different notions. RPC is mechanism, TCP/UDP - way of communication.
RPC concept has a lot of implementations in different languages: read wiki. TCP/UDP - is transport.
So, in your case, "using TCP/UDP" means "creating your own RPC, based on TCP/UDP". I suggest you to read about existing RPC implementations and select most acceptable, based on your task.
I'm developing a peer-to-peer communications network for use over a LAN in an industrial environment. Some messages are are just asynchronous, and don't require a response. Others are request-response. The request messages (and the async messages) are sent to a multicast group, and the replies to requests are sent unicast. Each endpoint, therefore, receives UDP packets that are sent to the multicast group, and also receives messages that are just sent to it using plain unicast.
So far it's working fine, but there doesn't seem to be any way in boost::asio to find out the destination address of a received UDP packet (using socket.async_receive_from) - whether it was sent to the multicast group or the actual interface. I can use the contents of the message to infer whether it was sent multicast or unicast, but it would be nice to be able to also check the destination address.
We are currently using Windows 7, but will be transitioning to Linux in the future.
Is there a way to find the destination address of a UDP packet received using boost::asio?
Unfortunately this is not possible with boost::asio, and usually is not "the way to do" it, as you try to access Transport Layer information at the Application Layer.
So you basically have two options:
a) Write non-portable system code with for example IP_PKTINFO or SO_BINDTODEVICE on Linux. Example Code can be found on the boost asio mailing list here
b) use two distinct sockets, one for the multicast and one for the unicast. You therefore need to specify a listen_address other than "0.0.0.0" on each socket.
udp::endpoint(address_v4::from_string("239.192.152.143"), 6771)
This Question on SO might also be helpful: Using a specific network interface for a socket in windows
Are there any viable alternatives to Winsock for C++? The reason I need of is because Winsock's raw sockets are not behaving properly (no, this is not fixable, don't waste your time asking) and WinPCAP can only monitor network traffic, not actually receive packets. I need something that can receive complete packets including any IP, UDP, TCP headers, etc. Does such a thing exist, and if not, why not?
WinPCAP can only monitor network traffic, not actually receive packets
Monitoring network traffic is equivalent to receiving packets. That's exactly what tools such as Wireshark do: read off your network card and reconstruct packet boundaries.
I need something that can receive complete packets including any IP, UDP, TCP headers, etc.
This is very much possible using the Winsock API. Have a look at the WSAIoctl function, specifically the SIO_RCVALL option - enabling this option will deliver ALL packets received on an interface to your socket. And these are raw IP packets starting with the IP header.
You could look at Boost.Asio. C++ cross-platform IO library. Support for UDP, TCP and ICMP.