I understand that HTTP2 in Jetty is mostly at the Connector, Transport and Channel levels.
I'm trying to decide which combination would be the best to transport binary data between client and server:
Jetty HTTP2 server with async servlet + Jetty HTTP2 client
Jetty HTTP2 server with sync servlet + Jetty HTTP2 client
Jetty HTTP2 server with async servlet + Netty HTTP2 client
GRPC client and server (both are default Netty based)
Details:
I would like to send binary data to my client and I would like the connections to be non-blocking/async. The number of concurrent client requests can be high and the server could take a few seconds (some times) to respond to some requests.
Each response is small chunk of binary data. I would've liked it if I could send Netty's ByteBufs directly as the response instead of copying to byte[] or ByteBuffer but that is not directly related to this particular question.
Method #4 is not my favorite because of the ProtoBuf wrapping (link) limitation (link).
Jetty references:
http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/http2/client/HTTP2Client.html
https://github.com/eclipse/jetty.project/tree/jetty-9.4.x/jetty-http2/http2-server/src/main/java/org/eclipse/jetty/http2/server
https://github.com/fstab/http2-examples/blob/master/multiplexing-examples/jetty-client/src/main/java/de/consol/labs/h2c/examples/client/jetty/JettyClientExample.java
https://groups.google.com/forum/#!topic/grpc-io/z0rhhetN1rE
Disclaimer, I am the Jetty HTTP/2 maintainer.
Given that you have a large number of clients and that processing could take seconds, I would recommend to go with option 1 - async servlet and Jetty HTTP/2 client.
With "async servlet" you have 2 flavors: 1) async processing with blocking I/O, or 2) async processing + async I/O.
Servlet async processing is triggered by the use of HttpServletRequest.startAsync().
Servlet async I/O is triggered by using ReadListener and WriteListener respectively with the ServletInputStream and ServletOutputStream.
You definitely want async processing because you have a large number of clients and processing in the order of seconds - this will optimize the use of server threads.
Whether or not to use async I/O should probably be measured, since you have small binary responses.
Blocking I/O is much much easier to code and debug, while async I/O is definitely more complicated to code and debug. Async I/O really shines when you have large contents and slow clients that may congest the TCP connection.
If you want to be fully async, go with async I/O. If you can tolerate a bit of blocking in exchange of simpler code, stay on blocking I/O.
Worth repeating, in both cases - either async I/O or blocking I/O - you want to use async processing.
Regarding the issue of copying data, if you are willing to cast down to Jetty classes, you can avoid the copy by directly writing a ByteBuffer to the ServletOutputStream subclass, see this example.
Finally, with the Jetty client you can use the high-level HttpClient with the HTTP/2 transport as detailed here. The benefit would be a high-level API that only deals with HTTP concepts, rather than using the low-level HTTP2Client that deals with HTTP/2 frames, streams and such.
Report back what you end up choosing, and how it goes for you !
Related
Okay, so a little context:
I have an app running on an embedded system that sends a few different requests over HTTP (using libcurl in C++) at the following intervals:
5 minutes
15 minutes
1 hour
24 hours
My goal: Reduce data consumption (runs over cellular)
We have both client and server side TLS authentication, so the handshake is costly. The idea is that we use persistent connections (at least for the shorter interval files) to avoid doing the handshake every time.
Unfortunately, after much tinkering I've figured out that the server is closing the connection before the intervals pass. Maybe this is something we can extend? I'll have to talk to the server side guys.
I was under the impression that was the reason the "TCP keep-alive" packets existed, but supposedly those "check the connection" not "keep it open" like the name suggests.
My idea is this:
Have my app send a packet (as small as possible) every 2 minutes or so (however long the timeout is) to "nudge" the connection into staying open.
My questions are:
Does that make any sense?
I don't suppose there is an easy way to do this in libcurl is there?
If so, how small could we get the request?
Is there an even easier way to do it? My only issue here is that all the connection stuff "lives" in libcurl.
Thanks!
It would be easier to give a more precise answer if you gave a little more detail on your application architecture. For example, is it a RESTful API? Is the use of HTTP absolutely mandatory? If so, what HTTP server are you using (nginx, apache, ...)? Could you consider websockets as an alternative to plain HTTP?
If you are at liberty to use something other than regular HTTP or HTTPs - and to use something other than libcurl on the client side - you would have more options.
If, on the other hand, if you are constrained to both
use HTTP (rather than a raw TCP connection or websockets), and
use libcurl
then I think your task is a good bit more difficult - but maybe still possible.
One of your first challenges is that the typical timeouts for a HTTP connection are quite low (as low as a few seconds for Apache 2). If you can configure the server you can increase this.
I was under the impression that was the reason the "TCP keep-alive" packets existed, but supposedly those "check the connection" not "keep it open" like the name suggests.
Your terminology is ambiguous here. Are you referring to TCP keep-alive packets or persistent HTTP connections? These don't necessarily have anything to do with each other. The former is an optional mechanism in TCP (which is disabled by default). The latter is an application-layer concept which is specific to HTTP - and may be used regardless of whether keep-alive packets are being used at the transport layer.
My only issue here is that all the connection stuff "lives" in libcurl.
The problem with using libcurl is that it first and foremost a transfer library. I don't think it is tailored for long-running, persistent TCP connections. Nonetheless, according to Daniel Stenberg (the author of libcurl), the library will automatically try to reuse existing connections where possible - as long as you re-use the same easy handle.
If so, how small could we get the request?
Assuming you are using a 'ping' endpoint on your server - which accepts no data and returns a 204 (success but no content) response, then the overhead - in the application layer - would be the size of the HTTP request headers + the size of the HTTP response headers. Maybe you could get it down to 200-300 bytes, or thereabouts.
Alternatives to (plain) HTTP
If you are using a RESTful API, this paradigm sort of goes against the idea of a persistent TCP connection - although I can not think of any reason why it would not work.
You might consider websockets as an alternative, but - again - libcurl is not ideal for this. Although I know very little about websockets, I believe they would offer some advantages.
Compared to plain HTTP, websockets offer:
significantly less overhead than HTTP per message;
the connection is automatically persistent: there is no need to send extra 'keep alive' messages to keep it open;
Compared to a raw TCP connection, the benefits of websockets are that:
you don't have to open a custom port on your server;
it automatically handles the TLS/SSL stuff for you.
(Someone who knows more about websockets is welcome to correct me on some of the above points - particularly regarding TLS/SSL and keep alive messages.)
Alternatives to libcurl
An alternative to libcurl which might be useful here is the Mongoose networking library. It would provide you with a few different alternatives:
use a plain TCP connection (and a custom application layer protocol),
use a TCP connection and handle the HTTP requests yourself manually,
use websockets - which it has very good support for (both as server and client).
Mongoose allows you to enable SSL for all of these options also.
I am building a REST service which internally calls other services and we use org.apache.cxf.jaxrs.client.WebClient to do this.
I want to use HTTP connection pooling to improve the performance but the documentation isn't very clear about how to do this or if this is even possible. Has anyone here done this?
The only other option I can think of it is to re-use clients but I'd rather not get into the whole set of thread-safety and synchronization issues that comes with that approach.
By default, CXF uses a transport based on the in-JDK HttpURLConnection object to perform HTTP requests.
Connection pooling is performed allowing persistent connections to reuse the underlying socket connection for multiple http requests.
Set system properties (default values)
http.keepalive=true
http.maxConnections=5
Increment the value of http.maxConnections to set the maximum number of idle connections that will be simultaneously kept alive, per destination.
In this post are explained some detail how it works
Java HttpURLConnection and pooling
When you need many requests executed simultaneosly CXF can also use the asynchronous apache HttpAsyncClient. Ser details here
http://cxf.apache.org/docs/asynchronous-client-http-transport.html
Is this even possible?
I know, I can make a one-way asynchronous communication, but I want it to be two-way.
In other words, I'm asking about the request/response pattern, but non-blocking, like described here (the 3rd option)
Related to Asynchronous, acknowledged, point-to-point connection using gSoap - I'd like to make the (n)acks async, too
You need a way to associate requests with replies. In normal RPC, they are associated by a timeline: the reply follows the response before another response can occur.
A common solution is to send a key along with the request. The reply references the same key. If you do this, two-way non-blocking RPC becomes a special case of two one-way non-blocking RPC connections. The key is commonly called something like request-id or noince.
I think that is not possible by basic usage,
The only way to make it two way is via response 'results of the call'
But you might want to use little trick
1] Create another server2 at client end and call that server2 from server
Or if thats not you can do over internet because of NAT / firewall etc
2] re architect your api so that client calls server again based on servers responce first time.
You can have client - server on both end. For example you can have client server on system 1 and system 2. (I specify sender as cient and receiver as server). You send async message from sys1 client to sys 2 server. On recieving message from sys1 you can send async response from sys 2 client to sys1 server back. This is how you can make async two way communication.
I guess you would need to run the blocking invocation in a separate thread, as described here: https://developer.nokia.com/Community/Wiki/Using_gsoap_for_web_services#Multithreading_for_non-blocking_calls
I am designing a client server socket program using TCP/IP.
The server listens on a certain port, the client program makes 2 connections to the server. One is for command and response and the other is for streaming of data.
For the command and response, I can use the normal blocking socket mode to receive the client command and send the server response.
For the streaming data, the server would wait for the client to send a start stream command and begins continuous sending of data to that client. The issue now is I need the handler to also listen on this connection for the stop stream command. Hence, I was thinking of making this connection non-blocking so that the receive would not block followed by a non-blocking send.
Is this method of implementing the server and client handler efficient?
Take a look at Boost::asio socket management layer. It's very well written.
http://www.boost.org/doc/libs/1_49_0/doc/html/boost_asio/tutorial/tutdaytime1.html
Yes it is very efficient.
You can use libraries like libevent.
From perspective of efficiency, the server should always be designed to use non-blocking sockets, and use event-driven asynchronous I/O architecture. Blocking sockets should be avoided at server side.
Fortunately, there've been a few mature open source frameworks you can use. Among them, libev is most lightweight.
I see most http codes in php.
I'm about to write some http calls in c++ using CURL.
Wonder if http is inherently blocking(opposed to non-blocking).
IE, when you send get/post message, your thread is blocked until it gets the response?
If it's not, is there a way to perform non-blocking http get or post with CURL?
Thank you
HTTP is a protocol, so it's not inherently blocking or non-blocking. The only thing resembling 'blocking behavior' in HTTP is that you can't send two requests or two responses at once in the same pipeline - you have to wait for the request to finish before sending another one.
So your real question about blocking operations should be about CURL - does it allow non-blocking IO?
The answer is that libcurl has something called the 'multi interface', which enables you to use it without blocking:
http://curl.haxx.se/libcurl/c/libcurl-multi.html
If you prefer a library that's better designed towards asynchronous IO, you can check out Boost.ASIO. I've never used it myself, but it seems to be popular:
http://www.boost.org/doc/libs/1_46_1/doc/html/boost_asio.html