Http Client with Source.tick - akka

Question

I am trying to connect http client to a http service exposed by server, the source should send request every 1 second for that I have crated following partial graphs:
def httpSourceGraph() = {
Source.fromGraph(GraphDSL.create() { implicit builder =>
val sourceOutLet = builder.add(Source.tick(FiniteDuration(0, TimeUnit.SECONDS), FiniteDuration(1,
TimeUnit.SECONDS),
HttpRequest(uri ="/test", method = HttpMethods.GET))).out
// expose outlet
SourceShape(sourceOutLet)
})
}
def httpConnFlow() = {
Flow.fromGraph(GraphDSL.create() { implicit builder =>
val httpSourceFlow = builder.add(Http(system).outgoingConnection(host = "localhost", port = 8080))
FlowShape(httpSourceFlow.in, httpSourceFlow.out)
})
}
the graph is composed as
val response= httpSourceGraph.via(httpConnFlow()).runForeach(println)
if the http server (localhost:8080/test) is up and running, everything works fine, every 1 second I can see the response coming back from the server. I am not able to any response in case of either server is down or it goes down later.
I think it should give me following error:
akka.stream.StreamTcpException: Tcp command [Connect(localhost/127.0.0.1:8080,None,List(),Some(10 seconds),true)] failed
This can tested with some wrong url as well. (domain name stackoverflow1.com and wrong url "/test")
Thanks for the help.
-Arun

Answer 1

I can propose one way to get the behavior that you are seeking. I think what's at the heart of your issue is that the Flow produced by Http().outgoingConnection will terminate when a failure is encountered. Once that happens, there is no more downstream demand to pull the requests from the Source and the whole flow stops. If you want something that will continue to emit elements downstream regardless of if the connection is lost then you might try and use a host connection pool instead of just a single connection. The pool will be more resilient to failures with individual connections and it's also setup from the get go to send either a Success or Failure downstream. A simplified version of your flow, using a host connection pool could be defined as follows:
val source =
Source.tick(
1 second,
5 second,
(HttpRequest(uri ="/", method = HttpMethods.GET), 1)
)
val connFlow = Http(system).
newHostConnectionPool[Int](host = "www.aquto.com", port = 80)
val sink = Sink.foreach[(util.Try[HttpResponse], Int)]{
case (util.Success(r), _ ) =>
r.entity.toStrict(10 seconds)
println(s"Success: ${r.status}")
case (util.Failure(ex), _) =>
println(s"Failure: ${ex.getMessage}")
}
source.via(connFlow).to(sink).run
I tested this out, unplugging my network connection in the middle of the test and this is what I see as output:
Success: 200 OK
Success: 200 OK
Failure: Tcp command [Connect(www.aquto.com/50.112.131.12:80,None,List(),Some(10 seconds),true)] failed
Failure: Tcp command [Connect(www.aquto.com/50.112.131.12:80,None,List(),Some(10 seconds),true)] failed
Failure: Tcp command [Connect(www.aquto.com/50.112.131.12:80,None,List(),Some(10 seconds),true)] failed
Success: 200 OK
Success: 200 OK