How can I test a response whose body is streamed from within an ecto transaction? - unit-testing

Question

I am attempting to write unit tests for an endpoint that streams its response in chunks. I can verify that the contents are fully and correctly streamed when accessed through my browser. But when I access the endpoint through my test suite, the response body is empty.
Example controller action
def stream_csv(conn, _params) do
conn = conn
|> put_resp_content_type("text/csv")
|> put_resp_header("content-disposition", "data.csv")
|> send_chunked(200)
Repo.transaction(fn ->
{:ok, conn} = chunk(conn, "csv data part")
end)
conn
end
Example unit test
test "stream endpoint" do
body = build_conn()
|> Phoenix.ConnTest.get("/stream_endpoint")
|> Phoenix.ConnTest.response(200)
assert body =~ "csv data part"
end
This will lead to an assertion failure, where body is an empty binary "".
I feel like there should be a way to wait on all the chunks before making assertions, or that I'm probably overlooking something obvious.
EDIT
The example I initially wrote works as expected. What seems to complicate things is when chunk is called from within a callback provided to Repo.transaction. I've updated the question and examples to better reflect the problem.

Answer 1

I had this problem and tried several ways to get to to work and ended up doing this
solution using mocks
So what is going on is that when we call Plug.Conn.chunk(conn, chunk), plug calls conn.adapter.chunk. From there, the chunk should be sent to the server (e.g. cowboy) for further handling. The conn is not aware of the chunk anymore.
To solve this, I moved the chunking to another functions with minimal side effects and easy mockable
defmodule MyApp.ControllerUtils do
use MyaAppWeb, :controller
#callback chunk_to_conn(map(), String.t()) :: map()
def chunk_to_conn(conn, current_chunk) do
conn |> chunk(current_chunk)
end
end
And in the response handler
def stream_csv(conn, _params) do
conn = conn
|> put_resp_content_type("text/csv")
|> put_resp_header("content-disposition", "data.csv")
|> send_chunked(200)
Repo.transaction(fn ->
{:ok, conn} = MyApp.ControllerUtils.chunk_to_conn(conn, "csv data part")
end)
conn
end
Now in your test you mock the chunking function to give you the chunk and use a something like Agent store and join the chunks or just assert them as they come.
import Mox
defp chunked_response_to_state(chunk, pid) do
current_chunk = Agent.get(pid, &Map.get(&1, :chunk_key))
Agent.update(pid, &Map.put(&1, :csv, current_chunk <> chunk))
end
setup do
MyApp.ControllerUtilsMock
|> stub(:chunk_to_conn, fn _, chunk -> chunk |> chunked_response_to_state(agent_pid) end)
{:ok, %{agent_pid: agent_pid}}
end
test "my test", state do
build_conn.get(somepath)
whole_chunks = Agent.get(state.agent_pid, &Map.get(&1, :chunk_key))
end