Running asynchronous code synchronously in separate thread - django

Question

I'm using Django Channels to support websockets and am using their concept of a group to broadcast messages to multiple consumers in the same group. In order to send messages outside of a consumer, you need to call asynchronuous methods in otherwise synchronous code. Unfortunately, this is presenting problems when testing.
I began by using loop.run_until_complete:
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.ensure_future(channel_layer.group_send(group_name, {'text': json.dumps(message),
'type': 'receive_group_json'}),
loop=loop))
Then the stacktrace read that the thread did not have an event loop: RuntimeError: There is no current event loop in thread 'Thread-1'.. To solve this, I added:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(asyncio.ensure_future(channel_layer.group_send(group_name, {'text': json.dumps(message),
'type': 'receive_group_json'}),
loop=loop))
And now the stacktrace is reading the RuntimeError: Event loop is closed, although if I add print statements loop.is_closed() prints False.
For context, I'm using Django 2.0, Channels 2, and a redis backend.
Update: I tried running this in a Python interpreter (outside of py.test to remove moving variables). When I ran the second code block, I did not get an Event loop is closed error (that may be due to something on Pytest's end whether its timeouts, etc). But, I did not receive the group message in my client. I did, however, see a print statement:
({<Task finished coro=<RedisChannelLayer.group_send() done, defined at /Users/my/path/to/venv/lib/python3.6/site-packages/channels_redis/core.py:306> result=None>}, set())
Update 2: After flushing redis, I added a fixture in py.test to flush it for every function as well as a session-scoped event loop. This time yielding yet another print from RedisChannelLayer:
({<Task finished coro=<RedisChannelLayer.group_send() done, defined at /Users/my/path/to/venv/lib/python3.6/site-packages/channels_redis/core.py:306> exception=RuntimeError('Task <Task pending coro=<RedisChannelLayer.group_send() running at /Users/my/path/to/venv/lib/python3.6/site-packages/channels_redis/core.py:316>> got Future <Future pending> attached to a different loop',)>}, set())

Answer 1

If channel_layer expects to reside in its own event loop in another thread, you will need to get a hold of that event loop object. Once you have it, you can submit coroutines to it and synchronize with your thread, like this:
def wait_for_coro(coro, loop):
# submit coroutine to the event loop in the other thread
# and wait for it to complete
future = asyncio.run_coroutine_threadsafe(coro, loop)
return future.wait()
wait_for_coro(channel_layer.group_send(group_name, ...), channel_loop)

Answer 2

By default, only the main thread gets an event loop and calling get_event_loop in other threads will fail.
If you need an event loop in another thread -- such as a thread handling an HTTP or WebSockets request -- you need to make it yourself with new_event_loop. After that you can use set_event_loop and future get_event_loop calls will work. I do this:
# get or create an event loop for the current thread
def get_thread_event_loop():
try:
loop = asyncio.get_event_loop() # gets previously set event loop, if possible
except RuntimeError:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
return loop
More here.