Is there any simplest way to run number of python request asynchronously? - django

Question

There is an API localhost:8000/api/postdatetime/, which is responsible for changing the online status of the driver. Every time a driver hits the API and if the response sent back then the driver status will be online otherwise the driver status will be offline.
Drivers need to give the response within every 10 seconds. If there is no response from the driver then he/she will be marked as offline automatically.
Currently what I am doing is getting all the logged in drivers and hitting the above-mentioned API one driver at a time.
For the simulation purpose, I populated thousands of drivers. To maintain the online-offline of thousands of drivers using my approach will leave almost many drivers offline.
The code for my approach is as described below:
online-offline.py
import requests
import random
import math
from rest_framework.authtoken.models import Token
from ** import get_logged_in_driver_ids
from ** import Driver
def post_date_time():
url = 'localhost:8000/api/postdatetime/'
while True:
# getting all logged in driver ids
logged_in_driver_ids = get_logged_in_driver_ids()
# filtering the driver who are logged in and storing their user id in the driver_query variable
driver_query = Driver.objects.only('id', 'user_id').filter(id__in=logged_in_driver_ids).values('user_id')
# storing the user id of driver in list
driver_user_ids = [driver['user_id'] for driver in driver_query]
# getting number of drivers who are supposed to hit the API
drivers_subset_count = math.ceil(len(driver_user_ids)*(random.randint(75, 85)/100))
# shuffle the driver ids list to randomize the driver id order
random.shuffle(driver_user_ids)
# getting the driver list of length drivers_subset_count
required_drivers = driver_user_ids[:drivers_subset_count]
for driver in required_drivers:
token = Token.objects.only('key', 'user_id').get(user_id=driver)
req_headers = {
'Content-Type': 'application/json',
'Accept': 'application/json',
'Authorization': 'Token' + ' ' + str(token)
}
response = requests.post(url=url, headers=req_headers)
print(response.text)
if __name == '__main__':
post_date_time()
Is there any idea that I can post request the localhost:8000/api/postdatetime/ API asynchronously and can handle about 2-3000 of drivers within 10 seconds?
I am really new at implementing async codes. I have read some articles on the aiohttp library but got confused at the time of implementing it.
This online-offline.py will run for the whole time of the simulation.
Any help will be highly appreciated. Thank you :)

Answer 1

In case of non-local API asyncio can help you. To make requests asynchronously you have to:
use special syntax (async def, await - read here for details)
make requests non-blocking way (so you can await them)
use asyncio.gather() of similar way to make multiple requests parallely"
start event loop
While it is possible to make requests library work with asyncio using threads it's easier and better to use already asyncio-compatible library like aiohttp.
Take a look at code snippets here *: they contain examples of making multiple concurrent requests. Rewrite you code same way, it'll look like:
import asyncio
async def driver_request(driver):
# ...
# use aiohttp to make request with custom headers:
# https://docs.aiohttp.org/en/stable/client_advanced.html#custom-request-headers
async def post_date_time():
# ...
await asyncio.gather(*[
driver_request(driver)
for driver
in required_drivers
])
asyncio.run(post_date_time())
Locally you won't see any effect by default, so to test it locally you have to make localhost send response with delay to emulate real-life network delay.
* In latest Python version event loop can be run with just asyncio.run()