#30451 closed New feature (fixed)
Add ASGI support to Django.
Reported by: | Andrew Godwin | Owned by: | Andrew Godwin |
---|---|---|---|
Component: | HTTP handling | Version: | dev |
Severity: | Normal | Keywords: | asgi |
Cc: | Petter Strandmark | Triage Stage: | Ready for checkin |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
The ASGI specification (http://asgi.readthedocs.io) is now stable and Django should add support for it in addition to WSGI. This doesn't require too many deep changes, unlike a proposed "async-ification" of Django to allow async views and similar, which will require a DEP.
This will allow both seamless hosting of Django on ASGI servers, and will also significantly improve performance for sites that have slow file uploads as a blocking operation, as this will run asynchronously rather than holding open a thread.
Change History (14)
comment:1 by , 6 years ago
comment:2 by , 6 years ago
Cc: | added |
---|
comment:3 by , 6 years ago
Summary: | Add ASGI support to Django → Add ASGI support to Django. |
---|---|
Triage Stage: | Unreviewed → Accepted |
comment:4 by , 6 years ago
...and will also significantly improve performance for sites that have slow file uploads as a blocking operation, as this will run asynchronously rather than holding open a thread.
From the conversation on the PR, it looks as if file uploads aren't yet enabled:
Florian: This probably cannot use the upload handlers for similar reasons?
Andrew: I haven't looked into that yet, but probably not. File input in general has been a pain in Channels and I expect the same here.
https://github.com/django/django/pull/11209#pullrequestreview-232567459
Is that right? If so do we need to pull this aspect into a separate ticket?
comment:5 by , 6 years ago
It looks like the patch is going to be merged. I feel like I should at least point out some downsides before this step is taken. (Note, I am not a Django dev, just a user).
The patch adds an unconditional asyncio import. It is quite heavy for something most users won't use at least initially. On my aging laptop:
- Import time: 50ms
- Memory usage: ~5MB
Command for import time: python3.7 -X importtime -c 'import asyncio' |& tail -1
Script for memory usage:
from resource import getrusage, RUSAGE_SELF before = getrusage(RUSAGE_SELF) import asyncio after = getrusage(RUSAGE_SELF) print(f'{after.ru_maxrss - before.ru_maxrss}kb')
The patch adds 2 dependencies, one direct asgiref
and one transitive async_timeout
.
The patch implements the ASGI specification, but that specification has not been adopted widely yet and has not been proposed for official status (that I know of) like WSGI has.
ASGI depends on asyncio, but there are other alternatives worth considering (like trio and curio). And of course there is gevent which has been used successfully by many companies (including mine) for async Djagno for some time.
The patch on its own will run all views in a thread pool (asgiref.sync_to_async -> loop.run_in_executor -> default ThreadPoolExecutor). The default number of workers used is the number of cores * 5, so for example an 8 core will use 40 workers. If postgresql is used with persistent connections, this is a hazard since postgresql doesn't like this many connections. So users will need to use a connection pool, or disable persistent connections.
In my opinion, it would be hasty be merge ASGI support before the overall plan is decided. If a DEP is later proposed, and rejected (I hope not!), will it still make sense to have merged this?
comment:6 by , 6 years ago
Regarding ASGI not being "official": Python core has made it reasonably clear to me that they don't want to have a PEP for ASGI, and giving one to WSGI was a bad idea in the first place. Thus, adoption is down to us just doing it. Flask have also said they are going to attempt to support it, as have Twisted.
The threads observation is an astute one, and I think worth calling out in the deployment documentation explicitly.
The async_timeout dependency is only for the async test harness in asgiref, and thus should never be imported under normal conditions.
The reason I have done this separately from the DEP is that it's not, in my eyes, strictly a new "feature" - instead, it is adding compatibility for a server-application protocol, and some last-resort async safety. Even if we didn't implement ASGI, I would still argue that we definitely need to make the ORM understand and protect itself from coroutine-based connection corruption, and that would involve importing asyncio (and probably asgiref for its helpful features in this area) anyway!
comment:7 by , 5 years ago
Triage Stage: | Accepted → Ready for checkin |
---|
The pull request for this is https://github.com/django/django/pull/11209