Under WSGI, multiple calls to asgiref.sync.async_to_sync within the same request do not share the same event loop.

[Mykhailo Havelia]

Each call creates a new thread with its own event loop. This commonly happens in middlewares and signals, causing a single request to spawn multiple threads and event loops, which is unnecessary and prevents reuse of async resources.

This could be addressed by introducing a per-request async context, for example as shown in this draft implementation:
​https://github.com/Arfey/django/pull/5/changes

[Kundan Yadav]

hey can i work on this issue ?

[Jacob Walls]

Thanks, but you should wait for it to be accepted first.

[Vishy Algo]

With asgiref >= 3.10, we can leverage AsyncSingleThreadContext to enforce a persistent execution context for the entire request lifecycle.

I suggest wrapping the WSGIHandler.__call__ logic in this context. Crucially, because the WSGI response iteration often outlives the __call__ stack frame (e.g. StreamingHttpResponse), we cannot use a simple with block or decorator. Instead, we must manually manage the context's lifecycle and attach the cleanup logic to response.close().

I’ve verified this behavior with a test asserting that the thread is successfully reused across calls.

django/core/handlers/wsgi.py

@@ -118 +118 @@
         self.load_middleware()
 
     def __call__(self, environ, start_response):
-        set_script_prefix(get_script_name(environ))
-        signals.request_started.send(sender=self.__class__, environ=environ)
-        request = self.request_class(environ)
-        response = self.get_response(request)
-
-        response._handler_class = self.__class__
-
-        status = "%d %s" % (response.status_code, response.reason_phrase)
-        response_headers = [
-            *response.items(),
-            *(("Set-Cookie", c.OutputString()) for c in response.cookies.values()),
-        ]
-        start_response(status, response_headers)
-        if getattr(response, "file_to_stream", None) is not None and environ.get(
-            "wsgi.file_wrapper"
-        ):
-            # If `wsgi.file_wrapper` is used the WSGI server does not call
-            # .close on the response, but on the file wrapper. Patch it to use
-            # response.close instead which takes care of closing all files.
-            response.file_to_stream.close = response.close
-            response = environ["wsgi.file_wrapper"](
-                response.file_to_stream, response.block_size
-            )
-        return response
+        async_context = AsyncSingleThreadContext()
+        async_context.__enter__()
+        try:
+            set_script_prefix(get_script_name(environ))
+            signals.request_started.send(sender=self.__class__, environ=environ)
+            request = self.request_class(environ)
+            response = self.get_response(request)
+
+            response._handler_class = self.__class__
+
+            status = "%d %s" % (response.status_code, response.reason_phrase)
+            response_headers = [
+                *response.items(),
+                *(("Set-Cookie", c.OutputString()) for c in response.cookies.values()),
+            ]
+            start_response(status, response_headers)
+
+            original_close = response.close
+
+            def close():
+                try:
+                    original_close()
+                finally:
+                    async_context.__exit__(None, None, None)
+
+            if getattr(response, "file_to_stream", None) is not None and environ.get(
+                "wsgi.file_wrapper"
+            ):
+                # If `wsgi.file_wrapper` is used the WSGI server does not call
+                # .close on the response, but on the file wrapper. Patch it to use
+                # response.close instead which takes care of closing all files.
+                response.file_to_stream.close = response.close
+                response = environ["wsgi.file_wrapper"](
+                    response.file_to_stream, response.block_size
+                )
+            return response
+        except Exception:
+            async_context.__exit__(None, None, None)
+            raise

    def test_async_context_reuse(self):
        """
        Multiple calls to async_to_sync within a single request share the
        same thread and event loop via AsyncSingleThreadContext.
        """
        async def get_thread_ident():
            return threading.get_ident()

        class ProbingHandler(WSGIHandler):
            def __init__(self):
                pass

            def get_response(self, request):
                t1 = async_to_sync(get_thread_ident)()
                t2 = async_to_sync(get_thread_ident)()
                return HttpResponse(f"{t1}|{t2}")

        app = ProbingHandler()
        environ = self.request_factory._base_environ(PATH_INFO="/")
        
        def start_response(status, headers):
            pass

        response = app(environ, start_response)
        content = b"".join(response).decode("utf-8")
        t1, t2 = content.split("|")

        self.assertEqual(
            t1, t2, 
            f"Failed: async_to_sync spawned new threads ({t1} vs {t2}). Context was not reused."
        )

[Mykhailo Havelia]

Replying to Vishy Algo:

Okay. I’m going to add tests for my MR and then submit it for code review.

[Mykhailo Havelia]

Replying to Vishy Algo:

I prepared an MR, but it seems it's not sufficient: ​https://github.com/django/django/pull/20552. I added a test that compares not just the thread, but the event loop as well, and the event loop isn't the same. asgiref uses the same thread, but it starts a new event loop for each function via asyncio.run (​https://github.com/django/asgiref/blob/main/asgiref/sync.py#L314).
I could try creating a loop myself and run each function via loop.run_until_complete, but that would require some additional investigation to see if it's feasible.

What do you think?

[Mykhailo Havelia]

Replying to Jacob Walls:

I've added an MR with changes to asgiref that should help with sharing the same event loop. There are still some issues on lower Python versions, but the overall approach can already be reviewed 😌
​https://github.com/django/asgiref/pull/542

[Jacob Walls]

Thanks Mykhailo. Can you help me quantify the performance impact? I'd like to know the upside for async users as well as the downside for WGSI users who have no async middleware or signals.

I notice your draft PR for the WSGIHandler has a nested function. We're starting to see tickets along the lines of "X created nested functions that take a gc pass to free". My understanding is that this is much less of a worry on Python 3.14 now that we have incremental gc, but still. I just want to know what WSGI-only users are paying.

Happy for you to reopen for another look with those details in hand, but full disclosure we're going to need a few more +1s from knowledgable parties to accept.

[Mykhailo Havelia]

Replying to Jacob Walls:

Hey. Thanks for response 😌

Async upside

Right now async views under the dev server go through async_to_sync, which runs them in a separate event loop. That breaks loop-bound resources (DB clients, HTTP clients, etc.) and we see errors like "event loop is closed". This change keeps async work in a consistent per-request loop, so async code behaves like it does under ASGI. The win is mainly correctness and lifecycle safety, not raw speed.

Cost for WSGI-only users

You're right that the current approach introduces a reference cycle by wrapping response.close. That means the response object is reclaimed on a GC pass instead of immediately via refcounting.

gc.disable()
gc.collect()

response = application(environ, start_response)
ref_response = weakref.ref(response)
response.close()
del response

print("before", ref_response() is None)
gc.collect()
print("after", ref_response() is None)

Output:

before False
after True

Mitigation

We can avoid the cycle entirely by moving the cleanup hook into the response layer (registering extra closers instead of wrapping close). That keeps refcount-based cleanup and makes the WSGI path effectively neutral.

class HttpResponseBase:
    def close(self):
        ...
        signals.request_finished.send(sender=self._handler_class)

        for closer in self._after_resource_closers:
            try:
                closer()
            except Exception:
                pass

        self._after_resource_closers.clear()

response._add_after_resorce_closer(cleanup_stack.close)

What do you think? 🙂

[Jacob Walls]

At a glance that cleanup strategy seems more attractive in order to unclutter the WSGIHandler, but we can defer that to PR review.

I think I understand the ask here; I'm just wondering why it was never designed this way. Our ​docs read:

Under a WSGI server, async views will run in their own, one-off event loop.
This means you can use async features, like concurrent async HTTP requests,
without any issues, but you will not get the benefits of an async stack.

So, am I to understand that there are, in fact, "issues" here, issues that are not resolvable by, "just use an ASGI stack"?

[Mykhailo Havelia]

Replying to Jacob Walls:

At a glance that cleanup strategy seems more attractive

Nice, glad that direction makes sense. I'll put together an MR with tests so we can try it out properly.

So, am I to understand that there are, in fact, "issues" here, issues that are not resolvable by, "just use an ASGI stack"?

I might be misunderstanding the question a bit 🙂 It does work correctly under ASGI, yes. But I'd still consider this an issue, because the behavior differs depending on the server stack, and that can lead to subtle lifecycle problems especially in development where people commonly use WSGI.

I've gotta fix this for async backends. ​https://github.com/Arfey/django-async-backend/issues/4#issuecomment-3749548648

[Jacob Walls]

I want to help you advance the django-async-backend, but I'm not certain that it needs to support WSGI right now in order to advance. I'll leave a comment on the ​new-features repo issue that tracks that work to get more input. I could be not understanding the intended audience of testers. Can't we test this with daphne's runserver?

About why it was designed this way, I did find a reference in the ​async DEP about the intended final design:

WSGI mode will run a single event loop around each Django call to make the async handling layer compatible with a synchronous server.

So that supports the idea that we would eventually get around to your proposal, but since DEP 0009 there have been other conversations to the effect of, "aren't we basically done except for the async database backend." And if that's ASGI-only for now, that might be enough?

I can check my understanding with the Steering Council early next week. Setting Someday/Maybe until then.

[Mykhailo Havelia]

Replying to Jacob Walls:

I want to help you advance the django-async-backend

I really appreciate that, thank you so much ☺️

I could be not understanding the intended audience of testers. Can't we test this with daphne's runserver?

We have a fairly large Django project that was set up long before async support existed. Part of it is now async (separate API endpoints), but a big portion is still fully sync. For development, we run everything through a single server so the whole system works together in one place.

Technically, yes. I could run async parts with Uvicorn/Gunicorn and keep the rest elsewhere. But then we'd need:

• two dev servers
• plus something to sit in front and combine them
• and both servers would need restarting on code changes

That makes the dev setup more complex and increases the barrier for other teams. Just running the backend requires extra infrastructure and configuration, which grows over time.

A concrete example is async Redis usage. It works fine under ASGI, but under WSGI we hit real limitations:

• connections can't be shared properly across requests
• we can't reliably close them at the end of the request because the expected async lifecycle and signals don't behave the same way

ASGI only sounds easy, but with a unified dev setup this is exactly where problems show up.

So that supports the idea that we would eventually get around to your proposal

DEP 0009 is 6 years old already 😅 still waiting. a bit ironic.

but since DEP 0009 there have been other conversations to the effect of, "aren't we basically done except for the async database backend."

I kind of wish that were true 🙂 From my side, there's still a lot of work left

And if that's ASGI-only for now, that might be enough?

In practice, a lot of internal Django code still ends up going through async - sync bridges. People wrap things in sync_to_async and assume they're writing fully async code. This is a really remarkable thread on this topic ​https://forum.djangoproject.com/t/asynctestclient/43841😅

I can check my understanding with the Steering Council early next week. Setting Someday/Maybe until then.

thank you so much 😌

btw: If you need any more info, just message me anytime.

[Jacob Walls]

Double-checked with the Steering Council:

• the reference I found to DEP 0009 is indeed out of date, we should not draw the conclusion from that that it's accepted to add a single event loop for WSGI handler
• the django-async-backend can be advanced and tested on ASGI-only setups, including for development Daphne's runserver command, among other options
• otherwise you can use a reverse proxy to direct traffic between WSGI and ASGI servers
• It is possible we would revisit this once the Django ecosystem becomes more interested in (& compatible with) free-threading Python builds

So Someday/Maybe feels appropriate for now.

[Mykhailo Havelia]

Replying to Jacob Walls:

Hi Jacob,
First of all, thanks so much for your really quick response 🙂

• the django-async-backend can be advanced and tested on ASGI-only setups ...
• otherwise you can use a reverse proxy to direct traffic between ...

Of course, we can do that 😔 I'll just go ahead without it for now.

the reference I found to DEP 0009 is indeed out of date, we should not draw the conclusion from that that it's accepted to add a single event loop for WSGI handler

That makes sense. Basically, Django lets us run sync code inside async temporarily (like a bridge while async backends aren't ready), but we can't run a fully async app on a sync server.

• Django forces it in some places, wrapping any async view with async_to_sync for some reason and middlewares too.
• Even some newer features, like task backends, require it.

    def call(self, *args, **kwargs):
        if iscoroutinefunction(self.func):
            return async_to_sync(self.func)(*args, **kwargs)
        return self.func(*args, **kwargs)

Because of this, we can't properly close async connections at the end of a request, which could lead to connection leaks at best.

• And to make things more confusing, Django documentation sometimes gives the impression that this is fine 😁

Async views will still work under WSGI, but with performance penalties, and without the ability to have efficient long-running requests.

Under a WSGI server, async views will run in their own, one-off event loop. This means you can use async features, like concurrent async HTTP requests, without any issues, but you will not get the benefits of an async stack.

In both ASGI and WSGI mode, you can still safely use asynchronous support to run code concurrently rather than serially. This is especially handy when dealing with external APIs or data stores.

Based on that, I have a couple of questions:

1. Would it be okay if I prepare an MR to fix the documentation and add some details about Daphne?
2. Can I open a ticket to discuss removing support for async views and middlewares under WSGI?

[Jacob Walls]

Would it be okay if I prepare an MR to fix the documentation and add some details about Daphne?

Yes, and it will also help me verify I'm on the same page with you.

Can I open a ticket to discuss removing support for async views and middlewares under WSGI?

I think the forum is the right place to start that discussion, it needs more acceptance than just one ticket triager.

I know Carlton's looking at the related asgiref PR, I just also nosied him on the Django PR, too.