#20238 closed New feature (fixed)
Support multiple open connections in LiveServerTestCase
Reported by: | Owned by: | Nadege | |
---|---|---|---|
Component: | Testing framework | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Accepted | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
Run a selenium test with LiveServerTestCase. Django starts StoppableWSGIServer. The StoppableWSGIServer is not multithread support. When the codes to be tested call urllib.urlopen to the same server (ie: StoppableWSGIServer) the selenium test will hang and fail because of time-out.
Change History (21)
comment:1 by , 11 years ago
Resolution: | → needsinfo |
---|---|
Status: | new → closed |
comment:2 by , 11 years ago
Just a clarification: please feel free to reopen this ticket if you can provide more detailed information.
comment:3 by , 11 years ago
Although this is not my ticket, I just ran into the same issue.
The problem occurs typically when, within a request/response cycle, another request is done to the same StoppableWSGIServer
. Now, the first request can never finish because the second request is waiting for the StoppableWSGIServer
to respond (but it never will because it's waiting for the first request...)
# view def index(self, request): if request.GET.get('inner_request') == 'true': # Within a request/response, perform another request to the same live server. # This one get's stuck. urllib.urlopen(request.build_absolute_uri()) return HttpResponse() # test class LiveServerTests(LiveServerTestCase): def test_request_within_request(self): response = requests.get(self.live_server_url, params={'inner_request': 'true'}) # This code is never reached (or the live server throws a timeout) self.assertEqual(response.status_code, 200)
The use case is somewhat fictional.
comment:4 by , 10 years ago
Resolution: | needsinfo |
---|---|
Status: | closed → new |
comment:5 by , 10 years ago
Resolution: | → wontfix |
---|---|
Status: | new → closed |
Considering that:
- "the use case is somewhat fictional",
- the ticket was reopened without any additional information,
- more threading will certainly create more race conditions on shutdown, especially when it comes to the database connections — it's taken months to eliminate most from LiveServerTestCase, and I'm sure there are still some left,
I don't think we should add threading without more compelling arguments.
comment:6 by , 8 years ago
Resolution: | wontfix |
---|---|
Status: | closed → new |
I'm hitting this problem, which means I have a concrete use case to offer.
I have a view that generates a PDF document by converting a HTML document served by the same server (with phantomjs). It works as follows:
- user makes a HTTP request for a PDF document
- pdf_view() starts a phantomjs subprocess
- phantomjs makes a HTTP request for the equivalent HTML document
- html_view() renders a HTML template
- phantomjs gets the HTML, converts it to PDF, and returns that
- pdf_view() returns the PDF
I cannot use that view with the LiveServerTestCase because it's single threaded. The thread runs pdf_view(), waits for phantomjs, but there's no thread available to run html_view(), so the test blocks there.
comment:7 by , 8 years ago
The following monkey-patch does the job for me.
import socketserver from django.test import testcases class ThreadedWSGIServer(socketserver.ThreadingMixIn, testcases.WSGIServer): """ Make Django's live server test case multi-threaded. See https://code.djangoproject.com/ticket/20238#comment:6 """ testcases.WSGIServer = ThreadedWSGIServer
I'm using the PostgreSQL database backend.
I don't think this proposal is sufficient for in-memory SQLite. A WSGIHandler subclass that replaces the database connection for each new thread by the database connection from the main thread is likely needed.
By the way, this piece of code in django.core.servers.basehttp
:
if threading: httpd_cls = type(str('WSGIServer'), (socketserver.ThreadingMixIn, WSGIServer), {}) else: httpd_cls = WSGIServer
could be replaced with:
httpd_cls = ThreadedWSGIServer if threading else WSGIServer
assuming the definition of ThreadedWSGIServer
above.
There's no reason (that I can think of) to define a class dynamically like that here. Saving a couple lines isn't worth the obfuscation.
comment:9 by , 8 years ago
Summary: | StoppableWSGIServer is not multithread support → Support multiple open connections in LiveServerTestCase |
---|---|
Triage Stage: | Unreviewed → Accepted |
Type: | Bug → New feature |
This was also requested in #25970.
comment:10 by , 8 years ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
comment:11 by , 8 years ago
Support for multiple open connections is also needed if you want to write a test that uses multiple browser instances as demonstrated in #27665 (closed as duplicate).
comment:12 by , 8 years ago
Has patch: | set |
---|---|
Needs tests: | set |
Version: | 1.4 → master |
I simply put Aymeric suggestion to a patch here: https://github.com/django/django/pull/7768
If someone has an idea for a test, please help.
comment:13 by , 8 years ago
Nadege also prepared a patch but hasn't submitted a PR yet. I was planning to take a look at her work after the holidays.
comment:15 by , 8 years ago
Needs tests: | unset |
---|
We actually made the sames changes, I added some tests as well.
They don't fail but hang whithout the patch.
I tried to do a test that could show potential database issue because aymeric sugested that might be a problem
but things seems good.
Patch is here https://github.com/django/django/pull/7832
comment:16 by , 8 years ago
Unless you're motivated to solve the Python 2 failures, we can target this patch for Django 2.0 which drops support for Python 2. The 1.11 feature freeze is Friday.
comment:18 by , 8 years ago
Patch needs improvement: | set |
---|
comment:19 by , 8 years ago
Patch needs improvement: | unset |
---|
comment:21 by , 4 years ago
FYI, I just filed #32416, which lists a possible regression caused by this fix.
Thank you for the report, however there is too little information here to identify if or where there is an issue. Could you please provide a detailed explanation of how to reproduce the issue you're facing, or even better, could you provide some code or a test case? Thanks!