Opened 4 years ago

Last modified 5 days ago

#20238 assigned New feature

Support multiple open connections in LiveServerTestCase

Reported by: jwxiao@… Owned by: Nadege
Component: Testing framework Version: 1.4
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Run a selenium test with LiveServerTestCase. Django starts StoppableWSGIServer. The StoppableWSGIServer is not multithread support. When the codes to be tested call urllib.urlopen to the same server (ie: StoppableWSGIServer) the selenium test will hang and fail because of time-out.

Change History (10)

comment:1 Changed 4 years ago by Julien Phalip

Resolution: needsinfo
Status: newclosed

Thank you for the report, however there is too little information here to identify if or where there is an issue. Could you please provide a detailed explanation of how to reproduce the issue you're facing, or even better, could you provide some code or a test case? Thanks!

comment:2 Changed 4 years ago by Julien Phalip

Just a clarification: please feel free to reopen this ticket if you can provide more detailed information.

comment:3 Changed 3 years ago by Joeri Bekker

Although this is not my ticket, I just ran into the same issue.

The problem occurs typically when, within a request/response cycle, another request is done to the same StoppableWSGIServer. Now, the first request can never finish because the second request is waiting for the StoppableWSGIServer to respond (but it never will because it's waiting for the first request...)

# view
def index(self, request):
    if request.GET.get('inner_request') == 'true':
        # Within a request/response, perform another request to the same live server.
        # This one get's stuck.
        urllib.urlopen(request.build_absolute_uri())
    return HttpResponse()

# test
class LiveServerTests(LiveServerTestCase):
    def test_request_within_request(self):
        response = requests.get(self.live_server_url, params={'inner_request': 'true'})

        # This code is never reached (or the live server throws a timeout)
        self.assertEqual(response.status_code, 200)

The use case is somewhat fictional.

Last edited 3 years ago by Joeri Bekker (previous) (diff)

comment:4 Changed 2 years ago by bdrennan@…

Resolution: needsinfo
Status: closednew

comment:5 Changed 2 years ago by Aymeric Augustin

Resolution: wontfix
Status: newclosed

Considering that:

  • "the use case is somewhat fictional",
  • the ticket was reopened without any additional information,
  • more threading will certainly create more race conditions on shutdown, especially when it comes to the database connections — it's taken months to eliminate most from LiveServerTestCase, and I'm sure there are still some left,

I don't think we should add threading without more compelling arguments.

comment:6 Changed 8 days ago by Aymeric Augustin

Resolution: wontfix
Status: closednew

I'm hitting this problem, which means I have a concrete use case to offer.

I have a view that generates a PDF document by converting a HTML document served by the same server (with phantomjs). It works as follows:

  • user makes a HTTP request for a PDF document
  • pdf_view() starts a phantomjs subprocess
  • phantomjs makes a HTTP request for the equivalent HTML document
  • html_view() renders a HTML template
  • phantomjs gets the HTML, converts it to PDF, and returns that
  • pdf_view() returns the PDF

I cannot use that view with the LiveServerTestCase because it's single threaded. The thread runs pdf_view(), waits for phantomjs, but there's no thread available to run html_view(), so the test blocks there.

comment:7 Changed 8 days ago by Aymeric Augustin

The following monkey-patch does the job for me.

import socketserver

from django.test import testcases


class ThreadedWSGIServer(socketserver.ThreadingMixIn, testcases.WSGIServer):
    """
    Make Django's live server test case multi-threaded.

    See https://code.djangoproject.com/ticket/20238#comment:6

    """


testcases.WSGIServer = ThreadedWSGIServer

I'm using the PostgreSQL database backend.

I don't think this proposal is sufficient for in-memory SQLite. A WSGIHandler subclass that replaces the database connection for each new thread by the database connection from the main thread is likely needed.


By the way, this piece of code in django.core.servers.basehttp:

    if threading:
        httpd_cls = type(str('WSGIServer'), (socketserver.ThreadingMixIn, WSGIServer), {})
    else:
        httpd_cls = WSGIServer

could be replaced with:

    httpd_cls = ThreadedWSGIServer if threading else WSGIServer

assuming the definition of ThreadedWSGIServer above.

There's no reason (that I can think of) to define a class dynamically like that here. Saving a couple lines isn't worth the obfuscation.

comment:8 Changed 8 days ago by Aymeric Augustin

jwxiao, I'm sorry for not understanding your report originally.

comment:9 Changed 5 days ago by Tim Graham

Summary: StoppableWSGIServer is not multithread supportSupport multiple open connections in LiveServerTestCase
Triage Stage: UnreviewedAccepted
Type: BugNew feature

This was also requested in #25970.

comment:10 Changed 5 days ago by Nadege

Owner: changed from nobody to Nadege
Status: newassigned
Note: See TracTickets for help on using tickets.
Back to Top