LiveServerTestCase in 1.10.4 doesnt close server ports properly

[Mike Hibbert]

I get the following error when using LiveServerTestCase along with lettuce:

Traceback (most recent call last):
  File "./manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/home/mike/Documents/python/bdd_older/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/home/mike/Documents/python/bdd_older/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 359, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/mike/Documents/python/bdd_older/local/lib/python2.7/site-packages/django/core/management/base.py", line 294, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/mike/Documents/python/bdd_older/local/lib/python2.7/site-packages/django/core/management/base.py", line 345, in execute
    output = self.handle(*args, **options)
  File "/home/mike/Documents/python/bdd_older/src/lettuce/lettuce/django/management/commands/harvest.py", line 191, in handle
    server.start()
  File "/home/mike/Documents/python/bdd_older/src/lettuce/lettuce/django/server.py", line 350, in start
    LiveServerTestCase.setUpClass()
  File "/home/mike/Documents/python/bdd_older/local/lib/python2.7/site-packages/django/test/testcases.py", line 1353, in setUpClass
    raise cls.server_thread.error
socket.error: [Errno 98] Address already in use

To reproduce the bug I've attached my Django project that can be run.

The error occurs when you try to run the tests. The first time through it works fine but if you try and run the tests again with about 1 - 2mins it will give the error above.

When I downgrade to 1.9.4 this works fine without the error.

Here's my code to reproduce the bug, the upload restrictions stopped me adding an attachment but you can see my test case here ​​https://www.dropbox.com/s/18uy88gz6frjyeg/bdd_app.tar.gz?dl=0

[Mike Hibbert]

The upload restrictions stopped me adding an attachment but you can see my test case here ​https://www.dropbox.com/s/18uy88gz6frjyeg/bdd_app.tar.gz?dl=0

[Tim Graham]

Could you please ​bisect to find the commit where the behavior changed?

[Mike Hibbert]

Replying to Tim Graham:

Could you please ​bisect to find the commit where the behavior changed?

I've installed and tested all releases between and the behavior begins to start happening with version 1.10a1 onwards

[Tim Graham]

Can you follow the bisect procedure that I linked? Your report doesn't contain enough details to explain why Django is at fault. It looks like ​lettuce isn't tested with Django 1.10 yet.

[Mike Hibbert]

Replying to Tim Graham:

Can you follow the bisect procedure that I linked? Your report doesn't contain enough details to explain why Django is at fault. It looks like ​lettuce isn't tested with Django 1.10 yet.

I'm not able to bisect it as I've checked the Django fork I made and I don't know enough about testing a test framework component enough to make it match the actual test case I've supplied.

I've narrowed it down to the fact that django.testcases.LiveServerTestCase implements the server creation and shutdown processes differently which is a breadcrumb.

My reason for suspecting Django to be at fault is that the lines of code that start the LiveServerTestCase from lettuce are the same and work for all previous versions of Django.

Also, lettuce does not implement any test server threading or port handling which is leveraged from Django so I'm pretty sure Django is the issue here unless there are some new considerations in the release notes that I haven't seen? I'm pretty sure the use case for LiveServerTestCase is as follows:

LiveServerTestCase.setUpClass()

# do some stuff then latter ..
LiveServerTestCase.tearDownClass()

This is correct isn't it?
.

[Mike Hibbert]

Actually, I think I've found it:

in 1.9.12:

def _create_server(self, port):
        return WSGIServer((self.host, port), QuietWSGIRequestHandler)

in 1.10.x:

def _create_server(self, port):
        return WSGIServer((self.host, port), QuietWSGIRequestHandler, allow_reuse_address=False)

Removing allow_reuse_address=False eliminates the bug.

[Mike Hibbert]

Should have mentioned that's in django.testcases.LiveServerThread

[Tim Graham]

#26011 is the ticket that made that change. It might be that the change reveals a bug in lettuce but I'm not sure -- maybe you can involve its author.

[Mike Hibbert]

Replying to Tim Graham:

#26011 is the ticket that made that change. It might be that the change reveals a bug in lettuce but I'm not sure -- maybe you can involve its author.

Aha! so it was put in there to allow testing windows to succeed. However, it's created a bug for non-windows users lol. Not like that ever happened before hehe.

I'll pop over to the lettuce guys but I'm pretty sure this is going to be a game of ping-pong with me as the ball as I doubt they will take ownership of your code changes.

Wouldn't a better solution be to check the host OS before hard coding this behavior?

[Marten Kenbeek]

I think the issue stems from ​this line and ​this one. By forcing Django to use a single port, the LiveServerTestCase won't try any other ports, so if that specific port is in use, it simply fails. allow_reuse_address = False may increase the live-span of connections that block the port, but lettuce would have the same problems when you use parallel testing of any sort, even when just running two separate test runners side by side (e.g. on a CI platform). Using an address range or port 0 largely avoids all these problems.

For now you can probably fix the error on your end by setting LETTUCE_SERVER_PORT = '8081-8179' in your test settings, which is the default range used by Django.

Django's current master has switched to using port 0 (#26791), which delegates the choice to the operating system. That should avoid any problems with addresses that are already in use, and it may be a good solution for lettuce to adopt as well.

[Tim Graham]

Sounds like no changes are required in Django.