#19856 closed Bug (fixed)
LiveServerTestCase + in-memory SQLite do not work together
Reported by: | anonymous | Owned by: | nobody |
---|---|---|---|
Component: | Testing framework | Version: | 1.4 |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Accepted | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
When running LiveServerTestCase with in-memory SQLite database (no NAME in settings) the liveserver and the test case share the connections. The static files serving done to by liveserver will cause concurrent requests while the test is running, and these concurrent requests will cause close_connection to be called (and also will cause session checking by cookies, and thus concurrent queries).
The browser is free to ask some static files even after the real request has finished. Such files might be images for example.
This causes two issues:
- the newly introduced close_connection -> transaction.abort() call will cause constant spamming of transaction.rollback while the test case is running. This will cause random failures.
- even if close_connection isn't called, there will be concurrent queries by two different threads, and at least some versions of sqlite do not like this at all (segfaults are possible).
Unfortunately this affects the just released 1.4.4. This only happens when running liveserver tests with in-memory SQLite DB. The workaround is to use file based SQLite when testing.
Change History (9)
comment:1 by , 12 years ago
Triage Stage: | Unreviewed → Accepted |
---|
comment:2 by , 12 years ago
comment:3 by , 12 years ago
Yeah, something along those lines is what I narrowed this down. The patch is this: https://github.com/akaariai/django/commit/aaa6eb91b2ef4dc564b04c2b6958965f5e7b870f
I tried to run the test admin_widgets.HorizontalVertical... test 10 times in a row with 10/10 success rate.
The transaction.abort() call does play a role in worsening this, as does static files serving. The static files serving launch request_finished() signal, and the concurrent page load will cause static files requests concurrently with the test. This makes the failure much louder as there will be transaction.abort() calls going on while test teardown is also going on. Or at least this is my current working theory, I have made so many false claims in this issue already that verifying my claims is a good idea...
request_finished should be removed from liveserver's static files serving, and likely also from the liveserver altogether if using a shared connection. Otherwise the transaction.abort() will ruin any transaction management going on in the test case itself. When not using shared connections, then the liveserver's connection should likely be closed, as otherwise it is possible that the liveserver's connection is still open when the tests finish.
All in all, transaction.abort() could cause some issues for users doing transaction management in their test case + using liveserver + using in-memory sqlite. This is hopefully an unlikely scenario. The test fix should be applied to 1.4-master, and likely use the try-except construct as used in admin_inlines tests.
comment:4 by , 12 years ago
Has patch: | set |
---|
Improved commit at https://github.com/akaariai/django/commit/88a253df90f26bd7a5ecbf9f45f1b50ca921fde1. I think this is ready for commit now.
comment:5 by , 12 years ago
Patch needs improvement: | set |
---|
I think wait_page_loaded
should be a method of AdminSeleniumWebDriverTestCase
.
comment:6 by , 12 years ago
Patch needs improvement: | unset |
---|
comment:7 by , 12 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
I saw on IRC that you narrowed this down to the test not waiting for the next page to be loaded. Maybe something like this would be more robust: https://github.com/django/django/blob/master/tests/regressiontests/admin_inlines/tests.py#L550-L557