Opened 9 months ago

Last modified 4 weeks ago

#29062 new Bug

"database table locked errors" when using sqlite in-memory database with LiveServerTestCase

Reported by: Juozas Masiulis Owned by: nobody
Component: Testing framework Version: 2.0
Severity: Normal Keywords: sqlite, testing, databases
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I am getting database table locked errors, when running tests which use sqlite in memory database.
This issue was introduced with this commit: https://github.com/django/django/commit/bece837829eafbc22f2598dadf82c9a8364b085a
I know this because reversing this commit resolves the issue for me.
If sqlite in memory testing is no longer supported, it should be mentioned in the documentation. If it is supported, there needs to be some parameter in the settings to use liveserver without multithreading. Or just make a new class liveservermultithreaded.

Change History (13)

comment:1 Changed 9 months ago by Tim Graham

Please provide a sample project that reproduces the problem.

comment:2 Changed 9 months ago by Tim Graham

Resolution: needsinfo
Status: newclosed
Summary: Sqlite in memory database does not work for testing."database table locked errors" when using sqlite in-memory database with LiveServerTestCase

comment:3 Changed 9 months ago by Juozas Masiulis

https://github.com/Euphorbium/django-bug-demo here is the sample project. Sorry for the mess in the test, the issue is with multiprocessing. The test pass with my version of django, check requirements, and fails with the unmodified django.

comment:4 Changed 9 months ago by Juozas Masiulis

Resolution: needsinfo
Status: closednew

comment:5 Changed 8 months ago by Tim Graham

Resolution: worksforme
Status: newclosed

Is the error consistent? I ran your sample project ten times and the test usually passes. Once in a while, I see this failure:

======================================================================
FAIL: testStuff (bugdemo.tests.TestDatabaseLocking)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django-bug-demo/bugdemo/tests.py", line 75, in testStuff
    self.assertEqual(Question.objects.count(), 30)
AssertionError: 29 != 30

I think you'll have to explain why Django is at fault to move this report forward. Possibly the problem might be dependent on the Python version. I tested with Python 3.4.7, 3.5.5, and 3.6.4.

comment:6 Changed 8 months ago by Juozas Masiulis

Resolution: worksforme
Status: closednew

It is a race condition. It is consistent in my system, but it gets locked by node process, and I did not want to go outside of python to demonstrate the bug. It needs something faster to lock the database consistently.

comment:7 Changed 8 months ago by Tim Graham

Unfortunately your comment doesn't help with my understanding of the problem. Can explain why Django is at fault or propose a solution? If not, I'm not sure how to move this issue forward.

comment:8 Changed 8 months ago by Tim Graham

Resolution: needsinfo
Status: newclosed

comment:9 Changed 8 months ago by Juozas Masiulis

Resolution: needsinfo
Status: closednew

Have you tried running it with my version, which is in the requirements.txt? That NEVER fails. This is proof that it is a django problem, and it was introduced in https://github.com/django/django/commit/bece837829eafbc22f2598dadf82c9a8364b085a

comment:10 Changed 8 months ago by Tim Graham

Just because the behavior changed doesn't mean Django is at fault. You'll need to investigate and explain why Django is at fault. It might be that your testing code is at fault.

comment:11 Changed 7 months ago by Tim Graham

Triage Stage: UnreviewedSomeday/Maybe

comment:13 Changed 4 weeks ago by Andrew Badr

Triage Stage: Someday/MaybeAccepted

Moving to 'Accepted' -- breaking test suites is a serious enough bug, and multiple people have hit it. Someday/Maybe is not appropriate here.

Note: See TracTickets for help on using tickets.
Back to Top