SQLite test database path not recognised when running tests in parallel

[Damian Posener]

How to replicate:

1. Have a DATABASES configuration with one or more SQLite DBs in directories, like this:

   DATABASES = {
       'default': {
           'ENGINE': 'django.db.backends.sqlite3',
           'NAME': BASE_DIR / 'db' / 'default' / 'db.sqlite3',
       },
       'other': {
           'ENGINE': 'django.db.backends.sqlite3',
           'NAME': BASE_DIR / 'db' / 'other' / 'db.sqlite3',
       }
   }
   

2. Run tests with both the --parallel=auto and --keepdb flags, i.e. python manage.py test --keepdb --parallel=auto

Expected behaviour: .sqlite3 files should be kept in db/default and db/other directories

Actual behaviour: default_X.sqlite3 and other_X.sqlite3 files are saved in the root directory instead

Why is this a problem? When running multiple test suites at once, this leads to filename collision if two test suites are using the same database name(s). It is impossible to run two Django test suites in parallel from the same directory without encountering this issue.

Setting DATABASES['<alias>']['TEST']['NAME'] does not seem to solve this either.

I've put together an example project to demonstrate the issue. Simply run python manage.py test --keepdb --parallel=auto from the base directory.

Happy to answer any other questions. :)

[Abhishek Srivastava]

Hi 👋 — I’m trying to reproduce this bug to help confirm .

Here’s what I did so far:

1. Created a test project with the DATABASES config exactly as described.
2. Ran migrate — the .sqlite3 file was created correctly in db/default/.
3. Added a simple TestCase with multiple dummy tests.
4. Ran python manage.py test --keepdb --parallel=auto.

Environment:

• Django version: 6.0.dev20250612075530 (editable install)
• Python version: 3.12.11
• SQLite version: 3.43.2

I only see db/default/db.sqlite3 — no additional default_1.sqlite3 or other_1.sqlite3 files appear in the project root.

Am I missing any step to trigger the parallel test DB creation?
Any suggestions are appreciated — thanks!

[Simon Charette]

Why is this a problem? When running multiple test suites at once, this leads to filename collision if two test suites are using the same database name(s). It is impossible to run two Django test suites in parallel from the same directory without encountering this issue.

I don't think that's something Django's test machinery claims to supports by the way, I can't think of multiple things that would break running multiple test suites against the same checkout of a projects.

Am I missing any step to trigger the parallel test DB creation?

You need to have at least two TestCase otherwise ​these is nothing to parallelize.

[Damian Posener]

Replying to abhishek1999:

Hi 👋 — I’m trying to reproduce this bug to help confirm .

Here’s what I did so far:

1. Created a test project with the DATABASES config exactly as described.
2. Ran migrate — the .sqlite3 file was created correctly in db/default/.
3. Added a simple TestCase with multiple dummy tests.
4. Ran python manage.py test --keepdb --parallel=auto.

Environment:

• Django version: 6.0.dev20250612075530 (editable install)
• Python version: 3.12.11
• SQLite version: 3.43.2

I only see db/default/db.sqlite3 — no additional default_1.sqlite3 or other_1.sqlite3 files appear in the project root.

Am I missing any step to trigger the parallel test DB creation?
Any suggestions are appreciated — thanks!

There is a demo project attached to this ticket as a zip file. If you extract that and run tests you can replicate the issue hopefully.

[Damian Posener]

Replying to Simon Charette:

I don't think that's something Django's test machinery claims to supports by the way, I can't think of multiple things that would break running multiple test suites against the same checkout of a projects.

We are using Django as a CMS for 14 different websites, and the nature of that beast is a lot of separate apps, each with their own separate test suites. Being able to run app tests in parallel saves us a bunch of time, but this database thing is a bit of a thorn in our side.

I think there's also some (probably minor) security risks around Django simply dumping it's databases copies in the current working directory. For one the directory might not actually be writable. If it is writable, it may contain other .sqlite3 files that aren't meant to be overwritten (Django does not seem to check whether the DBs are its own and will happily overwrite existing ones).

Personally I'd be happy if there was at least a way to specify where Django should store the files, but there seems to be no other option than to use the current working dir.

[Simon Charette]

Personally I'd be happy if there was at least a way to specify where Django should store the files, but there seems to be no other option than to use the current working dir.

Have you tried using the explicit TEST.NAME ​setting?

I missed that part

Setting DATABASES['<alias>']['TEST']['NAME'] does not seem to solve this either.

I think we can accept on the basis that TEST.NAME should work.

[Abhishek Srivastava]

You need to have at least two TestCase otherwise ​these is nothing to parallelize.

Thanks, I was able to reproduce the issue.

[Abhishek Srivastava]

I tried this:

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.sqlite3",
        "NAME": BASE_DIR / "db" / "default" / "db.sqlite3",
        "TEST": {
            "NAME": BASE_DIR / "db" / "default" / "test_db.sqlite3",
        },
    },
    "other": {
        "ENGINE": "django.db.backends.sqlite3",
        "NAME": BASE_DIR / "db" / "other" / "db.sqlite3",
        "TEST": {
            "NAME": BASE_DIR / "db" / "other" / "test_db.sqlite3",
        },
    },
}

I added ['TEST']['NAME'] to keep test DB files in separate folders.
I ran tests with:

python manage.py test --parallel=auto --keepdb

Observations:

• Django tries to create test DB copies with suffixes (_1, _2, etc.) for each worker process
• The base .sqlite3 test files were created in a different folder than the project root, as configured.
• But when running in parallel, Django does not handle the folder part when adding suffixes . It gives following error

  sqlite3.OperationalError: unable to open database file
  

[Simon Charette]

Damian, do you have a requirement that parallel test databases are backed by actual files instead of being in memory per worker process which should ensure isolation?

What I mean is that if you want to avoid collisions until this gets fixed you could simply set DATABASES['<alias>']['TEST']['NAME'] = ':memory:' which is a common setup to speed up tests.

[Damian Posener]

Replying to Simon Charette:

Damian, do you have a requirement that parallel test databases are backed by actual files instead of being in memory per worker process which should ensure isolation?

What I mean is that if you want to avoid collisions until this gets fixed you could simply set DATABASES['<alias>']['TEST']['NAME'] = ':memory:' which is a common setup to speed up tests and ensure isolation.

That would rather defeat the point of using --keepdb, don't you think? :D We do run with in-memory DBs for a few applications, but that's an inefficient choice when testing apps that have a load of migrations in them.