Opened 8 years ago

Last modified 6 months ago

#26822 new Bug

New migrations not applied on clone databases (sqlite) when using --parallel and --keepdb option

Reported by: Romain Garrigues Owned by: nobody
Component: Testing framework Version: 1.9
Severity: Normal Keywords:
Cc: romain.garrigues.cs@…, Sage Abdullah Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

I have noticed an issue when using --parallel and --keepdb option in test environment.

The first time I run my tests with both options, everything is fine.
If I run again my tests after having added a new migration, this migration is not applied to the clone databases, which makes the tests using these cloned databases fail.

After looking a bit on test.runner.setup_databases function, I noticed that the first database is created in connection.creation.create_test_db (with correct migrations applied to it), and then the cloned database creation happens in connection.creation.clone_test_db.

On sqlite, the function responsible of database cloning is db.backend.sqlite3.creation._clone_test_db, and in case of keepdb option, we don't touch the cloned databases:

if keepdb:
    return

But then, the migrations applied to the default database are not present in cloned ones.

I have noticed this behaviour on sqlite, I have not checked what happened on other databases.

If we remove this keepdb condition, that will work but slow down the test execution. A more interesting approach would probably be to apply migrations to all databases, not just the default one.

Any thoughts about that? Or maybe I have missed some details?

Change History (7)

comment:1 by Tim Graham, 8 years ago

Component: UncategorizedTesting framework
Triage Stage: UnreviewedAccepted

Seems like it would be helpful to fix that or document the limitation if it's not feasible.

comment:2 by Romain Garrigues, 8 years ago

I will try to propose something then!
I have the feeling that applying migrate to all cloned databases seems an interesting lead.

comment:3 by Romain Garrigues, 8 years ago

I have proposed a solution to fix this issue: https://github.com/django/django/pull/6884
Tell me what you think about this cloned database deletion/creation, even with keepdb option.

comment:4 by Romain Garrigues, 8 years ago

Has patch: set

comment:5 by Tim Graham, 8 years ago

Patch needs improvement: set

comment:6 by Simon Charette, 5 years ago

FWIW we worked around this issue by overriding our test suite runner's .parallel_test_suite.init_worker to perform a call_command('migrate', verbosity=0) after calling the Django defined init_worker. It feels like it should be performed at the create_test_db level though.

comment:7 by Sage Abdullah, 6 months ago

Cc: Sage Abdullah added
Note: See TracTickets for help on using tickets.
Back to Top