Opened 4 weeks ago

Closed 3 weeks ago

#36997 closed Bug (needsinfo)

test_double_replaced_migrations_are_checked_correctly fails intermittently under Windows Subsystem for Linux (WSL)

Reported by: Stewart Matheson Owned by:
Component: Migrations Version: dev
Severity: Normal Keywords:
Cc: gghez Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Under WSL running Ubuntu 24.04 I am seeing intermittent test failures on

migrations.test_commands.SquashMigrationsTests.test_double_replaced_migrations_are_checked_correctly

The failure message I get is

ERROR: test_double_replaced_migrations_are_checked_correctly (migrations.test_commands.SquashMigrationsTests.test_double_replaced_migrations_are_checked_correctly)
If replaced migrations are already applied and replacing migrations
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/stewart/code/open-source/python/install/3.14.3/lib/python3.14/unittest/case.py", line 58, in testPartExecutor
    yield
  File "/home/stewart/code/open-source/python/install/3.14.3/lib/python3.14/unittest/case.py", line 669, in run
    self._callTestMethod(testMethod)

  File "/home/stewart/code/open-source/python/install/3.14.3/lib/python3.14/unittest/case.py", line 615, in _callTestMethod
    result = method()
    ^^^^^^^^^^^^^^^
  File "/home/stewart/code/open-source/django/source/tests/migrations/test_commands.py", line 3246, in test_double_replaced_migrations_are_checked_correctly
    call_command("migrate", "migrations", interactive=False, verbosity=0)
    ^^^^^^^^^^^
  File "/home/stewart/code/open-source/django/source/django/core/management/__init__.py", line 195, in call_command
    return command.execute(*args, **defaults)
      ^^^^^^^^^^^^^^^^^
  File "/home/stewart/code/open-source/django/source/django/core/management/base.py", line 466, in execute
    output = self.handle(*args, **options)
    ^^^^^^^^^^^^^^^
  File "/home/stewart/code/open-source/django/source/django/core/management/base.py", line 113, in wrapper
    res = handle_func(*args, **kwargs)
    ^^^^^^^^^^^
  File "/home/stewart/code/open-source/django/source/django/core/management/commands/migrate.py", line 115, in handle
    executor = MigrationExecutor(connection, self.migration_progress_callback)
    ^^^^^^^^^^^^^^^
  File "/home/stewart/code/open-source/django/source/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
    ^^^^^^^^^^^^^^^
  File "/home/stewart/code/open-source/django/source/django/db/migrations/loader.py", line 59, in __init__
    self.build_graph()
    ^^^^^^^^^^^
  File "/home/stewart/code/open-source/django/source/django/db/migrations/loader.py", line 282, in build_graph
    self.load_disk()
    ^^^^^^^^^^^^^^^
  File "/home/stewart/code/open-source/django/source/django/db/migrations/loader.py", line 87, in load_disk
    module = import_module(module_name)
    ^^^^^^^
  File "/home/stewart/code/open-source/python/install/3.14.3/lib/python3.14/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
      ^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1398, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1371, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1335, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'tmpx72x4ve0.migrations'

This failure is intermittent however I'm able to trigger it around 90% of the time I run the test.

Steps to reproduce

  1. Create New Ubuntu Machine under WSL
  2. Install python 3.14.3.
  3. Follow the guide here https://docs.djangoproject.com/en/6.0/intro/contributing/ to the point of executing tests with the runtests.py runner.
  4. Observe the above failure.

Change History (7)

comment:1 by Tim Graham, 4 weeks ago

Resolution: needsinfo
Status: newclosed
Summary: test_double_replaced_migrations_are_checked_correctly fails intermittently under WSL.test_double_replaced_migrations_are_checked_correctly fails intermittently under Windows Subsystem for Linux (WSL)

Thanks for the report. Absent an investigation which confirms that Django is at fault, I think we'll have to triage this as "needsinfo". Anyone is welcome to investigate and reopen with more details.

comment:2 by Stewart Matheson, 4 weeks ago

The issue relates is a file system a race condition that is most likely being triggered by the unpredictable timings of the filesystem running under HyperV and WSL. After the test writes the first migration file to the /tmp folder there is a chance that the filesystem mtime update is still waiting in the FS journal. When the migration system performs a second read its reading the filesystem before the mtime has been updated from the previous write. Because of this timing python's file cache is not invalidated when the test expects it to be. The result is that the first migration will be over written and the test will fail when it attempts to squash migrations that don't exist because they where never written in the first place.

Running the same test on bare metal linux, docker linux or macos is unlikely to trigger this issue is their filesystems have timings that are more predictable than WSL.

Running

 importlib.invalidate_caches()

Each time after a migration file is written resolves this issue for me as python will bypass reading the mtime. Given all that I'm not too sure how you would reproduce this as I would have thought other WSL users would be running in to this issue.

comment:3 by Stewart Matheson, 4 weeks ago

Resolution: needsinfo
Status: closednew

Hi Tim,

I provided my investigation and reopened this ticket as you have suggested.

comment:4 by gghez, 3 weeks ago

Cc: gghez added
Has patch: set

comment:6 by Simon Charette, 3 weeks ago

I closed the PR as no one has been able to reproduce on WSL so far and thus the ticket hasn't been accepted; submitting a solution without a clear demonstration that Django is at fault is precipitated.

comment:7 by Natalia Bidart, 3 weeks ago

Resolution: needsinfo
Status: newclosed

Hello Stewart, thank you for the detailed investigation.

Based on the current debugging information (related to filesystem timing and mtime visibility under WSL), and considering that:

  1. the issue has not been reproduced by others,
  2. there is no clear indication that Django is violating expected guarantees on supported platforms,
  3. adding explicit cache invalidation in Django would also need stronger justification, as it would introduce overhead and compensate for environment-specific behavior,

I'll keep this in needsinfo as originally triaged.

To move this forward, we would need a reliably reproducible case or confirmation from multiple environments demonstrating that Django is at fault. If you are able to provide a minimal reliable reproducer or additional evidence that this occurs outside of WSL-specific conditions, we can revisit.

Note: See TracTickets for help on using tickets.
Back to Top