Module level queries in a project can crash migrations if the queried model schema changed.

[Sven Coenye]

Starting with

class Graduation(models.Model):
    ceremony_date = models.DateField()
    ...

    @classmethod
    def current_graduation(cls):
        early = date.today() - timedelta(183)
        late = date.today() + timedelta(183)
        
        return cls.objects.get(ceremony_date__gte=early, ceremony_date__lte=late)

class Graduate(models.Model):
    conferral    = models.ForeignKey(Graduation)
    first_name   = models.CharField(max_length = 14)
    last_name    = models.CharField(max_length = 19)
    ...

and

class TestReport(ListView):
    queryset = Graduate.objects.filter(conferral=Graduation.current_graduation()) \
                               .order_by("last_name", "first_name")

    template_name       = "graduation/reports/applicants.html"
    context_object_name = "applicants"

The object is to automatically show the applicants for the next upcoming graduation. This works like a charm until the Graduation model is modified:

class Graduation(models.Model):
    ceremony_date = models.DateField()
    crash_test    = models.CharField(max_length=10)
    ...

    @classmethod
    def current_graduation(cls):
        early = date.today() - timedelta(183)
        late = date.today() + timedelta(183)
        
        return cls.objects.get(ceremony_date__gte=early, ceremony_date__lte=late)

At this point

python manage.py makemigrations

crashes (and so does pretty much any invocation besides asking for --help)

Full stack trace attached, but the cause appears to be that the entire application is loaded, triggering execution of the class method. This in turn ends up asking the database for a column that does not exist yet as the ORM generates a select including all field names.

Not defining queryset on the ListView and using get_queryset() fixes the problem, but as I originally got bitten by this when rolling a changeset out to production, I thought the condition is worth documenting.

[Sven Coenye]

Stack trace of manage.py crash

[Simon Charette]

Hi scoenye,

From what I can see this has not much to do with migrations but is related to the fact you should never execute queries at module level.

This is related to #25454 and probably a large number of closed tickets.

I wonder if should simply prevent django.db from executing queries until django.apps.apps.ready or at least issue a RuntimeWarning.

We would have to go through deprecation but I'm pretty sure this would uncover a lot of existing application bugs and prevent future ones.

[Sven Coenye]

Thank you. I did search here, the docs, and Google, but it looks like my keywords were not a good match for the other reported manifestations.

Perhaps adding an explicit note to the CBV documentation would help those of us who are still feeling their way through Django avoid this pitfall? As-is, it looks like the only reference to this issue is in the testing documentation.

[Tim Graham]

I started ​a discussion on django-developers based on Simon's proposal to add a warning or prohibit module level queries. Let's open a new ticket with action items based on the results.