Enforce ordering of checks.

[David Szotten]

​https://github.com/django/django/pull/8967 changes the checks registry to use sets instead of lists. i may be missing it, but i can't see a ticket reference or any reason for the change

i just had a bug where the behaviour of a generated file (happened to be by ​https://github.com/adenh93/django-typomatic) depends on collecting classes, which in turn depends on the import order of files in my project

i had two different invocations get different orderings of checks causing different import orders

could we consider switching back to lists? or e.g. using sorted instead of list in get_checks ​https://github.com/django/django/blob/8036b53de61f16098a2f22c50621987182bdfaed/django/core/checks/registry.py#L96

[Carlton Gibson]

Hi David.

I don't think it's really part of the System Check API/contract that they're run in any order. I don't think making it such is a good idea. (Indeed it might be a feature that they're not...) We could throw in the sorted — what would it hurt? — but then folks would end up using/depending on it. As such I'm inclined to wontfix.

The change from the PR was so long ago now I can't see a realistic case for arguing there's a regression here. A set was the right collection type all along.

So to your problem, I think a change in django-typomatic is the way to go. Even if we accepted this here it would be roughly a year before it was isn an available version of Django. Applying some kind of sort in django-typomatic, on the other hand, could be available ≈immediately, and it's probably something it should do anyway…

[David Szotten]

to be clear, i wasn't suggesting guaranteeing any _particular order, but rather making sure the order is consistent between invocations and systems

django-typomatic was my example (and i fixed my issue by disabling the checks for the management command in question), but the general problem is hard-to-track-down bugs caused by different import orders on different systems (in my case i have yet to figure out what even _caused the iteration order to change)

(happy for you to re-close if you still think that's the best action here, but wanted to clarify my suggestion)

[Carlton Gibson]

Hey David. I'll reclose since it's best to take these kind of discussions to the DevelopersMailingList (as per the ​triage guidelines for wontfix tickets)

My thought was that really System Checks should be able to run in any order — like --random for a test runner — that was my point about Indeed it might be a feature.... Maybe that's not right though, so seeing what a wider audience think may be worthwhile.

...bugs caused by different import orders ...

I don't know but, my thought here is that ordering the system checks is at best a band-aid over a deeper problem here. Import order isn't really something that's under user-control. Relying on it seems to be asking for trouble... — but maybe others can speak differently.