Opened 4 years ago

Closed 3 years ago

#24397 closed Cleanup/optimization (fixed)

Speed up model rendering with apps.ready = False

Reported by: Marten Kenbeek Owned by: Marten Kenbeek
Component: Migrations Version: master
Severity: Normal Keywords:
Cc: Marten Kenbeek Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Whenever a model is rendered, the cache of several values in model._meta is cleared on each known model. This has a severe performance impact when rendering large amounts of models.

When apps.ready is False, the cache of model meta options is not cleared. This speeds up the initial rendering of the test suite significantly. The cache can then be cleared manually when all models are rendered, and apps.ready can be set back toTrue.

Results timing django.test.runner.DiscoverRunner.setup_databases:

apps.ready == True  - 42 seconds
apps.ready == False - 16 seconds

Change History (9)

comment:1 Changed 4 years ago by Marten Kenbeek

Cc: Marten Kenbeek added

comment:2 Changed 4 years ago by Tim Graham

Triage Stage: UnreviewedAccepted

comment:3 Changed 4 years ago by Markus Holtermann <info@…>

Resolution: fixed
Status: newclosed

In 888c9b6429a44824078a49fb1ebacf2e950cd887:

Fixed #24397 -- Sped up rendering multiple model states.

Set apps.ready to False when rendering multiple models. This prevents
that the cache on Model._meta is expired on all models after each time a
single model is rendered. Prevented that Apps.clear_cache() refills the
cache on Apps.get_models(), so that the wrong value cannot be cached
when cloning a StateApps.

comment:4 Changed 3 years ago by Marten Kenbeek

comment:5 Changed 3 years ago by Markus Holtermann

Resolution: fixed
Status: closednew

Reopen due to further improvements.

comment:6 Changed 3 years ago by Tim Graham

Patch needs improvement: set

Left some mostly cosmetic comments. Please advance to "Ready for Checkin" after updating.

comment:7 Changed 3 years ago by Marten Kenbeek

Patch needs improvement: unset
Triage Stage: AcceptedReady for checkin

I'm not too sure what the error message should be changed to, so I'll leave it as is.

comment:8 Changed 3 years ago by Tim Graham <timograham@…>

In 039d7881:

Refs #24397 -- Sped up model reloading in ProjectState.

Created bulk_update() context manager on StateApps. Sped up unregistering
models in reload_models() by using this context mananger.

comment:9 Changed 3 years ago by Tim Graham

Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.
Back to Top