Opened 5 months ago

Closed 3 months ago

#29413 closed Cleanup/optimization (fixed)

QuerySet.get_or_create()/update_or_create() shouldn't evaluate lazy defaults unless they are needed

Reported by: Viktor Danyliuk Owned by: Liuyang Qin
Component: Database layer (models, ORM) Version: 2.0
Severity: Normal Keywords: QuerySet get_or_create update_or_create lazy
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description (last modified by Viktor Danyliuk)

I wish to have ability to write something like this:

from django.utils.functional import lazy

obj, created = model.objects.get_or_create(
     key=jwt,
     defaults=lazy(self.get_defaults_for_model, dict)(jwt) 
)

But at the moment _extract_model_params prepare defaults before it's realy needed.
I think _extract_model_params should be separated to _prepare_model_lookup and _prepare_model_params.
So it will be possible to call _prepare_model_params when model.DoesNotExist or even inside _create_object_from_params.

Change History (9)

comment:1 Changed 5 months ago by Viktor Danyliuk

Description: modified (diff)

comment:2 Changed 5 months ago by Viktor Danyliuk

Description: modified (diff)

comment:3 Changed 5 months ago by Viktor Danyliuk

Easy pickings: set
Has patch: set

comment:4 Changed 5 months ago by Liuyang Qin

Owner: changed from nobody to Liuyang Qin
Status: newassigned

comment:5 Changed 5 months ago by Liuyang Qin

This function takes function and any number of classes. If to simplify, it returns wrapper(lets say "lazy function") instead of that function. At that point we can say that we turned function into lazy function. After that we can call this lazy function. Once called, it will return instance of proxy class, without calling the initial function instead of result of initial function. The initial function will be called only after we invoke any method on that result(proxy instance). *resultclasses here is the classes, instances of which are expected as results of the initial function
I wish to have ability to write something like this:

from django.utils.functional import lazy

obj, created = model.objects.get_or_create(
     key=jwt,
     defaults=lazy(self.get_defaults_for_model, dict)(jwt) 
)

But at the moment _extract_model_params prepare defaults before it's realy needed.
I think _extract_model_params should be separated to _prepare_model_lookup and _prepare_model_params.
So it will be possible to call _prepare_model_params when model.DoesNotExist or even inside _create_object_from_params.

comment:6 in reply to:  5 Changed 5 months ago by Viktor Danyliuk

The main issue is that defaults are processed before they are really needed.
If we have big defaults it will take some time to process it even if defaults are not needed (because object exists).

Replying to Liuyang Qin:

This function takes function and any number of classes. If to simplify, it returns wrapper(lets say "lazy function") instead of that function. At that point we can say that we turned function into lazy function. After that we can call this lazy function. Once called, it will return instance of proxy class, without calling the initial function instead of result of initial function. The initial function will be called only after we invoke any method on that result(proxy instance). *resultclasses here is the classes, instances of which are expected as results of the initial function

Last edited 5 months ago by Viktor Danyliuk (previous) (diff)

comment:7 Changed 5 months ago by Tim Graham

Summary: `QuerySet._extract_model_params` can/should be enchanced to support lazy defaultsQuerySet.get_or_create()/update_or_create() shouldn't evaluate lazy defaults unless they are needed
Triage Stage: UnreviewedAccepted

comment:8 Changed 4 months ago by Viktor Danyliuk

Is this going to be released? Can I help you somehow?

comment:9 Changed 3 months ago by Tim Graham <timograham@…>

Resolution: fixed
Status: assignedclosed

In 6ae7aaa:

Fixed #29413 -- Prevented evaluation of QuerySet.get_or_create()/update_or_create() defaults unless needed.

Removed the logic added in 81e05a418dc6f347d9627373288e773c477a9fe0 which
was obsolete since dbffffa7dc95fc62cbecfd00284bde62ee796f48.

Note: See TracTickets for help on using tickets.
Back to Top