Opened 10 days ago

Last modified 6 days ago

#29413 assigned Cleanup/optimization

QuerySet.get_or_create()/update_or_create() shouldn't evaluate lazy defaults unless they are needed

Reported by: Viktor Danyliuk Owned by: Liuyang Qin
Component: Database layer (models, ORM) Version: 2.0
Severity: Normal Keywords: QuerySet get_or_create update_or_create lazy
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description (last modified by Viktor Danyliuk)

I wish to have ability to write something like this:

from django.utils.functional import lazy

obj, created = model.objects.get_or_create(
     key=jwt,
     defaults=lazy(self.get_defaults_for_model, dict)(jwt) 
)

But at the moment _extract_model_params prepare defaults before it's realy needed.
I think _extract_model_params should be separated to _prepare_model_lookup and _prepare_model_params.
So it will be possible to call _prepare_model_params when model.DoesNotExist or even inside _create_object_from_params.

Change History (7)

comment:1 Changed 10 days ago by Viktor Danyliuk

Description: modified (diff)

comment:2 Changed 10 days ago by Viktor Danyliuk

Description: modified (diff)

comment:3 Changed 9 days ago by Viktor Danyliuk

Easy pickings: set
Has patch: set

comment:4 Changed 9 days ago by Liuyang Qin

Owner: changed from nobody to Liuyang Qin
Status: newassigned

comment:5 Changed 7 days ago by Liuyang Qin

This function takes function and any number of classes. If to simplify, it returns wrapper(lets say "lazy function") instead of that function. At that point we can say that we turned function into lazy function. After that we can call this lazy function. Once called, it will return instance of proxy class, without calling the initial function instead of result of initial function. The initial function will be called only after we invoke any method on that result(proxy instance). *resultclasses here is the classes, instances of which are expected as results of the initial function
I wish to have ability to write something like this:

from django.utils.functional import lazy

obj, created = model.objects.get_or_create(
     key=jwt,
     defaults=lazy(self.get_defaults_for_model, dict)(jwt) 
)

But at the moment _extract_model_params prepare defaults before it's realy needed.
I think _extract_model_params should be separated to _prepare_model_lookup and _prepare_model_params.
So it will be possible to call _prepare_model_params when model.DoesNotExist or even inside _create_object_from_params.

comment:6 in reply to:  5 Changed 6 days ago by Viktor Danyliuk

The main issue is that defaults are processed before they are really needed.
If we have big defaults it will take some time to process it even if defaults are not needed (because object exists).

Replying to Liuyang Qin:

This function takes function and any number of classes. If to simplify, it returns wrapper(lets say "lazy function") instead of that function. At that point we can say that we turned function into lazy function. After that we can call this lazy function. Once called, it will return instance of proxy class, without calling the initial function instead of result of initial function. The initial function will be called only after we invoke any method on that result(proxy instance). *resultclasses here is the classes, instances of which are expected as results of the initial function

Last edited 6 days ago by Viktor Danyliuk (previous) (diff)

comment:7 Changed 6 days ago by Tim Graham

Summary: `QuerySet._extract_model_params` can/should be enchanced to support lazy defaultsQuerySet.get_or_create()/update_or_create() shouldn't evaluate lazy defaults unless they are needed
Triage Stage: UnreviewedAccepted
Note: See TracTickets for help on using tickets.
Back to Top