Opened 6 years ago

Closed 6 years ago

#29413 closed Cleanup/optimization (fixed)

QuerySet.get_or_create()/update_or_create() shouldn't evaluate lazy defaults unless they are needed

Reported by: Viktor Danyliuk Owned by: Liuyang Qin
Component: Database layer (models, ORM) Version: 2.0
Severity: Normal Keywords: QuerySet get_or_create update_or_create lazy
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description (last modified by Viktor Danyliuk)

I wish to have ability to write something like this:

from django.utils.functional import lazy

obj, created = model.objects.get_or_create(
     key=jwt,
     defaults=lazy(self.get_defaults_for_model, dict)(jwt) 
)

But at the moment _extract_model_params prepare defaults before it's realy needed.
I think _extract_model_params should be separated to _prepare_model_lookup and _prepare_model_params.
So it will be possible to call _prepare_model_params when model.DoesNotExist or even inside _create_object_from_params.

Change History (9)

comment:1 by Viktor Danyliuk, 6 years ago

Description: modified (diff)

comment:2 by Viktor Danyliuk, 6 years ago

Description: modified (diff)

comment:3 by Viktor Danyliuk, 6 years ago

Easy pickings: set
Has patch: set

comment:4 by Liuyang Qin, 6 years ago

Owner: changed from nobody to Liuyang Qin
Status: newassigned

comment:5 by Liuyang Qin, 6 years ago

This function takes function and any number of classes. If to simplify, it returns wrapper(lets say "lazy function") instead of that function. At that point we can say that we turned function into lazy function. After that we can call this lazy function. Once called, it will return instance of proxy class, without calling the initial function instead of result of initial function. The initial function will be called only after we invoke any method on that result(proxy instance). *resultclasses here is the classes, instances of which are expected as results of the initial function
I wish to have ability to write something like this:

from django.utils.functional import lazy

obj, created = model.objects.get_or_create(
     key=jwt,
     defaults=lazy(self.get_defaults_for_model, dict)(jwt) 
)

But at the moment _extract_model_params prepare defaults before it's realy needed.
I think _extract_model_params should be separated to _prepare_model_lookup and _prepare_model_params.
So it will be possible to call _prepare_model_params when model.DoesNotExist or even inside _create_object_from_params.

in reply to:  5 comment:6 by Viktor Danyliuk, 6 years ago

The main issue is that defaults are processed before they are really needed.
If we have big defaults it will take some time to process it even if defaults are not needed (because object exists).

Replying to Liuyang Qin:

This function takes function and any number of classes. If to simplify, it returns wrapper(lets say "lazy function") instead of that function. At that point we can say that we turned function into lazy function. After that we can call this lazy function. Once called, it will return instance of proxy class, without calling the initial function instead of result of initial function. The initial function will be called only after we invoke any method on that result(proxy instance). *resultclasses here is the classes, instances of which are expected as results of the initial function

Last edited 6 years ago by Viktor Danyliuk (previous) (diff)

comment:7 by Tim Graham, 6 years ago

Summary: `QuerySet._extract_model_params` can/should be enchanced to support lazy defaultsQuerySet.get_or_create()/update_or_create() shouldn't evaluate lazy defaults unless they are needed
Triage Stage: UnreviewedAccepted

comment:8 by Viktor Danyliuk, 6 years ago

Is this going to be released? Can I help you somehow?

comment:9 by Tim Graham <timograham@…>, 6 years ago

Resolution: fixed
Status: assignedclosed

In 6ae7aaa:

Fixed #29413 -- Prevented evaluation of QuerySet.get_or_create()/update_or_create() defaults unless needed.

Removed the logic added in 81e05a418dc6f347d9627373288e773c477a9fe0 which
was obsolete since dbffffa7dc95fc62cbecfd00284bde62ee796f48.

Note: See TracTickets for help on using tickets.
Back to Top