Opened 3 weeks ago

Last modified 3 weeks ago

#37343 assigned Cleanup/optimization

Adjust SynchronousOnlyOperation exception raised on implicit related fetches to point at using select_related/prefech_related.

Reported by: Mykhailo Havelia Owned by: Mykhailo Havelia
Component: Database layer (models, ORM) Version: 6.1
Severity: Normal Keywords: async orm
Cc: Mykhailo Havelia, Simon Charette Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Description:

Django 6.1 added fetch modes (FETCH_ONE, FETCH_PEERS, FETCH_RAISE). I'd like to propose using them to improve the async ORM experience.

Problem

Async Django ORM doesn't support the active-record pattern for related objects. Accessing an unloaded relation triggers an implicit synchronous fetch:

book = await Book.objects.aget(pk=1)
book.author.name  # implicit FK fetch

Without select_related() / prefetch_related(), this raises SynchronousOnlyOperation. That error is confusing here: it describes the mechanism (a sync DB call inside an async context) rather than the actual mistake (accessing a relation that wasn't loaded). A developer new to async Django can't easily tell from it what to do next (or it can make a silent sync call if DJANGO_ALLOW_ASYNC_UNSAFE is true).

Proposal

When a query runs in async mode, default its fetch mode to FETCH_ASYNC instead of relying on the implicit-sync path. Accessing an unloaded field would then raise FieldFetchBlocked with a message pointing to the fix

FieldFetchBlocked: 'author' isn't fetched. Load it with select_related()/prefetch_related().

This turns a low-level error into an actionable ORM error, at the exact spot where the developer needs guidance.

Context

This came up in django-async-backend. We can hardcode a custom fetch function there to raise our own error, but it would be much better to have consistent behavior inside Django itself.

Change History (7)

comment:1 by Simon Charette, 3 weeks ago

Cc: Simon Charette added
Triage Stage: Unreviewed → Accepted
Type: New feature → Cleanup/optimization

We had a related discussion in #37218 which resulted in ad289c1de4c39cc2b35748dc7a5ac84c7cbdf10a.

I believe you are asking that FETCH_ASYNC (which should likely be FETCH_ASYNC_ERROR) provides a more helpful error message to complement the recently documentation.

When a query runs in async mode

I'm making assumption here but I assume you mean when QuerySet.a{method} methods are used to retrieve model instances? I wonder if a better approach would be to have FETCH_ONE and FETCH_PEERS raise an adjusted exception when they kick in an async context instead of overriding QuerySet.fetch_mode at a* methods call time. That would cover cases where a query runs in an async context but the resulting model instances have a related attribute accessed in a sync context (e.g. in mixed function colour code bases).

comment:2 by Mykhailo Havelia, 3 weeks ago

I believe you are asking that FETCH_ASYNC (which should likely be FETCH_ASYNC_ERROR) provides a more helpful error message to complement the recently documentation.

Yes 🙂

I wonder if a better approach would be to have FETCH_ONE and FETCH_PEERS raise an adjusted exception when they kick in an async context instead of overriding QuerySet.fetch_mode at a* methods call time.

Do you mean a try/except with get_running_loop() inside FETCH_ONE/FETCH_PEERS? I think we can, but it affects every sync ORM operation too. get_running_loop() should be a cheap operation since it just reads a field from thread-local, but I need to check.

comment:3 by Simon Charette, 3 weeks ago

Do you mean a try/except with get_running_loop() inside FETCH_ONE/FETCH_PEERS?

Could we instead have them try/except the exception currently raised by the database backend that you believe is confusing an re-raise with a clearer error message?

in reply to:  3 comment:4 by Mykhailo Havelia, 3 weeks ago

Replying to Simon Charette:

Do you mean a try/except with get_running_loop() inside FETCH_ONE/FETCH_PEERS?

Could we instead have them try/except the exception currently raised by the database backend that you believe is confusing an re-raise with a clearer error message?

Yes, I like it

comment:5 by Mykhailo Havelia, 3 weeks ago

Owner: set to Mykhailo Havelia
Status: new → assigned

comment:6 by Simon Charette, 3 weeks ago

Summary: Use FETCH_ASYNC as the default fetch mode for async ORM queries (replace SynchronousOnlyOperation on implicit related fetches) → Adjust SynchronousOnlyOperation exception raised on implicit related fetches to point at using select_related/prefech_related.

comment:7 by Mykhailo Havelia, 3 weeks ago

Has patch: set
Note: See TracTickets for help on using tickets.
Back to Top