Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#29727 closed Bug (fixed)

Lookup using F() on a non-existing model field doesn't throw an error

Reported by: Vidya Rani D G Owned by: Alexander Holmbäck
Component: Database layer (models, ORM) Version: 2.1
Severity: Release blocker Keywords: F() expressions
Cc: Mariusz Felisiak Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Fetching the value of a non-existing field on a model using F() object does not throw an error.

Instead, it is just giving the FK value of the instance.

Please find below a minimal test case to reproduce this issue.

# lineitem/models.py 

from django.db import models


class Account(models.Model):
    REGULAR = 'R'
    GOLD = 'G'
    PLATINUM = 'P'
    ACCOUNT_TYPE_CHOICES = (
        (REGULAR, 'Regular'),
        (GOLD, 'Gold'),
        (PLATINUM, 'Platinum'),
    )

    account_type = models.CharField(
        max_length=1,
        choices=ACCOUNT_TYPE_CHOICES,
        default=REGULAR,
    )


class Client(models.Model):
    name = models.CharField(max_length=50)
    registered_on = models.DateField()
    account_type = models.ForeignKey(Account, on_delete=models.CASCADE)

$ python manage.py shell

In [1]: from datetime import date, timedelta

In [2]: from models import Client, Account

In [3]: Client.objects.create(
   ...:     name='Jane Doe',
   ...:     account_type=Account.objects.create(account_type=Account.REGULAR),
   ...:     registered_on=date.today() - timedelta(days=36)
   ...: )
   ...: Client.objects.create(
   ...:     name='James Smith',
   ...:     account_type=Account.objects.create(account_type=Account.GOLD),
   ...:     registered_on=date.today() - timedelta(days=5)
   ...: )
   ...: Client.objects.create(
   ...:     name='Jack Black',
   ...:     account_type=Account.objects.create(account_type=Account.PLATINUM),
   ...:     registered_on=date.today() - timedelta(days=10 * 365)
   ...: )
   ...: 
   ...: 
Out[3]: <Client: Client object (6)>

In [4]: from django.db.models import F

In [5]: Client.objects.values(client_name=F('name'), account_name=F('account_type__name'))

Out[5]: <QuerySet [{'client_name': 'Jane Doe', 'account_name': 2}, {'client_name': 'James Smith', 'account_name': 3}, {'client_name': 'Jack Black', 'account_name': 4}, {'client_name': 'Jane Doe', 'account_name': 5}, {'client_name': 'James Smith', 'account_name': 6}, {'client_name': 'Jack Black', 'account_name': 7}]>

In [6]: print(queryset.query)
SELECT "lineitem_client"."name" AS "client_name", "lineitem_client"."account_type_id" AS "account_name" FROM "lineitem_client"

Attachments (1)

test_bogus_lookup.py (319 bytes ) - added by Matthew Schinckel 6 years ago.

Download all attachments as: .zip

Change History (20)

comment:1 by Matthew Schinckel, 6 years ago

I tried this in Django 1.11, and it raises the expected exception.

comment:2 by Curtis Maloney, 6 years ago

Needs tests: set
Triage Stage: UnreviewedAccepted

by Matthew Schinckel, 6 years ago

Attachment: test_bogus_lookup.py added

comment:3 by Matthew Schinckel, 6 years ago

Git bisect tells me it's:

2162f0983de0dfe2178531638ce7ea56f54dd4e7 is the first bad commit
commit 2162f0983de0dfe2178531638ce7ea56f54dd4e7
Author: Matthew Wilkes <git@matthewwilkes.name>
Date:   Sun Jun 18 16:53:40 2017 +0100

    Fixed #24747 -- Allowed transforms in QuerySet.order_by() and distinct(*fields).

comment:5 by Matthew Schinckel, 6 years ago

So, does it think it's a transform?

comment:6 by Curtis Maloney, 6 years ago

Even if it does, it should surely raise an error? I don't know of a transform called "name", do you?

comment:7 by Vidya Rani D G, 6 years ago

I tried this in Django 2.0 and it raises an exception.

in reply to:  7 comment:8 by Matthew Schinckel, 6 years ago

Replying to Vidya Rani D G:

I tried this in Django 2.0 and it raises an exception.

Okay, that's weird, because the test I wrote passed under 2.0

comment:9 by Alexander Holmbäck, 6 years ago

Added a test expecting FieldError when F() contains a join that ends with something other than a field or transform (test fails in 2.1a1 and above): https://github.com/alexh546/django/tree/ticket_29727

Edit: Patch is now included. I want to make sure that transforms work as expected so I'll look over the current tests and see if I should add some that combine joins and transforms.

Version 1, edited 6 years ago by Alexander Holmbäck (previous) (next) (diff)

comment:10 by Alexander Holmbäck, 6 years ago

Has patch: set
Owner: changed from nobody to Alexander Holmbäck
Status: newassigned

comment:11 by Alexander Holmbäck, 6 years ago

Needs tests: unset

comment:12 by Alexander Holmbäck, 6 years ago

Here's my take

In django.db.models.sql.Query:

setup_joins holds on to FieldException raised by names_to_path so that JoinInfo.transform_function can reraise if it doesn't find a satisfying transform.

It seems like resolve_ref doesn't need the return value from transform_function (unlike add_fields), so it didn't call it, which accidentally led FieldError to never be reraised.

When adding a call to transform_function from resolve_ref, all tests passes. Note how I assume targets to not be empty, that may be bad. Also, I haven't confirmed that there are tests for when joins and transforms are mixed, which could be relevant but also not.

comment:13 by Carlton Gibson, 6 years ago

Hi Alexander. Can you open a PR from your branch to master? Thanks.

comment:14 by Alexander Holmbäck, 6 years ago

Done: PR

Last edited 6 years ago by Tim Graham (previous) (diff)

comment:15 by Mariusz Felisiak, 6 years ago

Severity: NormalRelease blocker

IMO it should be marked as a "Release Blocker", because it is a regression in the 2.1 release.

comment:16 by Mariusz Felisiak, 6 years ago

Cc: Mariusz Felisiak added

comment:17 by Mariusz Felisiak, 6 years ago

Triage Stage: AcceptedReady for checkin

comment:18 by Tim Graham <timograham@…>, 6 years ago

Resolution: fixed
Status: assignedclosed

In f315d042:

Fixed #29727 -- Made nonexistent joins in F() raise FieldError.

Regression in 2162f0983de0dfe2178531638ce7ea56f54dd4e7.

comment:19 by Tim Graham <timograham@…>, 6 years ago

In bd5ce059:

[2.1.x] Fixed #29727 -- Made nonexistent joins in F() raise FieldError.

Regression in 2162f0983de0dfe2178531638ce7ea56f54dd4e7.

Backport of f315d0423a09dfe20dd4d4f6a0eb11fc8e45a665 from master

Note: See TracTickets for help on using tickets.
Back to Top