Opened 10 years ago

Closed 8 years ago

Last modified 2 months ago

#22288 closed Bug (fixed)

F() expression not compatible with __range field look up

Reported by: liushaohua86@… Owned by: Matthew Wilkes
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords:
Cc: github@…, josh.smeaton@…, matthew@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description (last modified by Tim Graham)

class TestModel(models.Model):
    a = models.SmallIntegerField()
    b = models.SmallIntegerField()

TestModel.objects.filter(a__range=(F('b')-1, F('b')+1)

TypeError: int() argument must be a string or a number, not 'ExpressionNode'

Change History (16)

comment:2 by Baptiste Mispelon, 10 years ago

Triage Stage: UnreviewedAccepted
Version: 1.5master

Hi,

I can reproduce this issue on latest master as well as on 1.6 so it doesn't appear to be related to the new custom lookup feature.

Regarding your formatting issues, you can make code blocks on trac using {{{ code goes here }}}, as explained on the WikiFormatting page.

Thanks.

comment:3 by Tim Martin, 10 years ago

Has patch: set

I've had a go at fixing this, the commit is here: https://github.com/django/django/compare/master...timmartin:bug22288

I'd like feedback on this fix, since I'm not sure I fully understand the potential wider impact of the changes I've made (though it passes all regression tests). It also looks like the changes in the pull request for #14030 will interact with this.

Last edited 8 years ago by Tim Graham (previous) (diff)

comment:4 by Anssi Kääriäinen, 10 years ago

Patch needs improvement: set

I checked the patch. It seems we need changes to too many places just to support something__range=(F('foo'), F('bar')). The main issue is that we have many rounds of value preparation.

The main issue here is that we just don't handle the case here value is an iterable correctly for value preparation. While the above mentioned commit touches the area around range lookups, we should likely fix all the lookups to lists of F() objects properly. Even if the only other example in core is somefield__in=[F()].

It might be better to wait for #14030 before applying this one.

In addition the tests of the above patch aren't correct, the code checks generated SQL, but that SQL differs between backends (quote character is not always ", and some backends IIRC use upper case table names).

Sorry for delay in commenting this issue.

comment:5 by Richard Eames, 9 years ago

Cc: github@… added

Any chance that this can be looked into now?

comment:6 by Josh Smeaton, 9 years ago

I think a side effect of https://github.com/django/django/pull/5090/ is that range and in lookups will now understand expressions properly. I'm not certain of this though, it just looks like similar code paths.

https://github.com/django/django/pull/5090/files#diff-b6b218ec29b7fb6a7d89868a94bfc73eR61

In fact, I'll incorporate the test case above into this patch to check.

Last edited 9 years ago by Josh Smeaton (previous) (diff)

comment:7 by Josh Smeaton, 9 years ago

Nope, the linked patch doesn't alter the path at all.

comment:8 by Tim Graham, 8 years ago

Description: modified (diff)

comment:9 by Matthew Wilkes, 8 years ago

I have had the same problem, albeit with date objects. I've begun working on a patch, WIP is at https://github.com/django/django/compare/master...MatthewWilkes:22288-iterables-and-F-objects

Current state is 1 failing test on sqlite (not tested on other backend) so certainly not ready for review at the moment. Posting here for visibility, as I've spoken to many of the people involved on IRC.

While working on this I've started to think that checking for expected parameter types in lookups being done by comparing their names is a bad idea, given the work done on custom lookups. Perhaps it would make sense for some of the value preparation to be offloaded onto the lookup, or at least the parsing of value data structures, to allow for richer lookups to use expressions too.

EDIT: On a practical note, I didn't intentionally unset the docs and tests parameters, they were unset on load and checking them changed the message to say that I'd changed it to checked. I guess they are newish fields?

Last edited 8 years ago by Matthew Wilkes (previous) (diff)

comment:10 by Josh Smeaton, 8 years ago

Cc: josh.smeaton@… added

comment:11 by Matthew Wilkes, 8 years ago

Cc: matthew@… added

I've added a WIP pull request at https://github.com/django/django/pull/6216

Any feedback on what additional testing would be required as well as on my general approach would be very much appreciated.

comment:12 by Matthew Wilkes, 8 years ago

#16487 is another instance of this

comment:13 by Matthew Wilkes, 8 years ago

I have updated the PR at https://github.com/django/django/pull/6216 - I believe this is in a good state but will need re-reviewing by ORM experts.

Is this something that should be against 1.10 or 1.11 at this stage?

comment:14 by Matthew Wilkes, 8 years ago

Owner: changed from nobody to Matthew Wilkes
Patch needs improvement: unset
Status: newassigned
Triage Stage: AcceptedReady for checkin

The PR is against 1.11 and ready for final review. Setting to ready for check-in following jarshwah's instructions on GitHub.

comment:15 by Tim Graham, 8 years ago

Patch needs improvement: set
Triage Stage: Ready for checkinAccepted

I left a few comments for improvement.

comment:16 by Tim Graham <timograham@…>, 8 years ago

Resolution: fixed
Status: assignedclosed

In 4f138fe5:

Fixed #22288 -- Fixed F() expressions with the range lookup.

comment:17 by Mariusz Felisiak <felisiak.mariusz@…>, 2 months ago

In 561f770:

Refs #22288 -- Corrected range lookup test names.

Note: See TracTickets for help on using tickets.
Back to Top