Opened 2 years ago

Closed 19 months ago

#30000 closed Bug (duplicate)

QuerySet constructed with .union() should raise an exception on unsupported filter() attempt.

Reported by: Thomas Hamann Owned by: Hasan Ramezani
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: queryset union
Cc: Prabakaran Kumaresshan Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Thomas Hamann)

I have a form that displays large lists of selectable records using ModelMultipleChoiceFields. These fields need to show a predetermined list of records, obtained through some database queries performed within the Form's class init block.
The QuerySet required is constructed by reaching through a few ForeignKey relationships to retrieve the required records and then using the .union() method to assemble the QuerySet.

# Retrieve original record:
 base = BaseModel.objects.filter(source_name=source_name).first()
# Get related record via ForeignKey relationship:
related_record = base.related_record
# related_record.other_related_things has ManyToMany relationship to another Model (one or more records).
# So get them:
other_related_things = related_record.other_related_things.all()
# Loop through these:
union_queryset = None 
for other_related_thing in other_related_things:
    # Each other_related_thing has a set of SelectableRecords associated with it, via a ForeignKey relationship from the latter model to the former one. This gets that QuerySet:
    other_related_thing_queryset = other_related_thing.selectablerecords_set.all()
    # Build union_queryset:
    if union_queryset is None:
        union_queryset = other_related_thing_queryset
        union_queryset = union_queryset_object.union(other_related_thing_queryset)
        # The resulting union_queryset is a unified queryset containing all of the required fields. 
        # If only one other_related_thing exists, the result of evaluating the union_queryset QuerySet ought to be the 
        # same as the result of evaluating SelectableRecords.objects.all()

However, when I pass the union_queryset to my ModelMultipleChoiceFields, validating the form returns all of the records contained in union_queryset instead of the records selected on the form. The form displays correctly.

The code used for this is:

self.fields['selectable_records'] = forms.ModelMultipleChoiceField(queryset=union_queryset, required=False,
                                                    label='Selectable records',

Do take note that the documentation for the .union() method (here) indicates there are some limits to the QuerySet returned, and I assume my problem may have to do with this. Still, it would be welcome if this were explained somewhere (should this behaviour be normal), as it is rather unexpected.

Change History (9)

comment:1 Changed 2 years ago by Thomas Hamann

Description: modified (diff)

comment:2 Changed 2 years ago by Thomas Hamann

Keywords: queryset union forms added

comment:3 Changed 2 years ago by Thomas Hamann

After some further testing it appears the unexpected behaviour comes from the fact that UNION querysets cannot be filtered, which the ModelMultipleChoiceField apparently does out of sight. Still, I believe that the observed behaviour is undesired, and the field should either accept the union queryset and work properly, or raise an exception if a union queryset is passed.

I also found that using

union_queryset = SelectableRecords.objects.filter(other_related_thing_name__in=other_related_things)

instead of the for-loop results in a queryset that actually works properly with the ModelMultipleChoiceField. Based on this StackOverflow question.

comment:4 Changed 2 years ago by Thomas Hamann

Type: UncategorizedBug

comment:5 Changed 2 years ago by Simon Charette

Component: FormsDatabase layer (models, ORM)
Keywords: forms removed
Summary: Passing QuerySet constructed with .union() causes ModelMultipleChoiceField to return wrong valuesQuerySet constructed with .union() should raise an exception on unsupported filter() attempt.
Triage Stage: UnreviewedAccepted

comment:6 Changed 2 years ago by Prabakaran Kumaresshan

Cc: Prabakaran Kumaresshan added

comment:7 Changed 2 years ago by Hasan Ramezani

Owner: changed from nobody to Hasan Ramezani
Status: newassigned

should we raise an exception for all filter() attempts? or just for unsupported one?
if we should we raise an exception just for unsupported, is there any way to find them?

comment:8 in reply to:  7 Changed 2 years ago by Thomas Hamann

Replying to Hasan Ramezani:

should we raise an exception for all filter() attempts? or just for unsupported one?
if we should we raise an exception just for unsupported, is there any way to find them?

Unsupported ones, I'd think. For my original problem, a simple check would be to do a count on the number of selections made in the form and comparing that to the number of records returned - normally they should match.

comment:9 Changed 19 months ago by Mariusz Felisiak

Resolution: duplicate
Status: assignedclosed
Version: 1.11master

filter() and exclude() are unsupported but doesn't raise an error yet. As ​per the documentation, "only LIMIT, OFFSET, COUNT(*), ORDER BY, and specifying columns (i.e. slicing, count(), order_by(), and values()/values_list()) are allowed on the resulting QuerySet.".

Duplicate of #27995.

See also a ticket #28519 to support these features.

Note: See TracTickets for help on using tickets.
Back to Top