Opened 7 weeks ago

Last modified 6 weeks ago

#30000 assigned Bug

QuerySet constructed with .union() should raise an exception on unsupported filter() attempt.

Reported by: thoha Owned by: Hasan Ramezani
Component: Database layer (models, ORM) Version: 1.11
Severity: Normal Keywords: queryset union
Cc: Prabakaran Kumaresshan Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by thoha)

I have a form that displays large lists of selectable records using ModelMultipleChoiceFields. These fields need to show a predetermined list of records, obtained through some database queries performed within the Form's class init block.
The QuerySet required is constructed by reaching through a few ForeignKey relationships to retrieve the required records and then using the .union() method to assemble the QuerySet.
e.g.:

# Retrieve original record:
 base = BaseModel.objects.filter(source_name=source_name).first()
# Get related record via ForeignKey relationship:
related_record = base.related_record
# related_record.other_related_things has ManyToMany relationship to another Model (one or more records).
# So get them:
other_related_things = related_record.other_related_things.all()
# Loop through these:
union_queryset = None 
for other_related_thing in other_related_things:
    # Each other_related_thing has a set of SelectableRecords associated with it, via a ForeignKey relationship from the latter model to the former one. This gets that QuerySet:
    other_related_thing_queryset = other_related_thing.selectablerecords_set.all()
    # Build union_queryset:
    if union_queryset is None:
        union_queryset = other_related_thing_queryset
    else:
        union_queryset = union_queryset_object.union(other_related_thing_queryset)
        # The resulting union_queryset is a unified queryset containing all of the required fields. 
        # If only one other_related_thing exists, the result of evaluating the union_queryset QuerySet ought to be the 
        # same as the result of evaluating SelectableRecords.objects.all()

However, when I pass the union_queryset to my ModelMultipleChoiceFields, validating the form returns all of the records contained in union_queryset instead of the records selected on the form. The form displays correctly.

The code used for this is:

self.fields['selectable_records'] = forms.ModelMultipleChoiceField(queryset=union_queryset, required=False,
                                                    label='Selectable records',
                                                    widget=forms.CheckboxSelectMultiple)

Do take note that the documentation for the .union() method (here) indicates there are some limits to the QuerySet returned, and I assume my problem may have to do with this. Still, it would be welcome if this were explained somewhere (should this behaviour be normal), as it is rather unexpected.

Change History (8)

comment:1 Changed 7 weeks ago by thoha

Description: modified (diff)

comment:2 Changed 7 weeks ago by thoha

Keywords: queryset union forms added

comment:3 Changed 7 weeks ago by thoha

After some further testing it appears the unexpected behaviour comes from the fact that UNION querysets cannot be filtered, which the ModelMultipleChoiceField apparently does out of sight. Still, I believe that the observed behaviour is undesired, and the field should either accept the union queryset and work properly, or raise an exception if a union queryset is passed.

I also found that using

union_queryset = SelectableRecords.objects.filter(other_related_thing_name__in=other_related_things)

instead of the for-loop results in a queryset that actually works properly with the ModelMultipleChoiceField. Based on this StackOverflow question.

comment:4 Changed 7 weeks ago by thoha

Type: UncategorizedBug

comment:5 Changed 7 weeks ago by Simon Charette

Component: FormsDatabase layer (models, ORM)
Keywords: forms removed
Summary: Passing QuerySet constructed with .union() causes ModelMultipleChoiceField to return wrong valuesQuerySet constructed with .union() should raise an exception on unsupported filter() attempt.
Triage Stage: UnreviewedAccepted

comment:6 Changed 7 weeks ago by Prabakaran Kumaresshan

Cc: Prabakaran Kumaresshan added

comment:7 Changed 6 weeks ago by Hasan Ramezani

Owner: changed from nobody to Hasan Ramezani
Status: newassigned

should we raise an exception for all filter() attempts? or just for unsupported one?
if we should we raise an exception just for unsupported, is there any way to find them?

comment:8 in reply to:  7 Changed 6 weeks ago by thoha

Replying to Hasan Ramezani:

should we raise an exception for all filter() attempts? or just for unsupported one?
if we should we raise an exception just for unsupported, is there any way to find them?

Unsupported ones, I'd think. For my original problem, a simple check would be to do a count on the number of selections made in the form and comparing that to the number of records returned - normally they should match.

Note: See TracTickets for help on using tickets.
Back to Top