Code

Opened 5 years ago

Closed 5 years ago

Last modified 3 years ago

#10417 closed (worksforme)

Some model instances returned from a query set do not have any attributes filled in.

Reported by: kevinvw Owned by: nobody
Component: Database layer (models, ORM) Version: master
Severity: Keywords: Q, "multi-table inheritance"
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

By combining two Q functions in my queryset with a OR (|) relation between them over a multi-table inherited model, i get the following bug:

Example of bug

>>> p = Project.objects.get(pk=1)

>>> for member in StaffMember.objects.all().filter(Q(publications_as_author__belongsTo = p) | Q(publications_as_coauthor__belongsTo = p)):
... 	print member.pk
... 
3
4

>>> for member in StaffMember.objects.all().filter(Q(publications_as_author__belongsTo = p) | Q(publications_as_coauthor__belongsTo = p)):
... 	print member.first_name
... 
Sven
None # << the bug, should be "Kevin". as proven on the line below.

>>> member = StaffMember.objects.get(pk=4)
>>> member.first_name
u'Kevin'

rozwell from #django had the following to say about this bug:
01:24 < rozwell> so it seems to me that this is happening when the queryset gets collapsed into a list of model instances

Necessary information to reproduce bug

class Publication(models.Model):
	titled = models.CharField(max_length=255)
	co_authors = models.ManyToManyField('StaffMember', related_name="publications_as_coauthor")
	belongsTo = models.ManyToManyField('Project', related_name="has_publications")
	authors = models.ManyToManyField('StaffMember', related_name="publications_as_author")

admin.site.register(Publication)

class Project(models.Model):
	named = models.CharField(max_length=255)
	hasStaffMembers = models.ManyToManyField('StaffMember', related_name="workson")

admin.site.register(Project)

class Person(models.Model):
	first_name = models.CharField(max_length=255)
	last_name = models.CharField(max_length=255)

# if you uncomment this line, you'll notice that the django shell throws an exception
# because it can't concattenate None to String.
# first_name is apparently None in that instance (but it's not.)
# and last_name is apparently "" in that instance (but again, it's not, in my case it was "Van Wilder".

#	def __str__(self):
#		return self.first_name + " " + self.last_name

admin.site.register(Person)

class StaffMember(Person):
	works_at = models.CharField(max_length=255)

SQL Query

The resulting SQL query from the queryset is as follows, but seemed okay:

>>> StaffMember.objects.all().filter(Q(publications_as_author__belongsTo = p) | Q(publications_as_coauthor__belongsTo = p)).query.as_sql()
('SELECT "wsdmsite_person"."id", "wsdmsite_person"."first_name", "wsdmsite_person"."last_name", "wsdmsite_staffmember"."person_ptr_id", "wsdmsite_staffmember"."works_at" FROM "wsdmsite_staffmember" LEFT OUTER JOIN "wsdmsite_publication_authors" ON ("wsdmsite_staffmember"."person_ptr_id" = "wsdmsite_publication_authors"."staffmember_id") LEFT OUTER JOIN "wsdmsite_publication" ON ("wsdmsite_publication_authors"."publication_id" = "wsdmsite_publication"."id") LEFT OUTER JOIN "wsdmsite_publication_belongsTo" ON ("wsdmsite_publication"."id" = "wsdmsite_publication_belongsTo"."publication_id") LEFT OUTER JOIN "wsdmsite_publication_co_authors" ON ("wsdmsite_staffmember"."person_ptr_id" = "wsdmsite_publication_co_authors"."staffmember_id") LEFT OUTER JOIN "wsdmsite_publication" T7 ON ("wsdmsite_publication_co_authors"."publication_id" = T7."id") LEFT OUTER JOIN "wsdmsite_publication_belongsTo" T8 ON (T7."id" = T8."publication_id") INNER JOIN "wsdmsite_person" ON ("wsdmsite_staffmember"."person_ptr_id" = "wsdmsite_person"."id") WHERE ("wsdmsite_publication_belongsTo"."project_id" = %s  OR T8."project_id" = %s )', (1, 1))
>>>

Notes

If I remove any of the two Q functions and the OR relationship, it works perfectly as you would expect it:

>>> p = Project.objects.get(pk=1)

>>> StaffMember.objects.all().filter(Q(publications_as_author__belongsTo = p))
[<StaffMember: StaffMember object>]
>>> StaffMember.objects.all().filter(Q(publications_as_author__belongsTo = p))[0].first_name
u'Kevin'

>>> StaffMember.objects.all().filter(Q(publications_as_coauthor__belongsTo = p))
[<StaffMember: StaffMember object>]
>>> StaffMember.objects.all().filter(Q(publications_as_coauthor__belongsTo = p))[0].first_name
u'Sven'

So i would guess it has something to do with the OR relationship.

Attachments (0)

Change History (5)

comment:1 in reply to: ↑ description Changed 5 years ago by kevinvw

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

Replying to kevinvw:

# if you uncomment this line, you'll notice that the django shell throws an exception
# because it can't concattenate None to String.
# first_name is apparently None in that instance (but it's not.)
# and last_name is apparently "" in that instance (but again, it's not, in my case it was "Van Wilder".

My apologies, this is false, last_name was also None. It is the " " string in between that the "Can't concat None to String" was referring to.

comment:2 Changed 5 years ago by jacob

  • milestone changed from 1.1 beta to 1.1
  • Triage Stage changed from Unreviewed to Accepted

comment:3 Changed 5 years ago by mtredinnick

I can't replicate this using the models given in the description.

If the original reporter is still seeing the problem with trunk, please try to simplify the test case a bit and provide more information about what data populates what. For example, all the ManyToMany fields really necessary? Can you get rid of the Project model entirely? What's the minimum amount of data you have to assign to these models to see the problem.

You should also fix the example to make things as correct as possible by removing those calls to admin.register() in case they're causing some weird side-effects (since you should not be calling register() from a models.py file).

If I take the current models, create two publications, assign a coauthor in one and a different author in the other one and assign them both to the same project, I don't have any missing data or crashes.

At the moment, this needs more information about how to repeat before we can work out what's going on.

comment:4 Changed 5 years ago by jacob

  • Resolution set to worksforme
  • Status changed from new to closed

Marking worksforme since Malcolm can't reporoduce. Please reopen if someone can provide a reproducible test case.

comment:5 Changed 3 years ago by jacob

  • milestone 1.1 deleted

Milestone 1.1 deleted

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.