Opened 9 years ago

Closed 5 years ago

Last modified 5 years ago

#7190 closed Uncategorized (fixed)

BooleanField does not return <type: 'bool'>

Reported by: Jeffrey Froman Owned by: nobody
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: BooleanField, type
Cc: Leo Soto M. Triage Stage: Accepted
Has patch: yes Needs documentation: yes
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Ramiro Morales)

In some cases, a BooleanField returns an integer instead of a bool. The following example uses a MySQL backend:

# models.py
from django.db import models

class Simple(models.Model):
    b = models.!BooleanField()

$ ./manage.py syncdb
Creating table djest_simple
$ python
Python 2.5 (r25:51908, Sep 10 2007, 13:30:49)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-8)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from djest.models import Simple
>>> simple = Simple.objects.create(b=True)
>>> simple.b
True
>>> simple_2 = Simple.objects.get(pk=simple.pk)
>>> simple_2.b
1

This may not be a problem for normal usage, but makes testing, particularly using doctest, much less elegant.

Attachments (7)

boolfield.diff (945 bytes) - added by Jeffrey Froman <django.tcijf@…> 9 years ago.
patch for django/db/models/fields/init.py
mysql.patch (1.2 KB) - added by Pim Van Heuven 8 years ago.
django-real-bools.diff (2.9 KB) - added by Alex Gaynor 7 years ago.
django-real-bools.2.diff (4.4 KB) - added by Alex Gaynor 7 years ago.
django-real-bools.3.diff (5.5 KB) - added by Alex Gaynor 7 years ago.
django-real-bools.4.diff (5.9 KB) - added by Alex Gaynor 7 years ago.
django-boolean_mysql_r12847.patch (4.1 KB) - added by George Vilches 7 years ago.

Download all attachments as: .zip

Change History (32)

Changed 9 years ago by Jeffrey Froman <django.tcijf@…>

Attachment: boolfield.diff added

patch for django/db/models/fields/init.py

comment:1 Changed 9 years ago by James Bennett

This is kind of tricky, because under the hood in Python bool is a subclass of int that only ever has two instances (which test equal to 0 and 1 for False and True, respectively). Not all DBs actually store a boolean value, either; some store a 0 or a 1, and return that. Given that, and Python's general bent toward duck typing, I'm not sure whether we should strictly ensure that it always returns a bool instance.

comment:2 Changed 8 years ago by Ramiro Morales

Description: modified (diff)

Changed 8 years ago by Pim Van Heuven

Attachment: mysql.patch added

comment:3 Changed 8 years ago by glassfordm

Note: the approach in the mysql.patch attachment handles some cases that the approach in the boolfield.diff attachment does not: for example, the former fixes results returned by calling Model.objects.value(), and so fixes the doctests in django/tests/regressiontests/model_inheritance_regress/models.py; the latter does not fix these doctests.

Also, a note from the django developer's mailing list:

Pim Van Heuven wrote:

The lack of boolean type coercion is more serious than it looks like at
first glance.
(starting from the example at http://code.djangoproject.com/ticket/7190)

In [1]: import django.newforms as forms
In [2]: from django.newforms.models import model_to_dict
In [3]: from simple.models import Simple
In [4]: simple_false = Simple.objects.create(b=False)
In [5]: simple_false_2 = Simple.objects.get(pk=simple_false.pk)

In [6]: class SimpleForm(forms.ModelForm):

...: b = forms.BooleanField(widget=forms.HiddenInput)
...: class Meta:
...: model = Simple
...:

In [7]: SimpleForm(data = model_to_dict(simple_false)).as_p()
Out[7]: u'<input type="hidden" name="b" value="False" id="id_b" />'

In [8]: SimpleForm(data = model_to_dict(simple_false_2)).as_p()
Out[8]: u'<input type="hidden" name="b" value="0" id="id_b" />'

When you POST the Out[8] form the value becomes bool("0") = True.
So when you save simple_false_2 based on the form the value is inverted
from False
to True.
Seems like a critical error...

The mysql.patch solves this issue too.

Pim.

comment:4 Changed 8 years ago by Leo Soto M.

Cc: Leo Soto M. added

comment:5 in reply to:  1 Changed 8 years ago by anonymous

Replying to ubernostrum:

This is kind of tricky, because under the hood in Python bool is a subclass of int that only ever has two instances (which test equal to 0 and 1 for False and True, respectively). Not all DBs actually store a boolean value, either; some store a 0 or a 1, and return that. Given that, and Python's general bent toward duck typing, I'm not sure whether we should strictly ensure that it always returns a bool instance.

I dont think we can get away with duck typing here since it causes problems with the tests, Model.objects.value() and when posting a form with a hidden input for a boolean field (see example below).
After a quick grep it seems that the old mysql, sqlite and the oracle driver all have some kind of boolean type coercion.
Besides the mysql.path confines the changes to the mysql backend.

comment:6 Changed 8 years ago by Paul Kenjora

I applied the patch to 0.97-pre-SVN-5 with no effect. Please advise if anyone else has applied the patch with success, maybe missing some code on the attachments?

comment:7 in reply to:  1 ; Changed 8 years ago by bear330

Replying to ubernostrum:

This is kind of tricky, because under the hood in Python bool is a subclass of int that only ever has two instances (which test equal to 0 and 1 for False and True, respectively). Not all DBs actually store a boolean value, either; some store a 0 or a 1, and return that. Given that, and Python's general bent toward duck typing, I'm not sure whether we should strictly ensure that it always returns a bool instance.

Yes, 1 means true in python, but not at this situation:

>>> 1 is True
False

I also encounter this problem in my code. I think it should alway return True or False for Boolean field. If I use:

if field is True

that will fail. but if I use:

if field

That will be true if field is any non false value. (list, tuple, string...)

To make BooleanField always return True or False, it only need to modify BooleanField's to_python method.

Thanks.

comment:8 in reply to:  7 Changed 8 years ago by bear330

Replying to bear330:

To make BooleanField always return True or False, it only need to modify BooleanField's to_python method.

if value in (True, False): return value

=>

if value in (True, False): return value == True

Thanks.

comment:9 Changed 8 years ago by Eric Holscher

milestone: 1.0 maybe
Triage Stage: UnreviewedDesign decision needed

comment:10 Changed 8 years ago by Jacob

Resolution: wontfix
Status: newclosed

This basically comes down to a limitation of MySQL and MySQLdb. The cast function technically would work, but it's silly to cast any tinyint to bool.

comment:11 Changed 8 years ago by (none)

milestone: 1.0 maybe

Milestone 1.0 maybe deleted

comment:12 Changed 8 years ago by stengleind

Needs documentation: set
Resolution: wontfix
Status: closedreopened

By the way, this killed some code of mine that depended on True being True.
My functional test had an assertEqual(True, someBooleanField) and it silently
broke when I switched backends from sqlite to MySQL.

If it is not going to be fixed, it needs to be very clearly documented (which
it is currently not).

-Dave

comment:13 Changed 8 years ago by Jacob

Resolution: wontfix
Status: reopenedclosed

Please don't reopen tickets closed by a core developer, and especially not if you're going to change the subject. If you disagree with the wontfix call, bring it up on django-dev; if you'd like something else done (more docs are almost always a good idea), open a new ticket.

comment:14 Changed 8 years ago by stengleind

Ticket done (http://code.djangoproject.com/ticket/8802).

I will bring this up on the mailing list because:

1 == True
True

1 is True
False

-Dave

comment:15 Changed 8 years ago by to.roma.from.djbug@…

Could you please elaborate what exactly prevents you from applying one of the patches? I searched the mailing list archives and the only obstacle I found was inconsistency with qs.values() which would return 0/1 rather than True/False, but to my mind it’s values()’s behavior that should be changed, not the other way round (#10412). You call the int->bool cast “silly”, and maybe it is, but what are your actual concerns—performance hit, inconsistency, something else?

comment:16 Changed 7 years ago by James Bennett

milestone: 1.2
Resolution: wontfix
Status: closedreopened
Triage Stage: Design decision neededAccepted

Reopening after discussion in #django-dev, targeting for 1.2.

Note that the eventual fix for this can't be backported into 1.1 since we documented the behavior.

Changed 7 years ago by Alex Gaynor

Attachment: django-real-bools.diff added

Changed 7 years ago by Alex Gaynor

Attachment: django-real-bools.2.diff added

Changed 7 years ago by Alex Gaynor

Attachment: django-real-bools.3.diff added

Changed 7 years ago by Alex Gaynor

Attachment: django-real-bools.4.diff added

comment:17 Changed 7 years ago by jkocherhans

Resolution: fixed
Status: reopenedclosed

(In [12578]) Fixed #7190. Boolean fields now return bool values instead of 1 or 0. Thanks, Alex Gaynor.

comment:18 Changed 7 years ago by George Vilches

Resolution: fixed
Status: closedreopened

Re-opening this per Alex Gaynor's request. The current solution does not work for MySQL, MySQL continues to return 0 and 1. The attached patch (with more complete tests on models!) fixes this.

Changed 7 years ago by George Vilches

comment:19 Changed 7 years ago by Russell Keith-Magee

Resolution: fixed
Status: reopenedclosed

(In [12900]) Fixed #7190 -- Corrected a problem with Boolean value handling on the MySQL backend. Thanks to George Vilches for the initial patch.

comment:20 Changed 5 years ago by Jacob

milestone: 1.2

Milestone 1.2 deleted

comment:11 Changed 5 years ago by Ramiro Morales

In [17588]:

Fixed #15169 -- Corrected handling of model boolean fields in MySQL spatial backend.

Thanks goes to zmsmith and others for reporting the issue and to Justin Bronn for the fix.

Refs #7190, r12578, r12900, #13293, r12939.

comment:12 Changed 5 years ago by kirkm@…

Easy pickings: unset
Resolution: fixed
Severity: Normal
Status: closedreopened
Type: Uncategorized
UI/UX: unset

MySQL 5.0.3 changed bit fields from being tinyints to being some other thing (see: http://dev.mysql.com/doc/refman/5.0/en/bit-field-literals.html). Django now sees them as hex strings like '\x01' for b'1'. As a result, BooleanFields in models no longer work as expected. Setting a single-bit BooleanField using True and False in django works (the database gets '\x01' and '\x00' respectively), but reading these back does not get you a Python bool, but rather the raw string, like '\x00'. I think we need to tweak the Django code again to handle this case.

comment:13 in reply to:  12 ; Changed 5 years ago by Ramiro Morales

Resolution: fixed
Status: reopenedclosed

Replying to kirkm@…:

MySQL 5.0.3 changed bit fields from being tinyints to being some other thing (see: http://dev.mysql.com/doc/refman/5.0/en/bit-field-literals.html). Django now sees them as hex strings like '\x01' for b'1'. As a result, BooleanFields in models no longer work as expected. Setting a single-bit BooleanField using True and False in django works (the database gets '\x01' and '\x00' respectively), but reading these back does not get you a Python bool, but rather the raw string, like '\x00'. I think we need to tweak the Django code again to handle this case.

Hmm this would mean the tests added in r1290 when fixing #7190: https://github.com/django/django/blob/stable/1.4.x/tests/regressiontests/model_fields/tests.py#L178 should fail when run against MySQL >= 5.0.3 which I don't think is the case.

We will need more information about the exact conditions under which you are seeing this. Bonus points if you can express them in a test case.

But please open another ticket instead of reopening this one.

comment:14 in reply to:  13 ; Changed 5 years ago by anonymous

Replying to ramiro:

Hmm this would mean the tests added in r1290 when fixing #7190: https://github.com/django/django/blob/stable/1.4.x/tests/regressiontests/model_fields/tests.py#L178 should fail when run against MySQL >= 5.0.3 which I don't think is the case.

Ah, I'm using django 1.3.1 which I guess explains it.

comment:15 in reply to:  14 Changed 5 years ago by anonymous

Replying to anonymous:

Replying to ramiro:

Hmm this would mean the tests added in r1290 when fixing #7190: https://github.com/django/django/blob/stable/1.4.x/tests/regressiontests/model_fields/tests.py#L178 should fail when run against MySQL >= 5.0.3 which I don't think is the case.

Ah, I'm using django 1.3.1 which I guess explains it.

That didn't explain it, but I did find the problem. When I added that column, I used the "bit" datatype instead of tinyint which is what Django wants to use. Once I switched it to a tinyint, everything started behaving properly. Sorry for the alarm!

Note: See TracTickets for help on using tickets.
Back to Top