Opened 2 years ago

Last modified 5 months ago

#23222 assigned Bug

Empty BinaryField != b'' on Python 2

Reported by: wkschwartz@… Owned by: Grzegorz Ślusarek
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: py2
Cc: cmawebsite@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by William Schwartz)


In Python 2, at least under SQLite, the initial value for an empty binary field behaves inconsistently. The ORM thinks it's an empty bytes: b''. The database connection manager thinks it's a buffer. Now, the buffer evaluates to False and has zero length. So it'll mostly work. But not always -- and most importantly to me, not in my unit tests!


Note this was not a problem under Python 3.4.

Steps to Reproduce

Using Python 2.7.8, SQLite, and either Django 1.7rc2 or Django @ edcc75e5ac5b9dc2f174580e7adacd3be586f8bd (HEAD at the time of this writing; the error exists in both places)

  1. Make a new project and a new app and add the app to
  2. Fill in app/ as follows
from django.db import models                                                                                                         
class BinModel(models.Model):                                                                                                        
    data = models.BinaryField()                                                                                                      
  1. Run from the command line:
(venv) $ ./ makemigrations app && ./ migrate && ./ shell
  1. Run from the resulting Python shell
>>> from app import models; m = models.BinModel();; n = models.BinModel.objects.get()
>>> == b''
<read-write buffer ptr 0x10eaa62b0, size 0 at 0x10eaa6270>
>>> == b''
>>> bool(
>>> len(
>>> bytes(

Note that the same problem persisted when I had a default value for the field. There was no problem under Python 3.4.

Change History (8)

comment:1 Changed 2 years ago by Collin Anderson

Cc: cmawebsite@… added
Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Severity: Release blockerNormal
Version: 1.7-rc-2master

I'm seeing the same behavior on django 1.6.0 and stable/1.6.x, so I don't think this is a release blocker.

Last edited 2 years ago by Collin Anderson (previous) (diff)

comment:2 Changed 2 years ago by Areski Belaid

I dont think it's SQLite related, I have the same output using PostgreSQL

comment:3 Changed 2 years ago by Collin Anderson

Summary: Empty BinaryField != b'' in Python 2/SQLiteEmpty BinaryField != b'' on Python 2

comment:4 Changed 2 years ago by Baptiste Mispelon

Triage Stage: UnreviewedAccepted

Marking this as accepted per previous comments.

This seem like it might be caused by six using buffers on python 2 [1].



comment:5 Changed 22 months ago by ppYang

I'm not sure, if it's only a problem with empty BinaryField.

I try to add a test function in model_fields.tests.BinaryFieldTests, as below:

    def test_retrieve_compare_non_empty_field(self):
        dm = DataModel(data=b'test')
        self.assertTrue( == b'test')
        dm = DataModel.objects.get(
        self.assertTrue( == b'test')

On python2, the first assertion will be passed, and the second one will be failed.

As the data field is a buffer type after retrieving, we can't do a simple compare.

We can see the definition:

class BinaryField(Field):
    def get_db_prep_value(self, value, connection, prepared=False):
        value = super(BinaryField, self).get_db_prep_value(value, connection, prepared)
        if value is not None:
            return connection.Database.Binary(value)
        return value

I can't go further as I know little about six, and hope this can help:)

Last edited 22 months ago by ppYang (previous) (diff)

comment:6 Changed 19 months ago by Grzegorz Ślusarek

Owner: changed from nobody to Grzegorz Ślusarek
Status: newassigned

comment:7 Changed 19 months ago by William Schwartz

Description: modified (diff)

comment:8 Changed 5 months ago by Tim Graham

Keywords: py2 added
Note: See TracTickets for help on using tickets.
Back to Top