Queryset repr is not displayed on IPython >= 3.0 REPL if a NotImplementedError is raised

[Michael Angeletti]

I'm running SQLite3 , and the following doesn't result in an error or any visible output:

In [5]: Spam.objects.distinct('field')
Out[5]: 
In [6]: isinstance(Spam.objects.distinct('field'), models.QuerySet)
Out[6]: True

The queryset is not evaluated for some reason.

In [7]: repr(Spam.objects.distinct('field'))
---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
...
NotImplementedError: DISTINCT ON fields is not supported by this database backend

Maybe this bug is really "queryset not evaluated when it should be", rather than "doesn't raise an error when it should", since the following is evaluated as expected:

In [8]: Spam.objects.all()
Out[8]: [<Spam: Salty>, <Spam: Delicious>, <Spam: Goo>]

[Michael Angeletti]

I'm running the following:

Python 3.4.3

ipython==4.0.0
django==1.8.6

[Simon Charette]

Hi yoyoma,

As you might know QuerySet are lazily evaluated.

In this case Django needs to determine which backend will be used to run the query against (e.g. routers can be involved) before raising the NotImplementedError. That's the reason why the error will only be raised at execution time.

[Michael Angeletti]

Hi charettes,

I appreciate the quick response, but I'm afraid you didn't read the entire report. The queryset, in the context I've given, should have been evaluated. See the first line of code in the first example.

Take for instance, the example from ​https://docs.djangoproject.com/en/1.8/topics/db/optimization/#understand-cached-attributes:

>>> entry = Entry.objects.get(id=1)
>>> entry.authors.all()   # query performed
>>> entry.authors.all()   # query performed again

Placing a queryset directly into the REPL should result in evaluation.

[Baptiste Mispelon]

What's the definition of your Spam model?

When I try to do Spam.objects.distinct('field'), I get the NotImplementedError (as expected).

Doing the isinstance() check however doesn't evaluate the queryset but that seems like the expected behavior to me.

[Michael Angeletti]

bmispelon,

Correct. Running isinstance(queryset) should NOT evaluate the queryset. However, placing queryset directly into the shell should (e.g., the example that you said you tried - what version of Django/Python/ipython are you running?).

The following was actually copy/pasted from my shell, this very moment:

In [4]: Answer.objects.distinct('text')
Out[4]: 
In [5]: Answer.objects.distinct('query')
Out[5]: 
In [6]: type(Answer.objects.distinct('query'))
Out[6]: django.db.models.query.QuerySet

Does that even look normal? Of course not. Unless an expression is None, its shell representation is usually visible.

My model looks exactly like this:

class Answer(models.Model):
    created_at = models.DateTimeField(
        _('created at'),
        default=timezone.now,
        editable=False
    )
    updated_at = models.DateTimeField(_('updated at'), auto_now=True)
    query = models.ForeignKey('queries.Query', verbose_name=_('query'))
    text = models.TextField(_('text'))

There isn't a custom manager.

[Simon Charette]

It looks like I skimmed through the report a bit too fast.

[Baptiste Mispelon]

I'm using Django 1.8.6 and I tried with both Python 3.5.0 and 2.7.10 (plain python shell).

I've also tried using the same model as you and I see the NotImplementedError (as expected).

[Baptiste Mispelon]

I do observe the same behavior as you when I install IPython though.

My guess is that IPython is doing weird things with QuerySet.__repr__. Not sure if that's a Django bug though...

[Simon Charette]

I can reproduce with IPython 4.0.0 but not with 2.3.1. Looks like the issue might be there.

[Simon Charette]

From testing different versions it looks like this behavior was introduced by IPython 3.0.0

[Simon Charette]

It looks like NotImplementedError are treated differently from other exceptions since IPython 3.0.0

In [1]: class Foo(object):
   ...:     def __repr__(self):
   ...:         raise Exception
   ...:     

In [2]: Foo()
... traceback
In [3]: class Foo(object):
    def __repr__(self):
        raise NotImplementedError
   ...:     

In [4]: Foo()
Out[4]: 

[Michael Angeletti]

charettes,

Thanks for looking into this. I decided to page through the ​3.0 release issues (55 pages!) on GitHub, searching each page for "exception". Perhaps a Google search would have been easier, but I found the following:

1. ​https://github.com/ipython/ipython/issues/6384
2. ​https://github.com/ipython/ipython/pull/6394

I haven't had a chance to look through them completely (gotta run some errands), but I wanted to add them in the meantime.

[Simon Charette]

Thanks for the investigation yoyoma.

I'm unsure about what should be done here. IMO IPython is trying to be too smart here by swallowing the NotImplementedError but I'm tentatively accepting until we figure it out.

[Tim Graham]

I don't think Django can do anything about it. Relevant ​commit in ipython.