Module level database queries in a project prevent register_hstore from being called

[Alessandro Reichert]

Using hstorefield through the webserver works fine, but for applications that don't use the webserver like management commands we got a problem with models that use hstorefield. Among these problems is retrieving the hstorefield object which is retrieved as 'str' insteado of a 'dict'.

I found out a workaround that may be the cause of the bug but I've not enought knowledge to propose a fix.

Here is how to replicate the problem:

# subaccount is a hstorefield of AccountEntry model

In [8]: entry = AccountEntry.objects.get(pk=1)

In [9]: entry.subaccount
Out[9]: '"store"=>"velocity"'
In [10]: entry.subaccount.__class__
Out[10]: str

To get it working I'd to do this trick:

In [12]: from psycopg2.extras import register_hstore

In [13]: from django.db import connection

In [14]: register_hstore(connection.connection, globally=True, unicode=True)

In [15]: entry.subaccount
Out[15]: '"store"=>"velocity"'

In [16]: entry2 = AccountEntry.objects.get(pk=2)

In [17]: entry2.subaccount
Out[17]: {u'store': u'velocity'}

In [18]: entry2.subaccount.__class__
Out[18]: dict

Note that:
1) We are using django 1.8.4, python 2.7.9, psycopg2 2.6.1, postgresql 9.4.4
2) We installed the hstore extension in the template1 and we have created the django database after the hstore extension creation.
3) We have in our INSTALLED_APPS the django.contrib.postresql

Thanks!

[Simon Charette]

Hi reichert,

It looks like django.contrib.postresql.apps.PostgresConfig.ready which is supposed to deal with register_hstore is never called.

From what I can see you reproduced using an iPython shell, was it lauched using manage.py shell and if it's not the case did you make sure to call django.setup() to make sure all apps were initialized?

[Craig de Stigter]

I've just had this issue. Reopening because it's marked needsinfo and I have some :)

For us, we're using manage.py shell correctly, but it looks like we're doing a database query in a silly place in our code _before_ the app cache is ready.

That is, we're forcing a database connection, which in turn sends the connection_created signal before the register_hstore signal handler is registered by PostgresConfig.ready.

This is obviously something dumb we're doing, but it'd be nice if Django was able to throw an error when the database connection is initialised too early. Then we would have found this bug ages ago. Would that be possible?

[Tim Graham]

I'm not sure if we could do that or not (I think probably a lot of projects make queries at import time so a warning rather than exception might be needed), but could you provide a minimal project to reproduce it?

[Craig de Stigter]

Sure thing. Here's a minimal test project I whipped up:

​https://github.com/craigds/django_25454

[Tim Graham]

Thanks, I'll accept the ticket for investigation by an interested person (could be you if you are so inclined). We can always reclassify it as a documentation issue if needed.

[Claude Paroz]

I am also experiencing this on Django's own test suite when running postgres_tests.test_hstore.

​This PR should solve this.

[Claude Paroz <claude@…>]

In 283b4684:

Fixed #25454 -- Ensured register_hstore_handler is called for all connections

Thanks Simon Charette for help with the patch.