Opened 3 years ago

Closed 10 months ago

#17656 closed Bug (worksforme)

collectstatic should not require a database

Reported by: jcspray@… Owned by: nobody
Component: contrib.staticfiles Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The collectstatic command, like all management commands, gets a call to self.validate() which requires a database to be up and running. This is annoying, because this command has no need for the database at all, and may well be run during packaging/deployment where a database may not be present.

Although hypothetically the STATICFILES_STORAGE setting could refer to something that stores static files in a database, the behaviour should be made more convenient for the default.

Change History (12)

comment:1 Changed 3 years ago by claudep

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted
  • Version changed from 1.3 to SVN

There is a flag called 'requires_model_validation' that commands can unset to not require validation.

comment:2 Changed 3 years ago by jezdez

  • Resolution set to fixed
  • Status changed from new to closed

In [17514]:

Fixed #17656 -- Stopped the collectstatic management command from running the database validation. Thanks, jcspray.

comment:3 Changed 22 months ago by anonymous

  • Resolution fixed deleted
  • Status changed from closed to new

comment:4 Changed 22 months ago by anonymous

This is broken again.... it loads translations here:

https://github.com/django/django/blob/master/django/core/management/base.py#L284

Which loads everything, any requires a database

comment:5 Changed 16 months ago by anonymous

I meet this bug on django version 1.4.10. when trying to write a multiple machine deployment script. I've stoped the database connection to avoid extra database hiting during the deployment. but collectstatic command still waiting for database connection until it is timed out.

comment:6 Changed 16 months ago by claudep

  • Resolution set to worksforme
  • Status changed from new to closed

This should not happen any more on a recent version of Django. Please provide a traceback if you can reproduce.

comment:7 Changed 10 months ago by MattBlack85

  • Resolution worksforme deleted
  • Status changed from closed to new

reopening, it seems like it still checks the database. In this case I just changed name and password.

Traceback (most recent call last):
  File "manage.py", line 12, in <module>
    execute_from_command_line(sys.argv)
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute
    django.setup()
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/__init__.py", line 21, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/apps/config.py", line 197, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/contrib/auth/models.py", line 40, in <module>
    class Permission(models.Model):
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/db/models/base.py", line 125, in __new__
    new_class.add_to_class('_meta', Options(meta, **kwargs))
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/db/models/base.py", line 300, in add_to_class
    value.contribute_to_class(cls, name)
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/db/models/options.py", line 166, in contribute_to_class
    self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/db/__init__.py", line 40, in __getattr__
    return getattr(connections[DEFAULT_DB_ALIAS], item)
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/db/utils.py", line 243, in __getitem__
    conn = backend.DatabaseWrapper(db, alias)
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 14, in __init__
    self.ops = PostGISOperations(self)
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 166, in __init__
    if self.spatial_version < (1, 3, 4):
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/utils/functional.py", line 55, in __get__
    res = instance.__dict__[self.func.__name__] = self.func(instance)
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 260, in spatial_version
    vtup = self.postgis_version_tuple()
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 434, in postgis_version_tuple
    version = self.postgis_lib_version()
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 414, in postgis_lib_version
    return self._get_postgis_func('postgis_lib_version')
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 404, in _get_postgis_func
    with self.connection.temporary_connection() as cursor:
  File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/db/backends/__init__.py", line 543, in temporary_connection
    cursor = self.cursor()
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/db/backends/__init__.py", line 165, in cursor
    cursor = self.make_debug_cursor(self._cursor())
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/db/backends/__init__.py", line 138, in _cursor
    self.ensure_connection()
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/db/backends/__init__.py", line 133, in ensure_connection
    self.connect()
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/db/backends/__init__.py", line 133, in ensure_connection
    self.connect()
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/db/backends/__init__.py", line 122, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 134, in get_new_connection
    return Database.connect(**conn_params)
  File "/home/matt/repos/uwncom/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect
    conn = _connect(dsn, connection_factory=connection_factory, async=async)
django.db.utils.OperationalError: FATAL:  role "234124323" does not exist

comment:8 Changed 10 months ago by timgraham

I'd guess this is probably impossible (or at least quite difficult) to make possible after the app loading refactor since all management commands now fully initialize Django.

comment:9 Changed 10 months ago by carljm

I think ideally "fully initialize Django" should not be equivalent to "require database connection." But you may be right that this will be difficult in practice.

comment:10 Changed 10 months ago by claudep

MattBlack85, the traceback shows that this is a PostGIS specific issue. I've created a new ticket to track that issue: #23514.
Now it is possible that this happens with a plain PostgreSQL connection, too, but it would be worth a try.

comment:11 Changed 10 months ago by MattBlack85

claudep, you're right, if I switch to postgres I can't reproduce this behaviour. This can be then closed, I'm going to add the traceback to the new ticket.

comment:12 Changed 10 months ago by claudep

  • Resolution set to worksforme
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.
Back to Top