Opened 10 years ago

Closed 10 years ago

#24064 closed Bug (fixed)

Spatialite tests could use or create real database

Reported by: Andriy Sokolovskiy Owned by: Andriy Sokolovskiy
Component: Database layer (models, ORM) Version: dev
Severity: Release blocker Keywords:
Cc: Andriy Sokolovskiy Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Modify your test config to be like this:

DATABASES = {
    'default': {
        'ENGINE': 'django.contrib.gis.db.backends.spatialite',
        'NAME': 'nonmemory',
        'TEST_NAME': ':memory:',
    },
    'other': {
        'ENGINE': 'django.contrib.gis.db.backends.spatialite',
        'NAME': 'nonmemory2',
        'TEST_NAME': ':memory:',
    }
}

Then run python runtests.py django.contrib.gis.tests.test_geoforms.
File with name nonmemory will be created, it is not correct.

More details about investigation:

Issue was found here:
https://github.com/django/django/pull/3677#issuecomment-68381272

Some investigation logs:

<truecoldmind> timograham, It is strange. Look at https://github.com/django/django/blob/51890ce8898f821d28f2f6fb6071c936e9bd88f0/django/contrib/gis/tests/utils.py#L39
<truecoldmind> This import causing problems.
<truecoldmind> For example if it is imported, python runtests.py backends django.contrib.gis.tests.test_geoforms will fail, but when I removed it (this import is not used in test_geoforms), tests passing correctly. Any ideas what it could be?
...
<truecoldmind> timograham, in this file connection.ops.spatial_version[0] used, which is performing some database operation. I will try to figure out what is the problem
<truecoldmind> without this call all is okay
<truecoldmind> timograham, calling cursor in https://github.com/django/django/blob/51890ce8898f821d28f2f6fb6071c936e9bd88f0/django/contrib/gis/db/backends/spatialite/operations.py#L211 causing problems. Does cursor close connection or something else?
...

timograham, when running tests, call to get spatialite version is calling cursor, which creates new connection, which is not a test database connection. I printed connection params, and saw that database name was ":memory:" instead of string which should be for new in-memory databases. It is telling us that test database was not used. You can check it in another way: sqlite allows to not specify database name, and if it will be not specified, and you will try to run `python runtests.py django.contrib.gis.tests.test_geoforms`, it will raise an exception "Please supply the NAME value.".
If I am right, this should be considered as another issue.
(:memory: was printed because it was specified in my settings as NAME)
Or, if running test_geoforms with NAME as some string, and TEST_NAME as :memory:, database file will be created. This is not correct, doesn't it?

Change History (8)

comment:1 by Andriy Sokolovskiy, 10 years ago

Component: UncategorizedDatabase layer (models, ORM)

comment:2 by Claude Paroz, 10 years ago

Triage Stage: UnreviewedAccepted

Basically, the problem is the type field of the SpatialiteGeometryColumns model depends on the Spatialite version. And as this happens at import time, the test machinery has not yet done its job.
See https://github.com/django/django/blob/51890ce8898f821d28f2f6fb6071c936e9bd88f0/django/contrib/gis/db/backends/spatialite/models.py#L16-L19

It's not the only location in Django where we could take advantage of a lazy model field instanciation, but I'm not aware of any such method currently. Ideas?

comment:3 by Andriy Sokolovskiy, 10 years ago

An idea how this could be resolved.
https://gist.github.com/coldmind/be61c330c7e56974bb51

This looks more good than creating meta magic on classes to make these fields to be dynamic

comment:4 by Tim Graham, 10 years ago

Severity: NormalRelease blocker

This is a release blocker as it causes the test from #12118 to fail on django-master-trusty.

comment:5 by Andriy Sokolovskiy, 10 years ago

Cc: Andriy Sokolovskiy added
Has patch: set
Owner: changed from nobody to Andriy Sokolovskiy
Status: newassigned

comment:6 by Tim Graham, 10 years ago

Patch needs improvement: set

Tests don't pass on Jenkins.

comment:7 by Andriy Sokolovskiy, 10 years ago

Patch needs improvement: unset

Tests are passing now, please review

comment:8 by Tim Graham <timograham@…>, 10 years ago

Resolution: fixed
Status: assignedclosed

In 839f431ef5b927c8b07ce33483b3a7b0dd17b761:

Fixed #24064 -- Prevented database access at compile time in spatialite models.

Note: See TracTickets for help on using tickets.
Back to Top