Code

Opened 5 years ago

Closed 5 years ago

Last modified 3 years ago

#9628 closed (fixed)

Using pysqlite2 instead of sqlite3 when needed

Reported by: mdh Owned by: jbronn
Component: Database layer (models, ORM) Version: 1.0
Severity: Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: yes
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

Loading SQLite extensions (such as SpatiaLite, which happens to be my use case) requires pysqlite 2.5.0, which is newer than the version that now ships as the sqlite3 module with Python. Currently Django always uses sqlite3 if it is present, even if the user has installed a newer version of pysqlite2.

The original reason for making sqlite3 the default was to “to do the right thing first and fallback” (#2772), which made the (seemingly reasonable) assumption that sqlite3 would become the undisputed correct name in the future. Unfortunately the package itself retained the pysqlite2 name, and only the Python-internal snapshots (which will inevitably become outdated) have the sqlite3 name.

So the question is, how can users take advantage of newer versions of pysqlite2, without having to go mucking around in their Python installation to disable the older sqlite3 module (or something equally unpleasant).

Options include:

  1. Leave sqlite3 as the default, and add a configuration setting that forces use of pysqlite2 if desired.
  1. Always try both sqlite3 and pysqlite2, and use whichever has the greater version number if both are present.
  1. Same as B, but with an optional configuration option to force one or the other if desired.
  1. Switch to making pysqlite2 the default, since that’s the correct name for the module if the user has explicitly installed it, and treat the Python-bundled version as the fallback.
  1. Develop arbitrarily convoluted bits of logic that pay attention to e.g. what Python version we're running under or the phase of the moon or something.

Several of these seem like reasonable options. For the moment I have created a simple patch that implements option B. I'm curious what others think about this.

It's not at all obvious to me how to write tests for a patch like this, since shouldn’t change the behavior in any way unless you happen to have a new version of pysqlite2 installed on your system. So, for the moment I have no tests. I would love some advice on what, if any, tests I could add.

Attachments (4)

use-pysqlite2-if-newer.diff (1.2 KB) - added by mdh 5 years ago.
A first patch to use whichever of sqlite3 and pysqlite2 is newer.
9628-r9781.diff (1.5 KB) - added by ramiro 5 years ago.
Another attempt at a patch for this that attampts to: Be more correct (no undefined exc, e1 vars), simpler, and appying what I learned from #8193
9628-r9791-optionD.diff (1.2 KB) - added by ramiro 5 years ago.
Patch implementing strategy D as per django-dev thread
9628-r9791-optionA.diff (1.5 KB) - added by ramiro 5 years ago.
Patch implementing strategy A as per django-dev discussion

Download all attachments as: .zip

Change History (10)

Changed 5 years ago by mdh

A first patch to use whichever of sqlite3 and pysqlite2 is newer.

Changed 5 years ago by ramiro

Another attempt at a patch for this that attampts to: Be more correct (no undefined exc, e1 vars), simpler, and appying what I learned from #8193

comment:1 Changed 5 years ago by mrts

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

comment:2 Changed 5 years ago by mrts

  • Needs documentation set
  • Patch needs improvement set
  • Triage Stage changed from Unreviewed to Design decision needed

Changed 5 years ago by ramiro

Patch implementing strategy D as per django-dev thread

Changed 5 years ago by ramiro

Patch implementing strategy A as per django-dev discussion

comment:3 Changed 5 years ago by ramiro

  • Patch needs improvement unset

I've attached two patches implementing both options D and A (in the last case the name choosen for the relevant setting.DATABASE_OPTIONS dictionary key is 'module').
If/when one of the patches is deemed correct, will add the needed documentation modifications.

comment:4 Changed 5 years ago by jbronn

  • milestone set to 1.1
  • Owner changed from nobody to jbronn
  • Status changed from new to assigned
  • Triage Stage changed from Design decision needed to Accepted

comment:5 Changed 5 years ago by jbronn

  • Resolution set to fixed
  • Status changed from assigned to closed

(In [10105]) Fixed #9628 -- Use pysqlite2 for database backend, if installed.

comment:6 Changed 3 years ago by jacob

  • milestone 1.1 deleted

Milestone 1.1 deleted

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.