Opened 6 years ago

Closed 9 months ago

#14286 closed New feature (fixed)

Support for BigAutoField

Reported by: hongrich Owned by: mmcnickle
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords:
Cc: Michael, Sergey Kolosov, ethan.jucovy@…, ludo@…, stodge@…, cmawebsite@…, drakkan, samir, mike@…, marti@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

1.2 added BigIntegerField, but there is still no support for using bigint as auto increment primary key. Could there be an official version of BigAutoField?

Reference of various attempts over the years:

http://djangosnippets.org/snippets/1244/

http://code.djangoproject.com/ticket/9625

http://code.djangoproject.com/ticket/13774

Attachments (2)

14286-Add_BigAutoField.diff (7.7 KB) - added by mmcnickle 6 years ago.
Adds BigAutoField as a new field type, with documentation.
bigauto.diff (9.2 KB) - added by jamercee 4 years ago.
BigAutoField patch file.

Download all attachments as: .zip

Change History (42)

comment:1 Changed 6 years ago by maaatt

Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset

In the meantime, it suffices to use a standard AutoField and hack the database like so:

In the table in question, alter the field like so: ALTER TABLE [table_name] ALTER COLUMN [field_name] [data_type], where [data_type] is one of the following:

MySQL: bigint AUTO_INCREMENT
Oracle: NUMBER(19)
PostgreSQL: bigserial
SQLite: integer

For every foreign key pointing to your field, you will also need to alter the column, but with these data types:

MySQL: bigint
Oracle: NUMBER(19)
PostgreSQL: bigint
SQLite: integer

This is a pain, but you only need to do it when creating tables, once. On the other hand, users shouldn't have to hack the database for such a simple feature, in my opinion. It would be super-cool if 1.3 included a fully-functional BigAutoField.

comment:2 Changed 6 years ago by Russell Keith-Magee

Triage Stage: UnreviewedAccepted

Of course, this can be easily handled as an external field definition, but given that BigInteger is in trunk, I suppose it makes sense for there to be a BigAutoField too.

comment:3 Changed 6 years ago by mmcnickle

Owner: changed from nobody to mmcnickle

Changed 6 years ago by mmcnickle

Attachment: 14286-Add_BigAutoField.diff added

Adds BigAutoField as a new field type, with documentation.

comment:4 Changed 6 years ago by mmcnickle

Has patch: set
Needs tests: set
Version: 1.2SVN

comment:5 Changed 5 years ago by Julien Phalip

Severity: Normal
Type: New feature

comment:6 Changed 5 years ago by Robin Jarry

Easy pickings: unset
UI/UX: unset

Hello guys,

Since 1.4 is to be released soon™, could this feature be integrated into it? I looked in the trunk and did not find it. It shouldn't have much impact on anything.

I could perform the integration of the patch proposed by mmcnickle, but since I'm new here, I'm not sure if I can :)

Thanks!!! Django is a wonderfull piece of software o/

comment:7 Changed 5 years ago by Michael

Cc: Michael added

comment:8 Changed 5 years ago by Sergey Kolosov

Cc: Sergey Kolosov added

comment:9 Changed 4 years ago by Anssi Kääriäinen

#18201 asked for tinyint and smallint based AutoFields. If bigint based fields is added in this ticket, it would make sense to add smallint based AutoField, too.

comment:10 Changed 4 years ago by Ethan Jucovy

Cc: ethan.jucovy@… added

comment:11 Changed 4 years ago by Jacob

#20001 has an alternate patch.

Changed 4 years ago by jamercee

Attachment: bigauto.diff added

BigAutoField patch file.

comment:12 Changed 4 years ago by jamercee

This week we upgraded to from 1.3 -> 1.5, and discovered BigAutoField was not yet included in Django. I know this issue has been referenced in a number of tickets over the years. In Django 1.3, we went with the mmcnickle approach discussed in this ticket, but found this difficult to merge into 1.5.

This time, we went with the approach approach suggested by pzinovkin in ticket 56 (https://code.djangoproject.com/attachment/ticket/56/56_bigint.diff). It required tweaking to get it to work cleanly with 1.5

Attached are patches derived from a git clone this morning.

comment:13 Changed 3 years ago by clintonb

What needs to be done to move this patch to completion and integration?

comment:14 Changed 3 years ago by anonymous

Someone from a big corporation that uses django to request it

comment:15 Changed 3 years ago by Aymeric Augustin

Judging by the ticket's flags, tests need to be written (at least).

Last edited 3 years ago by Aymeric Augustin (previous) (diff)

comment:16 Changed 3 years ago by Ludovico Magnocavallo

Cc: ludo@… added

comment:17 in reply to:  12 Changed 3 years ago by xsanch@…

Replying to jamercee:

This week we upgraded to from 1.3 -> 1.5, and discovered BigAutoField was not yet included in Django. I know this issue has been referenced in a number of tickets over the years. In Django 1.3, we went with the mmcnickle approach discussed in this ticket, but found this difficult to merge into 1.5.

This time, we went with the approach approach suggested by pzinovkin in ticket 56 (https://code.djangoproject.com/attachment/ticket/56/56_bigint.diff). It required tweaking to get it to work cleanly with 1.5

Attached are patches derived from a git clone this morning.

I tried to apply the patch to my 1.5 Django release:

patch -p1 --dry-run -r /var/tmp/reject.txt < bigauto.diff
patching file django/db/backends/init.py
Hunk #1 succeeded at 901 (offset -154 lines).
patching file django/db/backends/mysql/creation.py
Hunk #1 FAILED at 7.
1 out of 1 hunk FAILED -- saving rejects to file /var/tmp/reject.txt
patching file django/db/backends/oracle/creation.py
Hunk #1 FAILED at 17.
1 out of 1 hunk FAILED -- saving rejects to file /var/tmp/reject.txt
patching file django/db/backends/postgresql_psycopg2/creation.py
Hunk #1 FAILED at 11.
1 out of 1 hunk FAILED -- saving rejects to file /var/tmp/reject.txt
patching file django/db/backends/sqlite3/base.py
Hunk #1 succeeded at 214 (offset -34 lines).
patching file django/db/backends/sqlite3/creation.py
Hunk #1 FAILED at 9.
1 out of 1 hunk FAILED -- saving rejects to file /var/tmp/reject.txt
patching file django/db/models/fields/init.py
Hunk #1 succeeded at 213 (offset -2 lines).
Hunk #2 succeeded at 240 (offset -2 lines).
Hunk #3 succeeded at 531 (offset -8 lines).
Hunk #4 succeeded at 1158 (offset -8 lines).
Hunk #5 succeeded at 1174 (offset -8 lines).
patching file django/db/models/fields/related.py
Hunk #1 succeeded at 1092 (offset -21 lines).
patching file docs/ref/models/fields.txt
Hunk #1 FAILED at 331.
1 out of 1 hunk FAILED -- saving rejects to file /var/tmp/reject.txt

The version of Django is 1.5.0. Could someone tell me what am I doing wrong ?

Thanks,

Jorge

comment:18 Changed 2 years ago by Mike

Has there been any progress on this ticket? Is there a chance it will be included (and back-ported to 1.6)?

comment:19 Changed 2 years ago by Mike

Cc: stodge@… added

comment:20 Changed 2 years ago by Tim Graham

I don't see any progress. It would be included in Django 1.8 at the earliest. You can read about our support versions policy.

comment:21 Changed 22 months ago by Tadas Dailyda

I have patches for 1.6 and 1.7 ATM in my django fork:
Commit (patch) for 1.6: https://github.com/skirsdeda/django/commit/089fc7f165fb6cb7eb851b81a44660a986797946
Commit (patch) for 1.7: https://github.com/skirsdeda/django/commit/b0a567cc3fa7083ddcdca4f291b1e6d7fa85f6c7

It seems db backend implementation is changed a bit in master branch, so more work is needed. But I'm planning to do this sometime soonish.

comment:22 Changed 21 months ago by David Cramer

What needs done to make this happen?

It's pretty painful to modify these kinds of things currently and Sentry hit BIGINT quite a while back.

I'm attempting to hack this up but these are pretty deep internals and while I haven't given up I've yet to make things work.

Specifically the primary issue right now that we've hit with just doing it externally is with foreign keys:

https://github.com/getsentry/sentry/issues/1354#issuecomment-68393116

comment:23 Changed 21 months ago by David Cramer

In fact, I'd propose a new API be added to keys: "get_related_db_type" (or something along these lines)

I'm implement a custom ForeignKey locally that will do this, and allow BigAutoField to return BigIntegerField().db_type(connection)

comment:25 in reply to:  23 Changed 20 months ago by hhowe29

Replying to dcramer:

In fact, I'd propose a new API be added to keys: "get_related_db_type" (or something along these lines)

Yes please. Currently, ForeignKey.db_type is too tightly coupled to AutoField and IntegerField. There should be a new method that ForeignKey can call to ask the related field what SQL should be used to create a foreign key to that type. BitAutoField would return 'bigint'. AutoField would return a regular int. Most fields would simply defer to what db_type already returns.

Someone has already submitted this: #13774

comment:26 Changed 20 months ago by Collin Anderson

Cc: cmawebsite@… added

comment:27 Changed 20 months ago by drakkan

Cc: drakkan added

comment:29 Changed 14 months ago by samir

Cc: samir added

comment:30 Changed 14 months ago by Mike Fogel

Cc: mike@… added

comment:31 Changed 13 months ago by Marti

Cc: marti@… added

comment:32 Changed 13 months ago by Alexander

Needs tests: unset

comment:33 Changed 12 months ago by Alexander

Triage Stage: AcceptedUnreviewed

comment:34 Changed 12 months ago by Alexander

Triage Stage: UnreviewedReady for checkin

comment:35 Changed 12 months ago by Claude Paroz

Triage Stage: Ready for checkinAccepted

Sorry, but you are not supposed to mark your own patches as RFC.

comment:36 Changed 12 months ago by Tim Graham

Patch needs improvement: set

Added some comments for improvement on the pull request.

comment:37 Changed 12 months ago by Alexander

Patch needs improvement: unset

comment:38 Changed 11 months ago by Tim Graham

Patch needs improvement: set

I left another round of mostly cosmetic comments, looks like it's almost there.

comment:39 Changed 10 months ago by Alexander

Patch needs improvement: unset

Rebased on #13774.

Last edited 10 months ago by Alexander (previous) (diff)

comment:40 Changed 9 months ago by Tim Graham <timograham@…>

Resolution: fixed
Status: newclosed

In 2a7ce346:

Fixed #14286 -- Added models.BigAutoField.

Note: See TracTickets for help on using tickets.
Back to Top