Opened 7 years ago

Last modified 6 weeks ago

#13295 new New feature

Add a Meta.sequence option to models

Reported by: Max Shabalihin Owned by: nobody
Component: Database layer (models, ORM) Version: 1.1
Severity: Normal Keywords: sequence last_id postgresql
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: yes
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description (last modified by Jacob)

PostgreSQL users need to be able to override the name of the serial sequence. Mostly to match existing databases, but also to solve issues like #15682, #13179 and #1946.

Attachments (2)

postgresql_name_seq.diff (1.1 KB) - added by Max Shabalihin 7 years ago.
postgresql_name_seq.2.diff (2.5 KB) - added by Max Shabalihin 7 years ago.

Download all attachments as: .zip

Change History (11)

Changed 7 years ago by Max Shabalihin

Attachment: postgresql_name_seq.diff added

Changed 7 years ago by Max Shabalihin

Attachment: postgresql_name_seq.2.diff added

comment:1 Changed 7 years ago by Ramiro Morales

Keywords: sequence added; sequnce removed
milestone: 1.2
Needs documentation: set
Needs tests: set

comment:2 Changed 7 years ago by Russell Keith-Magee

Patch needs improvement: set
Triage Stage: UnreviewedAccepted

It's an edge case, but one that's probably worth supporting.

Regarding the patch --

  • it obviously needs tests.
  • It seems to ignore the creation of sequence names in the first place
  • It seems to be appending "_seq" to the name of the provided sequence name. Why? If I manually specify a sequence name, shouldn't it be used verbatim?
  • _meta.db_seq should always return the correctly constructed sequence name; we shoulnd't need to use 'if pk_name' or similiar tricks to work out if the sequence has been manually specified.

comment:3 Changed 7 years ago by Anssi Kääriäinen

At least tickets #8901, #13179 and #1946 deal with the same problem. There are actually a lot more issues than just fixing last_insert_id to use the custom db_seq. Postgresql truncates sequence names to 63 characters, and we need to deal with table creation, sequence resetting and sql_flush. We still need to think how to deal with problems presented by users trying to use the same sequence for multiple tables. And we need to take care of automatically generated m2m tables and their sequences.

First problem is ticket #8901. If it happens so that the table_name + column_name + _seq is longer than 63 characters, then the sequence name will be truncated to 63 characters in a complex way by postgresql. Complex here means that it is not easy to create the right sequence name for all supported versions of postgresql. Django currently doesn't work at all for tables where the table_nam + column_name is longer than 63 characters.

There is a function pg_get_serial_sequence(tablename, column) (see ticket #13179), which returns the sequence name for given column. Unfortunately this only works for tables having serial columns, that is

    id serial not null primary key

, not for tables having column of type

    id integer primary key default nextval('some_custom_sequence_name')

This means that using pg_get_serial_sequence will result in backwards incompatible behaviour for custom schemas where the default nextval way of specifying the sequence is used, and it can't be used with custom sequence names. Issuing postgresql command

    ALTER SEQUENCE tablename_colname_seq OWNED BY tablename.colname;

will fix the problem, but it is still a backwards incompatible change. It is impossible to have the sequence owned by multiple tables, so I don't think using pg_get_serial_sequence is a good solution.

The proposed solution:

Models get a new meta attribute, db_seq. In initialization we set up _meta.db_seq to always contain the right sequence name for the model, either the given db_seq or %s_%s_seq % (table_name, autofield_column_name). In model validation we check that the db_seq is shorter than 63 characters long, indifferent of if it is automatically generated or given as meta option. This should not be backwards incompatible, as currently if the sequence is longer than 63 characters django will not work.

In table generation, if db_seq is not automatically generated, we generate the pk column as "col_name integer not null primary key default next_val('db_seq')", else we will use the current behaviour of "col_name serial not null primary key". When using the next_val definition we need to generate the sequence before we generate the table. We need to also alter some other places (at least db/backends/postgresql/ functions sql_flush, last_insert_id, and sequence_reset_sql).

There is another problem: users will probably want to use the same sequence for multiple tables. In this case, we should create the sequence only once. But how to handle sequence_reset_sql? It should probably go through all the tables using the sequence and determine the maximum of all the ids in use. A simple solution for this would be to document that if you want to use the same sequence for multiple tables, Django currently can't manage the database schema for you. Note that the solution using pg_get_serial_sequence would break if the user tried to use the same sequence for multiple tables.

Still one more problem: what to do with autogenerated m2m tables. Here django currently uses table_name_id_seq as sequence name, but if I am not mistaken it is also possible for the sequence name to be longer than 63 characters here.

I think I can do the coding to get the proposed solution to work, but before starting the work it would be good to get a confirmation that the proposed solution has a chance to get accepted into 1.3.

comment:4 Changed 6 years ago by Julien Phalip

Severity: Normal
Type: New feature

comment:5 Changed 6 years ago by Jacob

#15682 was a dup.

comment:6 Changed 6 years ago by Jacob

As were #13179 and #1946.

comment:7 Changed 6 years ago by Jacob

Description: modified (diff)
Summary: alternate name sequence for postgresql in class meta db_seqAdd a Meta.sequence option to models

Updated title, description.

comment:8 Changed 6 years ago by Aymeric Augustin

Easy pickings: unset
UI/UX: unset

See also #16448.

comment:9 Changed 6 weeks ago by Jadson Ribeiro

I wonder if there is already a solution after so many years for this problem?

Note: See TracTickets for help on using tickets.
Back to Top