Add a Meta.sequence option to models

[Max Shabalihin]

PostgreSQL users need to be able to override the name of the serial sequence. Mostly to match existing databases, but also to solve issues like #15682, #13179 and #1946.

[Russell Keith-Magee]

It's an edge case, but one that's probably worth supporting.

Regarding the patch --

• it obviously needs tests.
• It seems to ignore the creation of sequence names in the first place
• It seems to be appending "_seq" to the name of the provided sequence name. Why? If I manually specify a sequence name, shouldn't it be used verbatim?
• _meta.db_seq should always return the correctly constructed sequence name; we shoulnd't need to use 'if pk_name' or similiar tricks to work out if the sequence has been manually specified.

[Anssi Kääriäinen]

At least tickets #8901, #13179 and #1946 deal with the same problem. There are actually a lot more issues than just fixing last_insert_id to use the custom db_seq. Postgresql truncates sequence names to 63 characters, and we need to deal with table creation, sequence resetting and sql_flush. We still need to think how to deal with problems presented by users trying to use the same sequence for multiple tables. And we need to take care of automatically generated m2m tables and their sequences.

First problem is ticket #8901. If it happens so that the table_name + column_name + _seq is longer than 63 characters, then the sequence name will be truncated to 63 characters in a complex way by postgresql. Complex here means that it is not easy to create the right sequence name for all supported versions of postgresql. Django currently doesn't work at all for tables where the table_nam + column_name is longer than 63 characters.

There is a function pg_get_serial_sequence(tablename, column) (see ticket #13179), which returns the sequence name for given column. Unfortunately this only works for tables having serial columns, that is

    id serial not null primary key

, not for tables having column of type

    id integer primary key default nextval('some_custom_sequence_name')

This means that using pg_get_serial_sequence will result in backwards incompatible behaviour for custom schemas where the default nextval way of specifying the sequence is used, and it can't be used with custom sequence names. Issuing postgresql command

    ALTER SEQUENCE tablename_colname_seq OWNED BY tablename.colname;

will fix the problem, but it is still a backwards incompatible change. It is impossible to have the sequence owned by multiple tables, so I don't think using pg_get_serial_sequence is a good solution.

The proposed solution:

Models get a new meta attribute, db_seq. In initialization we set up _meta.db_seq to always contain the right sequence name for the model, either the given db_seq or %s_%s_seq % (table_name, autofield_column_name). In model validation we check that the db_seq is shorter than 63 characters long, indifferent of if it is automatically generated or given as meta option. This should not be backwards incompatible, as currently if the sequence is longer than 63 characters django will not work.

In table generation, if db_seq is not automatically generated, we generate the pk column as "col_name integer not null primary key default next_val('db_seq')", else we will use the current behaviour of "col_name serial not null primary key". When using the next_val definition we need to generate the sequence before we generate the table. We need to also alter some other places (at least db/backends/postgresql/options.py functions sql_flush, last_insert_id, and sequence_reset_sql).

There is another problem: users will probably want to use the same sequence for multiple tables. In this case, we should create the sequence only once. But how to handle sequence_reset_sql? It should probably go through all the tables using the sequence and determine the maximum of all the ids in use. A simple solution for this would be to document that if you want to use the same sequence for multiple tables, Django currently can't manage the database schema for you. Note that the solution using pg_get_serial_sequence would break if the user tried to use the same sequence for multiple tables.

Still one more problem: what to do with autogenerated m2m tables. Here django currently uses table_name_id_seq as sequence name, but if I am not mistaken it is also possible for the sequence name to be longer than 63 characters here.

I think I can do the coding to get the proposed solution to work, but before starting the work it would be good to get a confirmation that the proposed solution has a chance to get accepted into 1.3.

[Jacob]

#15682 was a dup.

[Jacob]

As were #13179 and #1946.

[Jacob]

Updated title, description.