Code

Opened 6 years ago

Closed 6 years ago

Last modified 3 years ago

#8669 closed (fixed)

All remaining create() methods also force an insert (continuation of r8670)

Reported by: Richard Davies <richard.davies@…> Owned by: nobody
Component: Database layer (models, ORM) Version: master
Severity: Keywords:
Cc: richard.davies@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

#8419 / [8670] forced an SQL insert in the main create() and get_or_create() methods, preventing data loss from overwriting existing data - thanks Malcolm!

All other get_or_create() methods end up calling back to the main get_or_create().

However, there are a few extra create() methods which don't call back to the main create() - this patch ensures that those also force an insert, so cannot overwrite existing data.

Extra methods found via "find django -name '*.py' | xargs grep -F 'def create('" and "find django -name '*.py' | xargs grep -F 'def get_or_create('"

Attachments (6)

create-always-inserts.diff (1.9 KB) - added by Richard Davies <richard.davies@…> 6 years ago.
Fixes for 3 different create() methods
create-always-inserts.v2.diff (1.9 KB) - added by Richard Davies <richard.davies@…> 6 years ago.
create-always-inserts.v3.diff (1.9 KB) - added by Richard Davies <richard.davies@…> 6 years ago.
Fix test suite failures (was a silly one line error in last patch)
tests.diff (3.2 KB) - added by Richard Davies <richard.davies@…> 6 years ago.
Extra tests (actually test changeset 8670, but that's the base of this ticket too)
authors.diff (397 bytes) - added by Richard Davies <richard.davies@…> 6 years ago.
Have I done enough across various tickets to be in the authors list?
8669_tests.v2.diff (3.4 KB) - added by Richard Davies <richard.davies@…> 6 years ago.
Have learnt that I got it wrong in #8739. Here's a revised set of tests incorporating that learning. There's also good extra tests in the v2.diffs inside tickets #8739 and #8740, which could also go in already

Download all attachments as: .zip

Change History (16)

Changed 6 years ago by Richard Davies <richard.davies@…>

Fixes for 3 different create() methods

comment:1 Changed 6 years ago by mtredinnick

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Design decision needed

It's not clear that add() should do this. At some point, if you're playing with custom set primary keys, you have to act responsibly and by calling that method you are saying that you want to 'add' that object, which is what happens.

The generic relations one is probably in line with the other changes, but I'm going to think about the first two thirds of the patch a bit.

Changed 6 years ago by Richard Davies <richard.davies@…>

comment:2 Changed 6 years ago by Richard Davies <richard.davies@…>

Here's a different (better?) approach - which avoids changing add() and points everyone back to the main create() implementation, rather than maintaining multiple copies.

comment:3 Changed 6 years ago by mtredinnick

  • Triage Stage changed from Design decision needed to Ready for checkin

The second patch is the way to go. It's a little annoying that overriding add() many-to-many now won't get called in the create case, but that's an acceptable trade-off.

I haven't checked that the full test suite passes with this patch yet, but, on the surface, it looks good to go.

comment:4 Changed 6 years ago by mtredinnick

  • Patch needs improvement set
  • Triage Stage changed from Ready for checkin to Accepted

This patch (the second one, which is the one we're considering) causes a lot of failures in the test suite, even with SQLite. They need to be fixed. I'd also like some confirmation that the revised version runs against the full test suite on a database that supports transacations (one of the PostgreSQL backends or Oracle).

Finally, given the scope of the change and the fact that it's already caused some problems, some tests to verify the new functionality would be a good idea.

Changed 6 years ago by Richard Davies <richard.davies@…>

Fix test suite failures (was a silly one line error in last patch)

Changed 6 years ago by Richard Davies <richard.davies@…>

Extra tests (actually test changeset 8670, but that's the base of this ticket too)

comment:5 Changed 6 years ago by Richard Davies <richard.davies@…>

Note: tests.diff includes creating an empty file "tests/modeltests/create/_init_.py", which doesn't show up in the web view of the diff

Changed 6 years ago by Richard Davies <richard.davies@…>

Have I done enough across various tickets to be in the authors list?

comment:6 Changed 6 years ago by Richard Davies <richard.davies@…>

The full test suite succeeds on SQLite with all of the above 3 patches (create-always-inserts.v3.diff, tests.diff, authors.diff).

In my (somewhat unusual) PostgreSQL set up, the test suite fails in several ways on unmodified trunk, even without these patches. I'll continue working on this tomorrow, both to post bugs to make it work initially, and also then to verify my actual patches.

comment:7 Changed 6 years ago by Richard Davies <richard.davies@…>

#8739 is a separate bug report against save(force_insert=True) failing on PostgreSQL. Since create() is built on that function, save(force_insert=True) will have to be fixed to get create() working on postgreSQL - all the work in this ticket makes that situation no better and no worse.

Changed 6 years ago by Richard Davies <richard.davies@…>

Have learnt that I got it wrong in #8739. Here's a revised set of tests incorporating that learning. There's also good extra tests in the v2.diffs inside tickets #8739 and #8740, which could also go in already

comment:8 Changed 6 years ago by Richard Davies <richard.davies@…>

  • Patch needs improvement unset

OK, I've validated this against the full test suite on both sqlite3 and postgresql_psycopg2. It is ready to check in.

The patches:

  • create-always-inserts.v3.diff is the actual fix to the ticket
  • authors.diff adds me to the authors list
  • I have 3 sets of tests which could reasonably added to the test suite: 8669_tests.v2.diff (above), 8739_save_force_insert.v2.diff (on ticket #8739), 8740_save_force_update.v2.diff (on ticket #8740). In all cases, be careful to create _init_.py files, which the patches don't seem to do for me.

comment:9 Changed 6 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from new to closed

(In [8884]) Fixed #8669 -- Use a consistent version of create() across the board for
model/field instance creation. Based on a patch from Richard Davies.

comment:10 Changed 3 years ago by jacob

  • milestone 1.0 deleted

Milestone 1.0 deleted

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.