Code

Opened 4 years ago

Closed 3 years ago

Last modified 3 years ago

#15068 closed (invalid)

initial_data fixtures using natural keys fail in tests with multiple databases

Reported by: dcramer Owned by: nobody
Component: Testing framework Version: 1.2
Severity: Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

If a fixture using natural keys is referenced before that database is created (in tests), then the fixture fails to load due to a "table does not exist" error (though may also throw database does not exist, since it may not have been created yet).

The reason for this is that objects are instantiated before they are actually created in the database, but they mapper only does this if they're not using natural keys. In the case where they are, it attempts to look them up from the database, rather than the existing in-memory instances.

Attachments (0)

Change History (4)

comment:1 Changed 4 years ago by dcramer

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

A resolution which may help with some other bad behavior would be to create all databases and then process fixtures. This would also stop the additional unnecessary parsing of the fixtures files repeatedly (N databases).

comment:2 Changed 3 years ago by russellm

  • Component changed from Uncategorized to Testing framework
  • Resolution set to invalid
  • Status changed from new to closed
  • Triage Stage changed from Unreviewed to Accepted

I'm having difficulty in seeing how this situation would arise. Fixtures can't cross databases, and they are saved raw on the database they're being loaded onto. All test databases are created before the first test is created. I can't see how a test fixture -- which is loaded after all the databases have been configured -- could cause "table does not exist" or "database does not exist" errors.

Closing invalid; As with #15063, feel free to reopen if you can provide a comprehensive report that describes (or better yet, demonstrates) exactly how this would occur.

comment:3 Changed 3 years ago by dcramer

Natural keys require the data to exist in the database before they're referenced. With the way fixtures work, they're instantiated before running the "allow_syncdb" check. In our case, a database with the name "analytics" was loaded before "default", which contains very few tables. When it gets around to a fixture with a natural key, it attempts to instantiate it using the natural key code path, which attempts to validate a foreign key against a table that does not exist. I'm pretty certain this is valid, and while it may be an edge case (not sure how common natural key usage is), it's certainly very odd behavior.

I attempted to shift loading fixtures after syncdb was done (on every database), but that still doesnt solve it, as the fixtures load per database (not globally), and in this case, they still have the same issue. I think the proper fix would be to change natural keys so they're mapped in memory, but that requires quite a large change to Django itself I imagine, as last I knew relationships dont support associations if they dont have an identifier, which these wouldn't.

comment:4 Changed 3 years ago by russellm

Following discussion with David on IRC: This appears to be a case of #13252. His fixture had objects that didn't provide PKs, and he was assuming that this would cause "create if doesn't exist" type behavior.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.