Transaction error when doing loaddata

[Erik Allik <eallik@…>]

I'm using MySQL 4.1 with InnoDB tables and when doing ./manage.py loaddata data.json, I get the following traceback:

$ ./manage.py loaddata data.json 
Installing json fixture '/home/rime/www/rabakyla/packages/rime/data' from '/home/rime/www/rabakyla/packages/rime/data'.
Traceback (most recent call last):
  File "./manage.py", line 11, in ?
    execute_manager(settings)
  File "/home/rime/www/rabakyla/packages/django/core/management/__init__.py", line 301, in execute_manager
    utility.execute()
  File "/home/rime/www/rabakyla/packages/django/core/management/__init__.py", line 248, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/rime/www/rabakyla/packages/django/core/management/base.py", line 77, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/rime/www/rabakyla/packages/django/core/management/base.py", line 91, in execute
    output = self.handle(*args, **options)
  File "/home/rime/www/rabakyla/packages/django/core/management/commands/loaddata.py", line 157, in handle
    transaction.leave_transaction_management()
  File "/home/rime/www/rabakyla/packages/django/db/transaction.py", line 71, in leave_transaction_management
    raise TransactionManagementError("This code isn't under transaction management")
django.db.transaction.TransactionManagementError: This code isn't under transaction management

I initially used MyISAM tables (which are the default), then recreated the database with InnoDB tables, but it didn't help. I also discovered that the database was using latin1 instead of UTF-8 so I recreated the tables using UTF-8 as the default encoding which made the problem go away. But what disturbs me is that Django is not giving a good error message for this. I mean, latin1 and UTF8 are not even related to transactions, and even if they are due to some implementation details, it's really hard to make the guess. So I think loaddata should give a better error message!

(Running the latest revision of Django trunk)

[Erik Allik]

Has anyone else been able to reproduce/verify this? Just to be sure it wasn't something caused by a quirk in my setup.

[Russell Keith-Magee]

Can you provide some more explicit instructions on how to replicate this problem? Exact model definitions, version numbers (of Django, especially), and the fixture that causes the problem would be helpful. As it stands, I don't have enough detail to replicate the problem.

[Russell Keith-Magee]

In the absence of sufficient data to reproduce this problem, I'm closing this "worksforme". If you can provide detailed instruction on how to reproduce this problem, feel free to reopen.

[Erik Allik]

I'm getting the exact same traceback now. This time all tables are using UTF8 and InnoDB. Doing loaddata fixture.json

Traceback is at:http://dpaste.com/hold/77207/
MySQL version:5.0.51b
Django revision: 8362
Python 2.3.4

[Erik Allik]

Should I upload the JSON fixture somewhere, too? Or what else is needed to reproduce the problem? In a #django log, I saw someone else having the same problem with MySQL 4.4 and referring to this ticket, so I'm not the only one having this issue.

[Russell Keith-Magee]

What we need is enough detail so that we can reproduce the problem. This means a set of models and a JSON file that causes the problem when loaded.

[tom_payne]

Just as an aside, I found this ticket because I was experiencing the same problem and it turned out that my DB was out of sync with my models. (I found this by running syncdb, even though I wasn't adding new tables, it still raised an error.)

Fixing the DB fixed the problem.

Hope this helps someone!

Tom

[Graham King]

Changed the Milestone.

[tom_payne]

Well, I've had this problem again and have traced the error...

I was trying to insert null values into my booleanfield (which I shouldn't be doing - that's what NullBooleanField is for - see #5563), but I think it's more a question of raising any error like this.

Essentially, the bug is that you don't get a nice error message telling you what's wrong, rather you get an irrelevant Transaction error because the proper error message raises an exception.

The problem comes when this error is raised in loaddata.py (django/core/management/commands/loaddata.py). The except clause (line 124 onwards) tries to print an error message using traceback.format_exc, but this needs to be traceback.format_exception(sys.exc_type, sys.exc_value, sys.exc_traceback). The _exc shortcut, IIRC, is only available to print_exc.

I hope that's clear.

Tom

[tom_payne]

Oh, sorry, no version number. This is the case for 1.0.2 final

[Ramiro Morales]

Patch for this fixing usage of print-exc (unavailable on Python 2.3) and a superflous import

[Ramiro Morales]

tom_payne didn't report what version of Python he was using, but from what he describes he could be using Python 2.3. Uploaded a patch based in his findings.

[Karen Tracey]

(In [9770]) Fixed #8055: Replaced use of traceback.format_exc (new in Python 2.4) with something that works as far back as 2.3.

Thanks to Ramiro for diagnosis and patch.

[Karen Tracey]

(In [9772]) [1.0.X] Fixed #8055: Replaced use of traceback.format_exc (new in Python 2.4) with something that works as far back as 2.3.

Thanks to Ramiro for diagnosis and patch.

Backport of r9770 from trunk.

Also updated svn metadata; all trunk bugfixes have been merged to the 1.0.X branch.

[(none)]

Milestone post-1.0 deleted