Opened 13 years ago

Last modified 12 years ago

#12007 new Cleanup/optimization

djangoadmin loaddata gives odd error if it runs out of memory

Reported by: chrisw Owned by: nobody
Component: Core (Serialization) Version: 1.1
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Alex Gaynor)

Hi All,

I tried to import a 3.2GB xml file, partly to see if Django loads the whole file into memory when it's importing and partly to see how large pg client transactions work. Whichever caused the problem, I have a fair idea that even thought the machine has 4GB of memory, the process ran out of memory.

I'd expect a MemoryError if that were the case, what I actually got was:

  File "django/core/management/", line 362, in execute_manager
  File "/django/core/management/", line 303, in execute
  File "django/core/management/", line 195, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "django/core/management/", line 222, in execute
    output = self.handle(*args, **options)
  File "django/core/management/commands/", line 200, in handle
  File "django/db/", line 74, in leave_transaction_management
    raise TransactionManagementError("This code isn't under transaction management")
django.db.transaction.TransactionManagementError: This code isn't under transaction management



Change History (5)

comment:1 Changed 13 years ago by Russell Keith-Magee

Component: UncategorizedSerialization
Triage Stage: UnreviewedAccepted

For those looking to dig into this, it's important to identify whether this is a database issue (i.e., the database runs out of memory if you try to do a 3GB transaction) or a Django issue (i.e., a problem with our wrapper loading the data file into memory).

Its also important to note the analog with #5423 - that ticket deals with making dumpdata behave as an iterator. I'm not marking this as a dupe because the solution won't necessarily be the same, but it's worth noting if someone wants to tackle both problems.

comment:2 Changed 13 years ago by Alex Gaynor

Description: modified (diff)

Please use preview

comment:3 Changed 12 years ago by Peter Baumgartner

Severity: Normal
Type: Cleanup/optimization

comment:4 Changed 11 years ago by Aymeric Augustin

UI/UX: unset

Change UI/UX from NULL to False.

comment:5 Changed 11 years ago by Aymeric Augustin

Easy pickings: unset

Change Easy pickings from NULL to False.

Note: See TracTickets for help on using tickets.
Back to Top