Opened 7 years ago

Last modified 5 years ago

#12007 new Cleanup/optimization

djangoadmin loaddata gives odd error if it runs out of memory

Reported by: chrisw Owned by: nobody
Component: Core (Serialization) Version: 1.1
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Alex)

Hi All,

I tried to import a 3.2GB xml file, partly to see if Django loads the whole file into memory when it's importing and partly to see how large pg client transactions work. Whichever caused the problem, I have a fair idea that even thought the machine has 4GB of memory, the process ran out of memory.

I'd expect a MemoryError if that were the case, what I actually got was:

  File "django/core/management/", line 362, in execute_manager
  File "/django/core/management/", line 303, in execute
  File "django/core/management/", line 195, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "django/core/management/", line 222, in execute
    output = self.handle(*args, **options)
  File "django/core/management/commands/", line 200, in handle
  File "django/db/", line 74, in leave_transaction_management
    raise TransactionManagementError("This code isn't under transaction management")
django.db.transaction.TransactionManagementError: This code isn't under transaction management



Change History (5)

comment:1 Changed 7 years ago by russellm

  • Component changed from Uncategorized to Serialization
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

For those looking to dig into this, it's important to identify whether this is a database issue (i.e., the database runs out of memory if you try to do a 3GB transaction) or a Django issue (i.e., a problem with our wrapper loading the data file into memory).

Its also important to note the analog with #5423 - that ticket deals with making dumpdata behave as an iterator. I'm not marking this as a dupe because the solution won't necessarily be the same, but it's worth noting if someone wants to tackle both problems.

comment:2 Changed 7 years ago by Alex

  • Description modified (diff)

Please use preview

comment:3 Changed 5 years ago by baumer1122

  • Severity set to Normal
  • Type set to Cleanup/optimization

comment:4 Changed 5 years ago by aaugustin

  • UI/UX unset

Change UI/UX from NULL to False.

comment:5 Changed 5 years ago by aaugustin

  • Easy pickings unset

Change Easy pickings from NULL to False.

Note: See TracTickets for help on using tickets.
Back to Top