Opened 3 weeks ago

Last modified 8 days ago

#36732 assigned Cleanup/optimization

Sitemaps with i18n=True load the whole table in memory — at Initial Version

Reported by: Julien Palard Owned by:
Component: contrib.sitemaps Version: 5.2
Severity: Normal Keywords: sitemap, memory
Cc: Julien Palard, Roxane Triage Stage: Accepted
Has patch: yes Needs documentation: yes
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Hi!

It's a bit like: https://code.djangoproject.com/ticket/11572

when enabling i18n on a sitemap, _items uses the following code:

`python

# Create (item, lang_code) tuples for all items and languages.
# This is necessary to paginate with all languages already considered.
items = [

(item, lang_code)
for item in self.items()
for lang_code in self.get_languages_for_item(item)

]

`

The list comprehension loads the whole table times the number of languages in memory.

We tried with a table containing two millions elements and two languages: it does not fit in 16GB of memory and the process gets killed.

Change History (0)

Note: See TracTickets for help on using tickets.
Back to Top