Opened 13 years ago

Closed 11 years ago

#14894 closed Bug (fixed)

translation is not threadsafe

Reported by: Maxim Bublis Owned by: Sergey Kolosov
Component: Internationalization Version: 1.4
Severity: Normal Keywords: threadsafety translation
Cc: botondus@…, chris@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

I've got the following error in my multithreading application:

Traceback (most recent call last):
 File "/usr/local/lib/python2.6/site-packages/django/core/handlers/base.py", line 80, in get_response
   response = middleware_method(request)
 File "/usr/local/lib/python2.6/site-packages/django/middleware/locale.py", line 17, in process_request
   translation.activate(language)
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/__init__.py", line 66, in activate
   return real_activate(language)
 File "/usr/local/lib/python2.6/site-packages/django/utils/functional.py", line 55, in _curried
   return _curried_func(*(args+moreargs), **dict(kwargs, **morekwargs))
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/__init__.py", line 36, in delayed_loader
   return getattr(trans, real_name)(*args, **kwargs)
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 193, in activate
   _active[currentThread()] = translation(language)
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 177, in translation
   current_translation = _fetch(language, fallback=default_translation)
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 141, in _fetch
   if base_lang(lang) in [base_lang(trans) for trans in _translations]:

RuntimeError: dictionary changed size during iteration

The reason is not threadsafe global _translations dictionary in django.utils.translation.trans_real module. In one thread you could iterate ​over it and change it ​in another one, that could lead to the above RuntimeError.

Patch attached.

Attachments (2)

trans_multithreading_fix.diff​ (1.4 KB ) - added by Maxim Bublis 13 years ago.
Patch that fixes multithreading issue in translation
bug-14894-translation-thread-safety.diff​ (2.1 KB ) - added by Sergey Kolosov 11 years ago.
Patch which uses list() on _translations, so that it is Python 3.x complatible (where .keys() is an iterator); along with a regression test.

Download all attachments as: .zip

Change History (18)

by Maxim Bublis, 13 years ago

Attachment: trans_multithreading_fix.diff​ added

Patch that fixes multithreading issue in translation

comment:1 by Jannis Leidel, 13 years ago

Triage Stage: Unreviewed → Accepted

comment:2 by Maxim Bublis, 13 years ago

What about progressing this ticket to "Ready For Checkin" state?

comment:3 by Jannis Leidel, 13 years ago

Needs tests: set

comment:4 by Jannis Leidel, 13 years ago

Triage Stage: Accepted → Unreviewed

comment:5 by Maxim Bublis, 13 years ago

Trunk codebase has been slightly changed during past 2 months.
Iteration over _translations dictionary now is located in ​149 line
Changing _translations dictionary now is located in ​181 line

Version 0, edited 13 years ago by Maxim Bublis (next)

comment:6 by Béres Botond, 13 years ago

Cc: botondus@… added

comment:7 by Julien Phalip, 13 years ago

Severity: → Normal
Triage Stage: Unreviewed → Accepted
Type: → Bug

comment:8 by James Addison, 13 years ago

milestone: 1.3

comment:9 by Aymeric Augustin, 12 years ago

UI/UX: unset

Change UI/UX from NULL to False.

comment:10 by Aymeric Augustin, 12 years ago

Easy pickings: unset

Change Easy pickings from NULL to False.

comment:11 by Chris Adams, 12 years ago

Cc: chris@… added

comment:12 by Chris Adams, 12 years ago

Version: 1.2 → 1.4

comment:13 by Claude Paroz, 12 years ago

Patch needs improvement: set

I think the easier fix would be to iterate on _translations.keys(). No risk of overloading memory with languages codes. Properly testing this might be hard.

comment:14 by Aymeric Augustin, 11 years ago

Component: Core (Other) → Internationalization

comment:15 by Sergey Kolosov, 11 years ago

Owner: changed from nobody to Sergey Kolosov
Status: new → assigned

by Sergey Kolosov, 11 years ago

Attachment: bug-14894-translation-thread-safety.diff​ added

Patch which uses list() on _translations, so that it is Python 3.x complatible (where .keys() is an iterator); along with a regression test.

comment:16 by Florian Apolloner <florian@…>, 11 years ago

Resolution: → fixed
Status: assigned → closed

In acd0bb39df5c9ca486e49ec55ae34538242ce071:

Fixed #14894 -- Ensure that activating a translation doesn't run into threading issues.

Thanks to maxbublis for the report and sergeykolosov for the patch.

Note: See TracTickets for help on using tickets.
Back to Top