Opened 6 years ago

Closed 3 years ago

#14894 closed Bug (fixed)

translation is not threadsafe

Reported by: Maxim Bublis Owned by: Sergey Kolosov
Component: Internationalization Version: 1.4
Severity: Normal Keywords: threadsafety translation
Cc: botondus@…, chris@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

I've got the following error in my multithreading application:

Traceback (most recent call last):
 File "/usr/local/lib/python2.6/site-packages/django/core/handlers/base.py", line 80, in get_response
   response = middleware_method(request)
 File "/usr/local/lib/python2.6/site-packages/django/middleware/locale.py", line 17, in process_request
   translation.activate(language)
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/__init__.py", line 66, in activate
   return real_activate(language)
 File "/usr/local/lib/python2.6/site-packages/django/utils/functional.py", line 55, in _curried
   return _curried_func(*(args+moreargs), **dict(kwargs, **morekwargs))
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/__init__.py", line 36, in delayed_loader
   return getattr(trans, real_name)(*args, **kwargs)
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 193, in activate
   _active[currentThread()] = translation(language)
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 177, in translation
   current_translation = _fetch(language, fallback=default_translation)
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 141, in _fetch
   if base_lang(lang) in [base_lang(trans) for trans in _translations]:

RuntimeError: dictionary changed size during iteration

The reason is not threadsafe global _translations dictionary in django.utils.translation.trans_real module. In one thread you could iterate over it and change it in another one, that could lead to the above RuntimeError.

Patch attached.

Attachments (2)

trans_multithreading_fix.diff (1.4 KB) - added by Maxim Bublis 6 years ago.
Patch that fixes multithreading issue in translation
bug-14894-translation-thread-safety.diff (2.1 KB) - added by Sergey Kolosov 3 years ago.
Patch which uses list() on _translations, so that it is Python 3.x complatible (where .keys() is an iterator); along with a regression test.

Download all attachments as: .zip

Change History (18)

Changed 6 years ago by Maxim Bublis

Patch that fixes multithreading issue in translation

comment:1 Changed 6 years ago by Jannis Leidel

Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Triage Stage: UnreviewedAccepted

comment:2 Changed 6 years ago by Maxim Bublis

What about progressing this ticket to "Ready For Checkin" state?

comment:3 Changed 6 years ago by Jannis Leidel

Needs tests: set

comment:4 Changed 6 years ago by Jannis Leidel

Triage Stage: AcceptedUnreviewed

comment:5 Changed 6 years ago by Maxim Bublis

Trunk codebase has been slightly changed during past 2 months.
Iteration over _translations dictionary now is located in 149 line
Changing _translations dictionary now is located in 181 line

Version 0, edited 6 years ago by Maxim Bublis (next)

comment:6 Changed 6 years ago by Béres Botond

Cc: botondus@… added

comment:7 Changed 5 years ago by Julien Phalip

Severity: Normal
Triage Stage: UnreviewedAccepted
Type: Bug

comment:8 Changed 5 years ago by James Addison

milestone: 1.3

comment:9 Changed 5 years ago by Aymeric Augustin

UI/UX: unset

Change UI/UX from NULL to False.

comment:10 Changed 5 years ago by Aymeric Augustin

Easy pickings: unset

Change Easy pickings from NULL to False.

comment:11 Changed 4 years ago by Chris Adams

Cc: chris@… added

comment:12 Changed 4 years ago by Chris Adams

Version: 1.21.4

comment:13 Changed 4 years ago by Claude Paroz

Patch needs improvement: set

I think the easier fix would be to iterate on _translations.keys(). No risk of overloading memory with languages codes. Properly testing this might be hard.

comment:14 Changed 4 years ago by Aymeric Augustin

Component: Core (Other)Internationalization

comment:15 Changed 3 years ago by Sergey Kolosov

Owner: changed from nobody to Sergey Kolosov
Status: newassigned

Changed 3 years ago by Sergey Kolosov

Patch which uses list() on _translations, so that it is Python 3.x complatible (where .keys() is an iterator); along with a regression test.

comment:16 Changed 3 years ago by Florian Apolloner <florian@…>

Resolution: fixed
Status: assignedclosed

In acd0bb39df5c9ca486e49ec55ae34538242ce071:

Fixed #14894 -- Ensure that activating a translation doesn't run into threading issues.

Thanks to maxbublis for the report and sergeykolosov for the patch.

Note: See TracTickets for help on using tickets.
Back to Top