Code

Opened 3 years ago

Closed 11 months ago

#14894 closed Bug (fixed)

translation is not threadsafe

Reported by: maxbublis Owned by: sergeykolosov
Component: Internationalization Version: 1.4
Severity: Normal Keywords: threadsafety translation
Cc: botondus@…, chris@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

I've got the following error in my multithreading application:

Traceback (most recent call last):
 File "/usr/local/lib/python2.6/site-packages/django/core/handlers/base.py", line 80, in get_response
   response = middleware_method(request)
 File "/usr/local/lib/python2.6/site-packages/django/middleware/locale.py", line 17, in process_request
   translation.activate(language)
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/__init__.py", line 66, in activate
   return real_activate(language)
 File "/usr/local/lib/python2.6/site-packages/django/utils/functional.py", line 55, in _curried
   return _curried_func(*(args+moreargs), **dict(kwargs, **morekwargs))
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/__init__.py", line 36, in delayed_loader
   return getattr(trans, real_name)(*args, **kwargs)
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 193, in activate
   _active[currentThread()] = translation(language)
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 177, in translation
   current_translation = _fetch(language, fallback=default_translation)
 File "/usr/local/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 141, in _fetch
   if base_lang(lang) in [base_lang(trans) for trans in _translations]:

RuntimeError: dictionary changed size during iteration

The reason is not threadsafe global _translations dictionary in django.utils.translation.trans_real module. In one thread you could iterate over it and change it in another one, that could lead to the above RuntimeError.

Patch attached.

Attachments (2)

trans_multithreading_fix.diff (1.4 KB) - added by maxbublis 3 years ago.
Patch that fixes multithreading issue in translation
bug-14894-translation-thread-safety.diff (2.1 KB) - added by sergeykolosov 11 months ago.
Patch which uses list() on _translations, so that it is Python 3.x complatible (where .keys() is an iterator); along with a regression test.

Download all attachments as: .zip

Change History (18)

Changed 3 years ago by maxbublis

Patch that fixes multithreading issue in translation

comment:1 Changed 3 years ago by jezdez

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

comment:2 Changed 3 years ago by maxbublis

What about progressing this ticket to "Ready For Checkin" state?

comment:3 Changed 3 years ago by jezdez

  • Needs tests set

comment:4 Changed 3 years ago by jezdez

  • Triage Stage changed from Accepted to Unreviewed

comment:5 Changed 3 years ago by maxbublis

Trunk codebase has been slightly changed during past 2 months.
Iteration over _translations dictionary now is located in 149 line.
Changing _translations dictionary now is located in 181 line

Last edited 3 years ago by maxbublis (previous) (diff)

comment:6 Changed 3 years ago by bberes

  • Cc botondus@… added

comment:7 Changed 3 years ago by julien

  • Severity set to Normal
  • Triage Stage changed from Unreviewed to Accepted
  • Type set to Bug

comment:8 Changed 3 years ago by jaddison

  • milestone 1.3 deleted

comment:9 Changed 2 years ago by aaugustin

  • UI/UX unset

Change UI/UX from NULL to False.

comment:10 Changed 2 years ago by aaugustin

  • Easy pickings unset

Change Easy pickings from NULL to False.

comment:11 Changed 18 months ago by acdha

  • Cc chris@… added

comment:12 Changed 18 months ago by acdha

  • Version changed from 1.2 to 1.4

comment:13 Changed 18 months ago by claudep

  • Patch needs improvement set

I think the easier fix would be to iterate on _translations.keys(). No risk of overloading memory with languages codes. Properly testing this might be hard.

comment:14 Changed 13 months ago by aaugustin

  • Component changed from Core (Other) to Internationalization

comment:15 Changed 11 months ago by sergeykolosov

  • Owner changed from nobody to sergeykolosov
  • Status changed from new to assigned

Changed 11 months ago by sergeykolosov

Patch which uses list() on _translations, so that it is Python 3.x complatible (where .keys() is an iterator); along with a regression test.

comment:16 Changed 11 months ago by Florian Apolloner <florian@…>

  • Resolution set to fixed
  • Status changed from assigned to closed

In acd0bb39df5c9ca486e49ec55ae34538242ce071:

Fixed #14894 -- Ensure that activating a translation doesn't run into threading issues.

Thanks to maxbublis for the report and sergeykolosov for the patch.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.