wiki:DjangoSpecifications/Core/Threading

Context Navigation

Version 13 (modified by mrts, 17 years ago) ( diff )
--

Part of DjangoSpecifications

Threading improvements in Django

According to tickets #5632, #6950 and discussions http://groups.google.com/group/django-users/browse_frm/thread/a7d42475b66530bd, http://groups.google.com/group/django-developers/browse_thread/thread/fbcfa88c997d1bb3, at least the following components of Django are not entirely thread-safe:

django.template.loader
django.db.models
django.contrib.sessions

This specification outlines the changes that need to be implemented to solve these issues.

A related task is to identify other components not mentioned here that have threading issues.

See #1442 and http://groups.google.com/group/django-developers/browse_thread/thread/905f79e350525c95 for threading discussions.

Globals

There are three types of globals:

read-only globals that are never assigned to (THREAD-SAFE),
globals assigned to in a function by using the global keyword (NOT THREAD-SAFE),
global mutable data structures (lists and dictionaries, also instances) that are assigned to at module level but whose elements are modified in functions and that are accessed without using the global keyword (NOT THREAD-SAFE unless never modified).

The following modules' use of globals needs review.

See below for raw grep results.

THE FOLLOWING IS WORK IN PROGRESS.

Settings

FIXME: not reviewed. Replacing the global_settings with settings -- probably no major issues.

django/contrib/sites/models.py

Globals used:

SITE_CACHE

django/template

Globals used:

context.py: _standard_context_processors
__init__.py: invalid_var_format_string, libraries
loader.py: template_source_loaders

django/utils/translation

Globals used:

trans_real.py: _accepted, _active, _default, _translations

django/contrib/sites/models.py

Globals used:

SITE_CACHE

Raw `grep` results

Globals accessed with the `global` keyword

$ grep -r '^[[:space:]]*global ' django/ | egrep -v '(\.svn|\.html|\.css|\.pyc|_doctest\.py)' | sort | uniq

yields the following results

django/contrib/sites/models.py:        global SITE_CACHE
django/core/management/__init__.py:    global _commands
django/template/context.py:    global _standard_context_processors
django/template/__init__.py:                    global invalid_var_format_string
django/template/loader.py:    global template_source_loaders
django/utils/translation/trans_real.py:    global _accepted
django/utils/translation/trans_real.py:    global _active
django/utils/translation/trans_real.py:    global _default, _active
django/utils/translation/trans_real.py:        global _translations
django/utils/translation/trans_real.py:    global _translations

Out of these, django.core.management is not used in multi-threading context.

Global dictionaries

$ rgrep '^[[:alnum:]_]\+ *= *{' django | egrep -v '(\.svn|_doctest\.py)' | sort

yields the following results

django/conf/global_settings.py:ABSOLUTE_URL_OVERRIDES = {}
django/conf/global_settings.py:DATABASE_OPTIONS = {}          # Set to empty dictionary for default.
django/contrib/admin/utils.py:ROLES = {
django/contrib/admin/views/doc.py:DATA_TYPE_MAPPING = {
django/contrib/formtools/tests.py:test_data = {'field1': u'foo',
django/contrib/localflavor/ca/ca_provinces.py:PROVINCES_NORMALIZED = {
django/contrib/localflavor/in_/in_states.py:STATES_NORMALIZED = {
django/contrib/localflavor/us/us_states.py:STATES_NORMALIZED = {
django/contrib/sites/models.py:SITE_CACHE = {}
django/core/cache/__init__.py:BACKENDS = {
django/core/cache/__init__.py:DEPRECATED_BACKENDS = {
django/core/handlers/wsgi.py:STATUS_CODE_TEXT = {
django/core/serializers/__init__.py:BUILTIN_SERIALIZERS = {
django/core/serializers/__init__.py:_serializers = {}
django/core/servers/basehttp.py:_hop_headers = {
django/core/servers/fastcgi.py:FASTCGI_OPTIONS = {
django/core/urlresolvers.py:_callable_cache = {} # Maps view and url pattern names to their view functions.
django/core/urlresolvers.py:_resolver_cache = {} # Maps urlconf modules to RegexURLResolver instances.
django/db/backends/dummy/creation.py:DATA_TYPES = {}
django/db/backends/dummy/introspection.py:DATA_TYPES_REVERSE = {}
django/db/backends/mysql/creation.py:DATA_TYPES = {
django/db/backends/mysql/introspection.py:DATA_TYPES_REVERSE = {
django/db/backends/mysql_old/creation.py:DATA_TYPES = {
django/db/backends/mysql_old/introspection.py:DATA_TYPES_REVERSE = {
django/db/backends/oracle/creation.py:DATA_TYPES = {
django/db/backends/oracle/creation.py:REMEMBER = {}
django/db/backends/oracle/introspection.py:DATA_TYPES_REVERSE = {
django/db/backends/postgresql/creation.py:DATA_TYPES = {
django/db/backends/postgresql/introspection.py:DATA_TYPES_REVERSE = {
django/db/backends/postgresql_psycopg2/introspection.py:DATA_TYPES_REVERSE = {
django/db/backends/sqlite3/creation.py:DATA_TYPES = {
django/db/backends/sqlite3/introspection.py:BASE_DATA_TYPES_REVERSE = {
django/db/models/fields/related.py:pending_lookups = {}
django/db/models/query.py:LEGACY_ORDERING_MAPPING = {'ASC': '_', 'DESC': '-_', 'RANDOM': '?'}
django/db/transaction.py:dirty = {}
django/db/transaction.py:state = {}
django/dispatch/dispatcher.py:connections = {}
django/dispatch/dispatcher.py:senders = {}
django/dispatch/dispatcher.py:sendersBack = {}
django/template/__init__.py:libraries = {}
django/utils/dates.py:MONTHS = {
django/utils/dates.py:MONTHS_3 = {
django/utils/dates.py:MONTHS_3_REV = {
django/utils/dates.py:MONTHS_AP = { # month names in Associated Press style
django/utils/dates.py:WEEKDAYS = {
django/utils/dates.py:WEEKDAYS_ABBR = {
django/utils/dates.py:WEEKDAYS_REV = {
django/utils/_decimal.py:_condition_map = {ConversionSyntax:InvalidOperation,
django/utils/_decimal.py:_infinity_map = {
django/utils/simplejson/decoder.py:BACKSLASH = {
django/utils/simplejson/decoder.py:_CONSTANTS = {
django/utils/simplejson/encoder.py:ESCAPE_DCT = {
django/utils/termcolors.py:opt_dict = {'bold': '1', 'underscore': '4', 'blink': '5', 'reverse': '7', 'conceal': '8'}
django/utils/translation/trans_null.py:TECHNICAL_ID_MAP = {
django/utils/translation/trans_real.py:_accepted = {}
django/utils/translation/trans_real.py:_active = {}
django/utils/translation/trans_real.py:_translations = {}

Out of these, the following are read-only (i.e. not changed anywhere in code) or otherwise irrelevant: contrib/admin, formtools tests, localflavor mappings, core/cache, core/handlers, core/serializers/__init__.py:BUILTIN_SERIALIZERS, core/servers, db/backends, db/models/query.py, utils/dates.py, utils/_decimal.py, utils/simplejson, utils/termcolors.py, utils/translation/trans_null.py.

SITE_CACHE and everything in django.utils.translation.trans_real has already been listed under globals above.

_callable_cache and _resolver_cache in django/core/urlresolvers.py are used within the memoize decorator, result = func(*args) may be called more than once in utils/functional.py, but this should generally be a non-issue.

That leaves the following relevant global dicts not listed before:

django/core/serializers/__init__.py:_serializers = {}
django/db/models/fields/related.py:pending_lookups = {}
django/db/transaction.py:dirty = {}
django/db/transaction.py:state = {}
django/dispatch/dispatcher.py:connections = {}
django/dispatch/dispatcher.py:senders = {}
django/dispatch/dispatcher.py:sendersBack = {}
django/template/__init__.py:libraries = {}

Global lists

$ rgrep '^[[:alnum:]_]\+ *= *\[' django | egrep -v '(\.svn|_doctest\.py)' | sort

yields the following results

django/conf/urls/defaults.py:__all__ = ['handler404', 'handler500', 'include', 'patterns', 'url']
django/core/servers/basehttp.py:__all__ = ['WSGIServer','WSGIRequestHandler','demo_app']
django/core/servers/fastcgi.py:__all__ = ["runfastcgi"]
django/db/models/fields/__init__.py:BLANK_CHOICE_DASH = [("", "---------")]
django/db/models/fields/__init__.py:BLANK_CHOICE_NONE = [("", "None")]
django/template/__init__.py:builtins = []
django/template/loaders/app_directories.py:app_template_dirs = []
django/utils/checksums.py:__all__ = ['luhn',]
django/utils/_decimal.py:__all__ = [
django/utils/_decimal.py:rounding_functions = [name for name in Decimal.__dict__.keys() if name.startswith('_round_')]
django/utils/_decimal.py:_signals = [Clamped, DivisionByZero, Inexact, Overflow, Rounded,
django/utils/html.py:DOTS = ['&middot;', '*', '\xe2\x80\xa2', '&#149;', '&bull;', '&#8226;']
django/utils/html.py:LEADING_PUNCTUATION  = ['(', '<', '&lt;']
django/utils/html.py:TRAILING_PUNCTUATION = ['.', ',', ')', '>', '\n', '&gt;']
django/utils/simplejson/decoder.py:__all__ = ['JSONDecoder']
django/utils/simplejson/decoder.py:ANYTHING = [
django/utils/simplejson/encoder.py:__all__ = ['JSONEncoder']
django/utils/simplejson/__init__.py:__all__ = [
django/utils/simplejson/scanner.py:__all__ = ['Scanner', 'pattern']
django/utils/translation/__init__.py:__all__ = ['gettext', 'gettext_noop', 'gettext_lazy', 'ngettext',

Note: See TracWiki for help on using the wiki.

Download in other formats:

Plain Text

Issues

Context Navigation

Threading improvements in Django

Globals

Settings

django/contrib/sites/models.py

django/template

django/utils/translation

django/contrib/sites/models.py

Raw `grep` results

Globals accessed with the `global` keyword

Global dictionaries

Global lists

Download in other formats:

Django Links

Learn More

Get Involved

Get Help

Follow Us

Support Us

Issues

Context Navigation

Threading improvements in Django

Globals

Settings

django/contrib/sites/models.py

django/template

django/utils/translation

django/contrib/sites/models.py

Raw grep results

Globals accessed with the global keyword

Global dictionaries

Global lists

Download in other formats:

Raw `grep` results

Globals accessed with the `global` keyword