compilemessages breaks the virtualenv !

[Adrian Amaglio]

I have a django app that works well, in development and production.
I tried to translate the admin view so I created Meta subclasses, added verbose_name on my fields with the 'django.utils.translation.gettext_lazy' function.

Here is what I did :
Creation of virtualenv in my freshly pulled code

$ virtualenv djangovenv
Using base prefix '/usr'
New python executable in /home/py/git/court-metrage/djangovenv/bin/python
Installing setuptools, pip, wheel...done.

Sourcing and installing dependencies

$ source djangovenv/bin/activate
(djangovenv) $ pip install -r requirements.txt
Collecting django (from -r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/36/50/078a42b4e9bedb94efd3e0278c0eb71650ed9672cdc91bd5542953bec17f/   Django-2.1.5-py3-none-any.whl
Collecting pytz (from django->-r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/61/28/1d3920e4d1d50b19bc5d24398a7cd85cc7b9a75a490570d5a30c57622d34/   pytz-2018.9-py2.py3-none-any.whl
Installing collected packages: pytz, django
Successfully installed django-2.1.5 pytz-2018.9

Generating translation files

(djangovenv) $ ./manage.py makemessages -l fr_FR    
processing locale fr_FR

Here, I got my .po files with all the string from my model and I can run the server and everything is fine (seems logic as I did not change django files)

(djangovenv) $ ./manage.py runserver    
Performing system checks...    
    
System check identified no issues (0 silenced).    
January 25, 2019 - 16:48:51    
Django version 2.1.5, using settings 'courtmetrage.settings'    
Starting development server at http://127.0.0.1:8000/    
Quit the server with CONTROL-C. 

I generate the translation

(djangovenv) $ ./manage.py compilemessages
…
processing file django.po in /myrepo/djangovenv/lib/python3.7/site-packages/django/contrib/auth/locale/ne/LC_MESSAGES
…

The output is VERY verbose (1127 lines). I guess django generated admin lang for every locale ever. No errors so far…

But then, when I try to run manage.py with any parameters, I got an error :

ValueError: invalid token in plural form: EXPRESSION

The manage.py script is still running, maybe wating for file change.
The problem remains until I delete the virtualenv dir and recreate it…

I don’t know what is EXPRESSION, I tried to remove the line as some people advised on stackoverflow but the error remains…

Am I missing something ? How those basics steps can break my project ?
Running python 3.7 django 2.1 on manjaro linux

Full error and setting.py below :

---

Full error message :

(djangovenv) $ ./manage.py
Unhandled exception in thread started by <function check_errors.<locals>.wrapper at 0x7f430f40dae8>
Traceback (most recent call last):
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/utils/autoreload.py", line 225, in wrapper
    fn(*args, **kwargs)
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/core/management/commands/runserver.py", line 109, in inner_run
    autoreload.raise_last_exception()
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/utils/autoreload.py", line 248, in raise_last_exception
    raise _exception[1]
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/core/management/__init__.py", line 337, in execute
    autoreload.check_errors(django.setup)()
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/utils/autoreload.py", line 225, in wrapper
    fn(*args, **kwargs)
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/apps/registry.py", line 112, in populate
    app_config.import_models()
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/apps/config.py", line 198, in import_models
    self.models_module = import_module(models_module_name)
  File "/myrepo/djangovenv/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/contrib/auth/models.py", line 94, in <module>
    class Group(models.Model):
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/db/models/base.py", line 139, in __new__
    new_class.add_to_class(obj_name, obj)
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/db/models/base.py", line 305, in add_to_class
    value.contribute_to_class(cls, name)
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/db/models/fields/related.py", line 1583, in contribute_to_class
    self.remote_field.through = create_many_to_many_intermediary_model(self, cls)
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/db/models/fields/related.py", line 1051, in create_many_to_many_intermediary_model
    'verbose_name': _('%(from)s-%(to)s relationship') % {'from': from_, 'to': to},
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/utils/functional.py", line 149, in __mod__
    return str(self) % rhs
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/utils/functional.py", line 113, in __text_cast
    return func(*self.__args, **self.__kw)
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/utils/translation/__init__.py", line 75, in gettext
    return _trans.gettext(message)
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/utils/translation/trans_real.py", line 286, in gettext
    _default = _default or translation(settings.LANGUAGE_CODE)
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/utils/translation/trans_real.py", line 199, in translation
    _translations[language] = DjangoTranslation(language)
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/utils/translation/trans_real.py", line 97, in __init__
    self._add_installed_apps_translations()
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/utils/translation/trans_real.py", line 146, in _add_installed_apps_translations
    translation = self._new_gnu_trans(localedir)
  File "/myrepo/djangovenv/lib/python3.7/site-packages/django/utils/translation/trans_real.py", line 124, in _new_gnu_trans
    fallback=use_null_fallback,
  File "/usr/lib64/python3.7/gettext.py", line 533, in translation
    t = _translations.setdefault(key, class_(fp))
  File "/usr/lib64/python3.7/gettext.py", line 260, in __init__
    self._parse(fp)
  File "/usr/lib64/python3.7/gettext.py", line 402, in _parse
    self.plural = c2py(plural)
  File "/usr/lib64/python3.7/gettext.py", line 183, in c2py
    result, nexttok = _parse(_tokenize(plural))
  File "/usr/lib64/python3.7/gettext.py", line 116, in _parse
    nexttok = next(tokens)
  File "/usr/lib64/python3.7/gettext.py", line 93, in _tokenize
    raise ValueError('invalid token in plural form: %s' % value)
ValueError: invalid token in plural form: EXPRESSION
^C

settings.py

import os
from django.utils.translation import gettext_lazy as _

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

DEBUG = True

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = os.environ['SECRET_KEY'] if not DEBUG else 'm)5dci82fzm%d%p0rtj-sv9gd+#vamnceby#6sc4)$7_fa&@-b'

ALLOWED_HOSTS = ['localhost', '127.0.0.1']

# Application definition

INSTALLED_APPS = [
    'events.apps.EventsConfig',
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.locale.LocaleMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

STATICFILES_FINDERS = (
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    # other finders..
)

ROOT_URLCONF = 'courtmetrage.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
                'events.context_processors.nav',
            ],
        },
    },
]

WSGI_APPLICATION = 'courtmetrage.wsgi.application'

# Database

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}

# Password validation

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]

# Internationalization

LANGUAGE_CODE = 'fr_FR'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_L10N = True

USE_TZ = True

prefix_default_language=False

LANGUAGES = [
    ('fr', _('Français')),
    ('en', _('English')),
]

LOCALE_PATHS = [
    os.path.join(BASE_DIR, 'locale'),
]


# Static files (CSS, JavaScript, Images)

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static')

STATICFILES_DIRS = [
#    os.path.join(BASE_DIR, "static"),
]

[Ramiro Morales]

What are the paths of virtualrnv and the Django project? Do you have the virtualenv's lib/ directory under the Django project root dir?

[Adrian Amaglio]

/myrepo
        /djangovenv -> the virtualenv
                /lib
                ...
        /courtmetrage -> the django project main directory
                /settings.py
                /urls.py
                ...
        /events -> the app I tried to translate
                /models.py
                ...
        /locales -> the generated locales. Maybe that should go in my project main directory ?

So the lib directory is not under the django root directory

Edit : I tried to create a virtualenv in /myrepo/courtmetrage/venv, the problem is the same

[Claude Paroz]

Put your virtualenv folder outside of your base Django project (where your manage.py resides) and all should be fine.
When you run compilemessages you should not see any path from the virtualenv directory appear.

[Adrian Amaglio]

So I put the virtualenv in ~/.venv/courtmetrage, compilemessages only displays my fr_FR locale.
But I still got the main error when I type './manage.py'
ValueError: invalid token in plural form: EXPRESSION

Maybe I missed something about translation ?

When I run './manage.py runserver' the app still hangs, waiting for file change.

Edit : new precision : regardless of venv recreation, the error persists…
I have to delete the /locale/fr_FR/LC_MESSAGES/django.mo file to "fix" it. I guess the translation file is somewhat wrong…

[Claude Paroz]

It may be that Django couldn't determine the proper plural equation for your translation file. Try to rather use python manage.py makemessages -l fr and remove the fr_FR files.

[Adrian Amaglio]

It works ! What is the difference between fr_FR and fr ?
Thanks a lot !

[Claude Paroz]

fr is the generic French locale, fr_FR is France-specific variant, fr_CA the Canadian French variant, etc.