Add compilemessages --ignore option for excluding search paths

[Elger Jonker]

The makemessages command contain an ignore flag, this allows you to skip directories like 'vendor'.

In our project we have a directory called .tox, this directory is ignored during makemessages. Yet, when compilemessages is run, the .tox directory (which contains a lot of sources) is included with compiling translations.

Now we might have things wrong, but it seems strange to me that makemessages has ignore options, while compilemessages doesn't. Even if it's wrong, we might still want to ignore this folder because we're clumsy and it works just 'good enough'.

The command we'd like to use looks like:

django compilemessages --ignore vendor --ignore .tox -l EN

[Tim Graham]

I guess you want to use the --exclude option.

$ python manage.py compilemessages --ignore vendor
usage: manage.py compilemessages [-h] [--locale LOCALE] [--exclude EXCLUDE]
                                 [--use-fuzzy] [--version] [-v {0,1,2,3}]
                                 [--settings SETTINGS]
                                 [--pythonpath PYTHONPATH] [--traceback]
                                 [--no-color] [--force-color]
manage.py compilemessages: error: unrecognized arguments: --ignore vendor

See TicketClosingReasons/UseSupportChannels for places to get help before filing a bug report.

[Elger Jonker]

The --exclude command is about locales. This issue is about having too many directories being used in compilemessages.

Here is an example where i'm in need of an --ignore during compilemessages, including the line where the issue resides and an example output of where things go wrong:

Using this command:

django-admin compilemessages -l NL --exclude DE

And this filter on what directory constitutes a 'locale directory':
​https://github.com/django/django/blob/eac9ab7ebb1ce0cbbc79c4cf65e8f70b0635a240/django/core/management/commands/compilemessages.py#L69

It's clear that the filter is far too greedy. And in my case it results in a list of directories that should not be included. Hence the request for an --ignore option just like in makemessages.

The directory filter currently delivers this output on my machine, again: everything named 'locale' is included.

['conf/locale',
 'locale',
 'locale',
 './locale',
 './.tox/default/lib/python3.6/site-packages/import_export/locale',
 './.tox/default/lib/python3.6/site-packages/jet/locale',
 './.tox/default/lib/python3.6/site-packages/jet/dashboard/locale',
 './.tox/default/lib/python3.6/site-packages/constance/locale',
 './.tox/default/lib/python3.6/site-packages/adminsortable2/locale',
 './.tox/default/lib/python3.6/site-packages/leaflet/locale',
 './.tox/default/lib/python3.6/site-packages/dal_select2/locale',
 './.tox/default/lib/python3.6/site-packages/sphinx/locale',
 './.tox/default/lib/python3.6/site-packages/django_countries/locale',
 './.tox/default/lib/python3.6/site-packages/django_extensions/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/auth/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/admin/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/flatpages/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/sites/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/postgres/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/redirects/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/sessions/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/humanize/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/contenttypes/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/gis/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/admindocs/locale',
 './.tox/default/lib/python3.6/site-packages/django/conf/locale',
 './.tox/default/lib/python3.6/site-packages/debug_toolbar/locale',
 './failmap/locale',
 './failmap/organizations/locale',
 './failmap/game/locale',
 './failmap/map/locale',
 './vendor/dnscheck/engine/locale']

You can also run it for yourself, by creating a 'locale' directory somewhere and then run this part of the django source code:
​https://github.com/django/django/blob/eac9ab7ebb1ce0cbbc79c4cf65e8f70b0635a240/django/core/management/commands/compilemessages.py#L67

basedirs=[]
for dirpath, dirnames, filenames in os.walk('.', topdown=True): 
    for dirname in dirnames: 
        if dirname == 'locale': 
            basedirs.append(os.path.join(dirpath, dirname)) 

basedirs

[Ryan Siemens]

Hey Manu,

I have a patch in place for this. Do you mind if I take over?

[Manu Pascual]

Of course, I would have liked to do it this last weekend, but I was sick. So all your sir.

[Ryan Siemens]

I had initially implemented this to just drop a directory by name, but I think thats actually incorrect.
Consider a directory structure like so

./
 |- locale/
 |- foo/
   |- locale/
 |- bar/
   |- foo/
     |- locale/

With my current patch doing compilemessages --ignore=foo would drop ./foo and ./bar/foo from being searched, but I think it should only drop dirs relative to the path. So in this case only ./foo wouldn't be searched. Also I think I it should support globbing similar to makemessages --ignore. Thoughts or opinions?

Thanks.

[Carlton Gibson]

OK, yes. This seems a reasonable addition to mirror makemessages.

[Simon Charette]

Left a few comments for non-critical improvements.

[Ryan Siemens]

Resolved PR review comments. Thanks for the thorough review Simon!

[Simon Charette]

Except for the conflict in the 2.2 release notes that'll to be either fixed or changed to target 3.0 the patch looks RFC to me.

[Tim Graham <timograham@…>]

In bc9f0b3:

Refs #29973 -- Extracted helper functions from makemessages.

[Tim Graham <timograham@…>]

In a168e56:

Fixed #29973 -- Added compilemessages --ignore option.