django.apps.apps.get_model() does not allow namespaced (PEP 420) apps to get a model

[Jonas L.]

The django.contrib.auth app use django.apps.apps.get_model to load the AUTH_USER_MODEL from the settings. While doing so it leverage the shortcut to provide the app_name.ModelName.

The shortcut does not take into count that app_name can be a namespaced (PEP 420) app like my_namespace.my_app, and that we might want to load a model from my_namespace.my_app.User.

This result in the following error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/django/db/models/utils.py", line 15, in make_model_tuple
    app_label, model_name = model.split(".")
ValueError: too many values to unpack (expected 2)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/libretime-api", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/dist-packages/libretime/api/cli.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.7/dist-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.7/dist-packages/django/core/management/__init__.py", line 395, in execute
    django.setup()
  File "/usr/local/lib/python3.7/dist-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib/python3.7/dist-packages/django/apps/registry.py", line 114, in populate
    app_config.import_models()
  File "/usr/local/lib/python3.7/dist-packages/django/apps/config.py", line 301, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.7/dist-packages/django/contrib/admin/models.py", line 39, in <module>
    class LogEntry(models.Model):
  File "/usr/local/lib/python3.7/dist-packages/django/db/models/base.py", line 161, in __new__
    new_class.add_to_class(obj_name, obj)
  File "/usr/local/lib/python3.7/dist-packages/django/db/models/base.py", line 326, in add_to_class
    value.contribute_to_class(cls, name)
  File "/usr/local/lib/python3.7/dist-packages/django/db/models/fields/related.py", line 747, in contribute_to_class
    super().contribute_to_class(cls, name, private_only=private_only, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/django/db/models/fields/related.py", line 318, in contribute_to_class
    lazy_related_operation(resolve_related_class, cls, self.remote_field.model, field=self)
  File "/usr/local/lib/python3.7/dist-packages/django/db/models/fields/related.py", line 80, in lazy_related_operation
    return apps.lazy_model_operation(partial(function, **kwargs), *model_keys)
  File "/usr/local/lib/python3.7/dist-packages/django/db/models/fields/related.py", line 78, in <genexpr>
    model_keys = (make_model_tuple(m) for m in models)
  File "/usr/local/lib/python3.7/dist-packages/django/db/models/utils.py", line 24, in make_model_tuple
    "must be of the form 'app_label.ModelName'." % model
ValueError: Invalid model reference 'libretime.api.User'. String model references must be of the form 'app_label.ModelName'.

Some links:
​https://github.com/django/django/blob/e2f778d57947d168a875159e6df075255eea4bbc/django/contrib/auth/__init__.py#L160
​https://docs.djangoproject.com/en/3.2/ref/applications/#django.apps.apps.get_model

[Mariusz Felisiak]

Thanks for the ticket, however Django doesn't support namespace packages, see ​a discussion on the mailing list. It's also ​documented that AUTH_USER_MODEL is "dotted pair describes the name of the Django app ... and the name of the Django model that you wish to use as your user model".

[Jonas L.]

Replying to Mariusz Felisiak:

Thanks for the ticket, however Django doesn't support namespace packages, see ​a discussion on the mailing list. It's also ​documented that AUTH_USER_MODEL is "dotted pair describes the name of the Django app ... and the name of the Django model that you wish to use as your user model".

In that case the following documentation is misleading ​https://docs.djangoproject.com/en/3.2/ref/applications/#namespace-package

I understood that if I meet the requirements, using a namespace is possible. Or maybe I didn't understand it right ?

[Carlton Gibson]

PEP 420 is about being able to compose a package from code situated at different locations on the filesystem. These are known as namespace packages.

There's a ​Nested namespace packages example but it's not really about namespaces',' as in nesting that you get from folders, that we're familiar with from Namespaces are one honking great idea -- let's do more of those''

[Jonas L.]

There's a ​Nested namespace packages example but it's not really about namespaces',' as in nesting that you get from folders, that we're familiar with from Namespaces are one honking great idea -- let's do more of those

Sorry, I am not sure I understand your comment here.

I probably didn't explain enough my use case.

We have few services, playout, analyzer, api, and more. Only one of them is actually based on Django the api.

Our goal is to put them into the libretime namespace so we don't collide with other packages/modules. This would result in libretime.playout, libretime.analyzer, libretime.api.

The django app is still full of init.py files, we just wanted to wrap it into the namespace so we can separate the different services.

I understood the documentation like this: "Since our django app does not have anything anything outside the libretime/api folder, we can put it in the libretime namespace.

So based on my explanation, is it doable ? And how can we make the documentation less misleading, as it seem to perfectly fit my uses case ?

Thanks for your time, and sorry if I have trouble to understand your answers, the question was probably too short.

[Jonas L.]

Oooh, first let me apologize for not noticing the different between an App.name and an App.label.

I guess I could fix my issue by using a custom App.label, without dot.

My app now looks like this:

from pathlib import Path

from django.apps import AppConfig

here = Path(__file__).parent


class LibreTimeAPIConfig(AppConfig):
    name = "libretime.api"
    label = "libretime_api"
    path = here
    verbose_name = "LibreTime API"
    default_auto_field = "django.db.models.AutoField"

And it seem to be working.

Sorry again for this new comer error.

[Mariusz Felisiak]

Replying to Jonas L.:

Oooh, first let me apologize for not noticing the different between an App.name and an App.label.

I guess I could fix my issue by using a custom App.label, without dot.

Yes, I proposed a small clarification, see ​PR.

[GitHub <noreply@…>]

In fd881e8c:

Refs #33207 -- Clarified that AUTH_USER_MODEL expects an app label.

[Mariusz Felisiak <felisiak.mariusz@…>]

In ea66d1f2:

[4.0.x] Refs #33207 -- Clarified that AUTH_USER_MODEL expects an app label.

Backport of fd881e8cd9b7686ab8fcd32332100710a8ffaa10 from main