Opened 14 months ago

Last modified 3 weeks ago

#21699 new New feature

Provide a way to define a model without being registered into the app registry / Get rid of get_registered_model

Reported by: mitar Owned by: nobody
Component: Core (Other) Version: master
Severity: Normal Keywords: app-loading
Cc: mmitar@… Triage Stage: Someday/Maybe
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Provide a way to define a model without being registered into AppConfig. This is useful when you are creating dynamic models to be returned from querysets which have dynamic virtual fields (for example, created from data which can have arbitrary virtual fields stored). You do not want all those models to be stored in AppCache.

Change History (13)

comment:1 Changed 14 months ago by aaugustin

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

I know I'm insisting heavily, but can you formulate this feature request in terms of effects on the public API of the app registry, or any other public API of Django?

For example:

  • "I would like app_config.get_model(xxx) to return None when I've created a model in this way."
  • "I would like Django not to crash miserably with this stack trace when I've created a model in that way."

A request on the internal implementation of the app registry doesn't make sense because only public APIs are guaranteed to keep working as documented across versions. Why would you care that the model is or isn't referenced somewhere in the app registry, provided it's never returned or used by any API? In fact, if this feature gets implemented, it's likely that Django will register this model with some sort of "hidden" flag.

comment:2 Changed 14 months ago by charettes

I'm also curious to know why you'd need such a feature? There was no way of doing this with the legacy django.db.models.loading AFAIK.

Wouldn't making a django.apps.registry.Apps no-op subclass and specifying it as your DynamicModel.Meta.apps work?

class Void(django.apps.registry.Apps):
    def register_model(self, app_label, model):
        pass

void = Void()

Meta = type('Meta', (), {'apps': void})
MyDynamicModel = type('MyDynamicModel', (django.db.models.Model,), {'Meta': Meta})

comment:3 Changed 14 months ago by aaugustin

Setting Model._meta.auto_created most likely does exactly what the OP wants, but since they didn't specify, I can't be sure.

https://github.com/django/django/blob/98b52ae2/django/apps/registry.py#L222

comment:4 Changed 14 months ago by aaugustin

Model._deferred might be even more suitable in that case, since it's what Django does for its own dynamic models.

Of course it would be better to provide a formal API.

comment:5 Changed 14 months ago by aaugustin

  • Version changed from 1.6 to master

Maybe the goal is to avoid leaking memory when creating a large number of such models?

comment:6 Changed 14 months ago by mitar

I am just copying from #21698, to make things more clear here. So my comment how I see that unregistering could help:

So there are two ways to allow making custom subclasses of models which should not be registered. One is to somehow allow some configuration in Meta (#21682), another is to call current metaclass which adds it, but be able to remove it from registry immediately afterwards (a bad side-effect is that registration signal has been already made).


I didn't know about "no-op subclass" approach. This seems interesting idea.

Maybe the goal is to avoid leaking memory when creating a large number of such models?

Yes. So what we are doing is developing support for dynamic schema. Users can add dynamically fields from the many hop away models. In a way we make proxy fields to things you would need multiple hops and we do one big join instead in the background to not have to fetch them again and again as you are hopping around. (Those hops can cross content type relations as well.) (Which mean you can change concrete implementation of a model, but the rest of your code can still work as it is referencing them through registered names.) We add those fields as virtual fields into base model and create a dummy proxy model to keep them, when we return a new queryset for them. So the syntax is something like:

queryset = models.Node.objects.regpoint('config').registry_fields(
    name='core.general#name',
    type='core.type#type',
    router_id='core.routerid#router_id',
    project='core.project#project.name',
).regpoint('monitoring').registry_fields(
    last_seen='core.general#last_seen',
    network_status='core.status#network',
    monitored_status='core.status#monitored',
    health_status='core.status#health',
    peers='network.routing.topology#link_count',
).order_by('type')

So, after you fetch those additional fields, returned queryset contains a dynamically created proxy model to models.Node, with name and others added as virtual fields. As this proxy model is useful only for this particular queryset, we do not want to register it globally. core.status is a registered name for a particular registry point, which can be implemented by any model which announces that it is implementing it. So a concrete related model can change (be extended by an app), but your queryset does not have to change.

All this is a bit involved and we do not yet have good documentation, but you can check this description and our current implantation of creating a dynamic proxy model and unregistered immediately afterwards is here. We do not want that such models pile up.

I know this is not public API, but it would still be good to have some clear way to: or not register a model when created, or be able to remove it from the registry. As you are planing on rewriting AppConfig, I just wanted to raise a point that some internal API (which I understand can change at any time) would be useful here.

Last edited 14 months ago by mitar (previous) (diff)

comment:7 Changed 14 months ago by aaugustin

So we're talking of hacking the internals heavily here ;-)

The implementation of Apps has become very straightforward if you ignore all the deprecated code. In the short term I'd just go for del apps.all_models[app_label][model_name] or del apps.get_app_config(app_label).models[model_name], followed by apps.clear_cache(). The first one is marginally faster, the second one is marginally less likely to change. Make sure model_name is lowercase.

Yesterday I tried to stop storing deferred models with the app registry. It would have solved your use case nicely -- what you're doing looks a lot like deferred models. Sadly it didn't work.

This request has larger ramifications. Currently ModelBase.new always returns a class registered in the app registry. It's quite a hack, and it's done for bad reasons that no longer exist since Django 1.4 (new project layout). I'm going to keep the ticket open with the following scope: get rid of get_registered_model, and then figure out what the registration API becomes.

comment:8 Changed 14 months ago by aaugustin

  • Triage Stage changed from Unreviewed to Someday/Maybe

comment:9 Changed 14 months ago by mitar

So we're talking of hacking the internals heavily here ;-)

Yes. :-) But from such hacking new use cases emerge. :-)

Currently ModelBase.new always returns a class registered in the app registry.

Yes. This is why we register, and then try to unregister. So maybe then #21698 is a better approach. The issue there is that you still send a signal when a new model is registered, despite then removing it almost immediately. Maybe an alternative would be that API could provide a way to register a model without sending a signal, and then to unregister. But this is just a hack for current approach with get_registered_model.

comment:10 Changed 14 months ago by aaugustin

ModelBase.__new__ cannot return a class previously registered in the app registry any more.

get_registered_model is still used in ModelSignal.connect and in add_lazy_relation. In both cases it triggers different code paths depending on whether the target model was already created and its class_prepared signal was sent.

comment:11 Changed 13 months ago by aaugustin

  • Summary changed from Provide a way to define a model without being registered into AppConfig to Provide a way to define a model without being registered into the app registry / Get rid of get_registered_model

mitar, how should relations (foreign keys, one to one, many to many) between such a model and other models behave?

comment:12 Changed 5 months ago by kostko

mitar, how should relations (foreign keys, one to one, many to many) between such a model and other models behave?

Do you see any specific problems here?

comment:13 Changed 3 weeks ago by aaugustin

Yes. If you have multiple variants of a model with relations to another model, the reverse relations conflict on the other model.

Note: See TracTickets for help on using tickets.
Back to Top