Opened 4 years ago

Last modified 4 months ago

#16508 assigned New feature

Provide real support for virtual fields

Reported by: vzima Owned by: pirosb3
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords:
Cc: pirosb3 Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


I have created a virtual field based on GenericForeignKey, but I found out that it can not be properly used for model initiation.

Altered example from my test:

class Managing(models.Model):
    shortcut = models.CharField(max_length=20)
    name_cs = TranslationProxyField('name', 'cs', False)

class ManagingTranslation(models.Model):
    name = models.CharField(max_length=20)

class TranslationProxyField(object):
    Descriptor that is a virtual field that provides access to translations.
    def __init__(self, field_name, language_code=None, fallback=False):
        self._field_name = field_name
        self._language_code = language_code
        self._fallback = fallback
        names = [field_name]
        if language_code is not None:
            names.append(language_code.replace('-', '_'))
        if fallback:
            names.append(FALLBACK_FIELD_SUFFIX) = '_'.join(names)

    def contribute_to_class(self, cls, name):
        if != name:
            raise ValueError('Field proxy %s is added to class under bad attribute name.' % self)
        self.model = cls

        # Connect myself as the descriptor for this field
        setattr(cls, name, self)

# This results in Type error from Model.__init__, because of extra kwarg
Managing(shortcut='name', name_cs='jméno')

I tried to use pre_init signal, but I found out that I will not get instance argument, so I can not use that. This way is truly usable only for virtual fields which are like GenericForeignKey built on real model fields of object. When it gets more complicated, it can not be used. Confusing on this path is that instance is in providing_args of pre_init signal.

Currently I solve this by quite a hack which is deriving of TranslationProxyField from property instead of object. This works because of setting properties is allowed in Model.__init__.

In my opinion, a stable solution would be either providing an instance argument in pre_init signal or enable setting of virtual field in model initiation. Second option would also enable removal of pre_init connection from GenericForeignKey.

Exact examples can be found at github:

Change History (9)

comment:1 Changed 4 years ago by bpeschier

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

Virtual fields are also discussed shortly by Michal Petrucha for the Composite fields API [1] and in further responses; it seems like there are more use cases, but I would strongly suggest taking it up on the django-developers mailing list to further discuss it.


comment:2 Changed 3 years ago by akaariai

  • Type changed from Uncategorized to New feature

comment:3 Changed 15 months ago by pirosb3

  • Cc pirosb3 added
  • Owner changed from nobody to pirosb3
  • Status changed from new to assigned


I will be taking this ticked under consideration as we are currently planning a refactor of _meta.


comment:5 Changed 7 months ago by akaariai

I'm planning on tackling this issue. The initial plan is to:

  • Change ForeignObjectRel subclasses to real field instances. (For example, ForeignKey generates a ManyToOneRel in the related model). The Rel instances are already returned from get_field(), but they aren't yet field subclasses.
  • Allow direct usage of ForeignObjectRel subclasses. In certain cases it can be advantageous to be able to define reverse relations directly. For example, see
  • Partition ForeignKey to virtual relation field, and concrete data field. The former is the, the latter model.author_id's backing implementation.
  • Consider other cases where true virtual fields are needed.

My initial try will be a class structure like:

        RelationField (base class for ForeignKey, ManyToOneRel, maybe also ManyToManyField)
        ConcreteField (base class for concrete data fields, for example IntegerField, ...)

I expect this to cause a good amount of breakage in 3rd party apps. I'm going to use the old good "lets first break things, then fix those cases that are complained about".

comment:6 Changed 7 months ago by pirosb3

Thanks akaariai,

If you need any help on my side (given the work I did on Meta) please let me know.


comment:7 Changed 7 months ago by ovangle

Could I propose the following class structure:


The addition of an intermediary VirtualField class would be for future support of a CompositeField class, which isn't logically a RelationField, but is logically virtual (and subfields of the CompositeField would be limited to concrete fields for simplicity of implementation).

Last edited 7 months ago by ovangle (previous) (diff)

comment:8 Changed 4 months ago by nemesisdesign

I am also interested in this ticket.

See this other discussion on Django Developers:!topic/django-developers/_FmJRK3sJGs].

What's the proposed definition of VirtualField?

I have a proposal:

"A virtual field is a model field which it correlates to one or multiple concrete fields, but doesn't add or alter columns in the database."

Best regards

comment:9 Changed 4 months ago by collinanderson

I believe in the _meta refactor we simply defined it as "doesn't add or alter columns in the database.", something like field.column = None. It probably does correlates to one or multiple concrete fields, but it doesn't have to.

Also, one thing to note is that we don't want to distinguish (too much) between virtual and non virtual on the higher levels of the API (like Model Forms). Model Forms shouldn't care whether a field has a column in the database or not. So in that case, it doesn't make sense to have a _meta.virtual_fields like we used to do. But of course the ORM and migrations are going to want to know that they should ignore these fields.

Note: See TracTickets for help on using tickets.
Back to Top