Add support for Generic Relations / Table-Valued Expressions in the ORM

[Pravin]

This ticket proposes the implementation of a generic Relation API within the Django ORM to support Table-Valued Expressions (TVEs) and functions that return sets of rows.

A generic Relation API will lay the groundwork for cleanly implementing several advanced querying features across different databases, including:

• PostgreSQL generate_series() (and potential equivalents for other backends).
• Subqueries acting as derived tables directly within the FROM clause.
• Database abstraction for json_each() , JSON_TABLE and json_array_elements().
• Deprecation of FilteredRelation.

References & Context

• Django Forum Discussion: ​https://forum.djangoproject.com/t/proposal-add-generate-series-support-to-contrib-postgres/21947
• Django New Features Thread:​https://github.com/django/new-features/issues/25
• Official GSoC 2026 Announcement: ​https://www.djangoproject.com/weblog/2026/may/05/gsoc-2026-django-contributors/

[Jacob Walls]

Thanks. Just clarifying that Subquery is probably the easiest tool for validating the approach; the other bullet points mentioned above should probably get their own separate tickets later.

[Simon Charette]

As mentioned in a few locations already (forum, discord, github) I think that in order to even consider adding this feature we must add support for a generic CompositeField. We don't have a ticket for that but Lilly did some exploration work for it ​here that predated adding support for CompositePrimaryKey.

The reason is simple, we need a way to adequately represent the output_field of these Expression to remove the hack we have right now on sql.Query.output_field. Once we have that the problem of supporting table-like expressions becomes much simpler as we can make members of sql.Query.alias_map (which hosts table-like references) subclasses of Expression instead of opaque BaseTable and Join datastructures and augment them with compilation directives such as where they should live (inlined in the FROM clause, as a CTE) and how they should be combined (e.g. LATERAL JOIN)

From there things like

Author.objects.alias(
   book=Books.objects.values("title", "number_of_pages")
)

become much easier to implement as any expression that has .output_field.is_composite can be treated a candidate for alias_map inclusion.

[Jacob Walls]

Wonderful. Here's one ​forum post where Simon fleshes this idea out at more length.

Pravin started iterating on a design ​in a markdown document. In it, there was a first swing at representing a composite field with a columns dict, and then some ideas about "intercepting" this later, but I agree that Simon's proposal is more idiomatic, more general, and opens more doors.

Pravin, would you be open to reimagining your design document as a small-ish DEP that we could iterate on via a pull request workflow? From the experience doing something similar with the MAILERS implementation in Django 6.1, this won't arrest progress on the implementation, as you can be iterating on a proof-of-concept in parallel.

[Pravin]

Sound great, Yeah I will reform my design into small DEP .
Thank you.

[Shai Berger]

It seems like this may be related to the ​suggestion to add CTE support to the ORM.

[Simon Charette]

It seems like this may be related to the ​suggestion to add CTE support to the ORM.

I believe so. The moment we have

1. A way to express a tuple of expressions (CompositeField) as Expression.output_field
2. Adapted BaseTable and Join machinery to take advantage of 1.

The only part missing to add support for CTE is to allow compilation directives to be attached to table like objects (e.g. a TableExpression(Expression).output_field: CompositeField that would serve as a base class for BaseTable and Join) to inform the compilation layer (compiler.SQLCompiler) that the resulting SQL should prefix the query instead of being inlined in the FROM clause.

[Jacob Walls]

Feedback on ​draft DEP welcome.

[Jacob Walls]

The first prerequisite for this work (the CompositeField expression) will target #5929, which I just reopened. When we have that, we can come back to the set-returning function support.