Docs for custom db_type() fields are misleading about get_internal_type() -- can silently break exact-match lookups on Django 5.0+

[Robert Raposa]

Background

This ticket was prompted by a real production incident: a Django 5.2 upgrade silently broke
updates to existing rows once a table's primary keys passed 2³¹-1, in a field type that had
followed the docs' current advice (see below). This draft was written with AI assistance
(Claude), disclosed here for transparency, consistent with Trac's practice of asking reporters to
note AI tool use (see discussion on #36484).

Description

The "Emulating built-in field types" section of the custom-model-fields how-to
(​https://docs.djangoproject.com/en/6.1/howto/custom-model-fields/#emulating-built-in-field-types)
opens with:

If you have created a db_type() method, you don't need to worry about
get_internal_type() -- it won't be used much.

This is backwards for any field that widens or changes the *range* of a built-in type via
db_type()/rel_db_type() without also updating get_internal_type(). Since Django 5.0
(unchanged through 5.1 and 5.2; added in dde2537fbb04, "Fixed #27397 -- Prevented integer
overflows on integer field lookups"), exact-match lookups (IntegerFieldExact, via
IntegerFieldOverflow in django/db/models/lookups.py) call
connection.ops.integer_field_range(field.get_internal_type())
*before building any SQL*, and raise EmptyResultSet if the lookup value falls outside that
range -- silently returning zero rows, with no exception, no query sent to the database.
get_internal_type() is very much "used," just not where this doc section implies.

Related: #28702 hit the same shape in 2017 for CIText fields (db_type() wasn't consulted where
the docs implied it would be, this time for casting rather than range). A comment there noted the
docs "need[] to be adjusted" -- but the fix landed only for that specific case; the generic
misleading sentence above is still present today.

Reproduction

from django.db import models
from django.db.models import AutoField

class UnsignedBigIntAutoField(AutoField):
    def db_type(self, connection):
        return 'bigint UNSIGNED AUTO_INCREMENT' if connection.vendor == 'mysql' else super().db_type(connection)
    def rel_db_type(self, connection):
        return 'bigint UNSIGNED' if connection.vendor == 'mysql' else super().rel_db_type(connection)

class Widget(models.Model):
    id = UnsignedBigIntAutoField(primary_key=True)

Following the doc's advice literally -- defining db_type()/rel_db_type() and not touching
get_internal_type() -- leaves get_internal_type() at the inherited 'AutoField'. On MySQL,
once the table's AUTO_INCREMENT sequence passes 2³¹-1 (perfectly valid for an unsigned bigint
column, and not unusual for a long-lived high-traffic table):

>>> Widget.objects.filter(pk=4_312_695_480).exists()
False   # the row exists; this should be True
>>> Widget.objects.get(pk=4_312_695_480)
Widget.DoesNotExist

Model.save(force_update=True)'s internal filter(pk=...).exists() check hits the same path,
so this isn't just a read-lookup quirk: saves/updates to existing rows silently stop working
(raising DatabaseError("Forced update did not affect any rows.") from Django's own save()
machinery) once a table's ids cross that boundary, with no application-level error pointing at
the real cause.

Our fix

We hit this in production with a bigint-unsigned-PK AutoField subclass predating Django 5.0;
upgrading to Django 5.2 turned it into a live incident once ids passed 2³¹-1. Fix was a
one-line get_internal_type() override returning 'BigAutoField' (wide enough 64-bit signed
range to never clip a real value, even though the column is unsigned):
edx/edx-platform#495.

Possible follow-up (not proposing a patch here)

Django already has the plumbing to catch this class of bug at manage.py check time:
Field._check_backend_specific_checks() already calls
connections[alias].validation.check_field(), and MySQL's check_field_type()
(django/db/backends/mysql/validation.py) already receives both the field and its rendered
db_type(). Comparing that against connection.data_types.get(field.get_internal_type()) and
warning on a mismatch could have caught this years before any table's ids reached 2³¹. Flagging
the idea for consideration, not attaching an implementation -- #36484 suggests new
settings/behavior changes get a high bar here, so happy to defer on whether/how this should be
pursued.