Allow URLValidator to accept schema relative URLs

[Maciej Strömich]

Based on rfc1808#section-2.4.3 a valid schema relative URL is the one starting with //

URLValidator is not validating such urls.

>>> from django.core.validators import URLValidator
>>> value = '//example.com'
>>> URLValidator().__call__(value)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/opt/venv/lib/python3.8/site-packages/django/core/validators.py", line 110, in __call__
    raise ValidationError(self.message, code=self.code, params={'value': value})
django.core.exceptions.ValidationError: ['Enter a valid URL.']

[Maciej Strömich]

formatting

[Tim Graham]

The ​default list of accepted schemes is ['http', 'https', 'ftp', 'ftps'], so I wouldn't call this a bug. I guess the question is whether or not there should be a way to allow schema relative URLs (assuming adding // to schemes don't already work, I haven't tried it).

[Carlton Gibson]

Yes, I guess it's worth looking into whether we can add support for reasonable complexity. Thanks.

[Maciej Strömich]

The issue with scheme relative urls lies in this code block ​https://github.com/django/django/blob/stable/3.2.x/django/core/validators.py#L108-L110 in which scheme is split from url based on :// which in the absolute url makes total sense.

The ​default list of accepted schemes is ['http', 'https', 'ftp', 'ftps'], so I wouldn't call this a bug. I guess the question is whether or not there should be a way to allow schema relative URLs (assuming adding to schemes don't already work, I haven't tried it).

We were supporting schema relative urls with update to mentioned list in a way like this ["", "http", "https"] and it works in Django 2.2 well if you're allowing to define the url in your forms like ://example.com instead of //example.com.

Between Django2.2 and Django 3.2 ​https://github.com/django/django/commit/b41d38ae26b1da9519a6cd765bc2f2ce7d355007 was introduced which changed the behaviour of urlsplit() call.

When upgrading to Django 3.2 URLValidator started to throw TypeError exceptions because the default return value of urlsplit('://example.com).hostname is None and it wasn't even getting to the ValidationError line below.

In Django 2.2 the code was checking for netloc instead of hostname which even if the netloc is not correctly found would return an empty string '' and the len(urlsplit(.. code would pass because '' is less than 253 characters.

[Zoltán Szatmáry]

I've just made a PR for that. https://code.djangoproject.com/ticket/32874

[Carlton Gibson]

After review by the Django Security Team, we're going to close this as wontfix.

Protocol relative URLs are something of a legacy from times before HTTPS was the norm, and their use now is generally discouraged. (e.g. ​modern linters will flag them.)

What's more URLs are used in many non-web contexts, where the lack of a scheme is not valid.

As such it's not something we want to add to URLValidator.
As with other similar cases, if users want to accept such URLs in their application, a custom validator is the way forward.