add DomainNameValidator to validate Internet Domain Names

[michele]

Internet Domain Names are very present throughout, e.g. in URLs and e-mail addresses.

This patch adds django.core.validators.DomainNameValidator to check Internet Domain Names.

DomainNameValidator supports IDNA (utf) domain names, but it can reject them if optional accept_idna=False is passed to the constructor (defaults to accept).

DomainNameValidator is implemented as follows:

• derives RegexValidator
• uses the same RegEx of URLValidator (with the additional constrain to accept only 127 labels / 255 chars, as per RFC)
• if plain validation fails, converts UTF -> ASCII with IDNA encoding and attempts to validate again (unless accept_idna=False)

Based on this Validator, I will submit separate patches to add DomainNameField support for models.

[Claude Paroz]

I think it is a good idea to centralize domain name validation. I detected at least three points where we could use this validator (django/core/mail/message.py, django/core/validators.py (URLValidator), django/utils/html.py).

For DRY reasons, the domain regex part should be shared with URLValidator. In __init__, just use self.accept_idna = kwargs.pop('accept_idna', True)

[michele]

DomainNameValidator patch #2 - remove validator-own attributes before super()

[michele]

In attachment:domainnamevalidator_2.txt​ :

• fix the constructor argument as per claudep suggestion
• return a specific error message based on IDNA being accepted or not

[Berker Peksag]

​https://github.com/django/django/pull/3477

[Tim Graham]

This is blocked on #20003. Patch will need to be updated after that change.

[Thomas Güttler]

would be nice to have

[Berker Peksag]

Updated ​PR.

[Nina Menezes]

New PR: ​https://github.com/django/django/pull/18037.

[Natalia Bidart]

Thank you Nina for working on this! I have unset the patch needs improvement flag so this is listed in the "needs review" list in the ​dashboard.

[Nina Menezes]

Documentation now updated.

[Claude Paroz]

Somewhat naive question: are there still use cases to exclude IDNA domain names from such a validator, 12 years after the initial report?

[Sarah Boyce]

Replying to Claude Paroz:

Somewhat naive question: are there still use cases to exclude IDNA domain names from such a validator, 12 years after the initial report?

Good question, perhaps it makes sense to remove the accept_idna argument and not support excluding IDNA domain names for now. Then if someone has a use case this can be raised in a new ticket?
Nina - what do you think?

[Sarah Boyce]

(open question as to whether we should drop accept_idna)

[michele]

The reason for adding accept_idna had nothing to do with how "modern" the format was. It's a way to allow the developer to enforce ascii names.

Although python beautifully supports UTF, the developer is often required to handle non-ascii characters specially. For example, when checking their length, or storing them somewhere, or serializing them to somewhere else. The exceeding rarity of IDNA causes many to assume the domain is ASCII, only to run into exception later in production.

If nothing else, the presence of the accept_idna option forces the developer to think whether that's a relevant case for them, and what to do about it.

I wouldn't necessarily consider adding it if the patch was without, but I certainly see value in keeping it now it's already there.

PS: I'm the original submitter. Trac wouldn't send me a reset-password link.

[Sarah Boyce]

Replying to michele:

I wouldn't necessarily consider adding it if the patch was without, but I certainly see value in keeping it now it's already there.

PS: I'm the original submitter. Trac wouldn't send me a reset-password link.

Thank you for your input. In which case, I am marking the patch as "Ready for checkin".

[Claude Paroz]

OK, let's go for it, at least the parameter default is True.

[Sarah Boyce <42296566+sarahboyce@…>]

In 4971a9a:

Fixed #18119 -- Added a DomainNameValidator validator.

Thanks Claude Paroz for the review.

Co-authored-by: Nina Menezes <77671865+nmenezes0@…>

[Sarah Boyce <42296566+sarahboyce@…>]

In 5405912:

Fixed #36007 -- Removed dead code from URLValidator.

The "Trivial case failed. Try for possible IDN domain" handling was
obsoleted by ticket-20003, which adjusted the regular expressions to
allow all international domain names (Refs #20003).

Uses of ul were moved to DomainNameValidator in ticket-18119
(Refs #18119).