#23919 closed Cleanup/optimization (fixed)
Cleanups for when we drop Python 2 compatibility
Reported by: | Tim Graham | Owned by: | nobody |
---|---|---|---|
Component: | Core (Other) | Version: | |
Severity: | Normal | Keywords: | |
Cc: | cmawebsite@…, Tom Forbes | Triage Stage: | Accepted |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
This is a tracking ticket of things that we can remove when we drop Python 2 compatibility in Django 2.0. Please edit the description of the ticket as you come across new items.
#23905django.core.mail.message.make_msgid()
django.dispatch.weakref_backports
(remains for Python 3.4)django.http.cookie
workaroundsdjango.utils.2to3_fixes
django.utils.decorators.ContextDecorator
django.utils.encoding
@python_2_unicode_compatible
force/smart _unicode- either force/smart _text or force/smart _str PR
django.utils.html_parser.use_workaround
django.utils.http
functions likeurlquote_plus
-- I think the versions of these functions on Python 3 don't have the unicode characters bug we are working around PRStop usingdjango.utils.lru_cache
Figure the deprecation plan for(staying for now)django.utils.lru_cache
django.utils.six
from __future__ import unicode_literals
str()
stuff for environment variables, e.g. 0bfb53866199f366ed140d49938fd185e5898156str()
stuff fortype(name)
and__name__
Inheriting fromobject
indjango.core.servers.basehttp
(and perhaps other places) ala 4ee06ec3fc8e94d164afbd2f9c880c60c658a9acgit grep 'long int'
(mostly docs)django.utils._os [npath,upath]
In tests:contextlib.closing(self.urlopen
(contextlib.closing
no longer needed)Support for pysqlite (doesn't support Python 3)ReplacePRsuper(ClassName, self)
withsuper()
Remove(that's the default on Python 3)# -*- coding: utf-8 -*-
source file encoding- Evaluate replacement of custom
__del__
methods by weakref.finalize Remove__ne__
from objects already defining a__eq__
Remove note about PYTHONHASHSEED (see #26243)Remove Field.creation_counter (replace by metaclass(see comment:95)__prepare__
returningOrderedDict()
, https://docs.python.org/3/reference/datamodel.html#preparing-the-class-namespace)django.test.utils.reset_warning_registry()
ReplacePRerrno
checks byIOError
subclasses defined by PEP 3151django.utils.glob
(remains for Python 3.4 compatibility)django.utils.cookies.SimpleCookie
django.test.mock
Replacetempfile.mkdtemp
+ remove withtempfile.TemporaryDirectory
context managerReplaceio.open()
by a plainopen()
Evaluate need forassertRegex
,assertRaisesRegex
in tests. Some usage is merely to account for differences in messages between Python 2 and 3.- PR 7879re.U
,re.UNICODE
(default behavior of Python 3)
Change History (148)
comment:1 Changed 9 years ago by
comment:2 Changed 9 years ago by
Description: | modified (diff) |
---|
comment:3 Changed 9 years ago by
Description: | modified (diff) |
---|
comment:4 follow-up: 32 Changed 9 years ago by
I believe we can get rid of the Field.creation_counter hack. In Python 3 it is possible to store the attrs in a sorted dictionary. See http://stackoverflow.com/questions/4459531/how-to-read-class-attributes-in-the-same-order-as-declared.
comment:5 Changed 9 years ago by
Description: | modified (diff) |
---|
comment:6 Changed 9 years ago by
Description: | modified (diff) |
---|
comment:7 Changed 9 years ago by
Description: | modified (diff) |
---|
comment:8 Changed 9 years ago by
Cc: | cmawebsite@… added |
---|---|
Description: | modified (diff) |
comment:9 Changed 9 years ago by
Description: | modified (diff) |
---|
comment:10 Changed 9 years ago by
In far too many tests we rely on the fact six aliases builtins [like input] instead of using mock.
I submitted a patch recently to clean up a couple of these I found, but I am finding more.
comment:11 Changed 9 years ago by
Description: | modified (diff) |
---|
comment:12 Changed 8 years ago by
Description: | modified (diff) |
---|
comment:13 Changed 8 years ago by
Description: | modified (diff) |
---|
comment:14 Changed 8 years ago by
Description: | modified (diff) |
---|
comment:15 Changed 8 years ago by
Description: | modified (diff) |
---|
comment:16 Changed 8 years ago by
Description: | modified (diff) |
---|
comment:17 Changed 8 years ago by
Description: | modified (diff) |
---|
comment:18 Changed 8 years ago by
Description: | modified (diff) |
---|
comment:19 Changed 8 years ago by
Description: | modified (diff) |
---|
comment:20 Changed 8 years ago by
Description: | modified (diff) |
---|
comment:21 Changed 8 years ago by
Description: | modified (diff) |
---|
comment:22 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:23 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:24 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:25 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:26 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:27 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:28 Changed 7 years ago by
Note that I have a local branch which does some boring removals (unicode_literals, utf-8 preambles, some six usage). So ping me as soon as the Django 2.0 deprecated stuff is removed and I'll offer a patch for that.
comment:29 Changed 7 years ago by
I am also working to remove python2 workarounds by following a module by module approach.
comment:30 follow-up: 31 Changed 7 years ago by
If Claude has already prepared these changes, perhaps you should wait for his patch instead of duplicating his work?
comment:31 Changed 7 years ago by
Replying to Aymeric Augustin:
If Claude has already prepared these changes, perhaps you should wait for his patch instead of duplicating his work?
Yes I talked with Claude on IRC to pause before his work is merged.
comment:32 Changed 7 years ago by
Replying to Anssi Kääriäinen:
I believe we can get rid of the Field.creation_counter hack. In Python 3 it is possible to store the attrs in a sorted dictionary. See http://stackoverflow.com/questions/4459531/how-to-read-class-attributes-in-the-same-order-as-declared.
We could also remove this when we support python 3.6, as class attributes retain their order. I'm guessing this is a consequence (or the cause of) dictionaries retaining insertion order. https://docs.python.org/3.6/whatsnew/3.6.html#pep-520-preserving-class-attribute-definition-order
comment:35 Changed 7 years ago by
Triage Stage: | Someday/Maybe → Accepted |
---|
PR for removing Python 2 references from docs.
comment:40 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:41 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:42 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:46 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:47 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:49 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:55 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:63 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:64 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:69 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:75 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:77 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:79 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:87 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:91 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:92 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:95 Changed 7 years ago by
I had an attempt at dropping the creation_counter for model fields, but I *think* it might be more costly than the current method, considering so many things rely on the order being consistent. My attempt was here, with associated discussion I'll copy some of below: https://github.com/django/django/pull/7983
I'm not sure this is worth continuing with, at least not in the same way. Too many systems rely on the fields being ordered, which isn't going to be possible without maintaining another list of fields such as definition_order = []. This can be done for local fields, but then proxy/parent fields will need to be returned in the same definition order. This might be something like:
parent_fields + [field for field in self.definition_order if field in local_fields]
Then we'd need to account for private and hidden fields. But what's the point? We'd now be storing a separate list of all field references for each model, rather than a counter (int) on each field, and reordering fields in _get_fields() whether or not they need to be ordered. Of course I'd be happy to see a better solution that worked, I'm just giving up on this particular method for the moment.
comment:96 Changed 7 years ago by
Description: | modified (diff) |
---|
I also looked briefly at models field and manager usage of creation_counter
but it wasn't obvious if the __prepare__()
approach was feasible or if it would be simpler. I think we can defer that task for now.
comment:101 Changed 7 years ago by
Description: | modified (diff) |
---|
Can we remove empty init.py (and init.py-tpl) files?
comment:102 Changed 7 years ago by
IIRC there's a risk to making everything a namespace package: it means that if you have conflicting package / module names, they're somehow merged instead of having the first one shadow the other. (Perhaps someone more familiar with the topic can describe this more precisely.) I think it's safer to keep the __init__
files.
comment:103 Changed 7 years ago by
Given that Django 2.0 is targeting Python 3.5+, are there other things that could be cleaned up that were added to support <3.5 or could be used now that 3.5 is the minimum version?
For example, there are many places with the following pattern:
try: ... except ...: pass
These could be replaced with contextlib.suppress() which is available since 3.4.
Other potentially interesting things are:
- Use of unpacking generalisations
- Use of type hints and a type checker when building.
- Use of os.scandir() instead of os.listdir().
- Use of the enum and pathlib modules where appropriate.
- contextlib.redirect_stdout() and contextlib.redirect_stderr()
There may be more - this was just a quick look at the What's New pages for 3.4 and 3.5.
comment:104 Changed 7 years ago by
Usage of Python 3 features could be proposed in new tickets (e.g. #27804). Master is still supporting Python 3.4 right now as per django-developers discussion.
comment:108 Changed 7 years ago by
Description: | modified (diff) |
---|
Should we deprecate cache.has_key()
?
comment:109 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:110 Changed 7 years ago by
The "Python 3 is recommended" text here could probably be rephrased:
https://docs.djangoproject.com/en/dev/faq/install/#what-python-version-should-i-use-with-django
comment:123 Changed 6 years ago by
I was looking at potential cleanups I could help with and noticed a lot of os.path
functions that could be replaced with pathlib
- we have 219 occurrences of os.path
(excluding usages in tests).
Is this something worth spending time on? It's pretty easy to convert, and it makes path related code a lot more readable IMO (no need for nested, multiple os.path.* calls in a single line for example). In 3.5 and above there is also a read_text
method on paths which is quite useful in some cases.
comment:124 Changed 6 years ago by
Sure, start small and we can see if the changes look like an improvement.
comment:126 Changed 6 years ago by
Cc: | Tom Forbes added |
---|
comment:136 Changed 6 years ago by
Resolution: | → fixed |
---|---|
Status: | new → closed |
Various
nonlocal
workarounds (example).