#23919 closed Cleanup/optimization (fixed)
Cleanups for when we drop Python 2 compatibility
Reported by: | Tim Graham | Owned by: | nobody |
---|---|---|---|
Component: | Core (Other) | Version: | |
Severity: | Normal | Keywords: | |
Cc: | cmawebsite@…, Tom Forbes | Triage Stage: | Accepted |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
This is a tracking ticket of things that we can remove when we drop Python 2 compatibility in Django 2.0. Please edit the description of the ticket as you come across new items.
#23905django.core.mail.message.make_msgid()
django.dispatch.weakref_backports
(remains for Python 3.4)django.http.cookie
workaroundsdjango.utils.2to3_fixes
django.utils.decorators.ContextDecorator
django.utils.encoding
@python_2_unicode_compatible
force/smart _unicode- either force/smart _text or force/smart _str PR
django.utils.html_parser.use_workaround
django.utils.http
functions likeurlquote_plus
-- I think the versions of these functions on Python 3 don't have the unicode characters bug we are working around PRStop usingdjango.utils.lru_cache
Figure the deprecation plan for(staying for now)django.utils.lru_cache
django.utils.six
from __future__ import unicode_literals
str()
stuff for environment variables, e.g. 0bfb53866199f366ed140d49938fd185e5898156str()
stuff fortype(name)
and__name__
Inheriting fromobject
indjango.core.servers.basehttp
(and perhaps other places) ala 4ee06ec3fc8e94d164afbd2f9c880c60c658a9acgit grep 'long int'
(mostly docs)django.utils._os [npath,upath]
In tests:contextlib.closing(self.urlopen
(contextlib.closing
no longer needed)Support for pysqlite (doesn't support Python 3)ReplacePRsuper(ClassName, self)
withsuper()
Remove(that's the default on Python 3)# -*- coding: utf-8 -*-
source file encoding- Evaluate replacement of custom
__del__
methods by weakref.finalize Remove__ne__
from objects already defining a__eq__
Remove note about PYTHONHASHSEED (see #26243)Remove Field.creation_counter (replace by metaclass(see comment:95)__prepare__
returningOrderedDict()
, https://docs.python.org/3/reference/datamodel.html#preparing-the-class-namespace)django.test.utils.reset_warning_registry()
ReplacePRerrno
checks byIOError
subclasses defined by PEP 3151django.utils.glob
(remains for Python 3.4 compatibility)django.utils.cookies.SimpleCookie
django.test.mock
Replacetempfile.mkdtemp
+ remove withtempfile.TemporaryDirectory
context managerReplaceio.open()
by a plainopen()
Evaluate need forassertRegex
,assertRaisesRegex
in tests. Some usage is merely to account for differences in messages between Python 2 and 3.- PR 7879re.U
,re.UNICODE
(default behavior of Python 3)
Change History (148)
comment:1 by , 10 years ago
comment:2 by , 10 years ago
Description: | modified (diff) |
---|
comment:3 by , 10 years ago
Description: | modified (diff) |
---|
follow-up: 32 comment:4 by , 10 years ago
I believe we can get rid of the Field.creation_counter hack. In Python 3 it is possible to store the attrs in a sorted dictionary. See http://stackoverflow.com/questions/4459531/how-to-read-class-attributes-in-the-same-order-as-declared.
comment:5 by , 10 years ago
Description: | modified (diff) |
---|
comment:6 by , 10 years ago
Description: | modified (diff) |
---|
comment:7 by , 10 years ago
Description: | modified (diff) |
---|
comment:8 by , 10 years ago
Cc: | added |
---|---|
Description: | modified (diff) |
comment:9 by , 10 years ago
Description: | modified (diff) |
---|
comment:10 by , 10 years ago
In far too many tests we rely on the fact six aliases builtins [like input] instead of using mock.
I submitted a patch recently to clean up a couple of these I found, but I am finding more.
comment:11 by , 10 years ago
Description: | modified (diff) |
---|
comment:12 by , 9 years ago
Description: | modified (diff) |
---|
comment:13 by , 9 years ago
Description: | modified (diff) |
---|
comment:14 by , 9 years ago
Description: | modified (diff) |
---|
comment:15 by , 9 years ago
Description: | modified (diff) |
---|
comment:16 by , 9 years ago
Description: | modified (diff) |
---|
comment:17 by , 9 years ago
Description: | modified (diff) |
---|
comment:18 by , 9 years ago
Description: | modified (diff) |
---|
comment:19 by , 9 years ago
Description: | modified (diff) |
---|
comment:20 by , 8 years ago
Description: | modified (diff) |
---|
comment:21 by , 8 years ago
Description: | modified (diff) |
---|
comment:22 by , 8 years ago
Description: | modified (diff) |
---|
comment:23 by , 8 years ago
Description: | modified (diff) |
---|
comment:24 by , 8 years ago
Description: | modified (diff) |
---|
comment:25 by , 8 years ago
Description: | modified (diff) |
---|
comment:26 by , 8 years ago
Description: | modified (diff) |
---|
comment:27 by , 8 years ago
Description: | modified (diff) |
---|
comment:28 by , 8 years ago
Note that I have a local branch which does some boring removals (unicode_literals, utf-8 preambles, some six usage). So ping me as soon as the Django 2.0 deprecated stuff is removed and I'll offer a patch for that.
comment:29 by , 8 years ago
I am also working to remove python2 workarounds by following a module by module approach.
follow-up: 31 comment:30 by , 8 years ago
If Claude has already prepared these changes, perhaps you should wait for his patch instead of duplicating his work?
comment:31 by , 8 years ago
Replying to Aymeric Augustin:
If Claude has already prepared these changes, perhaps you should wait for his patch instead of duplicating his work?
Yes I talked with Claude on IRC to pause before his work is merged.
comment:32 by , 8 years ago
Replying to Anssi Kääriäinen:
I believe we can get rid of the Field.creation_counter hack. In Python 3 it is possible to store the attrs in a sorted dictionary. See http://stackoverflow.com/questions/4459531/how-to-read-class-attributes-in-the-same-order-as-declared.
We could also remove this when we support python 3.6, as class attributes retain their order. I'm guessing this is a consequence (or the cause of) dictionaries retaining insertion order. https://docs.python.org/3.6/whatsnew/3.6.html#pep-520-preserving-class-attribute-definition-order
comment:35 by , 8 years ago
Triage Stage: | Someday/Maybe → Accepted |
---|
PR for removing Python 2 references from docs.
comment:40 by , 8 years ago
Description: | modified (diff) |
---|
comment:41 by , 8 years ago
Description: | modified (diff) |
---|
comment:42 by , 8 years ago
Description: | modified (diff) |
---|
comment:46 by , 8 years ago
Description: | modified (diff) |
---|
comment:47 by , 8 years ago
Description: | modified (diff) |
---|
comment:49 by , 8 years ago
Description: | modified (diff) |
---|
comment:55 by , 8 years ago
Description: | modified (diff) |
---|
comment:63 by , 8 years ago
Description: | modified (diff) |
---|
comment:64 by , 8 years ago
Description: | modified (diff) |
---|
comment:69 by , 8 years ago
Description: | modified (diff) |
---|
comment:75 by , 8 years ago
Description: | modified (diff) |
---|
comment:77 by , 8 years ago
Description: | modified (diff) |
---|
comment:79 by , 8 years ago
Description: | modified (diff) |
---|
comment:87 by , 8 years ago
Description: | modified (diff) |
---|
comment:91 by , 8 years ago
Description: | modified (diff) |
---|
comment:92 by , 8 years ago
Description: | modified (diff) |
---|
comment:95 by , 8 years ago
I had an attempt at dropping the creation_counter for model fields, but I *think* it might be more costly than the current method, considering so many things rely on the order being consistent. My attempt was here, with associated discussion I'll copy some of below: https://github.com/django/django/pull/7983
I'm not sure this is worth continuing with, at least not in the same way. Too many systems rely on the fields being ordered, which isn't going to be possible without maintaining another list of fields such as definition_order = []. This can be done for local fields, but then proxy/parent fields will need to be returned in the same definition order. This might be something like:
parent_fields + [field for field in self.definition_order if field in local_fields]
Then we'd need to account for private and hidden fields. But what's the point? We'd now be storing a separate list of all field references for each model, rather than a counter (int) on each field, and reordering fields in _get_fields() whether or not they need to be ordered. Of course I'd be happy to see a better solution that worked, I'm just giving up on this particular method for the moment.
comment:96 by , 8 years ago
Description: | modified (diff) |
---|
I also looked briefly at models field and manager usage of creation_counter
but it wasn't obvious if the __prepare__()
approach was feasible or if it would be simpler. I think we can defer that task for now.
comment:101 by , 8 years ago
Description: | modified (diff) |
---|
Can we remove empty init.py (and init.py-tpl) files?
comment:102 by , 8 years ago
IIRC there's a risk to making everything a namespace package: it means that if you have conflicting package / module names, they're somehow merged instead of having the first one shadow the other. (Perhaps someone more familiar with the topic can describe this more precisely.) I think it's safer to keep the __init__
files.
comment:103 by , 8 years ago
Given that Django 2.0 is targeting Python 3.5+, are there other things that could be cleaned up that were added to support <3.5 or could be used now that 3.5 is the minimum version?
For example, there are many places with the following pattern:
try: ... except ...: pass
These could be replaced with contextlib.suppress() which is available since 3.4.
Other potentially interesting things are:
- Use of unpacking generalisations
- Use of type hints and a type checker when building.
- Use of os.scandir() instead of os.listdir().
- Use of the enum and pathlib modules where appropriate.
- contextlib.redirect_stdout() and contextlib.redirect_stderr()
There may be more - this was just a quick look at the What's New pages for 3.4 and 3.5.
comment:104 by , 8 years ago
Usage of Python 3 features could be proposed in new tickets (e.g. #27804). Master is still supporting Python 3.4 right now as per django-developers discussion.
comment:108 by , 8 years ago
Description: | modified (diff) |
---|
Should we deprecate cache.has_key()
?
comment:109 by , 8 years ago
Description: | modified (diff) |
---|
comment:110 by , 8 years ago
The "Python 3 is recommended" text here could probably be rephrased:
https://docs.djangoproject.com/en/dev/faq/install/#what-python-version-should-i-use-with-django
comment:123 by , 7 years ago
I was looking at potential cleanups I could help with and noticed a lot of os.path
functions that could be replaced with pathlib
- we have 219 occurrences of os.path
(excluding usages in tests).
Is this something worth spending time on? It's pretty easy to convert, and it makes path related code a lot more readable IMO (no need for nested, multiple os.path.* calls in a single line for example). In 3.5 and above there is also a read_text
method on paths which is quite useful in some cases.
comment:124 by , 7 years ago
Sure, start small and we can see if the changes look like an improvement.
comment:126 by , 7 years ago
Cc: | added |
---|
comment:136 by , 7 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
Various
nonlocal
workarounds (example).