Opened 4 years ago

Closed 4 years ago

#20989 closed Cleanup/optimization (fixed)

Remove useless list comprehensions

Reported by: jeroen.pulles@… Owned by: nobody
Component: Uncategorized Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

As I understand it, dict() and tuple() accept iterators, e.g.

l = [1, 2, 3]
tuple(v for v in l)

There are various calls to tuple() that place the iterator in a list comprehension before feeding the list to tuple(). For example in django/db/models/sql/query.py:

aliases = tuple([change_map.get(a, a) for a in aliases])

can be written as:

aliases = tuple(change_map.get(a, a) for a in aliases)

saving two characters, an extra iteration and list creation. The same goes for dict(), where dict([(v, v) for v in l]) can be rewritten as dict((v, v) for v in l).


A shell command like this can replace the calls. When I tried it, I got lots of errors in the unit tests, but my impression was that that didn't have anything to do with the edit...

find django/ -name '*.py' -print0 \
    | xargs -n 200 -0 grep -l 'tuple([[][^]]\+[]])' \
    | while read F
do 
    cp "$F" "$F.orig"
    sed -e 's/tuple([[]\(.*\)[]])/tuple(\1)/' "$F.orig" > "$F"
done

Attachments (1)

0001-Fixed-20989-Removed-useless-explicit-list-comprehens.patch (79.5 KB) - added by Simon Charette 4 years ago.

Download all attachments as: .zip

Change History (8)

comment:1 Changed 4 years ago by Marc Tamlyn

Triage Stage: UnreviewedAccepted
Type: UncategorizedCleanup/optimization

comment:2 Changed 4 years ago by Tim Graham <timograham@…>

Resolution: fixed
Status: newclosed

In c7d0ff0cad24dbf444f2e4b534377c3352c7aa98:

Fixed #20989 -- Removed explicit list comprehension inside dict() and tuple()

Thanks jeroen.pulles at redslider.net for the suggestion and
helper script.

comment:3 Changed 4 years ago by Simon Charette

I think the provided regex missed a few ones:

(django)simon@simon-laptop:~/workspace/django$ grep "dict(\[" django/ -R
django/db/backends/oracle/introspection.py:        return dict([(d[0], i) for i, d in enumerate(self.get_table_description(cursor, table_name))])
django/db/backends/mysql/introspection.py:        numeric_map = dict([(line[0], tuple([int(n) for n in line[1:]])) for line in cursor.fetchall()])
django/db/backends/mysql/introspection.py:        return dict([(d[0], i) for i, d in enumerate(self.get_table_description(cursor, table_name))])
django/db/models/sql/query.py:        return dict([
django/template/loader_tags.py:                    blocks = dict([(n.name, n) for n in
django/template/loader_tags.py:        values = dict([(name, var.resolve(context)) for name, var
django/template/defaulttags.py:        kwargs = dict([(smart_text(k, 'ascii'), v.resolve(context))
django/template/defaulttags.py:        values = dict([(key, val.resolve(context)) for key, val in
django/utils/html.py:    kwargs_safe = dict([(k, conditional_escape(v)) for (k, v) in
django/utils/termcolors.py:foreground = dict([(color_names[x], '3%s' % x) for x in range(8)])
django/utils/termcolors.py:background = dict([(color_names[x], '4%s' % x) for x in range(8)])
django/utils/cache.py:    cc = dict([_to_tuple(el) for el in
django/contrib/admin/filters.py:        self.date_params = dict([(k, v) for k, v in params.items()
django/contrib/messages/storage/cookie.py:            return dict([(key, self.process_messages(value))
django/core/management/__init__.py:                _commands.update(dict([(name, app_name)
(django)simon@simon-laptop:~/workspace/django$ grep "tuple(\[" django/ -R
django/db/backends/oracle/base.py:        return tuple([_rowfactory(r, self.cursor)
django/db/backends/oracle/base.py:        return tuple([_rowfactory(r, self.cursor)
django/db/backends/mysql/introspection.py:        numeric_map = dict([(line[0], tuple([int(n) for n in line[1:]])) for line in cursor.fetchall()])
django/db/models/query.py:                yield tuple([data[f] for f in fields])
django/db/models/sql/compiler.py:                    row = tuple(row[:aggregate_start]) + tuple([
django/template/base.py:        return self.msg % tuple([force_text(p, errors='replace')
django/contrib/gis/gdal/geometries.py:        return tuple([self[i] for i in xrange(len(self))])
django/contrib/gis/gdal/geometries.py:        return tuple([self[i].tuple for i in xrange(self.geom_count)])
django/contrib/gis/gdal/geometries.py:        return tuple([self[i].tuple for i in xrange(self.geom_count)])
django/contrib/gis/geos/coordseq.py:        else: return tuple([self[i] for i in xrange(n)])
django/contrib/gis/geos/polygon.py:        return tuple([self[i].tuple for i in xrange(len(self))])

comment:4 Changed 4 years ago by Tim Graham

Resolution: fixed
Status: closednew

comment:5 Changed 4 years ago by Simon Charette

Summary: tuple() and dict() accept iteratorsRemove useless list comprehensions

Added a patch which remove the missed ones and also consider the following cases:

  1. set and set.update
  2. OrderedDict
  3. list.extend
  4. join
  5. min,max,sum and any
  6. sorted

All tests pass on py2/py3 SQlite and Postgres.

comment:6 Changed 4 years ago by Claude Paroz

Has patch: set
Triage Stage: AcceptedReady for checkin

As far as tests are green, I'd say go ahead.

comment:7 Changed 4 years ago by Simon Charette <charette.s@…>

Resolution: fixed
Status: newclosed

In 11cd7388f77aa9d12ab6b57285c3801b237e241b:

Fixed #20989 -- Removed useless explicit list comprehensions.

Note: See TracTickets for help on using tickets.
Back to Top