id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
33002	Make DebugLexer.tokenize() more similar to Lexer.tokenize()	Chris Jerdonek	Chris Jerdonek	"Currently, [https://github.com/django/django/blob/8208381ba6a3d1613bb746617062ccf1a6a28591/django/template/base.py#L396-L418 DebugLexer.tokenize()] looks somewhat dissimilar to [https://github.com/django/django/blob/8208381ba6a3d1613bb746617062ccf1a6a28591/django/template/base.py#L352-L364 Lexer.tokenize()], even though they do more or less the same thing. For example, the `Lexer` class's implementation contains just one call to `self.create_token()`, whereas `DebugLexer` contains three calls to it.

This ticket is to make `DebugLexer.tokenize()` look visibly much more the same as `Lexer.tokenize()`, and in particular simpler. One advantage is that it will be more obvious on inspection that `DebugLexer` acts the same as `Lexer`. Another advantage is that it will be easier to keep the two implementations in step when applying optimizations to `Lexer`. It will be more maintainable and less likely to introduce bugs if the implementations for the two lexers don't diverge too much.

The idea will become more clear once the PR is posted.
"	Cleanup/optimization	closed	Template system	dev	Normal	fixed			Ready for checkin	1	0	0	0	0	0