Opened 88 minutes ago
#37235 new Bug
--debug-sql prints SQLParseError instead of falling back to unformatted SQL
| Reported by: | Jacob Walls | Owned by: | |
|---|---|---|---|
| Component: | Testing framework | Version: | 6.0 |
| Severity: | Release blocker | Keywords: | debug-sql |
| Cc: | Triage Stage: | Unreviewed | |
| Has patch: | no | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
The --debug-sql option in the test runner formats SQL with sqlparse. To fix uncontrolled resource consumption, version 0.5.5 raises SQLParseError instead of swallowing SQL with None.
I'm suggesting that we should catch the SQLParseError and fallback to the unformatted SQL. Right now, we're getting the traceback and its containing logging error spewed to the output. (Of course, this test and its neighbor could have used a lower limit, but at least that caught this issue!)
Happy to take backport advice: my first thought is at least 6.1, as it's a compatibility issue with an upstream dependency. I'd consider 6.0 as well.
./tests/runtests.py bulk_create.tests.BulkCreateTests.test_explicit_batch_size_respects_max_batch_size --debug-sql
Testing against Django installed in '/Users/jwalls/django/django' with up to 8 processes Found 1 test(s). Creating test database for alias 'default'... System check identified no issues (0 silenced). --- Logging error --- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/logging/__init__.py", line 1151, in emit msg = self.format(record) File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/logging/__init__.py", line 999, in format return fmt.format(record) ~~~~~~~~~~^^^^^^^^ File "/Users/jwalls/django/django/test/runner.py", line 53, in format record.args = (args[0], formatted_sql := format_sql(sql), *args[2:]) ~~~~~~~~~~^^^^^ File "/Users/jwalls/django/django/db/backends/base/operations.py", line 876, in format_debug_sql return sqlparse.format(sql, reindent=True, keyword_case="upper") ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jwalls/my314/lib/python3.14/site-packages/sqlparse/__init__.py", line 65, in format return "".join(stack.run(sql, encoding)) ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jwalls/my314/lib/python3.14/site-packages/sqlparse/engine/filter_stack.py", line 41, in run stmt = grouping.group(stmt) File "/Users/jwalls/my314/lib/python3.14/site-packages/sqlparse/engine/grouping.py", line 474, in group func(stmt) ~~~~^^^^^^ File "/Users/jwalls/my314/lib/python3.14/site-packages/sqlparse/engine/grouping.py", line 77, in group_brackets _group_matching(tlist, sql.SquareBrackets) ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jwalls/my314/lib/python3.14/site-packages/sqlparse/engine/grouping.py", line 37, in _group_matching raise SQLParseError( f"Maximum number of tokens exceeded ({MAX_GROUPING_TOKENS})." ) sqlparse.exceptions.SQLParseError: Maximum number of tokens exceeded (10000). Call stack: File "/Users/jwalls/django/./tests/runtests.py", line 786, in <module> failures = django_tests( File "/Users/jwalls/django/./tests/runtests.py", line 413, in django_tests failures = test_runner.run_tests(test_labels) File "/Users/jwalls/django/django/test/runner.py", line 1133, in run_tests result = self.run_suite(suite) File "/Users/jwalls/django/django/test/runner.py", line 1060, in run_suite return runner.run(suite) File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/runner.py", line 257, in run test(result) File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/suite.py", line 122, in run test(result) File "/Users/jwalls/django/django/test/testcases.py", line 314, in __call__ self._setup_and_call(result) File "/Users/jwalls/django/django/test/testcases.py", line 369, in _setup_and_call super().__call__(result) File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/case.py", line 725, in __call__ return self.run(*args, **kwds) File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/case.py", line 669, in run self._callTestMethod(testMethod) File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/case.py", line 615, in _callTestMethod result = method() File "/Users/jwalls/django/django/test/testcases.py", line 1591, in skip_wrapper return test_func(*args, **kwargs) File "/Users/jwalls/django/tests/bulk_create/tests.py", line 293, in test_explicit_batch_size_respects_max_batch_size Country.objects.bulk_create(objs, batch_size=max_batch_size + 1) File "/Users/jwalls/django/django/db/models/manager.py", line 87, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/Users/jwalls/django/django/db/models/query.py", line 899, in bulk_create returned_columns = self._batched_insert( File "/Users/jwalls/django/django/db/models/query.py", line 2178, in _batched_insert self._insert( File "/Users/jwalls/django/django/db/models/query.py", line 2140, in _insert return query.get_compiler(using=using).execute_sql(returning_fields) File "/Users/jwalls/django/django/db/models/sql/compiler.py", line 1936, in execute_sql cursor.execute(sql, params) File "/Users/jwalls/django/django/db/backends/utils.py", line 121, in execute with self.debug_sql(sql, params, use_last_executed_query=True): File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/contextlib.py", line 148, in __exit__ next(self.gen) File "/Users/jwalls/django/django/db/backends/utils.py", line 151, in debug_sql logger.debug( Message: '(%.3f) %s; args=%s; alias=%s' Arguments: (0.0017766660002962453, 'INSERT INTO "bulk_create_country" ("name", "iso_two_letter", "description") VALUES (\'Country 0\', \'\', \'\'), (\'Country 1\', \'\', \'\'), (\'Country 2\', \'\', \'\'), (\'Country 3\', \'\', \'\'), (\'Country 4\', \'\', \'\'), (\'Country 5\', \'\', \'\'), (\'Country 6\', \'\', ... truncated ... '', 'Country 984', '', '', 'Country 985', '', '', 'Country 986', '', '', 'Country 987', '', '', 'Country 988', '', '', 'Country 989', '', '', 'Country 990', '', '', 'Country 991', '', '', 'Country 992', '', '', 'Country 993', '', '', 'Country 994', '', '', 'Country 995', '', '', 'Country 996', '', '', 'Country 997', '', '', 'Country 998', '', '', 'Country 999', '', ''), 'default') . ---------------------------------------------------------------------- Ran 1 test in 0.081s OK