Version 9 (modified by trac, 10 years ago) ( diff )

--

Trac Ticket Queries

Table of Contents

In addition to reports, Trac provides support for custom ticket queries, which can be used to display tickets that meet specified criteria.

To configure and execute a custom query, switch to the View Tickets module from the navigation bar, and select the Custom Query link.

Filters

When you first go to the query page, the default filter will display tickets relevant to you:

  • If logged in then all open tickets, it will display open tickets assigned to you.
  • If not logged in but you have specified a name or email address in the preferences, then it will display all open tickets where your email (or name if email not defined) is in the CC list.
  • If not logged in and no name/email is defined in the preferences, then all open issues are displayed.

Current filters can be removed by clicking the button to the left with the minus sign on the label. New filters are added from the pulldown lists at the bottom corners of the filters box; 'And' conditions on the left, 'Or' conditions on the right. Filters with either a text box or a pulldown menu of options can be added multiple times to perform an Or on the criteria.

You can use the fields just below the filters box to group the results based on a field, or display the full description for each ticket.

After you have edited your filters, click the Update button to refresh your results.

Some shortcuts can be used to manipulate checkbox filters.

  • Clicking on a filter row label toggles all checkboxes.
  • Pressing the modifier key while clicking on a filter row label inverts the state of all checkboxes.
  • Pressing the modifier key while clicking on a checkbox selects the checkbox and deselects all other checkboxes in the filter.

The modifier key is platform and browser dependent. On Mac the modified key is Option/Alt or Command. On Linux the modifier key is Ctrl + Alt. Opera on Windows seems to use Ctrl + Alt, while Alt is effective for other Windows browsers.

Clicking on one of the query results will take you to that ticket. You can navigate through the results by clicking the Next Ticket or Previous Ticket links just below the main menu bar, or click the Back to Query link to return to the query page.

You can safely edit any of the tickets and continue to navigate through the results using the Next/Previous/Back to Query links after saving your results. When you return to the query any tickets which were edited will be displayed with italicized text. If one of the tickets was edited such that it no longer matches the query criteria , the text will also be greyed. Lastly, if a new ticket matching the query criteria has been created, it will be shown in bold.

The query results can be refreshed and cleared of these status indicators by clicking the Update button again.

Saving Queries

Trac allows you to save the query as a named query accessible from the reports module. To save a query ensure that you have Updated the view and then click the Save query button displayed beneath the results. You can also save references to queries in Wiki content, as described below.

Note: one way to easily build queries like the ones below, you can build and test the queries in the Custom report module and when ready - click Save query. This will build the query string for you. All you need to do is remove the extra line breaks.

Note: you must have the REPORT_CREATE permission in order to save queries to the list of default reports. The Save query button will only appear if you are logged in as a user that has been granted this permission. If your account does not have permission to create reports, you can still use the methods below to save a query.

You may want to save some queries so that you can come back to them later. You can do this by making a link to the query from any Wiki page. 

[query:status=new|assigned|reopened&version=1.0 Active tickets against 1.0]

Which is displayed as:

Active tickets against 1.0

This uses a very simple query language to specify the criteria, see Query Language.

Alternatively, you can copy the query string of a query and paste that into the Wiki link, including the leading ? character: 

[query:?status=new&status=assigned&status=reopened&group=owner Assigned tickets by owner]

Which is displayed as:

Assigned tickets by owner

Customizing the table format

You can also customize the columns displayed in the table format (format=table) by using col=<field>. You can specify multiple fields and what order they are displayed in by placing pipes (|) between the columns: 

[[TicketQuery(max=3,status=closed,order=id,desc=1,format=table,col=resolution|summary|owner|reporter)]]

This is displayed as:

Full rows

In table format you can also have full rows by using rows=<field>: 

[[TicketQuery(max=3,status=closed,order=id,desc=1,format=table,col=resolution|summary|owner|reporter,rows=description)]]

This is displayed as:

Results (1 - 3 of 35388)

1 2 3 4 5 6 7 8 9 10 11 →
Ticket Resolution Summary Owner Reporter
#37223 needsnewfeatureprocess Signing's JSONSerializer uses latin-1, causing silent mojibake with UTF-8-emitting serializers Paolo Melchiorre Paolo Melchiorre
Description

Summary

django.core.signing.JSONSerializer encodes its JSON payload with latin-1 and decodes it with latin-1. With the default ensure_ascii=True serializer the output is pure ASCII, so latin-1 and UTF-8 are indistinguishable and the bug is invisible. But any serializer that emits raw UTF-8 (orjson, msgspec, json.dumps(..., ensure_ascii=False)) produces non-ASCII bytes for non-ASCII data, and JSONSerializer.loads silently mojibakes them when decoding as latin-1. No BadSignature and no UnicodeDecodeError is raised — the data is returned corrupted as if it were valid.

Both SESSION_SERIALIZER and signing.dumps(serializer=...) are documented extension points. The scenario that bites is the rollback: a project runs a UTF-8-emitting serializer (e.g. OrjsonSerializer for SESSION_SERIALIZER), then switches back to the default — every token signed while the UTF-8 serializer was active comes back mojibaked, with no BadSignature and no UnicodeDecodeError. A staggered deploy across mixed patched/unpatched processes is the same class of risk, just less common. The django-orjson maintainer documented this as a one-way migration limitation in ​django-orjson PR #15 (warning boxes in docs/sessions.rst and docs/signing.rst, plus xfail tests for the cross-serializer case), rather than fixing it — the fix has to live in Django. This is a maintainer-documented limitation, not a user-reported data loss incident.

How to reproduce

JSONSerializer currently does: 

# django/core/signing.py
class JSONSerializer:
    def dumps(self, obj):
        return json.dumps(obj, separators=(",", ":")).encode("latin-1")

    def loads(self, data):
        return json.loads(data.decode("latin-1"))

A payload containing héllo, written by a UTF-8-emitting serializer as the bytes h\xc3\xa9llo, is decoded by loads as latin-1 into héllo. The HMAC is over the bytes (which are intact), so the signature still verifies and the corrupted value is returned to the caller as if it were valid. A minimal reproduction: 

from django.core.signing import JSONSerializer

# What a UTF-8-emitting serializer (e.g. orjson) would produce for {"text": "héllo"}
external_bytes = b'{"text":"h\xc3\xa9llo"}'

# Django's loads mojibakes it
assert JSONSerializer().loads(external_bytes) == {"text": "héllo"}  # silently corrupted
assert JSONSerializer().loads(external_bytes) != {"text": "héllo"}  # the real value is lost

Diagnosis

The latin-1 step was introduced in commit [58a086acfb] (Oct 2012, "Required serializer to use bytes in loads/dumps") as part of the Python 3 bytes-interface porting, chosen for consistency with the bytes interface rather than because latin-1 was required. With the default ensure_ascii=True serializer the output is always pure ASCII, so latin-1 and UTF-8 produce identical bytes and the bug is invisible — which is why it has gone unfixed since 2012. It surfaces the moment any non-default serializer is used.

The bug was surfaced while analyzing the pluggable JSON backend proposal (​#187), which had to carve core/signing.py out of the swappable backend because of this latin-1 step.

Scope

Only django/core/signing.py is in scope. Two contrib consumers are transitively covered because they use signing through the public API:

  • django/contrib/sessions/serializers.py — JSONSerializer is a re-export of signing.JSONSerializer.
  • django/contrib/sessions/backends/base.py and django/contrib/sessions/backends/signed_cookies.py — use signing.dumps / signing.loads.

django/contrib/messages/storage/cookie.py is not covered by this fix: it has its own MessagePartGatherSerializer.dumps and MessageSerializer.loads that call json.dumps(...).encode("latin-1") and json.loads(data.decode("latin-1")) directly (lines 68 and 73), independent of signing.JSONSerializer. That is the same bug pattern on a separate code path. The two are intentionally kept as separate tickets and separate PRs: they touch different modules, share no code path, and can land and be reverted independently. Keeping them separate also keeps each diff minimal and the review focused on one contract at a time. django/contrib/admin does not use signing directly.

Proposed direction

The natural fix is to switch the encode/decode step from latin-1 to utf-8 in both dumps and loads, leaving the separators=(",", ":") argument untouched so determinism is preserved. The exact shape of the change is left to the pull request.

Backwards compatibility

Fully backward-compatible. Existing signed tokens contain only ASCII bytes (ensure_ascii=True escapes every non-ASCII character to \uXXXX), and ASCII decodes identically under latin-1 and UTF-8, so every token already in the wild verifies and decodes to the same value. The compress=True path is covered by the same argument: existing compressed tokens are ASCII.

New tokens with non-ASCII data will contain raw UTF-8 bytes instead of \uXXXX escapes, so they are slightly shorter and round-trip correctly through the new loads.

Suggested test coverage

Tests should be added to the existing signing test module (tests/signing/tests.py) and should cover:

  • a regression test that feeds raw UTF-8 bytes (as a UTF-8-emitting external serializer would produce — not bytes JSONSerializer.dumps could ever produce, since ensure_ascii=True escapes everything to ASCII) to loads and asserts the original value is returned, not the latin-1 mojibake — this is the test that fails before the fix and passes after;
  • a round-trip of a non-ASCII payload (including combining marks and non-BMP characters) through dumps/loads.

The dumps encode step is not independently testable with the current JSONSerializer (ensure_ascii=True makes its output always ASCII), so no dumps-side regression test is warranted; that change is forward-looking and kept for symmetry with loads.

  • ​PR #21651 — Pull request implementing this fix.
  • ​new-features #187 — Pluggable JSON serialization/deserialization backend (this fix removes the signing carve-out from step 1).
  • ​django-orjson PR #15 — The django-orjson maintainer documented this as a one-way migration limitation (warning boxes + xfail tests for the cross-serializer case), not a user-reported data loss incident. Authoritative source for the mojibake behavior described here.
  • Django's contrib/sessions/backends/signed_cookies.py — uses signing.dumps(serializer=self.serializer) where self.serializer is SESSION_SERIALIZER; sessions routinely contain non-ASCII data (user names, locale), so the mixing scenario is reachable from Django core, not only from third-party packages.
  • Commit [58a086acfb] — introduced the latin-1 encode/decode step (Oct 2012).
#37219 fixed Add GeneratedField example to PostgreSQL search reference. Carlton Gibson Carlton Gibson
Description

I was reading the ​Performance section of the Postgres Full text search reference.

The requirement to use a trigger with SearchVectorField made me think of using a GeneratedField. I think that would be a good example to add.

#37218 fixed Document that lazy relation access is synchronous only Jacob Walls Jacob Walls
Description

See ​fiddle: 

from asgiref.sync import async_to_sync
from django.contrib.auth.models import User
from django.db import models


class Person(models.Model):
    name = models.CharField(max_length=100)
    user = models.ForeignKey(User, models.CASCADE)
    

@async_to_sync
async def async_view():
    qs = Person.objects.fetch_mode(models.FETCH_PEERS)
    async for el in qs:
        print(el.user)


def run():
    admin = User.objects.create(username="admin")
    instance = Person.objects.create(name='John Doe', user=admin)
    instance2 = Person.objects.create(name='Jane Doe', user=admin)
    async_view()


  File "/app/app/models.py", line 15, in async_view
    print(el.user)
          ^^^^^^^
  File "/django-pr/django/db/models/fields/related_descriptors.py", line 266, in __get__
    instance._state.fetch_mode.fetch(self, instance)
  File "/django-pr/django/db/models/fetch_modes.py", line 38, in fetch
    fetcher.fetch_many(instances)
  File "/django-pr/django/db/models/fields/related_descriptors.py", line 291, in fetch_many
    prefetch_related_objects(missing_instances, self.field.name)
  File "/django-pr/django/db/models/query.py", line 2701, in prefetch_related_objects
    obj_list, additional_lookups = prefetch_one_level(
                                   ^^^^^^^^^^^^^^^^^^^
  File "/django-pr/django/db/models/query.py", line 2876, in prefetch_one_level
    all_related_objects = list(rel_qs)
                          ^^^^^^^^^^^^
  File "/django-pr/django/db/models/query.py", line 434, in __iter__
    self._fetch_all()
  File "/django-pr/django/db/models/query.py", line 2239, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/django-pr/django/db/models/query.py", line 98, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/django-pr/django/db/models/sql/compiler.py", line 1622, in execute_sql
    cursor = self.connection.cursor()
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/django-pr/django/utils/asyncio.py", line 24, in inner
    raise SynchronousOnlyOperation(message)
django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.

We probably need to add AFETCH_PEERS and AFETCH_ONE fetch modes, and duplicate the fetch_{one,many}() methods into afetch_{one,many}() on the descriptors.

I can hustle this in before the release candidate if it seems sane.

1 2 3 4 5 6 7 8 9 10 11 →

Query Language

query: TracLinks and the [[TicketQuery]] macro both use a mini “query language” for specifying query filters. Filters are separated by ampersands (&). Each filter consists of the ticket field name, an operator and one or more values. More than one value are separated by a pipe (|), meaning that the filter matches any of the values. To include a literal & or | in a value, escape the character with a backslash (\).

The available operators are:

= the field content exactly matches one of the values
~= the field content contains one or more of the values
^= the field content starts with one of the values
$= the field content ends with one of the values

All of these operators can also be negated:

!= the field content matches none of the values
!~= the field content does not contain any of the values
!^= the field content does not start with any of the values
!$= the field content does not end with any of the values

The date fields created and modified can be constrained by using the = operator and specifying a value containing two dates separated by two dots (..). Either end of the date range can be left empty, meaning that the corresponding end of the range is open. The date parser understands a few natural date specifications like "3 weeks ago", "last month" and "now", as well as Bugzilla-style date specifications like "1d", "2w", "3m" or "4y" for 1 day, 2 weeks, 3 months and 4 years, respectively. Spaces in date specifications can be omitted to avoid having to quote the query string.

created=2007-01-01..2008-01-01 query tickets created in 2007
created=lastmonth..thismonth query tickets created during the previous month
modified=1weekago.. query tickets that have been modified in the last week
modified=..30daysago query tickets that have been inactive for the last 30 days

See also: TracTickets, TracReports, TracGuide, TicketQuery

Note: See TracWiki for help on using the wiki.
Back to Top