Syndication Feed Server ignores 'Last-Modified' date (Debian#926074)

[Christian Buhtz]

Hello,
First please take my apologize for writing such a broad ticket. But others, with more intel, I asked, didn't.

This is related to a (6 year old) Debian bug ticket: ​https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926074
In short: The modified date is ignored by the server.

The Debian folks telling me that Django is somehow involved and there should go a bug ticket and fix.
​https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926074#10

I can not tell because I lack of deeper understanding of the underlying technology.

I do have minimal working example demonstrating the problem with the Debian server.

>>> url = 'https://tracker.debian.org/pkg/feedparser/rss'
>>> import feedparser as fp
>>> a = fp.parse(url)
>>> import json
>>> print(json.dumps(a.headers, indent=4))
{
    "X-Clacks-Overhead": "GNU Terry Pratchett",
    "X-Xss-Protection": "1",
    "Transfer-Encoding": "chunked",
    "Last-Modified": "Sat, 30 Mar 2019 13:06:51 GMT",
    "Public-Key-Pins": "pin-
sha256=\"M9Wyk2x6hLne/VkyJWFx33+LB8s+kAaMbO49fnm5fEY=\"; pin-
sha256=\"ZsBWn0bekrum/1CJtBlg5yZ2eqRywN6YwGeBS31nH+s=\"; max-age=5184000",
    "Strict-Transport-Security": "max-age=15552000",
    "Content-Type": "application/rss+xml; charset=utf-8",
    "X-Content-Type-Options": "nosniff",
    "Connection": "close",
    "Referrer-Policy": "same-origin",
    "Content-Encoding": "gzip",
    "X-Frame-Options": "sameorigin",
    "Server": "Apache",
    "Vary": "Cookie,Accept-Encoding",
    "Date": "Sun, 31 Mar 2019 11:11:03 GMT"
}
>>> a.modified
'Sat, 30 Mar 2019 13:06:51 GMT'
>>> b = fp.parse(url, modified=a.modified)
>>> len(a.entries)
30
>>> len(b.entries)
30

I would expect "b.entries" should be (near) 0.

I also can not tell wich version of Django is used by Debian.

Maybe you can give some comments in the Debian ticket?

Regards,
Christian

[Natalia Bidart]

Hello Christian Buhtz, thank you for taking the time to create this ticket. I see from the messages in the Debian mailing list that you care deeply about this issue. Still, this is not an issue in Django: the Feed implementation from the syndication framework does handle item publication as shown in the Django source code. See the class Feed implementation at ​these relevant lines. This behavior is enabled via ConditionalGetMiddleware (which I'm not sure if the Debian tracker uses).

The issue you're experiencing could be caused by a variety of factors, including (but not limited to) the PackageNewsFeed view (from the ​Debian source code that defines item_pubdate()) returning timezone-naive datetimes or None.

Considering that this issue tracker is meant to track confirmed bugs in Django itself, and since this report seems to stem from application-specific code in the Debian project, I’ll close this ticket as invalid per our ​ticket triaging process. The best place to get further help would be one of the support forums listed at ​this link. If after debugging, you (or someone else) find that the problem lies in Django itself, please reopen the ticket with specific details and, ideally, a minimal Django project or failing test case that demonstrates the issue (using Django code only, we're not able to debug arbitrary third-party services).

[Tim Graham]

I didn't analyze this in detail but it looks like it could be a new feature request that the syndication framework only returns items published since the "Last-Modified" header.