Opened 7 years ago

Closed 7 years ago

#26073 closed Bug (duplicate)

Last-Modified http header not set to the latest last-mod in multi-maps sitemaps

Reported by: Eric Owned by: Eric
Component: contrib.sitemaps Version: 1.9
Severity: Normal Keywords: sitemap last-mod Last-Modified
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

at https://github.com/django/django/blob/master/django/contrib/sitemaps/views.py#L46

def sitemap(request, sitemaps, section=None,
            template_name='sitemap.xml', content_type='application/xml'):

    req_protocol = request.scheme
    req_site = get_current_site(request)

    if section is not None:
        if section not in sitemaps:
            raise Http404("No sitemap available for section: %r" % section)
        maps = [sitemaps[section]]
    else:
        maps = sitemaps.values()
    page = request.GET.get("p", 1)

    urls = []
    for site in maps:
        try:
            if callable(site):
                site = site()
            urls.extend(site.get_urls(page=page, site=req_site,
                                      protocol=req_protocol))
        except EmptyPage:
            raise Http404("Page %s empty" % page)
        except PageNotAnInteger:
            raise Http404("No page '%s'" % page)
    response = TemplateResponse(request, template_name, {'urlset': urls},
                                content_type=content_type)
    if hasattr(site, 'latest_lastmod'):
        # if latest_lastmod is defined for site, set header so as
        # ConditionalGetMiddleware is able to send 304 NOT MODIFIED
        lastmod = site.latest_lastmod
        response['Last-Modified'] = http_date(
            timegm(
                lastmod.utctimetuple() if isinstance(lastmod, datetime.datetime)
                else lastmod.timetuple()
            )
        )
    return response

we can see that when it gets the 'latest_lastmod' from site it only really depends on which was the last site in maps which might not always be the latest_lastmod from all items in maps.

I think this is a big SEO issue for websites that rely on their sitemaps being indexed by search engines as I noticed my website did not seem to have my newer Urls indexed by google since it probably saw the Last-Modified http header was not being newer that what they currently had (even if it contains new urls with newer last-mod)

Change History (2)

comment:1 Changed 7 years ago by Eric

Owner: changed from nobody to Eric
Status: newassigned

I will make a pull-request in the next couple of days (just need to setup a proper django dev install)

Last edited 7 years ago by Eric (previous) (diff)

comment:2 Changed 7 years ago by Tim Graham

Resolution: duplicate
Status: assignedclosed

Duplicate of #25989 which already has a patch. Feel free to review it using the PatchReviewChecklist and mark the ticket "Ready for checkin" when it looks good.

Note: See TracTickets for help on using tickets.
Back to Top