Opened 7 years ago

Closed 7 years ago

Last modified 4 years ago

#7793 closed (fixed)

Sitemaps generator does not care about 50,000 URL limit

Reported by: Julian Bez Owned by: nobody
Component: Contrib apps Version: master
Severity: Keywords: sitemaps
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

As stated in the protocol, each sitemap must have no more than 50,000 URLs.
http://sitemaps.org/protocol.php#index

The current sitemaps generator does not care about the limit and is not suited for larger sets of data.

The patch provided adds the paginator to sitemaps and creates a sitemap for every 50,000 URLs.

Attachments (3)

sitemaps-pagination.diff (3.7 KB) - added by Julian 7 years ago.
Patch that adds pagination
sitemaps-pagination.2.diff (3.9 KB) - added by julianb 7 years ago.
Updated: Limit is an attribute now
sitemaps-pagination.3.diff (4.0 KB) - added by julianb 7 years ago.
Fixed tiny mistake

Download all attachments as: .zip

Change History (10)

Changed 7 years ago by Julian

Patch that adds pagination

comment:1 Changed 7 years ago by julianb

  • Has patch set
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

comment:2 Changed 7 years ago by Simon Greenhill

  • milestone set to 1.0 beta
  • Triage Stage changed from Unreviewed to Ready for checkin

Changed 7 years ago by julianb

Updated: Limit is an attribute now

Changed 7 years ago by julianb

Fixed tiny mistake

comment:3 Changed 7 years ago by mtredinnick

I don't really have any way of testing this, but I've read it through with some care and it looks to be doing the right thing. The docs change might need to be added to a little in the future, since it doesn't really make sense to me and I know what the patch is meant to be doing. So attempts to fix that (in another ticket) will be welcomed.

comment:4 Changed 7 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from new to closed

(In [8088]) Fixed #7793 -- Handle sitemaps with more than 50,000 URLs in them (by using
pagination). Patch from Julian Bez.

The docs patch here could probably do with some rewording.

comment:5 Changed 6 years ago by anonymous

I am just trying to paginate the different sections to a lower limit, I tried to define the pagination limit to 1000 (otherwise the size of the view is too big to be in my memcached, memcached-limit: 1mb) to get more pages, but the index get's messed up and just list always the same "sitemap.xml" not the correct sections pages, should we reopen this ticket or is just my configuration problem ?

comment:6 Changed 6 years ago by anonymous

sorry for my last message, I found the problem was related to cache_view and urlresolve, I will send a message to the mailing list

comment:7 Changed 4 years ago by jacob

  • milestone 1.0 beta deleted

Milestone 1.0 beta deleted

Note: See TracTickets for help on using tickets.
Back to Top