Opened 7 years ago

Closed 5 years ago

Last modified 4 years ago

#8794 closed (fixed)

Profanity filter suffers from the Scunthorpe problem

Reported by: Daniel Pope <dan@…> Owned by: nobody
Component: contrib.comments Version: master
Severity: Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:


The implementation of the profanity filter suffers from the Scunthorpe Problem; ie. that it considers the town of Scunthorpe, amongst other innocuous words, to be profane.

Profanity filtering is A Hard Problem, and naïve solutions like this one cause frustrating problems to end-users.

Checking the current profanities list for false positives in a couple of word lists I had to hand also yields:

Shittah tree
Shittim wood

Obviously proper names are not in my dictionary, but they cause frequent and often more annoying problems.

I suggest to disable the filter by default so that scope of the problem is limited, and at the very least the filter must be restricted to re.match(r'\b' + word + '\b'). Users who need stricter profanity filters should have the responsibility for doing so, and potentially annoying their users themselves. Django should not be doing it for them.

Attachments (2)

8794.diff (2.6 KB) - added by thejaswi_puthraya 7 years ago.
git-patch of the latest checkout
t8794.diff (1.5 KB) - added by lukeplant 5 years ago.
Fix for the actual Scunthorpe bug.

Download all attachments as: .zip

Change History (9)

comment:1 Changed 7 years ago by thejaswi_puthraya

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Design decision needed

It's quite easy to toggle the COMMENTS_ALLOW_PROFANITIES setting. I am a -0 on this. Am marking this as a DDN.

comment:2 Changed 7 years ago by adrian

  • Triage Stage changed from Design decision needed to Accepted

I have come to believe that we shouldn't be distributing a list of profanities with Django. Problem is, we're committed to backwards compatibility -- so we'll have to leave those in until the next major release.

Changed 7 years ago by thejaswi_puthraya

git-patch of the latest checkout

comment:3 Changed 5 years ago by thejaswi_puthraya

  • milestone set to 2.0

Check #6290. This ticket has to be pushed for the next major revision ie 2.0.

comment:4 Changed 5 years ago by russellm

  • milestone changed from 2.0 to 1.3

This needs to be handled with our usual deprecation plan rather than just removing it. Bumping to the 1.3 release following a recent discussion on django-dev

Changed 5 years ago by lukeplant

Fix for the actual Scunthorpe bug.

comment:5 Changed 5 years ago by adrian

  • Resolution set to fixed
  • Status changed from new to closed

(In [13996]) Set default PROFANITIES_LIST setting to an empty tuple. This is technically backwards-compatible if you rely on a Web framework providing you with a woefully incomplete list of naughty words. Fixes #8794

comment:6 Changed 4 years ago by jacob

  • milestone 1.3 deleted

Milestone 1.3 deleted

comment:11 Changed 4 years ago by aaugustin

In [16935]:

Fixed #17012 - Removed references to the 'hasNoProfanities' validator. Refs #8794.

Note: See TracTickets for help on using tickets.
Back to Top