Opened 16 years ago

Closed 13 years ago

Last modified 12 years ago

#8794 closed (fixed)

Profanity filter suffers from the Scunthorpe problem

Reported by: Daniel Pope <dan@…> Owned by: nobody
Component: contrib.comments Version: dev
Severity: Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The implementation of the profanity filter suffers from the Scunthorpe Problem; ie. that it considers the town of Scunthorpe, amongst other innocuous words, to be profane.

Profanity filtering is A Hard Problem, and naïve solutions like this one cause frustrating problems to end-users.

Checking the current profanities list for false positives in a couple of word lists I had to hand also yields:

gobbledegook
snigger
Brushite
Cushite
Niggerhead
Peshito
Peshitto
Shittah
Shittah tree
Shittim
Shittim wood
Shittle
Shittlecock
Shittleness

Obviously proper names are not in my dictionary, but they cause frequent and often more annoying problems.

I suggest to disable the filter by default so that scope of the problem is limited, and at the very least the filter must be restricted to re.match(r'\b' + word + '\b'). Users who need stricter profanity filters should have the responsibility for doing so, and potentially annoying their users themselves. Django should not be doing it for them.

Attachments (2)

8794.diff (2.6 KB ) - added by Thejaswi Puthraya 15 years ago.
git-patch of the latest checkout
t8794.diff (1.5 KB ) - added by Luke Plant 13 years ago.
Fix for the actual Scunthorpe bug.

Download all attachments as: .zip

Change History (9)

comment:1 by Thejaswi Puthraya, 16 years ago

Triage Stage: UnreviewedDesign decision needed

It's quite easy to toggle the COMMENTS_ALLOW_PROFANITIES setting. I am a -0 on this. Am marking this as a DDN.

comment:2 by Adrian Holovaty, 16 years ago

Triage Stage: Design decision neededAccepted

I have come to believe that we shouldn't be distributing a list of profanities with Django. Problem is, we're committed to backwards compatibility -- so we'll have to leave those in until the next major release.

by Thejaswi Puthraya, 15 years ago

Attachment: 8794.diff added

git-patch of the latest checkout

comment:3 by Thejaswi Puthraya, 14 years ago

milestone: 2.0

Check #6290. This ticket has to be pushed for the next major revision ie 2.0.

comment:4 by Russell Keith-Magee, 13 years ago

milestone: 2.01.3

This needs to be handled with our usual deprecation plan rather than just removing it. Bumping to the 1.3 release following a recent discussion on django-dev

by Luke Plant, 13 years ago

Attachment: t8794.diff added

Fix for the actual Scunthorpe bug.

comment:5 by Adrian Holovaty, 13 years ago

Resolution: fixed
Status: newclosed

(In [13996]) Set default PROFANITIES_LIST setting to an empty tuple. This is technically backwards-compatible if you rely on a Web framework providing you with a woefully incomplete list of naughty words. Fixes #8794

comment:6 by Jacob, 12 years ago

milestone: 1.3

Milestone 1.3 deleted

comment:11 by Aymeric Augustin, 12 years ago

In [16935]:

Fixed #17012 - Removed references to the 'hasNoProfanities' validator. Refs #8794.

Note: See TracTickets for help on using tickets.
Back to Top