Add a model fuzz testing method to the test framework

[Adrian Holovaty]

​Fuzz testing is when random, crazy data is thrown at some code to make sure it can handle crazy input. The Django test system should provide an assertModelPassesFuzzTest() method (or pick a better name), which would take a model class as its argument and do the following:

• Insert a bunch of random data, including non-ASCII Unicode data, into a model instance and make sure the instance's __unicode__() doesn't throw UnicodeEncodeError or UnicodeDecodeError.
• Call every custom model method that doesn't have alters_data=True (or maybe include those, too?) and make sure nothing silly happens. (In this case, it will be hard to determine what "silly" means, without some hints from the model method as to its expected output.)

[Fraser Nevett]

Last change was me, sorry.

Adrian raised this ticket, so I assume it is correct to set to "Accepted".

[Fraser Nevett]

The attached patch (plus documentation) introduces two new assertions:

• assertModelUnicodePassesFuzzTest
• assertFunctionPassesFuzzTest

The first of these, assertModelUnicodePassesFuzzTest, assigns random unicode data to each of an instance of the model's fields and checks that calling the instance's unicode method doesn't raise a UnicodeError

I couldn't see how it would be possible to have an assertModelPassesFuzzTest (as Adrian suggested) that calls each of a models methods, as it would be very awkward to specify, for each method, valid argument types, response types and exceptions. Instead I have created assertFunctionPassesFuzzTest that allows you to call each method individually specifying valid inputs and outputs.

[Aymeric Augustin]

Change UI/UX from NULL to False.

[Aymeric Augustin]

Unicode-related problems and badly written unicode methods are less common than they used to be 5 years ago. I haven't encountered this class of problems frequently.

Given the lack of activity here, I suggest to abandon this ticket.

[Roman Gladkov]

Interesting feature. I want try to upgrade this patch for current version and to make tests for it.

[Roman Gladkov]

patch has been updated and tested in python 2.7.3/3.3.0. also added some tests.

[Tim Graham]

Agreed with Aymeric's comment from 19 months ago, it doesn't seem like this is something that Django needs to provide.