> djangobench --experiment=a415ce70bef6d91036b00dd2c8544aed7aeeaaed --control=cce47ff65a4dd3786c049ec14ee889e128ca7de9 Running all benchmarks Control: Django 3.0.dev20190620094849 (in git branch cce47ff65a4dd3786c049ec14ee889e128ca7de9) Experiment: Django 3.0.dev20190620102943 (in git branch a415ce70bef6d91036b00dd2c8544aed7aeeaaed) Running 'query_dates' benchmark ... Min: 0.001606 -> 0.001775: 1.1051x slower Avg: 0.001655 -> 0.001846: 1.1153x slower Significant (t=-9.536159) Stddev: 0.00010 -> 0.00010: 1.0812x larger (N = 50) Running 'url_resolve_nested' benchmark ... Min: 0.000218 -> 0.000254: 1.1657x slower Avg: 0.000739 -> 0.001145: 1.5495x slower Not significant Stddev: 0.00366 -> 0.00625: 1.7107x larger (N = 50) Running 'query_update' benchmark ... Min: 0.000251 -> 0.000326: 1.3017x slower Avg: 0.000271 -> 0.000347: 1.2813x slower Significant (t=-9.417966) Stddev: 0.00004 -> 0.00004: 1.2209x smaller (N = 50) Running 'query_filter' benchmark ... Min: 0.000818 -> 0.000947: 1.1588x slower Avg: 0.000867 -> 0.000993: 1.1450x slower Significant (t=-9.670772) Stddev: 0.00007 -> 0.00006: 1.2558x smaller (N = 50) Running 'query_latest' benchmark ... Min: 0.000846 -> 0.000912: 1.0786x slower Avg: 0.000889 -> 0.000968: 1.0884x slower Significant (t=-4.917320) Stddev: 0.00008 -> 0.00008: 1.1260x larger (N = 50) Running 'url_resolve_flat' benchmark ... Min: 0.168138 -> 0.185167: 1.1013x slower Avg: 0.174532 -> 0.188692: 1.0811x slower Significant (t=-5.142500) Stddev: 0.01697 -> 0.00954: 1.7789x smaller (N = 50) Running 'raw_sql' benchmark ... Min: 0.000079 -> 0.000129: 1.6284x slower Avg: 0.000085 -> 0.000140: 1.6438x slower Significant (t=-12.105940) Stddev: 0.00002 -> 0.00003: 1.5285x larger (N = 50) Running 'form_create' benchmark ... Min: 0.000080 -> 0.000115: 1.4410x slower Avg: 0.000114 -> 0.000151: 1.3214x slower Not significant Stddev: 0.00021 -> 0.00021: 1.0082x smaller (N = 50) Running 'url_reverse' benchmark ... Min: 0.000240 -> 0.000568: 2.3714x slower Avg: 0.000456 -> 0.000796: 1.7473x slower Not significant Stddev: 0.00149 -> 0.00151: 1.0169x larger (N = 50) Running 'query_aggregate' benchmark ... Min: 0.000530 -> 0.000634: 1.1961x slower Avg: 0.000565 -> 0.000667: 1.1808x slower Significant (t=-8.232631) Stddev: 0.00006 -> 0.00006: 1.0545x smaller (N = 50) Running 'multi_value_dict' benchmark ... Min: 0.000082 -> 0.000082: 1.0018x slower Avg: 0.000138 -> 0.000156: 1.1318x slower Not significant Stddev: 0.00004 -> 0.00013: 3.5026x larger (N = 50) Running 'query_prefetch_related' benchmark ... Min: 0.185224 -> 0.209212: 1.1295x slower Avg: 0.198212 -> 0.219117: 1.1055x slower Significant (t=-5.596483) Stddev: 0.02276 -> 0.01341: 1.6973x smaller (N = 50) Running 'query_exists' benchmark ... Min: 0.001489 -> 0.001786: 1.1994x slower Avg: 0.001549 -> 0.001904: 1.2292x slower Significant (t=-17.690485) Stddev: 0.00008 -> 0.00012: 1.5316x larger (N = 50) Running 'query_values' benchmark ... Min: 0.000460 -> 0.000526: 1.1428x slower Avg: 0.000489 -> 0.000557: 1.1392x slower Significant (t=-6.435646) Stddev: 0.00005 -> 0.00005: 1.0406x larger (N = 50) Running 'template_compilation' benchmark ... Min: 0.001122 -> 0.001123: 1.0012x slower Avg: 0.001291 -> 0.001275: 1.0120x faster Not significant Stddev: 0.00097 -> 0.00093: 1.0442x smaller (N = 50) Running 'query_all_converters' benchmark ... Min: 0.003720 -> 0.003900: 1.0482x slower Avg: 0.003840 -> 0.004097: 1.0669x slower Significant (t=-6.437233) Stddev: 0.00011 -> 0.00026: 2.3865x larger (N = 50) Running 'query_select_related' benchmark ... Min: 0.099899 -> 0.100411: 1.0051x slower Avg: 0.104357 -> 0.106177: 1.0174x slower Not significant Stddev: 0.00897 -> 0.01143: 1.2746x larger (N = 50) Running 'url_resolve_flat_i18n_off' benchmark ... Min: 0.167083 -> 0.189748: 1.1357x slower Avg: 0.169900 -> 0.194429: 1.1444x slower Significant (t=-13.818209) Stddev: 0.00920 -> 0.00854: 1.0763x smaller (N = 50) Running 'template_render_simple' benchmark ... Min: 0.000196 -> 0.000193: 1.0165x faster Avg: 0.000334 -> 0.000326: 1.0254x faster Not significant Stddev: 0.00093 -> 0.00089: 1.0472x smaller (N = 50) Running 'query_exclude' benchmark ... Min: 0.001017 -> 0.001183: 1.1631x slower Avg: 0.001130 -> 0.001245: 1.1013x slower Significant (t=-3.272947) Stddev: 0.00023 -> 0.00008: 2.7382x smaller (N = 50) Running 'startup' benchmark ... Skipped: Django 1.9 and later has changed app loading. This benchmark needs fixing anyway. Running 'query_order_by' benchmark ... Min: 0.000846 -> 0.000938: 1.1085x slower Avg: 0.000887 -> 0.000996: 1.1218x slower Significant (t=-7.212963) Stddev: 0.00007 -> 0.00008: 1.2756x larger (N = 50) Running 'query_all_multifield' benchmark ... Min: 0.091016 -> 0.088657: 1.0266x faster Avg: 0.096679 -> 0.095110: 1.0165x faster Not significant Stddev: 0.01005 -> 0.01066: 1.0597x larger (N = 50) Running 'template_render' benchmark ... Min: 0.005744 -> 0.006217: 1.0825x slower Avg: 0.007830 -> 0.008224: 1.0502x slower Not significant Stddev: 0.01269 -> 0.01233: 1.0295x smaller (N = 50) Running 'model_save_existing' benchmark ... Min: 0.020492 -> 0.025256: 1.2325x slower Avg: 0.021420 -> 0.026055: 1.2164x slower Significant (t=-32.109733) Stddev: 0.00080 -> 0.00063: 1.2665x smaller (N = 50) Running 'default_middleware' benchmark ... Min: -0.000001 -> -0.000007: 0.1436x faster Avg: 0.001445 -> 0.001505: 1.0420x slower Not significant Stddev: 0.01015 -> 0.01061: 1.0461x larger (N = 50) Running 'qs_filter_chaining' benchmark ... Min: 0.001312 -> 0.001669: 1.2721x slower Avg: 0.001367 -> 0.001769: 1.2941x slower Significant (t=-9.852406) Stddev: 0.00014 -> 0.00025: 1.8671x larger (N = 50) Running 'query_get_or_create' benchmark ... Min: 0.002449 -> 0.002925: 1.1942x slower Avg: 0.002634 -> 0.003081: 1.1699x slower Significant (t=-3.417605) Stddev: 0.00082 -> 0.00043: 1.9345x smaller (N = 50) Running 'query_in_bulk' benchmark ... Min: 0.000937 -> 0.001120: 1.1953x slower Avg: 0.000997 -> 0.001184: 1.1878x slower Significant (t=-8.731875) Stddev: 0.00011 -> 0.00010: 1.1355x smaller (N = 50) Running 'query_distinct' benchmark ... Min: 0.000611 -> 0.000688: 1.1252x slower Avg: 0.000647 -> 0.000735: 1.1360x slower Significant (t=-6.419391) Stddev: 0.00006 -> 0.00007: 1.1989x larger (N = 50) Running 'locale_from_request' benchmark ... Min: 0.000386 -> 0.000423: 1.0969x slower Avg: 0.000462 -> 0.000501: 1.0840x slower Not significant Stddev: 0.00046 -> 0.00047: 1.0221x larger (N = 50) Running 'query_get' benchmark ... Min: 0.001648 -> 0.002113: 1.2826x slower Avg: 0.001726 -> 0.008590: 4.9764x slower Significant (t=-5.990221) Stddev: 0.00012 -> 0.00810: 68.3286x larger (N = 50) Running 'query_iterator' benchmark ... Min: 0.000557 -> 0.000664: 1.1921x slower Avg: 0.000594 -> 0.000706: 1.1881x slower Significant (t=-7.739740) Stddev: 0.00007 -> 0.00007: 1.0311x larger (N = 50) Running 'query_all' benchmark ... Min: 0.053971 -> 0.054368: 1.0073x slower Avg: 0.058942 -> 0.059746: 1.0136x slower Not significant Stddev: 0.00962 -> 0.01066: 1.1076x larger (N = 50) Running 'model_delete' benchmark ... Min: 0.000724 -> 0.000877: 1.2125x slower Avg: 0.000778 -> 0.001258: 1.6172x slower Significant (t=-15.177560) Stddev: 0.00012 -> 0.00019: 1.6624x larger (N = 50) Running 'query_count' benchmark ... Min: 0.000444 -> 0.000529: 1.1923x slower Avg: 0.000482 -> 0.000571: 1.1847x slower Significant (t=-5.878392) Stddev: 0.00008 -> 0.00007: 1.0769x smaller (N = 50) Running 'query_values_10000' benchmark ... Min: 0.036362 -> 0.034332: 1.0591x faster Avg: 0.055256 -> 0.050081: 1.1033x faster Not significant Stddev: 0.01706 -> 0.02402: 1.4079x larger (N = 50) Running 'query_none' benchmark ... Min: 0.000286 -> 0.000345: 1.2070x slower Avg: 0.000364 -> 0.000418: 1.1467x slower Not significant Stddev: 0.00038 -> 0.00014: 2.7519x smaller (N = 50) Running 'query_delete_related' benchmark ... Min: 0.000581 -> 0.000799: 1.3738x slower Avg: 0.000698 -> 0.000919: 1.3165x slower Not significant Stddev: 0.00057 -> 0.00063: 1.1152x larger (N = 50) Running 'model_save_new' benchmark ... Min: 0.020373 -> 0.024828: 1.2186x slower Avg: 0.021451 -> 0.026129: 1.2181x slower Significant (t=-9.691248) Stddev: 0.00242 -> 0.00241: 1.0060x smaller (N = 50) Running 'query_raw_deferred' benchmark ... Min: 0.025999 -> 0.026282: 1.0109x slower Avg: 0.027795 -> 0.027985: 1.0068x slower Not significant Stddev: 0.00535 -> 0.00562: 1.0505x larger (N = 50) Running 'query_delete' benchmark ... Min: 0.000622 -> 0.000596: 1.0430x faster Avg: 0.000891 -> 0.000632: 1.4105x faster Significant (t=8.517416) Stddev: 0.00020 -> 0.00007: 2.8441x smaller (N = 50) Running 'url_resolve' benchmark ... Min: 0.017158 -> 0.026667: 1.5541x slower Avg: 0.017560 -> 0.027022: 1.5388x slower Significant (t=-36.435728) Stddev: 0.00128 -> 0.00131: 1.0230x larger (N = 50) Running 'form_clean' benchmark ... Min: 0.000031 -> 0.000032: 1.0159x slower Avg: 0.000034 -> 0.000034: 1.0050x faster Not significant Stddev: 0.00001 -> 0.00001: 1.5007x smaller (N = 50) Running 'model_creation' benchmark ... Min: 0.000381 -> 0.000440: 1.1566x slower Avg: 0.000469 -> 0.000482: 1.0270x slower Not significant Stddev: 0.00045 -> 0.00008: 5.7454x smaller (N = 50) Running 'query_raw' benchmark ... Min: 0.033103 -> 0.033827: 1.0219x slower Avg: 0.034986 -> 0.035563: 1.0165x slower Not significant Stddev: 0.00564 -> 0.00565: 1.0032x larger (N = 50) Running 'query_complex_filter' benchmark ... Min: 0.000232 -> 0.000277: 1.1898x slower Avg: 0.000245 -> 0.000291: 1.1915x slower Significant (t=-8.527667) Stddev: 0.00003 -> 0.00003: 1.1290x larger (N = 50) Running 'query_values_list' benchmark ... Min: 0.000439 -> 0.000518: 1.1796x slower Avg: 0.000471 -> 0.000547: 1.1623x slower Significant (t=-6.546009) Stddev: 0.00006 -> 0.00006: 1.0484x smaller (N = 50) Running 'query_annotate' benchmark ... Min: 0.001019 -> 0.001121: 1.0996x slower Avg: 0.001079 -> 0.001216: 1.1271x slower Significant (t=-3.289610) Stddev: 0.00010 -> 0.00028: 2.8625x larger (N = 50) Running 'l10n_render' benchmark ... Min: 0.005730 -> 0.005903: 1.0301x slower Avg: 0.007294 -> 0.008301: 1.1379x slower Not significant Stddev: 0.00974 -> 0.00992: 1.0192x larger (N = 50)