Ticket #16779: contrib-tutorial.txt

===================================
Writing your first patch for Django
===================================

Now that you've finished writing your first Django app, how about giving back to the community a little? Maybe you've found a bug in Django that you'd like to see fixed, or maybe there's a small feature you want added.

Contributing back to Django itself is the best way to see your own concerns addressed. This may seem daunting at first, but it's actually pretty simple. We'll walk you through the entire process, so you can learn by example.

In this tutorial, we'll walk through contributing a patch to Django for the first time. By the end of this tutorial, you should understand both the tools and the process involved. We'll be covering the following:

    * What you'll need to get started.
    * How to write your first patch.
    * Working together with the community.

.. note::
    
    For this tutorial, we'll be using `ticket 15315`__ as a case study. As part of this process, we will be going through all of the steps involved in writing that patch from scratch, including running Django's test suite. In order to do this though, we'll be working with an older revision of Django in this tutorial. When working on your own patch for another ticket, just make sure that you're using the current development revision of Django, and **not** an older revision like we are here.
    
    The patch for this ticket was generously written by SardarNL and Will Hardy, and it was applied to Django as `changeset 16659`__. Consequently, we'll be checking out the revision just prior to that, revision 16658.

.. admonition:: Where to get help:

    If you're having trouble going through this tutorial, please post a message
    to `django-users`__ or drop by `#django on irc.freenode.net`__ to chat
    with other Django users who might be able to help.

__ https://code.djangoproject.com/ticket/15315
__ https://code.djangoproject.com/changeset/16659
__ http://groups.google.com/group/django-users
__ irc://irc.freenode.net/django

What you'll need to get started
===============================

Subversion or Git
-----------------

For this tutorial, you'll need to have either Subversion or Git installed. These tools will be used to download the current development version of Django and to generate patch files for the changes you make.

To check whether or not you have one of these tools already installed, enter either ``svn`` or ``git`` into the command line. If you get messages saying that neither of these commands could be found, you'll have to download and install one of them yourself.

You can download and install Subversion from the `Apache's Subversion Binary Packages page`__.

Alternatively, you can download and install Git from `Git's download page`__.

__ http://subversion.apache.org/packages.html
__ http://git-scm.com/download

A Trac account
--------------

Next, you'll need an account for Trac, Django's online issue tracker. Trac allows community members to file tickets for bugs or feature requests, upload patches, discuss changes, and keep track of all the activity on the project.

If you don't have a Trac account yet, visit the `registration page`__ and create one. If you already have an account but have forgotten your password, you can reset it using the `password reset page`__. Having a Trac account will allow you to maintain a consistent identity when contributing to Django and build rapport within the community. In addition, it allows you to claim a ticket for yourself so that other community members know that you're currently working on a patch for it.

__ https://www.djangoproject.com/accounts/register
__ https://www.djangoproject.com/accounts/password/reset/

A ticket to work on
-------------------

Once you've setup a Trac account and are logged in, you'll need to find a ticket to work on. Trac has `a very powerful search engine`__ which allows you to filter tickets based on a plethora of criteria. For those of you who've never written a patch for Django before, pay special attention to tickets with the "easy pickings" criterion. These tickets are often much simpler in nature and are great for first time contributors.  Once you're familiar with contributing to Django, you can move on to writing patches for more difficult and complicated tickets.

Finding a ticket
~~~~~~~~~~~~~~~~

If you just want to get started, try taking a look at the list of `easy tickets that need patches`__ and the `easy tickets that have patches which need improvement`__. If you're familiar with writing tests, you can also look at the list of `easy tickets that need tests`__.

Claiming a ticket
~~~~~~~~~~~~~~~~~

Once you've found a ticket that looks interesting to you, make sure nobody else has claimed it. To do this, look at the "Assigned to" section of the ticket. If it's assigned to "nobody," then it's available to be claimed. Otherwise, somebody else is working on this ticket, and you can either find another bug/feature to work on or contact the developer working on the ticket to offer your help.

To claim a ticket for yourself, click the "accept" radio button under "Action" near the bottom of the page and then click "Submit changes." Keep in mind that once you've claimed a ticket, you have a responsibility to work on that ticket in a reasonably timely fashion. If you don't have time to work on it, either unclaim it or don't claim it in the first place!

.. admonition:: What do I do if a ticket already has a patch?

    Sometimes a ticket already has a patch, but that patch may still need some adjustments or changes before it can be added to Django. In these cases, the ticket is flagged with the "patch needs improvement" criterion. These tickets can provide a great opportunity to contribute to Django if you don't want to have to write a patch from scratch. Just make sure the ticket is unclaimed before you start your work, and pay particular attention to the suggestions and remarks in the ticket's change history.
        
    To work on an existing patch, you'll have to download the existing patch file from the ticket and then apply it to your local copy of Django before making your own modifications.  Detailed steps for this can be found below under `Applying an existing patch to Django`_.

__ https://code.djangoproject.com/query
__ https://code.djangoproject.com/query?status=new&status=reopened&has_patch=0&easy=1&col=id&col=summary&col=status&col=owner&col=type&col=milestone&order=priority
__ https://code.djangoproject.com/query?status=new&status=reopened&needs_better_patch=1&easy=1&col=id&col=summary&col=status&col=owner&col=type&col=milestone&order=priority
__ https://code.djangoproject.com/query?status=new&status=reopened&needs_tests=1&easy=1&col=id&col=summary&col=status&col=owner&col=type&col=milestone&order=priority

How to write your first patch
=============================

Checking out Django's development version
-----------------------------------------

The first step to contributing to Django is to check out the Django's current development revision using either Subversion or Git.

.. note::

    For the purposes of this tutorial, we'll be checking out an **older** revision of Django's trunk, specifically revision 16658; however, when working on your own patch for another ticket, make sure you remember to check out the **current** development revision.

From the command line, use the ``cd`` command to navigate to the directory where you'll want your local copy of Django to live.

Using Subversion
~~~~~~~~~~~~~~~~

If you have Subversion installed, you can run the following command to download revision 16658 of Django:

.. code-block:: bash
    
    svn checkout -r 16658 http://code.djangoproject.com/svn/django/trunk/

Using Git
~~~~~~~~~

Alternatively, if you prefer using Git, you can clone the official Django Git mirror (which is updated from the Subversion repository every five minutes), and then switch to revision 16658 as shown below:

.. code-block:: bash

    git clone https://github.com/django/django.git
    cd django
    git checkout 90e8bd48f2

.. note::

    If you're not that familiar with Subversion or Git, you can always find out more about their various commands by typing either ``svn help`` or ``git help`` into the command line.

Running Django's test suite
---------------------------

When contributing to Django it's very important that your code changes don't introduce bugs into other areas of Django.  One way to check that Django stills works after you make your changes is by running Django's entire test suite. If all the tests still pass, then you can be reasonably sure that your changes haven't completely broken Django. If you've never run Django's test suite before, it's a good idea to run it once beforehand just to get familiar with what its output is supposed to look like. 


Setting Django up to run the test suite
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Before we can actually run the test suite though, we need to make sure that your new local copy of Django is on your ``PYTHONPATH``; otherwise, the test suite won't run properly. We also need to make sure that there aren't any **other** copies of Django installed somewhere else that are taking priority over your newer copy (this happens more often than you might think). To check for these problems, start up the Python interpreter and follow along with the code below:

.. code-block:: python
    
    >>> import django
    >>> django
    <module 'django' from '/.../django/__init__.pyc'>

If you get an ``ImportError: No module named django`` after entering the first line, then you'll need to add your new copy of Django to your ``PYTHONPATH``. For more details on how to do this, read :ref:`pointing-python-at-the-new-django-version`.

If you didn't get any errors, then look at the path found in the third line (abbreviated above as ``/.../django/__init__.pyc``). If that isn't the directory that you put Django into earlier in this tutorial, then there is **another** copy of Django on your ``PYTHONPATH`` that is taking priority over the newer copy.  You'll either have to remove this older copy from your ``PYTHONPATH``, or add your new copy to the beginning of your ``PYTHONPATH`` so that it takes priority.

.. note::

    If you're a savvy Djangonaut you might be thinking that using ``virtualenv`` would work perfectly for this type of thing, and you'd be right (+100 bonus points). Using ``virtualenv`` with the ``--no-site-packages`` option isolates your local copy of Django from the rest of your system and avoids potential conflicts. Just make sure not to use ``pip`` to install Django, since that causes Django's setup.py script to be run. You'll still need to use Subversion or Git to check out a copy of Django, and then :ref:`manually add it to your ``PYTHONPATH``<pointing-python-at-the-new-django-version>`.

Actually running the Django test suite
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Once Django is setup properly, we can actually run the test suite. Simply ``cd`` into the Django ``tests/`` directory and run:

.. code-block:: bash

    ./runtests.py --settings=test_sqlite

If you get an ``ImportError: No module named django.contrib`` error, you still need to add your current copy of Django to your ``PYTHONPATH``. For more details on how to do this, read :ref:`pointing-python-at-the-new-django-version`.

Otherwise, sit back and relax. Django's entire test suite has over 4000 different tests, so it can take anywhere from 5 to 15 minutes to run, depending on the speed of your computer. 

.. note::
    
    While Django's test suite is running, you'll see a stream of characters representing the status of each test as it's run. ``E`` indicates that an error was raised during a test, and ``F`` indicates that a test's assertions failed. Both of these are considered to be test failures. Meanwhile, ``x`` and ``s`` indicated expected failures and skipped tests, respectively.  Dots indicated passing tests. 

Once the process is done, you should be greeted with a message informing you whether the test suite passed or failed. Since you haven't yet made any changes to Django's code, the entire test suite **should** pass.  If it doesn't, make sure you've followed all of the previous steps properly.

More information on how to run Django's test suite can be found in the :doc:`online documentation</internals/contributing/writing-code/unit-tests>`.

Writing a test for your ticket
------------------------------

In most cases, for a patch to be accepted into Django, it has to include tests. For bug fix patches, this means writing a regression test to ensure that the bug is never reintroduced into Django later on. A regression test should be written in such a way that it will fail while the bug still exists and pass once the bug has been fixed. For patches containing new features, you'll need to include tests which ensure that the new features are working correctly. They too should fail when the new feature is not present, and then pass once it has been implemented.

A good way to do this is to write your new tests first before making any changes to the code, and then run them to ensure that they do indeed fail. If your new tests don't fail, you'll need to fix them so that they do. After all, a regression test that passes regardless of whether a bug is present is not very helpful for preventing that bug's reoccurrence.

Now for our hands on example.

Writing a test for ticket 15315
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

`Ticket 15315`__ describes the following, small feature addition:
    
    Since Django 1.2 model forms can override widgets by specifying a ``'widgets'`` attribute in ``Meta``, similar to ``'fields'`` or ``'exclude'``. The ``modelform_factory`` function doesn't accept widgets, so the only way to specify it is by defining a new parent ``ModelForm`` with ``Meta`` and giving it as the ``'form'`` argument. This is more complex than it needs to be.

    The fix is to add a new keyword argument ``widgets=None`` to ``modelform_factory``.
    
In order to resolve this ticket, we'll modify the ``modelform_factory`` function to accept a ``widgets`` keyword argument. Before we make those changes though, we're going to write a test to verify that our modification actually functions correctly and continues to function correctly in the future.

Navigate to the Django's ``tests/regressiontests/model_forms_regress/`` folder and open the ``tests.py`` file. Find the :class:`FormFieldCallbackTests` class on line 264 and add the ``test_factory_with_widget_argument`` test to it as shown below:

.. code-block:: python
    
    class FormFieldCallbackTests(TestCase):
    
        def test_factory_with_widget_argument(self): 
            """ Regression for #15315: modelform_factory should accept widgets 
                argument 
            """ 
            widget = forms.Textarea() 
            
            # Without the widget, the form field should not have the widget set to a textarea 
            Form = modelform_factory(Person) 
            self.assertNotEqual(Form.base_fields['name'].widget.__class__, forms.Textarea) 
            
            # With a widget, the form field should have the widget set to a textarea 
            Form = modelform_factory(Person, widgets={'name':widget}) 
            self.assertEqual(Form.base_fields['name'].widget.__class__, forms.Textarea)
        
        def test_baseform_with_widgets_in_meta(self):
            ...

This test checks to see if the function ``modelform_factory`` accepts the new widgets argument specifying what widgets to use for each field. It also makes sure that those form fields actually use the specified widgets. Now we have the test for our patch written.

__ https://code.djangoproject.com/ticket/15315

Running your new test
~~~~~~~~~~~~~~~~~~~~~

Remember that we haven't actually made any modifications to ``modelform_factory`` yet, so our test is going to fail.  Let's run all the tests in the ``model_forms_regress`` folder to make sure that's really what happens. From the command line, ``cd`` into the Django ``tests/`` directory and run: 

.. code-block:: bash

    ./runtests.py --settings=test_sqlite model_forms_regress

If tests ran correctly, you should see that one of the tests failed with an error. Verify that it was the new ``test_factory_with_widget_argument`` test we added above, and then go on to the next step.

If all of the tests passed, then you'll want to make sure that you added the new test shown above to the appropriate folder and class. It's also possible that you have a second copy of Django on your ``PYTHONPATH`` that is taking priority over this copy, and thus it may actually be that copy of Django whose tests are being run. To check if this might be the problem, refer to the section `Setting Django up to run the test suite`_.

Making your changes to Django
-----------------------------

Next we'll actually add the functionality described in `ticket 15315`__ to Django.

Fixing ticket 1315
~~~~~~~~~~~~~~~~~~

Navigate to the ``trunk/django/forms/`` folder and open ``models.py``. Find the ``modelform_factory`` function on line 370 and modify it so that it reads as follows:

.. code-block:: python

    def modelform_factory(model, form=ModelForm, fields=None, exclude=None, 
                          formfield_callback=None,  widgets=None):
        # Create the inner Meta class. FIXME: ideally, we should be able to 
        # construct a ModelForm without creating and passing in a temporary 
        # inner class. 
     
        # Build up a list of attributes that the Meta object will have. 
        attrs = {'model': model} 
        if fields is not None: 
            attrs['fields'] = fields 
        if exclude is not None: 
            attrs['exclude'] = exclude
        if widgets is not None: 
            attrs['widgets'] = widgets 
     
        # If parent form class already has an inner Meta, the Meta we're 
        # creating needs to inherit from the parent's inner meta. 
        parent = (object,) 
        if hasattr(form, 'Meta'): 
            parent = (form.Meta, object) 
        Meta = type('Meta', parent, attrs) 
     
        # Give this new form class a reasonable name. 
        class_name = model.__name__ + 'Form' 
     
        # Class attributes for the new form class. 
        form_class_attrs = { 
            'Meta': Meta, 
            'formfield_callback': formfield_callback 
        } 
     
        form_metaclass = ModelFormMetaclass 
     
        if issubclass(form, BaseModelForm) and hasattr(form, '__metaclass__'): 
            form_metaclass = form.__metaclass__ 
     
        return form_metaclass(class_name, (form,), form_class_attrs) 
    
Notice that we're only actually adding three new lines of code. We've added a new ``widgets`` keyword argument to the function's signature, and then two more lines which set ``attrs['widgets']`` to the value of that argument whenever it's supplied.

Verifying your test now passes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Once you're done modifying ``modelform_factory``, we need to make sure that the test we wrote earlier passes now. To run the tests in the ``model_forms_regress`` folder, ``cd`` into the Django ``tests/`` directory and run: 

.. code-block:: bash

    ./runtests.py --settings=test_sqlite model_forms_regress

If everything passes, then we can move on. If it doesn't, make sure you correctly modified the ``modelform_factory`` function as shown above and copied the ``test_factory_with_widget_argument`` test correctly.

__ https://code.djangoproject.com/ticket/15315

Running Django's test suite (again)
-----------------------------------

Once you've verified that your patch and your test work correctly, it's a good idea to run the entire Django test suite to verify that your change hasn't introduced any bugs in other areas of Django. While successfully passing the entire test suite doesn't guarantee your code is bug free, it does help identify many bugs and regressions which otherwise would go unnoticed.

To run the entire Django test suite, ``cd`` into the Django ``tests/`` directory and run: 

.. code-block:: bash

    ./runtests.py --settings=test_sqlite

As long as you don't see any failures, you're good to go, and you can move on to generating a patch file that can be uploaded to Trac. If you do have any failures, you'll need to examine them in order to determine whether or not the modifications you made caused them.

Generating a patch file
-----------------------

Now it's time to actually generate a patch file that can be uploaded to Trac or applied to another copy of Django.

First, you'll need to navigate to your root Django directory (that's the one that contains ``django``, ``docs``, ``tests``, ``AUTHORS``, etc.). This is where we'll generate the patch file from.

Creating a patch file with Subversion
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you used Subversion to checkout your copy of Django, you can get a look at what your patch file will look like by running the following command:

.. code-block:: bash

    svn diff
    
This will display the differences between your current copy of Django (with your changes) and the revision that you initially checked out earlier in the tutorial. It should look similar to (but not necessarily the same as) the following:

.. code-block:: diff

    Index: django/forms/models.py
    ===================================================================
    --- django/forms/models.py  (revision 16658)
    +++ django/forms/models.py  (working copy)
    @@ -368,7 +368,7 @@
         __metaclass__ = ModelFormMetaclass
     
     def modelform_factory(model, form=ModelForm, fields=None, exclude=None,
    -                       formfield_callback=None):
    +                       formfield_callback=None, widgets=None):
         # Create the inner Meta class. FIXME: ideally, we should be able to
         # construct a ModelForm without creating and passing in a temporary
         # inner class.
    @@ -379,7 +379,9 @@
             attrs['fields'] = fields
         if exclude is not None:
             attrs['exclude'] = exclude
    -
    +    if widgets is not None: 
    +        attrs['widgets'] = widgets
    +    
         # If parent form class already has an inner Meta, the Meta we're
         # creating needs to inherit from the parent's inner meta.
         parent = (object,)
    
    Index: tests/regressiontests/model_forms_regress/tests.py
    ===================================================================
    --- tests/regressiontests/model_forms_regress/tests.py      (revision 16658)
    +++ tests/regressiontests/model_forms_regress/tests.py      (working copy)
    @@ -263,6 +263,20 @@
     
     class FormFieldCallbackTests(TestCase):
     
    +    def test_factory_with_widget_argument(self): 
    +        """ Regression for #15315: modelform_factory should accept widgets 
    +            argument 
    +        """ 
    +        widget = forms.Textarea() 
    +        
    +        # Without a widget should not set the widget to textarea 
    +        Form = modelform_factory(Person) 
    +        self.assertNotEqual(Form.base_fields['name'].widget.__class__, forms.Textarea) 
    +        
    +        # With a widget should not set the widget to textarea 
    +        Form = modelform_factory(Person, widgets={'name':widget}) 
    +        self.assertEqual(Form.base_fields['name'].widget.__class__, forms.Textarea) 
    +
         def test_baseform_with_widgets_in_meta(self):
             """Regression for #13095: Using base forms with widgets defined in Meta should not raise errors."""
             widget = forms.Textarea()

If the patch's content looks okay, you can run the following command to save the patch file to your current working directory.

.. code-block:: bash

    svn diff > myfancynewpatch.diff

You should now have a file in the root Django directory called ``myfancynewpatch.diff``. This patch file that contains all your changes.

Creating a patch file with Git
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For those who prefer using Git, the process is much the same as with Subversion.  To get a look at the content of your patch, run the following command instead:

.. code-block:: bash

    git diff
    
This will display the differences between your current copy of Django (with your changes) and the revision that you initially checked out earlier in the tutorial. It should look similar to (but not necessarily the same as) the following:
    
.. code-block:: diff

    diff --git a/django/forms/models.py b/django/forms/models.py
    index 527da5e..a93f21c 100644
    --- a/django/forms/models.py
    +++ b/django/forms/models.py
    @@ -368,7 +368,7 @@ class ModelForm(BaseModelForm):
         __metaclass__ = ModelFormMetaclass
     
     def modelform_factory(model, form=ModelForm, fields=None, exclude=None,
    -                       formfield_callback=None):
    +                       formfield_callback=None, widgets=None):
         # Create the inner Meta class. FIXME: ideally, we should be able to
         # construct a ModelForm without creating and passing in a temporary
         # inner class.
    @@ -379,6 +379,8 @@ def modelform_factory(model, form=ModelForm, fields=None, exclude=None,
             attrs['fields'] = fields
         if exclude is not None:
             attrs['exclude'] = exclude
    +    if widgets is not None: 
    +        attrs['widgets'] = widgets
     
         # If parent form class already has an inner Meta, the Meta we're
         # creating needs to inherit from the parent's inner meta.
    
    diff --git a/tests/regressiontests/model_forms_regress/tests.py b/tests/regressiontests/model_forms_regress/tests
    index 9860c2e..d2d9aa3 100644
    --- a/tests/regressiontests/model_forms_regress/tests.py
    +++ b/tests/regressiontests/model_forms_regress/tests.py
    @@ -263,6 +263,20 @@ class URLFieldTests(TestCase):
     
     class FormFieldCallbackTests(TestCase):
     
    +    def test_factory_with_widget_argument(self): 
    +        """ Regression for #15315: modelform_factory should accept widgets 
    +            argument 
    +        """ 
    +        widget = forms.Textarea() 
    +        
    +        # Without a widget should not set the widget to textarea 
    +        Form = modelform_factory(Person) 
    +        self.assertNotEqual(Form.base_fields['name'].widget.__class__, forms.Textarea) 
    +        
    +        # With a widget should not set the widget to textarea 
    +        Form = modelform_factory(Person, widgets={'name':widget}) 
    +        self.assertEqual(Form.base_fields['name'].widget.__class__, forms.Textarea) 
    +
         def test_baseform_with_widgets_in_meta(self):
             """Regression for #13095: Using base forms with widgets defined in Meta should not raise errors."""
             widget = forms.Textarea()

The Git patch format is slightly different than the Subversion format, but both are acceptable for patches.  Once you're done looking at the patch, hit the ``Q`` key to exit back to the command line.  If the patch's content looked okay, you can run the following command to save the patch file to your current working directory.

.. code-block:: bash

    git diff > myfancynewpatch.diff

You should now have a file in the root Django directory called ``myfancynewpatch.diff``. This patch file that contains all your changes.

Working together with the community
===================================

Now that you've generated your first patch, you can go on and find a ticket of your own to write a patch for. Before you do that, let's go over a few more useful points for contributing to Django.

Submitting a patch to Trac
--------------------------

Placeholder...

Applying an existing patch to Django
------------------------------------

Placeholder...


.. note::
    
    Some of this text comes from Django's documentation on :doc:`submitting patches</internals/contributing/writing-code/submitting-patches>`

# Alternately, have a conclusion section which contains summary and how to submit a patch to django, and then move "applying an existing patch" to an early section.