= Visualizing Django source code changeset dependencies =

When following the workflow I describe in '''![1]''' to be able to maintain a local copy of Django based on the 0.96 release with selected changes backported from the SVN trunk tip I soon found myself in following situation:

Certain changeset ''C'' that I'd want to use wasn't being considered by '''![2]'''{{{darcs}}} as a potential candidate so it didn't offer me to apply it.

Later the cause became evident: The ''C'' changeset wasn't being considered by {{{darcs}}} because I had previously answered "no" when it offered me to apply a (chronologically previous) changeset ''B'' on which ''C'' depends (dependency in this context means both ''B'' and ''C'' modify some overlaping section of some file).

Given the fact that in {{{darcs}}} patches (changesets, patchsets) are first class citizens, it is relatively easy (when compared with other revision control systems) to track and visualize the dependencies among them by using it together with some related '''![3]'''tools and '''![4]'''{{{Graphviz}}}.

If we add a couple of ad-hoc scripts and some tips from the experience gained to the mix we can get something like this (partial output):

{{{
#!html
<p style="text-align: center;">
<img src="http://rmorales.net/media/django-deps/django-deps-r5165-partial.png" alt="django deps r5165, partial, PNG format" />
</p>
}}}

== Scripts ==

=== filter1.sh ===
{{{
#!/bin/sh

# This filter changes the labels of the nodes of a Graphviz dot
# file as generated by darcs-deps to be just the patchset
# number.
#
# It depends on the labels being of the form
# "[tailor-project-name @ changeset-number]"
#
# It needs one command line parameter: The tailor-project-name
#

if test $# -ne 1; then
    echo "Usage:"
    echo "$(basename $0) tailor-project-name"
    exit 1
fi

/bin/sed "s/\[$1 @ \\([0-9]\+\\)\]/\1/g"
}}}

=== filter2.sh ===
{{{
#!/bin/sh

# This filter modifies the properties of the nodes of a Graphviz dot
# file to add a relative URL "href" property with the same value as
# the node label.
#
# It depends on the labels being the number of the revision/changeset
# represented by the node (possibly generated by filtering the darcs-deps
# output through filter1.sh).
#
# It has been created to facilitate the navigation of the Django trunk
# changeset dependancy graph, enabling (for the relevant output formats: pdf,
# image maps) the user to click on a node and getting a web browser window
# opened with the Django Trac page showing the corresponding changeset.
#
# For this to work, a graph level property href=http://code.djangoproject.com/changeset/
# must be also present, it can be added with a -Ghref=... commandline switch
# in the dot(1) invocation.

if test $# -ne 0; then
    echo "Usage:"
    echo "$(basename $0)"
    exit 1
fi

/bin/sed "s#label = \"\\([0-9]\+\\)\"#label = \"\1\", href=\"\1\"#g"
}}}

== Usage ==

{{{
ramiro@tabaqui:~/src/django/inmutable $ darcs-deps | filter1.sh django096 | filter2.sh | unflatten | dot -Tps2 -Gsize=50,50 -Ghref=http://code.djangoproject.com/changeset/ -o ~/django-deps.ps -Tpng -o ~/django-deps.png
Examining patch 249 of 249.

ramiro@tabaqui:~/src/django/inmutable $ cd

ramiro@tabaqui:~ $ poster -p1x3A4 -o django-deps-mpage.ps django-deps.ps

ramiro@tabaqui:~ $ ps2pdf django-deps-mpage.ps django-deps.pdf
}}}

=== Explanation ===

{{{darcs-deps}}} is ran on the {{{inmutable}}} {{{darcs}}} repository (please refer to '''![1]''' to see what that repository is and how to create it), it generates a {{{Graphviz}}} dot language description of the dependancy graph.

We filter that output first through our two custom filtering scripts (see above) to change some graph node attributes, and then through {{{unflatten(1)}}} (part of the {{{Graphviz}}} suite) to enhance the layout of the graphic representation.

Finally, we feed the result to {{{dot(1)}}} (also part of {{{Graphviz}}}) to get our final output. For some output methods (PDF [via Postscript], server/client side image maps) we'll be able to click on the nodes and get our Web browser to go to the Django Trac page associated with the changeset at hand.

For the PDF output we use {{{poster(1)}}} on the intermediate Postscript output to create a legible non-cropped multi-page output.

== Files ==

Generated as of revision [5165] (May 8 2007 3:43 UTC).

Some files are hosted elsewhere while the Trac crew solve the [http://trac.edgewall.org/ticket/4087 bug] that keeps us from attaching binary files to the Wiki and tickets.

=== DOT files ===

 * Graph as generated by {{{darcs-deps}}}, before post-processing: [attachment:django-deps-r5165-orig.dot django-deps-r5165-orig.dot]
 * After post-processing, before being feed to {{{dot}}}: [attachment:django-deps-r5165-final.dot django-deps-r5165-final.dot]

=== Example output ===

 * PNG output: [http://rmorales.net/media/django-deps/django-deps-r5165.png django-deps-r5165.png]
 * PDF output: [http://rmorales.net/media/django-deps/django-deps-r5165.pdf django-deps-r5165.pdf]

== Notes ==

 * The "Tailorization" node represents an initial synthetic patch created by {{{darcs}}} from which every other leaf node depends on, perhaps it would be better to find a way to eliminate it from the graph.
 * I attempted without success modifying the {{{darcs-deps}}} script to eliminate the nodes that have no other nodes depending on them, next try will be using {{{Graphviz}}}'s {{{gvpr(1)}}} tool.
 * The PDF output for this iteration has some problems, something that didn't happen before; it seems there is some non-deterministic interaction between {{{poster}}} and the {{{dot}}} PS output we feed to it. It would be nice to be able to create multi-page output  directly from {{{dot}}}, ideas welcome.

== References ==

 1. [http://seeonee.homeip.net:81/intdoc/DjangoPost096 Django: My post 0.96 branch]
 2. [http://darcs.net darcs]
 3. [http://scratchbox.org/~ttimonen/repos/darcs-deps/darcs-deps darcs-deps] (darcs repo: http://scratchbox.org/~ttimonen/repos/darcs-deps) (Perl script)
 4. [http://www.graphviz.org/ GraphViz]
 5. [http://www.cs.wisc.edu/~ghost/ Ghostscript]
 6. [ftp://ftp.ics.ele.tue.nl/pub/users/jos/poster/poster.tar.gz poster]

== Feedback ==

Please direct all feedback to [mailto:cramm0_at_gmail.com Ramiro Morales].