This describes a bare bones installation of Django on a USB stick using Movable Python. Installing a database of choice to the stick is left as an exercise for the reader.
1) Install a version of Movable Python (a trial version is available). This just goes onto your USB stick where you like. You'll have an executable, movepyw.exe that you'll run to start Python programs using Movable Python's version of Python rather than whatever may or may not be installed on the machine.
2) Get a suitable distribution of Django and put it someplace temporary. Unzip if necessary.
3) From the Django distribution, copy the django subdirectory and everything under it into the lib directory in Movable Python's directory structure. This is Movable Python's equivalent of site-packages.
4) Edit syspaths.pth in Movable Python's lib directory to add in the lines:
Note: with Portable Python 1.0 BETA create 'syspaths.pth' in 'Lib\site-packages'
5) You can delete your original temporary copy of the Django distribution.
You should now be able to run movepyw.exe, select django-admin.py and pass in the command line arguments for creating a site (e.g. startproject mysite). The mysite directory will go into one of two places...if you have the "f" checkbox checked, it'll go into the same directory as django-admin.py. If "f" is not checked, it'll go into the same directory as movepyw.exe.
Using movepyw.exe again, run manage.py from the newly created site and pass it "runserver" as the command line argument. If all of the above worked, the server should start as usual.