Django has textile, markdown and other markup languages as filters in the templates. But those filters can't do complex operations. For example we want such tags in our page content:

[art id="34"]
[art id="11"]

And we want them to become in the browser:

- <a href="/link/to/article34/">article title34</a><br>
- <a href="/link/to/article11/">article title11</a><br>

Something like BBcode but with the ability to respond do attributes passed in the tags. From some time I was using such tags in my PHP scripts. I called them “ContentBBcode". Now the time have come to djangoify the concept.

The Code – The Implementation

I've written the parser that finds and replaces ContentBBcode tags (CBC for short) with proper response, and I won't go to details about it. Put this code in a file and save it as cbcparser.py (I saved it in the application folder in which I want to use it):

import re
import sys
sys.path.append('wiki/cbcplugins/')

def parse_cbc_tags(text):
	# double: [tag]something here[/tag]
	tags = re.findall( r'(?xs)\[\s*rk:([a-z]*)\s*(.*?)\](.*?)\[(?=\s*/rk)\s*/rk:(\1)\s*\]''', text, re.MULTILINE)
	parsed_double = {}
	for tag in tags:
		k = str(tag[0]).strip()
		v = tag[1]
		v = v.split(' ')
		vals = {}
		vals['attributes'] = {}
		for attr in v:
			attr = attr.split('=')
			val = attr[1]
			attr[1] = val[1:-1]
			vals['attributes'][attr[0]] = attr[1]
		vals['code'] = tag[2]
		vals['tag'] = '[rk:' + tag[0] + ' ' + tag[1] + ']' + tag[2] + '[/rk:' + tag[0] + ']'
		if not parsed_double.has_key(k):
			parsed_double[k] = list()
		parsed_double[k].append(vals)
	
	for plugin in parsed_double:
		try:
			exec 'from ' + plugin + ' import *'
		except:
			pass
		else:
			text = render(parsed_double[plugin], text)
			
	# single: [tag]
	tags = re.findall('\[rk:([a-z]*) ([a-zA-z0-9 =.,"\']*)\]', text)
	parsed = {}
	for tag in tags:
		k = str(tag[0]).strip()
		v = tag[1]
		v = v.split(' ')
		vals = {}
		vals['attributes'] = {}
		for attr in v:
			attr = attr.split('=')
			val = attr[1]
			attr[1] = val[1:-1]
			vals['attributes'][attr[0]] = attr[1]
		vals['tag'] = '[rk:' + tag[0] + ' ' + tag[1] + ']'
		if not parsed.has_key(k):
			parsed[k] = list()
		parsed[k].append(vals)
	
	for plugin in parsed:
		try:
			exec 'from ' + plugin + ' import *'
		except:
			pass
		else:
			text = render(parsed[plugin], text)
	return text

Note on the line:

sys.path.append('wiki/cbcplugins/')

each tag for this parser is a python file ([art...] would need art.py). If don't put your plugins in PYTHONPATH then you need to append location of the plugin folder to PYTHONPATH. In my case wiki/ was my application folder and wiki/cbcplugins/ folder with plugins. Change the path to fit your needs.

Now we will make a template filter out of it. Create templategas folder in your application folder and create empty init.py file and cbc.py file with the code:

from project.aplication.cbcparser import *
from django import template

register = template.Library()

def cbc(value): # Only one argument.
    return parse_cbc_tags(value)

register.filter('cbc', cbc)

Where from project.aplication.cbcparser import * is the cbcparser.py importing. In my case it is from diamanda.wiki.cbcparser import *

And we are done. In a template in which you want to use it put

{% load cbc %}

and then to parse a string you just use:

{{ mystring|cbc }}

Now about the plugins. The parser tries to load them and if it succeeds it will call render function passing two variables – a dictionary with parsed data from the tag and the string. The tag should look like:

[rk:tagname attr1="value1" attr2="value2"]

Where tagname is the name of the tag and name of the plugin filename (tagname.py).

A basic plugin code would look like this:

def render(dic, text):
	for i in dic:
		text = text.replace(i['tag'], '<h1>Article ID ' + i['attributes']['id'] + '</h1>')
	return text

for a CBC:

[rk:art id="value"]

dic is a dictionary which has few dictionaries in it. attributes has all the attributes (as dictionaries), tag is a string containing the tag code which we replace.

attributes[ 'attrname' ] gives you value from given attribute from the tag ( attributes[ 'attr1' ] would give you value1). In the end the “rk:art" tag would become:

<h1>Article ID *value*</h1>

The parser supports also double tags:

[rk:tagname attr="value"]code here[/rk:tagname]

They work the same as single-tags with one difference – the code between tags is also available in the plugin as dic[ 'code' ]. For example for a tag:

[rk:codder lang="python"]a code here[/rk:codder]

We would create codder.py plugin:

def render(dic, text):
	for i in dic:
		text = text.replace(i['tag'], '<B>'+ i['attributes']['lang'] +'</B><pre><code>' + i['code'] + '</code></pre>')
	return text

Usage

• as a wrappers for JavaScript and similar code/widgets

• as a markup tags that need to get data from somewhere (database etc.)

Real Example

Now we will make a plugin for ​dp.syntaxhighlighter – a javascript based code highlighter. We have our codder.py plugin but it doesn't do any usefull things.

• Download the script archive and extract it to an empty folder
• Copy Styles and Scripts folders to your django media folder (I've placed them in MEDIA_ROOT/syntax/)
• You can open in a text editor one of examples to see how it works...

It works like this:

1. First we have call to the CSS

   <link type="text/css" rel="stylesheet" href="Styles/SyntaxHighlighter.css"></link>
   

2. Then we place code for highlighting in textareas:

   <textarea name="code" class="LANGNAME"> code here </textarea>
   

The name="code" class="LANGNAME" part make the textarea to work.

3. After all textareas we load JS files for languages we use and one main one (shCore.js):

   <script class="javascript" src="Scripts/shCore.js"></script>
   <script class="javascript" src="Scripts/shBrushCSharp.js"></script>
   <script class="javascript" src="Scripts/shBrushPhp.js"></script>
   <script class="javascript" src="Scripts/shBrushJScript.js"></script>
   <script class="javascript" src="Scripts/shBrushJava.js"></script>
   <script class="javascript" src="Scripts/shBrushVb.js"></script>
   <script class="javascript" src="Scripts/shBrushSql.js"></script>
   <script class="javascript" src="Scripts/shBrushXml.js"></script>
   <script class="javascript" src="Scripts/shBrushDelphi.js"></script>
   <script class="javascript" src="Scripts/shBrushPython.js"></script>
   <script class="javascript" src="Scripts/shBrushRuby.js"></script>
   <script class="javascript" src="Scripts/shBrushCss.js"></script>
   

4. At the end we initialize the script:

   <script class="javascript">
   dp.SyntaxHighlighter.HighlightAll('code');
   </script>
   

And thats all. Now how to make a plugin out of it? Like this:

def render(dic, text):
	# w3c will kill us for this :)
	text = '<link type="text/css" rel="stylesheet" href="/site_media/syntax/Styles/SyntaxHighlighter.css"></link>' + text
	langs = {}
	for i in dic:
		text = text.replace(i['tag'], '<textarea name="code" class="'+ i['attributes']['lang'] +'" rows="15" cols="90">' + i['code'] + '</textarea>')
		# what langs are used?
		langs[i['attributes']['lang']] = True
	
	# add the core JS
	text = text + '<script class="javascript" src="/site_media/syntax/Scripts/shCore.js"></script>'
	# add only those lang-JS files that we realy need. For example i limit it to two
	if langs.has_key('python'):
		text = text + '<script class="javascript" src="/site_media/syntax/Scripts/shBrushPython.js"></script>'
	if langs.has_key('xml'):
		text = text + '<script class="javascript" src="/site_media/syntax/Scripts/shBrushXml.js"></script>'
	# the end, activate the code
	text = text + '<script class="javascript">dp.SyntaxHighlighter.HighlightAll(\'code\');</script>'
	return text

• We changed paths to static files (JS and CSS). If you intend to use it often you could move CSS “injection" to HEAD in your template.
• Next we replace all “codder" tags with the textarea with language we want to use
• We make a dictionary which gathers all languages we use
• Next we inject the main JS file and then those language specific JS files we really need
• And at the end we add the initialization code.

Now CBC like:

[rk:codder lang="python"]
for foo in bar:
   print foo
[/rk:codder]

Will get highlighted. See a screenshot ​here