Oracle, Web, Script, SQLserver, Tips & Trick

July 16, 2011

Writing a domain specific language DSL with python

Filed under: phyton — sikathabis @ 5:14 am

Writing a domain specific language DSL with python

A domain-specific language is a piece of software designed to be useful for a specific task in a fixed problem domain, they’re gaining popularity because they enhance productivity and reusability of artifacts. DSLs also enable expression and validation of concepts at the level of abstraction of the problem domain, this approach is very useful when you need to describe a user interface, a business process, a database, or the flow of information.

// <![CDATA[
google_ad_client = “pub-9202186284112152”; /* 300×250 blog content */ google_ad_slot = “4279590471”; google_ad_width = 300; google_ad_height = 250;
// ]]>

The DSL concept isn’t new after all, special-purpose programming languages and all kinds of modeling, specification languages have always existed, but this term rise due the popularity of domain-specific model.

You can easily implement dsls using the ruby language, Java or even C# if you prefer, but this isn’t the main propose of this article. The sine qua non become visible when I was implementing a simple test case with python. Indeed, there are a lot of python BDD-like frameworks, mostly who are self claimed the silver bullet, that are mismatching a lot of basic principles, but like I said, we are talking about dsls =)

With python we can easily create a piece of software that expresses some basic desired behavior, like rspec does, but much more pythonic.

# coding: pyspec
    class Bow:
        def shot(self):
            print "got shot"

        def score(self):
            return 5

    describe Bowling:
        it "should score 0 for gutter game":
            bowling = Bow()
            assert that bowling.score.should_be(5)

we can easily make this test dsl a runnable piece of python code, whiteout writing incompressible regular expressions, just using the python codecs and tokenizer.

Fist of all we need to define a new encoding for pyspec – our pre defined spec file syntax – this neat hacking enables a new path to tokenize this file

import tokenize
    import codecs, cStringIO, encodings
    from encodings import utf_8

    class StreamReader(utf_8.StreamReader):
        def __init__(self, *args, **kwargs):
            codecs.StreamReader.__init__(self, *args, **kwargs)
            data = tokenize.untokenize(translate(
   = cStringIO.StringIO(data)

    def search_function(s):
        if s!='pyspec': return None
        utf8=encodings.search_function('utf8') # Assume utf8 encoding
        return codecs.CodecInfo(
            encode = utf8.encode,
            decode = utf8.decode,


Our tiny translate function defines a easy way to translate both describe and it into a traditional python class and method definition.

def method_for_it(token):
        return token.strip().replace(" ", "_").replace("\"","" ) + "(self)"

    def translate(readline):
        previous_name = ""
        for type, name,_,_,_ in tokenize.generate_tokens(readline):
            if type ==tokenize.NAME and name =='describe':
                yield tokenize.NAME, 'class'
            elif type ==tokenize.NAME and name =='it':
                yield tokenize.NAME, 'def'
            elif type == 3 and previous_name == 'it':
                yield 3, method_for_it(name)
                yield type,name
            previous_name = name

Clever isn’t ? Now, fell yourself free to fork this project on GitHub and finish the job =) maybe someday we can have a real BDD python framework.


  1. [codecs — Codec registry and base


  1. [tokenize — Tokenizer

for Python source][6]


Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at

Learning is not a spectator sport

Connor McDonald's Oracle blog

Teknologi Pendidikan

Just another weblog


Jurnalisme Yang Dapat Dipercaya

Qey050489's Blog

Ayo Bangkit sobat..!!!!!

B. S. Totoraharjo

Orang Bodoh Yang Tak Kunjung Pandai

culturekitchen foodlab

New Food and Art in Yogyakarta

Wahyu Catur Wibowo

Musafir Dunia Maya

SQL.... Still Learning

My Reference on SQL Server


"A distance is getting far if your heart is impatient" -ean-


Street Photography is like a box of a chocolates, you're never know what you're gonna get - Thomas Leuthard -

Keluarga Andrivito

-- Andrivito's Journal --


This site is about Latest NEWS, Articles, Videos, Books, Course, Presentations, Tips & Tricks related to I.T Pro's

Gaurav Saxena

|| तत् त्वम् असि ||

Be a good Moeslim

Menuju Pribadi Muslim Sejati yang Bertaqwa


Manajemen Sholat, renungan spiritual, Teknologi Informasi

Pelangi di khatulistiwa....

Mari berbagi kebaikan, pengalaman dan senyum denganku


Ceritanya saya, Pikirannya saya, Hatinya saya


sikathabis semua informasi disini

%d bloggers like this: