Testing & Code Generation: Generate function call permutations based on arguments

Hello all,

i’m kinda busy the last 2 years in a big home project which is evolving every day. New features are added, more web services written, more complicated database queries are formed…and after 2 years, this small baby has grown to a 17K lines of code distributed in more than 150 files.When i started it, i asked myself, “Should i write tests?” and after 0 seconds later i ofc took the decision to add tests for each feature in the project. Thus, a feature is considered complete, only if & only if there are appropriate tests for it, accompanied ofc with comments & encapsulated in test functions/cases with self-described names like “test_add_user_with_duplicated_email()” in order to know what this scenario/test case/function is about (yeah, if you write tests for each feature, by only looking at the test function names, all supported features of the system are revealed before your eyes 🙂 ).

OK, tests are good!

But…you need to admit it that OMG it is fucking boring sometimes! You need to think all corner cases, write much much mock code to simulate the behavior of the environment in which your feature will have an impact & you are really enthusiastic with continuing with the implementation of the next feature.

I am sure that many of you, when writing tests for a function, you often call the function multiple times by using different arguments & you assert the result of the function. In the faulty argument cases, you assert that the function returned false & true otherwise. Well, imagine you are testing a function that takes 4 arguments, well this is 2^4 = 16 different argument permutations for the function. What if the function takes 8, then it is 2^8 = 256 different permutations!! Are you going to write by hand 255 function calls with different arguments plus 1 correct in the end?

No man, i really have better things to do than this!

You know what i did? I wrote a program that generates all 256 function calls for me. And of course the amount of arguments could be arbitrary ofc.

def generate_truth_table(perms):
    b = str( bin ( perms ) )
    l = len(b)  1
    t = {}
    for i in xrange(0,perms):
        _b = str ( bin  (i) )
        s = ‘%s%s’ % ( ( l  len(_b ) ) * ‘0’ , _b)
        s = s.replace(‘b’,)
        s = ‘%s’ % s[1:]
        #print s
        t[i] = [int(c) for c in s]
    return tdef generate_function_call_permutation(func,args,indent=1):
    perms = pow(2, len(args) )
    t = generate_truth_table(perms )
    keys = t.keys()
    #print t,perms
    known_prefix = ‘known_’
    unknown_prefix = ‘unknown_’    func_perm = []
    for i in xrange(0,len(keys)):
        k = keys[i]
        #print k
        l = t[k]
        assert len(l) == len(args)
        gen_args = []
        for j in xrange(0,len(l)):
            arg = 
            if l[j] == 0:
                arg = ‘%s%s’ % (unknown_prefix,args[j])
                arg = ‘%s%s’ % (known_prefix,args[j])
        #print gen_args
        if i < len(keys) -1:
            current_func = ‘assert<b> </b>%s(%s)<b> </b>==<b> </b>False<b> </b>#%s’ % (func,‘,’.join(gen_args),‘,’.join([str(j) for j in t[k]]) )
            current_func = ‘assert<b> </b>%s(%s)<b> </b>#%s’ % (func,‘,’.join(gen_args),‘,’.join([str(j) for j in t[k]]) )
        #print current_func;exit(1)
        gen_args = []    for i in xrange(0,len(func_perm)):
        print indent*3*‘<b> </b>’,func_perm[i]
if __name__ == ‘__main__’:
    func = ‘ap.add_url’
    args = [‘artist_id’,‘desc’,‘url’]


The depicted python program (generate_function_call_permutations.py ) by getting as input a function name along with a list of the names of the function arguments, it generates first a truth table based on the amount of arguments:

for 3 arguments:

2^3 = 8 argument permutations


For each row of the previous table

For each argument:

If 0 is encountered, add the prefix “unknown_” in the argument’s name. Otherwise, add the prefix “known_” to the argument’s name.

With that way, we generate each tested function call based on the generated truth table & we assign the appropriate argument name based on the table.

Let’s see what our generator produces based on the previous output (func = ‘ap.add_url’ , args = [‘artist_id’,‘desc’,‘url’])

    assert ap.add_url(unknown_artist_id,unknown_desc,unknown_url) == False #0,0,0
    assert ap.add_url(unknown_artist_id,unknown_desc,known_url) == False #0,0,1
    assert ap.add_url(unknown_artist_id,known_desc,unknown_url) == False #0,1,0
    assert ap.add_url(unknown_artist_id,known_desc,known_url) == False #0,1,1
    assert ap.add_url(known_artist_id,unknown_desc,unknown_url) == False #1,0,0
    assert ap.add_url(known_artist_id,unknown_desc,known_url) == False #1,0,1
    assert ap.add_url(known_artist_id,known_desc,unknown_url) == False #1,1,0
    assert ap.add_url(known_artist_id,known_desc,known_url) #1,1,1


How to use it?

  • declare known_* & unknown_* variables & assign them a correct/expected value & an incorrect value respectively
  • copy & paste the generated code
  • & you are done!


def test_artist_portfolio(db_name=None):
    reset_db(db_name=db_name)    db = db_builder(db=db_name)
    attendees = 1000
    DIY = 1
    _city = ‘Athens’
    country = ‘Greece’
    name = ‘NoMeansNo’
    continent_id = 3
    country_id = 300
    state = 
    country_id = 300    setup_mock_city(‘Athens’,state,country_id,continent_id,db=db)
    setup_mock_artist(name,country,attendees,DIY,_city,db=db)    ap = artist_portfolio(db=db)

    known_artist_id, unknown_artist_id = 1, 2
    known_desc, unknown_desc = ‘this<b> </b>a<b> </b>description<b> </b>of<b> </b>the<b> </b>url’,
    known_url, unknown_url = https://www.youtube.com/watch?v=SIUmApXqJAk&#8217;,

    assert ap.add_url(unknown_artist_id,unknown_desc,unknown_url) == False #0,0,0
    assert ap.add_url(unknown_artist_id,unknown_desc,known_url) == False #0,0,1
    assert ap.add_url(unknown_artist_id,known_desc,unknown_url) == False #0,1,0
    assert ap.add_url(unknown_artist_id,known_desc,known_url) == False #0,1,1
    assert ap.add_url(known_artist_id,unknown_desc,unknown_url) == False #1,0,0
    assert ap.add_url(known_artist_id,unknown_desc,known_url) == False #1,0,1
    assert ap.add_url(known_artist_id,known_desc,unknown_url) == False #1,1,0
    assert ap.add_url(known_artist_id,known_desc,known_url) #1,1,1


Now go run your tests & if OK, go & implement some features, don’t waste your time in writing the same function over & over again!

Hope you enjoyed!



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s