The Best New Feature in unittest You Didn’t Know You Need

From time to time I like to read documentation of modules I think I know well. The python documentation is not a pleasant read but sometimes you strike a gem.

Same thing but slightly different Distinguishing Test Iterations

Let’s start with a simple function to check if a number is even

def is_even(n): return n % 2 == 0

And a simple test

class TestIsEven(TestCase): def test_should_be_even(self): self.assertTrue(is_even(2))

Nice, let’s add some more cases:

class TestIsEven(TestCase): ... def test_zero_should_be_even(self): self.assertTrue(is_even(0)) def test_negative_should_be_even(self): self.assertTrue(is_even(-2))

This is a simple example and we copied code three times. Let’s try to do better by writing a loop to iterate values we expect to be even:

class TestIsEven(TestCase): def test_should_all_be_even(self): for n in (2, 0, -2, 11): self.assertTrue(is_even(n))

This is starting to look more elegant, so what is the problem? I added an odd value, 11, to fail the test. Let’s run the test and see what it looks like:

F =================================================== FAIL: test_should_all_be_even (__main__.TestIsEven) ― ― ―― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― Traceback (most recent call last): File “subtest.py”, line 18, in test_should_all_be_even self.assertTrue(is_even(n)) AssertionError: False is not true

It failed as expected, but which value failed ?

Enter subTest

In python 3.4 there is a new feature called subTest . Lets see it in action:

class TestIsEven(TestCase): def test_should_all_be_even(self):
for n in (0, 4, -2, 11):
with self.subTest(n=n):
self.assertTrue(is_even(n))

Running this test produces the following output:

F
==========================================================
FAIL: test_should_all_be_even (__main__.TestIsEven) (n=11)
― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― Traceback (most recent call last): File “subtest.py”, line 23, in test_should_all_be_even self.assertTrue(is_even(n)) AssertionError: False is not true

So which value failed? 11! I t’s in the title .

How multiple failures look like?

F
===========================================================
FAIL: test_should_all_be_even (__main__.TestIsEven) (n=3)
― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― Traceback (most recent call last): File “subtest.py”, line 23, in test_should_all_be_even self.assertTrue(is_even(n)) AssertionError: False is not true ==========================================================
FAIL: test_should_all_be_even (__main__.TestIsEven) (n=5)
― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― Traceback (most recent call last): File “subtest.py”, line 23, in test_should_all_be_even self.assertTrue(is_even(n)) AssertionError: False is not true ==========================================================
FAIL: test_should_all_be_even (__main__.TestIsEven) (n=11)
― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― ― Traceback (most recent call last): File “subtest.py”, line 23, in test_should_all_be_even self.assertTrue(is_even(n)) AssertionError: False is not true

Exactly as if we wrote three separate test cases.

Profit!