Using Python's doctest Feature

Python has a neat automated tool for doing unit testing of a function or method called doctest. The doctest module searches for pieces of text, e.g., inside docstrings, that look like interactive Python sessions, and then executes those sessions to verify that they work exactly as shown.

For more details see https://docs.python.org/3/library/doctest.html

How to use the doctest feature?

1. Enter in the docstring of a function or method what looks like interactive Python sessions. Here is a sample with the two tests in bold:

   
   def date(month, day, year):
       ''' Returns a string of the date given integers for month, day, and year.
   
           Two doctests to unit test the function 
       >>> date(3, 29, 2015)
       '03/29/2015'
       >>> month= 4
       >>> day = 1
       >>> date(month, day, 2015)
       '04/1/2015'
   
       The blank line above delimits the expected output of the second test.
       '''
   
       return "{:02d}/{:02d}/{:04d}".format(month, day, year)
   
   import doctest
   doctest.testmod()  # test the whole module, i.e., this file

2. Add the following two lines at the end of the file so they will be run. Make sure to line up the code all the way to the left.

   import doctest
   doctest.testmod()

   then run the program in IDLE. The doctest module will run each line with '>>> ' and check that the function returns the same text as the expected output (if any) that is on the line after '>>> '. If the output from the run and the expected output differ even by one character, the doctest module prints a "failed test" message.

   The doctests MUST be in the first docstring after the function or method header. The doctest module expects the test to appear as if you had copied and pasted the test and output from a run in the Python interactive session. For example, it expects the lines to be properly indented in line with the quote mark of the docstring, it expects one space after the '>>>', and any expected output must immediately follow the final '>>> ' line containing the code. For example, you can't insert a comment or a blank line between the '>>> ' line and expected output. The doctest module considers the expected output (if any) to extend to the next '>>> ' or an all-whitespace line in the docstring (The all-whitespace line is not included in the expected output.).

   You may use several '>>> ' lines to set up the function before any output as shown above by setting month and day in the second test. Also, you may have many doctests in one docstring.

3. If all the tests pass, the doctest module doesn't print anything. Here is the output from the run of the above code.

   
   >>> 
   **********************************************************************
   File "__main__", line 9, in __main__.date
   Failed example:
       date(month, day, 2015)
   Expected:
       '04/1/2015'
   Got:
       '04/01/2015'
   **********************************************************************
   1 items had failures:
      1 of   4 in __main__.date
   ***Test Failed*** 1 failures.
   >>> 
   
   

   Here we observe that the second test's expected output 04/1/2015 is wrong and should be changed to 04/01/2015. After making that change in the second test, here is the output of a run of the modified code:

   
   >>> 
   >>> 
    

Many professional programmers write the unit tests in the docstring before they write any code in the function. Then they design and develop the code for the function until the function passes all the tests. This is a good habit to emulate!

Why use doctests?

If you already have test cases before you have written the code for a function, you can save a lot of time typing in tests with doctests. However, there are several more important reasons to use doctests. First, they document the function's good tests and aid in understanding what the function does. This is exceedingly valuable for someone (That someone might be you!) who might want to reuse your function in a different program. Or if that someone has to repair your code for some reason. A third reason -- It's easy to make modifications in one part of program that breaks something somewhere else. With the doctests in place, all you need to do is rerun the doctests to determine if anything was damaged.

When NOT to use doctests?

• When the output of a function or method may change for the same input such as when the function uses random numbers. The doctest module executes the text that looks like an interactive Python session and verifies the output is exactly the same. Even one character difference will cause the test to fail.
• When the function or method uses user input as with an input function. The docstring module doesn't expect to have user input.
• When the function or method uses graphics such as turtle graphics. The docstring module only checks what the function or method returns not what is drawn or displayed on a graphics screen. It can't tell if your turtle moved left or right!