Python Unittest Tutorial and Best Practices

Introduction

Overview

At Glinteco, both unit/integration/automation/stress tests are crucial for every single project. We believe "Great Developers must write tests for his own code"

Unit testing is a fundamental practice in software development that involves testing individual units or components of a software application to ensure they work as expected. A unit is the smallest testable part of any software - usually a function or method. By isolating and testing these units, developers can identify and fix bugs early in the development cycle, leading to more robust and reliable software.

Unit tests are IMPORTANT because they:

Verify the correctness of code.
Facilitate code refactoring by ensuring that changes do not break existing functionality.
Serve as documentation for the code, illustrating how it is supposed to work.
Help catch bugs early, reducing the cost of fixing them.
Improve the source code quality

Not many senior developers can manage this well due to many reasons, however, to level up our own programming expertise, writing unit test is a must.

Purpose

This article provides a completed guide to using Python's unittest framework for writing and running unit tests (We don't cover pytest here, we do have a pytest vs unittest comparison post). Whether you're a beginner or an experienced developer, this tutorial will help you understand the basics of unittest and teach you best practices for writing effective tests.

By the end of this article, you will:

Understand the basic structure of a unittest test case.
Learn how to write, run, and organize tests.
Explore advanced features such as test fixtures, mocking, and test suites.
Discover best practices to improve the quality and maintainability of your tests.
Identify common pitfalls and how to avoid them.

Unit testing with unittest is a powerful way to ensure your code is reliable and maintainable, hence, a better product. Let's start the unittest adventure.

Getting Started with Unittest

Installation

unittest is included as a Python's standard library - you don't need to install any additional packages to get started with unit testing in Python. Simply import unittest in your script, and you're ready to write tests.

Basic Structure

The core of the unittest framework revolves around the TestCase class. A test case is created by subclassing unittest.TestCase, and individual test methods are defined within this class. Each test method should start with the word test to ensure it is automatically recognized and executed by the unittest test runner. This seems a bit not pythonic, doesn't it?

Here's the basic structure of a unittest test case, name this file as test.py

import unittest



def count_e_letters(name):

    count = 0

    for letter in name:

        if letter == 'e':

            count += 1

    return count



class TestCountELetters(unittest.TestCase):

    def test_count_e_letters(self):

        count = count_e_letters("Joe")

        self.assertEqual(1, count)



if __name__ == '__main__':

    unittest.main()

Running Tests

To run the tests, save the script and execute it from the command line:

❯ python test.py . ---------------------------------------------------------------------- Ran 1 test in 0.000s

OK

If any test fails, unittest will provide detailed information about the failure, including the expected and actual results, helping you diagnose the issue.

❯ python -m unittest tests/test_strings.py

F.

======================================================================

FAIL: test_count_e_letters_with_a_none_input (tests.test_strings.TestCountELetters.test_count_e_letters_with_a_none_input)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/joe/Documents/WORK/GLINTECO/PROJECTS/INTERNAL/samples/unittest_tutorials/tests/test_strings.py", line 13, in test_count_e_letters_with_a_none_input

    self.assertEqual(1, count)

AssertionError: 1 != 0

With these basics covered, you can start writing simple tests using unittest. In the next section, we'll delve deeper into writing test cases with setup and teardown methods to manage test environments effectively.

Writing Test Cases

Setup and Teardown

In unit testing, it is common to prepare a specific environment before running tests and clean up afterward. unittest provides setUp and tearDown methods to handle this. The setUp method is called before each test method to set up any state that's shared across tests, and the tearDown method is called after each test method to clean up. This is similar to Ruby

Here's an example:

import unittest



class TestExample(unittest.TestCase):

    def setUp(self):

        self.user = User()



    def tearDown(self):

        self.user = None



    def test_name(self):

        self.assertEqual("Joe", self.user.name)



    def test_action(self):

        self.assertEqual("Playing Game", self.user.action())

As can be seen from above:

The setUp method initializes self.user as a new user before each test.
The tearDown method cleans up by setting self.user to None after each test.
Two test methods (test_name and test_action) use self.user to perform their assertions.

What is the difference between `setUp`/`tearDown` and `setUpClass/tearDownClass`

`setUp` and `tearDown`

Purpose: setUp and tearDown methods are used to set up and clean up resources needed for each individual test method.
Execution: setUp is called before each test method runs, and tearDown is called after each test method finishes.
Scope: The setup and teardown actions in these methods apply to each test method within the TestCase class. This means that if you have multiple test methods, setUp and tearDown will run multiple times—once for each test method.

import unittest



class TestExample(unittest.TestCase):

    def setUp(self):

        self.number = 1

        print("Setting up before a test method")



    def tearDown(self):

        self.number = None

        print("Tearing down after a test method")



    def test_addition(self):

        self.assertEqual(self.number + 1, 2)



    def test_subtraction(self):

        self.assertEqual(self.number - 1, 0)

Output:

Setting up before a test method

Tearing down after a test method

Setting up before a test method

Tearing down after a test method

`setUpClass` and `tearDownClass`

Purpose: setUpClass and tearDownClass are used to set up and clean up resources needed for the entire test case class, not just for individual test methods.
Execution: setUpClass is called once before any test methods are run, and tearDownClass is called once after all test methods have finished running.
Scope: The setup and teardown actions in these methods apply to the entire test case class. This means that if you have multiple test methods, setUpClass and tearDownClass will run only once for the whole class.

import unittest

class TestExample(unittest.TestCase): @classmethod def setUpClass(cls): cls.shared_resource = "Shared Resource" print("Setting up class resources")

@classmethod def tearDownClass(cls): cls.shared_resource = None print("Tearing down class resources")

def test_use_shared_resource(self): self.assertEqual(self.shared_resource, "Shared Resource")

def test_another_use_of_shared_resource(self): self.assertEqual(self.shared_resource, "Shared Resource")

Output:

Setting up class resources

Tearing down class resources

Assertions

Assertions are the key component of any test. They check whether a condition is true, and if not, they raise an error indicating that the test has failed. unittest provides a variety of assertion methods to check different conditions. Here are some commonly used assertions:

assertEqual(a, b): Checks if a is equal to b.
assertNotEqual(a, b): Checks if a is not equal to b.
assertTrue(x): Checks if x is True.
assertFalse(x): Checks if x is False.
assertIs(a, b): Checks if a is b.
assertIsNot(a, b): Checks if a is not b.
assertIsNone(x): Checks if x is None.
assertIsNotNone(x): Checks if x is not None.
assertIn(a, b): Checks if a is in b.
assertNotIn(a, b): Checks if a is not in b.
assertIsInstance(a, b): Checks if a is an instance of b.
assertNotIsInstance(a, b): Checks if a is not an instance of b.

Look at thise assertion statements, this again doesn't seem to be pythonic as it contains camelCase functions.

Here's an example using different assertions:

import unittest

class TestAssertions(unittest.TestCase): def test_assertions(self): self.assertEqual(1 + 1, 2) self.assertNotEqual(2 + 2, 5) self.assertTrue(3 < 5) self.assertFalse(5 < 3) self.assertIs(None, None) self.assertIsNot(1, None) self.assertIsNone(None) self.assertIsNotNone(1) self.assertIn(3, [1, 2, 3]) self.assertNotIn(4, [1, 2, 3]) self.assertIsInstance(3, int) self.assertNotIsInstance(3, str)

Running Tests

You can run tests in several ways:

Command Line: Run the test script directly from the command line. python test_script.py
Discover Tests: Use unittest’s built-in test discovery mechanism to find and run tests automatically. This is useful for larger projects with many test files.

python -m unittest discover

This command searches for test modules in the current directory and its subdirectories.
From an IDE: Most integrated development environments (IDEs) like PyCharm, VS Code, and Eclipse have built-in support for running unittest tests, providing a convenient graphical interface to run and debug tests.

project/

    ├── src/

    │   └── example.py

    └── tests/

        ├── __init__.py

        └── test_example.py

In tests/test_example.py:

import unittest from src.example import add

class TestMathFunctions(unittest.TestCase): def test_add(self): self.assertEqual(add(1, 2), 3)

Run the tests using:

python -m unittest discover -s tests

This command tells unittest to discover and run all tests in the tests directory.

writing unit tests in python

Advanced Features

Mocking

Mocking is a technique used in unit testing to replace real objects with mock objects that simulate the behavior of real objects. This is particularly useful for isolating the code being tested from its dependencies.

Sample use cases: you have two functions A and B, you already wrote tests for the function A, and the function B calls the function A. Now you have to write unit tests for the function B. Then you need to use Mock.

Python’s unittest.mock module provides a powerful framework for mocking objects. Here’s an example of how to use unittest.mock:

from unittest import TestCase from unittest.mock import MagicMock, patch

class TestMocking(TestCase): @patch('path.to.module.ClassName') def test_mocking(self, mock_class): instance = mock_class.return_value instance.method.return_value = 'mocked!'

result = instance.method() self.assertEqual(result, 'mocked!') mock_class.assert_called_once() instance.method.assert_called_once()

In this example:

@patch('path.to.module.ClassName') replaces ClassName in path.to.module with a mock object.
mock_class.return_value is the mock instance of the class.
instance.method.return_value = 'mocked!' sets the return value of the method to 'mocked!'.

Test Suites

Test suites allow you to group multiple test cases and test methods into a single suite, which can then be run together. This is useful for organizing tests and running a specific subset of tests.

Here’s an example of how to create and run a test suite:

import unittest

class TestMath(unittest.TestCase): def test_add(self): self.assertEqual(1 + 1, 2)

def test_subtract(self): self.assertEqual(2 - 1, 1)

class TestStringMethods(unittest.TestCase): def test_upper(self): self.assertEqual('foo'.upper(), 'FOO')

def test_isupper(self): self.assertTrue('FOO'.isupper()) self.assertFalse('Foo'.isupper())

if __name__ == '__main__': suite = unittest.TestSuite() suite.addTest(TestMath('test_add')) suite.addTest(TestMath('test_subtract')) suite.addTest(TestStringMethods('test_upper')) suite.addTest(TestStringMethods('test_isupper'))

runner = unittest.TextTestRunner() runner.run(suite)

In this example:

unittest.TestSuite() creates a test suite.
suite.addTest(TestMath('test_add')) adds individual test methods to the suite.
unittest.TextTestRunner() runs the test suite.

Skipping Tests

There are situations where you might want to skip certain tests. unittest provides decorators for this purpose:

@unittest.skip(reason): Unconditionally skip a test.
@unittest.skipIf(condition, reason): Skip a test if the condition is true.
@unittest.skipUnless(condition, reason): Skip a test unless the condition is true.

Here’s an example:

import unittest

class TestExample(unittest.TestCase): @unittest.skip("demonstrating skipping") def test_skip(self): self.fail("shouldn't happen")

@unittest.skipIf(1 == 1, "not testing this right now") def test_skip_if(self): self.fail("shouldn't happen")

@unittest.skipUnless(1 == 0, "not testing this right now") def test_skip_unless(self): self.fail("shouldn't happen")

In this example, all three tests are skipped for different reasons.

Best Practices

Adopting best practices in unit testing helps ensure your tests are effective, maintainable, and reliable. Here are some essential best practices for using the unittest framework:

Test Coverage

Aim for High Coverage: Strive for high test coverage to ensure that most of your code is tested. Use tools like coverage.py to measure how much of your code is covered by tests.

pip install coverage

coverage run -m unittest discover

coverage report -m

Name                    Stmts   Miss  Cover   Missing

-----------------------------------------------------

libs/__init__.py            0      0   100%

libs/strings.py            19      8    58%   15-24

tests/test_strings.py      19      0   100%

-----------------------------------------------------

TOTAL                      38      8    79%

This command will run your tests and generate a coverage report showing which parts of your code are covered.

Isolated Tests

Write Independent Tests: Ensure that each test is independent and does not rely on the state or outcome of another test. This prevents cascading failures and makes debugging easier.

import unittest

class TestIsolated(unittest.TestCase): def setUp(self): # remember how it is different to setUpClass? What if we use setUpClass here? self.sample_list = []

def test_add_to_list(self): self.sample_list.append(1) self.assertEqual(self.sample_list, [1])

def test_list_starts_empty(self): self.assertEqual(self.sample_list, [])

Clear Naming

Descriptive Test Names: Use clear and descriptive names for your test methods to indicate what they are testing. This makes it easier to understand what each test is verifying.

import unittest

class TestCalculator(unittest.TestCase): def test_add_two_numbers_return_true(self): self.assertEqual(add(1, 2), 3) def test_add_two_numbers_return_false(self): self.assertNotEqual(add(1, 2), 4)

def test_subtract_two_numbers(self): self.assertEqual(subtract(2, 1), 1)

DRY Principle

Avoid Repetition: Follow the DRY (Don't Repeat Yourself) principle by avoiding repetitive code in your tests. Use helper methods or classes to reduce redundancy.

import unittest

def add(a, b): return a + b

class TestCalculator(unittest.TestCase): def check_addition(self, a, b, expected): self.assertEqual(add(a, b), expected)

def test_add_positive_numbers(self): self.check_addition(1, 2, 3)

def test_add_negative_numbers(self): self.check_addition(-1, -2, -3)

This is a controlversial topic, many developers state that tests can be copied and duplicated, while others do not agree. You can find out more here - https://stackoverflow.com/questions/6453235/what-does-damp-not-dry-mean-when-talking-about-unit-tests.

Consistent Setup

Consistent Environment: Ensure a consistent setup and teardown process for your tests to avoid side effects. Use setUp and tearDown methods to prepare and clean up the test environment.

import unittest

class TestEnvironment(unittest.TestCase): def setUp(self): self.resource = "Setup"

def tearDown(self): self.resource = None

def test_resource_is_setup(self): self.assertEqual(self.resource, "Setup")

def test_resource_is_tear_down(self): self.assertIsNone(self.resource)

Test Edge Cases

Cover Edge Cases: Write tests for edge cases and boundary conditions to ensure your code handles all possible inputs and scenarios.

import unittest

def divide(a, b): return a / b

class TestEdgeCases(unittest.TestCase): def test_divide_by_zero(self): with self.assertRaises(ZeroDivisionError): divide(1, 0)

def test_divide_by_one(self): self.assertEqual(divide(1, 1), 1)

Use Mocks and Stubs

Mock External Dependencies: Use mocks and stubs to isolate the code being tested from external dependencies such as databases, network calls, or file systems.

from unittest import TestCase from unittest.mock import MagicMock

class TestWithMock(TestCase): def test_external_dependency(self): mock = MagicMock() mock.method.return_value = 'mocked!' result = mock.method() self.assertEqual(result, 'mocked!')

Sample unittest classes

Common Pitfalls

Even experienced developers can fall into some common pitfalls when writing unit tests. Recognizing and avoiding these pitfalls can help you write more effective and reliable tests.

Flaky Tests

Symptoms: Flaky tests are tests that sometimes pass and sometimes fail without any changes to the code. They create uncertainty and can undermine the confidence in your test suite. This is the most annoying test case - which is usually difficult to debug.

Causes: Common causes include dependency on external systems (e.g., databases, network), race conditions, and improper use of mocks.

Solution: Ensure tests are isolated, deterministic, and not dependent on external systems. Use mocking to simulate external dependencies.

import unittest from unittest.mock import MagicMock

class TestWithMock(unittest.TestCase): def test_flaky_dependency(self): mock = MagicMock() mock.method.return_value = 'mocked!' result = mock.method() self.assertEqual(result, 'mocked!')

Test Smells

Symptoms: Test smells are patterns in test code that indicate potential problems. Examples include overly complex tests, lack of assertions, and duplicated code.

Causes: Poor test design, lack of understanding of testing principles, and shortcuts taken during development.

Solution: Refactor test code regularly, follow best practices, and review tests as critically as production code.

import unittest

class TestSmells(unittest.TestCase): def test_without_assertion(self): # Bad practice: No assertion result = 1 + 1

def test_with_assertion(self): # Good practice: Includes assertion self.assertEqual(1 + 1, 2)

False Positives/Negatives

Symptoms: False positives occur when a test passes despite a bug in the code. False negatives occur when a test fails even though the code is correct. This happens when developers create many inputs, then the function, collect ouputs and write tests based on those input/output. They never check if they are expected or not.

Causes: Incorrect assertions, incomplete test coverage, and unreliable tests.

Solution: Write precise and meaningful assertions, ensure thorough test coverage, and regularly review and update tests.

import unittest

def add(a, b): return a + b

class TestFalsePositivesNegatives(unittest.TestCase): def test_false_positive(self): # Incorrect assertion leading to false positive self.assertEqual(add(2, 2), 5)

def test_false_negative(self): # Correct assertion but may fail due to external factors self.assertEqual(add(2, 2), 4)

Ignoring Test Failures

Symptoms: Ignoring test failures or temporarily disabling tests without fixing the underlying issue.

Causes: Time pressure, difficulty in diagnosing the failure, or perceived unimportance of the test.

Solution: Address test failures as soon as they occur. If necessary, use @unittest.expectedFailure to indicate known issues and document the reason.

import unittest

class TestIgnoringFailures(unittest.TestCase): @unittest.expectedFailure def test_known_issue(self): # Document the reason for the expected failure self.assertEqual(1 + 1, 3)

Overuse of Mocks

Symptoms: Tests become tightly coupled to the implementation rather than the behavior, leading to fragile tests.

Causes: Excessive mocking of internal details and dependencies, making tests difficult to maintain.

Solution: Mock only external dependencies and focus on testing the behavior rather than the implementation.

from unittest import TestCase, mock

def fetch_data(api_client): return api_client.get_data()

class TestOveruseOfMocks(TestCase): @mock.patch('path.to.api_client.APIClient.get_data') def test_fetch_data(self, mock_get_data): mock_get_data.return_value = {'data': 'mocked data'} result = fetch_data(mock_get_data) self.assertEqual(result, {'data': 'mocked data'})

By being aware of these common pitfalls and actively working to avoid them, you can write more reliable and effective unit tests.

You can also find some useful information in this Reddit Thread

Tools and Extensions

Enhancing your testing workflow with additional tools and extensions can significantly improve the efficiency and effectiveness of your tests. Here are some recommended tools and packages:

Test Runners

pytest: A powerful and user-friendly test runner that can run unittest tests and offers advanced features such as fixtures and plugins. It provides more detailed output and supports various plugins for additional functionality. And this is 100% compatible with unittest package

pip install pytest
pytest

Coverage Tools

coverage.py: A tool for measuring code coverage in Python programs. It works well with both unittest and pytest.

pip install coverage coverage run -m unittest discover coverage report -m

This will run your tests and generate a coverage report showing which parts of your code are covered and which are not.

Data Generation

Faker: A library for generating fake data, useful for creating test data without manually writing it.

pip install faker

Example usage:

from faker import Faker

fake = Faker()

class TestDataGeneration(unittest.TestCase): def test_fake_data(self): name = fake.name() address = fake.address() self.assertIsInstance(name, str) self.assertIsInstance(address, str)

Pre-commit Hooks

pre-commit: A framework for managing and maintaining multi-language pre-commit hooks. Pre-commit hooks are scripts that run before every commit to catch potential issues early.

pip install pre-commit pre-commit install

Example configuration (.pre-commit-config.yaml):

repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v3.7.1 hooks: - id: trailing-whitespace - id: end-of-file-fixer - id: check-yaml - repo: https://github.com/psf/black rev: 21.9b0 hooks: - id: black

This configuration will install hooks to remove trailing whitespace, fix end-of-file issues, check YAML files, and format code with black.

Advantages and Disadvantages of `unittest`

Pros

Built-in Standard Library:
- Availability: unittest is part of Python's standard library, meaning it requires no additional installation and is readily available in any Python environment.
- Documentation: Well-documented and supported by the Python community, making it easy to find resources and examples.
Comprehensive Assertions:
- Variety: Provides a wide range of assertion methods (assertEqual, assertTrue, assertFalse, assertRaises, etc.) to cover most testing scenarios.
- Clarity: Clear and readable assertions help in understanding test results and debugging failures.
Test Discovery:
- Automation: Automatically discovers and runs tests, which can be a huge time-saver in larger projects.
- Convenience: Easy to use with a simple command-line interface.
Setup and Teardown:
- Flexibility: Offers setUp and tearDown methods for preparing and cleaning up before and after each test, and setUpClass and tearDownClass for class-level setup and teardown.
Mocking Support:
- unittest.mock Module: Includes a powerful mocking library for simulating objects and functions, which is essential for isolating units of code during testing.
Compatibility:
- Integrations: Works seamlessly with other testing tools and libraries like coverage.py for code coverage, and CI tools like GitHub Actions for automated testing.

Cons

Verbosity:
- Boilerplate: Requires more boilerplate code compared to some other testing frameworks like pytest. Each test needs to be a method of a class derived from unittest.TestCase.
Less Intuitive:
- Learning Curve: Can be less intuitive for beginners compared to more modern frameworks like pytest, which use simpler syntax and less boilerplate.
Limited Fixture Flexibility:
- Complex Setups: While setUp and tearDown are useful, they can become cumbersome for complex test setups that require more flexible fixtures.
Assertion Errors:
- Less Informative: The default error messages for failed assertions can be less informative compared to those provided by pytest, which offers more detailed and user-friendly error messages. Look at this discussion - https://stackoverflow.com/questions/4319825/python-unittest-opposite-of-assertraises!
Less Extensible:
- Plugins and Extensions: Fewer plugins and extensions compared to pytest, which has a large ecosystem of plugins that extend its functionality. Pytest has a list of plugins.
Parameterization:
- Lack of Built-in Support: Does not have built-in support for parameterized tests, making it harder to run the same test with different inputs. This is something that pytest handles very elegantly with its @pytest.mark.parametrize decorator. Ref: https://docs.pytest.org/en/7.4.x/how-to/parametrize.html

Conclusion

Recap

In this article, we've covered the essentials of using the unittest framework for writing and running tests in Python, explored advanced features, common mistakes, and discussed best practices to ensure your tests are effective and maintainable.

Call to Action

Now that you have a solid foundation in unit testing with unittest, it's time to put this knowledge into practice. Start by writing tests for your existing projects, and gradually incorporate best practices to improve the quality and reliability of your code. Remember, effective testing is a continuous process that evolves with your codebase. Keep learning, experimenting, and refining your testing strategies to ensure your software remains robust and maintainable.

You can have a look at our sample code in this link and some exercises to play with here

Happy testing! Happy coding!