Python Unittest Tutorial and Best Practices
By JoeVu, at: June 3, 2024, 11:13 a.m.
Python Unittest Tutorial and Best Practices
Introduction
Overview
At Glinteco, both unit/integration/automation/stress tests are crucial for every single project. We believe "Great Developers must write tests for his own code"
Unit testing is a fundamental practice in software development that involves testing individual units or components of a software application to ensure they work as expected. A unit is the smallest testable part of any software - usually a function or method. By isolating and testing these units, developers can identify and fix bugs early in the development cycle, leading to more robust and reliable software.
Unit tests are IMPORTANT because they:
- Verify the correctness of code.
- Facilitate code refactoring by ensuring that changes do not break existing functionality.
- Serve as documentation for the code, illustrating how it is supposed to work.
- Help catch bugs early, reducing the cost of fixing them.
- Improve the source code quality
Not many senior developers can manage this well due to many reasons, however, to level up our own programming expertise, writing unit test is a must.
Purpose
This article provides a completed guide to using Python's unittest
framework for writing and running unit tests (We don't cover pytest here, we do have a pytest vs unittest comparison post). Whether you're a beginner or an experienced developer, this tutorial will help you understand the basics of unittest
and teach you best practices for writing effective tests.
By the end of this article, you will:
- Understand the basic structure of a
unittest
test case.
- Learn how to write, run, and organize tests.
- Explore advanced features such as test fixtures, mocking, and test suites.
- Discover best practices to improve the quality and maintainability of your tests.
- Identify common pitfalls and how to avoid them.
Unit testing with unittest
is a powerful way to ensure your code is reliable and maintainable, hence, a better product. Let's start the unittest
adventure.
Getting Started with Unittest
Installation
unittest
is included as a Python's standard library - you don't need to install any additional packages to get started with unit testing in Python. Simply import unittest
in your script, and you're ready to write tests.
Basic Structure
The core of the unittest
framework revolves around the TestCase
class. A test case is created by subclassing unittest.TestCase
, and individual test methods are defined within this class. Each test method should start with the word test
to ensure it is automatically recognized and executed by the unittest
test runner. This seems a bit not pythonic, doesn't it?
Here's the basic structure of a unittest
test case, name this file as test.py
import unittest
def count_e_letters(name):
count = 0
for letter in name:
if letter == 'e':
count += 1
return count
class TestCountELetters(unittest.TestCase):
def test_count_e_letters(self):
count = count_e_letters("Joe")
self.assertEqual(1, count)
if __name__ == '__main__':
unittest.main()
Running Tests
To run the tests, save the script and execute it from the command line:
❯ python test.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
If any test fails, unittest
will provide detailed information about the failure, including the expected and actual results, helping you diagnose the issue.
❯ python -m unittest tests/test_strings.py
F.
======================================================================
FAIL: test_count_e_letters_with_a_none_input (tests.test_strings.TestCountELetters.test_count_e_letters_with_a_none_input)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/joe/Documents/WORK/GLINTECO/PROJECTS/INTERNAL/samples/unittest_tutorials/tests/test_strings.py", line 13, in test_count_e_letters_with_a_none_input
self.assertEqual(1, count)
AssertionError: 1 != 0
With these basics covered, you can start writing simple tests using unittest
. In the next section, we'll delve deeper into writing test cases with setup and teardown methods to manage test environments effectively.
Writing Test Cases
Setup and Teardown
In unit testing, it is common to prepare a specific environment before running tests and clean up afterward. unittest
provides setUp
and tearDown
methods to handle this. The setUp
method is called before each test method to set up any state that's shared across tests, and the tearDown
method is called after each test method to clean up. This is similar to Ruby
Here's an example:
import unittest
class TestExample(unittest.TestCase):
def setUp(self):
self.user = User()
def tearDown(self):
self.user = None
def test_name(self):
self.assertEqual("Joe", self.user.name)
def test_action(self):
self.assertEqual("Playing Game", self.user.action())
As can be seen from above:
- The
setUp
method initializesself.user
as a new user before each test. - The
tearDown
method cleans up by settingself.user
toNone
after each test. - Two test methods (
test_name
andtest_action
) useself.user
to perform their assertions.
What is the difference between setUp
/tearDown
and setUpClass/tearDownClass
setUp
and tearDown
- Purpose:
setUp
andtearDown
methods are used to set up and clean up resources needed for each individual test method.
- Execution:
setUp
is called before each test method runs, andtearDown
is called after each test method finishes.
- Scope: The setup and teardown actions in these methods apply to each test method within the
TestCase
class. This means that if you have multiple test methods,setUp
andtearDown
will run multiple times—once for each test method.
import unittest
class TestExample(unittest.TestCase):
def setUp(self):
self.number = 1
print("Setting up before a test method")
def tearDown(self):
self.number = None
print("Tearing down after a test method")
def test_addition(self):
self.assertEqual(self.number + 1, 2)
def test_subtraction(self):
self.assertEqual(self.number - 1, 0)
Output:
Setting up before a test method
Tearing down after a test method
Setting up before a test method
Tearing down after a test method
setUpClass
and tearDownClass
- Purpose:
setUpClass
andtearDownClass
are used to set up and clean up resources needed for the entire test case class, not just for individual test methods.
- Execution:
setUpClass
is called once before any test methods are run, andtearDownClass
is called once after all test methods have finished running.
- Scope: The setup and teardown actions in these methods apply to the entire test case class. This means that if you have multiple test methods,
setUpClass
andtearDownClass
will run only once for the whole class.
import unittest
class TestExample(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.shared_resource = "Shared Resource"
print("Setting up class resources")
@classmethod
def tearDownClass(cls):
cls.shared_resource = None
print("Tearing down class resources")
def test_use_shared_resource(self):
self.assertEqual(self.shared_resource, "Shared Resource")
def test_another_use_of_shared_resource(self):
self.assertEqual(self.shared_resource, "Shared Resource")
Output:
Setting up class resources
Tearing down class resources
Assertions
Assertions are the key component of any test. They check whether a condition is true, and if not, they raise an error indicating that the test has failed. unittest
provides a variety of assertion methods to check different conditions. Here are some commonly used assertions:
assertEqual(a, b)
: Checks ifa
is equal tob
.
assertNotEqual(a, b)
: Checks ifa
is not equal tob
.
assertTrue(x)
: Checks ifx
isTrue
.
assertFalse(x)
: Checks ifx
isFalse
.
assertIs(a, b)
: Checks ifa
isb
.
assertIsNot(a, b)
: Checks ifa
is notb
.
assertIsNone(x)
: Checks ifx
isNone
.
assertIsNotNone(x)
: Checks ifx
is notNone
.
assertIn(a, b)
: Checks ifa
is inb
.
assertNotIn(a, b)
: Checks ifa
is not inb
.
assertIsInstance(a, b)
: Checks ifa
is an instance ofb
.
assertNotIsInstance(a, b)
: Checks ifa
is not an instance ofb
.
Look at thise assertion statements, this again doesn't seem to be pythonic as it contains camelCase functions.
Here's an example using different assertions:
import unittest
class TestAssertions(unittest.TestCase):
def test_assertions(self):
self.assertEqual(1 + 1, 2)
self.assertNotEqual(2 + 2, 5)
self.assertTrue(3 < 5)
self.assertFalse(5 < 3)
self.assertIs(None, None)
self.assertIsNot(1, None)
self.assertIsNone(None)
self.assertIsNotNone(1)
self.assertIn(3, [1, 2, 3])
self.assertNotIn(4, [1, 2, 3])
self.assertIsInstance(3, int)
self.assertNotIsInstance(3, str)
Running Tests
You can run tests in several ways:
-
Command Line: Run the test script directly from the command line.
python test_script.py
-
Discover Tests: Use
unittest
’s built-in test discovery mechanism to find and run tests automatically. This is useful for larger projects with many test files.python -m unittest discover
This command searches for test modules in the current directory and its subdirectories.
-
From an IDE: Most integrated development environments (IDEs) like PyCharm, VS Code, and Eclipse have built-in support for running
unittest
tests, providing a convenient graphical interface to run and debug tests.
project/
├── src/
│ └── example.py
└── tests/
├── __init__.py
└── test_example.py
In tests/test_example.py
:
import unittest
from src.example import add
class TestMathFunctions(unittest.TestCase):
def test_add(self):
self.assertEqual(add(1, 2), 3)
Run the tests using:
python -m unittest discover -s tests
This command tells unittest
to discover and run all tests in the tests
directory.
Advanced Features
Mocking
Mocking is a technique used in unit testing to replace real objects with mock objects that simulate the behavior of real objects. This is particularly useful for isolating the code being tested from its dependencies.
Sample use cases: you have two functions A and B, you already wrote tests for the function A, and the function B calls the function A. Now you have to write unit tests for the function B. Then you need to use Mock.
Python’s unittest.mock
module provides a powerful framework for mocking objects. Here’s an example of how to use unittest.mock
:
from unittest import TestCase
from unittest.mock import MagicMock, patch
class TestMocking(TestCase):
@patch('path.to.module.ClassName')
def test_mocking(self, mock_class):
instance = mock_class.return_value
instance.method.return_value = 'mocked!'
result = instance.method()
self.assertEqual(result, 'mocked!')
mock_class.assert_called_once()
instance.method.assert_called_once()
In this example:
@patch('path.to.module.ClassName')
replacesClassName
inpath.to.module
with a mock object.mock_class.return_value
is the mock instance of the class.instance.method.return_value = 'mocked!'
sets the return value of the method to'mocked!'
.
Test Suites
Test suites allow you to group multiple test cases and test methods into a single suite, which can then be run together. This is useful for organizing tests and running a specific subset of tests.
Here’s an example of how to create and run a test suite:
import unittest
class TestMath(unittest.TestCase):
def test_add(self):
self.assertEqual(1 + 1, 2)
def test_subtract(self):
self.assertEqual(2 - 1, 1)
class TestStringMethods(unittest.TestCase):
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
def test_isupper(self):
self.assertTrue('FOO'.isupper())
self.assertFalse('Foo'.isupper())
if __name__ == '__main__':
suite = unittest.TestSuite()
suite.addTest(TestMath('test_add'))
suite.addTest(TestMath('test_subtract'))
suite.addTest(TestStringMethods('test_upper'))
suite.addTest(TestStringMethods('test_isupper'))
runner = unittest.TextTestRunner()
runner.run(suite)
In this example:
unittest.TestSuite()
creates a test suite.suite.addTest(TestMath('test_add'))
adds individual test methods to the suite.unittest.TextTestRunner()
runs the test suite.
Skipping Tests
There are situations where you might want to skip certain tests. unittest
provides decorators for this purpose:
@unittest.skip(reason)
: Unconditionally skip a test.@unittest.skipIf(condition, reason)
: Skip a test if the condition is true.@unittest.skipUnless(condition, reason)
: Skip a test unless the condition is true.
Here’s an example:
import unittest
class TestExample(unittest.TestCase):
@unittest.skip("demonstrating skipping")
def test_skip(self):
self.fail("shouldn't happen")
@unittest.skipIf(1 == 1, "not testing this right now")
def test_skip_if(self):
self.fail("shouldn't happen")
@unittest.skipUnless(1 == 0, "not testing this right now")
def test_skip_unless(self):
self.fail("shouldn't happen")
In this example, all three tests are skipped for different reasons.
Best Practices
Adopting best practices in unit testing helps ensure your tests are effective, maintainable, and reliable. Here are some essential best practices for using the unittest
framework:
Test Coverage
Aim for High Coverage: Strive for high test coverage to ensure that most of your code is tested. Use tools like coverage.py
to measure how much of your code is covered by tests.
pip install coverage
coverage run -m unittest discover
coverage report -m
Name Stmts Miss Cover Missing
-----------------------------------------------------
libs/__init__.py 0 0 100%
libs/strings.py 19 8 58% 15-24
tests/test_strings.py 19 0 100%
-----------------------------------------------------
TOTAL 38 8 79%
This command will run your tests and generate a coverage report showing which parts of your code are covered.
Isolated Tests
Write Independent Tests: Ensure that each test is independent and does not rely on the state or outcome of another test. This prevents cascading failures and makes debugging easier.
import unittest
class TestIsolated(unittest.TestCase):
def setUp(self): # remember how it is different to setUpClass? What if we use setUpClass here?
self.sample_list = []
def test_add_to_list(self):
self.sample_list.append(1)
self.assertEqual(self.sample_list, [1])
def test_list_starts_empty(self):
self.assertEqual(self.sample_list, [])
Clear Naming
Descriptive Test Names: Use clear and descriptive names for your test methods to indicate what they are testing. This makes it easier to understand what each test is verifying.
import unittest
class TestCalculator(unittest.TestCase):
def test_add_two_numbers_return_true(self):
self.assertEqual(add(1, 2), 3)
def test_add_two_numbers_return_false(self):
self.assertNotEqual(add(1, 2), 4)
def test_subtract_two_numbers(self):
self.assertEqual(subtract(2, 1), 1)
DRY Principle
Avoid Repetition: Follow the DRY (Don't Repeat Yourself) principle by avoiding repetitive code in your tests. Use helper methods or classes to reduce redundancy.
import unittest
def add(a, b):
return a + b
class TestCalculator(unittest.TestCase):
def check_addition(self, a, b, expected):
self.assertEqual(add(a, b), expected)
def test_add_positive_numbers(self):
self.check_addition(1, 2, 3)
def test_add_negative_numbers(self):
self.check_addition(-1, -2, -3)
This is a controlversial topic, many developers state that tests can be copied and duplicated, while others do not agree. You can find out more here - https://stackoverflow.com/questions/6453235/what-does-damp-not-dry-mean-when-talking-about-unit-tests.
Consistent Setup
Consistent Environment: Ensure a consistent setup and teardown process for your tests to avoid side effects. Use setUp
and tearDown
methods to prepare and clean up the test environment.
import unittest
class TestEnvironment(unittest.TestCase):
def setUp(self):
self.resource = "Setup"
def tearDown(self):
self.resource = None
def test_resource_is_setup(self):
self.assertEqual(self.resource, "Setup")
def test_resource_is_tear_down(self):
self.assertIsNone(self.resource)
Test Edge Cases
Cover Edge Cases: Write tests for edge cases and boundary conditions to ensure your code handles all possible inputs and scenarios.
import unittest
def divide(a, b):
return a / b
class TestEdgeCases(unittest.TestCase):
def test_divide_by_zero(self):
with self.assertRaises(ZeroDivisionError):
divide(1, 0)
def test_divide_by_one(self):
self.assertEqual(divide(1, 1), 1)
Use Mocks and Stubs
Mock External Dependencies: Use mocks and stubs to isolate the code being tested from external dependencies such as databases, network calls, or file systems.
from unittest import TestCase
from unittest.mock import MagicMock
class TestWithMock(TestCase):
def test_external_dependency(self):
mock = MagicMock()
mock.method.return_value = 'mocked!'
result = mock.method()
self.assertEqual(result, 'mocked!')
Common Pitfalls
Even experienced developers can fall into some common pitfalls when writing unit tests. Recognizing and avoiding these pitfalls can help you write more effective and reliable tests.
Flaky Tests
Symptoms: Flaky tests are tests that sometimes pass and sometimes fail without any changes to the code. They create uncertainty and can undermine the confidence in your test suite. This is the most annoying test case - which is usually difficult to debug.
Causes: Common causes include dependency on external systems (e.g., databases, network), race conditions, and improper use of mocks.
Solution: Ensure tests are isolated, deterministic, and not dependent on external systems. Use mocking to simulate external dependencies.
import unittest
from unittest.mock import MagicMock
class TestWithMock(unittest.TestCase):
def test_flaky_dependency(self):
mock = MagicMock()
mock.method.return_value = 'mocked!'
result = mock.method()
self.assertEqual(result, 'mocked!')
Test Smells
Symptoms: Test smells are patterns in test code that indicate potential problems. Examples include overly complex tests, lack of assertions, and duplicated code.
Causes: Poor test design, lack of understanding of testing principles, and shortcuts taken during development.
Solution: Refactor test code regularly, follow best practices, and review tests as critically as production code.
import unittest
class TestSmells(unittest.TestCase):
def test_without_assertion(self):
# Bad practice: No assertion
result = 1 + 1
def test_with_assertion(self):
# Good practice: Includes assertion
self.assertEqual(1 + 1, 2)
False Positives/Negatives
Symptoms: False positives occur when a test passes despite a bug in the code. False negatives occur when a test fails even though the code is correct. This happens when developers create many inputs, then the function, collect ouputs and write tests based on those input/output. They never check if they are expected or not.
Causes: Incorrect assertions, incomplete test coverage, and unreliable tests.
Solution: Write precise and meaningful assertions, ensure thorough test coverage, and regularly review and update tests.
import unittest
def add(a, b):
return a + b
class TestFalsePositivesNegatives(unittest.TestCase):
def test_false_positive(self):
# Incorrect assertion leading to false positive
self.assertEqual(add(2, 2), 5)
def test_false_negative(self):
# Correct assertion but may fail due to external factors
self.assertEqual(add(2, 2), 4)
Ignoring Test Failures
Symptoms: Ignoring test failures or temporarily disabling tests without fixing the underlying issue.
Causes: Time pressure, difficulty in diagnosing the failure, or perceived unimportance of the test.
Solution: Address test failures as soon as they occur. If necessary, use @unittest.expectedFailure
to indicate known issues and document the reason.
import unittest
class TestIgnoringFailures(unittest.TestCase):
@unittest.expectedFailure
def test_known_issue(self):
# Document the reason for the expected failure
self.assertEqual(1 + 1, 3)
Overuse of Mocks
Symptoms: Tests become tightly coupled to the implementation rather than the behavior, leading to fragile tests.
Causes: Excessive mocking of internal details and dependencies, making tests difficult to maintain.
Solution: Mock only external dependencies and focus on testing the behavior rather than the implementation.
from unittest import TestCase, mock
def fetch_data(api_client):
return api_client.get_data()
class TestOveruseOfMocks(TestCase):
@mock.patch('path.to.api_client.APIClient.get_data')
def test_fetch_data(self, mock_get_data):
mock_get_data.return_value = {'data': 'mocked data'}
result = fetch_data(mock_get_data)
self.assertEqual(result, {'data': 'mocked data'})
By being aware of these common pitfalls and actively working to avoid them, you can write more reliable and effective unit tests.
You can also find some useful information in this Reddit Thread
Tools and Extensions
Enhancing your testing workflow with additional tools and extensions can significantly improve the efficiency and effectiveness of your tests. Here are some recommended tools and packages:
Test Runners
-
pytest
: A powerful and user-friendly test runner that can rununittest
tests and offers advanced features such as fixtures and plugins. It provides more detailed output and supports various plugins for additional functionality. And this is 100% compatible withunittest
packagepip install pytest
pytest
Coverage Tools
-
coverage.py
: A tool for measuring code coverage in Python programs. It works well with bothunittest
andpytest
.pip install coverage
coverage run -m unittest discover
coverage report -m
This will run your tests and generate a coverage report showing which parts of your code are covered and which are not.
Data Generation
-
Faker
: A library for generating fake data, useful for creating test data without manually writing it.pip install faker
Example usage:from faker import Faker
fake = Faker()
class TestDataGeneration(unittest.TestCase):
def test_fake_data(self):
name = fake.name()
address = fake.address()
self.assertIsInstance(name, str)
self.assertIsInstance(address, str)
Pre-commit Hooks
-
pre-commit
: A framework for managing and maintaining multi-language pre-commit hooks. Pre-commit hooks are scripts that run before every commit to catch potential issues early.pip install pre-commit
pre-commit install
Example configuration (.pre-commit-config.yaml
):repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.7.1
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- repo: https://github.com/psf/black
rev: 21.9b0
hooks:
- id: black
This configuration will install hooks to remove trailing whitespace, fix end-of-file issues, check YAML files, and format code withblack
.
Advantages and Disadvantages of unittest
Pros
-
Built-in Standard Library:
- Availability:
unittest
is part of Python's standard library, meaning it requires no additional installation and is readily available in any Python environment.
- Documentation: Well-documented and supported by the Python community, making it easy to find resources and examples.
- Availability:
-
Comprehensive Assertions:
- Variety: Provides a wide range of assertion methods (
assertEqual
,assertTrue
,assertFalse
,assertRaises
, etc.) to cover most testing scenarios.
- Clarity: Clear and readable assertions help in understanding test results and debugging failures.
- Variety: Provides a wide range of assertion methods (
-
Test Discovery:
- Automation: Automatically discovers and runs tests, which can be a huge time-saver in larger projects.
- Convenience: Easy to use with a simple command-line interface.
- Automation: Automatically discovers and runs tests, which can be a huge time-saver in larger projects.
-
Setup and Teardown:
- Flexibility: Offers
setUp
andtearDown
methods for preparing and cleaning up before and after each test, andsetUpClass
andtearDownClass
for class-level setup and teardown.
- Flexibility: Offers
-
Mocking Support:
unittest.mock
Module: Includes a powerful mocking library for simulating objects and functions, which is essential for isolating units of code during testing.
-
Compatibility:
- Integrations: Works seamlessly with other testing tools and libraries like
coverage.py
for code coverage, and CI tools like GitHub Actions for automated testing.
- Integrations: Works seamlessly with other testing tools and libraries like
Cons
-
Verbosity:
- Boilerplate: Requires more boilerplate code compared to some other testing frameworks like
pytest
. Each test needs to be a method of a class derived fromunittest.TestCase
.
- Boilerplate: Requires more boilerplate code compared to some other testing frameworks like
-
Less Intuitive:
- Learning Curve: Can be less intuitive for beginners compared to more modern frameworks like
pytest
, which use simpler syntax and less boilerplate.
- Learning Curve: Can be less intuitive for beginners compared to more modern frameworks like
-
Limited Fixture Flexibility:
- Complex Setups: While
setUp
andtearDown
are useful, they can become cumbersome for complex test setups that require more flexible fixtures.
- Complex Setups: While
-
Assertion Errors:
- Less Informative: The default error messages for failed assertions can be less informative compared to those provided by
pytest
, which offers more detailed and user-friendly error messages. Look at this discussion - https://stackoverflow.com/questions/4319825/python-unittest-opposite-of-assertraises!
- Less Informative: The default error messages for failed assertions can be less informative compared to those provided by
-
Less Extensible:
- Plugins and Extensions: Fewer plugins and extensions compared to
pytest
, which has a large ecosystem of plugins that extend its functionality. Pytest has a list of plugins.
- Plugins and Extensions: Fewer plugins and extensions compared to
-
Parameterization:
- Lack of Built-in Support: Does not have built-in support for parameterized tests, making it harder to run the same test with different inputs. This is something that
pytest
handles very elegantly with its@pytest.mark.parametrize
decorator. Ref: https://docs.pytest.org/en/7.4.x/how-to/parametrize.html
- Lack of Built-in Support: Does not have built-in support for parameterized tests, making it harder to run the same test with different inputs. This is something that
Conclusion
Recap
In this article, we've covered the essentials of using the unittest
framework for writing and running tests in Python, explored advanced features, common mistakes, and discussed best practices to ensure your tests are effective and maintainable.
Further Reading
For those looking to dive deeper into unit testing and related topics, here are some valuable resources:
- Official
unittest
Documentation: Comprehensive documentation on theunittest
framework.
- Python Testing with
pytest
: Official documentation forpytest
, an alternative testing framework with powerful features.
- Test-Driven Development with Python: A book by Harry J.W. Percival that covers test-driven development using Python.
- https://learnbyexample.github.io/py_resources/intermediate.html#testing
- https://faker.readthedocs.io/en/master/
Call to Action
Now that you have a solid foundation in unit testing with unittest
, it's time to put this knowledge into practice. Start by writing tests for your existing projects, and gradually incorporate best practices to improve the quality and reliability of your code. Remember, effective testing is a continuous process that evolves with your codebase. Keep learning, experimenting, and refining your testing strategies to ensure your software remains robust and maintainable.
You can have a look at our sample code in this link and some exercises to play with here
Happy testing! Happy coding!