Home
  • Sep 02, 2019
  • Post Nº 116

Measure Coverage with Coverage.py

Experience sharing on code coverage measurement of Python program using Coverage.py.

Overview

Today, I would like to share my recent experience with code coverage measurement using Coverage.py. I used it for my scripts about personal finance: mainly for aggregating account history from different banks. I use Python 3.7.2 with Coverage 4.5.3 and Circle CI.

After reading this article, you will understand:

  • How to setup Coverage.py in the existing Python project?
  • How to run tests with test coverage measured?
  • How to create a coverage report?
  • Advanced options to be enabled

Setup Coverage.py

Declare Coverage.py as a test-scoped dependency in requirements-tests.txt:

+coverage==4.5.3
 flak8==3.7.7
 pytest==5.0.1

Then, ensure the test requirements are installed during the build. In Circle CI, you can do it as follow:

steps:
  - ...
  - run:
      name: Install Test Dependencies
      command: |
        . venv/bin/activate
        pip install -r requirements-tests.txt

In your local machine, you can install it using pip:

pip install coverage

Measure Coverage

According to the official web page, you can run your program and gather data using coverage run:

# if you usually do:
#
#   $ python my_program.py arg1 arg2
#
# then instead do:

$ coverage run my_program.py arg1 arg2

In my case, I use pytest for testing. I also enabled level 2 verbosity (-vv) and strict mode (--strict) so that warnings become errors and make the build fail. Combined with coverage, the command is written as follows:

$ coverage run -m pytest -vv --strict

Here’s the screenshot of the execution on macOS:

Coverage run example

Coverage Report

Use coverage-report command to report on the results. This will print the results in terminal. In my case, I also enabled the option -m, --show-missing in my .coveragerc file to show line numbers of statements in each module that weren’t executed.

$ coverage report

Here’s the screenshot of the execution on macOS:

Coverage report example

Another way to see the result is via HTML page. You can see both the overview and a detailed view of each file.

$ coverage html

Here’s the screenshot of the overview page:

Coverage report HTML (overview)

Here’s the screenshot of the detail page:

Coverage report HTML (detail)

Advanced Options

Coverage.py options can be specified in a configuration file. This makes it easier to re-run coverage.py with consistent settings, and also allows for specification of options that are otherwise only available in the API.

The default name for configuration files is .coveragerc, in the same directory coverage.py is being run in. Most of the settings in the configuration file are tied to your source code and how it should be measured, so it should be stored with your source, and checked into source control, rather than put in your home directory.

In my case, for command “run”, I set the source directory to src and enabled the branch coverage measurement, in addition to the usual statement coverage; for command “report”, I added the show-missing option to see more detail about what is missing directly in terminal.

[run]
source = src
branch = True

[report]
show_missing = True

Conclusion

In this article, I explained the usage of coverage.py in my own Python project, including installation, coverage meansure, coverage report generation, and storing more options in configuration file. If you are interested in Coverage.py, I suggest you to go to their official documentation https://coverage.readthedocs.io/en/latest/index.html. There are clear and detailed explanation about different settings. Hope you enjoy this article, see you the next time!

References

Search