46  Package coverage: Python

We covered what code coverage measures, and how to calculate it with pytest-cov, in Testing: evaluating test suite quality. Here we measure the coverage of our package.

In a package, we add pytest-cov as a development dependency with Poetry:

poetry add --group dev pytest-cov

and run it through Poetry, for example poetry run pytest --cov=src.

46.1 Using our toy package example

Let’s go back to our previous packaging function and tests. For reference this was our function

def count_char(input_string):
    if not isinstance(input_string, str):
        raise TypeError(f"Expected input to be str, got {type(input_string)}")
    len(input_string)

and the tests we wrote

import pytest

from countchar.count_char import count_char


@pytest.fixture
def text_data():
    file_path = "tests/text.txt"
    with open(file_path, 'r') as file:
        lines = file.readlines()
    return lines


def test_line_1(text_data):
    string = text_data[0]
    assert count_char(string) == 34

def test_line_2(text_data):
    string = text_data[1]
    assert count_char(string) == 10

def test_line_3(text_data):
    string = text_data[2]
    assert count_char(string) == 8

We can now look at the line and branch coverage of our current package files

pytest tests/ --cov=countchar # line coverage
pytest --cov=countchar --cov-branch # branch coverage

and generate the document for a coverage report

pytest --cov=countchar --cov-report html

46.2 Reporting coverage badges

https://codecov.io/ is a service that can take our pytest information and give us a visualization of our test coverage and also give us a badge we can display in our repository.

More detail about creating and linking codecov to our repositor is in this section of the textbook: https://py-pkgs.org/08-ci-cd#recording-code-coverage

General steps:

  • log into https://codecov.io/ with your GitHub account
  • install code coverage github app (if you’re using personal account, your organization may have it already installed)
    • Make sure the application can see your repository in the app settings
  • find repo on codecov
  • click configure
  • setup code cov action
    • use github actions
    • select pytest
    • change --cov-report=xml
      • pytest --cov --cov-branch --cov-report=xml
    • get token info for CODECOV_TOKEN (from configure page)
    • when CI runs you will see a link in that particular step results of the coverage with a URL (else you probably didn’t have a token)
    • refresh the codecoverage repo page
  • badge
    • configuration page there’s a badge + graphs section
    • copy the markdown badge (we are using quarto + quartodoc, rst is for sphinx)