33  Package Documentation: Python

33.1 Learning objectives

• Generate well formatted function and package-level documentation for Python packages using Sphinx & Read the Docs

33.2 Documentation for Python packages

In Python, we use formatted docstrings to generate our code-level documentation. We then use a tool called sphinx to take those formatted docstrings to generate our API reference documentation for our package website, and several of our Markdown files in our packages GitHub repository to generate other pages for our package website (Contributing, Code of Conduct, etc). We then serve the website up on some platform, such as Read the Docs (others exist as well).

33.2.1 API reference docs

• We learned the basics of how to write formatted docstrings using numpy-style documentation.

• These docstrings can not only be accessed via ?function_name, but can also be used to automatically generate package-level documentation via sphinx

• We already did this with our toy pycounts package by:

  • adding our doc dependencies into our dev dependency group via `poetry add –group dev myst-nb sphinx-autoapi sphinx-rtd-theme
  • and then running make html from the docs directory

• The py-pkgs-cookiecutter template also has some extensions added to docs/conf.py that are needed for this to work.

• To have sphinx correctly render the docstring as package-level documentation, we need to either write our docstrings in the correct format for restructured text (RST) or we can use the sphinx extension napolean that can render Numpy- or Google-style docstrings (which are much easier for you to write and read).

33.2.2 Example of RST formatted docstrings:

""":type path: str
:param field_storage: The :class:`FileStorage` instance to wrap
:type field_storage: FileStorage
:param temporary: Whether or not to delete the file when the File
   instance is destructed
:type temporary: bool
:returns: A buffered writable file descriptor
:rtype: BufferedFileStorage
:example: my_funtion(4)
"""

33.2.3 Example of Numpy-style docstrings:

    """Summary line.

    Extended description of function.

    Parameters
    ----------
    arg1 : int
        Description of arg1
    arg2 : str
        Description of arg2

    Returns
    -------
    bool
        Description of return value

    Examples
    --------
    >>> my_funtion(4)
    """

33.2.4 Rendering the docs locally

It is not essential that you locally render the docs, as we will see next that Read the Docs does this for your on their remote machines, however it is a best practice to do so because it is a lot faster than Read the Docs and therefore editing and proof-reading is more efficient when done locally.

33.3 Adding new page to your package-level documentation

How do we add new pages to our Python package-level documentation? The pages that show up in the rendered document on ReadtheDocs are controlled by docs/index.yml.

Let’s take a look at the raw version of that file from our pypkgs-cookiecutter:

This results in a side bar on our webpage that looks like this:

Thus, to add new pages, we add them to the toctree list. They will then show up in that position in the rendered side bar.

What if we want headers in our side bar? To do this, we need to add multiple toctree’s and add a caption. Here’s an example:

Which will result in this rendering:

33.4 Vignettes/tutorials

It is common for packages to have vignettes/tutorials (think demos with narratives) showing how to use the package in a more real-world scenario than the documentation examples show. In your Python package, this ideally might go in an “Examples” section of the docs. The pypkgs-cookiecutter gives a template file for you to use as a starting place (docs/example.ipynb). As indicated in the section above you can rename this file and add others, just make sure you update the toctree in docs/index.md.

Here are some good examples of vignettes/tutorials from packages you know and love:

• https://altair-viz.github.io/case_studies/exploring-weather.html

• https://matplotlib.org/stable/tutorials/images.html

• https://pandas.pydata.org/docs/user_guide/groupby.html

33.4.1 Package websites (via Read the Docs)

The standard practice for hosting and sharing docs in the Python community is to use Read the Docs

• Similar to codecov.io, to use Read the Docs with our package, we need to link it to our GitHub repository

• Read the Docs then checks out the files from the GitHub repo and uses their remote machines to render and serve your documentation

• To do this, Read the Docs needs access to the packages pyproject.toml file. This is done via the creation of a .readthedocs.yml file in the root of your project that looks like this:

# .readthedocs.yaml
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

# Set the OS, Python version and other tools you might need
build:
  os: ubuntu-22.04
  tools:
    python: "3.9"
  jobs:
    post_create_environment:
      # Install poetry
      # https://python-poetry.org/docs/#installing-manually
      - pip install poetry
      # Tell poetry to not use a virtual environment
      - poetry config virtualenvs.create false
    post_install:
      # Install dependencies with 'docs' dependency group
      # https://python-poetry.org/docs/managing-dependencies/#dependency-groups
      - poetry install

# Build documentation in the "docs/" directory with Sphinx
sphinx:
  configuration: docs/conf.py

note 1 - the version of Python specified here has to be a version that your package can be installed with!

note 2 - all your documentation dependencies need to be in pyproject.toml for ReadtheDocs to be able to successfully render your docs!

33.4.2 Sphinx themes

Sometimes you want to have a different theme for your project’s docs. This is possible by changing html_theme in doc.conf.py. See the link below to view and read the docs for other Sphinx themes:

• Sphinx themes gallery: https://sphinx-themes.org/

33.4.3 Documentation metadata in pyproject.toml

To get your packages README and important links to show-up on the TestPyPI and PyPI pages for your package, add the following information to the [tool.poetry] table in pyproject.toml.

readme = "README.md"
homepage = "https://github.com/<github_username>/<github_repo>"
repository = "https://github.com/<github_username>/<github_repo>"
documentation = 'https://<package_name>.readthedocs.io'