Continuous Deployment and Publishing
Reproducible and Trustworthy Workflows for Data Science
Welcome
Introduction
1
How do reproducible and trustworthy workflows impact data science?
2
Introduction to the Bash Shell
Version Control with Git
3
SSH for authentication
4
Git basics: a review
5
Merge conflicts
6
Ignoring files with
.gitignore
7
Going back in time with Git
8
Git branches
Collaborating on GitHub
9
Pull requests
10
Code review on GitHub
11
Branching workflows: GitHub flow and Git flow
12
Project management on GitHub
13
Main branch protection
Project Organization
14
Filenames and data science project organization, Integrated development environments
Virtual Environments
15
Virtual environments
16
Virtual environments: R (renv)
17
Virtual environments: Python (uv)
18
Virtual environments: Python (conda)
19
Virtual environments: Python (conda-lock)
Containers
20
Introduction to containerization
21
Using and running containers
22
Containers: Docker Compose
23
Customizing and building containers
Data Validation
24
Data validation
25
Data validation: R (pointblank)
26
Data validation: Python (Pandera)
27
Data validation: Python (Deepchecks)
28
Data validation: Python (pointblank)
Scripts and Reports
29
Non-interactive scripts
30
Reproducible reports: Quarto
Pipelines
31
Data analysis pipelines with scripts
32
Data analysis pipelines with GNU Make
Software Testing for Data Science
33
Introduction to testing code for data science
34
Testing: R (testthat)
35
Testing: Python (pytest)
36
Testing: images
37
Testing: evaluating test suite quality
Packaging
38
Introduction to R & Python packages
39
Packaging: Python
40
Packaging: Python (uv)
41
Packaging: Python (Poetry)
42
Packaging: Python (Hatch)
43
Packaging: R
44
Packaging: R and Python tool philosophies
Testing Packages
45
Package testing: Python (pytest)
46
Package coverage: Python
Continuous Integration
47
Automated testing and continuous integration
48
GitHub Actions
49
GitHub Actions: matrix workflows
50
GitHub Actions: testing workflows
51
Case study: Python CI workflow (
pypkgs-cookiecutter
’s
ci.yml
)
52
Case study: R CI workflow (a simplified
check-release.yaml
)
Package Documentation
53
Package documentation: Python (Sphinx)
54
Package documentation: Python (quartodoc)
55
Package documentation: R
Continuous Deployment and Publishing
56
Deploying and publishing packages
57
Continuous deployment: Python (Poetry)
58
Continuous deployment: Python (Hatch and quartodoc)
59
Continuous deployment: R
60
Package peer review
61
Package versioning: Python
Licenses and Copyright
62
Copyright and licenses
Wrap Up
63
Workflows for reproducible and trustworthy data science wrap-up
Appendix
64
Defining functions in Python
65
Defining functions in R
66
Non-standard evaluation in R
67
Reproducible reports: R Markdown and Jupyter
68
Quarto and GitHub Pages
Archive
Continuous Deployment and Publishing
55
Package documentation: R
56
Deploying and publishing packages