# Filenames and data science project organization, Integrated development environments¶

Attribution: much of these notes come from Jenny Bryan’s talk “Naming things” (original slides and source code)

## Topic learning objectives¶

By the end of this topic, students should be able to:

1. Explain how project organization and file naming contribute to reproducible data science

2. Organize projects and name files in a sound manner

3. Use an integrated development environment (IDE) to create, edit and run a script (e.g., VScode in Python, or RStudio in R)

4. Use the IDE to access documentation/help, environment variables and efficiently navigate directories

5. Extend and customize an IDE with useful extensions/add-ons/tools such as linters and formatters

## Filenames - best practices¶

### What works, what doesn’t?¶

NO

myabstract.docx
Joe’s Filenames Use Spaces and Punctuation.xlsx
figure 1.png
fig 2.png
JW7d^(2sl@deletethisandyourcareerisoverWx2*.txt


YES

2014-06-08_abstract-for-sla.docx
joes-filenames-are-getting-better.xlsx
fig01_talk-scatterplot-length-vs-interest.png
fig02_talk-histogram-attendance.png
1986-01-28_raw-data-from-challenger-o-rings.txt


### Three principles for (file) names¶

3. Plays well with default ordering

#### Awesome file names :)¶

• Regular expression and globbing friendly

• Avoid spaces, punctuation, accented characters, case sensitivity

• Easy to compute on

• Deliberate use of delimiters

#### Globbing¶

Excerpt of complete file listing:

Example of globbing to narrow file listing:

#### Punctuation {.smaller}¶

Deliberate use of "-" and "_" allows recovery of meta-data from the filenames:

• "_" underscore used to delimit units of meta-data I want later

• "-" hyphen used to delimit words so my eyes don’t bleed

This happens to be R but also possible in the shell, Python, etc.

#### Recap: machine readable¶

• Easy to search for files later

• Easy to narrow file lists based on names

• Easy to extract info from file names, e.g. by splitting

• New to regular expressions and globbing? be kind to yourself and avoid

• Spaces in file names

• Punctuation

• Accented characters

• Different files named foo and Foo

• Name contains info on content

• Connects to concept of a slug from semantic URLs

#### Example¶

Which set of file(name)s do you want at 3 a.m. before a deadline?

#### Embrace the slug¶

slug filenames ![](img/slug_filenames.png)

slug

#### Recap: Human readable¶

Easy to figure out what the heck something is, based on its name

### Plays well with default ordering¶

#### Plays well with default ordering¶

• Put something numeric first

• Use the ISO 8601 standard for dates

• Left pad other numbers with zeros

#### Examples¶

Chronological order:

Logical order: Put something numeric first

#### Dates¶

Use the ISO 8601 standard for dates: YYYY-MM-DD

#### Left pad other numbers with zeros¶

If you don’t left pad, you get this:

10_final-figs-for-publication.R
1_data-cleaning.R
2_fit-model.R


which is just sad :(

#### Recap: Plays well with default ordering¶

• Put something numeric first

• Use the ISO 8601 standard for dates

• Left pad other numbers with zeros

#### Three principles for (file) names¶

3. Plays well with default ordering

#### Pros¶

• Easy to implement NOW

• Payoffs accumulate as your skills evolve and projects get more complex

## Project organization¶

A good project structure looks similar to this:

project/
├── data/              *.csv
│   ├── processed/
│   └── raw/
├── reports/           *.ipynb *.Rmd
├── src/               *.py *.R
├── doc/               *.md
└── environment.yaml (or renv.lock)


This can differ slightly between projects and for R the src directory is often just called R/, whereas for Python is has the same name as the project (project). You will learn more about project hierarchy when making packages.

### A tour de data science project organization¶

Below we link to three different data science projects shared via GitHub repositories. The first is a data analysis project, while the latter two are data science tools (an R and a Python package, respectively).

As you explore these projects, note the following similarities:

• Files related generally to the project are found in the project root (e.g., README, CODE_OF_CONDUCT.md, computational environment files, etc)

• Code files are generally found in the src or R directory

• data houses raw and processed data

• doc houses documentation files and or documents related to the project (sometimes there is both a doc and reports directory when there are two kinds of writing in a project).

• Other directories can be added as needed to group files with like/similar functions. Most important is that they have descriptive and obvious names related to the directory contents.

## Integrated development environments¶

Integrated development environments (IDEs) are software that provide comprehensive tools for programming in one place.

The classic Jupyter notebook interface is typically not considered an IDE, however the newer, JupyterLab interface is.

Other very popular IDEs for data science include RStudio and VSCode. We will now explore these and look for similarities to identify key IDE features important for developing code for data science.

Note: RStudio was originally developed to work with the R programming language, but now it can work with Python well too! If you would like to use RStudio with Python, please follow the instructions here on how to configure this: https://ttimbers.github.io/intro-to-reticulate/setup-instructions/setup-after-installing-python.html

Note 1: If you would like to use VS Code with Python, then please follow the instructions below on how to configure this:

1. Install the VSCode Python Extension in your VSCode application

2. Code can be sent line-by-line to the Python console via Shift + Enter. The first time you do this, an interactive iPython console will open up. Future Shift + Enter keystrokes during that work session will send code to that console.

3. A bash command line terminal console can be added by clicking the Terminal > New Terminal menu item, and then the “+” symbol and selecting “bash”.

Note 2: If you would like to use VS Code with R, please follow the instructions below on how to configure this:

1. Install the VSCode R Extension in your VSCode application

2. In the R console, install languageserver R package via install.packages("languageserver")

3. Install the radian Python package (which gives an improved R console in VS Code) via conda install radian

4. In the VS Code settings (found by clicking the Code > Preferences > Settings menu item) search for “r.path” and paste the path to radian under the textbox for your operating system (you can find the path to radian by typing which radian in the terminal). For example, on my Mac x86 laptop, the path is /Users/tiffany/opt/miniconda3/bin/radian

5. In the VS Code settings (Code > Preferences > Settings) search for “R: Bracketed Paste” and select the checkbox for this setting (this allows you to send code to the R console when it is split across lines without it breaking).

6. An interactive R console can be added by clicking the Terminal > New Terminal menu item, and then the “+” symbol and selecting “R Terminal”. Code can be sent line-by-line to the R console via Cmd + Enter (Mac) or Control + Enter (Windows).

### Key features of all of these IDEs includes:¶

• screen split into panes

• file browser

• convenient access to R/Python console and a terminal

• editor for code writing

• code autocompletion

• code/syntax highlighting