--- title: "1.3 Working with Files and Paths" --- ```{r} #| label: setup #| include: false knitr::opts_chunk$set( echo = TRUE, message = FALSE, warning = FALSE, fig.width = 6, fig.height = 3.6, fig.align = "center", dpi = 120, error = TRUE, fig.path = "_figs/files-and-paths/" ) library(tidyverse) ``` Find a file, tell R where it is, import it, and save the result. When working with data in R, we often need to: 1. Find a file on the computer. 2. Tell R where the file is located. 3. Understand how the data inside the file is organized. 4. Choose the appropriate function to import it. 5. Inspect and work with the data. 6. Save or export the results. A useful workflow is: Find → Path → Identify Format → Import → Inspect → Work → Export Work in an RStudio project so `getwd()` is the project folder. Download [students.csv](files/students.csv) and place it in a `data/` folder inside that project (or in the project folder itself). Note **1.2** is how you install and load the tidyverse. The `read_csv()` examples below assume `library(tidyverse)` has already been run, or you can write `readr::read_csv()`. --- ## 1. Files, Folders, and Paths A **path** tells R where a file or folder is located. Suppose our project has the following structure: my_project/ │ ├── analysis.R │ ├── data/ │ └── students.csv │ └── output/ The file `students.csv` is located inside the `data` folder. There are two main ways to describe its location: - Absolute path - Relative path --- ## 2. Absolute Paths An **absolute path** gives the complete location of a file on the computer. For example: ```{r} #| label: chunk-1 "C:/Users/John/Documents/my_project/data/students.csv" ``` We could therefore write: ```{r} #| label: chunk-2 #| eval: false students <- read.csv( "C:/Users/John/Documents/my_project/data/students.csv" ) ``` This may work on John's computer, but probably not on someone else's computer. For example, another student may have the project at: C:/Users/Sara/Desktop/my_project/ Therefore, absolute paths can make code difficult to share. Do not put them in work you submit. Use `/` in paths in R, even on Windows. --- ## 3. The Working Directory R has a **working directory**. Think of the working directory as: > **R's current location or starting point.** To find the current working directory: ```{r} #| label: chunk-3 getwd() ``` Think: getwd() = Where am I? For example, suppose: ```{r} #| label: chunk-4 getwd() ``` returns: "C:/Users/John/Documents/my_project" Then R currently considers `my_project` its starting location. In RStudio, **File → New Project** pins the working directory to the project folder. --- ## 4. Relative Paths A **relative path** describes the location of a file relative to our current location. Suppose: my_project/ │ ├── analysis.R │ ├── data/ │ └── students.csv │ └── output/ and our working directory is: my_project/ Instead of writing: ```{r} #| label: chunk-5 "C:/Users/John/Documents/my_project/data/students.csv" ``` we can simply write: ```{r} #| label: chunk-6 "data/students.csv" ``` Therefore: ```{r} #| label: chunk-7 #| eval: false students <- read.csv("data/students.csv") ``` means: > Starting from the current directory, go into the `data` folder and find `students.csv`. Relative paths are usually preferable because they make projects easier to move and share. --- ## 5. Understanding `.`, `..`, and `/` Three symbols are particularly useful when working with relative paths. Symbol Meaning `.` Current directory `..` Parent directory — one folder up `/` Separates folders ### `.` means "here" These are generally equivalent: ```{r} #| label: chunk-8 #| eval: false read.csv("data/students.csv") ``` and: ```{r} #| label: chunk-9 #| eval: false read.csv("./data/students.csv") ``` The second can be read as: > Start here → enter `data` → find `students.csv`. ### `..` means "go up one folder" Suppose: my_project/ │ ├── data/ │ └── students.csv │ └── scripts/ └── analysis.R If our current location is: my_project/scripts/ then the relative path to `students.csv` is: ```{r} #| label: chunk-10 "../data/students.csv" ``` Read this as: .. → go up to my_project data/ → enter the data folder students.csv → find the file So: ```{r} #| label: chunk-11 #| eval: false students <- read.csv("../data/students.csv") ``` ### Going up multiple folders Suppose: my_project/ │ ├── data/ │ └── students.csv │ └── R/ └── week1/ └── analysis.R If our current directory is: my_project/R/week1/ then: ```{r} #| label: chunk-12 "../../data/students.csv" ``` means: .. → R/ ../.. → my_project/ data/ → my_project/data/ 6. Why Prefer Relative Paths? Compare: ```{r} #| label: chunk-13 #| eval: false # Absolute path read.csv( "C:/Users/John/Documents/my_project/data/students.csv" ) ``` with: ```{r} #| label: chunk-14 #| eval: false # Relative path read.csv("data/students.csv") ``` If we send the entire `my_project` folder to another person, the absolute path will probably not work. The relative path can still work because the relationship between the folders has not changed. Therefore: > **When working within an R project, use relative paths whenever possible.** --- ## 7. Finding Files in R Before importing a file, we may need to determine where it is. Start with: ```{r} #| label: chunk-15 getwd() ``` > Where am I? Then: ```{r} #| label: chunk-16 list.files() ``` > What files and folders are here? For example: ```{r} #| label: chunk-17 list.files() ``` might return: "analysis.R" "data" "output" We can look inside a particular folder: ```{r} #| label: chunk-18 list.files("data") ``` which might return: "students.csv" The function: ```{r} #| label: chunk-19 dir() ``` can also be used to list files. --- ## 8. Searching for a Particular Type of File Suppose we want to find CSV files: ```{r} #| label: chunk-20 list.files(pattern = "\\.csv$") ``` To search inside subfolders as well: ```{r} #| label: chunk-21 list.files( recursive = TRUE, pattern = "\\.csv$" ) ``` This might return: "data/students.csv" To return full paths: ```{r} #| label: chunk-22 list.files( recursive = TRUE, pattern = "\\.csv$", full.names = TRUE ) ``` --- ## 9. Does the File Exist? Before importing: ```{r} #| label: chunk-23 file.exists("data/students.csv") ``` If R returns: ```{r} #| label: chunk-24 TRUE ``` the file exists at that path. If R returns: ```{r} #| label: chunk-25 FALSE ``` check: ```{r} #| label: chunk-26 getwd() list.files() list.files("data") ``` For directories: ```{r} #| label: chunk-27 dir.exists("data") ``` --- ## 10. Constructing Paths R provides `file.path()`: ```{r} #| label: chunk-28 file.path("data", "students.csv") ``` For longer paths: ```{r} #| label: chunk-29 file.path("data", "raw", "students.csv") ``` This is useful for constructing paths from individual folder and file names. --- ## 11. Selecting a File Manually Base R provides: ```{r} #| label: chunk-30 #| eval: false file.choose() ``` This opens a file-selection window. For example: ```{r} #| label: chunk-31 #| eval: false path <- file.choose() ``` Then: ```{r} #| label: chunk-32 path ``` shows the selected path. We could import the selected file using: ```{r} #| label: chunk-33 students <- read.csv(path) ``` This is convenient for quick interactive work. For reproducible scripts, however, relative paths are usually preferable: ```{r} #| label: chunk-34 #| eval: false students <- read.csv("data/students.csv") ``` --- ## 12. Before Importing: What Is Inside the File? Finding the file is only part of the problem. We also need to understand **how the data inside the file is organized**. Many data files are **delimited text files**. A **delimiter** is the character used to separate values. For example: name,age,grade John,20,85 Sara,21,92 The values are separated by: , Therefore, this is **comma-separated data**. Now consider: name age grade John 20 85 Sara 21 92 If the spaces between values are tab characters, this is **tab-separated data**. Other delimiters are also possible: John;20;85 uses a semicolon. And: John|20|85 uses a pipe character. The course file `students.csv` uses commas. Its columns are `name`, `grade`, and `passed`. The `name,age,grade` lines above are only to show what a delimiter looks like. --- ## 13. How Do We Determine the Delimiter? The file extension gives us a useful clue: Extension Usually means `.csv` Comma-separated values `.tsv` Tab-separated values `.txt` General text file; delimiter may vary However: > **The file extension is a clue, not a guarantee.** Someone could name a semicolon-separated file `students.csv`. Therefore, for an unfamiliar file, we can inspect its first few lines before importing it. Base R provides: ```{r} #| label: chunk-35 readLines() ``` For example: ```{r} #| label: chunk-36 readLines( "data/students.csv", n = 3 ) ``` R might display: "name,age,grade" "John,20,85" "Sara,21,92" We can clearly see commas. Or R might display: "name\tage\tgrade" "John\t20\t85" "Sara\t21\t92" Here: \t represents a **tab character**. Therefore, the data is tab-separated. --- ## 14. Choosing the Correct Import Function Once we know the delimiter, we can choose an appropriate function. Delimiter Example Base R readr/tidyverse Comma `,` `John,20,85` `read.csv()` `read_csv()` Tab `\t` `John 20 85` `read.delim()` `read_tsv()` Other `John;20;85` `read.table(sep = ";")` `read_delim(delim = ";")` For comma-separated data: ```{r} #| label: chunk-37 #| eval: false students <- read.csv("data/students.csv") ``` For tab-separated data: ```{r} #| label: chunk-38 students <- read.delim("data/students.tsv") ``` For another delimiter, such as `;`: ```{r} #| label: chunk-39 students <- read.table( "data/students.txt", sep = ";", header = TRUE ) ``` With `readr`: ```{r} #| label: chunk-40 #| eval: false students <- read_delim( "data/students.txt", delim = ";" ) ``` --- ## 15. The Relationship Between the Base R Functions The functions: ```{r} #| label: chunk-41 read.csv() read.delim() ``` are convenient variations of the more general: ```{r} #| label: chunk-42 read.table() ``` The main difference is that they provide useful default arguments for particular file formats. Conceptually: read.table() │ ├── comma separator → read.csv() │ └── tab separator → read.delim() Therefore, these are not completely unrelated functions. The difference is **default arguments**, which note **1.4** discusses in more detail. Similarly, `readr` provides: read_csv() → comma-separated read_tsv() → tab-separated read_delim() → specify another delimiter 16. Importing a CSV File CSV stands for: > **Comma-Separated Values** For example: name,age,grade John,20,85 Sara,21,92 David,19,78 Each row represents an observation, and each column represents a variable. Using Base R: ```{r} #| label: chunk-43 #| eval: false students <- read.csv("data/students.csv") ``` Using `readr`: ```{r} #| label: chunk-44 #| eval: false students <- read_csv("data/students.csv") ``` --- ## 17. File vs. R Object It is important to understand what happens when we import data: FILE ON COMPUTER OBJECT IN R students.csv --read.csv()--> students `students.csv` is a **file stored on the computer**. `students` is an **R object stored in memory**. They are not the same thing. --- ## 18. Inspecting Imported Data Never assume that the file was imported correctly. Start with: ```{r} #| label: chunk-45 head(students) ``` Then: ```{r} #| label: chunk-46 str(students) ``` and: ```{r} #| label: chunk-47 summary(students) ``` Other useful functions include: ```{r} #| label: chunk-48 tail(students) dim(students) nrow(students) ncol(students) names(students) ``` A useful habit is: ```{r} #| label: chunk-49 head(students) str(students) summary(students) ``` --- ## 19. A Common Sign of the Wrong Delimiter Suppose the original file contains: name,age,grade John,20,85 Sara,21,92 We expect **three columns**: name | age | grade If R imports everything as **one column**, something is probably wrong. One common reason is: > **The wrong delimiter was used.** Therefore, if an imported dataset does not look correct: ```{r} #| label: chunk-50 head(students) str(students) ``` Then inspect the original file: ```{r} #| label: chunk-51 readLines( "data/students.csv", n = 3 ) ``` and verify the delimiter. --- ## 20. Working with Imported Data Once imported, we normally work with the **R object**, not directly with the original file. For example: ```{r} #| label: chunk-52 students$grade ``` Calculate: ```{r} #| label: chunk-53 mean(students$grade) ``` or create a variable: ```{r} #| label: chunk-54 students$passed <- students$grade >= 80 ``` The R object has changed. However: > **Changing an R object does not automatically change the original file.** The original: data/students.csv remains unchanged. --- ## 21. Exporting Data with Base R To save our modified data: ```{r} #| label: chunk-55 #| eval: false write.csv( students, "output/students_updated.csv", row.names = FALSE ) ``` The workflow is: students.csv │ │ read.csv() ▼ R object: students │ │ analyze / modify ▼ Modified R object │ │ write.csv() ▼ students_updated.csv 22. Exporting with readr/tidyverse Using `readr`: ```{r} #| label: chunk-56 #| eval: false write_csv( students, "output/students_updated.csv" ) ``` The main functions can therefore be summarized as: Task Base R readr/tidyverse Read comma-separated `read.csv()` `read_csv()` Read tab-separated `read.delim()` `read_tsv()` Read other delimiter `read.table()` `read_delim()` Write CSV `write.csv()` `write_csv()` --- ## 23. Other Useful File Functions ### Current directory ```{r} #| label: chunk-57 getwd() ``` ### List files ```{r} #| label: chunk-58 list.files() ``` ### Preview a text file ```{r} #| label: chunk-59 readLines("data/students.csv", n = 3) ``` ### Check whether a file exists ```{r} #| label: chunk-60 file.exists("data/students.csv") ``` ### Check whether a directory exists ```{r} #| label: chunk-61 dir.exists("data") ``` ### Construct a path ```{r} #| label: chunk-62 file.path("data", "students.csv") ``` ### Select a file manually ```{r} #| label: chunk-63 #| eval: false file.choose() ``` ### Get file information ```{r} #| label: chunk-64 file.info("data/students.csv") ``` ### Create a directory ```{r} #| label: chunk-65 dir.create("results") ``` ### Copy a file ```{r} #| label: chunk-66 file.copy( "data/students.csv", "output/students_copy.csv" ) ``` ### Rename a file ```{r} #| label: chunk-67 file.rename( "output/students_copy.csv", "output/backup.csv" ) ``` ### Delete a file ```{r} #| label: chunk-68 file.remove("output/backup.csv") ``` You do not need to memorize all these functions. The important goal is to understand what each type of operation does. --- ## 24. Other Common File Types CSV is only one format. File type Base R / Common function readr/tidyverse `.csv` `read.csv()` `read_csv()` `.tsv` `read.delim()` `read_tsv()` `.txt` `read.table()` `read_delim()` `.rds` `readRDS()` — `.RData` `load()` — `.xlsx` `readxl::read_excel()` `readxl::read_excel()` The exact function changes depending on the file format, but the general workflow remains the same. --- ## 25. Saving R Objects Sometimes we want to save an R object in R's own format rather than exporting it as CSV. ```{r} #| label: chunk-69 #| eval: false saveRDS( students, "output/students.rds" ) ``` Later: ```{r} #| label: chunk-70 students <- readRDS( "output/students.rds" ) ``` This preserves an R object rather than converting it to a general text format such as CSV. --- ## 26. Complete Base R Workflow ```{r} #| label: chunk-71 #| eval: false # 1. Where am I? getwd() # 2. What files are here? list.files() list.files("data") # 3. Does the file exist? file.exists("data/students.csv") # 4. Look inside the file readLines( "data/students.csv", n = 3 ) # 5. Import using the appropriate function students <- read.csv( "data/students.csv" ) # 6. Inspect head(students) str(students) summary(students) # 7. Work with the data students$passed <- students$grade >= 80 # 8. Export write.csv( students, "output/students_updated.csv", row.names = FALSE ) ``` --- ## 27. The Same Basic Workflow with Tidyverse Tools ```{r} #| label: chunk-72 #| eval: false library(tidyverse) # Import students <- read_csv( "data/students.csv" ) # Inspect students glimpse(students) summary(students) # Export write_csv( students, "output/students_updated.csv" ) ``` Load `tidyverse` as in note **1.2**. We will learn additional tidyverse tools for manipulating data later. --- ## 28. The File Workflow to Remember When working with an unfamiliar data file, ask these questions: WHERE AM I? getwd() │ ▼ WHAT FILES ARE HERE? list.files() │ ▼ WHERE IS MY FILE? relative path │ ▼ DOES IT EXIST? file.exists() │ ▼ WHAT IS INSIDE IT? readLines() │ ▼ WHAT IS THE DELIMITER? comma / tab / other │ ▼ CHOOSE THE READER read.csv() / read.delim() read_csv() / read_tsv() │ ▼ IMPORT │ ▼ INSPECT head(), str(), summary() │ ▼ WORK WITH DATA │ ▼ SAVE / EXPORT write.csv() / write_csv() Three ideas to remember **1. A path tells R where a file is.** **2. A delimiter tells R how values inside a text file are separated.** **3. An import function reads the file and creates an R object that we can work with.** For an unfamiliar text data file: ```{r} #| label: chunk-73 #| eval: false # Find/check it file.exists("data/students.csv") # Look inside readLines("data/students.csv", n = 3) # Choose the appropriate reader students <- read.csv("data/students.csv") # Check that it worked head(students) str(students) ``` This habit—**find → inspect → import → verify**—will prevent many common file-import problems. Lab 1 asks you to import `students.csv`. Put it where a relative path can see it.