5.1 dplyr: rowwise() and across()
rowwise() summarizes across columns in one row. across() repeats a function on many columns.
Most calculations on a data frame go one of two ways: down a
column (the mean of exam1 over all students) or
across a row (one student’s mean over three exams).
Knowing which one you want tells you whether to reach for
across(), rowwise() with
c_across(), or rowSums() /
rowMeans().
1. mutate() works on whole columns
Before rowwise(), look at what mutate()
normally does.
exams <- tibble(
student = c("A", "B", "C"),
exam1 = c(80, 70, 90),
exam2 = c(90, 85, 95),
exam3 = c(70, 75, 85)
)
exams |> mutate(difference = exam1 - exam2, total = exam1 + exam2 + exam3)## # A tibble: 3 × 6
## student exam1 exam2 exam3 difference total
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 A 80 90 70 -10 240
## 2 B 70 85 75 -15 230
## 3 C 90 95 85 -5 270It looks as if mutate() subtracts row by row. It does
not. Inside mutate(), exam1 and
exam2 are whole columns, so R computes
c(80, 70, 90) - c(90, 85, 95)## [1] -10 -15 -5- and + are vectorized:
they already work element by element. No rowwise() is
needed.
2. Why mean() needs rowwise()
Now try the average score of each student:
exams |> mutate(avg = mean(c(exam1, exam2, exam3)))## # A tibble: 3 × 5
## student exam1 exam2 exam3 avg
## <chr> <dbl> <dbl> <dbl> <dbl>
## 1 A 80 90 70 82.2
## 2 B 70 85 75 82.2
## 3 C 90 95 85 82.2Every row gets the same number. c(exam1, exam2, exam3)
glues the three whole columns into one vector of nine
values,
c(exams$exam1, exams$exam2, exams$exam3)## [1] 80 70 90 90 85 95 70 75 85and mean() returns one mean of all nine, repeated on
every row.
rowwise() changes the context: the next verbs run
once per row.
exams |>
rowwise() |>
mutate(avg = mean(c(exam1, exam2, exam3))) |>
ungroup()## # A tibble: 3 × 5
## student exam1 exam2 exam3 avg
## <chr> <dbl> <dbl> <dbl> <dbl>
## 1 A 80 90 70 80
## 2 B 70 85 75 76.7
## 3 C 90 95 85 90Row 1 is mean(c(80, 90, 70)) = 80, row 2 is
mean(c(70, 85, 75)) = 76.67, row 3 is
mean(c(90, 95, 85)) = 90.
rowwise() is a group of one
group_by(carrier) makes one group per carrier, each with
many rows. rowwise() makes each row its own
group. So:
group_by(): the context is a group of rows.rowwise(): the context is one row.
rowwise(name) also keeps name as an
identifier when you summarize():
df <- tibble(name = c("Mara", "Hadley"), x = 1:2, y = 3:4, z = 5:6)
df |> mutate(m = mean(c(x, y, z)))## # A tibble: 2 × 5
## name x y z m
## <chr> <int> <int> <int> <dbl>
## 1 Mara 1 3 5 3.5
## 2 Hadley 2 4 6 3.5# one mean for the whole table
df |>
rowwise(name) |>
summarize(m = mean(c(x, y, z)), .groups = "drop")## # A tibble: 2 × 2
## name m
## <chr> <dbl>
## 1 Mara 3
## 2 Hadley 4# one mean per rowFinish with ungroup() (or
.groups = "drop"). Otherwise later verbs keep working one
row at a time.
3. c_across(): the values of the current row
Typing c(exam1, exam2, exam3) gets long. Inside
rowwise(), c_across(exam1:exam3) collects the
values of those columns in the current row into a
vector. For student A it is c(80, 90, 70).
exams |>
rowwise() |>
mutate(
avg_score = mean(c_across(exam1:exam3)),
max_score = max(c_across(exam1:exam3)),
min_score = min(c_across(exam1:exam3)),
sd_score = sd(c_across(exam1:exam3))
) |>
ungroup()## # A tibble: 3 × 8
## student exam1 exam2 exam3 avg_score max_score min_score sd_score
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 A 80 90 70 80 90 70 10
## 2 B 70 85 75 76.7 85 70 7.64
## 3 C 90 95 85 90 95 85 5c_across() selects columns with the same helpers as
select():
df <- tibble(id = 1:6, w = 10:15, x = 20:25, y = 30:35, z = 40:45)
rf <- df |> rowwise(id)
rf |> mutate(total = sum(c_across(w:z)))## # A tibble: 6 × 6
## # Rowwise: id
## id w x y z total
## <int> <int> <int> <int> <int> <int>
## 1 1 10 20 30 40 100
## 2 2 11 21 31 41 104
## 3 3 12 22 32 42 108
## 4 4 13 23 33 43 112
## 5 5 14 24 34 44 116
## 6 6 15 25 35 45 120rf |> mutate(total = sum(c_across(where(is.numeric))))## # A tibble: 6 × 6
## # Rowwise: id
## id w x y z total
## <int> <int> <int> <int> <int> <int>
## 1 1 10 20 30 40 100
## 2 2 11 21 31 41 104
## 3 3 12 22 32 42 108
## 4 4 13 23 33 43 112
## 5 5 14 24 34 44 116
## 6 6 15 25 35 45 120Missing values
na.rm = TRUE works as usual. It drops the
NAs in that row only.
quizzes <- tibble(
student = c("A", "B", "C"),
quiz1 = c(80, 90, NA),
quiz2 = c(70, NA, 85),
quiz3 = c(90, 80, 95)
)
quizzes |>
rowwise() |>
mutate(quiz_avg = mean(c_across(quiz1:quiz3), na.rm = TRUE)) |>
ungroup()## # A tibble: 3 × 5
## student quiz1 quiz2 quiz3 quiz_avg
## <chr> <dbl> <dbl> <dbl> <dbl>
## 1 A 80 70 90 80
## 2 B 90 NA 80 85
## 3 C NA 85 95 90For B this is
mean(c(90, NA, 80), na.rm = TRUE) = 85.
starwars (in dplyr) is the usual practice table. Without
rowwise(), max(height, mass, birth_year) is
one number for the whole column stack, repeated on every row.
4. across() repeats a function on columns
across() lives inside
mutate(), summarize(), filter(),
and the other data-masking verbs. It does not change the data by
itself.
Two arguments:
.cols: tidyselect, defaulteverything().fns: a function, a named list of functions, or a formula~ median(.x, na.rm = TRUE)..xis the current column.
Write where(is.numeric), not
where(is.numeric()). where() wants the
function itself, not the result of calling it.
starwars |>
summarize(across(where(is.numeric), ~ median(.x, na.rm = TRUE)), .groups = "drop")## # A tibble: 1 × 3
## height mass birth_year
## <int> <dbl> <dbl>
## 1 180 79 52starwars |>
group_by(species) |>
filter(n() > 1) |>
summarize(
across(c(sex, gender, homeworld), ~ length(unique(.x))),
n = n()
)## # A tibble: 9 × 5
## species sex gender homeworld n
## <chr> <int> <int> <int> <int>
## 1 Droid 1 2 3 6
## 2 Gungan 1 1 1 3
## 3 Human 2 2 15 35
## 4 Kaminoan 2 2 1 2
## 5 Mirialan 1 1 1 2
## 6 Twi'lek 2 2 1 2
## 7 Wookiee 1 1 1 2
## 8 Zabrak 1 1 2 2
## 9 <NA> 1 1 3 4Grouping variables are skipped so you do not summarize them by accident.
A named list makes several summaries per column:
min_max <- list(
min = ~ min(.x, na.rm = TRUE),
max = ~ max(.x, na.rm = TRUE)
)
starwars |> summarize(across(where(is.numeric), min_max))## # A tibble: 1 × 6
## height_min height_max mass_min mass_max birth_year_min birth_year_max
## <int> <int> <dbl> <dbl> <dbl> <dbl>
## 1 66 264 15 1358 8 896The new columns are named column_function:
height_min, height_max, mass_min,
and so on.
Here across() is the right tool because we want
statistics for each column separately. Combining one
character’s height, mass, and birth year with c_across()
would not mean anything.
Order matters. If you compute n = n() first,
n is numeric, so a later
across(where(is.numeric), sd) will try to take the SD of
that constant and give NA. Put n = n()
last.
across() replaces the old *_if(),
*_at(), and *_all helpers.
select() and rename() already use tidyselect;
use rename_with() to change names with a function.
5. across() vs c_across()
The names are close, the jobs are not.
across()applies a function to each selected column separately.c_across()combines the selected values within the current row.
xyz <- tibble(name = c("A", "B", "C"), x = c(10, 20, 30), y = c(20, 40, 60), z = c(30, 60, 90))One mean per column (mean(x),
mean(y), mean(z)):
xyz |> summarize(across(x:z, mean))## # A tibble: 1 × 3
## x y z
## <dbl> <dbl> <dbl>
## 1 20 40 60xyz |> mutate(across(x:z, ~ .x * 2))## # A tibble: 3 × 4
## name x y z
## <chr> <dbl> <dbl> <dbl>
## 1 A 20 40 60
## 2 B 40 80 120
## 3 C 60 120 180One mean per row (mean(c(10, 20, 30)),
mean(c(20, 40, 60)), …):
xyz |>
rowwise() |>
mutate(row_mean = mean(c_across(x:z))) |>
ungroup()## # A tibble: 3 × 5
## name x y z row_mean
## <chr> <dbl> <dbl> <dbl> <dbl>
## 1 A 10 20 30 20
## 2 B 20 40 60 40
## 3 C 30 60 90 60A picture to remember:
x y z
----------------
Row 1 10 20 30 -> c_across()
Row 2 20 40 60 -> c_across()
Row 3 30 60 90 -> c_across()
| | |
v v v
across()across() goes down the columns. c_across()
goes along a row. The c is for combine, as
in c().
6. rowSums() and rowMeans()
For sums and means across a row, base R already has vectorized
functions. They take a set of columns, and across() hands
them over. No rowwise() is needed.
Two ways to get each student’s total:
exams |>
rowwise() |>
mutate(total = sum(c_across(exam1:exam3))) |>
ungroup()## # A tibble: 3 × 5
## student exam1 exam2 exam3 total
## <chr> <dbl> <dbl> <dbl> <dbl>
## 1 A 80 90 70 240
## 2 B 70 85 75 230
## 3 C 90 95 85 270exams |> mutate(total = rowSums(across(exam1:exam3)))## # A tibble: 3 × 5
## student exam1 exam2 exam3 total
## <chr> <dbl> <dbl> <dbl> <dbl>
## 1 A 80 90 70 240
## 2 B 70 85 75 230
## 3 C 90 95 85 270Same answer. The second is shorter and much faster on a big table,
because it does not split the data into one group per row.
rowMeans() works the same way, and both take
na.rm = TRUE:
quizzes |>
mutate(
quiz_avg = rowMeans(across(quiz1:quiz3), na.rm = TRUE),
quiz_total = rowSums(across(quiz1:quiz3), na.rm = TRUE)
)## # A tibble: 3 × 6
## student quiz1 quiz2 quiz3 quiz_avg quiz_total
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 A 80 70 90 80 240
## 2 B 90 NA 80 85 170
## 3 C NA 85 95 90 180It also works with where():
df |> mutate(total = rowSums(across(where(is.numeric))))## # A tibble: 6 × 6
## id w x y z total
## <int> <int> <int> <int> <int> <dbl>
## 1 1 10 20 30 40 101
## 2 2 11 21 31 41 106
## 3 3 12 22 32 42 111
## 4 4 13 23 33 43 116
## 5 5 14 24 34 44 121
## 6 6 15 25 35 45 126So why learn rowwise()? Because there is no
rowSd() or rowMax() in base R. For any other
per-row statistic, rowwise() with c_across()
is the general tool:
exams |>
rowwise() |>
mutate(highest = max(c_across(exam1:exam3)), spread = sd(c_across(exam1:exam3))) |>
ungroup()## # A tibble: 3 × 6
## student exam1 exam2 exam3 highest spread
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 A 80 90 70 90 10
## 2 B 70 85 75 85 7.64
## 3 C 90 95 85 95 57. When you do not need rowwise()
Using several columns does not by itself call for
rowwise(). These are all vectorized:
xyz |> mutate(total = x + y + z, difference = x - y, ratio = x / y, bigger = x > y)## # A tibble: 3 × 8
## name x y z total difference ratio bigger
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <lgl>
## 1 A 10 20 30 60 -10 0.5 FALSE
## 2 B 20 40 60 120 -20 0.5 FALSE
## 3 C 30 60 90 180 -30 0.5 FALSErowwise() is for functions like mean(),
max(), min(), sd() that take a
vector and return one value, when you want that value
for each row.
So do not ask “Am I using several columns?” Ask:
- Same function on each column separately? Use
across(). - Combine values from several columns within each
row? Use
rowwise()withc_across()(orrowSums()/rowMeans()for sums and means).
8. Choosing the right tool
| Goal | Typical code |
|---|---|
| One statistic of one column | summarize(avg = mean(exam1)) |
| The same statistic for several columns | summarize(across(exam1:exam3, mean)) |
| The same change to several columns | mutate(across(exam1:exam3, ~ .x + 5)) |
| Arithmetic that combines columns | mutate(total = exam1 + exam2 + exam3) |
| Sum across each row | mutate(total = rowSums(across(exam1:exam3))) |
| Mean across each row | mutate(avg = rowMeans(across(exam1:exam3))) |
| Any other statistic across each row | rowwise(), then
mutate(top = max(c_across(exam1:exam3))), then
ungroup() |
Remember: across() keeps the selected columns
separate. c_across()
combines the selected values of the current row into
one vector. And before you type rowwise(), check whether
the operation is already vectorized, or whether rowSums() /
rowMeans() already does the job.
9. Practice
- For each
starwarscharacter, the maximum ofheight,mass, andbirth_year(na.rm = TRUE). Who is largest? Try it with and withoutrowwise(). - Median of every numeric column, by
speciesandgender, plusn(), sorted byndescending.