Lesson 15 of 17 · Week 07 · Tue Nov 10

Reshaping Data with tidyr

Questions

How do I change the layout of a data frame?

Objectives

Tell long, wide and intermediate layouts apart.
Reshape with pivot_longer() and pivot_wider().
Summarize several metrics at once in long form.

Long, wide and in between

The same numbers can be laid out in different shapes:

Long: one row per single measurement, with columns saying what was measured (country, year, metric, value).
Wide: one row per unit (say, a country), with a column for every year × metric combination (country, lifeExp_1967, lifeExp_2007, pop_1967, …).
In between: gm itself is one row per country-year with a column per metric.

Data is tidy when each variable is a column, each observation is a row, and each value has its own cell. ggplot2 and dplyr work best on tidy (usually longer) data. Tables for people to read are often wider. The tidyr package moves between the shapes.

Adapted from Software Carpentry's “Data Frame Manipulation with tidyr” episode (CC BY 4.0, The Carpentries). Text, examples and exercises have been rewritten for this course.

Wide to long: pivot_longer()

library(tidyr)
library(dplyr)
gm_long <- gm |>
  pivot_longer(cols = c(lifeExp, pop, gdpPercap),   # the columns to stack
               names_to = "metric",                 # their names go here
               values_to = "value")                 # their values go here
head(gm_long)                                       # 3 rows per country-year now

Choose columns by name, by range (lifeExp:gdpPercap), or by exclusion (-c(country, continent, year)).

Long to wide: pivot_wider()

gm_long |>
  pivot_wider(names_from = metric, values_from = value)       # back to gm's shape

gm |>
  select(country, year, lifeExp) |>
  pivot_wider(names_from = year, values_from = lifeExp,
              names_prefix = "le_")                          # one column per year

names_from can be several columns; their values are glued together (with names_sep) into the new column names. Every row of the wide result is one combination of the columns you didn't use.

Long data + dplyr

gm_long |>
  group_by(continent, metric) |>
  summarize(mean_value = mean(value))

In long form, one grouped summary covers all three metrics at once.

Exercises

Each exercise gives you up to 10 checked attempts. Use Run as many times as you like to experiment first, then click Check answer when you're ready. Your exercises use numbers/subsets unique to you, so answers won't match a classmate's exactly.

Exercise 1 — Which shape?

0 / 10 attempts used

Click Run to print three small tables built from the same data (in a random order). For each, decide whether it's "long", "wide" or "intermediate" (as defined above), and store your three answers, in order, in shapes.

Pasting isn't accepted here — please type your answer so it reflects your own work.

Exercise 2 — Summarize in long form

0 / 10 attempts used

Make gm_long by pivoting lifeExp, pop and gdpPercap into metric/value columns, using only the rows from year yr (click Run). Then make means: the mean of each metric for each continent, with columns continent, metric and mean_value.

Pasting isn't accepted here — please type your answer so it reflects your own work.

Exercise 3 — Back to the start

0 / 10 attempts used

gm_long (all years, three metrics) is ready for you. Use pivot_wider() to turn it back into gm_back, with one row per country-year and columns lifeExp, pop and gdpPercap. Check that it matches gm.

Pasting isn't accepted here — please type your answer so it reflects your own work.

Exercise 4 — Ludicrously wide

0 / 10 attempts used

Starting from gm_long restricted to the two years shown when you click Run, make super_wide with one row per continent. Every other combination of metric, country and year becomes its own column (use names_from = c(metric, country, year)). Why are so many cells NA?

Pasting isn't accepted here — please type your answer so it reflects your own work.

Key Points

Tidy data: variables in columns, observations in rows, one value per cell.
pivot_longer() stacks columns into name/value pairs; pivot_wider() spreads them back out.
Long data plus group_by() summarizes many metrics in one step.

Proof of completion

Once you've attempted every exercise above, download a PDF summary of your work on this lesson — your name, the exercises you passed, attempt counts, timing, and your final code for each.