Lesson 5 of 17 · Week 02 · Tue Oct 6

Exploring Data Frames

The gm dataset

From here on, most examples use Gapminder data: life expectancy, population and GDP per person by country and year. In the exercise boxes it's already loaded as a data frame called gm: 22 countries × 4 years (1967, 1987, 2002, 2007) = 88 rows, with columns country, continent, year, lifeExp, pop and gdpPercap.

On your own computer, save the full dataset (gapminder_data.csv) into your data/ folder and read it with a relative path:

gapminder <- read.csv("data/gapminder_data.csv", stringsAsFactors = TRUE)

Everything below works the same on either. Only the sizes differ.

Adapted from Software Carpentry's “Exploring Data Frames” episode (CC BY 4.0, The Carpentries). Text, examples and exercises have been rewritten for this course.

First look

Get oriented before you analyse anything:

str(gm)          # every column: its type and first values
summary(gm)      # per-column summaries
dim(gm)          # rows, columns   (also nrow(), ncol())
colnames(gm)
head(gm)         # first 6 rows; head(gm, 10) for 10
tail(gm, 3)      # last 3 rows
typeof(gm)       # "list"  - remember, a data frame is a list of columns
length(gm)       # number of columns, for the same reason

How to read str(): the first line gives the object's class and its size (obs. = rows, variables = columns). Each line after that is one column: its name, its type (num, int, chr, logi, or Factor w/ N levels), and its first few values. For a factor, the numbers shown are the codes pointing at its levels.

The top and bottom rows are often the tidiest part of a file. To spot problems in the middle, look at a random handful of rows: gm[sample(nrow(gm), 5), ].

Adding columns and rows

gm$pop_millions <- gm$pop / 1e6           # new column from existing ones
flag <- gm$lifeExp > 70
gm2 <- cbind(gm, long_lived = flag)       # or bind a vector as a column

extra <- data.frame(country = "Chile", continent = "Americas", year = 2012,
                    lifeExp = 79.6, pop = 17.4e6, gdpPercap = 15300,
                    pop_millions = 17.4)
gm3 <- rbind(gm, extra)                   # rows must have the same columns

cbind() adds columns (lengths must match the number of rows); rbind() stacks data frames with the same column names. With factor columns, rbind() adds any new category as a new level.

Removing rows and missing values

gm[-3, ]                 # everything except row 3
gm[, -6]                 # everything except column 6
na.omit(df)              # drop every row with an NA anywhere

Be careful with na.omit(): it drops the whole row if any column is missing.

Scripts you can re-run: source()

Once your loading and cleaning code works, put it in a script (e.g. scripts/load-data.R) so you can re-run it in one step. source("scripts/load-data.R") runs a whole script file, and everything it creates appears in your session. In Positron, the Source ▷ button at the top of the editor does the same for the open file.

Exercises

Each exercise gives you up to 10 checked attempts. Use Run as many times as you like to experiment first, then click Check answer when you're ready. Your exercises use numbers/subsets unique to you, so answers won't match a classmate's exactly.