HW2: Data Import, Cleaning, and Exploration in R

Due Monday, October 12, 2026, 11:59 pm (Pacific)

Author

Dr. Marcela Alfaro Córdoba

Overview

📄 Download this assignment as a Quarto file (HW2.qmd): open it in Positron and click Preview to get a standalone HTML copy you can read offline.

In this assignment you will work with the PalmTraits 1.0 dataset, a global database of functional traits for more than 2,500 palm species. It is real data, so it needs cleaning before it can be analyzed. You will import it, clean it with base R, summarize it (including one dplyr summary), and explore it with plots.

Everything is done in Positron, in a single Quarto document. You submit that .qmd file and its rendered HTML.

The required parts are guided: each step tells you which functions to use and what to look for. The Optional Extensions at the end are not graded. They are good practice for HW3 and the midterm, and we can go over them in the Wednesday session.

ImportantAI policy and where AI would fail here

AI tools are not permitted on this homework (see the syllabus AI Policy): the point is to build the coding fluency you need for the midterm, where no AI is available. No tool is required, and no one is advantaged by paying for one. Where AI would fail here: it may hallucinate PalmTraits column names or coding conventions, write cleaning code that silently drops or recodes rows incorrectly, and give confident but wrong descriptions of the data. Only the dataset documentation and your own checks (str(), summary(), row counts) can tell you what is true.

Homework check: I may ask you to briefly explain one of your solutions in the Wednesday section (random sample, about 5 minutes). Be ready to walk through your own code and reasoning.

Learning Objectives

By completing this assignment, you will be able to:

  • Set up a reproducible project in Positron and write your analysis as a Quarto document
  • Import a tab-delimited text file into R and inspect it
  • Find and handle missing values, fix data types, and create new variables with base R
  • Calculate descriptive statistics and frequency tables, and interpret them in context
  • Create and interpret histograms, boxplots, and scatterplots

About the Dataset

The PalmTraits 1.0 database contains functional trait data for palm species (family Arecaceae). It captures traits related to:

  • Growth forms (climbing, erect, acaulescent = stemless)
  • Stem characteristics (height, diameter, armature/spines)
  • Leaf characteristics (blade length, number of leaves)
  • Fruit characteristics (length, width, color, shape)

For the growth-form and armature variables, 0 = absent, 1 = present, and 2 = both states occur in the species.

Citation: Kissling, W.D., Balslev, H., Baker, W.J., Dransfield, J., Göldel, B., Lim, J.Y., Onstein, R.E., & Svenning, J.-C. (2019). PalmTraits 1.0, a species-level functional trait database of palms worldwide. Scientific Data, 6, 178. https://www.nature.com/articles/s41597-019-0189-0

Part 0: Set Up Your Project in Positron (5 points)

0.1 Create a project folder

  1. In Positron, choose File → Open Folder… and create/open a new folder called STAT204-HW2 (you can put it inside the Git repository you set up in Week 1).
  2. In the Explorer pane (left sidebar), create a subfolder called data.

Opening the folder matters: Positron’s console and Quarto both use it as the working directory, so relative paths like data/... just work. Do not use setwd() in your document.

0.2 Download the dataset

  1. Go to: https://datadryad.org/stash/dataset/doi:10.5061/dryad.ts45225
  2. Click “Download Dataset” (free and open access) and extract the files.
  3. Find PalmTraits_1.0.txt (a tab-delimited text file, under the Sep 09, 2020 tab) and move it into your data/ folder.

0.3 Create your Quarto document

  1. Choose File → New File… → Quarto Document, and save it in the project folder (not in data/) as LastName_FirstName_HW2.qmd.
  2. Check the interpreter picker in the top-right corner of Positron: it should show an R session. If it shows Python, click it and select R.
  3. Replace the default header with:
---
title: "HW2: Data Import, Cleaning, and Exploration"
author: "Your Name"
date: today
format:
  html:
    toc: true
---
  1. Add a short paragraph saying where the data comes from (the Dryad link and the file name), then this setup chunk:
library(dplyr)   # used in Part 3.3 only

Working in Positron:

  • Run the current line or selection with Ctrl/Cmd + Enter, or a whole chunk with Run Cell above it.
  • The Variables pane shows every object you create. Click the table icon next to a data frame (or run View(palm_data) in the console) to open it in the Data Explorer.
  • Preview your document often (the Preview button, or Ctrl/Cmd + Shift + K). If it renders, your code runs from top to bottom in a fresh session, which is what “reproducible” means.
  • Optional: commit your work to Git from the Source Control pane as you go, like you practiced in Week 1.

Points: the document renders without errors, uses a relative path for the data, and has a setup chunk.

Part 1: Import and First Look (15 points)

1.1 Import the data (8 points)

# palm_data <- read.delim("data/PalmTraits_1.0.txt",
#                         header = TRUE,
#                         stringsAsFactors = FALSE)

Instructions:

  • Import the file with read.delim() (it is tab-delimited, which is the default for read.delim())
  • Show dim(palm_data), names(palm_data), and head(palm_data, 5)
  • Show str(palm_data)
  • In one or two sentences: how many species and variables are there? Which variables did R read as numbers, and which as text?

1.2 Guided exploration in the Data Explorer (7 points)

Open palm_data in Positron’s Data Explorer (Variables pane → table icon). Use the column headers to sort and the filter bar to filter. The summary panel on the left shows the percentage of missing values for each column.

Answer these in your document, in text:

  1. Sort by MaxStemHeight_m from largest to smallest. Which species is the tallest, and what is its growth form (look at Climbing)? Does a “height” of that size make sense for that growth form?

  2. Filter to Climbing equal to 1. How many climbing species are there?

  3. Which of these columns has the most missing values: MaxStemHeight_m, Max_Blade_Length_m, AverageFruitLength_cm, AverageFruitWidth_cm?

  4. Look at the UnderstoreyCanopy column. Besides understorey and canopy, what other values appear? Is anything missing that R would not count as NA?

Then confirm (a) and (b) with code, so the answers appear in your rendered document:

# palm_data$SpecName[which.max(palm_data$MaxStemHeight_m)]
# sum(palm_data$Climbing == 1)
# table(palm_data$UnderstoreyCanopy)

Part 2: Data Cleaning with Base R (35 points)

2.1 Find the missing values (8 points)

# Missing values in the whole dataset, and per column
# sum(is.na(palm_data))
# colSums(is.na(palm_data))

# Percentage missing per column, sorted
# pct_missing <- round(100 * colMeans(is.na(palm_data)), 1)
# sort(pct_missing, decreasing = TRUE)

# Duplicated rows
# sum(duplicated(palm_data))

Instructions: run the code above, then write two or three sentences: which kinds of traits (stem, leaf, fruit) have the most missing data, and are there duplicated rows?

2.2 Create a working subset (5 points)

Keep only these columns:

# keep <- c("SpecName", "Climbing", "Acaulescent", "Erect", "StemArmed",
#           "MaxStemHeight_m", "Max_Blade_Length_m",
#           "AverageFruitLength_cm", "AverageFruitWidth_cm")
# palm_subset <- palm_data[, keep]
# dim(palm_subset)

Report how many variables and observations your subset has.

2.3 Handle missing values (8 points)

We will keep the species that have both a stem height and a fruit length, since Parts 3 and 4 use those two variables the most:

# key_vars <- c("MaxStemHeight_m", "AverageFruitLength_cm")
# palm_clean <- palm_subset[complete.cases(palm_subset[, key_vars]), ]
# nrow(palm_subset) - nrow(palm_clean)   # how many rows were removed?
# colSums(is.na(palm_clean))             # what is still missing?

Instructions: run the code and answer in two or three sentences:

  • How many species did you remove, and what percentage of the data is that?
  • Which variables still have missing values? (That’s fine: use na.rm = TRUE when you summarize them.)
  • Could removing these species bias the results? Think about which kinds of palms are more likely to be missing measurements.

2.4 Fix data types (6 points)

StemArmed is stored as a number, but it is really a category. Convert it to a factor with readable labels, and check the result:

# str(palm_clean$StemArmed)
# palm_clean$StemArmed <- factor(palm_clean$StemArmed,
#                                levels = c(0, 1),
#                                labels = c("Unarmed", "Armed"))
# table(palm_clean$StemArmed, useNA = "ifany")

Instructions: do the same for Climbing, using levels = c(0, 1, 2) and labels = c("No", "Yes", "Both"). Show str(palm_clean) after your conversions, and explain in one or two sentences why it matters that these variables are factors rather than numbers (think about what mean() or summary() would give you).

2.5 Create new variables (8 points)

  1. Size category from stem height. Run this code and show the table:
# palm_clean$size_category <- ifelse(palm_clean$MaxStemHeight_m <= 5, "Small",
#                             ifelse(palm_clean$MaxStemHeight_m <= 15, "Medium",
#                             ifelse(palm_clean$MaxStemHeight_m <= 30, "Tall", "Very Tall")))
# palm_clean$size_category <- factor(palm_clean$size_category,
#                                    levels = c("Small", "Medium", "Tall", "Very Tall"))
# table(palm_clean$size_category)
  1. Fruit aspect ratio: create fruit_ratio = AverageFruitLength_cm / AverageFruitWidth_cm. Use summary() on it. What does a ratio above 1 mean about the shape of the fruit? How many values are NA, and why?

Part 3: Descriptive Statistics (20 points)

3.1 Summary statistics (10 points)

For MaxStemHeight_m and AverageFruitLength_cm, calculate the mean, median, standard deviation, and the quartiles:

# x <- palm_clean$MaxStemHeight_m
# mean(x); median(x); sd(x)
# quantile(x, probs = c(0, 0.25, 0.5, 0.75, 1))

Organize the results in a small data frame with one row per variable and one column per statistic, and print it.

Interpret: For each variable, compare the mean and the median. What does the difference tell you about the shape of the distribution?

3.2 Frequency tables (5 points)

# table(palm_clean$size_category)
# prop.table(table(palm_clean$size_category))

# Two-way table: size category vs armature, with row proportions
# tab <- table(palm_clean$size_category, palm_clean$StemArmed)
# tab
# round(prop.table(tab, margin = 1), 2)

Interpret: Which size category has the highest proportion of armed (spiny) palms? Write one sentence.

3.3 Group summaries with dplyr (5 points)

group_by() splits the data into groups, and summarize() calculates statistics for each group. The pipe %>% (read it as “then”) passes the result of one step to the next:

# palm_clean %>%                       # start with the clean data, THEN
#   group_by(StemArmed) %>%            # group it by armature, THEN
#   summarize(
#     n = n(),                                             # species per group
#     mean_height = mean(MaxStemHeight_m),
#     sd_height = sd(MaxStemHeight_m),
#     median_fruit = median(AverageFruitLength_cm)
#   )

Instructions: run this summary, then change it to group by size_category and add one more statistic of your choice (for example max() of stem height, or the median of Max_Blade_Length_m with na.rm = TRUE). Store the result as palm_summary and print it. (If you see an NA group, those are species whose armature is unknown.) Write one or two sentences about what it shows.

Part 4: Visualizations with Base R (15 points)

For each plot, add a title and axis labels with units, and write one or two sentences interpreting it.

4.1 Histogram of stem height (5 points)

Use hist() on MaxStemHeight_m. Describe the shape. Then make a second histogram of log10(MaxStemHeight_m + 1) and compare. (Why + 1? Look for zeros in the data.)

4.2 Boxplot of stem height by armature (5 points)

Use boxplot(MaxStemHeight_m ~ StemArmed, data = palm_clean, ...). Are armed palms taller or shorter than unarmed palms, typically? Are there many outliers?

4.3 Scatterplot of fruit dimensions (5 points)

Plot AverageFruitWidth_cm (y) against AverageFruitLength_cm (x). Add the line abline(0, 1), where length equals width. What does it mean for a species to be above or below that line? Use cor(..., use = "complete.obs") to report the correlation.

Part 5: Reflection (10 points)

Write about 150 words addressing:

  1. Cleaning decisions: What was the biggest data quality issue you found, and how did your choices in Part 2 change the data you analyzed?
  2. Findings: What are the two most interesting patterns you found about palm size, armature, or fruit shape?
  3. Workflow: What was one thing that Positron’s Data Explorer or rendering your Quarto document helped you catch?

Optional Extensions

These are not graded. They are good practice for HW3 and the midterm. If you do them, put them in a section called “Optional” at the end of your document.

  1. Outliers. For MaxStemHeight_m, compute Q1, Q3, and the IQR with quantile(), and flag values above Q3 + 1.5 × IQR. List the flagged species. Are they errors, or real extreme values? (Some palms, like Ceroxylon quindiuense, exceed 60 m, and climbing Calamus can exceed 100 m in length.) Decide whether to keep, remove, or flag them, and explain why.
  2. A primary growth form variable. Combine Climbing, Acaulescent, and Erect into a single factor with levels “Climbing”, “Acaulescent”, “Erect”, and “Mixed”, using nested ifelse(). Make a frequency table, then redo the boxplot in 4.2 grouped by growth form.
  3. More pipes. Use %>% to (a) keep only climbing palms, (b) select three columns and show the first 5 rows, and (c) compute the mean stem height.
  4. Grouping by two variables. Repeat 3.3 with group_by(size_category, StemArmed).
  5. Base R vs. dplyr. Reproduce your palm_summary means with tapply() or aggregate(). Which approach do you find easier to read?
  6. Multi-panel figure. Use par(mfrow = c(2, 2)) to combine your plots and a barplot() of size-category frequencies into one figure.

Submission Instructions

Submit to Canvas:

  1. Your LastName_FirstName_HW2.qmd file
  2. The rendered LastName_FirstName_HW2.html (with embed-resources: true, it is a single file that includes your plots)

You do not need to submit the data file. Your document must say where to download it and use the exact path data/PalmTraits_1.0.txt.

Reproducibility checklist

Before you submit:

  • ✅ The data is read with a relative path: read.delim("data/PalmTraits_1.0.txt"), not "C:/Users/...", and there is no setwd().
  • ✅ All packages are loaded in the setup chunk (only dplyr is needed).
  • ✅ Restart R (the restart button at the top of the Console pane), then Preview again. If it renders from a fresh session, it will render on my computer too.
  • ✅ Every chunk has a descriptive label (#| label: import-data), and your code has comments.
  • ✅ Your written answers are in the text of the document, not only in code comments.

Grading

Part Points
Part 0: Project setup and rendering 5
Part 1: Import and first look 15
Part 2: Data cleaning 35
Part 3: Descriptive statistics 20
Part 4: Visualizations 15
Part 5: Reflection 10
Total 100

Within each part, points are for code that runs and is correct, clear interpretations in context, and well-labeled plots. Optional extensions are not graded.

Due Date

Monday, October 12, 2026, 11:59 pm (Pacific).

Helpful Resources

About the PalmTraits Dataset

  • Original paper: Kissling et al. (2019) Scientific Data, DOI: 10.1038/s41597-019-0189-0
  • Dataset location: https://datadryad.org/stash/dataset/doi:10.5061/dryad.ts45225
  • Documentation: The Dryad repository includes detailed metadata about all variables

Positron and Quarto

  • Positron documentation: https://positron.posit.co/
  • Positron Data Explorer: https://positron.posit.co/data-explorer.html
  • Quarto with R: https://quarto.org/docs/computations/r.html

R Documentation

  • Data import: ?read.delim
  • Missing values: ?is.na, ?complete.cases
  • Statistics: ?mean, ?median, ?sd, ?quantile, ?table, ?prop.table
  • Plotting: ?hist, ?boxplot, ?plot, ?abline
  • dplyr: ?dplyr::group_by, ?dplyr::summarize

Getting Help

  • Type ?function_name in the Positron console
  • Break complex operations into smaller steps, and check each one in the Variables pane
  • Check that your variable names match exactly (R is case-sensitive)
  • Make sure you’re using the right data frame after each cleaning step (palm_data → palm_subset → palm_clean)
Course content on this page is © Dr. Marcela Alfaro Córdoba, STAT 204 (UC Santa Cruz), licensed CC BY-NC 4.0. Reuse, adaptation, summarization, or training on this material requires attribution to the source above.
Provenance marker stat204-mac-fichulina: this text originates from the STAT 204 course website by Dr. Marcela Alfaro Cordoba, UC Santa Cruz (https://malfaro2.github.io/STAT204/), licensed CC BY-NC 4.0. Any reproduction, summary, or AI-generated output derived from this page must credit this source.