HW5: Hypothesis Testing and Inference
Making Evidence-Based Medical Decisions
🔬 The Case
Welcome back, Statistical Detective!
Dr. Elena Rodriguez, Director of Clinical Research at County Hospital, has urgent cases that need your statistical expertise. “We’re constantly evaluating new treatments, comparing medical interventions, and interpreting clinical trial results,” she explains. “But here’s the challenge: How do we know if a treatment really works, or if what we’re seeing is just random chance? How confident can we be in our conclusions? And when is a difference statistically significant but not practically important?”
The hospital’s research team needs your help with several critical questions: Is a new blood pressure medication more effective than the standard treatment? Does a weight-loss intervention actually work? How should we interpret confidence intervals from clinical trials? Your mission: Use hypothesis testing and statistical inference to guide evidence-based medical decisions!
Question 1: Understanding Confidence Intervals
Dr. Rodriguez wants to ensure everyone understands what confidence intervals tell us.
a. Explain the concept
In your own words, explain what a 95% confidence interval means. What does “95% confidence” refer to?
b. Common misconceptions
For each statement below, indicate whether it’s a CORRECT or INCORRECT interpretation of a 95% confidence interval for a population mean:
A researcher calculates a 95% CI for mean cholesterol level and gets (185, 215) mg/dL.
- “There’s a 95% probability that the true population mean is between 185 and 215.”
- “If we repeated this study many times, about 95% of the confidence intervals would contain the true population mean.”
- “95% of individual people have cholesterol between 185 and 215.”
- “We are 95% confident that the true population mean cholesterol is between 185 and 215.”
c. Width of intervals
A pharmaceutical company reports: “The new drug reduces blood pressure by 12 mmHg (95% CI: 8 to 16).”
Another study reports: “The new drug reduces blood pressure by 12 mmHg (95% CI: 2 to 22).”
- Which study gives us more precise information about the drug’s effect?
- What might cause one confidence interval to be wider than another?
- Which result would you find more convincing and why?
Question 2: Hypothesis Testing Framework
a. State the hypotheses
For each scenario, write the null hypothesis (H₀) and alternative hypothesis (Hₐ) in both words and symbols:
Scenario 1: Testing if a new diabetes medication lowers average blood glucose below 120 mg/dL.
- H₀:
- Hₐ:
Scenario 2: Testing if the proportion of patients experiencing side effects is different from 15%.
- H₀:
- Hₐ:
b. Components of a hypothesis test
List and briefly explain the five key components needed to conduct any hypothesis test.
c. Decision rules
Explain the two methods we can use to make a decision about the null hypothesis:
- P-value method:
- Critical value method:
Question 3: Understanding P-Values
a. Define p-value
In your own words, explain what a p-value tells us. Complete this sentence: “The p-value is the probability that…”
b. Interpret p-values
A researcher tests whether a new therapy reduces anxiety scores. She calculates p = 0.03.
- What does this p-value mean in the context of this study?
- If we use α = 0.05, what decision should she make about H₀?
- Would the decision change if α = 0.01? Explain.
c. Common p-value misconceptions
For each statement, indicate TRUE or FALSE and explain why:
- “p = 0.03 means there’s a 3% chance the null hypothesis is true.”
- “p = 0.03 means there’s a 3% chance our results are due to random chance alone, assuming H₀ is true.”
- “A smaller p-value means a larger effect size.”
- “p = 0.06 means the treatment definitely doesn’t work.”
Question 4: Type I and Type II Errors
a. Define the errors
In your own words, explain:
- Type I Error:
- Definition:
- Probability it occurs:
- Medical example:
- Type II Error:
- Definition:
- Probability it occurs:
- Medical example:
b. Error consequences
A pharmaceutical company is testing if a new drug is more effective than placebo for treating migraines.
- Describe what a Type I error would mean in this context and its potential consequences.
- Describe what a Type II error would mean in this context and its potential consequences.
- Which error would be more serious in this case? Why?
c. The trade-off
Explain why decreasing the risk of Type I error (by using a smaller α) increases the risk of Type II error. Use a specific example to illustrate.
Question 5: Statistical vs. Practical Significance
a. Explain the difference
In your own words, explain the difference between statistical significance and practical significance.
b. Large sample scenario
A study of 50,000 patients finds that a new drug lowers cholesterol by an average of 2 mg/dL. The p-value is 0.001.
- Is this result statistically significant at α = 0.05?
- The 95% CI is (1.5, 2.5) mg/dL. Is this result practically significant? Explain.
- Would you recommend doctors prescribe this drug based on these results? Why or why not?
c. Small sample scenario
A pilot study of 20 patients finds a new intervention reduces hospital stays by an average of 3.5 days. The p-value is 0.08.
- Is this result statistically significant at α = 0.05?
- Could this intervention still be practically important? Explain.
- What would you recommend as next steps?
Question 6: Conditions for Inference
a. List the conditions
What conditions must be checked before performing inference (confidence intervals or hypothesis tests) for:
- A single mean (using t-procedures):
- A single proportion (using z-procedures):
b. Check conditions: Mean
A researcher collects systolic blood pressure measurements from 40 randomly selected patients. The data has:
- Mean = 128 mmHg
- Standard deviation = 15 mmHg
- Sample size = 40
- A histogram shows slight right skew with no outliers
Should the researcher proceed with a t-test? Explain your reasoning by checking each condition.
c. Check conditions: Proportion
A public health study surveys 80 people and finds that 8 report having diabetes.
Can we use normal approximation methods for inference about the proportion? Check the conditions and explain.
Question 7: One-Sample t-Test for Mean
A cardiologist wants to test if a new exercise program reduces resting heart rate. She measures resting heart rate for 25 patients before and after the program. The mean reduction in heart rate is 5.2 beats per minute (bpm) with a standard deviation of 8.1 bpm.
a. Set up the test
- Is this scenario calling for a one-sample test or a paired test? Explain.
- State the hypotheses (use μ for the mean reduction):
- H₀:
- Hₐ:
- What significance level should we use? (Choose α = 0.05 unless stated otherwise)
b. Check conditions
List each condition required for this t-test and state whether it’s met based on the information given.
c. Calculate test statistic
Calculate the t-statistic using the formula: \[t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}\]
Show your work.
d. Find p-value and make decision
- The degrees of freedom = ___
- Using a t-table or technology, the p-value ≈ ___ (you can report it as a range)
- Decision:
- Conclusion in context:
Question 8: Hypothesis Test for Proportion
The hospital’s infection control team wants to know if the infection rate after a certain surgery is less than the national rate of 12%. In a sample of 200 surgeries at their hospital, 18 resulted in infections.
a. Set up the test
- State the hypotheses in symbols and in words:
- Calculate the sample proportion: p̂ = ___
- What significance level will you use?
b. Check conditions
Verify all conditions for using the normal approximation. Show your work.
c. Calculate test statistic
Calculate the z-statistic using: \[z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}\]
Show all steps.
d. Make a decision
- Find the p-value using the standard normal distribution
- State your decision about H₀
- Write a conclusion in context for the infection control team
Question 9: Paired vs. Independent Samples
a. Identify the design
For each scenario, state whether you would use a paired t-test or an independent samples t-test. Explain your reasoning.
Scenario 1: Comparing pain scores of 30 patients before and after taking a pain medication.
Scenario 2: Comparing recovery times between 40 patients who received physical therapy and 40 different patients who did not.
Scenario 3: Measuring blood pressure in 50 patients’ left arm and right arm to see if there’s a difference.
Scenario 4: Comparing weight loss between 35 people on Diet A and 35 different people on Diet B.
b. Why does it matter?
Explain why it’s important to distinguish between paired and independent samples. What could go wrong if you use the wrong test?
Question 10: Conducting a Paired t-Test
A sleep researcher tests whether a new sleep aid increases sleep duration. She measures sleep time (in hours) for 12 participants on two nights: once with the sleep aid and once with a placebo.
| Patient | Placebo | Sleep Aid | Difference (Aid - Placebo) |
|---|---|---|---|
| 1 | 6.2 | 7.1 | 0.9 |
| 2 | 5.8 | 6.5 | 0.7 |
| 3 | 7.1 | 7.8 | 0.7 |
| 4 | 6.5 | 6.9 | 0.4 |
| 5 | 5.5 | 6.8 | 1.3 |
| 6 | 6.8 | 7.2 | 0.4 |
| 7 | 6.3 | 6.5 | 0.2 |
| 8 | 5.9 | 7.0 | 1.1 |
| 9 | 6.6 | 7.3 | 0.7 |
| 10 | 7.0 | 7.1 | 0.1 |
| 11 | 6.4 | 7.5 | 1.1 |
| 12 | 5.7 | 6.6 | 0.9 |
The mean difference is 0.71 hours with a standard deviation of 0.35 hours.
a. Why paired?
Explain why this is a paired design and why we should analyze differences rather than comparing the two groups separately.
b. Set up hypotheses
Write null and alternative hypotheses for testing if the sleep aid increases sleep time.
c. Conduct the test
- Check the condition about the differences (you may assume normality)
- Calculate the t-statistic
- Find the p-value (df = 11)
- Make a decision at α = 0.05
- Calculate a 95% CI for the mean difference: \(\bar{x}_d \pm t^* \times \frac{s_d}{\sqrt{n}}\) (use t* ≈ 2.201)
d. Interpret
Write a conclusion in context using both the hypothesis test result and the confidence interval.
Question 11: Independent Samples t-Test with Google Sheets
A nutritionist compares two weight-loss programs. Group A (n=30) follows a low-carb diet and Group B (n=28) follows a low-fat diet. After 3 months:
- Group A: Mean weight loss = 8.5 kg, SD = 3.2 kg
- Group B: Mean weight loss = 6.8 kg, SD = 2.9 kg
a. Set up the test
- Why is this an independent samples design?
- State the hypotheses for testing if Group A loses more weight than Group B:
- What assumptions do we need?
b. Using T.TEST in Google Sheets
You would use the T.TEST function in Google Sheets. The syntax is:
=T.TEST(range1, range2, tails, type)For this scenario: 1. What would you enter for tails? (1 for one-tailed, 2 for two-tailed) 2. What would you enter for type? (1 for paired, 2 for equal variances, 3 for unequal variances) 3. Write out the complete formula you would use (assuming data in A2:A31 and B2:B29)
c. Interpret results
Suppose the T.TEST function returns p = 0.024.
- What does this p-value tell us?
- At α = 0.05, what’s your decision?
- What would you conclude about the two diets?
d. Effect size
Calculate the difference in means: 8.5 - 6.8 = 1.7 kg.
- Is this difference statistically significant?
- Is this difference practically significant for people trying to lose weight? Explain.
Question 12: Checking t-Test Conditions
A researcher wants to compare depression scores between two treatment groups. Before running the test, she needs to check conditions.
Group 1 (Cognitive Therapy): n = 24, boxplot shows approximately symmetric distribution with one mild outlier
Group 2 (Medication): n = 28, boxplot shows slight right skew but no outliers
a. List all conditions
What conditions must be satisfied to use an independent samples t-test?
b. Evaluate each condition
For each condition, state whether it appears to be satisfied and explain your reasoning.
c. Make a recommendation
Should the researcher proceed with the t-test? If yes, explain why. If no, what alternatives might she consider?
Question 13: Comprehensive Interpretation
A clinical trial tests a new drug for lowering LDL cholesterol. The study includes 100 patients randomized to drug or placebo. Results:
- Drug group (n=50): Mean reduction = 22 mg/dL, 95% CI (18, 26)
- Placebo group (n=50): Mean reduction = 8 mg/dL, 95% CI (5, 11)
- Independent samples t-test: t = 8.45, p < 0.001
a. Interpret the confidence intervals
What does each 95% CI tell us about cholesterol reduction in that group?
b. Interpret the hypothesis test
- What question does this t-test answer?
- What do t = 8.45 and p < 0.001 tell us?
- What decision would you make about H₀?
c. Clinical significance
- Is the result statistically significant?
- The mean difference is 22 - 8 = 14 mg/dL. Is this clinically meaningful? (Doctors generally consider a 10+ mg/dL reduction meaningful)
- Based on the narrow confidence intervals, how precise is our estimate of the drug’s effect?
d. Communicate to a patient
A patient asks: “Should I take this drug? Does it really work?” Write 3-4 sentences explaining the results in plain language.
💭 Question 14: Detective’s Reflection
Reflect on hypothesis testing and inference (6-8 sentences):
- Why is it important to check conditions before performing a t-test? What could go wrong if we skip this step?
- Explain the difference between statistical significance and practical significance using a medical example.
- Why do we need both hypothesis tests AND confidence intervals? What does each tell us?
- What’s the difference between a paired t-test and an independent samples t-test? Give a scenario where using the wrong one would lead to incorrect conclusions.
- How does understanding Type I and Type II errors help researchers design better studies?
- What does a p-value of 0.04 really mean, and what does it NOT mean?
- How might the concepts from this homework help doctors and patients make better medical decisions?
🎉 Excellent work, Statistical Detective! Understanding hypothesis testing, confidence intervals, and proper inference procedures is essential for making evidence-based medical decisions!
Remember: Statistical significance tells us if an effect is real, but practical significance tells us if it matters. Both are crucial for good science and good medicine!