What does a p-value of 0.05 actually mean?

It is a convention, not a law of nature. A p-value below 0.05 means that if there were truly no difference or no relationship, you would see a result this extreme less than five percent of the time. It does not measure the size or importance of an effect, and it does not give the probability that your hypothesis is true.

When should I use a t-test and when something else?

Use a t-test to compare the means of two groups with a numeric outcome and reasonably symmetric data. For more than two groups use an analysis of variance with aov. For heavily skewed data or very small samples consider a rank-based test such as wilcox.test.

Does a strong correlation mean one variable causes the other?

No. Two variables can move together because of a third factor, by coincidence, or in the opposite direction from what you assumed. Establishing cause needs a designed experiment or careful causal reasoning beyond correlation or regression.

Statistics in R: Correlation, t-tests, and Linear Regression

Introduction

R was created by statisticians, and its built-in functions for tests and models are one of its biggest strengths. Running a test takes one line. The hard part is knowing what the output means and what it does not. This lesson uses the mtcars dataset to compare groups and to model fuel economy, with an emphasis on interpreting results honestly rather than only producing numbers.

Prerequisites

Data frames and the pipe from earlier lessons
Basic ideas of averages and spread; no advanced statistics needed

Step-by-Step Guide

Step 1: Describe the data first

Before any test, look at the numbers and a plot. Compute the mean, standard deviation, and count for each group, and draw a histogram or boxplot. Tests can return a significant result for data with errors or extreme outliers, and a quick look will catch it. In mtcars, the am column records the transmission, where 0 is automatic and 1 is manual, and mpg is the fuel economy. Convert am to a labeled factor to make output readable.

library(tidyverse)

cars <- mtcars |>
  mutate(transmission = factor(am, labels = c("automatic", "manual")))

cars |>
  group_by(transmission) |>
  summarise(n = n(), mean_mpg = mean(mpg), sd_mpg = sd(mpg))
# automatic: n 19, mean about 17.1
# manual:    n 13, mean about 24.4

Step 2: Measure the relationship with correlation

Correlation measures how strongly two numeric variables move together, from -1 for a perfect opposite relationship to 1 for a perfect matching one, with 0 meaning no straight-line relationship. The weight of a car and its fuel economy have a correlation of about -0.87, which is strong and negative: heavier cars get fewer miles per gallon. Correlation does not show cause, and it only captures straight-line patterns, so always pair it with a scatter plot.

cor(mtcars$wt, mtcars$mpg)        # about -0.87
cor.test(mtcars$wt, mtcars$mpg)   # adds a confidence interval and p-value

Step 3: Compare two groups with a t-test

A two-sample t-test asks whether the means of two groups differ by more than random variation would explain. The formula interface reads as 'mpg by transmission'. R uses the Welch version by default, which does not assume equal variances. The output reports the t statistic, degrees of freedom, a p-value, a confidence interval for the difference, and the group means. Here the p-value is about 0.0014, so a gap this large would be unlikely if the true means were equal. The p-value is the probability of seeing a result at least this extreme under the null hypothesis, not the probability that the hypothesis is true.

t.test(mpg ~ transmission, data = cars)
# t about -3.77, p-value about 0.0014
# mean in automatic about 17.1, mean in manual about 24.4

Step 4: Fit a linear regression with lm

A linear model predicts a numeric outcome from one or more predictors. The lm function takes a formula and a data frame. The summary reports a coefficient for each predictor with its standard error and p-value. For fuel economy against weight, the intercept is about 37.3 and the slope is about -5.3, and weight is measured in thousands of pounds. This means each additional 1000 pounds is associated with about 5.3 fewer miles per gallon, on average. The multiple R-squared of about 0.75 means weight accounts for roughly three quarters of the variation in mpg across these cars.

model <- lm(mpg ~ wt, data = mtcars)
summary(model)
coef(model)             # intercept about 37.29, wt about -5.34

predict(model, newdata = data.frame(wt = 3))   # about 21.3 mpg

Step 5: Add predictors and compare

Adding a second predictor lets you see how an effect changes when you account for other variables. Adding horsepower to the model reduces the weight coefficient to about -3.9 and raises R-squared to about 0.83. The coefficients change because weight and horsepower are related to each other, so each one now describes the effect while holding the other constant. Prefer adjusted R-squared when comparing models with different numbers of predictors, because plain R-squared never decreases when you add a predictor.

model2 <- lm(mpg ~ wt + hp, data = mtcars)
summary(model2)$adj.r.squared
anova(model, model2)    # does the extra predictor improve the fit?

Step 6: Check the assumptions and the limits

A regression is only trustworthy if its assumptions roughly hold: the relationship is linear, residuals have similar spread across the range of fitted values, and they are roughly normal. Call plot on the model to see diagnostic charts, and look for curved patterns, funnel shapes, and points with high leverage. Be honest about limits. This dataset has only 32 cars from the early 1970s, the results are associations rather than proof of cause, and a small p-value says nothing about whether the effect is large enough to matter in practice. Report the effect size and a confidence interval, not only the p-value.

par(mfrow = c(2, 2))
plot(model)
confint(model)     # confidence intervals for the coefficients

FAQs

What does a p-value of 0.05 actually mean?: It is a convention, not a law of nature. A p-value below 0.05 means that if there were truly no difference or no relationship, you would see a result this extreme less than five percent of the time. It does not measure the size or importance of an effect, and it does not give the probability that your hypothesis is true.
When should I use a t-test and when something else?: Use a t-test to compare the means of two groups with a numeric outcome and reasonably symmetric data. For more than two groups use an analysis of variance with aov. For heavily skewed data or very small samples consider a rank-based test such as wilcox.test.
Does a strong correlation mean one variable causes the other?: No. Two variables can move together because of a third factor, by coincidence, or in the opposite direction from what you assumed. Establishing cause needs a designed experiment or careful causal reasoning beyond correlation or regression.

Summary

You have run a correlation, a t-test, and simple and multiple linear regressions, and you know how to read their key outputs: effect sizes, confidence intervals, p-values, and R-squared. Always look at plots, check assumptions, and be careful about claiming cause. The final lesson shows how to wrap your analysis into a report that anyone can regenerate.