Inferential Statistics: Tests, p-Values & Confidence

Interpret Inferential Statistics before reaching for a formula

In an A/B test for checkout completion, report the conversion difference with a confidence interval, not only a p-value. A statistically detectable lift may still be too small to matter commercially.

• Hypothesis testing asks how compatible observed data are with a null model; it does not prove the alternative hypothesis true.
• A p-value is a probability calculated under the null assumption, not the probability that the null hypothesis is correct.
• Confidence intervals show a range of parameter values supported by the data and make effect size visible.
• Statistical significance without practical significance can lead to decisions that are numerically impressive but operationally useless.

Calculate a small case and verify it in Python

For Inferential Statistics, predict one visible result before each snippet runs—a row count, shape, statistic, ranking or error pattern—then compare it with the printed checkpoint.

# Inferential Statistics · example 1
from scipy import stats
a = [52,49,55,50,48,53]
b = [58,56,60,57,59,61]
t_stat, p_value = stats.ttest_ind(a, b, equal_var=False)
print(round(t_stat,3), round(p_value,4))

Inferential Statistics checkpoint 1: Run the cell and inspect the printed values before changing anything.

# Inferential Statistics · example 2
import pandas as pd
import numpy as np

df = pd.read_csv('practice.csv')
x = df.select_dtypes('number').iloc[:,0].dropna()
print('n=', len(x), 'mean=', round(x.mean(),2), 'median=', round(x.median(),2))

Inferential Statistics checkpoint 2: n, mean and median are printed from the first numeric column.

# Inferential Statistics · example 3
q1, q3 = x.quantile([0.25,0.75])
iqr = q3 - q1
print('std=', round(x.std(ddof=1),2), 'IQR=', round(iqr,2))

Inferential Statistics checkpoint 3: Standard deviation and IQR provide two different views of spread.

# Inferential Statistics · example 4
rng = np.random.default_rng(42)
boot = [rng.choice(x, size=len(x), replace=True).mean() for _ in range(1000)]
print(np.percentile(boot,[2.5,50,97.5]).round(2))

Inferential Statistics checkpoint 4: The three values are the bootstrap 2.5th, 50th and 97.5th percentiles.

# Inferential Statistics · example 5
x2 = pd.concat([x, pd.Series([x.max()*5])], ignore_index=True)
print('mean change:', round(x2.mean()-x.mean(),2))
print('median change:', round(x2.median()-x.median(),2))

Inferential Statistics checkpoint 5: The mean usually moves more than the median when one extreme value is added.

Stress-test the result against assumptions and outliers

Self-check with answers

Practise interpretation, not only arithmetic