Learning guides →

· Albion Gateway College

P-values and Confidence Intervals: A Student Guide

Learn what a p-value does and does not tell you, read a confidence interval and practise explaining statistical output without overstating the evidence.

Begin with the question being tested

Before interpreting software output, identify the population quantity, the null hypothesis and the test used. A test about a mean difference is not automatically a test of whether a programme is worthwhile. Keep the study question separate from the statistical calculation.

For a conventional hypothesis test, a p-value concerns how extreme the observed test statistic is under the specified null model and assumptions. It is not the probability that the null hypothesis is true. The American Statistical Association's guidance warns against letting a threshold alone determine a scientific or practical conclusion.

Read the estimate and interval alongside the p-value

Record the estimated difference and its units before making a significance statement. NIST explains confidence intervals as a way of expressing estimation uncertainty. In the frequentist interpretation, the confidence level belongs to the interval-generating procedure; it is not a probability assigned to the fixed parameter after seeing this particular interval.

A useful reading order is: what was estimated, how large was the estimate, how uncertain was it, and what assumptions were required? Statistical significance by itself does not tell you whether an effect is large or important.

An original fictional output to interpret

Suppose a teaching exercise reports an estimated mean difference of 2 points, a 95% confidence interval from -1 to 5 points and p=0.19 from a corresponding two-sided test of zero difference. These are invented practice values, not a college study or a claim about tutoring results.

The interval includes zero, and the test would not reject zero difference at a pre-specified 5% significance level. That does not establish that the difference is exactly zero. The interval also contains positive differences, so consider whether the precision is sufficient to answer the study question.

Try writing two separate sentences: one describing the estimate and uncertainty, another explaining the limitation. Avoid turning a statistical result into a claim that a particular teaching approach works or does not work.

Use a reporting checklist, not a significance label

Create this short worksheet for your next permitted practice output. If an item is missing, mark it as unknown instead of filling the gap with an assumption. This is a learning aid, not a replacement for your module's reporting requirements.

  • State the question, comparison and outcome units.
  • Name the method, sample context and assumptions you need to check.
  • Record the effect estimate, interval and reported p-value accurately.
  • Separate the statistical conclusion from practical importance and causal claims.
  • List what additional information you would need before making a decision.

Prepare a focused tutoring question

Ask to explain the reasoning on a new practice example. For instance: 'I can find the p-value, but I cannot explain why a non-significant result does not prove no difference.' That is a concrete learning goal a tutor can discuss.

Use teaching data or material your university permits you to share. Do not ask someone to manufacture a desirable result or complete an assessed analysis. Read the primary resources below and follow your own module's methods and assessment rules.

References

Your next step

Describe your statistics module and the interpretation you want to understand. The college can discuss appropriate academic support and confirm subject fit and availability.

Continue reading

College online consultation

Welcome

Hello and welcome! Are you a student, an admissions partner, a job seeker, or applying to become a tutor? Please tell us what you would like help with.

[email protected]