Definition
The significance level — written α (alpha) — is the probability threshold a researcher fixes in advance for rejecting the null hypothesis when it is actually true. It is the false-positive rate the study is willing to tolerate. By far the most common convention is α = 0.05, meaning the researcher accepts a one-in-twenty chance of declaring a true null hypothesis false. More stringent fields use 0.01 or 0.001; exploratory work sometimes uses 0.10.
α is a decision the analyst makes before looking at the data. The observed p-value is then compared to α: if the p-value is smaller, the null is rejected and the result is called statistically significant at that level.
Why it matters
How it works
The significance level fixes the type I error rate of the test — the probability of rejecting a true null hypothesis. If the null is true and you repeat the experiment many times, on average a proportion equal to α of those experiments will produce a false-positive significant result. The remaining (1 − α) will correctly fail to reject. There is a symmetric quantity, the type II error rate β, which is the probability of failing to reject a false null. Lowering α (making the test stricter) raises β (more real effects get missed); raising α (making the test looser) lowers β. The two cannot both be minimised without increasing sample size.
The dependence on α is the reason significance must be declared before the analysis. If the analyst can choose the threshold after seeing the p-value, every result becomes significant in retrospect and the test loses its meaning. The same logic applies to running many tests: at α = 0.05, twenty independent null hypotheses will produce roughly one false positive on average. Honest practice either lowers α for each test (the Bonferroni correction divides α by the number of comparisons) or controls a different error rate such as the false discovery rate.