4 Neutral trials: If the difference between interventions is not statistically significant, is there truly no difference?
Neutral trials (i.e. trials without a statistically significant difference in the primary outcome) should not all be interpreted equally, as they will differ in the degree in which they rule out a difference between interventions. It is important to be able to recognize that “no statistically significant difference” does not mean “no difference”.
Neutral trials are also sometimes referred to as “negative trials”.
Checklist Question
| Does the confidence interval (CI) exclude a clinically important difference? |
Does the confidence interval exclude a clinically important difference?
If the 95% CI is wide enough to include a clinically important difference, it remains possible that a future trial with greater precision, or a meta-analysis of multiple trials, may find a clinically meaningful difference.
| Outcome | Combination | Steroid Alone | Absolute risk difference (95% CI) |
| Mortality | 27% | 38% | -11% (-22% to +5%) |
The uncertainty of the estimated reduction represented by the 95% CI means that the trial could not exclude the possibility of an absolute reduction in mortality with combination therapy as high as 22%.
A primary outcome is an outcome from which trial design choices are based (e.g. sample size calculations). Primary outcomes are not necessarily the most important outcomes.
Absolute risk difference is the risk in one group compared to (minus) the risk in another group over a specified period of time. For example, if the absolute risk of myocardial infarction over 5 years was 15% for the comparator and 10% for the intervention, then the absolute risk difference was 5% (15% - 10%) over 5 years. See here for further discussion.
