{"id":52,"date":"2021-05-22T00:04:42","date_gmt":"2021-05-22T04:04:42","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/?post_type=chapter&p=52"},"modified":"2022-01-12T23:02:33","modified_gmt":"2022-01-13T04:02:33","slug":"secondary-outcomes-can-conclusions-be-made-from-outcomes-other-than-the-primary-one","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/chapter\/secondary-outcomes-can-conclusions-be-made-from-outcomes-other-than-the-primary-one\/","title":{"raw":"Secondary outcomes: Can conclusions be made from outcomes other than the primary one?","rendered":"Secondary outcomes: Can conclusions be made from outcomes other than the primary one?"},"content":{"raw":"Most trials designate one outcome as a [pb_glossary id=\"1517\"]\"primary\" outcome[\/pb_glossary] (or 2-3 \"co-primary outcomes\") and all other outcomes as [pb_glossary id=\"1532\"]\"secondary\" outcomes[\/pb_glossary]. Designation of an outcome as \"primary\" is done to determine and justify sample size calculations prior to conducting a study. In other words, the [pb_glossary id=\"1517\"]primary outcome[\/pb_glossary] is not necessarily the most clinically important (it often isn't), and should not be the sole consideration as to whether an intervention is \"better\" than a comparator.\r\n\r\nThe interpretation of [pb_glossary id=\"1532\"]secondary outcomes[\/pb_glossary] requires additional considerations. The probability of finding a difference simply due to chance increases as the number of outcomes increases.\r\n

Checklist Questions<\/h1>\r\n\r\n\r\n\r\n\r\n\r\n\r\n
Are we trying to find a difference in a [pb_glossary id=\"1532\"]secondary outcome[\/pb_glossary] when there was no statistically significant difference between groups for the [pb_glossary id=\"1517\"]primary outcome[\/pb_glossary]?<\/td>\r\n<\/tr>\r\n
Was the [pb_glossary id=\"1532\"]secondary outcome[\/pb_glossary] one of a small number of secondary endpoints defined in the original protocol? If there was a positive finding, were there appropriate statistical adjustments made?<\/td>\r\n<\/tr>\r\n
Does the [pb_glossary id=\"1532\"]secondary endpoint[\/pb_glossary] result make sense in the context of the [pb_glossary id=\"290\"]primary[\/pb_glossary] (and other secondary) outcome findings?<\/td>\r\n<\/tr>\r\n
Was there an unexpected positive finding for a rare outcome?<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Data-mining: Are we trying to find a difference in a secondary outcome when there was no statistically significant difference between groups for the primary outcome?<\/h1>\r\nAuthors may emphasize statistically significant differences in [pb_glossary id=\"1532\"]secondary outcomes[\/pb_glossary] when they fail to find a statistically significant difference in the [pb_glossary id=\"290\"]primary outcome[\/pb_glossary].\r\n\r\nFor example, a review (Khan MS et al.<\/a>) of 93 cardiovascular [pb_glossary id=\"704\"]RCTs[\/pb_glossary] found that spin (i.e. reporting strategies highlighting benefits despite a non-statistically significant [pb_glossary id=\"1517\"]primary outcome[\/pb_glossary]) was present in 57% of abstracts and 67% of main texts.\r\n
E.g. In the FIELD trial (Keech A et al.<\/a>), the difference in the [pb_glossary id=\"1517\"]primary outcome[\/pb_glossary] (coronary events up to 5 years) was not statistically significant between fenofibrate and placebo in patients with type 2 diabetes<\/em>. In their conclusions, authors highlighted marginally statistically <\/em>significant reduction in 3 of 9 [pb_glossary id=\"1532\"]secondary[\/pb_glossary] efficacy outcomes (total cardiovascular events, non-fatal myocardial infarction, and revascularization).<\/em><\/div>\r\n

Minimizing multiplicity:\u00a0<\/span><\/span>Was the secondary outcome one of a small number of secondary endpoints defined in the original protocol? If there was a positive finding, were there appropriate adjustments made?<\/span><\/span><\/h1>\r\nMore comparisons increase the risk of finding a difference when there is none, as depicted:\r\n\r\n[caption id=\"attachment_776\" align=\"alignnone\" width=\"1024\"]\"\" Graph 1. Probability of at least one false positive result by number of outcomes tested assuming no difference and threshold for statistical significance <0.05.[\/caption]\r\n\r\nDepending on the context, it may be justified to adjust for multiplicity when considering multiple outcomes. Adjusting for multiplicity is a statistical method of requiring lower p-values<\/a> to account for multiple comparisons. There are multiple methods for calculating the stricter margin (Bonferroni test, Holm test, etc.). There is no consensus on when to adjust for multiplicity, but the following can act as general guidance:<\/span>\r\n\r\n\r\n\r\n\r\n
Table 10. Circumstances where adjusting for multiplicity may be necessary.<\/caption>\r\n
Circumstance<\/strong><\/td>\r\nWhether Adjustment is Necessary<\/b><\/td>\r\n<\/tr>\r\n
At least one outcome is positive and the outcome is intended to inform future research rather than be incorporated directly into clinical practice (i.e. exploratory)<\/td>\r\nAdjustments in the analysis are not warranted as such findings are used only to generate hypotheses<\/td>\r\n<\/tr>\r\n
At least one outcome is positive and the outcome is intended to directly inform clinical practice (i.e. confirmatory)<\/td>\r\nAdjustments may be necessary if:\r\n- More than one dose is compared\r\n- More than one [pb_glossary id=\"1517\"]primary outcome[\/pb_glossary] is used\r\n- The [pb_glossary id=\"1517\"]primary outcome[\/pb_glossary] was assessed in multiple different population<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Consistency: Does the secondary endpoint result make sense in the context of the primary (and other secondary) outcome findings?<\/span><\/span><\/h1>\r\nOutcomes with similar pathophysiology (e.g. myocardial infarction and ischemic stroke with antihypertensive agents) should move in the same direction (both increased or both decreased), whereas outcomes with opposing pathophysiology (e.g. myocardial infarction and bleeding with antiplatelets) should move in opposite directions.\r\n
E.g. In FIELD (Keech A et al.<\/a>), the [pb_glossary id=\"1532\"]secondary outcome[\/pb_glossary] of non-fatal myocardial infarction was statistically significantly less<\/strong> with the fenofibrate group compared to placebo. However, all-cause mortality, coronary death, deep vein thrombosis, and pulmonary embolism occurred more<\/strong> frequently in the fenofibrate group.<\/em><\/div>\r\n

Was there an unexpected positive finding for a rare outcome?<\/h1>\r\nOne should be skeptical whenever an unexpected statistically significant reduction is found in a rare [pb_glossary id=\"1532\"]secondary outcome[\/pb_glossary], particularly when there is no difference in the [pb_glossary id=\"1517\"]primary outcome[\/pb_glossary].\r\n
E.g. The ELITE trial (Pitt B, Segal R, et al.<\/a>) comparing losartan to captopril in 722 elderly heart failure patients failed to find a significant difference in the incidence of the [pb_glossary id=\"1517\"]primary outcome[\/pb_glossary], increase in serum creatinine (11% in both groups). There was, however, an unexpected reduction in all-cause mortality with losartan vs. captopril (5% vs 9%, p<\/a>=0.04). The follow-up ELITE II trial (Pitt B, Poole-Wilson PA, et al.<\/a>) with its larger sample of 3152 patients and a [pb_glossary id=\"1517\"]primary outcome[\/pb_glossary] of mortality found no reduction in \u2013 and in fact numerically higher \u2013 mortality with losartan vs captopril (18% vs 16%, p<\/a>=0.16).<\/em><\/div>","rendered":"

Most trials designate one outcome as a \"primary\" outcome<\/a> (or 2-3 “co-primary outcomes”) and all other outcomes as \"secondary\" outcomes<\/a>. Designation of an outcome as “primary” is done to determine and justify sample size calculations prior to conducting a study. In other words, the primary outcome<\/a> is not necessarily the most clinically important (it often isn’t), and should not be the sole consideration as to whether an intervention is “better” than a comparator.<\/p>\n

The interpretation of secondary outcomes<\/a> requires additional considerations. The probability of finding a difference simply due to chance increases as the number of outcomes increases.<\/p>\n

Checklist Questions<\/h1>\n\n\n\n\n\n\n
Are we trying to find a difference in a secondary outcome<\/a> when there was no statistically significant difference between groups for the primary outcome<\/a>?<\/td>\n<\/tr>\n
Was the secondary outcome<\/a> one of a small number of secondary endpoints defined in the original protocol? If there was a positive finding, were there appropriate statistical adjustments made?<\/td>\n<\/tr>\n
Does the secondary endpoint<\/a> result make sense in the context of the primary<\/a> (and other secondary) outcome findings?<\/td>\n<\/tr>\n
Was there an unexpected positive finding for a rare outcome?<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Data-mining: Are we trying to find a difference in a secondary outcome when there was no statistically significant difference between groups for the primary outcome?<\/h1>\n

Authors may emphasize statistically significant differences in secondary outcomes<\/a> when they fail to find a statistically significant difference in the primary outcome<\/a>.<\/p>\n

For example, a review (Khan MS et al.<\/a>) of 93 cardiovascular RCTs<\/a> found that spin (i.e. reporting strategies highlighting benefits despite a non-statistically significant primary outcome<\/a>) was present in 57% of abstracts and 67% of main texts.<\/p>\n

E.g. In the FIELD trial (Keech A et al.<\/a>), the difference in the primary outcome<\/a> (coronary events up to 5 years) was not statistically significant between fenofibrate and placebo in patients with type 2 diabetes<\/em>. In their conclusions, authors highlighted marginally statistically <\/em>significant reduction in 3 of 9 secondary<\/a> efficacy outcomes (total cardiovascular events, non-fatal myocardial infarction, and revascularization).<\/em><\/div>\n

Minimizing multiplicity:\u00a0<\/span><\/span>Was the secondary outcome one of a small number of secondary endpoints defined in the original protocol? If there was a positive finding, were there appropriate adjustments made?<\/span><\/span><\/h1>\n

More comparisons increase the risk of finding a difference when there is none, as depicted:<\/p>\n

\"\"
Graph 1. Probability of at least one false positive result by number of outcomes tested assuming no difference and threshold for statistical significance <0.05.<\/figcaption><\/figure>\n

Depending on the context, it may be justified to adjust for multiplicity when considering multiple outcomes. Adjusting for multiplicity is a statistical method of requiring lower p-values<\/a> to account for multiple comparisons. There are multiple methods for calculating the stricter margin (Bonferroni test, Holm test, etc.). There is no consensus on when to adjust for multiplicity, but the following can act as general guidance:<\/span><\/p>\n\n\n\n\n\n
Table 10. Circumstances where adjusting for multiplicity may be necessary.<\/caption>\n
Circumstance<\/strong><\/td>\nWhether Adjustment is Necessary<\/b><\/td>\n<\/tr>\n
At least one outcome is positive and the outcome is intended to inform future research rather than be incorporated directly into clinical practice (i.e. exploratory)<\/td>\nAdjustments in the analysis are not warranted as such findings are used only to generate hypotheses<\/td>\n<\/tr>\n
At least one outcome is positive and the outcome is intended to directly inform clinical practice (i.e. confirmatory)<\/td>\nAdjustments may be necessary if:
\n– More than one dose is compared
\n– More than one primary outcome<\/a> is used
\n– The primary outcome<\/a> was assessed in multiple different population<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Consistency: Does the secondary endpoint result make sense in the context of the primary (and other secondary) outcome findings?<\/span><\/span><\/h1>\n

Outcomes with similar pathophysiology (e.g. myocardial infarction and ischemic stroke with antihypertensive agents) should move in the same direction (both increased or both decreased), whereas outcomes with opposing pathophysiology (e.g. myocardial infarction and bleeding with antiplatelets) should move in opposite directions.<\/p>\n

E.g. In FIELD (Keech A et al.<\/a>), the secondary outcome<\/a> of non-fatal myocardial infarction was statistically significantly less<\/strong> with the fenofibrate group compared to placebo. However, all-cause mortality, coronary death, deep vein thrombosis, and pulmonary embolism occurred more<\/strong> frequently in the fenofibrate group.<\/em><\/div>\n

Was there an unexpected positive finding for a rare outcome?<\/h1>\n

One should be skeptical whenever an unexpected statistically significant reduction is found in a rare secondary outcome<\/a>, particularly when there is no difference in the primary outcome<\/a>.<\/p>\n

E.g. The ELITE trial (Pitt B, Segal R, et al.<\/a>) comparing losartan to captopril in 722 elderly heart failure patients failed to find a significant difference in the incidence of the primary outcome<\/a>, increase in serum creatinine (11% in both groups). There was, however, an unexpected reduction in all-cause mortality with losartan vs. captopril (5% vs 9%, p<\/a>=0.04). The follow-up ELITE II trial (Pitt B, Poole-Wilson PA, et al.<\/a>) with its larger sample of 3152 patients and a primary outcome<\/a> of mortality found no reduction in \u2013 and in fact numerically higher \u2013 mortality with losartan vs captopril (18% vs 16%, p<\/a>=0.16).<\/em><\/div>\n
definition<\/span>