{"id":5,"date":"2021-01-18T15:55:53","date_gmt":"2021-01-18T20:55:53","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/2021\/01\/18\/chapter-1\/"},"modified":"2024-03-13T15:45:38","modified_gmt":"2024-03-13T19:45:38","slug":"chapter-1","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/chapter\/chapter-1\/","title":{"raw":"Risk of bias: Are the results internally valid?","rendered":"Risk of bias: Are the results internally valid?"},"content":{"raw":"Randomization is the core of the [pb_glossary id=\"704\"]RCT[\/pb_glossary] and ensures that the play of chance dictates whether any given participant is assigned the intervention or comparator(s). Because of this, baseline characteristics tend to be similar between randomized groups, though imbalances can still occur by chance. So, at the start of the trial, each group should tend to have a similar probability of experiencing any outcome. If this similarity is properly maintained (i.e. neither [pb_glossary id=\"193\"]bias[\/pb_glossary] nor confounding<\/a> introduces differences between groups), then any differences in outcomes will either be due to either treatment allocation or to chance.\r\n\r\nTowards this objective of maintaining similar groups, other strategies (such as blinding of participants, clinicians, and investigators) are often implemented to minimize differences in care between groups over the course of the trial. If these strategies are not successful, then any differences in outcomes between groups could also be attributable to these differences in care, thus introducing [pb_glossary id=\"193\"]bias[\/pb_glossary].\r\n\r\nThe [pb_glossary id=\"105\"]internal validity[\/pb_glossary] sections of this chapter will focus on describing key sources of [pb_glossary id=\"193\"]bias[\/pb_glossary] in [pb_glossary id=\"704\"]RCTs[\/pb_glossary], how to identify them, and how to evaluate their impact on observed study results.\r\n

Checklist Questions<\/h1>\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
Was the [pb_glossary id=\"48\"]sequence generation[\/pb_glossary] random?<\/td>\r\n<\/tr>\r\n
Was the allocation sequence concealed until participants were enrolled and assigned to interventions?<\/td>\r\n<\/tr>\r\n
Were participants, clinicians, outcome assessors, and investigators blinded?<\/td>\r\n<\/tr>\r\n
Were there deviations from the intended intervention that arose because of the above?<\/td>\r\n<\/tr>\r\n
Could measurement or ascertainment of the outcome have differed between intervention groups?<\/td>\r\n<\/tr>\r\n
Could assessment of the outcome have been influenced by knowledge of intervention received?<\/td>\r\n<\/tr>\r\n
Did participants from the comparator group receive the intervention from the intervention group (or vice versa)?<\/td>\r\n<\/tr>\r\n
Were data for key outcomes available for all, or nearly all, participants randomized?<\/td>\r\n<\/tr>\r\n
Were patients analyzed in the groups to which they were randomized ([pb_glossary id=\"39\"]ITT[\/pb_glossary]), or did researchers only count participants who were adherent to their study treatment ([pb_glossary id=\"138\"]per protocol[\/pb_glossary]) or completed the full trial duration (completer analysis)?<\/td>\r\n<\/tr>\r\n
Are the [pb_glossary id=\"39\"]ITT[\/pb_glossary] methods appropriate?<\/td>\r\n<\/tr>\r\n
Are any important outcomes included in the study protocol but absent from the publication? Is this justified?<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Allocation Bias: Were patients app<\/span><\/span>ropriately randomized with allocation concealment?<\/span><\/span>\u00a0<\/span><\/h1>\r\n

Sequence generation (i.e. randomization)<\/strong><\/h2>\r\n

Unclear or inadequate [pb_glossary id=\"48\"]sequence generation[\/pb_glossary] exaggerates [pb_glossary id=\"119\"]relative benefits[\/pb_glossary] of an intervention on average by ~11% (Savovi\u0107 J et al.<\/a>).<\/p>\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
Table 2. Adequate and inadequate randomization methods.<\/caption>\r\n
Adequate<\/span> Randomization<\/strong><\/td>\r\nInadequate<\/span> Randomization<\/strong><\/td>\r\n<\/tr>\r\n
Computer-generated random sequence generation (preferred)<\/strong><\/td>\r\nQuasi-randomization (e.g. alternation by case number or date of birth)<\/td>\r\n<\/tr>\r\n
Random numbers table<\/td>\r\nTreatment assignment left to the discretion of the clinician<\/td>\r\n<\/tr>\r\n
Coin toss<\/td>\r\n<\/td>\r\n<\/tr>\r\n
Drawing cards<\/td>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Allocation concealment<\/span><\/span><\/strong><\/h2>\r\n

Unclear or inadequate <\/span>[pb_glossary id=\"34\"]allocation concealment[\/pb_glossary] exaggerate [pb_glossary id=\"119\"]relative benefits[\/pb_glossary] of an intervention on average by ~7% (Savovi\u0107 J et al.<\/a>).<\/p>\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
Table 3. Adequate and inadequate allocation concealment methods.<\/caption>\r\n
Adequate<\/span> allocation concealment<\/strong><\/td>\r\nInadequate<\/strong><\/span>\u00a0allocation concealment<\/strong><\/td>\r\n<\/tr>\r\n
Central randomization (look for \"interactive web-response system\" or \"interactive voice-response system\" within a study manuscript) (preferred)<\/strong><\/td>\r\nAllocation scheme posted on a bulletin board<\/td>\r\n<\/tr>\r\n
Coded identical drug boxes\/vials<\/td>\r\nNon-opaque, non-tamper proof envelopes<\/td>\r\n<\/tr>\r\n
Sequentially-numbered, tamper-proof, sealed opaque envelopes (preferably lined with cardboard or foil)<\/td>\r\n<\/td>\r\n<\/tr>\r\n
On-site locked computer system<\/td>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Blinding: Were participants, treating clinicians, outcome assessors, or investigators aware of treatment assignment during the trial?<\/h1>\r\nLack of or unclear blinding is associated with an average ~13% exaggeration of the [pb_glossary id=\"119\"]relative benefits[\/pb_glossary] of an intervention for dichotomous outcomes (Savovi\u0107 J et al.<\/a>), and a 68% exaggeration of [pb_glossary id=\"119\"]relative benefits[\/pb_glossary] for subjective continuous outcomes (Hr\u00f3bjartsson A et al.<\/a>).\r\n\r\nNote that [pb_glossary id=\"37\"]double-blinding[\/pb_glossary] does not have a standardized definition and, consequently, further examinations are needed to ascertain exactly who was blinded (Lang TA et al<\/a>).\r\n

Blinding of participants and clinicians<\/strong><\/h2>\r\n\r\n\r\n\r\n
Table 4. Adequate and inadequate blinding methods for participants and clinicians.<\/caption>\r\n
Adequate<\/span> blinding of participants and clinicians<\/strong><\/td>\r\nInadequate<\/span> blinding of participants and clinicians<\/strong><\/td>\r\n<\/tr>\r\n
Used an identical placebo\/control product without indication that treatments were distinguishable<\/td>\r\nPROBE: Prospective randomized open-label, blinded endpoint trial (open-label refers to trial that has non-blinding as part of the design, and does not refer to cases where blinding is simply inadequate)<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nBlinding of outcome assessors<\/span><\/span><\/strong>\u00a0<\/span>\r\n\r\nAwareness of treatment allocation by participants and clinicians may introduce performance bias<\/a>, whereas awareness of allocation by outcome assessors may introduce detection bias<\/a>. This is compounded when participants or their clinicians are also the outcome assessors (e.g. patient aware of treatment allocation asked to rate their pain or fill out a quality-of-life questionnaire). Lack of blinding is a particularly important source of [pb_glossary id=\"193\"]bias[\/pb_glossary] with the use of subjective outcomes - one review (Wood L et al.<\/a>) found that lack of blinding exaggerated the [pb_glossary id=\"103\"]OR[\/pb_glossary] of subjective outcomes by ~30%. Conversely, the same review found no statistically significant [pb_glossary id=\"193\"]bias[\/pb_glossary] was introduced by lack of blinding for objective outcomes. This review provided evidence that all-cause mortality is a particularly resistant to detection [pb_glossary id=\"193\"]bias[\/pb_glossary] even when trials are not blinded.\r\n\r\n\r\n\r\n
Table 5. Adequate blinding methods for outcome accessors and difficult situations to blind.<\/caption>\r\n
Adequate<\/span> blinding of outcome assessors<\/strong><\/td>\r\nDifficult<\/span> situations to blind<\/strong><\/td>\r\n<\/tr>\r\n
Independent central adjudication committee adjudicated outcomes<\/td>\r\nThe intervention has an effect on a readily-measurable biomarker or the drug has an easily observable adverse effect profile (e.g. iron causing darkened stools)<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
\r\n
E.g. #1 Among several concerns raised by the FDA regarding the PLATO trial (DiNicolantonio JJ et al.<\/a>), a [pb_glossary id=\"704\"]RCT[\/pb_glossary] comparing ticagrelor vs. clopidogrel for patients with acute coronary syndrome, it was noted that blinding was not sufficiently protected. This is because the \u201cdummy capsules\u201d (identical in appearance to the ticagrelor containing capsule) could be opened, revealing a clopidogrel tablet cut in half. This could unblind both patients and sponsor site monitors (who were given unused capsules). There were also concerns that too many groups involved in the trial had access to treatment assignments (and could subsequently become unblinded). These concerns cast doubts on the [pb_glossary id=\"105\"]internal validity[\/pb_glossary] of both the efficacy and safety outcomes, especially when combined with additional concerns by the FDA regarding the inaccuracy of reported events.<\/em><\/div>\r\nSome situations initially thought to be impossible to blind can be successfully blinded with some ingenuity.\r\n
E.g. #2 In ROCKET-AF (Patel MR et al.<\/a>), INR was measured centrally and clinicians taking care of patients on rivaroxaban were given dummy INR values for which to adjust the warfarin-placebo dose.<\/em><\/div>\r\n

Were there differences between groups in the receipt of co-interventions?<\/strong><\/h2>\r\nCo-interventions may introduce [pb_glossary id=\"193\"]bias[\/pb_glossary] if they affect the outcomes of interest and are distributed differently between groups.\r\n
E.g. #3 CONTACT (Roddy E et al.<\/a>), an unblinded non-inferiority trial<\/a> comparing naproxen and colchicine for acute gout attacks, found no difference in pain control between groups. However, co-intervention analgesic use (e.g. acetaminophen, ibuprofen) was 42% in the colchicine group and only 25% in the naproxen group. This raises the possibility that pain control would have been inferior in the colchicine group had it not been for the additional analgesic use.<\/em><\/div>\r\n

Was outcome monitoring assessed consistently between groups? If no, then was this likely to bias the results?<\/strong><\/h2>\r\nOutcome measures ought to be consistent between groups with regards to:\r\n