{"id":96,"date":"2021-05-22T23:38:19","date_gmt":"2021-05-23T03:38:19","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/?post_type=chapter&#038;p=96"},"modified":"2022-01-12T23:30:43","modified_gmt":"2022-01-13T04:30:43","slug":"results-of-the-systematic-review-i-e-which-trials-were-included","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/chapter\/results-of-the-systematic-review-i-e-which-trials-were-included\/","title":{"raw":"Results of the systematic review","rendered":"Results of the systematic review"},"content":{"raw":"The quality of the [pb_glossary id=\"1099\"]systematic review[\/pb_glossary] depends both on the quality of the individual studies and the aggregate characteristics of these studies. If the aggregate results are missing studies, contain predominantly poorly conducted studies, or are highly [pb_glossary id=\"107\"]heterogeneous[\/pb_glossary] then this will likely warrant lower confidence in the results.\r\n<h1>Checklist Questions<\/h1>\r\n<table class=\"grid\" style=\"border-collapse: collapse;width: 100%;height: 90px\" border=\"0\">\r\n<tbody>\r\n<tr style=\"height: 18px\">\r\n<td style=\"width: 100%;height: 18px\">Do all inclusions &amp; exclusions of trials make sense?<\/td>\r\n<\/tr>\r\n<tr style=\"height: 18px\">\r\n<td style=\"width: 100%;height: 18px\">Are you aware of any relevant studies that were not identified\/included in this review?<\/td>\r\n<\/tr>\r\n<tr style=\"height: 18px\">\r\n<td style=\"width: 100%;height: 18px\">Did reviewers adequately assess individual trials for risk of [pb_glossary id=\"193\"]bias[\/pb_glossary]?<\/td>\r\n<\/tr>\r\n<tr style=\"height: 18px\">\r\n<td style=\"width: 100%;height: 18px\">Was each component reported separately, or summarized with a composite quality score?<\/td>\r\n<\/tr>\r\n<tr style=\"height: 18px\">\r\n<td style=\"width: 100%;height: 18px\">Are there any differences between studies that should preclude [pb_glossary id=\"1101\"]meta-analysis[\/pb_glossary]?<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<h1><span class=\"TextRun BCX9 SCXW174162427\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun BCX9 SCXW174162427\">Risk of bias<\/span><\/span><span class=\"TextRun BCX9 SCXW174162427\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun BCX9 SCXW174162427\">\u00a0<\/span><\/span><span class=\"TextRun BCX9 SCXW174162427\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun BCX9 SCXW174162427\">with<\/span><\/span><span class=\"TextRun BCX9 SCXW174162427\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun BCX9 SCXW174162427\">in trials (internal validity): Did reviewers adequately assess for (&amp; report) risk of bias?<\/span><\/span><\/h1>\r\nRisk of [pb_glossary id=\"193\"]bias[\/pb_glossary] should be evaluated by using a tool that is specific to [pb_glossary id=\"704\"]RCTs[\/pb_glossary]. The Cochrane risk of bias tool (version 1 (<a href=\"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/back-matter\/references\/\" target=\"_blank\" rel=\"noopener\">Higgins JPT et al.<\/a> 2011) or 2 (<a href=\"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/back-matter\/references\/\" target=\"_blank\" rel=\"noopener\">Sterne JAC et al.<\/a> 2019)) evaluates the risk of individual trial [pb_glossary id=\"193\"]biases[\/pb_glossary] and offers the most transparent assessment of trial [pb_glossary id=\"105\"]internal validity[\/pb_glossary] (see <a href=\"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/chapter\/chapter-1\/\" target=\"_blank\" rel=\"noopener\">NERDCAT-RCT<\/a> for more information regarding [pb_glossary id=\"105\"]internal validity[\/pb_glossary]). ROBIS-I (<a href=\"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/back-matter\/references\/\" target=\"_blank\" rel=\"noopener\">Sterne JA et al.<\/a>) is a similar tool available for appraising risk of [pb_glossary id=\"193\"]bias[\/pb_glossary] in observational trials.\r\n<h2><strong>Quality Scores<\/strong><\/h2>\r\n\u201cQuality scores\u201d such as the Jadad score are more closely related to reporting quality than methodological issues, and lead to wide variability in conclusions on \u201cquality\u201d based on the score used. In particular, the Jadad score is considered obsolete and is a poor measure of risk of [pb_glossary id=\"193\"]bias[\/pb_glossary].\r\n<h1><span class=\"TextRun BCX9 SCXW121707707\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun BCX9 SCXW121707707\">Methodological &amp; clinical heterogeneity: Is it appropriate to perform a meta-analysis?<\/span><\/span><\/h1>\r\n<ul>\r\n \t<li>Methodological [pb_glossary id=\"107\"]heterogeneity[\/pb_glossary]: Are there methodological differences (e.g. risk of [pb_glossary id=\"193\"]bias[\/pb_glossary]) between studies?<\/li>\r\n \t<li>Clinical [pb_glossary id=\"107\"]heterogeneity[\/pb_glossary]: Are there any differences in clinical characteristics between the individual trials (i.e. any component of [pb_glossary id=\"323\"]PICO[\/pb_glossary]) that preclude pooling the trials together in a [pb_glossary id=\"1101\"]meta-analysis[\/pb_glossary]?<\/li>\r\n \t<li>Is the impact of any of these characteristics tested in a subgroup analysis or meta-regression?<\/li>\r\n<\/ul>\r\nTesting possible sources of [pb_glossary id=\"107\"]heterogeneity[\/pb_glossary] may identify causes for statistical [pb_glossary id=\"107\"]heterogeneity[\/pb_glossary] identified in the [pb_glossary id=\"1101\"]meta-analysis[\/pb_glossary] (e.g. the intervention may only appear beneficial in trials at high risk of [pb_glossary id=\"193\"]bias[\/pb_glossary], but not in those at low risk).\r\n\r\n<span class=\"TextRun SCXW11926480 BCX9\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW11926480 BCX9\">See <a href=\"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/chapter\/subgroups-were-additional-comparisons-made-on-segments-of-the-study-population\/\" target=\"_blank\" rel=\"noopener\">NERDCAT-RCT<\/a> to learn more on how to\u00a0<\/span><\/span><span class=\"TextRun SCXW11926480 BCX9\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW11926480 BCX9\">appraise validity\u00a0<\/span><\/span><span class=\"TextRun SCXW11926480 BCX9\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW11926480 BCX9\">of subgroup<\/span><\/span><span class=\"TextRun SCXW11926480 BCX9\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW11926480 BCX9\"> effects.<\/span><\/span>","rendered":"<p>The quality of the <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_1099\">systematic review<\/a> depends both on the quality of the individual studies and the aggregate characteristics of these studies. If the aggregate results are missing studies, contain predominantly poorly conducted studies, or are highly <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_107\">heterogeneous<\/a> then this will likely warrant lower confidence in the results.<\/p>\n<h1>Checklist Questions<\/h1>\n<table class=\"grid\" style=\"border-collapse: collapse;width: 100%;height: 90px\">\n<tbody>\n<tr style=\"height: 18px\">\n<td style=\"width: 100%;height: 18px\">Do all inclusions &amp; exclusions of trials make sense?<\/td>\n<\/tr>\n<tr style=\"height: 18px\">\n<td style=\"width: 100%;height: 18px\">Are you aware of any relevant studies that were not identified\/included in this review?<\/td>\n<\/tr>\n<tr style=\"height: 18px\">\n<td style=\"width: 100%;height: 18px\">Did reviewers adequately assess individual trials for risk of <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_193\">bias<\/a>?<\/td>\n<\/tr>\n<tr style=\"height: 18px\">\n<td style=\"width: 100%;height: 18px\">Was each component reported separately, or summarized with a composite quality score?<\/td>\n<\/tr>\n<tr style=\"height: 18px\">\n<td style=\"width: 100%;height: 18px\">Are there any differences between studies that should preclude <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_1101\">meta-analysis<\/a>?<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h1><span class=\"TextRun BCX9 SCXW174162427\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun BCX9 SCXW174162427\">Risk of bias<\/span><\/span><span class=\"TextRun BCX9 SCXW174162427\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun BCX9 SCXW174162427\">\u00a0<\/span><\/span><span class=\"TextRun BCX9 SCXW174162427\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun BCX9 SCXW174162427\">with<\/span><\/span><span class=\"TextRun BCX9 SCXW174162427\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun BCX9 SCXW174162427\">in trials (internal validity): Did reviewers adequately assess for (&amp; report) risk of bias?<\/span><\/span><\/h1>\n<p>Risk of <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_193\">bias<\/a> should be evaluated by using a tool that is specific to <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_704\">RCTs<\/a>. The Cochrane risk of bias tool (version 1 (<a href=\"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/back-matter\/references\/\" target=\"_blank\" rel=\"noopener\">Higgins JPT et al.<\/a> 2011) or 2 (<a href=\"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/back-matter\/references\/\" target=\"_blank\" rel=\"noopener\">Sterne JAC et al.<\/a> 2019)) evaluates the risk of individual trial <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_193\">biases<\/a> and offers the most transparent assessment of trial <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_105\">internal validity<\/a> (see <a href=\"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/chapter\/chapter-1\/\" target=\"_blank\" rel=\"noopener\">NERDCAT-RCT<\/a> for more information regarding <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_105\">internal validity<\/a>). ROBIS-I (<a href=\"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/back-matter\/references\/\" target=\"_blank\" rel=\"noopener\">Sterne JA et al.<\/a>) is a similar tool available for appraising risk of <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_193\">bias<\/a> in observational trials.<\/p>\n<h2><strong>Quality Scores<\/strong><\/h2>\n<p>\u201cQuality scores\u201d such as the Jadad score are more closely related to reporting quality than methodological issues, and lead to wide variability in conclusions on \u201cquality\u201d based on the score used. In particular, the Jadad score is considered obsolete and is a poor measure of risk of <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_193\">bias<\/a>.<\/p>\n<h1><span class=\"TextRun BCX9 SCXW121707707\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun BCX9 SCXW121707707\">Methodological &amp; clinical heterogeneity: Is it appropriate to perform a meta-analysis?<\/span><\/span><\/h1>\n<ul>\n<li>Methodological <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_107\">heterogeneity<\/a>: Are there methodological differences (e.g. risk of <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_193\">bias<\/a>) between studies?<\/li>\n<li>Clinical <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_107\">heterogeneity<\/a>: Are there any differences in clinical characteristics between the individual trials (i.e. any component of <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_323\">PICO<\/a>) that preclude pooling the trials together in a <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_1101\">meta-analysis<\/a>?<\/li>\n<li>Is the impact of any of these characteristics tested in a subgroup analysis or meta-regression?<\/li>\n<\/ul>\n<p>Testing possible sources of <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_107\">heterogeneity<\/a> may identify causes for statistical <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_107\">heterogeneity<\/a> identified in the <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_1101\">meta-analysis<\/a> (e.g. the intervention may only appear beneficial in trials at high risk of <a class=\"glossary-term\" aria-haspopup=\"dialog\" aria-describedby=\"definition\" href=\"#term_96_193\">bias<\/a>, but not in those at low risk).<\/p>\n<p><span class=\"TextRun SCXW11926480 BCX9\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW11926480 BCX9\">See <a href=\"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/chapter\/subgroups-were-additional-comparisons-made-on-segments-of-the-study-population\/\" target=\"_blank\" rel=\"noopener\">NERDCAT-RCT<\/a> to learn more on how to\u00a0<\/span><\/span><span class=\"TextRun SCXW11926480 BCX9\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW11926480 BCX9\">appraise validity\u00a0<\/span><\/span><span class=\"TextRun SCXW11926480 BCX9\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW11926480 BCX9\">of subgroup<\/span><\/span><span class=\"TextRun SCXW11926480 BCX9\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW11926480 BCX9\"> effects.<\/span><\/span><\/p>\n<div class=\"glossary\"><span class=\"screen-reader-text\" id=\"definition\">definition<\/span><template id=\"term_96_1099\"><div class=\"glossary__definition\" role=\"dialog\" data-id=\"term_96_1099\"><div tabindex=\"-1\"><p>A review that systematically identifies all potentially relevant studies on a research question. The aggregate of studies is then evaluated with respect to factors such as risk of bias of individual studies or heterogeneity among results. The qualitative combination of results is a systematic review.<\/p>\n<\/div><button><span aria-hidden=\"true\">&times;<\/span><span class=\"screen-reader-text\">Close definition<\/span><\/button><\/div><\/template><template id=\"term_96_107\"><div class=\"glossary__definition\" role=\"dialog\" data-id=\"term_96_107\"><div tabindex=\"-1\"><p>Refers to variability between studies in a systematic review. It can refer to clinical differences, methodological differences, or variable results between studies. Heterogeneity occurs on a continuum and, in the case of heterogeneity amongst results, can be expressed numerically via measures of statistical heterogeneity. See <a href=\"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/chapter\/results-of-the-meta-analysis-i-e-what-do-the-pooled-results-of-the-trials-show\/\">here<\/a> for a further discussion of statistical heterogeneity.<\/p>\n<\/div><button><span aria-hidden=\"true\">&times;<\/span><span class=\"screen-reader-text\">Close definition<\/span><\/button><\/div><\/template><template id=\"term_96_193\"><div class=\"glossary__definition\" role=\"dialog\" data-id=\"term_96_193\"><div tabindex=\"-1\"><p>Systematic deviation of an estimate from the truth (either an overestimation or underestimation) caused by a study design or conduct feature. See the <a href=\"https:\/\/catalogofbias.org\/biases\/\">Catalog of Bias<\/a> for specific biases, explanations, and examples.<\/p>\n<\/div><button><span aria-hidden=\"true\">&times;<\/span><span class=\"screen-reader-text\">Close definition<\/span><\/button><\/div><\/template><template id=\"term_96_1101\"><div class=\"glossary__definition\" role=\"dialog\" data-id=\"term_96_1101\"><div tabindex=\"-1\"><p>A meta-analysis is a quantitative combination of the data obtained in a systematic review.<\/p>\n<\/div><button><span aria-hidden=\"true\">&times;<\/span><span class=\"screen-reader-text\">Close definition<\/span><\/button><\/div><\/template><template id=\"term_96_704\"><div class=\"glossary__definition\" role=\"dialog\" data-id=\"term_96_704\"><div tabindex=\"-1\"><p>Randomized controlled trials are those in which participants are randomly allocated to two or more groups which are given different treatments.<\/p>\n<\/div><button><span aria-hidden=\"true\">&times;<\/span><span class=\"screen-reader-text\">Close definition<\/span><\/button><\/div><\/template><template id=\"term_96_105\"><div class=\"glossary__definition\" role=\"dialog\" data-id=\"term_96_105\"><div tabindex=\"-1\"><p>The extent to which the study results are attributable to the intervention and not to bias. If internal validity is high, there is high confidence that the results are due to the effects of treatment (with low internal validity entailing low confidence).<\/p>\n<\/div><button><span aria-hidden=\"true\">&times;<\/span><span class=\"screen-reader-text\">Close definition<\/span><\/button><\/div><\/template><template id=\"term_96_323\"><div class=\"glossary__definition\" role=\"dialog\" data-id=\"term_96_323\"><div tabindex=\"-1\"><p>An acronym for \"patient, intervention, comparator, and outcome\". These are the four basic elements of a study. For instance, a study may examine an elderly population (P) to understand the effects of statin therapy (I) compared to placebo (C) in terms of cardiovascular events (O). Sometimes extended to PICO(T) to include the time at which outcomes were assessed, or (D)PICO to incorporate the study design.<\/p>\n<\/div><button><span aria-hidden=\"true\">&times;<\/span><span class=\"screen-reader-text\">Close definition<\/span><\/button><\/div><\/template><\/div>","protected":false},"author":1318,"menu_order":2,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[],"contributor":[],"license":[],"class_list":["post-96","chapter","type-chapter","status-publish","hentry"],"part":25,"_links":{"self":[{"href":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/wp-json\/pressbooks\/v2\/chapters\/96","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/wp-json\/wp\/v2\/users\/1318"}],"version-history":[{"count":25,"href":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/wp-json\/pressbooks\/v2\/chapters\/96\/revisions"}],"predecessor-version":[{"id":1896,"href":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/wp-json\/pressbooks\/v2\/chapters\/96\/revisions\/1896"}],"part":[{"href":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/wp-json\/pressbooks\/v2\/parts\/25"}],"metadata":[{"href":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/wp-json\/pressbooks\/v2\/chapters\/96\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/wp-json\/wp\/v2\/media?parent=96"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/wp-json\/pressbooks\/v2\/chapter-type?post=96"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/wp-json\/wp\/v2\/contributor?post=96"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/rickyturgeon\/wp-json\/wp\/v2\/license?post=96"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}