{"id":2039,"date":"2023-11-26T19:38:13","date_gmt":"2023-11-27T00:38:13","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/?post_type=chapter&#038;p=2039"},"modified":"2025-10-30T08:57:23","modified_gmt":"2025-10-30T12:57:23","slug":"vulnerability-multipliers-discussion","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/chapter\/vulnerability-multipliers-discussion\/","title":{"raw":"Vulnerability multipliers: Discussion","rendered":"Vulnerability multipliers: Discussion"},"content":{"raw":"<div>\r\n<h2><img class=\"alignnone size-full wp-image-2663\" src=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/JB_website-image.jpg\" alt=\"\" width=\"1650\" height=\"1275\" \/><\/h2>\r\n<h2>Limitations and future development<\/h2>\r\n<p style=\"font-weight: 400\">It is not enough to estimate vulnerability multipliers and assume those which produce the best statistical hindcast fit are appropriate. Ecological reasoning and hypothesis testing must support statistical inference as it should when balancing Ecopath models[footnote]Link, J.S. 2010. Adding Rigor to Ecological Network Models by Evaluating a Set of Pre-balance Diagnostics: A Plea for PREBAL. Ecol. Model. 221:1582-1593. <a href=\"http:\/\/dx.doi.org\/10.1016\/j.ecolmodel.2010.03.012\">10.1016\/j.ecolmodel.2010.03.012<\/a>,[\/footnote], estimating primary production anomalies[footnote]e.g., Serpetti N, Baudron AR, Burrows M, Payne BL, Helaou\u00ebt P, Fernandes PG, Heymans J (2017) Impact of ocean warming on sustainable fisheries management informs the Ecosystem Approach to Fisheries. Scientific Reports 7:13438 <a href=\"https:\/\/doi.org\/10.1038\/s41598-017-13220-7\">https:\/\/doi.org\/10.1038\/s41598-017-13220-7<\/a>[\/footnote] and incorporating environmental drivers.[footnote]e.g., Mackinson S. 2014. Combined analyses reveal environmentally driven changes in the North Sea ecosystem and raise questions regarding what makes an ecosystem model\u2019s performance credible? CJFAS. <a href=\"https:\/\/doi.org\/10.1139\/cjfas-2013-0173\">https:\/\/doi.org\/10.1139\/cjfas-2013-0173<\/a>)[\/footnote]\u00a0Part of this process should include a critical evaluation of calibration time series, as these, and their associated uncertainty, drive the statistical estimation of vulnerability multipliers. Using data with inherent inconsistencies will lead to variable and potentially biased estimates. Equally important is the lack of missing reference time series. Time series produce constraints, and when estimating vulnerability multipliers for groups without time series the lack of constraints allows the fitting procedure to explore a broad parameter space to let such groups indirectly impact other groups with time series. \u00a0All of the above, can impact model derived management advice.<\/p>\r\n<p style=\"font-weight: 400\">As shown in the case studies in the previous chapters (<a href=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/chapter\/cast-study-fitting-impact-on-vulnerability-multipliers\/\">#1<\/a>, <a href=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/chapter\/case-study-2-how-fitting-impacts-advice\/\">#2<\/a>), <em>F<sub>MSY<\/sub><\/em> estimates can change in response to estimated vulnerability multipliers and their impacts on predator consumption rates, albeit with most changes being relatively conservative[footnote]See Supplemental Figure 3 in Bentley et al. 2024[\/footnote]. With high vulnerability multiplier values, species are more sensitive to changes in <em>F<\/em> and are therefore capable of recovering faster in the absence of fishing pressure, meaning maximum sustainable yields are achieved at lower fishing pressures (Figure 1a).\u00a0\u00a0 As predator consumption rates increase with higher vulnerability multiplier values, prey experience higher predation rates, reducing the yield that can be obtained by fishing at <em>F<sub>MSY <\/sub><\/em>(Figure 1b). Unreliable vulnerability multipliers are not easily apparent when reviewing model hindcast simulations against calibration time series data. Comparing Ecosim <em>F<sub>MSY<\/sub><\/em> to other estimates, or proxies (e.g., natural mortality), is one approach to assess vulnerability multipliers and has been demonstrated for the ICES key-run models of the North Sea[footnote]ICES. 2016. Report of the Working Group on Multispecies Assessment Methods (WGSAM), 9\u201313 November 2015, Woods Hole, USA. <a href=\"https:\/\/ices-library.figshare.com\/articles\/report\/Report_of_the_Working_Group_on_Multispecies_Assessment_Methods_WGSAM_\/18614894\">ICES<\/a> CM 2015\/SSGEPI:20. 206pp.[\/footnote] and Irish Sea[footnote]ICES. 2019b. Working group on multispecies assessment methods (WGSAM). ICES Scientific Reports. 1:320. Doi: 10.17895\/ices.pub.5758[\/footnote]. Simulating models beyond observations, under alternate fishing or environmental scenarios, can also highlight issues with vulnerability multipliers by exploring group sensitivities and whether simulated responses to change falls outside of what might be considered ecologically reasonable. Future developments should also consider dependencies between vulnerability multipliers, whether correlation exists between vulnerability multipliers, and how this may impact the ability of a search routine to find stable solutions.<\/p>\r\n\r\n<div style=\"font-weight: 400\">\r\n<div>\r\n<div><\/div>\r\n<\/div>\r\n<div><img class=\"alignnone size-full wp-image-2041\" src=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/Bentley-et-al.-2024.-Figure-9.png\" alt=\"\" width=\"918\" height=\"452\" \/><\/div>\r\n<div>\r\n\r\n<strong>Figure 1. Effects of vulnerability multipliers on derived sustainable fishing advice. Estimations of fishing mortality at which MSY is achieved (<em>F<sub>MSY<\/sub><\/em>). Based on Walters et al., 2005.<\/strong>[footnote]Walters CJ, Christensen V, Martell SJ, Kitchell JF. 2005. Possible ecosystem impacts of applying MSY policies from single-species assessment. ICES Journal of Marine Science 62:558 - 568. <a href=\"https:\/\/doi.org\/10.1016\/j.icesjms.2004.12.005\">https:\/\/doi.org\/10.1016\/j.icesjms.2004.12.005<\/a>[\/footnote]\r\n<div style=\"font-weight: 400\">\r\n<div>\r\n<div><\/div>\r\n<\/div>\r\n<div>\r\n<p style=\"font-weight: 400\">For EwE models to be of operational use, it should be possible to explain why estimated vulnerability multipliers are realistic. This could be based on knowledge of species' ecology, carrying capacity, or natural mortality. We envisage that the future development of Ecosim will encourage users to think more critically when calibrating models by building options to restrict the statistical vulnerability optimisation routine. The objective of this would be to enable users to constrain vulnerability multiplier estimation using <em>a priori <\/em>knowledge where, importantly, data is available to justify doing so. Increased control over the search for vulnerability multipliers could be used to set upper and lower parameter limits, or limits determined by carrying capacity, and penalise parameter combinations which operate outside of predefined limits.<\/p>\r\n<p style=\"font-weight: 400\">Such constraints would also have important implications for Ecospace: the spatial-temporal component of EwE. High vulnerability multipliers, and the large increases in predation mortality which they enable, can have disproportionately large impacts in Ecospace when prey are restricted to small areas (as predators are able to deplete them rapidly). Vulnerability multipliers in Ecospace require further consideration given how spatial heterogeneity may impact species physiology, habitat carrying capacity, and predator-prey interaction rates. Spatial considerations are implicit within the vulnerability concept and enable spatial considerations to be integrated indirectly into Ecosim. The necessity for vulnerability multipliers, or at least those in Ecosim which go some way to indirectly recognising spatial heterogeneity, may be negated by the explicit consideration of spatial heterogeneity in Ecospace. Alternate vulnerability multiplier combinations may be needed depending on the priority use of Ecosim or Ecospace and the different mechanistic role vulnerability multipliers may play across the two components.<\/p>\r\n\r\n<h2>Recommendations<\/h2>\r\n<p style=\"font-weight: 400\">Calibration methods for EwE are not prescriptive. Any one of the methods included in Figure 2 of the <a href=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/chapter\/vulnerability-and-vulnerability-multipliers\/\">Vulnerability and vulnerability multipliers<\/a> chapter, or new methods, may be suitable for use if they can be justified. That said, the <a href=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/chapter\/cast-study-fitting-impact-on-vulnerability-multipliers\/\">first case study<\/a> showed that predator vulnerability multipliers are more likely to re-emerge than predator-prey vulnerability multipliers, and that re-emergence is impacted by data quality. Below we provide best practice recommendations to evaluate the appropriateness of vulnerability multipliers and their impact on model uncertainty:<\/p>\r\n\r\n<ul>\r\n \t<li><strong>Recommendation <\/strong><strong>1: Limit the number of vulnerability multipliers to be estimated. <\/strong>The most efficient way to limit the number of parameters is to estimate by predator, add individual predator-prey combinations if you have specific arguments for why this is necessary. Avoid estimating vulnerabilities for groups without time series as the lack of constraints can lead to unrealistic estimates.<\/li>\r\n \t<li><strong>Recommendation 2: Explain vulnerability multipliers. <\/strong>Provide justifications for setting initial vulnerability multipliers (or keeping the default). If estimating vulnerability multipliers using a statistical routine, check if they make sense relative to the exploitation and ecology of the predator and the ecology of the predator-prey interaction.<\/li>\r\n \t<li><strong>Recommendation 3: Sense check carrying capacities.<\/strong> Vulnerability multipliers augment the upper limit for predator consumption rates, which dictates how predators respond to changes in mortality rates (e.g., release from fishing pressure or predation) or in prey biomass. It is important to review how predator biomass responds to such changes and critically evaluate whether the changes are plausible and whether the limits of estimates should be constrained (i.e., setting upper and lower limits).<\/li>\r\n \t<li><strong>Recommendation 4: Look beyond goodness of fit when evaluating model performance. <\/strong>Combinations of vulnerability multipliers that achieve the best statistical fit (i.e., <em>SS<\/em> and <em>AIC<sub>c<\/sub><\/em>) do not necessarily produce the \"best\" model, if other model outputs, such as indicators, <em>F<sub>MSY<\/sub><\/em> reference points, and forward projections, are unlikely. Assessment of wider model performance should be undertaken to review vulnerability multipliers.<\/li>\r\n \t<li><strong>Recommendation 5: Perform vulnerability multiplier sensitivity analyses.<\/strong> It is best practice to acknowledge and communicate model uncertainty. Calibrating Ecosim models, and thereby choosing one of multiple approaches to estimate vulnerability multipliers, introduces additional uncertainty into the process. Exploring model performance under alternate calibration approaches tests the sensitivity of model outputs to changes in vulnerability multipliers and identifies which vulnerability multipliers consistently emerge.<\/li>\r\n<\/ul>\r\n\"\r\n\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox shaded\"><strong>Attribution <\/strong>This chapter is based on Bentley JW, Chagaris D, Coll M, Heymans JJ, Serpetti N, Walters CJ and Christensen V. 2024. Calibrating ecosystem models to support marine Ecosystem-based Management. ICES Journal of Marine Science, <a href=\"https:\/\/doi.org\/10.1093\/icesjms\/fsad213\">https:\/\/doi.org\/10.1093\/icesjms\/fsad213<\/a>. Adapted based on CC BY License. Rather than citing this chapter, please cite the source.<\/div>","rendered":"<div>\n<h2><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-2663\" src=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/JB_website-image.jpg\" alt=\"\" width=\"1650\" height=\"1275\" srcset=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/JB_website-image.jpg 1650w, https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/JB_website-image-300x232.jpg 300w, https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/JB_website-image-1024x791.jpg 1024w, https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/JB_website-image-768x593.jpg 768w, https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/JB_website-image-1536x1187.jpg 1536w, https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/JB_website-image-65x50.jpg 65w, https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/JB_website-image-225x174.jpg 225w, https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/JB_website-image-350x270.jpg 350w\" sizes=\"auto, (max-width: 1650px) 100vw, 1650px\" \/><\/h2>\n<h2>Limitations and future development<\/h2>\n<p style=\"font-weight: 400\">It is not enough to estimate vulnerability multipliers and assume those which produce the best statistical hindcast fit are appropriate. Ecological reasoning and hypothesis testing must support statistical inference as it should when balancing Ecopath models<a class=\"footnote\" title=\"Link, J.S. 2010. Adding Rigor to Ecological Network Models by Evaluating a Set of Pre-balance Diagnostics: A Plea for PREBAL. Ecol. Model. 221:1582-1593. 10.1016\/j.ecolmodel.2010.03.012,\" id=\"return-footnote-2039-1\" href=\"#footnote-2039-1\" aria-label=\"Footnote 1\"><sup class=\"footnote\">[1]<\/sup><\/a>, estimating primary production anomalies<a class=\"footnote\" title=\"e.g., Serpetti N, Baudron AR, Burrows M, Payne BL, Helaou\u00ebt P, Fernandes PG, Heymans J (2017) Impact of ocean warming on sustainable fisheries management informs the Ecosystem Approach to Fisheries. Scientific Reports 7:13438 https:\/\/doi.org\/10.1038\/s41598-017-13220-7\" id=\"return-footnote-2039-2\" href=\"#footnote-2039-2\" aria-label=\"Footnote 2\"><sup class=\"footnote\">[2]<\/sup><\/a> and incorporating environmental drivers.<a class=\"footnote\" title=\"e.g., Mackinson S. 2014. Combined analyses reveal environmentally driven changes in the North Sea ecosystem and raise questions regarding what makes an ecosystem model\u2019s performance credible? CJFAS. https:\/\/doi.org\/10.1139\/cjfas-2013-0173)\" id=\"return-footnote-2039-3\" href=\"#footnote-2039-3\" aria-label=\"Footnote 3\"><sup class=\"footnote\">[3]<\/sup><\/a>\u00a0Part of this process should include a critical evaluation of calibration time series, as these, and their associated uncertainty, drive the statistical estimation of vulnerability multipliers. Using data with inherent inconsistencies will lead to variable and potentially biased estimates. Equally important is the lack of missing reference time series. Time series produce constraints, and when estimating vulnerability multipliers for groups without time series the lack of constraints allows the fitting procedure to explore a broad parameter space to let such groups indirectly impact other groups with time series. \u00a0All of the above, can impact model derived management advice.<\/p>\n<p style=\"font-weight: 400\">As shown in the case studies in the previous chapters (<a href=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/chapter\/cast-study-fitting-impact-on-vulnerability-multipliers\/\">#1<\/a>, <a href=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/chapter\/case-study-2-how-fitting-impacts-advice\/\">#2<\/a>), <em>F<sub>MSY<\/sub><\/em> estimates can change in response to estimated vulnerability multipliers and their impacts on predator consumption rates, albeit with most changes being relatively conservative<a class=\"footnote\" title=\"See Supplemental Figure 3 in Bentley et al. 2024\" id=\"return-footnote-2039-4\" href=\"#footnote-2039-4\" aria-label=\"Footnote 4\"><sup class=\"footnote\">[4]<\/sup><\/a>. With high vulnerability multiplier values, species are more sensitive to changes in <em>F<\/em> and are therefore capable of recovering faster in the absence of fishing pressure, meaning maximum sustainable yields are achieved at lower fishing pressures (Figure 1a).\u00a0\u00a0 As predator consumption rates increase with higher vulnerability multiplier values, prey experience higher predation rates, reducing the yield that can be obtained by fishing at <em>F<sub>MSY <\/sub><\/em>(Figure 1b). Unreliable vulnerability multipliers are not easily apparent when reviewing model hindcast simulations against calibration time series data. Comparing Ecosim <em>F<sub>MSY<\/sub><\/em> to other estimates, or proxies (e.g., natural mortality), is one approach to assess vulnerability multipliers and has been demonstrated for the ICES key-run models of the North Sea<a class=\"footnote\" title=\"ICES. 2016. Report of the Working Group on Multispecies Assessment Methods (WGSAM), 9\u201313 November 2015, Woods Hole, USA. ICES CM 2015\/SSGEPI:20. 206pp.\" id=\"return-footnote-2039-5\" href=\"#footnote-2039-5\" aria-label=\"Footnote 5\"><sup class=\"footnote\">[5]<\/sup><\/a> and Irish Sea<a class=\"footnote\" title=\"ICES. 2019b. Working group on multispecies assessment methods (WGSAM). ICES Scientific Reports. 1:320. Doi: 10.17895\/ices.pub.5758\" id=\"return-footnote-2039-6\" href=\"#footnote-2039-6\" aria-label=\"Footnote 6\"><sup class=\"footnote\">[6]<\/sup><\/a>. Simulating models beyond observations, under alternate fishing or environmental scenarios, can also highlight issues with vulnerability multipliers by exploring group sensitivities and whether simulated responses to change falls outside of what might be considered ecologically reasonable. Future developments should also consider dependencies between vulnerability multipliers, whether correlation exists between vulnerability multipliers, and how this may impact the ability of a search routine to find stable solutions.<\/p>\n<div style=\"font-weight: 400\">\n<div>\n<div><\/div>\n<\/div>\n<div><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-2041\" src=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/Bentley-et-al.-2024.-Figure-9.png\" alt=\"\" width=\"918\" height=\"452\" srcset=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/Bentley-et-al.-2024.-Figure-9.png 918w, https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/Bentley-et-al.-2024.-Figure-9-300x148.png 300w, https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/Bentley-et-al.-2024.-Figure-9-768x378.png 768w, https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/Bentley-et-al.-2024.-Figure-9-65x32.png 65w, https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/Bentley-et-al.-2024.-Figure-9-225x111.png 225w, https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-content\/uploads\/sites\/1902\/2023\/11\/Bentley-et-al.-2024.-Figure-9-350x172.png 350w\" sizes=\"auto, (max-width: 918px) 100vw, 918px\" \/><\/div>\n<div>\n<p><strong>Figure 1. Effects of vulnerability multipliers on derived sustainable fishing advice. Estimations of fishing mortality at which MSY is achieved (<em>F<sub>MSY<\/sub><\/em>). Based on Walters et al., 2005.<\/strong><a class=\"footnote\" title=\"Walters CJ, Christensen V, Martell SJ, Kitchell JF. 2005. Possible ecosystem impacts of applying MSY policies from single-species assessment. ICES Journal of Marine Science 62:558 - 568. https:\/\/doi.org\/10.1016\/j.icesjms.2004.12.005\" id=\"return-footnote-2039-7\" href=\"#footnote-2039-7\" aria-label=\"Footnote 7\"><sup class=\"footnote\">[7]<\/sup><\/a><\/p>\n<div style=\"font-weight: 400\">\n<div>\n<div><\/div>\n<\/div>\n<div>\n<p style=\"font-weight: 400\">For EwE models to be of operational use, it should be possible to explain why estimated vulnerability multipliers are realistic. This could be based on knowledge of species&#8217; ecology, carrying capacity, or natural mortality. We envisage that the future development of Ecosim will encourage users to think more critically when calibrating models by building options to restrict the statistical vulnerability optimisation routine. The objective of this would be to enable users to constrain vulnerability multiplier estimation using <em>a priori <\/em>knowledge where, importantly, data is available to justify doing so. Increased control over the search for vulnerability multipliers could be used to set upper and lower parameter limits, or limits determined by carrying capacity, and penalise parameter combinations which operate outside of predefined limits.<\/p>\n<p style=\"font-weight: 400\">Such constraints would also have important implications for Ecospace: the spatial-temporal component of EwE. High vulnerability multipliers, and the large increases in predation mortality which they enable, can have disproportionately large impacts in Ecospace when prey are restricted to small areas (as predators are able to deplete them rapidly). Vulnerability multipliers in Ecospace require further consideration given how spatial heterogeneity may impact species physiology, habitat carrying capacity, and predator-prey interaction rates. Spatial considerations are implicit within the vulnerability concept and enable spatial considerations to be integrated indirectly into Ecosim. The necessity for vulnerability multipliers, or at least those in Ecosim which go some way to indirectly recognising spatial heterogeneity, may be negated by the explicit consideration of spatial heterogeneity in Ecospace. Alternate vulnerability multiplier combinations may be needed depending on the priority use of Ecosim or Ecospace and the different mechanistic role vulnerability multipliers may play across the two components.<\/p>\n<h2>Recommendations<\/h2>\n<p style=\"font-weight: 400\">Calibration methods for EwE are not prescriptive. Any one of the methods included in Figure 2 of the <a href=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/chapter\/vulnerability-and-vulnerability-multipliers\/\">Vulnerability and vulnerability multipliers<\/a> chapter, or new methods, may be suitable for use if they can be justified. That said, the <a href=\"https:\/\/pressbooks.bccampus.ca\/ewemodel\/chapter\/cast-study-fitting-impact-on-vulnerability-multipliers\/\">first case study<\/a> showed that predator vulnerability multipliers are more likely to re-emerge than predator-prey vulnerability multipliers, and that re-emergence is impacted by data quality. Below we provide best practice recommendations to evaluate the appropriateness of vulnerability multipliers and their impact on model uncertainty:<\/p>\n<ul>\n<li><strong>Recommendation <\/strong><strong>1: Limit the number of vulnerability multipliers to be estimated. <\/strong>The most efficient way to limit the number of parameters is to estimate by predator, add individual predator-prey combinations if you have specific arguments for why this is necessary. Avoid estimating vulnerabilities for groups without time series as the lack of constraints can lead to unrealistic estimates.<\/li>\n<li><strong>Recommendation 2: Explain vulnerability multipliers. <\/strong>Provide justifications for setting initial vulnerability multipliers (or keeping the default). If estimating vulnerability multipliers using a statistical routine, check if they make sense relative to the exploitation and ecology of the predator and the ecology of the predator-prey interaction.<\/li>\n<li><strong>Recommendation 3: Sense check carrying capacities.<\/strong> Vulnerability multipliers augment the upper limit for predator consumption rates, which dictates how predators respond to changes in mortality rates (e.g., release from fishing pressure or predation) or in prey biomass. It is important to review how predator biomass responds to such changes and critically evaluate whether the changes are plausible and whether the limits of estimates should be constrained (i.e., setting upper and lower limits).<\/li>\n<li><strong>Recommendation 4: Look beyond goodness of fit when evaluating model performance. <\/strong>Combinations of vulnerability multipliers that achieve the best statistical fit (i.e., <em>SS<\/em> and <em>AIC<sub>c<\/sub><\/em>) do not necessarily produce the &#8220;best&#8221; model, if other model outputs, such as indicators, <em>F<sub>MSY<\/sub><\/em> reference points, and forward projections, are unlikely. Assessment of wider model performance should be undertaken to review vulnerability multipliers.<\/li>\n<li><strong>Recommendation 5: Perform vulnerability multiplier sensitivity analyses.<\/strong> It is best practice to acknowledge and communicate model uncertainty. Calibrating Ecosim models, and thereby choosing one of multiple approaches to estimate vulnerability multipliers, introduces additional uncertainty into the process. Exploring model performance under alternate calibration approaches tests the sensitivity of model outputs to changes in vulnerability multipliers and identifies which vulnerability multipliers consistently emerge.<\/li>\n<\/ul>\n<p>&#8221;<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"textbox shaded\"><strong>Attribution <\/strong>This chapter is based on Bentley JW, Chagaris D, Coll M, Heymans JJ, Serpetti N, Walters CJ and Christensen V. 2024. Calibrating ecosystem models to support marine Ecosystem-based Management. ICES Journal of Marine Science, <a href=\"https:\/\/doi.org\/10.1093\/icesjms\/fsad213\">https:\/\/doi.org\/10.1093\/icesjms\/fsad213<\/a>. Adapted based on CC BY License. Rather than citing this chapter, please cite the source.<\/div>\n<div class=\"media-attributions clear\" prefix:cc=\"http:\/\/creativecommons.org\/ns#\" prefix:dc=\"http:\/\/purl.org\/dc\/terms\/\"><h2>Media Attributions<\/h2><ul><li >Calibrating Ecosystem Models figure by Jacob Bentley       <\/li><li >From Bentley et al. 2024. Figure 9       <\/li><\/ul><\/div><hr class=\"before-footnotes clear\" \/><div class=\"footnotes\"><ol><li id=\"footnote-2039-1\">Link, J.S. 2010. Adding Rigor to Ecological Network Models by Evaluating a Set of Pre-balance Diagnostics: A Plea for PREBAL. Ecol. Model. 221:1582-1593. <a href=\"http:\/\/dx.doi.org\/10.1016\/j.ecolmodel.2010.03.012\">10.1016\/j.ecolmodel.2010.03.012<\/a>, <a href=\"#return-footnote-2039-1\" class=\"return-footnote\" aria-label=\"Return to footnote 1\">&crarr;<\/a><\/li><li id=\"footnote-2039-2\">e.g., Serpetti N, Baudron AR, Burrows M, Payne BL, Helaou\u00ebt P, Fernandes PG, Heymans J (2017) Impact of ocean warming on sustainable fisheries management informs the Ecosystem Approach to Fisheries. Scientific Reports 7:13438 <a href=\"https:\/\/doi.org\/10.1038\/s41598-017-13220-7\">https:\/\/doi.org\/10.1038\/s41598-017-13220-7<\/a> <a href=\"#return-footnote-2039-2\" class=\"return-footnote\" aria-label=\"Return to footnote 2\">&crarr;<\/a><\/li><li id=\"footnote-2039-3\">e.g., Mackinson S. 2014. Combined analyses reveal environmentally driven changes in the North Sea ecosystem and raise questions regarding what makes an ecosystem model\u2019s performance credible? CJFAS. <a href=\"https:\/\/doi.org\/10.1139\/cjfas-2013-0173\">https:\/\/doi.org\/10.1139\/cjfas-2013-0173<\/a>) <a href=\"#return-footnote-2039-3\" class=\"return-footnote\" aria-label=\"Return to footnote 3\">&crarr;<\/a><\/li><li id=\"footnote-2039-4\">See Supplemental Figure 3 in Bentley et al. 2024 <a href=\"#return-footnote-2039-4\" class=\"return-footnote\" aria-label=\"Return to footnote 4\">&crarr;<\/a><\/li><li id=\"footnote-2039-5\">ICES. 2016. Report of the Working Group on Multispecies Assessment Methods (WGSAM), 9\u201313 November 2015, Woods Hole, USA. <a href=\"https:\/\/ices-library.figshare.com\/articles\/report\/Report_of_the_Working_Group_on_Multispecies_Assessment_Methods_WGSAM_\/18614894\">ICES<\/a> CM 2015\/SSGEPI:20. 206pp. <a href=\"#return-footnote-2039-5\" class=\"return-footnote\" aria-label=\"Return to footnote 5\">&crarr;<\/a><\/li><li id=\"footnote-2039-6\">ICES. 2019b. Working group on multispecies assessment methods (WGSAM). ICES Scientific Reports. 1:320. Doi: 10.17895\/ices.pub.5758 <a href=\"#return-footnote-2039-6\" class=\"return-footnote\" aria-label=\"Return to footnote 6\">&crarr;<\/a><\/li><li id=\"footnote-2039-7\">Walters CJ, Christensen V, Martell SJ, Kitchell JF. 2005. Possible ecosystem impacts of applying MSY policies from single-species assessment. ICES Journal of Marine Science 62:558 - 568. <a href=\"https:\/\/doi.org\/10.1016\/j.icesjms.2004.12.005\">https:\/\/doi.org\/10.1016\/j.icesjms.2004.12.005<\/a> <a href=\"#return-footnote-2039-7\" class=\"return-footnote\" aria-label=\"Return to footnote 7\">&crarr;<\/a><\/li><\/ol><\/div>","protected":false},"author":1909,"menu_order":7,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":["jacob-bentley","chagaris","martacoll","sheila","natalia-serpetti","carl-j-walters-e0zd3ow3zk","villy"],"pb_section_license":""},"chapter-type":[],"contributor":[64,79,76,68,78,74,60],"license":[],"class_list":["post-2039","chapter","type-chapter","status-publish","hentry","contributor-carl-j-walters-e0zd3ow3zk","contributor-chagaris","contributor-jacob-bentley","contributor-martacoll","contributor-natalia-serpetti","contributor-sheila","contributor-villy"],"part":1094,"_links":{"self":[{"href":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-json\/pressbooks\/v2\/chapters\/2039","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-json\/wp\/v2\/users\/1909"}],"version-history":[{"count":11,"href":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-json\/pressbooks\/v2\/chapters\/2039\/revisions"}],"predecessor-version":[{"id":4047,"href":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-json\/pressbooks\/v2\/chapters\/2039\/revisions\/4047"}],"part":[{"href":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-json\/pressbooks\/v2\/parts\/1094"}],"metadata":[{"href":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-json\/pressbooks\/v2\/chapters\/2039\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-json\/wp\/v2\/media?parent=2039"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-json\/pressbooks\/v2\/chapter-type?post=2039"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-json\/wp\/v2\/contributor?post=2039"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/wp-json\/wp\/v2\/license?post=2039"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}