{"id":2027,"date":"2023-11-26T19:03:22","date_gmt":"2023-11-27T00:03:22","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/?post_type=chapter&p=2027"},"modified":"2025-10-30T08:57:20","modified_gmt":"2025-10-30T12:57:20","slug":"case-study-2-how-fitting-impacts-advice","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/chapter\/case-study-2-how-fitting-impacts-advice\/","title":{"raw":"Case study: How fitting impacts advice","rendered":"Case study: How fitting impacts advice"},"content":{"raw":"

The Irish Sea EwE model[footnote]Bentley, J.W., Serpetti, N., Fox, C.J., Heymans, J.J. and Reid, D.G., 2020. Retrospective analysis of the influence of environmental drivers on commercial stocks and fishing opportunities in the Irish Sea. Fisheries Oceanography, 29(5), pp.415-435. https:\/\/doi.org\/10.1111\/fog.12486<\/a>[\/footnote] and Northwest Atlantic Continental Shelf EwE model[footnote]Chagaris, D., Drew, K., Schueller, A., Cieri, M., Brito, J. and Buchheister, A., 2020. Ecological reference points for Atlantic menhaden established using an ecosystem model of intermediate complexity. Frontiers in Marine Science, 7, p.606417. https:\/\/doi.org\/10.3389\/fmars.2020.606417<\/a>[\/footnote] (hereafter called NWACS-MICE) have both been used to inform fisheries advice for their respective regions using ecological\/ecosystem reference points[footnote]Howell, D., Schueller, A.M., Bentley, J.W., Buchheister, A., Chagaris, D., Cieri, M., Drew, K., Lundy, M.G., Pedreschi, D., Reid, D.G. and Townsend, H., 2021. Combining ecosystem and single-species modelling to provide ecosystem-based fisheries management advice within current management systems. Frontiers in Marine Science, 7, p.607831. https:\/\/doi.org\/10.3389\/fmars.2020.607831<\/a>[\/footnote]. Both models were designed to focus on commercial fisheries however, they have very different structures in terms of model complexity (Table 1). The two models were used to demonstrate the outcomes and management implications of vulnerability multiplier (kj<\/sub><\/em>\u00a0<\/em>or kij<\/sub><\/em>) estimation and compared estimates of FMSY<\/sub><\/em> and ecosystem indicators. Ecosystem indicators selected for this analysis included total system biomass, commercial biomass, total catch, system diversity (Kempton\u2019s Q<\/em>), the trophic level of the catch, and the trophic level of the community. Estimates of FMSY<\/sub><\/em> and ecosystem indicators were compared across the following nine fitting approaches:<\/p>\r\n\r\n

    \r\n \t
  1. Predator kj<\/sub><\/em>\u00a0values, where the number of parameters estimated is one less than the number of available calibration time series (K-1).<\/em><\/li>\r\n \t
  2. Predator kj<\/sub><\/em>\u00a0<\/em>values estimated for all functional groups with time series.<\/li>\r\n \t
  3. Predator kj<\/sub><\/em>\u00a0<\/em>values using the automated stepwise fitting approach, where the applied kj<\/sub><\/em> values are taken from the model with the lowest AICc<\/sub><\/em>, and the vulnerabilities are reset to the default (2) at each fitting iteration.<\/li>\r\n \t
  4. Predator kj<\/sub><\/em>\u00a0<\/em>values using the automated stepwise fitting approach, where the applied kj<\/sub><\/em>\u00a0values are taken from the model with the lowest AICc<\/sub><\/em>, and the vulnerabilities are retained from previous fitting iterations.<\/li>\r\n \t
  5. Predator-prey kij<\/sub><\/em>\u00a0values, where the number of parameters estimated is one less than the number of available calibration time series (K-1).<\/em><\/li>\r\n \t
  6. Predator-prey kij<\/sub><\/em>\u00a0<\/em>values using the automated stepwise fitting approach, where the applied \u00a0values are taken from the model with the lowest AICc<\/sub><\/em>, and the vulnerabilities are reset to the default (2) at each fitting iteration.<\/li>\r\n \t
  7. Predator-prey kij<\/sub><\/em>\u00a0<\/em>values using the automated stepwise fitting approach, where the applied \u00a0values are taken from the model with the lowest AICc<\/sub><\/em>, and the vulnerabilities are retained from previous fitting iterations.<\/li>\r\n \t
  8. Predator-prey kij<\/sub><\/em>\u00a0values using a repeated manual stepwise fitting approach, where the estimated kij<\/sub><\/em>\u00a0(K-1)<\/em> are retained from one iteration to the next (with a total of 5 iterations) and the final configuration is that with the lowest AICc<\/sub><\/em>, as was done in Chagaris et al.,[footnote]Chagaris et al. 2020. op. cit<\/em>.[\/footnote].<\/li>\r\n \t
  9. A combination of predator kj<\/sub><\/em>\u00a0and predator-prey kij<\/sub><\/em>\u00a0<\/em>values using the methods outlined by Bentley et al., (2020). Predator kj<\/sub><\/em>\u00a0values were estimated using the automated stepwise fitting approach in #3. Predator-prey kij<\/sub><\/em>\u00a0values were estimated using a manual stepwise fitting approach and the remaining degrees of freedom. The number of additional predator-prey kij<\/sub><\/em>\u2019s was determined by their AICc <\/sub><\/em>score (note this approach was only carried out for the Irish Sea model\u00a0 in this study).<\/li>\r\n<\/ol>\r\n\"\"\r\n
    \r\n
    \r\n
    \r\n

    Alternate fitting approaches led to the emergence of different vulnerability multipliers in the corresponding models of best fit (as determined by sum of squared deviations and AICc<\/sub><\/em>) for the Irish Sea (Figure 1) and NWACS-MICE (Figure 2). Different fitting approaches impacted estimates of FMSY<\/sub><\/em> in both models due to changes in species sensitivity to F with alternate vulnerability multipliers. Despite the increased complexity of the Irish Sea model, the patterns in FMSY<\/sub><\/em> variability are comparable between models, with certain species having consistent FMSY<\/sub><\/em> estimates across the nine approaches for vulnerability multiplier estimation. This includes cod and whiting for the Irish Sea and striped bass for the NWACS. As demonstrated in case study 1<\/a> (previous chapter), these species are opportunistic feeders which are also predated on by higher trophic levels, giving them a relatively high degree of connectivity within the food web models, which may constrict the emergence of vulnerability multipliers. Additionally, these groups have experienced a period of collapse, and in some cases recovery, which provides contrast for the model to estimate the vulnerability multipliers. Fitting approaches with similar properties resulted in more closely related FMSY<\/sub><\/em> estimates[footnote]See Supplemental figure 3 in Bentley et al. 2024[\/footnote]. For example, FMSY<\/sub><\/em> estimates generated by approaches, which searched for vulnerability multipliers by \"predator\" tended to be more similar to each other when compared to those generated by approaches which searched by \"predator-prey\", and vice versa<\/em>. This emergent trend is perhaps most clearly seen in the FMSY<\/sub><\/em> estimates for menhaden and bluefish adults from the NWACS-MICE model (Figure 2).<\/p>\r\n

    The approach used to estimate vulnerability multipliers had an impact on the derived ecosystem indicators (Figure 1b and Figure 2b). These impacts were relatively small, most deviations being within the range of 5-10% when compared to indicators from the published models. Trophic level indicators were particularly robust across estimation approaches, despite often larger differences being observed in diversity (Kempton\u2019s Q) catch and commercial biomass indicators. Balanced reconfiguration within the ecosystem models (i.e., increases in some species and decreases in others with similar trophic levels) enabled the trophic indicators to remain similar across approaches. However, the dissimilarity in trophic level of the catch in the NWACS-MICE model was generally higher across scenarios where vulnerability multipliers were searched by \"predator\". This reflects the higher FMSY<\/sub><\/em> reference points for adult weakfish and bluefish and lower FMSY reference points for menhaden and herring produced under the same fitting approaches. Overall, the Irish Sea EwE model showed greater dissimilarity in derived indicators than the NWACS-MICE model. This outcome is likely linked to the increased complexity of the Irish Sea model, and how a repeated search provides the opportunity to adjust more predator-prey vulnerability multipliers. This may be less of a concern for low complexity models as the parameter space is smaller, increasing the likelihood that the same vulnerability multipliers will be adjusted.<\/p>\r\n\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n

    \"\"<\/div>\r\n
    \r\n\r\nFigure 1. Irish Sea EwE outputs under alternate fitting approaches. Vulnerability multipliers for\u00a0 the Irish Sea Ecosim model were estimated following seven alternate fitting approaches. The impacts of alternate fitting approaches and vulnerability multiplier estimates are shown for (a) estimates of FMSY<\/sub><\/em> (b) indicators of ecosystem structure and function. The impacts of vulnerability multiplier estimates on indicator simulations are illustrated by comparing new simulations against the simulations from the published model. The published Irish Sea model has vulnerability multiplier values estimated using the predator and predator-prey approach.<\/strong>[footnote]Bentley et al., 2020. op. cit<\/em>.[\/footnote]\r\n
    \r\n
    \r\n
    <\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
    <\/div>\r\n
    \r\n
    \r\n
    \"\"<\/div>\r\n<\/div>\r\n
    \r\n
    \r\n\r\nFigure 2. NWACS-MICE EwE outputs under alternate fitting approaches. Vulnerability\u00a0\u00a0 multipliers for the NWACS-MICE Ecosim model were estimated following seven alternate fitting approaches. The impacts of alternate fitting approaches and vulnerability multiplier estimates are shown for (a) estimates of FMSY<\/sub><\/em> (b) indicators of ecosystem structure and function. The impacts of vulnerability multiplier estimates on indicator simulations are illustrated by comparing new simulations against the simulations from the published model. The published NWACS-MICE model has vulnerability multiplier values estimated using the manual repeated predator-prey vulnerability multiplier search approach.<\/strong>[footnote]Chagaris et al., 2020. op cit.<\/em>[\/footnote]\r\n
    \r\n
    \r\n
    <\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
    Attribution <\/strong>This chapter is based on Bentley JW, Chagaris D, Coll M, Heymans JJ, Serpetti N, Walters CJ and Christensen V. 2024. Calibrating ecosystem models to support marine Ecosystem-based Management. ICES Journal of Marine Science, https:\/\/doi.org\/10.1093\/icesjms\/fsad213<\/a>. Adapted based on CC BY License. Rather than citing this chapter, please cite the source.<\/div>","rendered":"

    The Irish Sea EwE model[1]<\/sup><\/a> and Northwest Atlantic Continental Shelf EwE model[2]<\/sup><\/a> (hereafter called NWACS-MICE) have both been used to inform fisheries advice for their respective regions using ecological\/ecosystem reference points[3]<\/sup><\/a>. Both models were designed to focus on commercial fisheries however, they have very different structures in terms of model complexity (Table 1). The two models were used to demonstrate the outcomes and management implications of vulnerability multiplier (kj<\/sub><\/em>\u00a0<\/em>or kij<\/sub><\/em>) estimation and compared estimates of FMSY<\/sub><\/em> and ecosystem indicators. Ecosystem indicators selected for this analysis included total system biomass, commercial biomass, total catch, system diversity (Kempton\u2019s Q<\/em>), the trophic level of the catch, and the trophic level of the community. Estimates of FMSY<\/sub><\/em> and ecosystem indicators were compared across the following nine fitting approaches:<\/p>\n

      \n
    1. Predator kj<\/sub><\/em>\u00a0values, where the number of parameters estimated is one less than the number of available calibration time series (K-1).<\/em><\/li>\n
    2. Predator kj<\/sub><\/em>\u00a0<\/em>values estimated for all functional groups with time series.<\/li>\n
    3. Predator kj<\/sub><\/em>\u00a0<\/em>values using the automated stepwise fitting approach, where the applied kj<\/sub><\/em> values are taken from the model with the lowest AICc<\/sub><\/em>, and the vulnerabilities are reset to the default (2) at each fitting iteration.<\/li>\n
    4. Predator kj<\/sub><\/em>\u00a0<\/em>values using the automated stepwise fitting approach, where the applied kj<\/sub><\/em>\u00a0values are taken from the model with the lowest AICc<\/sub><\/em>, and the vulnerabilities are retained from previous fitting iterations.<\/li>\n
    5. Predator-prey kij<\/sub><\/em>\u00a0values, where the number of parameters estimated is one less than the number of available calibration time series (K-1).<\/em><\/li>\n
    6. Predator-prey kij<\/sub><\/em>\u00a0<\/em>values using the automated stepwise fitting approach, where the applied \u00a0values are taken from the model with the lowest AICc<\/sub><\/em>, and the vulnerabilities are reset to the default (2) at each fitting iteration.<\/li>\n
    7. Predator-prey kij<\/sub><\/em>\u00a0<\/em>values using the automated stepwise fitting approach, where the applied \u00a0values are taken from the model with the lowest AICc<\/sub><\/em>, and the vulnerabilities are retained from previous fitting iterations.<\/li>\n
    8. Predator-prey kij<\/sub><\/em>\u00a0values using a repeated manual stepwise fitting approach, where the estimated kij<\/sub><\/em>\u00a0(K-1)<\/em> are retained from one iteration to the next (with a total of 5 iterations) and the final configuration is that with the lowest AICc<\/sub><\/em>, as was done in Chagaris et al.,[4]<\/sup><\/a>.<\/li>\n
    9. A combination of predator kj<\/sub><\/em>\u00a0and predator-prey kij<\/sub><\/em>\u00a0<\/em>values using the methods outlined by Bentley et al., (2020). Predator kj<\/sub><\/em>\u00a0values were estimated using the automated stepwise fitting approach in #3. Predator-prey kij<\/sub><\/em>\u00a0values were estimated using a manual stepwise fitting approach and the remaining degrees of freedom. The number of additional predator-prey kij<\/sub><\/em>\u2019s was determined by their AICc <\/sub><\/em>score (note this approach was only carried out for the Irish Sea model\u00a0 in this study).<\/li>\n<\/ol>\n

      \"\"<\/p>\n

      \n
      \n
      \n

      Alternate fitting approaches led to the emergence of different vulnerability multipliers in the corresponding models of best fit (as determined by sum of squared deviations and AICc<\/sub><\/em>) for the Irish Sea (Figure 1) and NWACS-MICE (Figure 2). Different fitting approaches impacted estimates of FMSY<\/sub><\/em> in both models due to changes in species sensitivity to F with alternate vulnerability multipliers. Despite the increased complexity of the Irish Sea model, the patterns in FMSY<\/sub><\/em> variability are comparable between models, with certain species having consistent FMSY<\/sub><\/em> estimates across the nine approaches for vulnerability multiplier estimation. This includes cod and whiting for the Irish Sea and striped bass for the NWACS. As demonstrated in case study 1<\/a> (previous chapter), these species are opportunistic feeders which are also predated on by higher trophic levels, giving them a relatively high degree of connectivity within the food web models, which may constrict the emergence of vulnerability multipliers. Additionally, these groups have experienced a period of collapse, and in some cases recovery, which provides contrast for the model to estimate the vulnerability multipliers. Fitting approaches with similar properties resulted in more closely related FMSY<\/sub><\/em> estimates[5]<\/sup><\/a>. For example, FMSY<\/sub><\/em> estimates generated by approaches, which searched for vulnerability multipliers by “predator” tended to be more similar to each other when compared to those generated by approaches which searched by “predator-prey”, and vice versa<\/em>. This emergent trend is perhaps most clearly seen in the FMSY<\/sub><\/em> estimates for menhaden and bluefish adults from the NWACS-MICE model (Figure 2).<\/p>\n

      The approach used to estimate vulnerability multipliers had an impact on the derived ecosystem indicators (Figure 1b and Figure 2b). These impacts were relatively small, most deviations being within the range of 5-10% when compared to indicators from the published models. Trophic level indicators were particularly robust across estimation approaches, despite often larger differences being observed in diversity (Kempton\u2019s Q) catch and commercial biomass indicators. Balanced reconfiguration within the ecosystem models (i.e., increases in some species and decreases in others with similar trophic levels) enabled the trophic indicators to remain similar across approaches. However, the dissimilarity in trophic level of the catch in the NWACS-MICE model was generally higher across scenarios where vulnerability multipliers were searched by “predator”. This reflects the higher FMSY<\/sub><\/em> reference points for adult weakfish and bluefish and lower FMSY reference points for menhaden and herring produced under the same fitting approaches. Overall, the Irish Sea EwE model showed greater dissimilarity in derived indicators than the NWACS-MICE model. This outcome is likely linked to the increased complexity of the Irish Sea model, and how a repeated search provides the opportunity to adjust more predator-prey vulnerability multipliers. This may be less of a concern for low complexity models as the parameter space is smaller, increasing the likelihood that the same vulnerability multipliers will be adjusted.<\/p>\n<\/div>\n<\/div>\n<\/div>\n

      \"\"<\/div>\n
      \n

      Figure 1. Irish Sea EwE outputs under alternate fitting approaches. Vulnerability multipliers for\u00a0 the Irish Sea Ecosim model were estimated following seven alternate fitting approaches. The impacts of alternate fitting approaches and vulnerability multiplier estimates are shown for (a) estimates of FMSY<\/sub><\/em> (b) indicators of ecosystem structure and function. The impacts of vulnerability multiplier estimates on indicator simulations are illustrated by comparing new simulations against the simulations from the published model. The published Irish Sea model has vulnerability multiplier values estimated using the predator and predator-prey approach.<\/strong>[6]<\/sup><\/a><\/p>\n

      \n
      \n
      <\/div>\n<\/div>\n<\/div>\n<\/div>\n
      <\/div>\n
      \n
      \n
      \"\"<\/div>\n<\/div>\n
      \n
      \n

      Figure 2. NWACS-MICE EwE outputs under alternate fitting approaches. Vulnerability\u00a0\u00a0 multipliers for the NWACS-MICE Ecosim model were estimated following seven alternate fitting approaches. The impacts of alternate fitting approaches and vulnerability multiplier estimates are shown for (a) estimates of FMSY<\/sub><\/em> (b) indicators of ecosystem structure and function. The impacts of vulnerability multiplier estimates on indicator simulations are illustrated by comparing new simulations against the simulations from the published model. The published NWACS-MICE model has vulnerability multiplier values estimated using the manual repeated predator-prey vulnerability multiplier search approach.<\/strong>[7]<\/sup><\/a><\/p>\n

      \n
      \n
      <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      Attribution <\/strong>This chapter is based on Bentley JW, Chagaris D, Coll M, Heymans JJ, Serpetti N, Walters CJ and Christensen V. 2024. Calibrating ecosystem models to support marine Ecosystem-based Management. ICES Journal of Marine Science, https:\/\/doi.org\/10.1093\/icesjms\/fsad213<\/a>. Adapted based on CC BY License. Rather than citing this chapter, please cite the source.<\/div>\n

      Media Attributions<\/h2>