{"id":2012,"date":"2023-11-26T17:45:32","date_gmt":"2023-11-26T22:45:32","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/?post_type=chapter&p=2012"},"modified":"2025-10-30T08:57:17","modified_gmt":"2025-10-30T12:57:17","slug":"statistical-approaches-for-estimating-vulnerability-multipliers","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/ewemodel\/chapter\/statistical-approaches-for-estimating-vulnerability-multipliers\/","title":{"raw":"Statistical approaches for estimating vulnerability multipliers","rendered":"Statistical approaches for estimating vulnerability multipliers"},"content":{"raw":"

It is becoming more common for Ecosim vulnerability multipliers and primary production anomalies to be estimated using statistical routines, with Heymans et al.[footnote]Heymans, J.J., Coll, M., Link, J.S., Mackinson, S., Steenbeek, J., Walters, C. and Christensen, V., 2016. Best practice in Ecopath with Ecosim food-web models for ecosystem-based management. Ecological Modelling, 331, pp.173-184. https:\/\/doi.org\/10.1016\/j.ecolmodel.2015.12.007<\/a>[\/footnote] demonstrating that it is best practice to estimate vulnerability multipliers by fitting model simulations to time series reference data. Longer time series are preferable as they provide an opportunity to explore important drivers of change and tend to have strong contrast in the data, which improves the model\u2019s ability to estimate parameters, leading to more confidence in our assessment of ecosystem dynamics.<\/p>\r\nModel fitting that includes estimation of primary production anomalies is basically the ecosystem equivalent of estimating recruitment and mortality anomalies in state-space approaches to parameter estimation for single-species models, a key difference being that the anomaly estimates may be informed by correlated variation in time series patterns of multiple species.\r\n

The quality (precision, informative contrast over time) of time series data is important, especially if the fitted model is to be used for management purposes, as vulnerability multipliers (and thus predation rate changes), will be used in forward simulations to times beyond the data. In Ecosim, users can weight time series data to represent how reliable or variable time series are compared to the other reference time series. Low weights imply that the data either has high variance or is unreliable (e.g., underestimated or uncertain catches). Weightings impact the contribution of time series to the assessment of model performance, where a weight of 0 indicates that the time series will not be used in the calculation of \"goodness of fit\". Weightings can be assigned based on a qualitative assessment of data pedigree (e.g., based on data origin), or by using more quantitative information, such as confidence intervals from survey estimates, the retrospective analyses of stock assessment models, or signal to noise ratio assessments[footnote]Heymans et al., 2016. op. cit.<\/em>[\/footnote].<\/p>\r\n

The procedure for estimating vulnerability multipliers and production anomalies that improve the fit of model simulation to calibration data is based on minimization of a sum of squares (SS<\/em>) of prediction errors, which is then checked for overparameterization using the Akaike Information Index (AIC)<\/em>[footnote]Akaike, H., 1998. Information theory and an extension of the maximum likelihood principle. In Selected papers of Hirotugu Akaike (pp. 199-213). Springer, New York, NY. https:\/\/doi.org\/10.1007\/978-1-4612-1694-0_15<\/a>[\/footnote] [footnote]Cavanaugh, J.E. and Neath, A.A., 2019. The Akaike information criterion: Background, derivation, properties, application, interpretation, and refinements. Wiley Interdisciplinary Reviews: Computational Statistics, 11(3), p.e1460. https:\/\/doi.org\/10.1002\/wics.1460<\/a>[\/footnote].<\/p>\r\n

The SS<\/em> is used to calculate a log likelihood criterion (Figure 1), assuming normally distributed deviations of log model predictions from log observations, evaluated at the conditional maximum likelihood estimate of the prediction error variance and scaled in the case of relative observations (y)<\/em> by the maximum likelihood estimate of the relative simulation scaling factor (q)<\/em> in the equation y = q x X<\/em>,\u00a0 where X<\/em> is the absolute observation. Model fitting then proceeds by numerical search procedures to seek parameter values that minimize SS.<\/p>\r\n

When generating a set of model fits under different fitting hypotheses or methods for choosing what parameters to include in the SS, AIC<\/em> is then used to identify the model of best fit. AIC<\/em> is a tool for model selection that penalizes for fitting too many parameters relative to the time series available for estimating the SS and is calculated as<\/p>\r\n[latex]AIC = n \\cdot \\ln (\\frac{minSS}{n}) + 2K\\tag{1}[\/latex]\r\n

where n<\/em>\u00a0is the total number of observations, or time series values, from the loaded calibration time series and K<\/em>\u00a0is the number of parameters estimated. When sample size is small, there is a large probability that AIC<\/em> will select models with too many estimated parameters (i.e., overfit models). The modified AICc<\/sub><\/em> can be used to address this potential overfitting by including a correction for small sample sizes<\/p>\r\n[latex]AIC_c = AIC + 2K \\cdot (\\frac{K-1}{n-K-1})\\tag{2}[\/latex]\r\n

As a rule of thumb[footnote]Burnham, K. P., & Anderson, D. R. (2004). Multimodel Inference: Understanding AIC and BIC in Model Selection. Sociological Methods & Research, 33(2), 261\u2013304. https:\/\/doi.org\/10.1177\/0049124104268644<\/a>[\/footnote], AICc<\/sub><\/em> should be used unless n\/K<\/em> > ~40. In other words, unless the number of estimated parameters equates to a minimum 2-3% of the amount of data, use AICc<\/sub><\/em>. AICc<\/sub><\/em> should therefore be used when assessing EwE model performance.<\/p>\r\nA word of caution, the AIC calculations assume that the observations are independent whereas timeseries data such as typically used for ecosystem modelling have high autocorrelation. For this reason, it is advisable to test the impact on assumptions about n<\/em> on model selection.\r\n\r\n\"\"\r\n

\r\n
\r\n
\r\n
Figure 1.\u00a0<\/span> Overview of the Ecopath and Ecosim modelling process. Using log likelihood criteria, vulnerability multipliers or production anomalies (e.g., climate or nutrient loading) may be estimated based on a non-linear search routine and vulnerability multiplier (vulmult) estimation. Prediction (fitting) failures after each estimation trial then inform judgmental changes in model structure and parameters. B is biomass, Z<\/em> is total mortality, C<\/em> is catch, W<\/em> is average weight. Subscript 0 refers to the Ecopath model base year, and CC<\/em> to carrying capacity. Bcc<\/sub>\/B0<\/sub> refers to vulmult.\u00a0From Christensen and Walters<\/span><\/strong>[footnote]Christensen and Walters. 2011. Op. cit.[\/footnote]<\/span>.<\/span><\/strong><\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n

Multiple approaches have been developed to statistically estimate vulnerability multipliers (see Figure 2 in Vulnerability and vulnerability multipliers<\/a> chapter). They can be estimated for predators, providing a single multiplier limit to all of a given predator\u2019s base predation rates, and they can be estimated for individual predator-prey relationships, which assumes that the multiplier limits are heterogeneous across prey. This choice tends to be associated with user preference, ecological justification, or determined based on the approach that produces the best fit model. Whether estimating predator or predator-prey vulnerability multipliers, there are a few ways to select which vulnerability multipliers should be estimated:<\/p>\r\n\r\n

    \r\n \t
  1. manual selection based on a priori knowledge or species priority;<\/li>\r\n \t
  2. select vulnerability multipliers for groups with calibration time series; or<\/li>\r\n \t
  3. select the most sensitive vulnerability multipliers (i.e., those that when changed have the largest impact on SS<\/em>).<\/li>\r\n<\/ol>\r\nManually selecting vulnerability multipliers allows for an early integration of ecological information but may lead to a sub-optimal model fit if the SS<\/em> is not sensitive to the selected multipliers. Conversely, the sensitivity search may optimise model fit but it is purely statistical and does not know what makes sense ecologically. Only estimating vulnerability multipliers for groups with time series acknowledges that, to some degree, the parameter should be constrained by the available time series. The level of group connectedness within the food web (e.g., group consumption and predation) may also constrain the parameter search if changes in vulnerability multipliers impact the \u00a0contribution of other groups.\u00a0 Groups that do not have informative time series or, have low connectedness in the food web, have widely variable estimated vulnerability multipliers \u2013 search routines can change those without any penalty incurred.\r\n

    Choosing how many vulnerability multipliers to estimate, without overfitting is another point of confusion and discussion. The number of vulnerability multipliers that can be potentially estimated is often significantly more than the data available to constrain simulations. EwE best practices suggest that a conservative number of Degrees of Freedom (DoF) and therefore parameters to estimate is one less than the number of calibration time series available[footnote]Heymans et al., 2016. op. cit<\/em>.[\/footnote]. This approach recognizes that values within time series are highly autocorrelated, viewing each time series as an \"independent observation\", but\u00a0 the \u00a0approach could be overly conservative, especially if long time series are available\u2013 especially if the contrast (ups and downs) and are not just one-way trajectories.<\/p>\r\n

    Both manual and automated statistical calibration routines are available in Ecosim to search for vulnerability multipliers. The manual approach can be arduous when testing multiple fitting hypotheses (e.g., with or without fishing effort or primary production anomalies) as the number of plausible fitting combinations can easily reach the hundreds, if not thousands, increasing the likelihood of user error. In the past, users have overcome this issue by only testing the nth fitting scenario (e.g., 5, 10, 15 vulnerability multipliers etc.)[footnote]e.g., Alexander, K.A., Heymans, J.J., Magill, S., Tomczak, M.T., Holmes, S.J. and Wilding, T.A., 2015. Investigating the recent decline in gadoid stocks in the west of Scotland shelf ecosystem using a foodweb model. ICES Journal of Marine Science, 72(2), pp.436-449. https:\/\/doi.org\/10.1093\/icesjms\/fsu149<\/a>[\/footnote] However this approach risks overlooking the vulnerability multiplier combination, which produces the best statistical fit. The stepwise fitting procedure developed by Scott et al.[footnote]Scott, E., Serpetti, N., Steenbeek, J. and Heymans, J.J., 2016. A Stepwise Fitting Procedure for automated fitting of Ecopath with Ecosim models. SoftwareX, 5, pp.25-30. https:\/\/doi.org\/10.1016\/j.softx.2016.02.002<\/a>[\/footnote] automates this process, allowing for a broad exploration of the parameter space which accelerates the process and removes the problem of user error. Recent improvements to the automated approach have increased the computational speed by enabling multiple fitting scenarios to be tested simultaneously using computers multithreading capabilities (J. Steenbeek, pers. comm.).<\/p>\r\n

    Novel approaches to estimate vulnerability multipliers using the manual and automated fitting routines have also been developed for two EwE models which are being used operationally to inform fisheries catch advice. Bentley et al.,[footnote]Bentley, J.W., Serpetti, N., Fox, C.J., Heymans, J.J. and Reid, D.G., 2020. Retrospective analysis of the influence of environmental drivers on commercial stocks and fishing opportunities in the Irish Sea. Fisheries Oceanography, 29(5), pp.415-435. https:\/\/doi.org\/10.1111\/fog.12486<\/a>[\/footnote] employed an approach, which combined searches for predator vulnerability multipliers and predator-prey vulnerability multipliers, whereas the approach developed by Chagaris et al.[footnote]Chagaris, D., Drew, K., Schueller, A., Cieri, M., Brito, J. and Buchheister, A., 2020. Ecological reference points for Atlantic menhaden established using an ecosystem model of intermediate complexity. Frontiers in Marine Science, 7, p.606417. https:\/\/doi.org\/10.3389\/fmars.2020.606417<\/a>[\/footnote] uses the manual fitting tool in Ecosim to iteratively estimate the \u00a0most sensitive predator-prey vulnerability multipliers over multiple sequential (repeated) tuning iterations.[footnote]Full methodologies for these approaches are provided in the Bentley et al. 2024, Supplementary Material.[\/footnote].<\/p>\r\n

    It is worth reiterating that statistical estimation of vulnerability multipliers does not necessarily have any bearing on ecology. While it is possible to exclude vulnerability multipliers from the search routine, there is currently no mechanism to include prior information or ecologically sensible bounds to constrain the limits for vulnerability multipliers included in the search routine. A judgement evaluation following the formal estimation of vulnerability multipliers should be applied to:<\/p>\r\n\r\n

      \r\n \t
    1. reflect on the ecological assumptions attached to estimated vulnerability multipliers,<\/li>\r\n \t
    2. assess how realistic functional group simulations are (in hindcast and future), and<\/li>\r\n \t
    3. understand and fix issues with model structure and parameterization (Figure 1) .<\/li>\r\n<\/ol>\r\nIt is possible to view the fit of each functional group to calibration time series and its contribution to the overall SS<\/em> in Ecosim via the Ecosim > Output > Ecosim group plots<\/em> form. This is often used to screen issues with model simulations, such as contradicting trends or misalignment in initial time steps, and direct fixes.\r\n

      However, what is often not accounted for when estimating vulnerability multipliers is their impacts on the advice products such as FMSY<\/sub><\/em> or food web indicators. The focus is often only on the goodness of fit of the model, but the impacts of estimated vulnerability multipliers on predictions and reference points should be evaluated[footnote]e.g., Rehren, J., Coll, M., Jiddawi, N., Kluger, L.C., Omar, O., Christensen, V., Pennino, M.G. and Wolff, M., 2022. Evaluating ecosystem impacts of gear regulations in a data-limited fishery\u2014comparing approaches to estimate predator\u2013prey interactions in Ecopath with Ecosim. ICES Journal of Marine Science 79(5):1624-1636. https:\/\/doi.org\/10.1093\/icesjms\/fsac077<\/a>[\/footnote]. We next provide two case studies to explore how alternate fitting approaches impact the emergence of vulnerability multipliers and how vulnerability multipliers impact model outputs.<\/p>\r\n\r\n

      \r\n
      \r\n
      \r\n
      \r\n
      \r\n
      <\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
      \r\n
      Attribution\r\n<\/strong>This chapter is based on Bentley JW, Chagaris D, Coll M, Heymans JJ, Serpetti N, Walters CJ and Christensen V. 2024. Calibrating ecosystem models to support marine Ecosystem-based Management. ICES Journal of Marine Science, https:\/\/doi.org\/10.1093\/icesjms\/fsad213<\/a>. Adapted based on CC BY License. Rather than citing this chapter, please cite the source.<\/div>\r\n<\/div>","rendered":"

      It is becoming more common for Ecosim vulnerability multipliers and primary production anomalies to be estimated using statistical routines, with Heymans et al.[1]<\/sup><\/a> demonstrating that it is best practice to estimate vulnerability multipliers by fitting model simulations to time series reference data. Longer time series are preferable as they provide an opportunity to explore important drivers of change and tend to have strong contrast in the data, which improves the model\u2019s ability to estimate parameters, leading to more confidence in our assessment of ecosystem dynamics.<\/p>\n

      Model fitting that includes estimation of primary production anomalies is basically the ecosystem equivalent of estimating recruitment and mortality anomalies in state-space approaches to parameter estimation for single-species models, a key difference being that the anomaly estimates may be informed by correlated variation in time series patterns of multiple species.<\/p>\n

      The quality (precision, informative contrast over time) of time series data is important, especially if the fitted model is to be used for management purposes, as vulnerability multipliers (and thus predation rate changes), will be used in forward simulations to times beyond the data. In Ecosim, users can weight time series data to represent how reliable or variable time series are compared to the other reference time series. Low weights imply that the data either has high variance or is unreliable (e.g., underestimated or uncertain catches). Weightings impact the contribution of time series to the assessment of model performance, where a weight of 0 indicates that the time series will not be used in the calculation of “goodness of fit”. Weightings can be assigned based on a qualitative assessment of data pedigree (e.g., based on data origin), or by using more quantitative information, such as confidence intervals from survey estimates, the retrospective analyses of stock assessment models, or signal to noise ratio assessments[2]<\/sup><\/a>.<\/p>\n

      The procedure for estimating vulnerability multipliers and production anomalies that improve the fit of model simulation to calibration data is based on minimization of a sum of squares (SS<\/em>) of prediction errors, which is then checked for overparameterization using the Akaike Information Index (AIC)<\/em>[3]<\/sup><\/a> [4]<\/sup><\/a>.<\/p>\n

      The SS<\/em> is used to calculate a log likelihood criterion (Figure 1), assuming normally distributed deviations of log model predictions from log observations, evaluated at the conditional maximum likelihood estimate of the prediction error variance and scaled in the case of relative observations (y)<\/em> by the maximum likelihood estimate of the relative simulation scaling factor (q)<\/em> in the equation y = q x X<\/em>,\u00a0 where X<\/em> is the absolute observation. Model fitting then proceeds by numerical search procedures to seek parameter values that minimize SS.<\/p>\n

      When generating a set of model fits under different fitting hypotheses or methods for choosing what parameters to include in the SS, AIC<\/em> is then used to identify the model of best fit. AIC<\/em> is a tool for model selection that penalizes for fitting too many parameters relative to the time series available for estimating the SS and is calculated as<\/p>\n

      [latex]AIC = n \\cdot \\ln (\\frac{minSS}{n}) + 2K\\tag{1}[\/latex]<\/p>\n

      where n<\/em>\u00a0is the total number of observations, or time series values, from the loaded calibration time series and K<\/em>\u00a0is the number of parameters estimated. When sample size is small, there is a large probability that AIC<\/em> will select models with too many estimated parameters (i.e., overfit models). The modified AICc<\/sub><\/em> can be used to address this potential overfitting by including a correction for small sample sizes<\/p>\n

      [latex]AIC_c = AIC + 2K \\cdot (\\frac{K-1}{n-K-1})\\tag{2}[\/latex]<\/p>\n

      As a rule of thumb[5]<\/sup><\/a>, AICc<\/sub><\/em> should be used unless n\/K<\/em> > ~40. In other words, unless the number of estimated parameters equates to a minimum 2-3% of the amount of data, use AICc<\/sub><\/em>. AICc<\/sub><\/em> should therefore be used when assessing EwE model performance.<\/p>\n

      A word of caution, the AIC calculations assume that the observations are independent whereas timeseries data such as typically used for ecosystem modelling have high autocorrelation. For this reason, it is advisable to test the impact on assumptions about n<\/em> on model selection.<\/p>\n

      \"\"<\/p>\n

      \n
      \n
      \n
      Figure 1.\u00a0<\/span> Overview of the Ecopath and Ecosim modelling process. Using log likelihood criteria, vulnerability multipliers or production anomalies (e.g., climate or nutrient loading) may be estimated based on a non-linear search routine and vulnerability multiplier (vulmult) estimation. Prediction (fitting) failures after each estimation trial then inform judgmental changes in model structure and parameters. B is biomass, Z<\/em> is total mortality, C<\/em> is catch, W<\/em> is average weight. Subscript 0 refers to the Ecopath model base year, and CC<\/em> to carrying capacity. Bcc<\/sub>\/B0<\/sub> refers to vulmult.\u00a0From Christensen and Walters<\/span><\/strong>[6]<\/sup><\/a><\/span>.<\/span><\/strong><\/div>\n<\/div>\n<\/div>\n<\/div>\n

      Multiple approaches have been developed to statistically estimate vulnerability multipliers (see Figure 2 in Vulnerability and vulnerability multipliers<\/a> chapter). They can be estimated for predators, providing a single multiplier limit to all of a given predator\u2019s base predation rates, and they can be estimated for individual predator-prey relationships, which assumes that the multiplier limits are heterogeneous across prey. This choice tends to be associated with user preference, ecological justification, or determined based on the approach that produces the best fit model. Whether estimating predator or predator-prey vulnerability multipliers, there are a few ways to select which vulnerability multipliers should be estimated:<\/p>\n

        \n
      1. manual selection based on a priori knowledge or species priority;<\/li>\n
      2. select vulnerability multipliers for groups with calibration time series; or<\/li>\n
      3. select the most sensitive vulnerability multipliers (i.e., those that when changed have the largest impact on SS<\/em>).<\/li>\n<\/ol>\n

        Manually selecting vulnerability multipliers allows for an early integration of ecological information but may lead to a sub-optimal model fit if the SS<\/em> is not sensitive to the selected multipliers. Conversely, the sensitivity search may optimise model fit but it is purely statistical and does not know what makes sense ecologically. Only estimating vulnerability multipliers for groups with time series acknowledges that, to some degree, the parameter should be constrained by the available time series. The level of group connectedness within the food web (e.g., group consumption and predation) may also constrain the parameter search if changes in vulnerability multipliers impact the \u00a0contribution of other groups.\u00a0 Groups that do not have informative time series or, have low connectedness in the food web, have widely variable estimated vulnerability multipliers \u2013 search routines can change those without any penalty incurred.<\/p>\n

        Choosing how many vulnerability multipliers to estimate, without overfitting is another point of confusion and discussion. The number of vulnerability multipliers that can be potentially estimated is often significantly more than the data available to constrain simulations. EwE best practices suggest that a conservative number of Degrees of Freedom (DoF) and therefore parameters to estimate is one less than the number of calibration time series available[7]<\/sup><\/a>. This approach recognizes that values within time series are highly autocorrelated, viewing each time series as an “independent observation”, but\u00a0 the \u00a0approach could be overly conservative, especially if long time series are available\u2013 especially if the contrast (ups and downs) and are not just one-way trajectories.<\/p>\n

        Both manual and automated statistical calibration routines are available in Ecosim to search for vulnerability multipliers. The manual approach can be arduous when testing multiple fitting hypotheses (e.g., with or without fishing effort or primary production anomalies) as the number of plausible fitting combinations can easily reach the hundreds, if not thousands, increasing the likelihood of user error. In the past, users have overcome this issue by only testing the nth fitting scenario (e.g., 5, 10, 15 vulnerability multipliers etc.)[8]<\/sup><\/a> However this approach risks overlooking the vulnerability multiplier combination, which produces the best statistical fit. The stepwise fitting procedure developed by Scott et al.[9]<\/sup><\/a> automates this process, allowing for a broad exploration of the parameter space which accelerates the process and removes the problem of user error. Recent improvements to the automated approach have increased the computational speed by enabling multiple fitting scenarios to be tested simultaneously using computers multithreading capabilities (J. Steenbeek, pers. comm.).<\/p>\n

        Novel approaches to estimate vulnerability multipliers using the manual and automated fitting routines have also been developed for two EwE models which are being used operationally to inform fisheries catch advice. Bentley et al.,[10]<\/sup><\/a> employed an approach, which combined searches for predator vulnerability multipliers and predator-prey vulnerability multipliers, whereas the approach developed by Chagaris et al.[11]<\/sup><\/a> uses the manual fitting tool in Ecosim to iteratively estimate the \u00a0most sensitive predator-prey vulnerability multipliers over multiple sequential (repeated) tuning iterations.[12]<\/sup><\/a>.<\/p>\n

        It is worth reiterating that statistical estimation of vulnerability multipliers does not necessarily have any bearing on ecology. While it is possible to exclude vulnerability multipliers from the search routine, there is currently no mechanism to include prior information or ecologically sensible bounds to constrain the limits for vulnerability multipliers included in the search routine. A judgement evaluation following the formal estimation of vulnerability multipliers should be applied to:<\/p>\n

          \n
        1. reflect on the ecological assumptions attached to estimated vulnerability multipliers,<\/li>\n
        2. assess how realistic functional group simulations are (in hindcast and future), and<\/li>\n
        3. understand and fix issues with model structure and parameterization (Figure 1) .<\/li>\n<\/ol>\n

          It is possible to view the fit of each functional group to calibration time series and its contribution to the overall SS<\/em> in Ecosim via the Ecosim > Output > Ecosim group plots<\/em> form. This is often used to screen issues with model simulations, such as contradicting trends or misalignment in initial time steps, and direct fixes.<\/p>\n

          However, what is often not accounted for when estimating vulnerability multipliers is their impacts on the advice products such as FMSY<\/sub><\/em> or food web indicators. The focus is often only on the goodness of fit of the model, but the impacts of estimated vulnerability multipliers on predictions and reference points should be evaluated[13]<\/sup><\/a>. We next provide two case studies to explore how alternate fitting approaches impact the emergence of vulnerability multipliers and how vulnerability multipliers impact model outputs.<\/p>\n

          \n
          \n
          \n
          \n
          \n
          <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
          \n
          Attribution
          \n<\/strong>This chapter is based on Bentley JW, Chagaris D, Coll M, Heymans JJ, Serpetti N, Walters CJ and Christensen V. 2024. Calibrating ecosystem models to support marine Ecosystem-based Management. ICES Journal of Marine Science,           https:\/\/doi.org\/10.1093\/icesjms\/fsad213<\/a>. Adapted based on CC BY License. Rather than citing this chapter, please cite the source.<\/div>\n<\/div>\n

          Media Attributions<\/h2>