{"id":345,"date":"2025-10-07T14:46:27","date_gmt":"2025-10-07T18:46:27","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/?post_type=chapter&p=345"},"modified":"2026-05-27T14:50:52","modified_gmt":"2026-05-27T18:50:52","slug":"monitoring-system","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/chapter\/monitoring-system\/","title":{"raw":"12.4 Establishing a monitoring system","rendered":"12.4 Establishing a monitoring system"},"content":{"raw":"Once the focus is determined, the next step consists of identifying indicators, targets, and sources of data. Table 12.2 provides an illustration of the different focal points and the corresponding criteria, indicators, targets, data sources, and collection processes.\r\n\r\nChoosing targets to monitor is important. According to Champagne et al. (2011d), targets can be identified from different sources such as best practice guidelines, institutional norms, actual practices in similar organizations, and experts\u2019 views. The source will be identified according to the monitoring system\u2019s purpose. The evaluator may choose to select representative targets, such as the mean, or opt for elitist ones (Champagne et al., 2011d). For example, an education program aimed at excellence may choose not to compare its programs to the domain\u2019s average achievement but instead compare itself to the best ones or specific targets of excellence in sub-areas. Targets can be ambitious but should be achievable (Markiewicz & Patrick, 2022\/2016).\r\n\r\nMonitoring can adopt a longitudinal perspective and track progress over time, using the initial data collected as a baseline for monitoring evolution over time. Monitoring can also be used to compare performance across programs or similar organizations or units (McDavid et al., 2019).\r\n\r\nWhen designing a monitoring system, the evaluator should be specific about the areas that will be monitored and ensure measurable indicators are selected, data is accessible, and the organization can collect and analyze it (Markiewicz & Patrick, 2022\/2016).\u00a0<\/strong>\r\n\r\nTable <\/strong>12<\/strong>.<\/strong>2<\/strong> Developing the Monitoring Plan<\/em>\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
Focus<\/strong><\/td>\r\nCriteria<\/strong><\/td>\r\nSample Questions<\/strong><\/td>\r\nIndicator<\/strong><\/td>\r\nTarget<\/strong><\/td>\r\nData Source<\/strong><\/td>\r\nWhose responsibility and timing<\/strong><\/td>\r\n<\/tr>\r\n
Structure<\/strong><\/td>\r\nFidelity<\/td>\r\nIs the intervention implemented as planned?<\/td>\r\nInvestments\r\n\r\nRecruitment of professional\r\n\r\nQualifications<\/td>\r\nAs set in the plan<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n
Process<\/strong><\/td>\r\nFidelity<\/td>\r\nAre all activities taking place?\r\n\r\nAre the costs respected?<\/td>\r\nAmount and number of activities delivered\r\n\r\nOperation costs<\/td>\r\nAs set in the plan<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n
\u00a0<\/strong><\/td>\r\nReach<\/td>\r\nIs the target population participating as expected?<\/td>\r\nTarget population versus participants<\/td>\r\nAs set in the plan<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n
\u00a0<\/strong><\/td>\r\nQuality<\/td>\r\nAre the activities offered according to best practices?<\/td>\r\nWait times\r\n\r\nSurvival rates<\/td>\r\nAccording to best practices<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n
\u00a0<\/strong><\/td>\r\n<\/td>\r\nAre clients satisfied (interpersonal dimensions of quality)?<\/td>\r\nSatisfaction rates\r\n\r\nDissatisfaction<\/td>\r\nAgreed on levels\r\n\r\nNb of complaints<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n
Results<\/strong><\/td>\r\nGoal achievements<\/td>\r\nDetermined according to the logic model, for example, are students succeeding?<\/td>\r\nGraduate rates\r\n\r\nPost-graduate employment\r\n\r\nStudies duration<\/td>\r\nBest results in similar programs<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n
\u00a0<\/strong><\/td>\r\nEnvironmental impacts: pollution, biodiversity, land and waters<\/td>\r\nIs the intervention meeting its environmental commitments?<\/td>\r\nGreenhouse gas emissions\r\n\r\nUse of plastic\r\n\r\nGreening of parking lots<\/td>\r\nEstablished targets<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n
\u00a0<\/strong><\/td>\r\nImpacts on human systems<\/td>\r\nIs the intervention contributing to better health, greater equity and more prosperity?<\/td>\r\nMental health indicators\r\n\r\nRates of burn out\r\n\r\nDiversity of beneficiaries\r\n\r\nProportion of spending in local business<\/td>\r\nAccording to best practices or agreed pre-set targets<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nSource: Adapted from Markiewicz, A., & Patrick, I. (2022). Developing Monitoring and Evaluation Frameworks<\/em>. SAGE Publications, and from Champagne, F., Hartz, Z., Brousselle, A., & Contandriopoulos, A. P. (2011d). L\u2019appr\u00e9ciation normative. In A. Brousselle, F. Champagne, A. P. Contandriopoulos, & Z. Hartz (Eds.), L\u2019\u00e9valuation: concepts et m\u00e9thodes. Deuxi\u00e8me \u00e9dition mise \u00e0 jour.<\/em> Les Presses de l\u2019Universit\u00e9 de Montr\u00e9al.<\/sub>","rendered":"

Once the focus is determined, the next step consists of identifying indicators, targets, and sources of data. Table 12.2 provides an illustration of the different focal points and the corresponding criteria, indicators, targets, data sources, and collection processes.<\/p>\n

Choosing targets to monitor is important. According to Champagne et al. (2011d), targets can be identified from different sources such as best practice guidelines, institutional norms, actual practices in similar organizations, and experts\u2019 views. The source will be identified according to the monitoring system\u2019s purpose. The evaluator may choose to select representative targets, such as the mean, or opt for elitist ones (Champagne et al., 2011d). For example, an education program aimed at excellence may choose not to compare its programs to the domain\u2019s average achievement but instead compare itself to the best ones or specific targets of excellence in sub-areas. Targets can be ambitious but should be achievable (Markiewicz & Patrick, 2022\/2016).<\/p>\n

Monitoring can adopt a longitudinal perspective and track progress over time, using the initial data collected as a baseline for monitoring evolution over time. Monitoring can also be used to compare performance across programs or similar organizations or units (McDavid et al., 2019).<\/p>\n

When designing a monitoring system, the evaluator should be specific about the areas that will be monitored and ensure measurable indicators are selected, data is accessible, and the organization can collect and analyze it (Markiewicz & Patrick, 2022\/2016).\u00a0<\/strong><\/p>\n

Table <\/strong>12<\/strong>.<\/strong>2<\/strong> Developing the Monitoring Plan<\/em><\/p>\n\n\n\n\n\n\n\n\n\n\n\n
Focus<\/strong><\/td>\nCriteria<\/strong><\/td>\nSample Questions<\/strong><\/td>\nIndicator<\/strong><\/td>\nTarget<\/strong><\/td>\nData Source<\/strong><\/td>\nWhose responsibility and timing<\/strong><\/td>\n<\/tr>\n
Structure<\/strong><\/td>\nFidelity<\/td>\nIs the intervention implemented as planned?<\/td>\nInvestments<\/p>\n

Recruitment of professional<\/p>\n

Qualifications<\/td>\n

As set in the plan<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Process<\/strong><\/td>\nFidelity<\/td>\nAre all activities taking place?<\/p>\n

Are the costs respected?<\/td>\n

Amount and number of activities delivered<\/p>\n

Operation costs<\/td>\n

As set in the plan<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
\u00a0<\/strong><\/td>\nReach<\/td>\nIs the target population participating as expected?<\/td>\nTarget population versus participants<\/td>\nAs set in the plan<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
\u00a0<\/strong><\/td>\nQuality<\/td>\nAre the activities offered according to best practices?<\/td>\nWait times<\/p>\n

Survival rates<\/td>\n

According to best practices<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
\u00a0<\/strong><\/td>\n<\/td>\nAre clients satisfied (interpersonal dimensions of quality)?<\/td>\nSatisfaction rates<\/p>\n

Dissatisfaction<\/td>\n

Agreed on levels<\/p>\n

Nb of complaints<\/td>\n

<\/td>\n<\/td>\n<\/tr>\n
Results<\/strong><\/td>\nGoal achievements<\/td>\nDetermined according to the logic model, for example, are students succeeding?<\/td>\nGraduate rates<\/p>\n

Post-graduate employment<\/p>\n

Studies duration<\/td>\n

Best results in similar programs<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
\u00a0<\/strong><\/td>\nEnvironmental impacts: pollution, biodiversity, land and waters<\/td>\nIs the intervention meeting its environmental commitments?<\/td>\nGreenhouse gas emissions<\/p>\n

Use of plastic<\/p>\n

Greening of parking lots<\/td>\n

Established targets<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
\u00a0<\/strong><\/td>\nImpacts on human systems<\/td>\nIs the intervention contributing to better health, greater equity and more prosperity?<\/td>\nMental health indicators<\/p>\n

Rates of burn out<\/p>\n

Diversity of beneficiaries<\/p>\n

Proportion of spending in local business<\/td>\n

According to best practices or agreed pre-set targets<\/td>\n<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Source: Adapted from Markiewicz, A., & Patrick, I. (2022). Developing Monitoring and Evaluation Frameworks<\/em>. SAGE Publications, and from Champagne, F., Hartz, Z., Brousselle, A., & Contandriopoulos, A. P. (2011d). L\u2019appr\u00e9ciation normative. In A. Brousselle, F. Champagne, A. P. Contandriopoulos, & Z. Hartz (Eds.), L\u2019\u00e9valuation: concepts et m\u00e9thodes. Deuxi\u00e8me \u00e9dition mise \u00e0 jour.<\/em> Les Presses de l\u2019Universit\u00e9 de Montr\u00e9al.<\/sub><\/p>\n","protected":false},"author":20,"menu_order":4,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[49],"contributor":[],"license":[],"class_list":["post-345","chapter","type-chapter","status-publish","hentry","chapter-type-numberless"],"part":333,"_links":{"self":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters\/345","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/users\/20"}],"version-history":[{"count":4,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters\/345\/revisions"}],"predecessor-version":[{"id":696,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters\/345\/revisions\/696"}],"part":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/parts\/333"}],"metadata":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters\/345\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/media?parent=345"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapter-type?post=345"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/contributor?post=345"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/license?post=345"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}