35 Community-Based Research for Food Systems Change: A Collaborative Praxis [Award Winning Paper]

*Best Paper Award For Contribution to Practice

Abstract

Organizations that operate community food initiatives (CFIs), such as community gardens, farmer training workshops, and food literacy projects, are challenged to provide evidence of the tangible contributions they make towards advancing food security, food sovereignty and/or just and healthy local food systems. Critiqued within the literature for failing to address income as a root cause of food insecurity (Martin, 2018, p. 116), acknowledge how they reinforce inequitable systems of power (Guthman, 2008, pp. 434–436), and offer meaningful alternatives to the neoliberal economic system (Moragues-Faus, 2017, p. 471), academic and community stakeholders have called for more focused and robust evaluation practices for CFIs (Knight, 2013, p. 30; Levkoe & Blay-Palmer, 2018, p. 50; Loopstraw & Tarasuk, 2013, p. 55).

Unfortunately, for many organizations, addressing these critiques through evaluation remains elusive due to a lack of conceptual frameworks, financial supports, and literature (Barbour, Ho, Davidson, & Palermo, 2016, p. 128). Accordingly, this reflective paper reviews the results of an applied doctoral research project investigating how organizations operating CFIs within Central Vancouver Island can evaluate their impacts from a food systems perspective. Structured through community-based research methodology and framed by the food regime analysis and food sovereignty theoretical frameworks, this qualitative study employed semi-structured interviews, document analysis, and an analysis of food systems definitions to identify the evaluation method and indicators required to revaluate CFIs’ food systems impacts. The findings supported the drafting of a food systems-focused evaluation framework for the study’s community research partners, including an evaluation logic model, matrix, and plans. In addition, the process of conducting community-university research revealed avenues for improving the application of CBR and social sciences theory within applied, collaborative doctoral research.

Keywords: community-based research; community food initiatives; evaluation

Introduction

This reflective research paper summarizes the conduct and results of an applied, collaborative doctoral research project that studied how organizations operating community food initiatives (CFIs) within Central Vancouver Island can evaluate their impacts from a food systems perspective. First, an outline of the project’s applied context opens the paper to reveal the study’s purpose, site, and research partners and describe how the community-university research collaboration was founded, nurtured, and maintained through community-based research (CBR) methodology. Second, the study’s theoretical framework is addressed to define the rationale and tensions associated with the application of social science theory in an applied research setting. Third, the presentation of findings describes how the research team (also referred to as the CBR Team) and I applied the results to inform the drafting of a food systems-focused evaluation framework (i.e., the evaluation logic model, matrix, and plans). Fourth, this paper closes with a discussion of the challenges and opportunities the CBR Team and I faced in balancing our distinct expectations (me as a doctoral candidate and the research partners as grassroots community organizations interested in actionable results) with the epistemological demands of CBR and offers recommendations to the academy pertinent to how best prepare students and community stakeholders for collaborative research.

Applied Research Context

This research project grew from an operational challenge my colleagues and I faced while working at the Cowichan Green Community (CGC), a registered not-for-profit organization whose mission is to “cultivate food, community, and resilience” (CGC, 2021, para. 2) within the Cowichan Region of Central Vancouver Island. As program staff responsible for organizing community-based food events, initiatives, research projects, and workshops, our jobs frequently entailed evaluation. As we faced high staff turnover rates, limited evaluation expertise and resources, overambitious workloads, and restrictive funding, our evaluation practices mainly consisted of funder-driven evaluations that often failed to deliver meaningful data to our organization. For example, such evaluations required our team to gather generalized program metrics, including details on outputs, participation rates, program activities, and program expenditures. Few, if any, asked or supported our team to gather data on the broader, systems-level impacts of our work or contemplate how our initiatives met our organization’s mission statement. Although we recognized that funders did not wish to burden CGC with unrealistic evaluation processes and also needed to collect data to evaluate the merits of their funding opportunities, we felt that when organizations direct vital energy to evaluation, the results needed to be impactful.

Consequently, our team sought to develop an evaluation framework that would assist us to evaluate our work in relation to the systems-level critiques we faced, namely: (a) can CFIs genuinely tackle the root causes of food insecurity?; (b) can CFIs present democratic alternatives to neoliberalism?; and (c) can CFIs amend social injustices? Further, we hoped to use the results of such an evaluation framework to communicate our impacts (and limits) to funders and local governments, who, in our experience, failed to recognize that the impacts of our CFIs evolved over a project’s lifespan and occurred outside the scope of our place of work. Thus, aware that CGC had a responsibility to progress our field of practice by addressing this knowledge gap, our staff began to explore how we might evaluate our work’s impacts on our local food system. In pursuit of this goal, we applied (unsuccessfully) for funding to support evaluation, discussed and defined food-based evaluation matrices, partnered with a local university to engage students on the idea of evaluating CFIs, and recruited interns to develop evaluation concepts. With each activity, we made small steps towards our goal. Unfortunately, without dedicated evaluation funding, we were unable to actualize any one idea.

Fortuitously, our investigation into evaluation practices surfaced in tandem with my acceptance to Royal Roads University’s (RRU) Doctor of Social Sciences (DSocSci) program, where I chose to dedicate my research to this applied problem. With CGC willing to form a community-university research partnership, we commenced a five-year, multi-stakeholder research collaboration with the Cowichan Food Security Coalition, which served as an advisory team, and the Nanaimo Foodshare Society, a registered charity in Nanaimo, BC, that also faced barriers to evaluating their CFIs. In November 2017, we formalized our working relationship and designed a research project to answer: “How can the impacts of CFIs on Central Vancouver Island be evaluated from a food systems perspective?”

Methodological Framework

Due to the collaborative nature of this study, our methodological selection rested upon eight requirements, including: (a) the interdisciplinary trajectory of food studies and food systems research in Canada (Koç, Bancerz, & Speakman, 2017, pp. 3–6); (b) the growing body of literature calling for more community-university research collaborations in the area of food systems and evaluation (Barbour et al., 2016, p. 128; Barton, Wrieden, & Anderson., 2011, p. 593); (c) the inter/trans-disciplinary focus of RRU’s DSocSci program (RRU, 2014, pp. 3–4); (d) my axiological and epistemological outlook which favoured applied, inter/trans-disciplinary research that fosters an equitable sharing of power, relationship-building, and reciprocity; (e) the boundaries and expectations of doctoral research that defined our timeline and matters such as intellectual property; (f) the collaborative nature of this study; (g) CGC and NFS’s need for a flexible participation model; and (h) CGC and NFS’s desire for action-oriented results.

To meet these demands, we selected CBR, a branch of action-oriented and participatory research that offers an alternative approach to positivist epistemology and knowledge creation. By dispelling research hierarchies and the notion that the academy is the custodian of knowledge, CBR encourages the active collaboration and participation of academic and community partners to co-design knowledge creation instruments and meaningfully share data collection, analysis, and knowledge mobilization processes. Inspired by the writings of Etmanski, Hall, and Dawson (2014), Israel, Schulz, Parker, and Becker (1998), Ochocka and Janzen (2014), Smith (2008), and previous CBR research within food studies (Andrée et al., 2014; Rojas, Black, Orrego, Chapman, & Valley, 2017), we designed a qualitative study consisting of three data collection methods. Specifically, throughout 2018, we conducted 17 semi-structured interviews with leaders of CFIs operating on Vancouver Island and Coast, a document analysis of six academic and 19 community-published reports on the evaluation of CFIs, and an analysis of 45 definitions to food systems terms (e.g., food justice, food literacy, food security, food sovereignty, Indigenous food sovereignty, and the right to food) retrieved from the public websites and/or social media channels (e.g., Facebook and Twitter) of 27 organizations in British Columbia that operate CFIs.

Though I was an emerging scholar new to CBR, and CGC and NFS felt our research methodology was too fluid or unbound (concerned by CBR’s broad interpretations and lack of defined protocols), we established procedures to guide our working relationship and our implementation of CBR. First, we agreed to adapt Kemmis, McTaggart, and Nixon’s (2014) Action Research Cycle (planning, action, observation, and reflection) (p. 18) as part of our methodology. In doing so, we applied the Action Research Cycle to ensure we consistently reviewed our research steps and took the necessary time to plan the study’s subsequent stages. Second, we agreed to formalize the working relationship between CGC, NFS, and me through a voluntary Memorandum of Understanding (MOU). The MOU defined our research team’s operational norms relevant to communication, document and resource sharing, inducements, meeting schedules, project scope, and responsibilities, as well as policies and procedures on anonymity and information use, confidentiality, conflicts of interest, conflict resolution, disclosure and retention of information, knowledge mobilization, possible negative effects, and the right to withdraw. With the support of RRU’s Office of Research Ethics and faculty, the drafting of the MOU required our team to unpack the tenants of RRUs ethics application and review process, discuss the minutia of our collaboration, define how to operationalize CBR within the context of a doctoral study, and grapple with issues sensitive to research collaborations, including the delineation and sharing of intellectual property. Although not without its contentions, the MOU served as a beacon to remind ourselves of our individual and shared responsibilities to the research project and each other and as a reference point to inform our collective decisions.

Theoretical Framework

In drawing from Powers’ (2010) description of a theory’s role in research—to outline what society currently knows of a phenomenon, elicit questions for future inquiry, and change and advance to indicate a progression in what is known of a phenomenon (p. 5)—the CBR Team and I searched for a theoretical framework to account for both the historical events that resulted in CFIs and the current circumstances requiring their further evaluation. In consultation with the CBR Team, I met these requirements by pairing food regime analysis and food sovereignty. Equally rooted in political-economic discourse, food regime analysis frames the development of the global, industrial food system as part of the colonial and neocolonial agendas (Friedmann & McMichael, 1989, pp. 93–95; Friedmann, 2009, p. 336), and food sovereignty offers a critique of the historical and contemporary economic and political systems, or regimes, that disenfranchise local or small-scale food producers and consumers (Gürcan, 2018, pp. 324–326, Wiebe & Wipf, 2011, pp. 3–8). Together, they frame the evolution and future trajectory of market-based food systems and offer a historical yet modern lens to analyze local to global food systems and their discontents.

In practice, I applied the theoretical framework post-analysis to contextualize this project’s findings and identify gaps within the evaluation framework. From a myopic focus on agrarian food systems (over aquatic, Indigenous, or non-market food systems) to issues of power that perpetuate food insecurity, I applied theory to reveal the evaluation framework’s limitations and identify opportunities for change. While a positive outcome, the use of theory within this study proved an area of contention as the CBR Team found the theoretical application burdensome. Faced with action-oriented mandates and a lack of time, neither organization possessed the resources to explore the theoretical literature, contemplate Powers’ articulation of theory, or assess how theory could inform our evaluation framework. To this, CGC and NFS acknowledged and respected the need for theory within an academic context and understood why I believed that it was critical to review our findings and the evaluation framework through the perspectives of food regime analysis and food sovereignty. Still, the use of theory distanced both research partners from my analysis and, equally so, our findings—an outcome counter to our understanding of CBR’s tenants. Consequently, as we prepare to close our study, the experience leaves me questioning how I could have improved the implementation of theory within this research context and how academia can make social sciences theory more accessible to community stakeholders.

Results

The analysis generated seven key findings that challenged and broadened our research team’s understanding of evaluation. We drew upon each theme within the design of our evaluation framework, including our approach to the evaluation method and indicator development. These findings included:

1. The evaluation should be cost-effective, accessible to a broad audience, quick to administer, and accompanied by supports. Should our evaluation framework fail to account for the operational landscape of CFIs and prove too technical to implement, the evaluation would not become a part of CGC or NFS’s work culture.
2. The evaluation must be culturally safe. As clearly identified in the literature (Blanchet-Cohen, Geoffroy, & Hoyos, 2018, pp. 21–29; Hanberger, 2018, pp. 114–119; Patton, 2011, p. 247; Patton, 2012, p. 76), evaluations must guarantee respect for culture. However, the interview data revealed that many staff of CFIs on Vancouver Island and Coast felt ill-equipped to evaluate from a culturally informed lens. This data, linked to concepts of anti-racism, decolonization, food justice, and Indigenous food sovereignty, suggested that staff require further resources or training to conduct culturally safe evaluations and that organizations bear a responsibility to prepare staff to take on this essential yet sensitive work.
3. The evaluation should empower staff in their work. The evaluation framework must operate from a position of appreciative inquiry or evaluative inquiry (Preskill & Torres, 1999, p. 44), whereby the process and results are not an exercise focused on deficits but a process of ongoing exploration and learning dedicated to evolving and improving CFIs. Such a focus may increase staff willingness to evaluate and the value they place over their work.
4. The evaluation should be adaptive to evolving food discourses. One of the most impactful outcomes of this study was a discussion of the breadth of food terms employed to define CFIs. From food justice, food literacy, food sovereignty, food systems, and more, there are a multitude of terms and countless definitions to describe a CFI’s purpose and proposed impacts. As such, this evaluation framework needs to contend with food discourse and the ever-changing lexicon employed in food studies and food systems work while remaining accessible to a general audience. Thus, the evaluation framework needs to tacitly connect to key food terms without burdening the process in technical language.
5. The evaluation needs to reflect the scaled impacts of CFIs. In order to recognize the numerous actors and arenas that CFIs interact with or impact, the evaluation framework needs to account for the various sub-systems (within a local food system) that CFIs intersect, including the participant, organization, community, and local food system. This finding confirmed our systems approach to the evaluation framework.
6. The evaluation should support organizations to evaluate the intangible systems-level impacts of CFIs. The findings indicated that we could not alone evaluate the food systems impacts of CFIs. Instead, we also need to evaluate their broader, interdependent systems- level impacts. From decolonization to emergency preparedness, the evaluation framework needs to facilitate a process to capture both food systems and the broader systems indicators that equally affect the attainment of just and healthy local food systems. As identified in Table 1 (Preliminary Logic Model Template), the Indicator Themes and the four Indicator Levels (e.g., participant, organization, community and local food system) represent this perspective. Likewise, in Table 2 (Preliminary Evaluation Matrix Template), the systems-informed evaluation questions mirror this outlook.
7. The community evaluated should develop the evaluation. While our interview data revealed that many staff of CFIs would appreciate access to standardized food systems evaluation frameworks, interviewees also described that it is best practice to develop evaluations directly with those engaged or impacted. A sentiment mirrored in the collaborative and participatory evaluative literature (Cousins & Chouinard, 2012, p. 10; Fetterman, Rodríguez-Campos, Wandersman, & O’Sullivan, 2014, p. 145; Schones, Murphy-Berman, & Chambers, 2000, p. 55), a collaborative approach to evaluation may increase buy-in from all stakeholders and the implementation of evaluation results. For interview participants, this was a critical demand of evaluation within an Indigenous or cross-cultural context and/or with vulnerable populations. In practice, this indicated that the evaluation framework should not offer prescriptive evaluation indicators. Instead, the framework should offer indicator themes related to a food system and the broader systems that CFIs intersect. Users of the evaluation framework are, accordingly, encouraged to draft their initiative-specific indicators through the lens of the diverse indicator themes presented in Table 1.

In summary, the findings of this project informed not only the method that this study’s evaluation framework for CFIs required but the types of indicators relevant to a food systems-focused evaluation. As a result, this study produced a draft evaluation framework for CGC and NFS that we hope will support each organization to design and evaluate their work in a manner reflective of the scaled impacts of their work and the various stakeholders they engage. However, as this study approaches completion, we have additional plans for our findings and research collaboration. First, the evaluation framework possesses gaps and shortcomings that we need to amend. As highlighted in the above outline to the theoretical framework, our evaluation framework is data-driven and is missing several viewpoints, including anti-racist, gendered, non-agrarian, and non-market perspectives. Second, we need to test the evaluation framework for its efficacy. Although our initial research plan was to test the evaluation framework, our ambitious research goals conflicted with busy work schedules and the onset of the COVID-19 pandemic. This test will now likely take place outside the confines of this dissertation. Third, we need to reimagine the evaluation framework. As exhibited by Tables 1 and 2, the evaluation framework, despite our best efforts, does not fully align with our first finding’s demand for an easy-to-use framework. While the logic model format was selected as this approach speaks to funders and is congruent with best practices in the evaluation field, CGC, NFS, and the Cowichan Food Security Coalition believe it is overly complicated. Therefore, our future work may look to how we can repurpose our data to generate other equally useful expressions of program measurement or assessment. One option is to develop an online assessment tool for organizations to identify where and how their proposed initiatives intend to impact their local food system.

Table 1 Preliminary Logic Model Template