Rebecca Wilson-Mah (rebecca.wilsonmah@royalroads.ca) is an Associate Professor in the School of Tourism and Hospitality Management at Royal Roads University, Victoria BC Canada.

Rationale

The mini field research pilot study offers graduate students in a research methodology course at Royal Roads University’s MA program in Tourism Management (IHMN 550) the opportunity to practice a research method of their choice. Typically, the students select a research method they are considering for their major research project or thesis research. The mini field research pilot study is the final assignment and functions as a capstone assignment for the course. This assignment is suitable as a capstone assignment for students who have already completed a proposal which includes a preliminary literature review, identification of a research question, and potential research methodology and methods.

This assignment contributes to student learning by taking the student from an understanding related to ‘learning about’ to practice based application and ‘learning to do.’ The intent of this assignment is to support the development of 1) planning skills for actual field research; 2) practice in conducting a research method; 3) basic practice using an analytical technique; and 4) the development of reflective capacity as a researcher.

The development of this assessment was informed by Mueller’s (2005) work on authentic assessment and the application of knowledge and skills to perform real-world tasks. The assessment takes the student into the activities of a researcher: specifically, the planning and thinking that occurs prior to using a research method. The assessment also takes the students through the practice of the research method and the reflection on this practice with others. Bridging the knowledge-to-practice gap, this authentic assessment also encourages self-awareness, decision making, and reflection (Boud, 2007). Furthermore, the mini field research pilot study builds the student’s capacity to receive and integrate feedback, adapt, and continuously improve after the course ends (Boud, 2000).

The final step of this mini field research pilot study includes the opportunity for peer learning. Students each share a voice-over presentation in small pods of three or four students. The opportunity to find out what two other researchers learned in their mini field research pilot study enriches the learning possibilities in this assignment. The students’ learning is expanded outside their own pilot study to the practice-based learning of two other peers.

The learning outcomes associated with this assessment are:

1. Effectively uses research methodologies, data sources, concepts, and theories to investigate, evaluate, and analyze findings that inform planning, decision-making, and actions.
2. Demonstrates integrity and ethical behaviour when collaborating, synthesizing findings, problem solving, conducting research, and reporting findings or recommendations.
3. Effectively gathers, analyzes, synthesizes, prepares, and presents written materials, accurately utilizing the American Psychology Association (APA) style.

Overview

The mini field research pilot study creates authentic engagement as learners practice a research method of their choice. All learners are asked to develop an invitation to participate in the study that includes a participation information sheet and consent form. Through application, the steps for ethical research practice are understood more deeply. The assignment is a confidence builder as learners connect more deeply with their proposed study and build their skills as researchers. A course ethics review is in place for this assignment. Instructors who wish to adopt the mini field research pilot study are advised to complete a course ethical review at their institution for this assignment before proceeding.

The intent of this assignment is to provide students with an appreciation and understanding of actual field research and practice in conducting research. Furthermore, the assignment encourages students to consider and practice reflexivity as a researcher. In advance of starting this assignment, invite students to watch this short video from Sage Publications on Pilot Studies. In this video Eva Mikuska (2018) reflects on piloting ideas and research methods and she shares a useful overview of what she learned from her own pilot study. She raises some useful considerations that students may wish to think about as they reflect on the purpose and practice of a mini field research pilot study.

Step 1: Select a Research Method

Ask students to select a research method and carry out a mini-research pilot project for that research method. If your students have written a research proposal, advise them to consider selecting a research method that was included in their proposal. If students have not completed this step, I recommend setting aside time to review different research method choices. Be sure to provide students with the research consent form (Appendix 1), and the introductory letter (see Appendix 2). These supporting documents are customized by the student and the instructor.

The research method could be a pilot design for a survey shared with a small number of respondents. Alternatively, the method could be practicing a semi-structured interview design with one or two participants or some form of observation. Students might also like to try a creative method such as visual explorer, or a collaborative group method.

Step 2: Research, Plan, and Pilot a Research Method

It will be important for students to undertake some research on their chosen research method and to follow the key steps and practices recommended for the method. If you use a particular course text, direct the students to refer to the textbook or associated readings. Ask students to aim to be systematic in their practice and to think ahead to plan out their approach. Advise students to make notes on their learning as they pilot their research method. Were there any surprises?  What worked and what did not work as anticipated?

Step 3: Voice-Over Presentation

Explain that on completion of the pilot study students will share their field research project with their peers and instructor through a short voice over presentation (maximum 12 minutes). Advise the students to share: 1) the research method they chose and why, 2) their experience with the process, 3) the data they collected, and 4) their reflection on what was learned from practicing a research method.

Ask students to consider and explain how they would analyze the data and if any adjustments are needed to their data collection method. Encourage your students to be reflective and share what they learned openly (e.g. what worked, what did not work, what you would do differently next time). Direct students to refer to the assignment rubric for further details and prompts (Appendix 3).

In this final step the students create a voice-over presentation to share with small pods of three or four students. Invite your students to share their thoughts on the value of peer learning. Emphasize the opportunity to learn and share with others, and the benefits of reciprocity.

Student feedback on this assignment is resoundingly positive. Students share that they enjoy the freedom to pursue their own research interests in a hands-on way, and that through practice they can situate themselves as the researcher.

Reflection

The following common distinguishing features of authentic assessment held some congruence to the mini field research pilot project assignment.

1. Aligned: Authentic assessment aligns with the course instruction and learning approaches; the choices are intentional in these areas (Lindstrom, Taylor, & Weleschuck, 2017). The mini-field research pilot project offered students the opportunity to apply their learning in the course to pilot a research method of their own choice. With support from their instructor, each student made their own choices. This decision making mirrored the decisions they will make as principal researchers in their own major research project, and this is a confidence builder. This assessment also builds on the previous ‘draft research assignment’ that precedes the mini field research pilot project.
2. Integrated: Authentic assessment provides the space for students to receive and integrate feedback; to reflect and continuously improve after the course ends (Boud, 2000). The voice-over presentation delivered to a small group of fellow students offers the possibility for integration of feedback and questions from peers. This step can be further supported by including a peer feedback forum immediately after the presentations.
3. Supportive: This assessment is learner-centred and creates linkages between previous assignments and future courses (e.g., a major research project or thesis). By providing choice and student discretion in the final assessment, within a framework of clear expectations (Appendix 3) the students can drive their own learning bus and identify their own stops, starts, speed and progression.
4. Contextual: This assignment takes the students outside the course into the real world of research practice. The students plan and practice a research method that relates directly to their draft research proposal. The two assignments work together to support active engagement in research in a real-world professional context. The students think, plan, apply and reflect – all important aspects of the research process.

I have used this assessment in several research methodology courses over the past four years, and each time I reflect, refine, and improve. In the most recent iteration, I placed more emphasis on identifying and practicing an analytical process. This has rounded out the learning available in this assessment. Students now identify and apply an analytical technique to a small data set thus gaining applied experience with data analysis.

References

Boud, D. (2000). Sustainable assessment: Rethinking assessment for the learning society. Studies in Continuing Education, 22, 151-167.

Boud, D. (2007). Reframing assessment as if learning were important. In D. Boud & N. Falhikov (Eds.) Rethinking assessment in higher education: Learning for the longer term (pp. 14-26). New York, NY: Routledge.

Lindstrom, G., Taylor, L., & Weleschuk, A. (2017). Guiding Principles for Assessment of Students’ Learning. Calgary, AB: Taylor Institute for Teaching and Learning, University of Calgary. http://studentassessment.ucalgaryblogs.ca/files/2017/06/Guiding-Principles-for-Assessment-of-Student-Learning-FINAL.pdf

Mikuska, E. (Academic). (2018). The importance of pilot studies [Video]. SAGE Research Methods https://dx.doi.org/10.4135/9781526450180

Mueller, J. (2005). The authentic assessment toolbox: enhancing student learning through online faculty development. Journal of Online Learning and Teaching, 1(1), 1-7.

Appendix 1: Sample Consent Form

RESEARCH CONSENT FORM

Mini Field Pilot Research Project – IHMN 550 (Research Methodology)

My name is [your name here] and I am a graduate student in a Master of Arts Program in Tourism Management at Royal Roads University. I am currently taking a course in research methodology, and the final assignment is a Mini Field Research Pilot Study. The intent of this assignment is to practice a research method. The research method I have chosen to practice is [your research method here].

My credentials with Royal Roads University can be established by contacting my course instructor,  [name and contact information here]. The Research Ethics Board can be contacted via [name and contact information here].

Thank you for your interest in supporting the development of researchers at [institution name here].

This document constitutes an agreement to participate in my Mini Field Pilot Research Project. If you choose to participate, your anonymity will be protected, and no names or distinguishing data will be associated with the data collected.

On completion of this project I will submit an assignment to my course instructor. The assignment will not be publicly accessible.

If you do choose to participate in this project you are free to withdraw at any time without prejudice.

By signing this letter, you give free and informed consent to be included in this project as outlined above.

Name: (Please Print): __

Signed: __

Date: __

Appendix 2: Sample introduction letter

[date here]

To whom it may concern:

Re: Mini Field Research Pilot Study

At Royal Roads University we encourage our students to learn through practice and experience in real world settings.

The student: [your name here]  is currently taking a course in research methodology (Research Methodology, IHMN 550). The final assignment is a Mini Field Research Pilot Study. The intent of this assignment is to provide the student with an opportunity to pilot (practice) a research method. For example, this could be to pilot a design for a survey shared with a small number of people, or to facilitate a small focus group.

If you choose to participate, your anonymity will be protected, and no names or distinguishing data will be associated with the data collected. On completion of the pilot study the student will destroy the data collected.

Thank you for considering participation in a mini field research pilot study. If you have any questions, please do not hesitate to discuss them directly with the student, or with me, the instructor at the contact listed below.

Sincere regards,

[name and contact information here].

Appendix 3: Sample Rubric

Mini Field Pilot Research Study

The intent of this assignment is to provide you with 1) an appreciation and understanding of actual field research; and 2) practice in conducting research. Furthermore, the assignment encourages you to consider 3) your reflective capacity as a researcher.

Rubric

1. Analysis and clarity /10

Share the choice of research method by providing a coherent explanation of how the research method relates to the research topic and research questions, research design, and other aspects of the study (feasibility, ethics, participant sample etc.).

2. Research /10

Situate the research method in the literature related to your research method. For example, if you used a survey, what data and processes did you refer to (secondary sources) before you decided on the survey design and distribution? What sources did you use to help design your survey?

3. Preparation and planning /20

Share the planning and preparation completed in advance of conducting the research – how did you prepare and plan? What skills did you need to think about?  How did you introduce the study to the research participant etc.?  If you were to scale this up with more participants, what would you need to do?  Provide our preparation documents if relevant (for example, interview schedule, participant information sheet, field notes, or consent form).

4. Practice a research method /20

Explain what you did, and how it went. Review the data you collected and comment on it. Did your practice go as you expected?

How did you record your data?  Did you count or interpret?

5. Identify an analytical strategy and practice /10

Select an analytical strategy and complete some initial data analysis.

What analytical strategy did you choose and why?

6. Reflect on learning /20

Reflect on and share what you learned from the pilot. This assessment promotes learning through practice. What did you learn about yourself, the method, the research topic?

7. Presentation, Composition, APA /10

Consider your audience: your instructor and your fellow researchers. Create the presentation for them and think about their interests and questions. Present professionally: visual, written, spoken etc.

The learning outcomes associated with this assessment are:

1.2: Effectively uses research methodologies, data sources, concepts and theories to investigate, evaluate, and analyze findings that inform planning, decision-making and actions.

1.4: Demonstrates integrity and ethical behaviour when collaborating, synthesizing findings, problem solving, conducting research and reporting findings or recommendations.

3.1: Effectively gathers, analyzes, synthesizes, prepares and presents written materials, accurately utilizing the APA reference method.