A. Assessing Collaborative, Experiential and Discussion-based Learning

What makes a good online discussion that facilitates learning? How do you know it when you see it?

In an online course, there are endless options for assessment; for the purposes of this workshop, we will be focusing on assessing learning within synchronous and asynchronous activities. As you can probably imagine, online discussion activities seem at first to be difficult to assess as they rely upon subjective assessment. However, we can use performance assessments to make transparent to students the elements that we are looking for in their efforts. This section will explain how you can do that.

Much can be said about assessing learners; however, it is our intention to provide laser-like focus to this section. We concentrate on assessing learners in the context of collaborative, experiential and discussion-based learning activities. The synchronous and asynchronous affordances that online discussion forums and web-conferencing tools provide make student-to-student interactions easy. Online learning makes it possible to design learning activities that encourage students to engage with each other using these tools. Our role as instructors, then, is to how best to provide students with meaningful feedback as we engage these tools to facilitate their learning.

And of primary of importance to note: it’s important that participants know how they will be evaluated up front. This means you must determine how you will rate or measure contributions and communicate this information to participants at the start of the course or activity.

The Goal of Assessment

Let’s start at the beginning by considering what is the goal of assessment in our learning environments. Very simply: the goal of assessment is to evaluate participants’ learning, and give them feedback on their progress toward learning outcomes. All assessments must be fair and consistent. This means that all students are assessed in an open and straightforward manner: no tricks or hidden agendas, or different applications of the rules to different students.

What we are asking you to do is to think about the choices and rationale that you make when you design and create your course’s learning assessments. In an online course, we must be more deliberate in planning for assessment and planning in advance. Determining how the various evaluation activities work together in your course go a long way in ensuring that:

• the timing of the assessment is fair. Students are ready for the assessment based on the work that has preceded it.
• the scope of the assessment is fair. The assessment is aligned with the learning outcomes of your course, in terms of both breadth as well as level of achievement.
• the conditions of the assessment is fair. You have scheduled assessments over your course so that students have time to complete them, and to ensure they have access to the resources to complete them.
• students benefit from formative feedback during the delivery of the course so that they may have the chance to apply what they have learned from the feedback to subsequent assessments. Planning and scheduling assessments in advance ensure that your online students have a chance to identify errors in their understanding and learn from their earlier mistakes so that they may then be able to turn around and apply their revised understanding.

Sometimes facilitators equate “being fair” with “being objective.” In many cases, though, it is impossible to eliminate the subjective element. The learning you are evaluating does not always lend itself to clear-cut yes or no answers–a requirement for completely “objective” grading. However, subjectivity is not, in itself, a bad thing. It is what allows you latitude to use your own knowledge and experience–the knowledge and experience that you draw on every day as a facilitator and in your other work–to help people become more proficient. The important thing is using fair, consistent criteria to help you evaluate and assess.

Key take-aways:

• Focus on what’s important (what will make a difference to student knowledge, understanding and performance in the future).
• Feedback is a conversation. Foster a supportive environment where this conversation can flourish.
• Focus on the actionability of the feedback you give.

Assessing for Evidence of Student Learning

When you are planning and designing online learning activities, you have the choice of including individual or group assignments. There are times when an individual assignment is the best choice. Individual assignments give you an opportunity to evaluate each participant’s mastery of the competency being learned. At other times, group work can be more useful; it can enhance learning to have participants work together to learn from each other. Group assignments do save you from having to review individual assignments; however, it is rarely a time-saver, since they tend to be large and more complex.

Whether the assignment is an individual or group effort, the process of evaluation is the same. Your goal is to ensure that participants have demonstrated their mastery of learning outcomes measured by the assignment. In doing this, you want to be fair and consistent to everyone. A systematic, analytic approach can help you do this.

Develop Learning Outcomes

To assess a participant, the first step is to clearly define and state what it is the participant must be able to “do.” These statements have various names: learning objectives, performance objectives, and learning outcomes. (We shall use the term “learning outcome.”) Learning outcomes describe the knowledge, skills and/or attitudes the learner is expected to have when they have completed the course, lesson, or program. It is a good exercise to construct a learning outcome, and then set the criteria to measure it. Doing this will help you focus on the critically important components of participant assignments when you begin marking them: that which is stated in the outcome should be what you are evaluating.

Clearly stated learning outcomes allow you, as the facilitator, to have clear criteria against which to measure participants and provide feedback. Clear outcomes also inform participants of how they will be measured. Your role is to help participants acquire skills and knowledge that will help them on the job. By providing feedback on how they are doing, you help them focus on areas where they need to improve to meet the learning outcomes.

Create Marking Criteria

Set Criteria

In order to assign a mark, even with a pass/fail type of assessment, you need to set criteria for measuring participants’ success at achieving the learning outcomes. Marking criteria are sometimes referred to as rubrics, marking guides, marking scales, marking schemes, or scoring schemes. The important issue is not the name, but how well the criteria are developed so as to be fair and consistent. Established criteria will help participants self assess and give them guidance as to what is expected for a given grade.

When you develop scales for your own courses, you can set the marking scale to whatever makes sense. For example, you can use a one to five scale (excellent, above average, acceptable, unacceptable) or a pass/fail scheme.

What’s important is using it consistently.

Establish Measurement Scales

Before you can evaluate discussion contributions you need to develop criteria against which you can measure them. Developing criteria is easier if you have specific learning outcomes to work from. One form of measurement is a Likert scale. These scales are usually designed on a 1-4 or 1-5 value scale: 1 is the top level and 4 or 5 is an unacceptable level of work.

The analytic approach means that before you begin marking each assignment you determine what should be in each response, and then apply that determination to each of the assignments you mark.

For example, if you are going to assess your students’ efforts in a discussion forum, you might follow these steps:

Step 1: Create (or identify) an ideal response.

• An ideal response might be one that you and experts in your field would consider excellent or really, really good. Crafting an ideal response means asking yourself what ‘excellent’ looks like.
• An “ideal” response is one to which you would assign a grade of 9–Response Meets Expectations. (You may choose to challenge learners to add more detail and ultimately exceed expectations. That’s your choice. Learners, though, must have the option of doing what is required and nothing extra.)

Step 2: Analyze the components of the ideal response.

• You may find it easiest to list what is required in the form of questions to be answered.

Step 3: Determine what absolutely must be present for the response to be satisfactory, and what can be missing or partially present.

• If a critical component is missing, then the response will be incomplete.

Step 4: Read responses and assign points.

A special note about Assessing Participation:

Participation is not attendance, we don’t give marks just for showing up. Maybe it’s better to call it “Contributions” rather than “Participation” in order to escape the trap that we sometimes fall into which is trying to justify marks when a student has done nothing other than be present.