Summary

A primary role of the DBES is to facilitate course transformation projects as a means of developing faculty expertise around teaching and learning. Focused course development will typically involve deep collaboration between the DBES and faculty (and often a faculty working group), as well as drawing on results from the education literature and data from the course. During the course transformation, the DBES can act like a partner and tactful coach for the teaching faculty, taking on some of the legwork and developing materials but gradually turning these tasks over to the faculty member. This chapter outlines the resources and processes in developing a course using a backward design framework, beginning with the establishment of learning goals, followed by assessments and effective pedagogy, and ending with documenting and disseminating the work. Sustainable use is often the biggest challenge in this work, and ongoing communication with departmental faculty throughout the project is key.

What is a course transformation?

To undertake a course transformation, a guiding framework is important. You may make other choices in your initiative, but the SEI used the backward design framework, in which learning goals, assessments, and instruction are aligned across the course.

1. Learning goals: what should students learn? Develop clearly articulated learning goals at both the course and topic levels to guide student learning and faculty teaching.
2. Assessments: what are students learning? Develop measurements which allow assessment of student achievement of the learning goals, both before and after the transformation.
3. Instruction: which instructional approaches improve student learning? Develop instructional techniques, including active learning approaches, that foster deliberate practice so that students develop expertise related to the learning goals. This includes pre-class, in-class, and post-class methods (e.g., clicker questions, lecture, worksheets, pre-class reading, homework, etc.).

Successful course transformations include thoughtful application of these three items within the context of a course. For a full description of the elements and steps of a course transformation in the SEI, see Course Transformation Outcomes and Timeline.

Below are some examples of this in practice, primarily in CU Boulder’s Department of Physics.

Course transformation resources

This chapter only briefly touches on a complex subject. We have documented our accumulated wisdom in enacting course transformations within several resources listed below. Additionally, see Chapter 6: Setting the Stage for discussion of how the department can lay a productive groundwork for course transformations.

Instructor Guidance Page

Our actively managed, online instructor guidance collection includes information on course transformation, the use of clickers, a video collection, and several other white papers.

• Course Transformation Guide. Our Course Transformation Guide is a detailed PDF which summarizes principles of learning and suggestions for effective design of learning goals, assessments, and class activities. (This is also reprinted in the Appendix of Wieman [2017]).
• Clicker booklet. An Instructor’s Guide to the Effective Use of Personal Response Systems (Clickers) in Teaching is a PDF booklet with effective practices for writing questions and facilitating discussion using audience response systems.
• Example videos. Our SEI video collection includes demonstrations of implementing different active learning strategies, along with supporting documents.

How do you start the project?

This section describes the interactions between DBESs and course instructors once course instructors have made the step to engage in a course transformation, typically during a planning term prior to the start of the course transformation. Sometimes course instructors expect the DBES to come up with ideas, but course instructors need to direct the project based on their own interests in order to achieve maximum impact on faculty and students. To ensure everything works smoothly during the flurry of the teaching semester, the departmental director should help facilitate the structures and planning (see Chapter 5: Departmental Leadership). See Chapter 8: Partnering with Faculty for more general advice for DBESs about working with and coaching faculty, and see Chapter 9: DBES Development for information about establishing trust and credibility in the department.

It is not the job of the DBES to engage faculty in the initiative, at least not at the start of the project. Rather, the department director should be identifying potential faculty, generating agreements, and setting clear expectations for faculty planning to work with the DBES on course transformations. By the time the DBES is in contact with a course instructor, the instructor should have (in principle) agreed to make changes in their course. See Chapter 6: Setting the Stage for detail on how the departmental leaders can outline clear expectations for a faculty member’s engagement in the project

Determine the course instructor’s interests

While the DBES isn’t responsible for brokering the initial agreement to engage, the course instructor may not have a very good idea of what they have signed up to do. The DBES will need to be prepared to learn about the course instructor’s interests and what they perceive as problems in their course, and show them examples of course approaches. Your main job during the planning term is to get to know the course instructor. Once you understand what the course instructor is comfortable with and interested in, you can proceed.

Meet with the course instructor(s) prior to the course transformation

For a course requiring a major redesign, it is highly desirable for the DBES and course instructor to start collaborating some months before the course is taught to develop learning goals and to begin designing course materials and assessments. During this time you can establish your personal and professional credibility by demonstrating your expertise, sound judgment, and ability to be a supportive team member. You might see yourself as a project manager and cheerleader for the faculty work during the rest of the term. See Chapter 8: Partnering with Faculty. See also Chapter 6: Setting the Stage for advice on planning the work by departmental leaders.

Meet with instructional assistants

If a course includes learning assistants and teaching assistants (undergraduate and/or graduate), they can be powerful allies. Learning assistants and/or teaching assistants have eyes and ears in the classroom, and may be well-versed in pedagogy (for example, through taking a course). Thus, they can suggest changes to faculty, inform you about what is happening in the course, and provide instructors with direct access to the student perspective. Include them in meetings with course instructors during the course transformation. This potential role of instructional assistants is a good reason for you to undertake teaching assistant training projects (which many in the SEI have done).

Facilitate consensus among multiple course instructors

Often there is more than one course instructor involved in a course transformation. In these cases, it is important for the DBES to focus on supporting group consensus as much as possible. In some cases, one person can be particularly keen on the project and more highly involved. This isn’t necessarily a bad thing, especially if that member has some authority in the course, such as a course coordinator.

Meet with other department faculty

Discuss the course instructors’ priorities for the course and their learning goals for students. We recommend having these discussions individually first, followed by working group meetings, to establish relationships and identify faculty interests democratically. It is particularly useful to have discussions with faculty who have taught the course or teach succeeding courses. See the SEI Research Interview Guide for tips on conducting faculty interviews.

Observe the course

You can also observe classes, establishing and documenting how the course is operating at the outset; this will make it much easier to describe the changes made as part of the transformation. This familiarity with the course and where it is currently effective or ineffective can help shape your plans for what to tackle and when. This is also an opportunity in general to start practicing documentation of the course transformation.

Start collecting data to support and guide efforts

The DBES will often be involved in observing and assessing student learning in the course one or two terms before it is transformed. There are valuable opportunities to collect data at the start of a project that will not be available later. In particular, don’t miss opportunities to capture data on student learning in the course. Document important knowledge and skills that the students need or are assumed to have at the start of the course. Gathering such baseline data is often challenging due to a lack of time or available assessments and the fact that course instructors may not want to measure a ‘bad’ instance of their course.

Determining what to measure can be part of your early conversations with the course instructor. (What do they think students should learn and what data would be most interesting to them?) You can also start collecting ideas for what to look into from the literature (e.g., what misconceptions do students tend to have with this course content?) and from previous course instructors (e.g., what kinds of essential problems do students seem unable to solve at the end of the course?). Some kinds of assessments will take substantial time to arrange or have immovable deadlines (like a start-of-term survey), so you will want to figure out what needs to be measured as early as possible. That said, using a quickly developed assessment of a few important learning outcomes is much better than nothing.

Use your departmental director to negotiate conflicts

If you find that the course instructor is resistant or balks at every proposed change, speak with your departmental director. Your director can help renegotiate the course transformation agreement with the course instructor.

How can you best use meetings with faculty?

DBESs can use several types of faculty meetings to engage faculty and use their expertise to further the course transformation work.

In all cases, it is important to use effective facilitation techniques, active listening, and thoughtful questions to keep the discussion on track. See the documents above for suggestions on how to ensure that all faculty voices are heard and that the group process is effective. For meetings with faculty in general, document your conversations and consider asking the faculty member(s) to review your notes to see if you understood correctly, and to highlight any agreements or decisions that were made.

How do you design learning goals?

Learning goals define what a student should be able to do as a result of learning about course content. (Note that learning goals, outcomes, and objectives were terms often used interchangeably in the SEI, though others make distinctions between them.) One of the key roles of the DBES is to facilitate the development and/or improvement of such learning goals to serve as strong guides for the development of the course. Students often respond very positively to having clear learning goals for a course and come to rely upon them (and notice their absence) once they become normative in the department.

Note, however, that while learning goals are important, they can sometimes be a difficult place for faculty to start, and so you may wish to use smaller, more accessible changes as a starting point early in a collaboration. See handouts below for recommendations on developing high-quality learning goals and working productively with faculty.

In some cases, DBESs have also facilitated discussion of learning outcomes for curricular alignment and the major as a whole. Useful resources for this process are at Degree Programs: Design, Development and Assessment from the Science Education Resource Center.

How do you assess student outcomes?

There are many measures of student outcomes that can be gathered to guide course development and/or gauge success of the transformation. Choose assessment measures that are the most appropriate for the learning goals developed and are also the most important to the faculty. This way, the assessments can guide faculty efforts. While DBESs may carry much of this work, it should be done in collaboration with the teaching team for it to have the greatest impact.

Below are some key strategies for developing assessments to measure student prior knowledge, document student learning, and provide feedback to faculty.

Collect baseline data

There are some crucial opportunities for data collection at the outset of the course transformation. This can be a challenge, but it is necessary to help uncover where the existing course is ineffective, which will guide choices in the transformation.

Draw on ideas from elsewhere

See the Course Transformation Guide for information on specific assessment strategies. For detail on developing validated conceptual inventories which survey student learning (akin to the Force Concept Inventory), see Development of Validated Instruments to Measure Student Learning of Expert-Like Thinking (Adams and Wieman, 2010). The Earth, Ocean and Atmospheric Sciences department at UBC, one of the most successful departments in the SEI, has many specific examples of course-specific assessments. Your DBES community will also be a valuable source of ideas.

Use a variety of data sources

Validated pre-/post-survey measures are extremely useful, especially in cases where the course transformation results are intended for publication. Make sure to use the data that is most readily available and will guide improvements. Validated conceptual assessments are useful, but may be impractical to develop. Alternatively, instructor-developed measures, such as high quality exams, diagnostic tests, or carefully developed activities or homework are equally helpful for guiding improvements. You may need to seek guidance from those experienced with educational assessment on writing and later coding such problems.

Measure affective outcomes, such as student confidence, enjoyment, interest, sense of belonging, and other perceptions related to the field of study, as these are important outcomes for promoting persistence and learning.

Include faculty perspectives

It is essential to talk to the course instructor(s) about what is the most interesting to look into as well as what they think students should learn. Ask the instructor(s) whether there is any data on student learning, attitudes, etc. they are particularly interested in seeing. What are students expected to know at the start of the course? What topics or skills do the students have the most difficulty with? Do they have examples of past student work that are available for review? You can also gain valuable insight from previous course instructors and instructors of follow-on courses. See the SEI Research Interview Guide for tips on conducting student interviews.

Include student perspectives

Be sure to talk to students as well. If you have not done something like this before, you may be surprised at how much you learn from the process; there really is no clear substitute for this. Try talking to students informally after class or after an exam. Interviewing students individually or in small groups can yield great insights into their learning and course experiences, especially in combination with other data sources. What students say is also very powerful in affecting what instructors do—anonymous interview data can be a powerful tool in convincing faculty to try something new. That said, be sure to also measure systematically what students think via surveys and careful tests, rather than relying on anecdotes or common beliefs. Note that students can become survey fatigued; be mindful of other ongoing student surveys across the department or campus and coordinate when possible.

Document, document, document!

Observe courses before and during the transformation and take field notes. See the SEI Course Observation guide for guidance in conducting casual observations and PhysPort’s Assessment page for a list of formal observation protocols. These will help structure your thinking and recording, which makes later interpretation much easier. Similarly, write detailed summary notes immediately after a student or faculty interview, as well as meetings with faculty members or teaching teams, while things are clear in your memory. Even if you are unable to address all the observations and ideas you collect during the teaching term, these notes will be extremely valuable in your reflections once the term is over. You will be the only person collecting such data, and you will not remember details later.

Summarize your findings

Distill your findings into a concise and relevant summary, and communicate them to both the faculty you are working with and the department in general (in collaboration with the department director) to generate greater engagement. The summaries will also be of use to anyone interested in results in later years (including future you!), and can form the basis for a relevant external publication.

How do you develop course materials?

Development of curricular materials is best done in partnership with faculty, though DBESs may do a little more work at the beginning of a partnership in order to give faculty a model to start from. Some suggestions for creation of curricular material are below, many of which involve professional development of the DBES (described more fully in Chapter 3: DBES Success) and collaboration with the faculty member (see Chapter 8: Partnering with Faculty).

Design with learning goals in mind

This will ensure that the course design is coherent and aligned with what the faculty value.

Develop your own knowledge

Keep abreast of the research literature to create a solid knowledge of possible instructional strategies. Many campuses have teaching and learning centers that host teaching workshops, which can be very useful for gathering ideas. Your DBES community will also be a valuable source of ideas. And observe other classes: sit in and watch to gather ideas, especially if you know that a class is using something innovative.

Meet frequently with the faculty member throughout the term

Often the course instructor’s role is to develop drafts and examples, and provide materials to the DBES, while the DBES’s role is to review literature, convene meetings, and give feedback in a more facilitative role. Typically, DBESs and the course instructors will have regular weekly meetings to share thoughts and feedback on the course. It is important for you to be flexible and be able to adjust to the faculty member’s priorities. Meeting agendas might include development of learning goals, brainstorming how to achieve those learning goals, reviewing materials and giving feedback, and sharing results.

Fade out your development of course materials

It is more impactful to coach faculty in developing or changing materials than to do that work yourself. The course instructor is more likely to use the material the next year and use the approaches in future courses. Don’t shy away from doing some legwork in the beginning; sharing the load for the course transformation work with a DBES is part of the benefit to faculty who decide to transform their course, however gradually turn things over to the course instructor. In this way, the new methods become part of their practice, rather than relying on you to develop materials. This idea of scaffolding (smaller expectations at first, more assistance from you) and then fading (raising expectations as expertise is developed, reducing your assistance) is the same idea for faculty development as it is for teaching students.

Review course materials

In order for you to give feedback on course materials, it is best for the course instructor to send you materials at least two days prior to its use in the classroom. You must then provide any feedback in a timely fashion to give the course instructor time to incorporate feedback and to maintain a positive working relationship where your contribution is valued.

Observe the course regularly

A DBES typically attends most or all class periods in order to observe the use of materials and student engagement and learning. Take detailed field notes and consider the use of validated observation protocols, which were developed to collect systematic data on classroom behaviors. See the SEI Course Observation guide for a framework for observing active learning courses, Chapter 8: Partnering with Faculty for suggestions on giving feedback after course observations, and PhysPort’s Assessment page for a list of observation protocols. We recommend using the Course Observation Protocol for Undergraduate STEM (COPUS) due to its ease of use and growing body of available comparison data (Stains et al., 2018).

Practice, practice, practice

The DBES, faculty, and instructional assistants may have to run a certain activity several times in order for it to work smoothly and effectively.

Design for modular, partial use

Having a perfect but complicated course approach can be intimidating to future course instructors who may want to try to use it, and the approach may not be robust to small curricular or other changes in the course. Can activities ‘degrade gracefully’ if they do not go as planned either due to shifts in the student population or to instructor constraints (less experience, reduced time for part of the course, etc.)? For example, do all the worksheets fit into a tight sequence, or can one be dropped easily due to a new time pressure?

When possible, designing course materials modularly can support future sustainability. Detailed guidance on developing course activities is in the Course Transformation Guide.

How do you wrap up the work?

Talk to future instructors of the course

Especially in the first year or two after a substantial course project, make a point to reach out—with assistance from your departmental director—to new course instructors teaching the course. Walk them through the course materials and approach, offering them your assistance.

Disseminate within the department

Make sure to make the work visible within the department to build faculty engagement and celebrate teaching successes (see Chapter 4: Central Organization). For example, present outcomes from the course transformation during faculty meetings, create a departmental newsletter, and reconvene faculty working groups to share results.

Create course materials packages

As noted in Chapter 4: Central Organization, packages of course materials for use outside the institutions are a way of documenting the outcomes of the project. Creating a package of course materials is a necessary (though not sufficient) step towards synthesizing and documenting the outcomes of the project. Create an organized set of materials that can be used by other faculty. Include a set of notes on what was done, what worked, and what areas need further improvement. Examples of such course archive packages are online at our course archives page (archives in Physics may be the most detailed).

Publish and present the work

It is valuable to have some focus on formal presentation and/or publication of the work for your own career development and to share ideas with the broader community.

For further reading

SEI Resource documents:

1. Teaching-related resources from SEI: Resources page at cwsei.ubc.ca
2. Example course packages: SEI course archives page
3. Recommendations for course design: SEI Course Transformation Guide.
4. Full course design steps and timeline in the SEI: Course Transformation Outcomes and Timeline
5. Tips on conducting student and faculty interviews: SEI Research Interview Guide
6. Recommendations for developing learning goals with faculty: Facilitating Learning Goal Discussions
7. Recommendations for facilitating faculty discussions: Facilitating Faculty Discussions
8. Recommendations for effective use of clickers: An Instructor’s Guide to the Effective Use of Personal Response Systems (clickers)
9. Framework for semi-structured course observations: SEI Course Observation guide
10. Demonstrations of different active learning strategies: SEI video collection
11. Examples of course-specific assessments: UBC EOAS research page

Annotated bibliography

Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., Norman, M. K., & Mayer, R. E. (2010). How learning works: Seven research-based principles for smart teaching. San Francisco, CA: Jossey-Bass.

Outlines major principles for how students learn.

Chasteen, S., & Perkins, K. (2014). Change from within: The Science Education Initiative. In M. McDaniel, R. Frey, S. Fitzpatrick, & H. L. Roediger (Eds.), Integrating cognitive science with innovative teaching in STEM disciplines [E-reader version]. Retrieved from https://openscholarship.wustl.edu/circle_book/7/.

Gives a step-by-step approach to course transformation with an upper-division physics course as a concrete example.

Chasteen, S. V., Perkins, K. K., Beale, P. D., Pollock, S. J., & Wieman, C. E. (2011). A thoughtful approach to instruction: Course transformation for the rest of us. Journal of College Science Teaching, 40(04).

Early paper describing the SEI model of course transformation, similar to Chasteen and Perkins (2014).

Chasteen, S. V., Wilcox, B., Caballero, M. D., Perkins, K. K., Pollock, S. J., & Wieman, C. E. (2015). Education transformation in upper-division physics: The Science Education Initiative model, outcomes, and lessons learned. Phys. Rev. ST Phys. Educ. Res., 11, 020110.

Describes the CU SEI outcomes of transformations in upper division physics in the most detail.

Felder, R. M., & Brent, R. (2016). Teaching and learning STEM: A practical guide. San Francisco, CA: Jossey-Bass.

A practical and readable guide to instructional design and implementation in STEM, full of useful examples.

Fink, L. D. (2013). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco, CA: Jossey-Bass.

A practical classic for instructional design, including course planning, learning goals, and instructional approaches.

Handelsman, J., Miller, S., & Pfund, C. (2006). Scientific teaching. New York, NY: W.H. Freeman.

Describes the scientific approach to teaching, which underlies the SEI model.

Jones, D. J., Madison, K. W., & Wieman, C. E. (2015). Transforming a fourth year modern optics course using a deliberate practice framework. Phys. Rev. ST Phys. Educ. Res. 11, 020109.

This paper describes a course transformation project, with an explicit focus on deliberate practice.

Kober, N. (2015). Reaching students: what research says about effective instruction in undergraduate science and engineering. Washington, DC: The National Academies Press.

This resource identifies the department as the unit of change (albeit using the SEIs as an example) and provides general methods about instructional effectiveness. Chapter 7: “Creating Broader Contexts That Support Research-Based Teaching and Learning” is particularly relevant to SEI-like initiatives.

Stains, M. et al. (2018, March 30). Anatomy of STEM teaching in North American universities. Science, 359(6383), 1468-1470. doi:10.1126/science.aap8892.

Analysis of COPUS course observation data across a wide variety of STEM courses.

Wieman, C. (2007). Course transformation case study. Retrieved from http://cwsei.ubc.ca/resources/files/Course_transformation_case_study.pdf.

One of the earliest course transformation projects, undertaken by Wieman and McKagan, focusing on modern physics.

Wieman, C. (2017). Improving how universities teach science: Lessons from the Science Education Initiative. Cambridge, MA: Harvard University Press.

Authoritative resource on the SEI model and its impacts, including a detailed description of the rationale for the SEI, lessons learned, and data on departmental outcomes.

Wiggins, G., & McTighe, J. (2005). Understanding by design (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.

The original ‘backwards design’ model of instruction which inspired SEI course development approaches.