Duration of initiative: 2011-2017
Case study written: Spring 2018

Our efforts to reform STEM education at Brown University took the form of an interdepartmental community of practice supported by instructors, staff, administrators, and department chairs, with the potential to engage many courses across the institution. Through the AAU Undergraduate STEM Education Initiative project, instructors at Brown implemented ‘problem solving sessions’ (PSS) as a replacement for recitation sections to foster more collaborative small group experiences and to build more community among students. For 4 academic years, the community of practice built around Brown University’s AAU Undergraduate STEM Education Initiative project (called “Changing the Culture of Introductory Science”) worked to improve retention in STEM fields by redesigning 11 courses from 4 departments.

In this case study, we describe in detail Brown’s context and our project’s evolution, and we highlight successes to be emulated as well as lessons learned. From our experiences as a project site, we advocate for community and interconnectedness across an institution as the central themes for success when embarking on a STEM education reform journey.

Project demographics

Brown University

• Institution type: urban, Ivy League, research-intensive private university.
• Size: ~9,400 students, and ~4,700 faculty and staff.
• Of note: first in the Ivy League to establish an engineering program.

Departments involved in the initiative

• Eligible departments: all STEM departments.
• Participating departments: Physics, Chemistry, Applied Mathematics, and Engineering.

Historical context

Reason for the initiative

Student retention in STEM disciplines had become a growing concern at Brown, with some departments, notably Physics and Chemistry, retaining less than a quarter of initially interested students. Two of the major reasons students cite for leaving STEM are 1) because of the lack of a sense of community with their instructors and peers, and 2) because of their non-ideal experiences in introductory level courses, specifically. However, many faculty teaching introductory STEM courses are hesitant to shift their pedagogy away from the traditional lecture format toward active learning to decrease attrition rates.

Early context

Years before the start of our AAU Undergraduate STEM Initiative project work, a group of open-minded STEM faculty gathered within the flexible and neutral learning space at Brown University’s Science Center and initiated a collaboration to build the interpersonal relationships necessary for productive discussions about pedagogical reform. The community building process began with a series of informal conversations about the challenges facing academics in the sciences, including education reform. These conversations, called Science Fridays, started during the Spring of 2009 as a way to facilitate communication across departmental boundaries about teaching and learning in STEM disciplines, and to provide a space for faculty to converse together without pressure from administrators or students.

Science Fridays continue today, independent of any formal departmental affiliation. There is no formal agenda during the conversations, invitations to attend come from faculty peers, and attendance is completely voluntary. Topics of discussion arise out of the particular interests of the participants with no expectation that the ideas discussed must be put into practice. As they discuss and experiment with evidence-based teaching practices and learner-centered approaches in their own courses, participants become informal mentors to one another, which spreads best practices in a fluid, low-risk manner. This freedom to discuss without specified goals or agendas has led to significant course-level changes that arose organically from the conversation and were tailored to the needs of the participants and their respective departments. Over the course of years of discussion in Science Fridays, the collective group was able to identify and develop mathematics competency as both a requirement for success and barrier to retention in STEM disciplines.

The AAU Undergraduate STEM Education Initiative Project

In 2011, the AAU launched a five-year initiative to improve the quality of undergraduate teaching and learning in STEM, entitled the AAU Undergraduate STEM Education Initiative. The initiative sought proposals demonstrating solid and actionable plans to change the culture of undergraduate STEM education through evidence-based practices, rather than reinventing the wheel in course after course and wasting time re-proving techniques like active learning or hands-on experiences. The conversations and community around STEM education reform that grew from Science Fridays created a strong platform for the AAU proposal; the chairs of several departments as well as the staff at the Science Center were poised for action when the call for submissions to the AAU Undergraduate STEM Education Initiative was announced. The acceptance of our proposal, Changing the Culture of Introductory Science, made Brown one of only 8 institutions to be designated as a project site (out of 62 AAU institutions total).

A single postdoctoral fellow served as the Science Education Specialist (SES) across departments

A project-specific postdoctoral researcher (equivalent to the DBES in the SEI Handbook) provided pedagogical training, met regularly with instructors, observed PSS, and coordinated data collection and analysis across all courses within the project scope. Instructors and facilitators met weekly or bi-weekly with the SES for consultations on implementing problem solving sessions, observation debriefs, and reports on the data analysis results of ongoing course evaluations.

The initiative focused on introductory and required courses

All courses chosen for redesign were either introductory courses or required courses taken by a large portion of students entering into majors within the associated departments. To address challenges of large enrollments and limited staffing, and to build confidence and community among Brown students, the stakeholders involved in this project reimagined introductory and required courses to include thoughtfully constructed, small group experiences in which instructors made a concerted effort to show real connections to the lecture material, develop community, and provide more active, context-rich practice with the vital mathematical skills needed to succeed in STEM disciplines and careers. The first departments to participate, Chemistry and Physics, were part of the original AAU proposal to improve students’ competency with mathematics. As additional faculty and departments became aware of the project, they were allowed to join without a formal proposal process. This resulted in the inclusion of Engineering and Applied Mathematics.

The initiative focused on integrating problem solving sessions (PSS) into recitations

The common strategy used for each of the courses involved the integration of PSS into what was previously a large, traditional lecture-based instruction format, sometimes without any type of recitation. This was because efforts to redesign STEM courses at Brown University focused on aligning the redesigns with the institutional culture, rather than pushing against it. Faculty at Brown often pride themselves on sharing their research by incorporating cutting edge topics into their lectures. Thus, project work centered on innovative activities outside of the lecture to preserve faculty members’ preferred platform to interact with their students.

While there were variations depending on course context, in general, PSS were between one and two hours long and held in flexible learning spaces with hexagonal tables and movable whiteboards designed for collaborative groups. Students worked in teams of two to four with about one facilitator for every three to five teams in a session. PSS facilitators, including faculty, postdocs, graduate students, and undergraduate students, met weekly with the postdoctoral researcher to develop worksheet packets comprised of discussion-provoking problems deeply rooted in real world contexts. These packets were designed to provide students with enough problems to work at their own pace during the session, as well as additional problems to practice outside of the session. Facilitators encouraged students to take their time to understand each problem before moving on.

The purpose of this emphasis on deeply engaging with some content, rather than rushing through all the content, was to reduce students’ performance-related stress within the problem solving session environment. At the beginning of each session, facilitators briefly introduced the content covered by the packet, either via a two- to five-minute lecture or with an introductory problem that students worked on individually before discussing with their teams. Facilitators then circulated around the room, encouraged intra- and inter-team discussion, expanded on problems with relevant disciplinary context and their own expertise, and modeled useful classroom practices. These evidence-based teaching practices included diagramming problems on whiteboards, avoiding controlling language and behaviors, making frequent short visits to student teams, asking exploratory questions rather than providing answers, and giving students space to struggle and discuss with one another. These behaviors were emphasized during facilitation training and were observed by the SES over the duration of the project.

A Community of Practice among course instructors was developed

To connect project work across the various involved departments, a Community of Practice (CoP) was developed for course instructors, including faculty, postdocs, graduate students, and even undergraduate teaching assistants. Many of the CoP’s activities revolved around professional development for instructors, enabling them to develop awareness about and reflect upon how their courses impact students, both cognitively and affectively. Training materials and resources for this community drew upon resources from the Sheridan Center for Teaching and Learning, research literature on collaborative learning, and firsthand accounts of instructors and facilitators from previous semesters about their challenges and triumphs (after Fall 2013). Participants in the project-supported training commented that it was nice to have a mix of departmental backgrounds and experience levels within the training environment; that is, faculty appreciated having students in the room, physicists appreciated having engineers in the room, and so on.

What were the lessons learned from the initiative?

Centralized STEM education expertise was critical

Using the Science Center as a ‘discipline-neutral’ space to house project discussions allowed for a broad range of stakeholders to participate in the evolution of project activities over time. The Science Center also kept any particular department from receiving special treatment or taking ownership of the SES; rather than having a PHYS education specialist or a CHEM education specialist and leaving the other three departments on the periphery of the project, all interested instructors, regardless of discipline, received a proportional amount of support for their activities from the SES. Further, the targeted expertise of the SES was pivotal in supporting instructors’ pursuit of improved pedagogy and filling in the gaps for instructors who were splitting their time between their teaching, their disciplinary research, their service, and their discipline-based education research (DBER). By shouldering the responsibility of staying up-to-date on and communicating the DBER literature relevant to project activities, the SES reduced the ‘drinking-from-a-fire-hose’ frustration that faculty new to the DBER landscape often face and streamlined the overall implementation of the project to avoid reinvention of the wheel in each newly involved course.

End-of-term changeovers of department chairs disrupted progress

Because the project relied heavily on the support of department chairs and because department chairs serve for fixed terms, the replacement of those department chairs whose terms ended in the middle of the project caused disruptions in courses where the instructors were less engaged. For example, in one physics course, because the instructors of the course were very engaged in the project, the changeover of the department chair mid-project had almost no impact on the progress of project implementation, data collection, or data analysis. However, in another course in a different department, the changeover of the department chair caused a significant disruption and loss of a portion of a data set because the course instructors were neutral or even hostile to the project without the sustained urging of the previous department chair (despite the new chair being similarly committed to DBER). To address this issue, committed department chairs should be coached to not only advocate for DBER from their position of authority within their department, but also cultivate engagement in their faculty, rather than relying on their position to coerce faculty into implementing evidence-based practices the faculty themselves do not believe in or connect with personally.

Exclusively project-owned spaces save logistical headaches

The flexible learning spaces used for project activities were shared for non-project purposes, many of which were unsupervised undergraduate study sessions of various types. This shared use resulted in instructors experiencing several logistical headaches using the space. For example, markers, erasers, and other whiteboard-relevant supplies often went missing, leaving instructors to scramble to find enough for their PSS of 30+ students to work together on whiteboards during the session. Further, instructors of PSSs that met in the morning would often arrive to find the room in a sub-optimal or lecture-like arrangement and would then be forced to delay the start of their session to rearrange the room; some of these instructors had physical limitations that made it difficult for them to move the sometimes un-wheeled hexagonal tables, and would have to recruit their own students to help. To address this issue, having spaces that are exclusively reserved for project activities and managed by the project staff could save time and increase the positive effect of instructors using the space.

Faculty accountability was low

Because course modifications focused on recitation, the lecturing faculty had no responsibility to participate in the project or to make changes to their lecture. In some cases, there was limited to no connection between the lecturing faculty and the SES because a separate head instructor was assigned to the PSS portion of the course. While this ‘lecture-and-PSS-as-separate-entities’ arrangement was sometimes the only way to implement the project work in a given course, the student experience suffered because the experience within PSS was predictably disconnected from the experience within lecture. To address this issue, lecturing faculty should be welcomed and encouraged to attend PSS planning and consultation meetings even if they are not formally responsible for any portion of the PSS experience for students and/or are uninterested in incorporating evidence-based practices into their lecture. However, if this type of encouragement is ineffective, an example of a proactive solution adopted in some courses was to have the head instructor of the PSS attend the lecture for the course such that the pacing and content of the PSS worksheets could reflect the pacing and content of the rest of the course. This arrangement bypassed the faculty accountability problem despite the lecturing faculty never attending any of the PSS planning.

Junior instructors were sometimes forced to ‘mentor up’

The nature of the project meant that many of the graduate students and postdocs who were recruited or chose to become involved were the junior instructors most interested and engaged in thinking about DBER in their own teaching. The more senior instructors (usually faculty) were less experienced with DBER and sometimes only involved because they had been assigned one of the courses that had been previously involved in the project. This relative difference in DBER experience level, coupled with the fundamental hierarchy of a primary instructor overseeing a group of TAs, meant that it fell to some junior instructors to serve as DBER mentors to their senior instructors in some contexts. For the most part, this dynamic did not pose a significant problem, but in the minority of instances where the junior instructor was highly knowledgeable about DBER and the senior instructor was both inexperienced with and hostile to DBER, it fell to the SES to mediate a very tenuous situation; maintain the engagement of the senior instructor without crushing the enthusiasm of the junior instructor. This mediation work was draining for both the SES and the junior instructors.

A lone SES as a project’s main change agent is not ideal

The SES faced several challenges as a lone change agent, including the lack of time to fully direct the project and the lack of a peer network. Managing ongoing course feedback, data collection, and data reporting, as well as mediating the personalities of diverse groups of people, was more than a full-time job. In general, the SES spent between 50 and 80 hours every week on project activities; this time included approximately 10-15 hours of observations, 5-10 hours of course consultations, 5-10 hours of other project-relevant meetings, 10-20 hours of data collection/analysis, 10-20 hours of email/in-person scheduling/mediation/organizational work, and the remainder writing/drafting/editing project-relevant websites/reports/manuscripts. Further, the SES was an island; there were no other members of staff that had a similar job description, even in a different disciplinary context, so there was no peer with whom to commiserate or troubleshoot. The SES also often lacked the guidance of the lecturing faculty in a particular course because of the separated nature of some PSS course arrangements. Without the capacity to bounce ideas off of another person well-versed in the DBER literature (i.e., without a research group equivalent) most of the project activities came from only one perspective, that of the lone SES. To address this issue, we recommend hiring at least two project-dedicated SESs and housing them within a more defined structure in a similar center to the Science Center. With the full support of two (or more) people, the overall management of the project would not only be distributed to a more reasonable 40-hour work week, but also the distributed work could be completed with more attention to detail and time for creative reflection.

What were the key outcomes of the initiative?

Problem solving sessions have become part of the infrastructure of multiple courses in multiple STEM departments. The departments have bought into these approaches.

A drive has emerged from the upper administration to infuse problem solving as a form of critical thinking within the Brown curriculum. Our AAU program required both a bottom up and a top down commitment as well as a department to department commitment to the goals as laid out by PCAST and AAU. The associated interactions have led the upper administration to embrace these goals more generally for Brown students. Concretely, the Sheridan Center for Teaching and Learning at Brown is conducting a Problem Solving Course Development Institute (PSCDI) to assist faculty members adding new problem solving activities to their courses. Moreover, a staff member was hired to implement this broader strategy and is currently teaching a course on problem solving for developing an ongoing stream of facilitators.

Students in STEM at Brown are beginning to look for problem solving session opportunities in their courses. The majority of them report that they are key vehicles for their learning.

How do I get more information?

Website: https://www.aau.edu/stem-status-report-engagement-aau-universities.

This website provides the STEM Status Report from the AAU, posted on October 3, 2017.

Website: https://www.aau.edu/education-service/undergraduate-education/undergraduate-stem-education-initiative.

This website showcases the overall AAU Undergraduate STEM Education Initiative, including Brown University as a project site.

Related paper: Closing the Achievement Gap in STEM: A Two-Year Reform Effort at Brown University. Full Paper at the 2016 ASEE Northeast Section Conference’s website: http://egr.uri.edu/wp-uploads/asee2016/73-1064-1-DR.pdf.

Contact: for more details please contact David Targan (david_targan@brown.edu).