Summary

While the structure will vary by institution, some sort of central organization or management is critically important in creating a vision and oversight for the initiative, building community for and training Discipline-Based Education Specialists (DBESs), communicating with stakeholders, and coordinating daily project operations. The central organization in the SEI acted as a highly involved funding agency, soliciting and funding proposals from departments, continually clarifying the DBES role, and providing ongoing oversight of projects through monthly DBES progress reports and regular meetings with DBESs and departmental directors. This oversight can help the central organization head off several common hurdles, such as poor departmental leadership, lack of departmental ownership of courses, and curricular issues, including over-packed curricula or entrenched course design. Troubleshooting such departmental issues is described in this chapter, in “Departmental factors which help and hinder” (Chapter 5: Departmental Leadership), and in Wieman (2017; especially Chapters 4-6).

What is the central organization?

The central organization provides important management for the initiative. This central organization is likely to take different forms at different institutions. In the SEI, a new organization (called “SEI Central”) was formed with its own management structure and budget. At other institutions, the initiative may be housed and managed within an existing unit, such as a teaching and learning center. Regardless of the specific structure, it is important to clearly identify an organization or unit that will be responsible for the tasks outlined in this chapter.

What does the central organization do?

SEI Central provided complete management and oversight for the SEI. This may be adapted at other institutions depending upon local context.

What human and financial resources are required?

The range of duties described above requires significant staff, financial resources, and time. We suggest reviewing our handout on SEI Strategic Planning Approaches, which includes critical questions to consider and external resources to help you plan your change initiative.

Allow 5-10 years for full change

The initial intended duration of the SEI was 5-7 years to allow several departments to engage under a 5-year grant. However, the initiative at UBC lasted for a total of 10 years, primarily due to a staggering of the work over time in different departments. Other initiatives have lasted just a few years (see Appendix 1: Case Studies), either due to grant funding cycles or pilot projects. Because culture change requires a longer-term engagement (Kezar, 2014), a period of 5 years was chosen for the length of departmental grants in the SEI. Carl Wieman considered this to be long enough to accomplish change, but short enough so that the work achieves a certain sense of urgency. That said, achieving a sense of urgency was one of the unsolved problems of the SEI. See Wieman (2017; p. 136).

Sufficiently staff the central organization

The intention of the SEI was to staff the central organization at a minimal level, enabling greater investment in the departmental work. UBC’s SEI Central included the equivalent of approximately 2 full-time staff (2 FTE), with a director (~50% FTE), an associate director (~100% FTE), and an administrative assistant (~50% FTE), along with technical support. This lasted for the initial 3-4 years as the DBES count grew from 7 to 20, after which staffing and the number of DBESs was gradually reduced. Meanwhile, the CU SEI included a director (20% FTE), an associate director (50% FTE), an administrative assistant (50% FTE), and a few temporary research staff positions. This staffing level was insufficient to effectively lead the work, especially once the director, Carl Wieman, left for a position in the White House and the associate director was consequently tasked with the duties of both positions within the existing 50% FTE hours of that position. In organizations housed in an existing unit, ensure that adequate staff time is available for activities directly related to the initiative.

Provide sufficient central budget

When creating a budget for the central organization and project management, be sure to consider:

• central staff
• equipment
• website development
• travel, including conference travel and other expenses, for central staff (travel expenses for DBESs in the SEI were included in departmental budgets)
• budget for events
• visitor hosting
• dissemination of the work, including time to collect, analyze, and publish outcomes from the initiative (which may involve publication fees)

In the SEI, funding for SEI Central accounted for about 20% of the project budget, of which staff salaries comprised the majority.

Provide sufficient departmental budget to create real change

Provide adequate budget for departments; typically, the bulk of the expenses will be DBES salaries and faculty incentives. The full funding amount for the SEI was based on a set of calculations by Carl Wieman (Wieman, 2017), which assumed that for an institution to create major change, it must invest 5-10% of its budget in that change effort. Given this assumption, the total required departmental funding in the SEI would need to be $10M USD for each institution, spread over 5 years and across 5 departments (or $2M per large research department), in order to support the goal of influencing each university’s teaching culture.

This target was largely met at UBC, where average departmental funding was $1.3M spent over 6-7 years, with a total budget for the initiative of $11M, of which $9M was funded by UBC and $2M through donations. At CU Boulder, however, the total budget was $5.3M, and departmental funding averaged $650K. This funding level did not achieve the same level of impact.

The approximate expenses in each category at UBC are listed in the table below, as a portion of the overall $11M budget

How do you solicit and fund effective proposals?

The central organization circulates a Request for Proposals (RFP) to eligible departments, typically addressed to the heads of the departments. Departments initiate their engagement in the initiative by submitting a proposal. The central organization then reviews proposals and make funding decisions, acting as a highly involved funding agency. Be aware that departments simply may not know how to submit a reasonable proposal for educational work and that proposals may not always reflect consensus and/or the vision in the department as a whole. This can make it difficult to evaluate proposals on their potential to enact change in the department. The following recommendations are based on how we addressed those challenges. For example RFPs and funded proposals, see the UBC CWSEI Funding website, CU Boulder TRESTLE site, or Cornell University’s 2016 RFP.

Advertise the initiative

Visit departmental faculty meetings shortly after the RFP is sent out to discuss the research on STEM education and its improvement, as well as to discuss the proposal process. In the SEI, these visits (conducted by SEI Director Carl Wieman) often resulted in good discussions about teaching improvements and undergraduate education at the department level, even when a department did not submit a proposal. Because the RFP was fairly comprehensive, creating such faculty engagement was important in the SEI.

Help departments develop strong proposals

Departments may not write a stellar, fundable proposal at the beginning. It can be useful to provide clear guidance around proposal preparation and specific feedback on how to improve proposals. The SEI often worked with departments to help them develop a proposal with enough specificity to be fundable, which also helped to clarify expectations for the work. In many cases, the SEI funded pilot projects to enable departments to begin some of the work proposed, then provided full funding when the department submitted a more specific and realistic proposal.

Seek out departments with strong departmental leadership and ownership

The level of leadership and engagement in the initiative is important to consider. Note that in the SEI, the faculty reaction to Carl Wieman’s initial visit to advertise the initiative was a fairly good indicator of whether the department would be successful in leading SEI work. Also, the number of grants available was strategically chosen to send a message that funding was likely if a department made a serious effort.

Below are some questions to consider when reviewing proposals:

• Is there evidence of faculty consensus and engagement across the department, or does the proposal seem led by an individual champion only?
• Is there an individual willing to serve as departmental director?
• Are there existing structures in the department (such as a curriculum committee) for overseeing undergraduate education? Are those structures likely to be a support or a hindrance to effecting change? (In the SEI, existing structures were more commonly a hindrance.)
• Is the department distracted by other looming priorities or does the proposal come at an opportune moment (e.g., creation of a new major)?
• How ready is the department for change?
• Are there any red flags? See “Factors that affected departmental success” below and “Departmental factors that help and hinder” in Chapter 5: Departmental Leadership

Below are several common pitfalls and hallmarks of successful departments that we found during the SEI.

Use a proposal structure that includes clear deliverables

The specificity required in proposals enabled more accurate judgments as to the feasibility of the work. Require proposals to contain at least some discussion of which courses will be transformed, a suggested schedule of transformation, and identification of faculty who teach the courses. Such information will provide guidance at the start of the departmental project, and while plans may—and often should—change, working out this level of detail can help to avoid the common problem of faculty who have been assigned to course transformations but are not personally committed to the work, and encourages the department to discuss targets in advance. After a departmental project really gets rolling, new ideas, projects, courses, and interests will emerge and take things in directions likely not anticipated in the proposal, to the benefit of the project and the overall initiative.

Allow faculty incentives within the proposal

Be sure to allow direct incentives for faculty to participate in the work (such as reduced teaching loads, support for teaching or research assistants, and/or summer salary) in order to provide more direct benefit to faculty for their engagement when there are institutional disincentives for teaching improvements. In addition to providing needed support, direct incentives also provide an additional layer of accountability. In the SEI, faculty typically valued such incentives, and so when they were at risk of losing those incentives, faculty were often responsive to making necessary changes in the face of inadequate progress.

Make funding or incentives contingent on progress

You might grant funding on an annual basis, with funding in future years contingent on good progress. In the SEI, this structure provided accountability and allowed SEI Central to make adjustments in the few extreme cases in which a department was not living up to expectations. While in some cases departments changed their approaches when faced with the possibility of losing funding, this threat was not always a real motivator for faculty and departments to change their behavior—especially when money was not being allocated to faculty incentives (so there was no fear of losing them) and when the DBES was not highly valued. Discontinuing funding (or not renewing the DBES position) was, instead, a way to avoid wasting money.

Fund departments at an appropriate level and duration for the work proposed

Adjust the timing, size, and duration of grants awarded in accordance with department readiness to engage in the work. In reviewing proposals, give serious consideration as to whether departments will be able to spend larger amounts of funding productively. While department proposals in the SEI were typically fully funded for five years, most had a duration of six-to-seven years to allow for ramp up (planning and hiring DBESs), and wrap up (archiving and creating structures for sustainability). Allow funding to be carried over from year to year, as projects may take longer than anticipated.

How do you supervise the work?

Set clear, shared expectations between departments, DBESs, and the central organization. In the SEI, lack of shared expectations often resulted in poor follow-through on course transformations, unclear supervision of the DBES, and, ultimately, low faculty and DBES morale. Over time, the following structures were used to inform the partnerships between SEI Central, DBESs, and departments, resulting in better faculty uptake and lower DBES attrition in later years. For more on common departmental issues, see “Departmental factors which help and hinder” (Chapter 5: Departmental Leadership) and Wieman (2017; especially Chapters 4-6).

Oversight of departmental work

Identify clear expectations at the start of the work

The interactive proposal process serves to establish clear expectations for the work. Emphasize that funding after the first year will be contingent on good (though not necessarily rapid) progress. You may also wish to develop a memorandum of understanding (MOU) with each department to formally establish expectations as was done in post-SEI years at UBC. See examples of this and other MOUs in Appendix 1: Case Studies.

Provide additional oversight from the central organization

The central organization can handle training, research advising, convening a cross-departmental community for DBESs, and meeting once or twice per term with departments for progress updates. The central organization should probably not be responsible for hiring and general HR activities related to the DBES, regular meetings with the DBES to discuss progress, advocating on behalf of a DBES within their department, or seeking out faculty to work with DBES on projects. This distinction should be clarified with departments.

Support effective data collection

While departmental groups are responsible for assessing course transformation outcomes, data collection, analysis, and reporting can be facilitated substantially by the central organization, as described below.

Meet with the department to discuss the DBES role multiple times

Don’t assume that one meeting will do the trick; the role will likely need clarification over time. For detail on what is, and is not, expected to be part of a DBES’s position, see Chapter 2: What Is a DBES? Lack of clarity about the purpose and potential contribution of a DBES could result in confusion and wasted time—or worse—between a DBES and partnering faculty. For details on setting expectations, see Chapter 5: Departmental Leadership for establishing the initiative in the department and Chapters 7-8 about collaboration between DBESs and faculty.

Reinforce the value of and work needed for publications

Remind the department as needed that the DBES needs adequate time to write and publish results—if this is not prioritized, it will often fall prey to the more urgent matters associated with facilitating course improvement work. Considering that research is a secondary aspect of the DBES role, the SEI was very effective in contributing to the research literature. (See our full list of publications.) Experience with discipline-based education research (DBER) or Scholarship of Teaching and Learning (SoTL) publishing can vary substantially in a department, and so it can fall to the central unit to advise on DBES research and publication projects. It can be productive to push DBESs to submit a paper when it is ‘good enough’, rather than spending too much time refining it.

The process of DBES supervision

In the SEI, the following reports and meetings helped to support productive oversight and facilitate clear communication between DBESs, SEI Central, and departments. We suggest using some combination of similar mechanisms to stay engaged with the departmental work, ensuring the DBESs’ time and energy are well spent.

Have the departmental director serve as the DBES’s immediate supervisor

It is important to establish the location of the DBESs within a clear chain of command. Who does the DBES report to? Who manages their daily activities? It is critical that the DBES is seen as a member of the department, but the central organization may need to provide professional development for the DBES and oversight for the department’s efforts. In the SEI, a faculty member in the department served as the departmental director, and was charged with overseeing the DBES and the departmental work. This ensured that the DBES was clearly situated as a member of the department and their supervisor was intimately connected with the departmental culture, politics, and daily functioning.

Have DBESs submit monthly progress reports

DBESs should submit brief (one-to-three pages) progress reports monthly, to be shared with the departmental director and the central organization. These reports can then form the basis of monthly meetings while providing an opportunity for DBESs to reflect upon their work and document progress, which can be motivating since progress might otherwise be difficult to see. These reports can be reviewed with particular attention as to whether:

• The DBES was able to effectively manage multiple demands on their time
• The DBES needed access to particular resources (people, literature, etc.)
• The department was setting reasonable expectations for the DBES
• The DBES was wasting time due to lack of departmental coordination or support

Hold monthly departmental team meetings

About once a month, it’s useful for the central organization, the departmental director, and the DBES to meet to discuss progress in the department. The DBES progress report ensures that these meetings are focused and productive. These meetings can be quite informal, and in the SEI they resulted in formative feedback and troubleshooting for the departmental work.

Hold quarterly meetings with the group of departmental directors

Such meetings enable departments to share practices and approaches (such as methods for incentivizing faculty) and to ensure functioning SEI teams within departments. UBC’s SEI Central held meetings among departmental directors, called the CWSEI Working Group.

Organize occasional corrective departmental meetings

You may need to call meetings between the central organization and the departmental director, and perhaps the department chair, to discuss roles and expectations, such as the responsibilities of the DBES (for example, if the DBES is failing to attend training sessions or submit monthly reports).

In the SEI, there were several common (and often predictable) issues that arose within departments, including lack of clear departmental leadership, lack of departmental ownership of courses, and problems with the overarching curriculum. Discussion of these issues is in “Departmental factors which help and hinder” (see Chapter 5: Departmental Leadership), as well as in Wieman (2017, especially Chapters 4-5). Keep an eye out for such issues and address them proactively in these departmental discussions.

How can you create community?

Given the many roles that a DBES can take on (Chapter 2: What Is a DBES?) and the supports required for the DBES to achieve success (Chapter 3: DBES Success), a vibrant professional community is very important for helping the DBES ramp up to maximal effectiveness in a short period. In the SEI, SEI Central provided regular professional development events, social events, a central meeting space, and an annual End of Year Event each year in April to showcase the work for that year, all to foster professional development, sharing of best practices, collaboration, and a sense of community across the SEI. See Chapter 3: DBES Success for detail.

Additionally, DBESs and initiative organizers can make use of national networks to create cross-campus communities. One of the projects building on the SEI, the Transforming Education, Stimulating Teaching and Learning Excellence (TRESTLE), focuses on creating community within and across campuses, and provides professional engagement opportunities for initiative leaders and for DBESs across campus.

How can you lay the groundwork for sustainability?

It is important to plan your initiative with an eye to sustainability and future engagement. It is not sufficient to create change and expect individual faculty to maintain the work. In addition to such cultural changes, you can create a favorable climate for continued engagement by making successes visible, engaging administrators, collecting persuasive data on initiative success, and seeking to build on the work in the future.

Encourage department events to promote early conspicuous successes

Department talks offer a valuable way of promoting discussion and generating interest for future teaching development work. These can be informal (‘lunch and learn’), part of an established series for teaching and learning, or the occasional slot in a department’s colloquium series. These talks can be encouraged (and perhaps initially funded) by the central organization, but it is best if they are organized within the department and clearly supported by the department chair.

Visibly celebrate success through cross-departmental events

Use public events to showcase the work in departments to others at the institution. The SEI featured an End of Year Event, provided as an example in Chapter 3: DBES Success. This is a valuable opportunity for DBESs and departments to engage professionally around teaching and learning. You may also wish to invite speakers from other institutions to highlight a topic related to ongoing work or to seed new ideas for course transformation and research studies. These are a further channel for informal interaction across departments for those who attend and provide valuable networking opportunities for DBESs and faculty.

Collect data on the initiative

Collecting data on the impact of the initiative can be challenging for a few reasons. If there are multiple departments involved in the initiative, you will need to work with each set of department leaders to collect this data. You will also want to collect data at several points, to see how the initiative progresses over time (e.g. baseline data, data during implementation, and data on sustainability post-initiative). Departments may engage at different times, and thus you will need to plan for collecting baseline data several times.

In the SEI, we assessed the following:

• Changes and outcomes from course transformations
• Changes in faculty teaching practices
• Changes in departmental attitudes and structures
• Outcomes from the initiative as a whole

Please see our handout, SEI Assessment Metrics for a list of indicators used in the SEI.

Engage with higher administration

It will be important to highlight success to higher administration at the institution; in many cases this will be the funding body for the initiative. Having regularly updated reports and summaries can be very helpful in bringing new administrators up to speed on the current state and success of your initiative.

Ideally, your dean can be a champion for the work. Support at the dean’s level is critical for creating institutional priority for a multi-departmental initiative, especially given the traditionally low prioritization of teaching improvements. Such engagement by the dean will require regular updates from the central organization.

Deans may be particularly interested in the perspective offered by UBC Dean Simon Peacock on the UBC CWSEI (Dolan et. al., 2016). Dean Peacock describes how the UBC CWSEI dealt with university policies (such as teaching evaluations) and how UBC required first year courses to develop learning goals as a support for SEI efforts.

Consider supporting the public archiving of course materials

Creating a package of course materials is a necessary (though not sufficient) step towards synthesizing and documenting the outcomes of departmental projects. Many transformed courses were accompanied by an organized set of materials, usable by other faculty. See Chapter 7: Course Transformation.

Course packages were collected in public archives in a custom database and online at the CU SEI’s Course Archives page. One challenge in creating such public archives was that faculty are often reluctant to share materials for a few reasons. First, they may not have ownership of all materials, as some items may have been borrowed from other sources (other faculty in the department or commercial publishers). Second, they may not be willing to share material that isn’t yet perfected, which is an unrealistic goal. These challenges may be mitigated by having an eye to public dissemination early in the initiative, addressing ownership issues in the early expectation setting phase, and carefully documenting the sources of all materials.

Another challenge was that there was typically no clear incentive or structure for creating or maintaining these course packages. In the SEI, it was difficult to urge faculty and DBESs to contribute materials and include the package in the public archive. Once the DBES completed their position, maintenance of the public course package was rare.

Lastly, it was difficult to create a single, centrally-used system to make the archiving process easier for DBESs and faculty. Considerable time and expense was devoted to creating an online system, which was poorly used in the SEI. To house course materials, we currently recommend either collaborating with another established campus unit (such as a library or IT department) or using a system that is already familiar to departmental faculty, such as Dropbox, Google Drive, or a page on your institution’s website with downloadable zip files.

Seek future funding or engagement with national networks

After the initiative is over, it will be useful to find ways to continue to engage those who were involved and continue to spread the work. This may include seeking external or internal funding to continue the initiative or related work, connecting to national networks (such as TRESTLE), and creating ongoing events or opportunities for faculty to work on course transformation. For information on how the SEI continued engagement post-initiative, see the UBC CWSEI and CU SEI entries in Appendix 1: Case Studies.

For further reading

SEI resource documents:

1. Teaching-related resources from SEI: Resources tab at cwsei.ubc.ca
2. Recommendations for planning an initiative: SEI Strategic Planning Approaches
3. Example requests for proposals and funded proposals: SEI Funding website
4. SEI publications: Full list of publications on SEI website
5. List of indicators used in evaluating the SEI: SEI Assessment Metrics
6. Example course packages: SEI course archives page
7. Alternate example course packages: Page of downloadable zip files at CU

Annotated bibliography

Accelerating Systemic Change Network. Retrieved from http://ascnhighered.org.

The Accelerating Systemic Change Network (ASCN) is a network of individuals and institutions, formed with the goal of more quickly advancing STEM education programs. The website includes literature resources, a blog, and other information about higher education reform.

Carey, S. J. (Ed.). (2015). Navigating institutional change for student success in STEM. Peer Review, 17(2).

This special issue, sponsored by the PKAL institutional change project, provides guidance for campus leaders on developing local capacity, assessment, and strategic planning—including a readiness tool for assessing capacity for change.

Dolan, E. L., Lepage, G. P., Peacock, S. M., Simmons, E. H., Sweeder, R., & Wieman, C. (2016). Improving undergraduate STEM education at research universities: A collection of case studies. Tucson, AZ: Research Corporation for Science Advancement. Retrieved from https://www.aau.edu/key-issues/improving-undergraduate-stem-education-research-universities-collection-case-studies.

Includes a chapter by UBC Dean of Science Simon Peacock on the SEI in which he describes how the UBC CWSEI dealt with university policies (such as teaching evaluations) and how UBC required first year courses to develop learning goals as a support for SEI efforts.

Eckel, P., Green, M., Hill, B., & Mallon, W. (1999). On Change III: Taking charge of change: A primer for colleges and universities. An occasional paper series of the ACE Project on leadership and institutional transformation. Washington, DC: American Council on Education.

A practical guide to leading change within an institution, including strategies and information on supporting change agents.

Elrod, S., & Kezar, A. (2016). Increasing student success in STEM: A guide to systemic institutional change. Washington, DC: Association of American Colleges & Universities.

A step-by-step guidebook for campus leaders at the department or institution level for improving student achievement in undergraduate STEM.

Fry, C.L. (Ed.). (2014). Achieving systemic change: A sourcebook for advancing and funding undergraduate STEM education. Washington, D.C.: Association of American Colleges and Universities. Retrieved from https://www.aacu.org/sites/default/files/files/publications/E-PKALSourcebook.pdf.

A practical sourcebook that discusses the rationale for change in higher education, building institutional capacity, changing faculty practices, and tracking and sustaining improvement.

Kezar, A. (2009). Change in higher education: not enough, or too much? Change: The Magazine of Higher Learning, 41(6), 18-23.

Discusses the challenge of change in college campuses, using insights from studies on change leadership. Also discusses the importance of change agents.

Kezar, A. (2014). How colleges change: Understanding, leading, and enacting change. New York, NY: Routledge.

A more detailed scholarly volume on the topic of leadership and academic change.

Kotter, J. P. (2012). Leading change. Boston, MA: Harvard Business Review Press.

Describes eight change strategies for leading top-down change within an organization. This model informed the SEI approach, and there are many short articles describing the general approach. While not fully accounting for emergent change, this is a useful model.

Kotter, J. (2012, November). Accelerate! Harvard Business Review, 45-58.

Describes eight processes that can help organizations accelerate change, such as developing a sense of urgency and strategic vision.

Walter, E. M., Beach, A., Henderson, C., & Williams, C. T. (2015). Describing instructional practice and climate: Two new instruments. In G. C. Weaver, W. D. Burgess, A. L. Childress, & L. Slakey (Eds.). Transforming Institutions: Undergraduate Stem Education for the 21st Century. West Lafayette, IN: Purdue University Press.

Describes the Survey of Climate for Instructional Improvements (SCII), a tool which can be used to measure department-wide attitudes and norms regarding instructional practice.

Walter, E. M., Henderson, C. R., Beach, A. L., & Williams, C. T. (2016). Introducing the Postsecondary Instructional Practices Survey (PIPS): A concise, interdisciplinary, and easy-to-score survey. CBE-Life Sciences Education, 15(4).

PIPS is an instrument which may be used to measure instructional practice and change.

Wieman, C. (2017). Improving how universities teach science: Lessons from the Science Education Initiative. Cambridge, MA: Harvard University Press.

Authoritative resource on the SEI model and its impacts, including a detailed description of the rationale for the SEI, lessons learned, and data on departmental outcomes.

Wieman, C., & Gilbert, S. (2014). The Teaching Practices Inventory: A new tool for characterizing college and university teaching in mathematics and science. CBE-Life Sciences Education, 13(2), 552-569.

The Teaching Practices Inventory was used to measure instructional practice and change in the SEI.

Williams, C. T., Walter, E. M., Henderson, C., & Beach, A. L. (2015). Describing undergraduate STEM teaching practices: A comparison of instructor self-report instruments. International Journal of STEM Education, 2(1), 18.

A review of instruments which may be used to measure instructional practice and change.