Lorilee R. Sandmann
Director, Community Outreach
Michigan State University
Annette Abrams
Assistant Director, Office of the Vice Provost for University Outreach
Michigan State University
Lester Manderscheid
Professor, Agricultural Economics
Michigan State University
Pat McConeghy
Associate Dean, Graduate Studies and Research, College of Arts and Letters
Michigan State University
Merry Morash
School of Criminal Justice
Michigan State University
Susan Smalley
Program Leader, Extension, Agriculture and Natural Resources
Michigan State University
Diane Zimmerman
Associate Director, Office of the Vice Provost for University Outreach
Michigan State University
Evaluating Quality Outreach
How do we know if what we are accomplishing in higher education outreach is of quality? Business and industry have been developing minimum standards of quality as well as measures of quality enhancement. However, educational institutions and providers are struggling with both the concept and assessment of quality.
In 1993, the Provost's Committee on University Outreach recommended that Michigan State University establish a system for monitoring, measuring, and evaluating outreach. However, measurement processes are premised on a common understanding of quality. Therefore, an eleven-member faculty committee was appointed to further develop MSU's thinking about planning and evaluating quality outreach. This session introduced the committee's work and provided an opportunity for participants to apply the dimensions of quality to case examples.
Instilling a Quality Culture
Rather than beginning to articulate what a quality outreach initiative would be, the faculty committee framed a broader charge. Their work was guided by exploring the following questions:- How do we encourage discussion about what quality outreach means among faculty, staff, administrators, and university collaborators?
- How do we develop a common understanding of what constitutes quality outreach, and the language to describe it?
- How do we assist academic units in articulating definitions and expectation for outreach consistent with their mission, values, and context?
- How do we assist units in planning and evaluating outreach research and teaching initiatives consistent with standards of quality?
- How do we inform the rewards system to recognize outreach achievements in tenure, promotion, and annual merit salary decisions?
- What are suggestions for documenting and reporting accomplishments in outreach?
- Finally, what are the aids that help units communicate, both internally and externally, about their outreach activities and their impact?
- Both quantitative and qualitative indicators are essential for evaluating the quality of outreach activities.
- Evaluation is useful at all stages of the process: for planning purposes; for formative and developmental purposes; and for summative, outcome purposes.
- Evaluation is necessary both at the unit and the individual level.
- Documentation must be tailored for its particular purpose.
Dimensions of Quality
When asked to describe characteristics of successful or outstanding research and teaching outreach initiatives, other symposium participants suggested that they should be timely, addressing an issue that is important to people now. Outreach should be flexible, creative, innovative, and novel in order to meet changing needs, done in a way that says that this is cutting edge or original. Quality outreach addresses both the how and the why. It has instructional and research components, teaching the standards and values of a profession. Outreach goals are designed to be beneficial to university, the discipline, the faculty member and to the constituent, shared, and created through a shared language.Quality outreach makes a difference or meets needs. It has a long-term, overall, sustained impact. Resources should be wisely used, shared with partners and constituents and collaboratively allocated. In fact, there is broad involvement of stakeholders on multiple levels. While meeting the needs of all stakeholders, quality outreach stretches them so that they discover things they didn't know. Objectives are achieved in terms of the learner's goals and exceed expected outcomes. More and more clients expect quality to be there; but they also want to be sure there is real value.
Four dimensions of quality define the MSU faculty committee's definition of quality.
- Significance: Project goals are significant. A target audience is identified. Issues and opportunities are consistent with the university and unit missions. Resources are used wisely.
- Contextualization: Process is matched to the individual situation. The effort is collaborative and sensitive to diversity, and uses comprehensive and appropriate methods.
- Scholarly characteristics and contributions: Timely knowledge is generated, but also applied and preserved. Contributions are original. Clients and peers are included in scholarly assessments.
- External and internal impacts: Impact is made on issue or client and capacity is built for sustainability. Mutual university-constituent/community connections and benefits result from the effort, but also have an impact on the university.
Applying a Matrix of Quality
These four dimensions have been formulated in a matrix that includes suggested components, with examples of both qualitative and quantitative indicators that may be used as evidence and documentation. Participants used the matrix on several case studies, and then discussed the usefulness of the product. Some felt the work represented "an appropriate and helpful integration of both the quality movement and demands for accountability." The matrix illuminated prospective program deficiencies, such as a disconnect with the target audience and the objectives of the project; passivity on the part of students and learners; lack of sensitivity to diversity; failure to involve constituents or stakeholders in the impact assessment; and so forth. The matrix's potential use in program improvement was evident.One participant noted that there was too little documentation in the cases to support a decision. This is typical, and the cases and matrix illustrate the nature of the documentation that would be helpful to make informed judgments about the value of any project. Another felt that the fact that the reported outcomes were uninteresting suggested that the reporting of impacts, both internal and external, needs to be engaging. One felt that outreach quality can be assessed to some extent by using familiar measures already employed to assess teaching and research while another believed that the quality matrix is too comprehensive to be used in all situations.
Some felt that university and academic unit teaching and research programs typically do not receive this type of scrutiny. Perhaps standards of quality being used for outreach are more explicit and rigorous than those for teaching and research. Used as a review tool, the matrix would be very useful prior to submitting a proposal to an outside funding agency. More importantly, the approach can serve as a model for central administration and units to use in planning and assessing all parts of its mission.
"Just because administrators say it is so, does not mean it is," said another participant. That is, just because administrators would like such principles of outreach incorporated into the life of a unit and faculty member does not necessarily mean it will happen. Each unit and every faculty member need to interpret the concept of outreach and its assessment for themselves.