APPENDIX C UNIVERSITY OF OREGON ACADEMIC PRODUCTIVITY PLAN ASSESSMENT AND PRODUCTIVITY MEASUREMENT


This Appendix is a close companion of Section IX B of the University of Oregon Academic Productivity Plan. That section describes the annual "University of Oregon Portfolio" and its role in reporting progress in the three main objectives of higher education: 1) The discovery and creation of new knowledge; 2) the application of knowledge in undergraduate and graduate education; and 3) the application of knowledge in service to society. Below is a more complete description of these objectives and examples of operational measures of them.

1. THE DISCOVERY AND CREATION OF NEW KNOWLEDGE

Creative and scholarly research is an integral part of the missions of the University of Oregon. These missions enable the University to attract faculty in a wide range of disciplines who wish to dedicate their careers to the integration of teaching with scholarly and creative pursuits. In a very real sense, therefore, the University is defined by the quality and dedication of the faculty.

How then can the success of the University's faculty in their scholarly and creative endeavors be assessed? The best methods of assessment are those integrated into the structure and normal operations of the University, consistent with the principles of continuous quality improvement.

At the individual level, faculty are evaluated both formally and informally in a variety of ways as part of our faculty retention, promotion, and merit-pay systems. For example, each instructional unit develops specific criteria to be used in merit evaluations of individual faculty in the areas of teaching, research, and service. In addition to internal evaluations, faculty are also often recognized externally by prestigious awards or by appointment to national and international academies, boards, or societies.

At the program or college level, faculty are collectively evaluated during periodic program reviews involving external evaluators. These reviews deal explicitly with research productivity and the reputation of the faculty within each program. At this time, detailed manifestations of faculty research productivity (e.g., books, refereed papers, performances, commissioned works, scholarly research grants, national and international visibility) are assessed within the context appropriate to the particular discipline. Significantly, program reviews also examine the integration of faculty research with the graduate training of students, the quality of graduate students matriculating in departments, the extent and quality of graduate student research, and the pattern of success for graduates of the program. In some instances, program reviews are conducted by accreditation societies or boards. Beyond these periodic program evaluations, many disciplines present occasional evaluations of the productivity and reputation of research departments in the discipline. These offer independent, although often narrow, evaluations of research productivity.

At the University level, detailed evaluation criteria are more difficult to apply owing to the diverse nature of our various programs. Thus, evaluations of University research productivity must rely, in the main, on appropriate evidence from the component programs. In some instances, however, aggregations of particular assessment measures (e.g., counts of scholarly research grants and recipients of prestigious awards and appointments) may be meaningful. University membership in the prestigious Association of American Universities, as one of only fifty-eight universities, clearly marks our research distinction, as does our attraction to many of the best students and postdoctoral fellows from around the world.

Operational Measures of the Discovery and Creation of New Knowledge

2. THE APPLICATION OF KNOWLEDGE IN UNDERGRADUATE AND GRADUATE EDUCATION
 
A. Undergraduate Education

Providing high quality undergraduate education is a central component of the University of Oregon's mission. Since the mid-1980s efforts have been made nationwide to measure the effectiveness and the efficiency of undergraduate education. In its March 1992 report, the OSSHE task force on assessment stressed several key principles to keep in mind about such measurement. Most importantly, the report stressed that assessment must match institutional mission. Further, assessment must make use of multiple indices of outcomes. Institutions must resist simplistic quantification of outcomes such as comparisons of performance on standardized tests, even if those tests are achievement based; moreover, it is completely inappropriate to make statements of outcomes based on testing instruments designed to measure aptitude.

To measure the productivity of our undergraduate teaching, we will use a wide variety of instruments, evaluation approaches, and indices of enrollment and retention, student progress, graduation, and overall satisfaction.

Enrollment and retention measures address such questions as: Does the institution enroll and retain students who are capable of benefiting from the resources on campus? What are the entry indicators used for success? Do these indicators and the recruitment efforts produce a cohort of students well suited to the educational program of the institution? What level of attrition is appropriate to the institution?

Measures of student progress toward degrees deal with such questions as: Are academic requirements well articulated and coordinated with the resources available to allow students to meet those requirements in a timely fashion? Are there indicators of intermediate achievement that provide coordination of student needs with necessary resources? Are learning and teaching resources marshaled in an efficient and timely manner to ensure that students are learning actively and are progressing toward graduation? What barriers are there to student progress and how are they overcome? What is the breadth and depth of undergraduate learning and achievement?

To measure graduation productivity, not only is a count of graduates important, but also assessment of the quality of their learning and of the learning experience, consistent with the University's goals and missions. Methodology can include retrospective (or statistical cohort) studies of alumni, assessing their professional achievements and long- term satisfaction.

B. Graduate Education

University instruction of graduate students involves teaching, research, and creative activity at the highest level. Graduate teaching and graduate programs are an extension of the University's research mission and an essential component of outreach to the community, State, and nation. Graduate teaching and research inform the undergraduate programs in the disciplines by focusing faculty and student effort on the most current scholarship, technology, and creative endeavors. The University's graduate programs enable faculty and graduate students to pursue the creation of knowledge while training the next generation of professionals in business, industry, government, and education. Graduate education is a fundamental component of the University's tripartite mission of teaching, research, and service.

Assessment and productivity measures for graduate education include enrollment, entry indicators, special recruitment, graduation achievement, employment and placement profiles, extramural funding, student satisfaction and experiences, student satisfaction and experiences, teaching innovations and evaluations, student research and publication, faculty teaching load, and faculty-student collaboration.

Operational Measures Of The Application Of Knowledge In Undergraduate And Graduate Education:

A. Undergraduate Education

B. Graduate Education
  • Teaching evaluations
  •  Program review
  •  Teaching load
  •  Special instruction (e.g., seminars, minicourses)
  •  Individualized instruction or supervision
  •  Dissertations and theses chaired
  •  Dissertations and theses served on


    • 3. THE APPLICATION OF KNOWLEDGE IN SERVICE TO SOCIETY

    The third higher educational objective may be evaluated by measures of significance to the University and to OSSHE and of service to the institution and society.

    "Significance" here describes the degree to which various departments, schools, colleges, and programs form the fabric of the University's missions and charter, and to which various units in the University interweave in this fabric to meet the needs of the State and society in general. Measures of significance directly reflect the University's statement of purpose. The values of the measures are likely to change little, if at all, over time, but they are important factors in University, OSSHE, and legislative planning.

    Service to the university, and parallel activities in the public interest, are obligations of every faculty member. Such service normally comprises two elements:

    According to the University Charter, the president and the professors constitute the University, and the faculty are the immediate government of the institution. Because of this long-standing tradition of faculty governance, the University relies upon effective, regular participation of faculty in many aspects of University life beyond teaching, academic advising, and research. Academic and administrative service internal to the University includes membership on faculty and governance bodies and service on departmental, college, school, University or State System committees or task forces. Such commitments can be substantial, including: service on the University Senate; curriculum development; preparation for University or departmental accreditation; evaluation of candidate files for admission; review and evaluation of faculty promotion and tenure files; serving on departmental or University search committees; strategic planning; budgeting; intercollegiate athletics; and service on a variety of problem-solving task forces.

    Service and activities on behalf of the larger community and the public are also encouraged and recognized as important to the missions and reputation of the university and its departments. Such community and public service reflects the professional expertise of the faculty member and is often related to professional growth, scholarship and teaching. Examples of service vary widely by academic discipline but include membership on local, state, national or international committees, conducting workshops, giving public presentations or performances, serving on editorial boards, etc. Faculty may also contribute to the formulation of goals and policies for national, professional, and community bodies, and for units of State government. Faculty also consult with a variety of private sector concerns, ranging from computer companies to law firms and banks, from multinational corporations to small businesses.

    Because departments have different ways of providing service, and have different expectations about providing service, the kind and amount of faculty service vary widely from school to school, department to department, and individual to individual. Quality and effectiveness of service, therefore, are difficult to assess. However, because of the tradition of faculty governance and participation in the life of the University, and because community and public service often relate closely to professional growth, scholarship, and teaching, academic and administrative service and activities on behalf of the larger community are best evaluated at the departmental level. Currently, this assessment is done at tenure, promotion, merit, and post-tenure evaluations, with service always one of the criteria considered.

    Operational Measures of the Application of Knowledge in Service to Society:

    A. Significance to the University's Missions
    4. CONTEXT: ADMINISTRATION, SUPPORT SERVICES, FUNDING, AND RESOURCES

    The research, teaching and service activities of the University take place in a context of administrative, academic, and student support services, provided by available resources. We plan to monitor the level and quality of administrative and support services, as well as to measure funding and resources.

    Administrators within the University play many roles. One of their functions is to act as mediators of the communitarian interests of the faculty, staff, and students, and, in particular, to apportion resources. They must also exercise leadership, by persuasion and by personal example. Senior administrators, such as the president, vice presidents, and deans, also represent the University's fiduciary and legal interests and are spokespersons to external audiences. In all of these roles, administrators must be principled, ethical, and informed.

    Support services form the backbone of the University. They include a wide range of student services, as well as all of the individuals and offices needed to make a large enterprise function. Secretarial staff, the Library, and the Physical Plant Department are a few of the more visible examples of support services; the list includes dozens of different types of activities.

    Funding issues are central to a discussion of productivity. In the case of the University of Oregon, the charge from the Chancellor to develop a productivity plan began from the belief that "we must challenge ourselves to teach the same number of students that we did in 1991-93 with fewer resources available."

    As a public institution, the University has the obligation to provide access to as many qualified students as is feasible. Access is affected by the total funds the State is willing to invest, tuition policy, and other funds such as scholarships and loans. Measures which concern public policy makers might involve the ratio of tuition to state dollars, and the ratio of state dollars plus tuition to outputs. These ratios sometimes also take the form of costs per student or per external dollar raised or per program or per SCH, since costs must be funded from a combination of revenue sources, by far the largest of which are state appropriations and tuition.

    Ultimately, improvements in productivity or efficiency should result in dollar savings for the same level of output, a higher level of output for the same number of dollars, or a higher quality output for the same number of dollars. Since the number of dollars available will be constrained by a combination of legislation and policy, careful monitoring of the impact of the elements of the productivity plan on these funding/cost ratios will be an essential part of making the plan produce the desired outcomes. Research-intensive universities absolutely must invest major funds in computers, networks, and educational technology. This requirement will certainly drive up the cost of higher education, however it is measured, but the alternative is to have graduates untrained in state-of-the-art technology and thus unable to compete in the changing job markets of the world.

    Operational Measures of Context:

    [Submitted by: Charley Wright
    Tue, 29 Mar 94 14:56:32 PST] [Copyright 1994, University of Oregon]
    Web page spun on 25 June 2002 by Peter B Gilkey 202 Deady Hall, Department of Mathematics at the University of Oregon, Eugene OR 97403-1222, U.S.A. Phone 1-541-346-4717 Email:peter.gilkey.cc.67@aya.yale.edu of Deady Spider Enterprises