University of Oregon
Lane Education Network
National Telecommunications and Information Administration
Telecommunications and Information Infrastructure Assistance Program
Award Number 41-40-94029

Evaluation Plan

The Lane Education Network is a consortium of educational, government, health care and public interest groups who are creating a high-speed fiberoptic network connection in order to develop applications which will serve their respective clients and constituencies. Evaluating the progress of this network development will have three different aspects. First, the technical component - purchasing equipment, leasing lines, creating connections, and maintaining the network - will be reviewed and reported in quarterly technical reports to the Program Officer at the Department of Commere. Second, partnership meetings will be held periodically to review plans for the development of applications, communicate organizational activities, facilitate connections among faculty, teachers, staff, students and others who can collaborate on demonstration projects, and generally review progress and plan future actions. Third, the demonstration projects will be evaluated by a variety of means, including surveys, outcomes assessment, user counts, and system logs.

Technical reports will be made quarterly, in accord with the requirements of the grant. These reports will be submitted January 15, 1995, April 15, 1995, July 15, 1995, October 15, 1994, January 15, 1996 and April 15, 1996. They will contain:

A partnership meeting was held on October 18, 1994, following a local press conference to announce the NTIA award. A meeting is currently scheduled on November 21, 1994, and we anticipate that meetings of either technical or other staff will be held approximately monthly through the implementation phase. A revised timetable for the project is as follows:

4J workstations in classrooms                           1 July 94 - 1 Sept 94
Symantec router installation                            1 June 94 - 1 Nov 94
Hub site room prep/power conditioner                    1 Nov 94 - 1 Dec 94
Router and network connection: Sacred Heart             1 Jan 95 - 1 Mar 95
Multimedia server installation                          1 Dec 94 - 15 Jan 95
Hub router installation                                 1 Jun 94
Router installation: LCC, 4J, Springfield, Bethel, EPL  1 Feb 95 - 1 Mar 95
Circuit installation: LCC, 4J, Springfield, Bethel, EPL 1 Feb 95 - 15 Mar 95
Access point room preparation                           1 Nov 94 - 1 Feb 95
Access point installation: UO                           1 Dec 94 - 1 Jan 95
Access point installation: LCC, Springfield, 4J, Bethel 1 Dec 95 - 1 Mar 95
Leased circuit upgrades: Springfield, 4J                1 Jan 95 - 1 Mar 95
Access point installation: Eugene Public Library        1 Mar 95 - 1 July 95
LCC network upgrade                                     1 July 94 - 1 Sept 95
network fully operational                               1 March 96
During the implementation period, most of the possible evaluation will be technical milestones: system connectivity established, reliability maintained, equipment installed, access points opened, World Wide Web access page established, developed and expanded. The primary indicator of success will be numbers of users accessing the WWW page and volumes of packets being sent over the network.

As demonstration projects are developed and implemented, each will be evaluated on its own terms. The projects contained in the original proposal had individual evaluation plans included, which can be taken as prototypes of the kinds of evaluation schemes which will be used on subsequent projects. Some of the original projects are changing somewhat as implementation nears. For instance, the Eugene School District proposal to use the network for Chineese instruction has been impeded by the departure of the Chineese language teacher who proposed the project. They are evaluating the idea of providing Russian language instruction instead. These projects are not envisioned to be developed sufficiently to fully evaluate during the grant period. Nonetheless, in addition to evaluating the numbers of users served and volumes of data moved, all demonstration projects will be evaluated for their educational or service outcomes, and new projects reviewed in light of the experience of the previous ones. Examples of the evaluations envisioned are included in the text of the evaluation plans from the proposal.

University of Oregon/Bethel School District Mentoring Triads

The main component of our evaluation plan is centered on changing practices in high school science and math classes, which we plan to ascertain directly by observing instruction and measuring student performance. In addition we plan to determine the indirect effects from this project by interviewing students, teachers, and faculty. Our primary objective is to change the manner in which teachers provide instruction, requiring us to analyze the instructional planning process, review both instructional and training materials, observe interactions among faculty-teacher as well as teacher-student and faculty-student dyads. Finally but most importantly, we plan to measure student learning outcomes. With each of the teacher participants, we will develop a professional portfolio, allowing us to track their development and monitor changes made in classrooms. This portfolio will include samples of projects completed by students as well as their problem-solving efforts.

We will also need to assess teacher, student and faculty perceptions, using both formal surveys and informal interviews. We will evaluate the applicability of project resources, ease of access (both human and technological), classroom use and application, and commitment and support of faculty and project personnel. Finally, we will document a number of systems lever variables to determine how much "take" occurred in the field. For example, the bulletin board would be analyzed for the numbers and types of interactions, the threads in dialogue among teachers, students and faculty, and the source and distribution of material exchanged.

Springfield School District High Speed Communication Infrastructure

The district and university partners will evaluate the effectiveness of this project by:

  1. Tracking the number of users of the information system, the services they accessed, the number of problems they encounter.
  2. Interviewing participating teachers, students, and faculty to evaluate learning, staff development, and changes in classroom practice.
The Eugene Electronic Public Library

Sacred Heart Health System: On- Line Data Sharing with a School- Based Teen Clinic

Currently all statistics and trends regarding teen visits that occur at the High School Teen Clinics are kept in manual logs. After the implementation of the on- line clinic management software, numerous statistical reports will be available to establish benchmark measurements of clinic utilization and overlapping SHHS clinical utilization. The first major evaluation factor that will be reported will be accurate baseline utilization reports to establish usage patterns and assist in school wide health care needs assessments. Surveys of students utilizing the teen clinic will be taken both at the beginning of the school year and at the end of the school year. The educational and treatment work of the teen clinic will then be measured by seeing if the surveys show improvements in beliefs on the part of students that pregnancy, sexual activity, and leaving school are costly to them and if alternative behaviors are being promoted. We will also measure if students feel a greater sense of personal control over their own life situations. Based on the results of this evaluation, other programs will be developed for use on the network to enhance the goal of improved health for teens.

Eugene School District 4J Interactive, Multi-media Language Courses

In addition to measuring the success rate on the SAT Exam, the School District will measure student competencies directly, and will administer surveys to all personnel and students involved. University Education faculty and 4J language faculty will be asked to aid in evaluation and will participate in developing generalized applications from the information gleaned.

Lane Community College Desktop to Desktop Training

Lane Community College will evaluate project and program performance through usage statistics and through collaborative evaluation. College computer laboratory usage is monitored closely at present and this project's electronic classroom likewise will be monitored through automated student check-in and check-out procedures. The services and programs will be evaluated through faculty and student surveys, other end-user surveys, and student performance assessment by Symantec and Springfield School District teachers.

University of Oregon/Lane OnLine On-Line Training Materials

Lane OnLine will establish procedures that will enable it to evaluate its performance and the impact of its services on the community. These will be developed by UO staff and tested using Lane OnLine subscribers, with a pre-test/experience/post-test protocol, as well as survey questionnaires and open-ended suggestion opportunities. Other procedures will include electronic tracking of usage, surveys, and independent research. Because this is a new model for community networking in an era when interest in such services is growing, it is anticipated that Lane OnLine and the Lane Education Network will both become models for other communities. Therefore, LOL bears a special responsibility to insure that its programs and services undergo regular evaluation.