The Nirvana Teacher Center (NTC) understands the importance of having a robust and structured evaluation process for its programs and activities that is cyclical in nature. This process begins with a variety of needs assessments and continues through to a summative evaluation at the end of the year. To that end, NTC utilized the Evaluation Program Planner Worksheet (Appendix C) to plan for an overall evaluation process and the Evaluation Focus Worksheet (Appendix D) to focus the impact evaluation. These two worksheets enable the Policy Board to determine the focus and priorities of the annual evaluation. The audience for the evaluation begins with the Policy Board and Center staff but is not limited to them. An evaluation sub-committee of the Policy Board is responsible for analyzing evaluation data and making recommendations to the Policy Board. This assists Center staff in developing new programs and refining existing programs. In view of the fact that all member districts in the Consortium contribute a per teacher allocation, it is important that Superintendents understand the impact of NTC’s programs and activities on their teachers. Teachers, administrators and our local Legislators are also audiences for the evaluation as evidenced by the completed program planner attached.

Using the Criteria on the Evaluation Planner Worksheet and data from the 2001-2002 End of the Year Report, NTC proposes the following evaluation plan:

1. On average, NTC holds twenty (20) workshops per year. A workshop is defined as a short, one hour to one-day event generally to provide awareness and some knowledge of a topic. Given this number, NTC will ask participants to complete an end of activity evaluation for a selected number of workshops. (Appendix E)

2. During the year, approximately forty (40) courses are offered through the Center. A course is defined as a fifteen-hour event on a topic. Generally, participants gain skills and knowledge and apply that knowledge and skill in their classrooms. For Twenty-five (25) of these activities, the Center plans to have participants complete end of activity evaluations. (Appendix F). For ten (10) activities, NTC will send a follow-up questionnaire (Appendix G) six months after the conclusion of a course. The aim of the follow-up survey is to gather data on the impact of the program on teacher practice and to gather specific information on how teacher perceive observable, positive impact on student performance. By design, some courses will be planned so that the next to last and last sessions are one month apart. At the last session, Instructors will gather information similar to that of the follow-up survey.

3. Instructors are required to complete a questionnaire at the conclusion of the courses they teach. (Appendix H) The Policy Board Evaluation Committee and Center staff review and analyze the questionnaires.

4. This year, we have developed a multi-year evaluation plan in order to better determine impact on teacher practice and student performance. This plan will focus on one program: a new literacy collaborative with __________ College, the National Writing Project and Consortium districts. More specific details of the evaluation are described in Section III of the Continuation Program Plan.

HOW DOES NTC KNOW...

...that teachers and other participants are satisfied with the programs and services provided by the Center.

Back to Top

Evaluation Surveys

On a Likert Scale of one to four with four being the highest, participants rate satisfaction with the programs in the following areas:

a. subject knowledge of the instructor

b. course content

c. impact on participant’s practice

d. benefit to students and

e. success in meeting the Instructor’s goals.

Self-Assessment Questionnaires

In some activities such as the Grants Programs and the Professional Circles Program, NTC has designed self-assessment forms specific to the activity. For example, participants in the Professional Circles Program are required to submit a log of each session and a portfolio of the content of the circle. (Appendix J) The Evaluation Sub-Committee of the Policy Board will review these. In addition, in some programs, participants are asked to produce representative work as evidence of their learning.

Personal Interviews

The Director and/or members of the Policy Board are often present for some or all of many activities. This results in conversation that gives the Director or Policy Board member insight into the program.

Documentation

The NTC utilizes its record-keeping procedures as part of its evaluation process. These include:

a. Course and activity registrations

b. Contact Records. These are logs of use such as telephone logs, resource room logs etc.

c. Course Design/Proposal

d. Course Materials

e. Correspondence and

f. Databases such as the Activities and Mini-Grant required by SED.

Staff /Policy Board Judgment

The Director, Instructors and Policy Board members often discuss the outcomes of a program. Again, these conversations provide insight into the program and may affect its design.

...that each of the Center’s outcomes were satisfactorily met

Each year, NTC determines outcomes as part of its planning process. Activities are planned to meet the outcomes. Each activity has its own goals to be met. After evaluating activities and programs collecting quantitative and qualitative information on each, and after reviewing all of the activity results within an outcome, NTC can assess whether the outcome has been met. For example, NTC will use an excel spreadsheet to collate information for all activities using the Course/Workshop Questionnaire (Appendix K) This information will be used to develop a report. (Appendix L). These reports will assist the Policy Board in determining whether a specific outcome has been met.

...that the Center is meeting its Mission and Goals and is operating according to the Teacher Center Standards.

All of NTC’s outcomes and activities are aligned with one or more of the New York State Teacher Center Standards. As described in Section Two, the Program Narrative, NTC has developed a rubric and an Action Plan for the last two years to assess their performance with relation to the standards. The first year, NTC concentrated on the Governance Standard. The second year, the emphasis was on the Program Standard. NTC will continue to emphasize the Program Standard for the 2003-2004 program year to address those areas that were not addressed fully. The Director uses this action plan as a guidance document throughout the year: and at year’s end, she, along with the Self-Study Committee reviews the year’s progress. Other evaluative tools and methods are used to assess the efficiency of operations, public relations, budget implementation and staffing. For example, each year the Director id evaluated by the Personnel Committee, and the Director, who reports to the Policy Board, evaluates the NTC Secretary. Appendices M/N are samples of the evaluations.

Back to Top

Measuring impact on teaching and learning is a very time intensive process. In developing a plan to measure impact, factors such as staff and resource allocation must be considered. NTC utilized the Evaluation Focus Worksheet to determine the focus for an in depth evaluation that could assess not only impact on teacher practice but also impact on student performance. NTC will study the implementation of the National Writing Project Collaboration as a focus for a three-year study on teacher practice and student performance. NTC will use this summer to design the specifics of the Evaluation; however components will include pre and post test information, paper and pencil surveys, documentation of change in teacher practice through the use of journals or focus groups, and examining student work to determine improved student performance. NTC may also consider observation techniques in its plan. In addition, NTC will utilize the following in an effort to collect data in specific activities in the process described in the overall evaluation plan at the beginning of Section III.

Follow-Up Survey/ Focus Groups

For a specified number of activities, NTC will send the follow-up survey to participants. NTC will inform them at the beginning of an activity in an effort to get a better response return. In at least two activities, NTC will conduct a focus group with some participants and ask questions related to change in teacher practice and in some cases, improved student performance. There are many ways to do this, but NTC will ask participants to bring in evidence of student work to discuss and share with other members of the focus group.

Program Design

Another way to assess change in teacher practice or improved student performance is to design programs in a way that builds in data gathering. For example, if a course is a 30 hour course, space the meeting time over a longer period of time so that teachers can actually report change in practice as evidenced in their classrooms, or if for some reason, participants could not implement change, the reasons why. NTC will design its 2003-2004 program in such a way to meet this goal in at least two programs.

The Policy Board plays an important role in the Evaluation Process. It is most heavily involved in developing goals and criteria and establishing priorities. It is also very much involved in the judgment phase of the evaluation. A sub-committee of the Policy Board reviews much of the data collected and draws conclusions and identifies implications for future programming. Their findings and recommendations are presented to the full Policy Board. In addition, the Policy Board engages in the Self-Study annually. This activity is highly interactive and results in some very fine reflective and responsive practices at NTC.

The Director develops the overall evaluation plan with the help of NTC staff. This plan is then presented to the Policy Board for its endorsement. The Director and staff then carry out the plan: distributing surveys, collecting and summarizing data and planning and conducting focus groups. During the year adjustments to programs may be made based upon formative data. In this case, the Policy Board has opportunity for input at regularly scheduled Policy Board meetings.

The entire process becomes a cycle of determining needs, implementing programs and/or activities, assessing and/or evaluating, modifying if necessary, then a return to identifying needs.

Back to Top