Past Reports

A message from Hayley Cavino New York SIG Evaluator
hcavino@uc.syr.edu
315-443-9219

A message from the NY SIG Evaluation team:

The New York State Improvement Grant (SIG) is now in its fifth year. Over the life of the project the evaluation design has shifted in response to federal and state stakeholder requirements for outcome evaluation of SIG/SPDG initiatives. In particular, the needs of the various stakeholders have served to inform and shape the evaluation response including: the initial logic model, participation in the ongoing National evaluation, the developing New York State performance plan, Federal reporting requirements, and the ‘collective call’ to use scientific and evidence based principles to strengthen programming and evaluation. More information about selected methodological approaches and utilizing a logic model based design is available at:

http://systemschange.syr.edu/lc/uploads/syschg_lc/The_NY_SIG_Initiative.ppt

The purpose of SIG, as it is designed in New York State, is to assist school districts and Institutes of Higher Education in designing and developing the capacity, resources, and expertise to address specific problems in schools related to performance gaps between general and special education students, and disproportionality of minority students in special education classification, placement, and achievement. Evaluation team members from Syracuse University's Office of Professional Research & Development (OPRD) worked with the New York State Education Department, Office of Vocational and Educational Services for Individuals with Disabilities (VESID) and program staff in an ongoing way to develop, review, and refine the evaluation design in each of the programs five years.

The logic model served as a touchstone for the initial evaluation design effort, and continues to serve as the framework from which to plan SIG program activities and concurrently inform the evaluation process. State and federal stakeholder needs have impacted the logic model, the nature of programming offered through SIG, and the evaluation design. In particular – three distinct developments have demanded pro-active and timely programmatic and evaluative responses from both program coordinators and the evaluation team: the National evaluation of SIG State implementation and outcomes, the development and implementation of the New York State Performance Plan, and federal reporting requirements that include measurement of evidence and scientific-based practice. The requirements of these State and Federal stakeholders have a common thread, namely, to improve the rigor of SIG/SPDG programming and evaluation through the development and support of scientific and evidence based practice. The New York State SIG evaluation design has and continues to evolve to meet these changes.

Inspiring Success:

To identify the evaluation methods, evaluation team members reviewed the program logic model, which serves as the framework from which to plan SIG program activities and concurrently inform the evaluation process.  A multi-method approach was further developed and refined to gather information about:

• The implementation of the SIG program,
• The degree to which program activities match stated goals and objectives,
• The strengths of the program to date and lessons learned,
• Challenges to the success of the program and strategies used to overcome these challenges,
• Opportunities for program improvement, and
• Intermediate outcomes related to the programs key strategies and activities.

The following data collection methods were utilized in each of the five years of the project evaluation to date.

Further information about the specific methods indicated above and the primary formative and summative outcomes related to the SIG effort in New York can be found at the link below. The linked document contains reporting related to scientifically-based practices that are sustained though on-going and comprehensive efforts and documentation of different ways of working:

http://systemschange.syr.edu/lc/uploads/syschg_lc/SUMMARY_OF_EVAL_NY_SIG.doc

Further Information:

Further information concerning program evaluation of SIG work can be found at the web-site for the Higher Education Support Center (HESC) at Syracuse University. The most recent annual evaluation report can be found at:

http://www.systemschange.syr.edu/uploaded/2004-2005SIGevaluationReport.pdf