Past Reports

Measuring Up

In the 4 th year of the first 5-year award, the Wisconsin SIG is a complex and multi-faceted undertaking that encompasses twenty-three interdependent--yet independent--SIG personnel development projects. A major challenge in evaluating Wisconsin ’s SIG is to determine across all 23 SIG personnel development projects whether what was intended to be accomplished was actually accomplished at the end of the 5-year award period. Another major challenge is to determine whether independently developed products maintain a consistency of special education policy messages across products.

To evaluate the work processes and quality of Wisconsin ’s SIG project-related activities, the evaluation design is guided by three conceptual frameworks. See paper located at: http://www.wisconsinsig.org/e&a/SIG_EVAL_Conceptual_framework.pdf

Wiscoinsin’s SIG Life Cycle

When viewed as a five-year period of sustained personnel development efforts, the “life cycle” of Wisconsin ’s SIG is conceptually divided into three stages. Stage I --- The “start up” period. During the first-to-second years, the focus was on developing and refining personnel development products. Stage II was given over to the “ramp up” of product implementation and dissemination activities throughout Wisconsin . Attention was given to dissemination, maintaining high quality of delivery, responsiveness to consumer feedback and satisfaction ratings, and refinement of products. Currently in Stage III, Wisconsin ’s SIG is focusing on scaling up and institutionalizing project activities and products.  

Adaptation of CIPP Evaluation Model

Vital Signs

To secure information about the quality and intensity of SIG work related activities during each of the three SIG life cycle stages, “Vital Signs” focus data collection activities for informing each vital sign. The 7 vital signs are aligned with three components of the CIPPI Evaluation Model and find expression of importance within each of the three stages of the SIG’s life cycle. An interactive graphic representation of the current status of Wisconsin’s SIG Vital Signs can be viewed at the Wisconsin SIG Evaluation website located at (click on the vital sign to view data) http://www.wisconsinsig.org/e&a/evaluation.html

Evaluating Consistency of Message Across Wisconsin SIG Products

This component of the evaluation is designed to examine the success of the improved collaboration among agencies providing services to children with disabilities throughout Wisconsin articulated in Goal II of SIG. As new products are released from the 23 SIG projects, each is reviewed by a panel of experts using content analysis procedures. A paper explaining this evaluation process is located at

http://www.wisconsinsig.org/e&a/consistency_of_message_report.pdf

Evaluation for Consistency of Message is conducted in two phases.

First, a consistency of message analysis is conducted to identify and compare the policy messages emanating from each of the 23 SIG-funded project’s products. These project products include information booklets, handbooks, training manuals, PowerPoint slide presentations and other publicly disseminated documents. These analyses seek to identify common policy messages contained in the documents.

Second, as a measure of the consistency about what the SIG projects are advocating within the expanded special education community, a consistency of message analysis is conducted comparing the messages emerging from SIG-funded projects with messages found in the community-at-large. This analysis examines products from Local Education Agencies (LEAs), advocacy groups, community-based organizations and other state agencies, such as Health and Human Services.