Evaluating the ECS Programming for Children with Severe Disabilities

ASSIGNMENT 2: WHAT MODEL IS APPROPRIATE TO EVALUATE THIS PROGRAM AND WHY?

Based on the limited information provided, the ECS Programming for Children with Severe Disabilities appears to have the goal of helping to improve the functioning of children with severe disabilities in a learning environment either at home or at a center (e.g., daycare, kindergarten). Considering that Individual Program Plans, with clear objectives and goals, are designed for each participant based on his/her needs, the desired outcome of each program may be different. However, one overall goal that should be apparent in each individual program would be to make some sort of gains in the child’s development and/or functioning; in other words, there should be benefits to participating in the program otherwise the program is no longer feasible.

Before finalizing the decision of which approach or model to use to evaluate the ECS Program, a consultation with the stakeholders would have to occur. Stakeholders, or those who are interested in having the evaluation done, should have some input in the direction that evaluation will take. There is likely a reason why the stakeholders have requested that an evaluation be done (such as conflict or complaints from participants, budget requirements, the desire to replicate the program elsewhere, etc.) and part of my job as an evaluator would be to help answer the questions/needs of those stakeholders. Designing the evaluation should be a collaborative effort between the evaluator and stakeholders.

POSSIBLE MODELS/APPROACHES TO THE EVALUATION

The model that I would recommend to evaluate the ECS Program would depend on what question(s) stakeholders want answered through my evaluation. However, the following describes potential models for evaluating the ECS Program and the situations under which each model may be more appropriate to apply.

Summative Goals-Based Evaluation: The Goal is to Determine Whether or Not the Program is Working as Intended

A potential goal of the evaluation may be to determine whether or not the intended objectives and goals of the Individual Program Plans are actually being achieved. There could be numerous motivations for performing a summative goals-based evaluation on the ECS Program. A few examples include identifying areas of the program that should be retained, cut or improved upon in the future, determining whether or not, and to what extent the goals of the program are being reached and the costs associated with obtaining those goals. If the purpose of the evaluation is to examine the extent to which the goals of the program are being achieved, the primary evaluation model that I would recommend would be a goals-based evaluation.

Outcomes-Based Evaluation: The Goal is to Determine if Participants are Benefiting from the Program

Perhaps for publicity reasons the stakeholders would like to examine the efficacy of the programs being applied to each participant. Or, the program may be working so well (or so poorly) that an assessment of participant outcomes is required to make decisions about the future implementation of the program. In these instances, an outcomes-based evaluation may be best suited; the evaluation would aim to assess the benefits and drawbacks of completing the program. In other words, an outcomes-based evaluation would help determine whether or not participants demonstrate improvement from participating in the program and whether those improvements outweigh the costs associated with it. Such an evaluation would help stakeholders decide whether or not to keep or eliminate the program (or certain aspects of it).

Process Evaluation: The Goal is to Replicate the Program Elsewhere or to Inform Stakeholders about the Costs and Benefits of the Program

Perhaps the program has been previously evaluated and has been shown to be quite successful in achieving its goals and there is now a desire to replicate the program elsewhere. Maybe there have been complaints regarding the program, or changes to the program have been made over the years and the expenditure of resources needs to be assessed. In these instances, I would recommend a process evaluation which provides a detailed description of the program’s process, or how the program actually works or runs. The results obtained from a process evaluation could facilitate the implementation of the program at different locations, reveal how resources (including staff, money, etc.) are being utilized and identify potential inefficiencies in the program. If the process of the program is to be evaluated, a process evaluation would be best suited.

Formative Evaluation: The Program is Relatively New and the Goal is to Obtain Feedback regarding Program Improvement

It may be the case that the stakeholders want feedback regarding areas of the program to improve or to test-run the implementation of the program. This would be more likely to occur if the program is somewhat in its infancy and is still being developed. The goal of a formative evaluation of the ECS Program would be to identify areas of the program that could be developed or improved, and to help guide the program towards that improvement.

An Eclectic Approach: There are Numerous Questions to be Answered

Depending upon the purpose of the evaluation and the questions that stakeholders want answered, one model may prove to be better suited than the others. However, it may also be that a more eclectic approach, or a mixture of the models, would be preferred. For example, the stakeholders may want an evaluation of participants’ outcomes (outcomes based) as well as to test run new materials (formative) while attempting to replicate the program at another location (process). Answering all of these questions would not be permitted by strictly following one model while disregarding others; rather a mixture of the models and methods may be the best vehicle to arrive at the desired answers.

Depending on the questions that stakeholders want answered and the resources available to complete the evaluation, the direction and scope of the evaluation may change. My job as an evaluator of the ECS Program would be to first identify the purpose of the evaluation through consultation with stakeholders and to then decide on the most appropriate model or models to follow in order to provide the stakeholders with the information they need to make their decisions.