|
|
|
Program Evaluation -- (Cont'd)
Steps to Carry Out an Evaluation Lay the groundwork for the evaluation -- Learn the answers to three critical questions: Exactly what is being evaluated? Why is it being evaluated? Who will use the information and how? This is accomplished through interviews with key managers and staff, through reading relevant documents, and literature reviews. Formulate relevant evaluation questions -- Meet with client to discuss specific questions to be addressed by the evaluation: program objectives and activities help suggest appropriate evaluation issues. Then set question priority, and submit a preliminary draft of the questions to the client for approval. Next, in close consultation with the client, determine potential indicators for each question (indicators may be thought of as the kind of information that is necessary to answer the evaluation question). A critical step in a summative evaluation is to select the best measures for assessing outcomes. Determine data sources and methodologies -- Knowing the evaluation questions and what information is needed to address them, the evaluator must then decide how it should be gathered. Available data sources suggest appropriate methodologies. For example, program participants and non-participants (i.e., the comparison group) are usually surveyed because that is the most efficient way to gather information from them. Program managers are generally interviewed because they usually aren't too numerous and evaluators need to ask them a lot of open-ended questions to learn about the program. Process evaluations require a detailed description of the program as planned and implemented. The process evaluator needs to understand the day-to-day reality of the program. Much of the focus is on how the program is perceived by participants, management and staff. This calls for an array of methods, mostly qualitative, including a document review; interviews with management, staff and other key stakeholders; observations of the program in action; focus groups with stakeholders and/or participants; and perhaps a survey of participants. Summative evaluations call for a determination of outcomes, thus the central method is usually a survey of participants and a survey of the control or comparison group. A review of any management information system data is also crucial. It is also important to learn the point of view of key stakeholders about the strengths and weaknesses of the program, the continuing need for the program, alternatives to the program, and suggestions for improvement. Interviews and focus groups are the usual means of gathering such information. If program services differ across sites or the program consists of a number of different projects, case studies of selected sites/projects can provide a wealth of important information. |
|
André Paradis Tel: (905) 335-0307 Fax: (905) 335-0080 andre@measureresults.com Webmaster |