|
Home >>
Program and Project Evaluation •Planning the Evaluation
Stakeholders
Identification of the primary stakeholders and their key concerns is
essential for the development of an appropriate evaluation plan. Determining the audiences
who are interested in the results of the evaluation and how the information will be used by
the various groups will guide all aspects of the evaluation. For example, evaluation results
may be used by an administrator as a basis for financial decisions, while staff may use the
information for implementing changes to the program. These differing needs must be reflected
in the evaluation questions and information collection strategies, in order to provide
results that will be of interest to each of the stakeholders.
Possible stakeholders include:
- Administrators
- Outside funding sources
- Program staff
- Program participants (i.e., students)
- Faculty
- Parents
- Alumni
- Special interest groups
- Community members/organizations
Constraints
Possible constraints need to be considered while developing the evaluation
questions. Constructing an evaluation question that cannot be reasonably addressed due to specific
circumstances sets the process up for failure. For example, an evaluation question that requires
hours of interviews may not be feasible on a limited budget or if results need to be available
within a relatively short period of time.
Possible constraints include:
- Budgetary
- Time
- ‘Political’ situations
- Availability of data sources
Evaluation Questions
The most important element of a successful evaluation is the evaluation
questions. These serve to focus the entire evaluation process and determine the type of information
to collect, the strategy for gathering the information, and the appropriate analysis options. Taking
the time to formulate the appropriate research questions is crucial to providing useful results.
Poorly constructed, inappropriate, or unrealistic questions lead to irrelevant data that do not
satisfactorily address the reasons for conducting the evaluation.
The purpose of the evaluation, the stakeholders, and the acknowledgment of
possible constraints are all important factors that must be considered in the process of developing
the evaluation questions. Additionally, the examination of the goals and objectives of the program
should drive the process. Goals and objectives describe what the program is striving to accomplish.
Goals depict the general programmatic outcomes, while objectives specify more specific outcomes. It
is common to have several objectives for each program goal.
The following is an example of a possible program goal and a set of
corresponding objectives. Based on this information an appropriate evaluation question may be:
Does the program provide students with opportunities to increase their
academic preparedness?
| Goal 1: Program participants will be more academically prepared as a result of their experience. |
| Objectives |
| a. |
Increase the number of program participants who make use of learning resources
(e.g., tutoring, Writing Center) |
| b. |
Increase the number of students participating in study groups |
| c. |
Increase the number of program participants who graduate from
SU |
Budget
A common ‘rule of thumb’ is to allocate approximately 10% of the yearly total program or project
budget to evaluation costs.
Routine costs include, but are not limited to, the following:
- Evaluation plan
- Data collection
- Developing or purchasing evaluation instrument(s)
- Printing (survey, cover letter)
- Mailing (envelopes, postage)
- Formatting on-line surveys
- Conducting interviews, focus groups, observations
- Gathering documents and records
- Data processing
- Data entry
- Scanning
- Programming
- Analysis and Interpretation
- Statistical
- Narrative
- Document and record
- Report Writing/Presentation
On average, CSTL charges $30 per hour for these evaluation services. The design of an evaluation
for a grant proposal, however, is free of charge.
Ethics
The rights of participants involved in an evaluation must be respected and protected. It is
imperative that participants are aware of the following:
- The purpose for the evaluation.
- The role of the participant in the evaluation.
- How the information the participant provides will be used/reported.
- The participant’s involvement in the evaluation is voluntary and s/he may choose to withdraw at
any point in the process.
- Confidentiality will be maintained in the collection, storage, and reporting of the information
shared by the participant.
Syracuse University provides policies and guidelines regarding ethics approval for human
subjects. Information on Institutional Review Board (IRB) approval and participant consent forms is
available on the Office of Sponsored Programs website.
The IRB Handbook: Policies and Guidelines - Syracuse University
IRB Application for Expedited and Full Board Review
IRB – Instructions for Completing Informed Consent
(also includes an example)
|
.GIF)
