|
Program and Project Evaluation
Introduction to Evaluation
Planning the Evaluation
Collecting Information
Analyzing and Interpreting
Reporting |
Home >>
Program and Project Evaluation Collecting Information
In this phase of the evaluation plan, information is gathered to
address the evaluation questions, thus it must be relevant and sufficient in quantity.
Although the evaluation questions drive the selection of data collection techniques, there
are a number of factors to consider.
- What information needs to be collected?
- The information that needs to be collected is delineated by the
evaluation questions.
- Who are the information sources? (target audiences)
- People (student participants current, past, withdrawn, perspective;
support staff tutors, counselors; program staff; faculty; parents; administrators)
- Documents
- Records
- Observations
- How much information should be collected?
- Entire population
- Sample of the population
- How should the information be collected? (methodology)
- Surveys (paper, web-based, scan form)
- Focus groups
- Interviews (face-to-face, telephone)
- Observations (events, behaviors, level of engagement)
- Document analysis (program documents, activity logs, student work)
- Record analysis (university student record system, attendance records)
- Testing (pre-test, post-test)
- Literature review
- Other existing data sources (retention data, institutional survey data)
Some evaluation questions are best addressed by using a variety of data
collection techniques. For example, a survey may be administered to gather information from a large
number of people, with follow-up interviews or focus groups conducted with certain respondents to
obtain more in-depth information. Additionally, using several different sources of information helps
to substantiate the findings. For instance, if exploring a tutoring program, a data collection
strategy may include a survey and/or focus groups for students, a survey and/or interviews for
tutors, and a record analysis of students attendance behaviors. Triangulation, or using multiple data gathering strategies from
several sources, helps to more completely explore the evaluation questions.
Although the evaluation questions drive the data collection techniques,
the practicality of the approach must also
be taken into consideration. Time, cost, and scope of the evaluation must be addressed. The amount
of time to develop the data collection instruments (e.g., survey, focus group protocol, document
analysis guidelines), gather the information (e.g., distribute the survey, conduct the focus groups,
review documents), and analyze the data must realistically meld with the timetable of the program
staff and administrators. Budgetary resources must be compatible with the cost of the evaluation.
The scope or magnitude of the evaluation is often closely tied to time and budget. For example, if
an information gathering plan includes interviewing twenty participants but financial resources are
limited and a short timeline exists before information is needed for planning purposes or decision
making, the practicality of the approach comes into question. Although the data collection design is
driven by the evaluation questions, it must also be realistic.
|
.GIF)
