Three assessment techniques comprising:

1. the one-minute paper,
2. documented and discussed problem solution, and
3. student-generated problems were applied as active steps in the teaching/learning of the course "Research in Broadcasting" during Spring 1994.

A total of 23 students were enrolled in this class. Modifications were made in the assessment techniques during the semester in terms of frequency and timing of the techniques so that they coincided with the introduction of new or complex material. Observations of learning improvement were assessed in a number of waysfirstly, on the basis of the number and nature of questions posed in the one-minute papers over the course of the semester, secondly, through a pre- and post-course assessment of the understanding of ten communication concepts, thirdly, comparing the approximation of class/instructor teaching goal inventories and finally through the appropriate application of research concepts in the group research projects.

The use of assessment techniques enhanced student learning.

One-minute paper
This technique appeared to be successful in helping students better understand the subject especially when new concepts were being introduced.

The technique was applied six times and the students were asked to write down questions for which they requested further clarification. Students took the one-minute paper seriously and used it as an opportunity to ask for explanations, elaborations and to raise points including their assessment of the techniques. The concerns were primarily about the new vocabulary they were learning e.g. gross rating points, cost per thousand which were then addressed in the subsequent meeting.

The procedure continued during the semester and seemed to help students be more expressive and start asking questions orally in class. Toward the end of the semester, fewer students turned in one-minute papers even though the majority thought that it was "a valuable tool." Other comments included:

...this technique was very effective for me to tell you exactly what areas I needed help on.

...one-minute papers were helpful in going over or recapping the confusing concepts during the next class.

...I thought your use of feedback was very helpful. It gave everyone a chance to voice their worries/concerns/problems.

In one assessment near the end of the semester, two papers were received. One student wrote, "I have no questions, but thank you for asking." The other said, "At first, the one- minute paper was good. After a while, I didn't care about it. In fact, I didn't even fill it out. I liked all the class assignments." Perhaps, by this time the students had learned the material. They praised the use of the one-minute paper and stressed that it should continue to be used.

Documented and discussed problem solution
The use of a problem solution assessment technique helped students to better understand probability sampling and ratings computation.

During the fall semester, two assignments were developed in which students would analyze problem solutions in class. The first focused on the analysis of different sampling designs which they would identity in various publications. Each student would bring an example to class, describe the study and identify the factors which characterized it as a probability or non probability sample.

The second assignment involved a mathematical exercise in the computation of and application of ratings. Student trials of the assignments during the fall semester revealed positive responses and learning improvement for the sampling design problem solution but hostility toward the computation assignment. Comments received in the one-minute papers during the fall indicated that students were angry at having to do a "math" assignment when they were not in a math class. This assignment which required computing and projecting market ratings from results of a telephone coincidental was therefore not included as an assignment in the spring semester but as an example of how calculations and projections are made. Another similar assignment was developed as a class exercise.

One-minute papers indicated students' positive attitude and learning:

...In-class problem solving was useful! You kept it moving along, which is important. I usually hate it when students are expected to teach themselves through 'guided discovery.' But if the instructor retains control, as you did, it can work.

...Problem solving was a good way to make us think and participate.

...This technique was great! It forced us to work out real-world problems among ourselves.

...Application of principles!

Student-generated problems
Student groups working on research to address student generated problems led to active class engagement in conceptualizing the research studies, fieldwork, statistical analyses, report writing and project presentations. Examples of student- generated problems included assessing the impact of NBA advertising on sports purchase, surveying listener interest in an AOR alternative, tracing the evolution of Syracuse's urban contemporary radio station, evaluating the success of pay-per-view television and testing the accuracy of ratings data.

One-minute papers showed positive responses to the student generated problems:

...Group project was valuable in working together to decide what to do, how to do it, evaluating it, etc.

...This project was a fantastic experience for me to combine what we learned to come up with practical research. It was effective to see and implement these concepts. We all worked well together, too.

Overall, learning of specialized communications knowledge occurred.

The teaching inventory comparison between the instructor's assessment of importance and the class' extent of learning specific skills showed a close approximation (overall average difference of 0.31 ) for the different skills groups. See accompanying charts. Students rated discipline specific skills the highest in amount of learning attained (3.75 on a 1-5 scale where 1 was "not at all" and 5 was "a great deal"). This was followed by higher order thinking skills (3.71) and work preparation skills (3.59). These skill groups were similarly ranked by the instructor (4.25, 4, 3.88).

Results from the pre- and post course assessment of the understanding of research concepts showed that understanding improved for the ten concepts listed: sampling error, syndicated research, probability sampling, response rate, Nielsen Station Index, gross rating point, household rating, telephone coincidental, audience availability, and people meter. See chart, "Understanding of concepts." These concepts covered the general areas of methodology, sampling, computations and research fundamentals. Post-course understanding increased by an average 1.58 from 2.88 to 4.43 on a 1-5 scale where 1 represents "never heard of it" and 5 represents "able to explain."

Further learning improvements can be made for "sampling error" which received a post- course mean score of 3.61.

In the presentation of the group research projects, students appropriately used the concepts learned in the course of the semester: probability sample, quota sample, convenience sample, household and person ratings, census validation, response rate, sampling error, telephone survey, telephone coincidental, etc. They were also able to assess the validity and reliability of their data.

Teaching and learning
Student learning improved when the effort was made to find out what and how students think of the way the topics are presented and when more relevant explanations are offered to students in terms the latter can understand. Each class is different in terms of composition, mix of talent, background, motivation and level of performance. Instructors need to adjust to the level of the class being taught. One-minute papers were a non-threatening means for students to dialogue with the teacher without subjecting themselves to the scrutiny of their class colleagues. The technique also provided the instructor with feedback which was used to clarify concerns raised and improve the class experience.

Classroom assessment
The three techniques (one-minute paper, documented problem and solution, student generated problems) applied during this course were valuable tools in motivating students to more actively participate in the learning process. Students appeared to appreciate the effort expended and the interest showed in obtaining feedback and providing strategies for greater student involvement.

Impact on the class
Students seemed to be more satisfied with their classroom experience, learned and remembered more. They assessed their learning experience positively. Overall spring class evaluations increased statistically though not significantly from the fall figures. One-minute papers indicated that:

...The class as a whole was extremely beneficial and was a positive experience.

...l feel the objectives of the class have been met.

From the instructor's perspective, the use of classroom assessment techniques was a very rewarding effort. The one-minute paper was an extremely useful device to both assess student learning and obtain feedback on instruction. It was convenient to administer and the comments obtained were well worth the effort. All three assessment techniques applied will continue to be used in this and other classes. Also, the assessment information will be shared with other colleagues interested in enhancing the student learning experience.

[back]