Academic Exchange Quarterly Fall 2005 ISSN 1096-1453 Volume 9, Issue 3
To cite, use print source rather than this on-line version which may not reflect print copy format requirements or text lay-out and pagination.
Combining peer assessment and role play into one activity can produce an effective assessment method. Incorporating peer assessment into the project results in reduced faculty grading time and increased student learning. This paper describes an assignment that combined the two, the results, and recommendations that faculty can use to readily adapt the technique to their own content areas.
Good faculty members are constantly seeking ways to both involve students in classes and increase learning. Long lectures can be boring for students, and studies consistently show that discussion-oriented teaching methods result in increased student engagement and learning. Role play, or simulation, is an established teaching methodology that has been used in education for many years (Bernstein[PC1], Scheerhorn, & Ritter, 2002; Taylor & Walford, 1978; Van Ments, 1983). It is viewed as a particularly powerful strategy because it can create learning experiences that are varied and realistic. In role play situations, students are active participants rather than passive observers. Participants must make decisions, solve problems, and react to the results of their decisions. Collier (1998, 2000) has written about the emotional experience for the student and the increased learning as a result of this level of participation.
With the recent increased emphasis on assessment in higher education, a logical extension of using role play or simulation as a teaching method is to use it as an evaluation of student learning. However, a drawback to using it as an assessment tool is the extensive amount of time that is required for the faculty member to review and appraise the performance of each student (Bernstein[PC2], Scheerhorn, & Ritter, 2002). Peer assessment as a method to evaluate student performance has increased substantially over recent years as a way to reduce faculty time, and some studies have reported that it can increase the understanding of students conducting the assessment as well (Bangert, 2003). Peer assessment typically involves students exchanging assignments and providing feedback to their peers through the use of rating scales or checklists (Airisian[PC3], 1996; Topping & Ehly, 1998). This technique has been used to evaluate student performance in a wide variety of disciplines such as psychology (Haaga, 1993), mathematics (Bangert, 2003; Earl, 1986), biology (Falchikov[PC4], 1986), geography (Mowl & Pain, 1995[PC5]), and computer science (Marcoulides & Simkin[PC6], 1991). The project described in this paper combined role play and peer assessment into one assignment used in an education course.
Peer Assessment to Evaluate Role Play
The purpose of the project/assignment described in this paper was to explore the use of peer assessment to evaluate role play in a graduate level higher education administration course. By using peer assessment, I hoped to increase the feasibility and benefit of using role play as an assessment instrument in the classroom. This paper describes the assignment, feedback from students about the value of the project, and lessons learned, and then provides recommendations that faculty can use to implement this technique in their own classes. Feedback from the students was obtained through a written questionnaire completed immediately at the end of the activity.
Description of the Project/Assignment
As a faculty member in an Adult and Higher Education Graduate Program in a 4-year institution located in a rural environment, I decided that our program would provide a good arena to explore combining role play and peer assessment. Students are typically nontraditional age with job experience, and some travel distances of over 100 miles to take courses in the program. Most courses are taught in 3-hour evening time periods or in a weekend format. The assignment was used in “Resource and Program Management,” a required course for students in the program, and a variety of topics are covered throughout the semester. Managing human resources is an area of emphasis, and students are presented with principles of listening and communication skills. As an active learning activity, students are given an exercise designed for practicing communication skills as a manager or supervisor in a role play situation. In the past, students were divided into groups of three and asked to practice a scenario involving an employee with problems on the job. One student acted as the manager, another as the employee experiencing difficulty on the job, and the third student was an observer of the role play. Sessions were video taped and reviewed by each group, and then the experience was discussed as a class. No grade was given to the students.
While students reported learning from the practice sessions, I sought to use the simulations as an assessment of student learning of the listening and communication skills presented in class. In an attempt to convert the activity to an assessment technique, in subsequent semesters I tried grading the role play of each student. However, reviewing and evaluating each tape for a grade was found to be too time intensive to be feasible. A review of literature describing the successful use of peer evaluation to assess student learning motivated me to combine the two into one assessment activity. The role play assignment was first used during the spring semester of 2004. Revisions were made to the assignment based upon student reports and my own evaluation of it, and the assignment was given again during the spring semester of 2005.
To begin the project, students were divided into groups of three and given a five-page handout containing all the material needed to complete the assignment. The cover sheet described the activity in detail and delineated how the project was to be graded. The remaining pages included the observer assessment sheet, the self-assessment sheet, the role play scenarios, and, finally, the student evaluation of the project.
The directions explained that one person in each group was to be the manager, another the employee, and the third the observer and time keeper. The person acting as the manager was to practice the listening and communication skills discussed in class in response to the scenarios acted out by the employee. The observer was to take notes during the role play, lead a critique of the session during a replay of the taped session, and complete the observer assessment sheet. The role of each student rotated so that each individual had the opportunity to be a manager, employee, and observer. Students were instructed that each role play was to last a minimum of 5 minutes and a maximum of 8 minutes.
Fifty percent of students’ grades was derived from their performance as managers, and 50% was based on their observer roles. The manager portion of the grade was determined by both the actual acting out of this role and the self-assessment. In other words, if a student was dissatisfied with her performance in the role play, she could write up what it would take to improve her performance, and this assessment would be factored into her “manager” grade. The observer role was graded in order to motivate students to give serious and quality feedback. It was also hoped that students would learn how to improve their own behavior by critiquing others.
The observer form was a one-page sheet containing a column that listed each desired behavior (e.g., creating a positive atmosphere, paraphrasing, using open-ended questions, probing statements to encourage additional information, etc.). To the left of each desired behavior were four columns that described various levels of performance exhibited for each behavior. Rather than providing a likert numerical rating, each column provided a short description of the behavior in a rubric style, such as “creates positive atmosphere by beginning with friendly small talk,” “begins session in a friendly manner with the business at hand,” “begins session with business at hand in a condescending manner,” and so forth. During the session and after the group critique, the observer circled the words that best described the behavior of the manager. At the bottom of the page was a space where observers were asked to provide examples of statements made by the manager during the role play. Also, the observer was asked to describe the primary “plus” of the interview as well as a “wish.” This feedback structure was designed to keep critiques constructive and to create a positive experience for students.
The manager self-assessment sheet asked the manager to describe “what I did that was a plus” and “my wishes for another time.” Students were required to list at least two pluses and one wish so that they were forced to identify positive aspects of their behavior.
Three distinct scenarios were provided for each group, and from these, students were to choose one to use for each role play; alternatively, they were allowed to create their own scenario of a similar nature. The scenarios that were provided were based upon real-life experiences of the author in previous administrative positions.
The evaluation form was a survey that asked for student feedback about the project itself. Students were asked to rate eight statements using a 5-category likert scale of “Strongly agree,” “Agree,” “Neutral,” “Disagree,” and “Strongly disagree.” The eight statements included the following assertions: “This activity helped me to practice the listening skills discussed in class,” “this activity helped me to improve my listening skills by practicing them,” “this activity should be used in future classes,” and “the observer sheet worked well in assessing how I did.” The student evaluation sheet was anonymous and was handed in separately from the other sheets that were to be graded.
All sheets and tapes were collected at the end of class. Each tape was then reviewed in conjunction with both the self-assessment sheet and the observer sheet for each student.
Combining peer evaluation and role play resulted in a positive assessment technique that both reduced the amount of faculty time required to grade students and was rated by students as a beneficial learning experience. I spent approximately 10-15 minutes grading each role play. In previous classes with no peer assessments, time spent grading each role play averaged 25 minutes and in some cases even longer. I found that the student observers provided excellent feedback to the student managers and, in many cases, gave much more in-depth information than I had in the past in my grading. The observers focused on the positive aspects of the managers’ behaviors; however, they were not reluctant to offer constructive criticism and ideas for improvement.
The students’ self-assessment of their own role play was thorough and well constructed. They provided specific examples of their “pluses” as well as what they “wished” they had said or done. However, they usually listed more “wishes” than “pluses,” so it was important to incorporate the group critiques of the session to provide the students with a more complete listing of their positive behaviors.
Student evaluations of the assignment were positive. Likert scores were all in the “agree” or “strongly agree” ratings. All students either “agreed” or “strongly agreed” that the activity should be used in future classes. Specific comments from the students included the following: “I always like to put what I’m learning into practice,” “this was a good way to assess where my current skills are and what areas I need to improve,” “everyone should take this class and do this activity,” and “the critique sessions generated a lot of helpful discussions and gave me ideas that I had not thought about.”
Following are some of the student “wishes” that were expressed: “I wish we could have had the opportunity to do the taping again to see if we could incorporate the group’s feedback into another meeting,” “I think it would have been helpful to tape the critique, as this provided much more feedback on the pluses and wishes section, and then I could have reviewed it later.” Also, feedback from the first semester resulted in the observer sheet being reduced from two pages in length to one.
7. Incorporating the role play self-assessment as part of the grade reduces some student anxiety. Students reported that my grading them on their self-assessments as well as on their role play eliminated some of their apprehension about being recorded. Since my objective was for students to learn what is good communication and listening skills, I was comfortable incorporating the self-assessment as part of the grade.
[PC7]Airisian, P. W. (1996). Assessment
in the classroom.
Bangert, A. (2003). An exploratory study of the effects of peer assessment activities on
student motivational variables that impact learning. Journal of Student Centered
Learning, 1, [PC8](2), 69-76.
Bernstein[PC9], J., Scheerhorn, S., & Ritter, S. (2002). Using simulations and collaborative
teaching to enhance introductory courses. College Teaching, 50, (1)[PC10], 9-64.
Collier, K. (1998). Once more with feeling – identification, representation and the
affective aspects of role play in experience-based education. In J. Rolfe,
Saunders, & T. Powell (Eds.), The
internal simulation and gaming research yearbook (vol. 6, pp. 145-153).
Collier, K. (2000). Dramatic changes – a new action model for role-play practice. In J.
Rolfe, D. Saunders, & T. Powell (Eds.), The internal simulation and gaming
research yearbook (vol.
8, pp. 47-58).
Earl, S. (1986). Staff and peer assessment of a group project designed to promote skills
of capability. Programmed Learning and Educational Technology, 25, 327-339.
Falchikov, N. (1986)[PC11]. Product comparisons and process benefits of collaborative peer
group and self-assessment. Assessment and Evaluation in Higher Education,
Haaga,[PC12] D. (1993). Peer review of term papers in graduate psychology courses. Teaching
Psychology, 20, 28-32.
Mowl, G., & Pain, R. ([PC13]1995). Using self and peer assessment to improve students’ essay
writing: A case study from geography. Innovations in Education and Training
International, 32, 324-335.
[PC14]Marcoulides, G., & Simkin, M. (1991). Evaluating student papers: The case for peer
review. Journal of Educational Business, 167, 80-83.
Taylor, J., & Walford, R.
(1978). Learning and the simulation game.
Open University Press.
[PC15]Topping, K. & Ehly, W. (1998). Introduction to peer assisted learning. In K. Topping &
(Eds.), Peer assisted learning.
Van Ments, M. (1983). The effective use of role-play: A handbook for teachers and
[PC1]Check spelling on Bernstein c.f. p. 2 and Refs.
[PC2]Check spelling on Bernstine c.f. p. 1 and Refs.
[PC3]Check spelling of this author—c.f. Refs.
[PC4]This source is not cited in your Refs. You have Falchikov & Goldfinch 2000.
[PC5]You cite 1995 in Refs. for Mowl & Pain
[PC6]Spelling for these authors does not match those in Refs.
[PC7]This does not match the spelling in your citation on p. 2
[PC8]If this is the issue #, it should be in parentheses and not italicized, like this: Learning, 1(2), 69-76.
[PC9]Check spelling of Berstien
[PC10]If this is the issue #, it should read: Teaching, 50(1), 9-64.
[PC11]Not cited in text. You have Falchikov 1986 on p. 2
[PC12]I moved this citation to alphabetize.
[PC13]You cite 1996 on p. 2
[PC14]These spellings do not match citation on p. 2
[PC15]Citation on p. 2 includes a second author (Ehly)