Academic Exchange Quarterly Spring 2001: Volume 5, Issue 1
Assessing the Effectiveness of Problem‑Based Learning in Higher Education: Lessons from the Literature
Claire H. Major,
Major is an assistant professor of higher education administration. <email@example.com>. Palmer is an assistant professor in the Counseling, Rehabilitation and Student Development department <betsy‑firstname.lastname@example.org>.
Problem‑Based Learning (PBL) is an innovative educational approach that is gaining prominence in higher education. A review of the literature of PBL outcomes summarizes, across multiple studies, the positive effects of problem‑based learning. Since PBL brings with it unique challenges to traditional assessment, however, this study suggests alternative approaches. Alternative assessment may provide additional insight into the effectiveness of PBL and other alternative pedagogies.
Traditional instruction, such as the typical lecture‑based session that developed before textbooks were mass‑produced, often involves delivering as much information as possible as quickly as possible. The lecture method was one of the most effective and efficient ways to disseminate information and has often been used for this end. Because many faculty members are poor lecturers, and because students are often poor participants in the lecture, this type of instruction has often allowed students to be passive in the classroom. Students, not knowing how to be active participants in the lecture, have relied on transcription, memorization, and repetition for learning.
In recent decades, however, we have learned a great deal from cognitive science research about the nature of learning. Students construct knowledge; they do not take it in as it is disseminated, but rather they build on knowledge they have gained previously (Cross, 1998). They benefit from working together, and they may learn best from teaching each other (Annis, 1983; McKeachie, et al., 1986). Research also suggests that students learn best in the context of a compelling problem (Ewell, 1997); they learn through experience (Cross, 1999). In short, students learn through making cognitive connections, social connections, and experiential connections (Cross, 1990). Because they make these connections differently, students do not learn in the same way. This relatively new information suggests that teaching is a complex activity, and it necessitates the emergence and development of approaches to instruction that are consistent with what we know about the way that learning happens (Ewell, 1997). This new understanding has given rise to the notion of a paradigm shift in higher education, one from a focus on teaching to a focus on learning (Barr and Tagg 1995). New "powerful pedagogies" emphasizing learning, such as project‑based learning, inquiry‑based learning, case‑based learning, research‑based learning, situation‑based learning, action learning, and Problem‑Based Learning (PBL) intimate that alternative pedagogies may be gaining in prominence and may ultimately become the dominant classroom paradigm.
PBL is an educational approach in which complex problems serve as the context and the stimulus for learning. In PBL classes, students work in teams to solve one or more complex and compelling "real world" problems. They develop skills in collecting, evaluating, and synthesizing resources as they first define and then propose a solution to a multi‑faceted problem. In most PBL classes, students also summarize and present their solutions in a culminating experience. The instructor in a PBL class facilitates the learning process by monitoring the progress of the learners and asking questions to move students forward in the problem‑solving process. Unlike traditional classrooms, the faculty member is not the sole resource for content or process information, but instead guides students as they search out appropriate resources.
approach had its start in the 1960s at
Finally, educators and employers
alike began to call for change in undergraduate institutions (Jones, 1997).
They also wanted students who could think critically, solve problems, and work
in teams. The 1998 Boyer Report, Reinventing
undergraduate education: A blueprint for
In developing expectations about the outcomes that PBL can bring, we can learn from the research that has been conducted in medical schools. Therefore, in this article, we review recent literature on PBL from medical and professional schools that outlines PBL effectiveness compared to traditional instruction. We also review recent literature from professional schools and undergraduate programs, which suggests that traditional assessment may not be adequate to evaluate the effectiveness of PBL as an instructional method. This research shows that PBL provides students with the opportunity to gain theory and content knowledge and comprehension. In addition, PBL helps students develop advanced cognitive abilities such as critical thinking, problem solving, and communication skills. PBL also can improve students' attitudes toward learning. Thus, as a pedagogical technique, problem‑based learning promotes the kinds of active learning that many educators advocate (Barr & Tagg, 1995).
Several studies focus on the change in knowledge and skill levels that occur with PBL instruction. A few studies show slight decreases in knowledge of basic sciences (Albanese & Mitchell, 1993). Other studies show that on tests of medical knowledge, students in traditional programs scored higher than students in the PBL curriculum (Schmidt, Dauphinee, & Patel, 1987; de Vries, Schmidt, & de Graaff, 1989). Overall, most studies show no significant difference between the knowledge that PBL students and non‑PBL students acquire about sciences (Albanese & Mitchell, 1993). However, students who acquired knowledge in the context of solving problems have been shown to be more likely to use it spontaneously to solve new problems than individuals who acquire the same information under more traditional methods of learning facts and concepts through lectures (Bransford, Franks, Vye, & Sherwood, 1989). In addition, students in the problem‑based learning environment have developed stronger clinical competencies although the differences were small and non‑significant (de Vries, Schmidt, & de Graaff, 1989). A study conducted in a nutrition and dietetics course found that PBL students perceived that they developed stronger thinking and problem‑solving skills, effective communication skills, and sense of personal responsibility than did students who received lectures (Lieux, 1996).
Much of the medical school research shows that student attitudes toward learning do change. Students in PBL courses often report greater satisfaction with their experiences than non‑PBL students. For example, PBL medical students at Harvard reported their studies to be more engaging, difficult, and useful than did non‑PBL students (Albanese & Mitchell, 1993). Additional research studies document that students who experience PBL have substantially more positive attitudes toward the instructional environment than do students in more traditional programs. PBL students tend to give high ratings for their training whereas students in traditional programs are more likely to describe their training as boring and irrelevant (de Vries, Schmidt, & de Graaff, 1989; Schmidt, Dauphinee, & Patel, 1987). These changes in attitudes are marked by an impact on student retention. In countries with high dropout rates among medical school students, such as in Holland, students in the PBL medical program were much more likely to graduate and do so in less time than students in the more traditional curriculum (de Vries, Schmidt, & de Graaff, 1989). In addition, attendance was significantly higher in the PBL class than in the lecture version (Lieux, 1996).
Research also shows changes in
student study behavior. Coles (1985) and Newble and
Research of PBL
in medical schools, as seen in the reviews by Albanese and Mitchell (1993),
However, PBL presents some unique challenges for assessment. Because the focus of this pedagogy is primarily on learning to learn and less on mastery of a particular body of knowledge, traditional methods of course assessment such as examinations may not be very effective (Major, 1999). If traditional assessment is a good measure of traditional pedagogy, it stands to reason that an alternative assessment may be necessarily a better measure for an alternative pedagogy, such as PBL. Using alternative assessment in the case of PBL can help bridge the gap between instruction and assessment. Authentic assessment uses tasks developed from realistic activities in the professional world (Nightingale, Te Wiata, Toohey, Ryan, Hughes & Magin 1996). Nightingale, Te Wiata, et al., (1996) define authentic assessment tasks as "complex simulations, case studies, . . . or multi‑faceted projects . . . assessing a range of knowledge, skills and attitudes in the one assessment task."
Some signs of a movement in this direction exist. Recently studies have begun to investigate PBL outcomes, such as teamwork or presentation skills, that may not be associated with traditional lecture methods. Cockrell, Caplow, & Donaldson (2000), for example, recently conducted a study examining students' perspectives on their learning as members of collaborative groups. The researchers, using interpretive methods, found that the collaborative groups fostered students' sense of ownership of the knowledge that was created over the semester. The researchers also suggested that within the groups, leadership moved from student to student as situations arose and resolved. More studies like this one are needed to determine the effectiveness of PBL in higher education.
These recent studies show that it
is time to think outside of the box on how we assess the effectiveness of PBL and how we think about its outcomes. Alternative
assessment which is authentic to the learning environment can have a positive
influence. An alternative assessment measure might include constructed response
items, essays, writing samples, oral presentations, exhibitions, experiments,
and/or portfolios (
As with any assessment planning process, the principles of good assessment can help guide practitioners. Instructors should begin by knowing what they want their students to achieve and how they want students to get there. They should consider that learning is a multidimensional activity; including knowledge and abilities as well as values, attitudes, and habits of the mind. In addition, when formulating learning goals, faculty members should think of learning in the larger context of the educational community. Assessment of whether goals have been attained and learning has occurred should have a clearly stated purpose, one related to the learning. Thus, it should focus on comparing educational goals and expectations with performance. In addition, assessment should be ongoing, throughout a semester, rather than occurring only at the end (American Association for Higher Education, 1992).
While each particular PBL instructional environment is unique, and therefore merits its own unique assessment strategy, several alternative assessment techniques seem particularly appropriate for the PBL learning environment.
Outside Evaluation by Experts. Since many PBL experiences involve a culminating experience such as a presentation, written projects, or portfolio, systematic evaluation of these capstone projects by a team of outside experts can afford one means of assessing student performance.
Content Analysis of Projects. Because PBL uses ill‑structured problems with many possible solutions, each student group within a class may pursue a slightly different domain of knowledge. To assess the range of content knowledge learned by students in the class, instructors may need to evaluate across assignments and groups to look for the variety of resources students are collecting. Project analyses may also be useful in assessing skills such as researching, critical analysis, or writing.
Focus Groups. One method that can prove useful for assessing outcomes such as teamwork or leadership is the use of focus groups. Students can offer perspectives on their experience within the problem‑solving group and may be able to reflect on their own growth across the experience.
Peer Evaluations. Many PBL instructors use some form of peer evaluation as a means of moderating individual student behavior within the group setting. However, these evaluations can also be used to gather data about the level of skill development across individuals in the class.
Journals or Activity Logs. Students in PBL classes often do the bulk of the work for a project outside the formal classroom. The work completed outside the classroom can be difficult to assess, so many instructors require students to keep a log or journal of the work they complete for the project. Mid‑semester and/or end of semester evaluations of these journals or logs can provide excellent evidence of student learning throughout the project.
Personal Reflections. For many students, the type of active learning that PBL requires is an unusual experience. One method for assisting students in their metacognitive understanding of the PBL process is to ask them to reflect on the experience of PBL at key points in the process. Qualitative analyses of these reflections can offer supportive evidence for many process‑type outcomes such as developing critical thinking or research skills.
These techniques focus on the contextual nature of PBL, requiring the students to produce an authentic product that is related to the problem and to make judgments about their performances.
The academy and its constituents have called for change in the way that teaching and learning are carried out in undergraduate education, and change is happening on many campuses and in many classrooms. Assessment must keep pace with these trends. We must begin to look beyond traditional measures toward new and different ways of knowing. PBL is just one of the many pedagogies that illustrate this fact, but educators' experience with PBL can shed new light on what, why, and how we are doing in the academy.
Albanese, M.A. (1993). Problem‑based learning: A review of literature on its outcomes and implementation issues. Academic medicine: Journal of the Association of American Medical Colleges, 68 (1), 52‑81.
American Association for
Higher Education (1992). Nine principles
of good practices for assessing student learning.
Annis, L. F. (1983). The process and effects of peer tutoring. Human Learning, 2, 39‑47.
Barr, R., & Tagg, J. (1995). From teaching to learning: A new paradigm for undergraduate education. Change, 27 (6), 12‑25.
Barrows, H. S. (1996). Problem‑based learning in medicine and beyond: A brief overview. New Directions for Teaching and Learning, 68, 3‑12.
Blumberg, P. & Michael, J. A. (1992). Development of self‑directed learning behaviors in a partially teacher‑directed problem‑based learning curriculum. Teaching and Learning in Medicine, 4 (1), 3‑8.
Boud, D. & Feletti, G. I. (1991). The
challenge of problem‑based learning.
The Boyer Commission on
Educating Undergraduates in the
Bransford, J. D., Franks, J. J., Vye, N. J., & Sherwood, R. D. (1989). New Approaches to
Instruction: Because Wisdom Can't Be Told. In S. Vosiadou
& A. Ortony (Eds.), Similarity and Analogical Reasoning (pp. 470‑297).
Cockrell, K. S., Caplow, J. A. H., & Donaldson, J. F. (2000). A Context for Learning: Collaborative groups in the problem‑based learning environment. Review of Higher Education 23 (3), 347‑363.
Coles, C. R. (1985). Differences between conventional and problem‑based curricula in their students' approaches to studying. Medical Education, 19 (4), 308‑309.
Cross, K. P. (1998). Opening windows on learning: The cross
papers number 2.
Cross, K. P. (1999). Learning is about making connections: the
cross papers number 3.
M., Schmidt, H. G. & de Graaf, E. (1989). Dutch
comparisons: Cognitive and motivational effects of problem‑based learning
on medical students. In Schmidt, H. G., Lipkin, M.,
de Vries, M. W. & Greep,
J. M. (eds), New
Directions for Medical Education: Problem‑based learning and community
oriented medical education. (pp. 230‑240) .
Ewell, P. T. (1997). Organizing for learning: A new imperative. AAHE Bulletin, 50 (4), 3‑6.
Jones, E. A. (1997). Identifying college graduates' essential
skills in reading and problem solving: Perspectives of faculty, employers, and
Lieux, E. M. (1996). A comparative study of learning in lecture versus problem‑based format. About Teaching, 50, 25‑27.
Major, C. (1999). Connecting what we know and what we do through problem‑based learning. AAHE Bulletin, 51 (1), 7‑9.
McKeachie, W. J., Pintrich,
P. R. Lin, Y.‑G, & Smith, D. A. F. (1986). Teaching and Learning in the College Classroom: A Review of the
National Education Goals
Panel. (1992) . The national education
goals report: Building a nation of learners.
Nightingale, P., Te Wiata,
Nolte, J., Eller, P. & Ringel, S. P. (1988). Shifting toward problem‑based learning in a medical school neurobiology course. Proceedings of the Annual Conference on Research in Medical Education, 27, 66‑71.
Schmidt, H. G., Dauphnee, W. D., & Patel, V. L. (1987). Comparing the effects of problem‑based and conventional curricula in an international sample. Journal of Medical Education, 62 (4), 305‑315.
Worthen, B. (1993). Critical issues that will determine the future of alternative assessment. Phi Delta Kappan, 74, (6), 444‑448, 450‑454.