A New Way to Troubleshoot Student Learning
To see a full presentation of results from the two PALD studies at AERA conducted by the Chatterji group, visit Outcomes Executive Summary and Mapping Cognitive Pathways.
Mapping Cognitive Pathways PDF
To learn more about TC's Assessment and Evaluation Research Initiative, including its mission and current projects, visit AERI Mission Statement & Projects
The studies, led by Madhabi
Chatterji, Associate Professor of Measurement and Evaluation in the Department of Organization and Leadership
The method requires teachers to break down math problem solving – or any academic task they want students to learn -- into a set of connected skills and concepts required by students to solve the tasks. They organize the tasks by difficulty. They then assess student performance at each step to understand precisely where students make errors or show lack of understanding. The underlying philosophy of PALD is to re-conceive testing in general as a diagnostic process embedded within instruction, rather than simply as an instrument for testing students at the end of a semester or year or for sorting students by merit.
In one paper, “Proximal Assessment for Learner Diagnosis (PALD): A Study of Classroom Practices and Early Teacher and Student Outcomes”, Chatterji and colleagues Douglas Ready, Nancy Koh, Linda Choi and Radhika Iyengar compare classrooms in which teachers used PALD, with others at the same schools that did not, to assess how teachers’ use of PALD practices affected student achievement in math on local and standardized tests.
In another paper, “Mapping Cognitive Pathways in Mastering Long Division,” Chatterji and colleagues Nancy Koh, Howard Everson and Pearl Solomon present a case study of one class that was exposed to PALD-trained teachers in both grade 5-6, documenting changes in their learning gaps and cognitive development as they became experts in long division.
“The fundamental questions we are asking through this work are, ‘Can we train teachers to look at children and their learning processes more diagnostically? And if teachers gain the skills to conduct close-up examinations of where children’s learning stalls, would they take actions to turn things around?’” Chatterji says. “We were also interested in determining if signs of students’ progress will show up on standardized tests in the long run, and if the attitudes of both teachers and students towards tests and testing also change – will they come to believe more in diagnostic tests as an aid to teaching and learning?
“The answer to these questions, based on our preliminary studies, is Yes – particularly in terms of teacher attitudes towards assessment and student outcomes in grade 6 where more students had continuing exposure to PALD.” Some results in PALD fifth grade classes were on a par or inferior to those in non-PALD fifth grade classes. The authors believe that in these instances, the novelty of the PALD method for the teachers, lower levels of student exposure, and teachers’ lower math subject matter knowledge, might have all affected the PALD implementation and early outcomes.
The assessments of students that PALD-trained teachers use are based on “situated tasks” that draw on embedded concepts and skills. Thus, problems in long division are designed to reflect real situations in which a fifth or sixth grader will need to apply long-division skills. For example, children were asked: If cookies come in packs of 12, and there are 215 fifth graders in your school, how many packs of cookies must be purchased in order to give each fifth grader one cookie on the last day of school?
“It’s a more contextualized method of assessment, teaching and student evaluation that stresses diagnosis,” Chatterji says.” You wouldn’t say in the end, simply, whether a student got a problem right or wrong. You’d say what he or she did right, where the mistakes occurred, and what needs to be done to help the child take the next cognitive step to learn. So, PALD’s a much more fine-grained approach to analysis than current grading and standardized testing methods. And along the way, you’re teaching children meta-cognitive skills – that is, to think about what they’re learning and where they’re falling down.”
In conducting their studies at Hillcrest Elementary, El Dorado Elementary, Elmwood Elementary and Colton Elementary, Chatterji and colleagues at first met with some resistance from teachers who balked at the extra labor involved in implementing PALD methodology.
Initially teachers were up in arms because of the time it takes to implement PALD. But after a year, participant teachers became the greatest advocates for the PALD approach.
“Using proximal assessment to
evaluate student understanding is an effective method to ensure positive
student outcomes,” says Andrea LaMantia, a sixth grade teacher at
LaMantia says that while she initially thought using PALD would be time consuming, “in the long run, it saved time. Learning math is contingent on understanding embedded concepts. Students often give up once they get confused. Eliminating this confusion early in the process not only helps achievement but builds self-confidence as well.”
(Ms. LaMantia is available for interviews with the media.)
Chatterji is Co-Director (with
Edmund Gordon) of the Assessment and Evaluation Research Initiative (AERI) at
Teachers College, affiliated with The Campaign for Educational Equity. She has
just returned from