(64f) Rapid Analysis of Written Responses to Reveal Student Misconceptions in Thermodynamics

Traditionally, engineering courses utilize computations to evaluate student competence. Problem solving typically involves identification of the governing principles and proper application of engineering models. Many of todayâ??s students have challenges applying models due to misconceptions about the underlying principles. Collection and assessment of short answer written (constructed) responses to conceptual questions can provide insights into why students have difficulties solving numerical problems. Such assessments are seldom used because the evaluation process is time consuming, especially when assessing the percentages of students with certain misconceptions. Accurate automated assessment of constructed responses can provide rapid feedback to instructors about the distribution of understanding/misconceptions even in large classes, providing opportunities for the instructor to reemphasize correct application of principles. In the future, such assessments may also be valuable for self-instruction. The Automated Analysis of Constructed Response (AACR) research group (www.msu.edu/~aacr) at Michigan State University is leading a multi-institutional collaboration (TUES 3 and WIDER funding) with several connected goals: 1) create a national web portal to provide automated analysis of constructed responses (AACR) to conceptual questions in a variety of STEM areas; 2) use the resulting reports to focus community collaborations between STEM education researchers and instructors; 3) transport AACR innovations through ongoing faculty professional development; 4) expand the range of STEM disciplines in which we pursue this research from biology into chemistry, chemical engineering, physics/astronomy, and statistics; 5) engage in ongoing project evaluation for continuous quality improvement; and 6) lay the foundation for sustainability.

This presentation provides a background on the progress in automated assessment of thermodynamics concepts, initially focusing on internal energy. The presentation will provide an overview of the criteria used for question development, the methods used to refine questions, the techniques applied for automated assessment, the reports provided to the instructor and how the reports are helpful in understanding the distribution of misconceptions. The presentation will describe opportunities for participation in the question development process and the status of our current set of questions.