(107f) Assessing Authentic Problem-Solving in Heat Transfer

Problem-solving is an essential skill for engineers-in-training to develop. Although engineering programs stress the importance of teaching problem-solving skills, there are frequently reported gaps between the skills graduating engineers have and what employers want. Part of the reason for this gap is that many of the problems students solve, particularly in engineering science courses like thermodynamics and fluid mechanics, bear little resemblance to the authentic, unstructured problems they will be expected to solve as engineers. Recent work has characterized authentic problem-solving as a set of 29 decisions common across experts in different STEM fields. These decisions represent specific, measurable skills that can be practiced with appropriate feedback to develop problem-solving expertise. In this paper, we describe the development of an assessment to measure decision-making in the context of heat transfer. This assessment represents a departure from typical assessments in that it is not designed to rank students relative to one another, but to measure the progression of students’ expertise toward a standard established by a consensus of experts. We tested the assessment with 14 engineers highly skilled in heat transfer and transport phenomena, as well as 12 undergraduate engineering students who had previously taken a heat transfer course. From this data, we developed a scoring rubric that maximizes the discrimination between expert thinking and student thinking. On many decisions, students are far from expert-like, whereas for others, students are much closer to expert thinking. In some areas where expert responses differ from one another, our think-aloud interviews reveal how the specific decisions made by experts depend on their interpretations of the goals in an unstructured problem-solving context. We hope to encourage other educators to use this assessment in their courses to measure how well they are preparing their students to solve real-world engineering problems.