(134f) Initial Analysis of the Effects of Homework Wrappers in a Problem-Solving Course

A previous study spanning 30 years examined the combined effects of (a) changing from lecture to flipped class course delivery, (b) changing to predominantly effort-based homework grading and (c) introducing homework wrappers as a required component of homework assignments. That study took place in a junior-level, problem-solving course, specifically a kinetics and reaction engineering course. Prior to these changes the data showed a weak effect of homework completion upon exams scores, and the exams scores distribution of students who completed an above-average amount of homework was statistically equivalent to that for students who did not. After these changes, the effect of homework completion upon exams scores more than doubled (from 0.171 to 0.435 exam points per percent homework submitted) and the average exams scores of students who submitted an above-average amount of homework was 12.45 points higher than students who did not.

In this study, the effects of homework wrappers are examined in greater detail. The rationale for introducing homework wrappers is that student learning is improved by practice coupled with targeted feedback. Grading a problem and returning it along with a correct solution allows students to understand what they did wrong. The intended purpose of homework wrappers in the present context is to cause the students to examine the thought processes they employed when solving the problem and identify and correct points where their approach was flawed or their understanding was incomplete. The previous study showed that wrapper use correlated with higher exams scores, but it did not examine whether the wrappers were actually providing this level of feedback to the students.

Prior to assigning a given type of problem to the students as homework, they were instructed on how to identify that problem type and a general approach for solving that type of problem was presented. A problem of that type was then solved as an in-class learning activity. The activity was scaffolded to emphasize the implementation of the generalized approach to solving that type of problem. Only then was a problem of that type assigned as homework. As implemented here, homework wrappers were follow-up assignments wherein students are asked reflect upon their solution to a problem, write an analysis of their thinking at the time they formulated their problem solution, comparing it to the general approach, and identify knowledge and skills they were missing, etc.

Kinetics data analysis problems were selected for study. These are one of several problem types wherein the students must formulate mole and energy balances for a reactor in order to solve the problem. This problem type involves parameter estimation using those balances and experimental data. Other problem types use the same balance equations, but in a different way. Kinetics data analysis is considered relatively early in the course. As a consequence, kinetics data analysis problems appear in at least one mid-term exam and again in the final exam.

Each homework assignment involving a kinetics data analysis problem was followed up with a homework wrapper. The student response on each wrapper was coded to indicate points that the student identified as a point where their thinking was flawed or the understanding was incomplete. Separately, each submitted assignment was reviewed by the author who coded the actual student solution to indicate points that the expert identified as a point where their thinking was flawed or the understanding was incomplete. For each assignment, the student and expert analyses were compared. This analysis is critical because it indicates whether students can self-generate appropriately targeted feedback on their homework solutions. The agreement was tracked for the entire semester to see whether agreement changes as the students become more familiar and comfortable with the wrappers. These results may also suggest ways to modify the wrappers to improve agreement.

The data were also analyzed to track individual students over time. The number of times a student made the same conceptual mistake in successive assignments or exams was recorded. The results for students who had self-identified that mistake were compared to results for students who failed to do so. Emergence of mistakes (following a previous assignment where that mistake was not made) and recurrence of mistakes were also classified as either problem-type identification or problem formulation procedure. It was expected that problem-type identification was more likely to emerge or recur, especially on exams that were longer after the homework assignments, because it is more difficult to provide authentic practice in problem-type identification.