(534g) Student Learning in Hands-On, Remote, and Virtual Laboratory Experiences: What Works (or doesn't)

Background Student learning outside the classroom may be complemented by a variety of structures. Laboratory and project work are typical modes. However, lab work can be hands-on, remote, or based upon simulations (virtual). Ideally, remote experiments can be conducted at any time from any place. They are particularly useful for students at universities where resources are severely limited and there is no access to any significant experimental equipment. Simulations are also accessible 24/7, do not require substantive resources (unless the software is expensive) and can provide efficient and easy ways to study multiple variables. Multiple runs are frequently impossible or too costly with real lab equipment whether remotely operated or not.

In all contexts, understanding student learning is critical. Each mode has unique advantages and disadvantages that contribute to learning. Basically, knowing what works and what doesn't is necessary to structure course material and inform implementation of any or all modes in a given course.

At WPI distillation is taught in the sophomore year throughout a project-based spiral curriculum. All projects are team-based. We introduce basic concepts early in the sophomore year then revisit distillation throughout the year with successively more complex assignments and projects. This includes at least two lab experiences. The first experiment uses a batch column operated at total reflux to introduce students to multistage distillation including efficiency and energy balances at total reflux. A follow-on course explores pressure swing distillation without a lab component. Near the end of the year, the batch column operated at a constant external reflux ratio is used again in a lab project. This project engages students in process dynamics and challenges them to compare differences between theory and reality using the Rayleigh analysis. Large enrollments forced us to ?outsource? lab work by taking advantage of the available but remotely-operated column at UTC. The column is the same as WPI's but is accessible and controllable through a web-based interface. Methodology At the last AIChE meeting we presented a study that showed student learning in remote and hands-on environments was essentially equivalent. We have supplemented that work by comparing learning in the same course described above in three cohorts: hands-on, remote, and virtual. The virtual column was constructed using Aspen BatchSep®.

Teams were four students each (teams were self-selected), and they were randomly assigned to cohorts. Seven teams ran the local column (hands-on), six teams ran the UTC column (remote) and seven teams used the simulation (virtual). Remote and hands-on teams conducted identical experiments with identical assignments. Virtual teams had an identical base case assignment but made additional simulation runs. Analysis and project reporting requirements were the same for all cohorts.

Evaluation had three components with direct and indirect assessments, giving us some degree of triangulation. All cohorts completed surveys managed by a third partner: UIUC. The online survey included closed-end, Likert-scaled responses and open-ended questions. The open-ended questions included student attitudes but also probed team processing since we wanted to learn more about how each team handled roles, logistics, scheduling, data collection, and analysis.

Course instructors compared final reports from all teams. A pre/post in-class quiz compared individual learning for students in the remote cohort compared to hands-on cohort in a quantitative manner. Qualitative measures include evaluating student attitudes about the experience and assessing students' improvement in ABET outcomes such as: (b) ? design, analysis and interpretation of data; (d) ? functioning on multidisciplinary teams; (g) ? effective remote communication; (h) - have broad education necessary to understand the impact of engineering solutions in a global, economic, and environmental societal context; and (k) - using techniques, skills, and modern engineering tools (such as computers and web interfaces) necessary for engineering practice. Results Logistics As previously reported, having completed two cycles of this pilot we have a fairly good understanding of what it takes to put 20 teams through this lab project. It would have been essentially impossible to run all teams through the local WPI lab or the UTC lab, during the time allotted for the course. However, splitting effort between WPI, UTC, and Aspen makes logistics workable. We learned that running either the remote or virtual column presents different but no more burdensome scheduling issues than running the local column. Student Attitudes Survey results will not be available until sometime after May, 2010. Previous results indicated that there are few, if any, serious issues. Students don't mind being assigned to cohorts nor was there any indication of teams feeling they lost out on something because they weren't in a different cohort. New results involving virtual team experiences will be available over the summer and will be analyzed in time for the conference.

Student Learning All teams produced a significant final report that was graded by the WPI instructor using the same rubrics. We evaluated presentation quality, analysis of results, and demonstration of appropriate concepts. We also examined learning issues as they specifically related to two important concepts: distillation column dynamics and comparison of theory and experiment using the Rayleigh analysis. We also gave a pre/post quiz regarding simple concepts in these areas to compare differences among cohorts.

Our previous content analysis of the reports showed there were common deficiencies that were independent of remote, or hands-on cohorts. These included the typical mistakes with energy balance calculations; and another typical confusion over quantitative comparison of experiment to Rayleigh analysis. Students expect exact comparisons yet reality is never that predictable. In too many cases, the report presentation quality was marginal. This result is not unusual given the end-of-semester time demands.

Our report analysis is ongoing and details will not be available as of the deadline abstract. However, a preliminary read showed that the virtual cohort may have shown some interesting positive learning gains when compared to remote or hands-on. This, at first, seems surprising but virtual students had the advantage of being able to make several runs and may show learning gains because of that. The authors note that this is very preliminary and will only be confirmed (or not) when the content analysis is done. Previously we've used the psychology of ?presence? (Ma and Nickerson, ACM Computing Surveys, v. 68 no. 3, article 7, 2006) as a framework for interpretation. Student learning in lab environments is connected to their perception of presence and the process of collaboration. Presence can be physical, telepresence, or virtual (for example, a simulation). In our case the remote teams experienced telepresence, local teams experienced physical presence, and simulation teams experience virtual presence. Hence a hands-on lab that has minimal physical interaction may not be significantly different than a remote one where the interaction is mediated via a computer. In fact, the UTC column has three cameras that can be manipulated by students bringing an element of physical presence into the mix. However, the Aspen simulation is viewable only through the standard Aspen interface. By meeting time we will also know more about students' collaboration process, and how that dynamic affected interactions, and hence learning, within each cohort.