(721d) Attention Aware Systems in Process Control Rooms through Real-Time Pupillometry

Abstract

The role of control room operator’s in many complex, highly-automated plants is predominately on process monitoring and supervision. The performance of the plant operator under critical situation is significantly affected by poor HMI (Human Machine Interface), high alarm rates, fatigue, stress, high workload and harsh environments. Lately, there are significant developments in HMI design leading to a user friendly display in the control room; however, less attention is paid towards understanding the cognitive behavior and attention level of the plant operators which could be used in development of HMI. The cognitive abilities of the human are limited and therefore, it is necessary to understand the mental state of the operator. Negligence to realize the operator capability and attention could result in a workload bombardment and probably lead to accidents like Esso Longford & Texas City refinery explosion.

In the recent past, there has been an increased emphasis in the development of attention aware HMI systems; those that that are capable of adapting to and supporting human attentional processes, especially in situations involving multi-tasking and in highly dynamic environments (Tsianos et al. (2009), Weibel et al. (2012)). Studies have shown that metrics computed eye tracking data can be used as a method of identifying users’ actual behavior in a hypermedia setting. Researcher are currently utilizing point of eye gaze from eye trackers as an alternate HMI (Human Machine Interface) paradigm with mouse movements and selections made based on user’s eye movements  (Roda and Thomas (2006)). In the field of avionics, attempts have been made to design HMI that are capable of assessing the pilot’s focus of attention and make predictions about it in the near future and also determine when their attention needs to be redirected for efficient operation (Weibel et al. (2012)). However, to the best of our knowledge, these opportunities are yet to be explored in the chemical process industries.  In this work, we make a novel attempt to utilize eye tracker data in understanding the attention level of control room operators, a vital information in development of attention aware HMI system.

Task evolved pupillometry response (TEPR), analysis of changes in pupil diameter during execution of tasks, is one of the commonly used approaches to analyze the attention level and mental state of a human under critical situations (Christine et al. (2011)). Pupillometry has been widely used to evaluate attention level (Laeng et al. (2012)), decision making process (Marshall 2000) and stress levels (Pedrotti et al. 2014). Experiments have revealed that during decision making, pupil dilation increases before making a decision. More recently, Jiang et al. (2013) reported that when participants performed a simulated surgical task with varying difficulty levels and different demands of mental workload, the pupil size increased significantly during the harder task and decreased rapidly while performing the easier task; changes in pupil size can therefore be used to measure the mental workload.  TEPR has also been used to as an online measure to track the attention level of drivers (Di Stasi et al. (2010)).

In this study, we attempt to develop an online measure to identify the attention level of control room operator during plant abnormalities. To accomplish this, students having control theory background have performed the role of an operator in controlling the simulated ethanol plant under abnormalities. The experiment is interactive and the participant can use the HMI to get process information and manipulate the control valves. During the task, the participants take various actions to assess the situation and eliminate the disturbance by searching for appropriate information and perform corrective actions (opening / closing valves).  These actions taken by the participant using the HMI including mouse movements, mouse clicks and slider movements are recorded during the experiment. TX300 eye tracker is used to record the eye gaze data of the participants. Analysis of data from eye tracker revealed that the majority of the participants had their pupil dilated after the occurrence of alarm, revealing the high cognitive work load due to plant abnormality. Further, clearance of last alarm (end of the task) results in decrease of cognitive demand which is reflected by contraction (decrease) of the pupil size for majority of the participants. These preliminary results indicate that analysis of pupil size variations along with key events during the task could help in development of an online measure for identifying the attention level of control room operators.

Reference

Tsianos, N., Germanakos, P., Lekkas, Z., Mourlas, C., & Samaras, G. (2009, July). Eye-tracking users' behavior in relation to cognitive style within an e-learning environment. In Advanced Learning Technologies, 2009. ICALT 2009. Ninth IEEE International Conference on (pp. 329-333). IEEE.

Weibel, N., Fouse, A., Emmenegger, C., Kimmich, S., & Hutchins, E. (2012, March). Let's look at the cockpit: exploring mobile eye-tracking for observational research on the flight deck. In Proceedings of the Symposium on Eye Tracking Research and Applications (pp. 107-114). ACM.

Roda, C., and Thomas, J. 2006. Attention aware systems: Theories, applications, and research agenda. Computers in Human Behavior 22, 4, 557–587

Laeng, Bruno, Sylvain Sirois, and Gustaf Gredebäck. "Pupillometry A Window to the Preconscious?." Perspectives on psychological science 7.1 (2012): 18-27.

Marshall, Sandra P. "Method and apparatus for eye tracking and monitoring pupil dilation to evaluate cognitive activity." U.S. Patent No. 6,090,051. 18 Jul. 2000.

Pedrotti, M., Mirzaei, M. A., Tedesco, A., Chardonnet, J. R., Mérienne, F., Benedetto, S., & Baccino, T. (2014). Automatic stress classification with pupil diameter analysis. International Journal of Human-Computer Interaction, 30(3), 220-236.

Jiang, X., Zheng, B., Tien, G., Atkins, M. S., 2013. Pupil response to precision in surgical task execution. In: MMVR.

pp. 210–214.

Di Stasi, L. L., Marchitto, M., Antolí, A., Baccino, T., & Cañas, J. J. (2010). Approximation of on-line mental workload index in ATC simulated multitasks.Journal of Air Transport Management, 16(6), 330-333.