(182c) Predicting Goal Attainment in Control-Oriented Behavioral Interventions Using a Data-Driven System Identification Approach

Concepts from process systems engineering, notably system identification and model predictive control, have been increasingly applied to the development of control-oriented, personalized interventions for improving health behaviors such as daily walking; these can prevent chronic illnesses such as heart disease, cancer, and diabetes [1]. The approach taken in [1] to generate these improved interventions relies on mobile health technologies (e.g., smart watches) and experimental trials such as the Control Optimization Trial (COT) [2]. The COT is a data-driven intervention optimization strategy for individual subjects that integrates dynamic modeling via system identification with control algorithms such as model predictive control (MPC). Hence, the COT provides a unified framework to understand individual participant characteristics over time and as a result of changes in environmental context and external factors.

Goal attainment, defined as the difference between the daily number of steps taken and the declared goals, is an important consideration in the success of the COT, and has been studied in prior work [3]. It has been established that setting goals is generally positively associated with behavior, but too ambitious goals will negatively impact goal attainment. As a result, it is important to determine an optimal goal setting zone that promotes the desired physical activity and enables participants to remain engaged in an intervention. In this work we (in contrast to [3]), consider goal attainment as a dynamical process that is a function of input signals (e.g., daily goals) and environmental context (e.g., predicted stress, temperature). We take an idiographic modeling approach, meaning that participant models are developed individually and are not studied in clusters with other participants. The personalized nature of the approach allows us to consider individual responsiveness towards different inputs [4].

The analysis in this paper is based on the Just Walk intervention study [5]. Just Walk is an mHealth intervention based on system identification principles with the goal to promote physical activity in sedentary adults. In this study, 18 middle-aged participants, generally healthy but inactive, are provided with two primary input signals: daily step goals (ambitious but doable) and expected points. The actual number of steps walked per day (behavior) is observed and reward points (that can be cashed for gift cards) are granted to participants when the goals are met (or exceeded). Exogenous factors such as Temperature, Predicted Busyness, Weekday/Weekend are also recorded. The input signals are generated using multisine functions that are deterministic yet pseudo-random in nature, and are designed to cover a frequency interval of interest [6]. The study spans across 80 days, comprising five cycles (sub-experiments) of 16 days each with a sampling time T = 1 day.

For estimation purposes, we rely on multi-input Auto-Regressive with eXogenous (ARX) input models to predict behavior. As ARX estimation is a linear least squares problem, it facilitates examining a range of orders and features (i.e., input signals and exogenous/external factors). To improve validity of the estimated model and reduce dependence on specific times of the intervention, we explore a generalized approach that evaluates multiple combinations of sub-experiments to specify estimation and validation data (with at least two cycles used for validation). Goodness of fit is evaluated through calculating the normalized root mean square error (NRMSE) on both validation cycles and the overall data. While exhaustive evaluation of model orders and model features (in order to determine optimal models) is possible, it can also require significant time; hence we consequently evaluate a Discrete Simultaneous Perturbation Stochastic Approximation (DSPSA) [7] search routine which is near-optimal but significantly reduces computational time.

Our analysis indicates that goal attainment is a highly personalized phenomenon; for example, characteristics that best describe goal attainment in a specific individual may be goals, expected points, and granted points (without any influence from contextual factors) while for a different individual, temperature, or whether the day is a weekday or a weekend, will have an important role. The optimal combination of experiments for individual participants can be determined through our analysis, with step responses from ARX models providing a sense of the dynamic nature of goal attainment that is not provided by traditional static methodological approaches. Model validation analysis on the population of Just Walk participants indicates that linear models are generally adequate to explain phenomena that could be expected to be very nonlinear.

Further study of models across all participants contrasting both goal attainment and behavior shows that both behavior and goal attainment are negatively and positively associated with expected and granted points, respectively. This implies that a participant decreases their activity if they are provided with expected points, regardless of changes in goals, whereas increases in granted points further motivates the participant to take more steps, thus contributing to goal attainment. Responses to other system features can be similarly understood through this analysis.

In summary we conclude that system identification principles can be instrumental in better understanding behavior dynamics both at an individual and societal level through a data-driven approach, and in providing guidelines for the design of experiments that successfully implement proper adaptive intervention signals that promote the desired behavior.

References:

[1] Rivera, D.E., Hekler, E.B., Savage, J.S., Downs, D.S. (2018). Intensively adaptive interventions using control systems engineering: two illustrative examples. In: Collins, L., Kugler, K. (eds) Optimization of Behavioral, Biobehavioral, and Biomedical Interventions. Statistics for Social and Behavioral Sciences. Springer, Cham. https://doi.org/10.1007/978-3-319-91776-4_5

[2] Chevance, G., Baretta, D., Golaszewski, N., Takemoto, M., Shrestha, S., Jain, S., Rivera, D. E., Klasnja, P., & Hekler, E. (2021). Goal setting and achievement for walking: A series of N-of-1 digital interventions. Health Psychology, 40(1), 30–39. https://doi.org/10.1037/hea0001044

[3] Hekler, E. B., Rivera, D. E., Martin, C. A., Phatak, S. S., Freigoun, M. T., Korinek, E., ... & Buman, M. P. (2018). Tutorial for using control systems engineering to optimize adaptive mobile health interventions. Journal of Medical Internet Research, 20(6), e8622. doi: 10.2196/jmir.8622

[4] Phatak, S. S., Freigoun, M. T., Martín, C. A., Rivera, D. E., Korinek, E. V., Adams, M. A., ... & Hekler, E. B. (2018). Modeling individual differences: A case study of the application of system identification for personalizing a physical activity intervention. Journal of Biomedical Informatics, 79, 82-97.

[5] Freigoun, M. T., Martín, C. A., Magann, A. B., Rivera, D. E., Phatak, S. S., Korinek, E. V., & Hekler, E. B. (2017, May). System identification of Just Walk: A behavioral mHealth intervention for promoting physical activity. In 2017 American Control Conference (ACC) (pp. 116-121).

[6] Martín, C. A., Rivera, D. E., & Hekler, E. B. (2015). Design of informative identification experiments for behavioral interventions. IFAC-PapersOnLine, 48(28), 1325-1330.

[7] Rachael T. Kha, Daniel E. Rivera, Predrag Klasnja, and Eric Hekler. “Model Personalization in Behavioral Interventions using Model-on-Demand Estimation and Discrete Simultaneous Perturbation Stochastic Approximation”, 2022 American Control Conference, Atlanta, Georgia: in press