(377b) Image Based Control Virtualization

Image Based Control (IBC) has seen wide-ranging applications in the control of autonomous robots [1, 2]. In chemical engineering, IBC has been used to gather process information, such as the online monitoring of crystal size distributions [3, 4, 5], and optimal combustion practices [6]. Virtual test beds for simulating IBC can be useful in setting baseline parameters, developing ideal scenarios, as well as controlled introduction of noise into a target system. The use of virtual image test beds is common in robotics, for applications like drone flight control [7], manipulation of surgical robots [8], and spatial exploration with mobile devices [9]. The target process envisioned in this work is the manipulation of monomers suspended in free space that self-assemble, also known as Directed Self-Assembly (DSA).

For this work, Blender, a free and open source 3D modeling and animation software, is used to simulate and control an IBC controlled process. The work aims to simulate an ideal IBC example of a DSA process. The process in this example involves the manipulation of multiple nanorods in two dimensions to achieve a desired configuration. The state of the rods, their position and orientation in the medium, are captured by the virtual camera in the Blender environment. Hypothetical control is enacted by translating each rod along its axis at a fixed velocity. This desired configuration is defined in relative distance and orientation, and as such the state of the system is a compact formulation of the positions of individual rods. Since the motion of the rods in the medium will be largely directed by Brownian Dynamics, a stochastic model predictive controller (SMPC) is set up to derive optimal control.

[1] Rafik Mebarki and Vincenzo Lippiello. “Image-based control for aerial manipulation”. In: Asian Journal of Control 16.3 (2014), pp. 646–656.
[2] Christophe Collewet and Fran¸cois Chaumette. “A contour approach for image-based control on objects with complex shape”. In: Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000)(Cat. No. 00CH37113). Vol. 1. IEEE. 2000, pp. 751–756.
[3] Xue Z Wang, Kevin J Roberts, and Caiyun Ma. “Crystal growth measurement using 2D and 3D imaging and the perspectives for shape control”. In: Chemical Engineering Science 63.5 (2008), pp. 1173–1184.
[4] Sebastian Maaß et al. “Automated drop detection using image analysis for online particle size monitoring in multiphase systems”. In: Computers & chemical engineering 45 (2012), pp. 27–37.
[5] Paul A Larsen, James B Rawlings, and Nicola J Ferrier. “An algorithm for analyzing noisy, in situ images of high-aspect-ratio crystals to monitor particle size distribution”. In: Chemical Engineering Science 61.16 (2006), pp. 5236–5248.
[6] Junghui Chen et al. “Design of image-based control loops for industrial combustion processes”. In: Applied energy 94 (2012), pp. 13–21.
[7] Dongliang Zheng et al. “Image-based visual servoing of a quadrotor using virtual camera approach”. In: IEEE/ASME Transactions on Mechatronics 22.2 (2016), pp. 972–982.
[8] Brian C Becker et al. “Vision-based control of a handheld surgical micromanipulator with virtual fixtures”. In: IEEE Transactions on Robotics 29.3 (2013), pp. 674–683.
[9] Richard Werner Nelem Pazzi, Azzedine Boukerche, and Tingxue Huang. “Implementation, measurement, and analysis of an image-based virtual environment streaming protocol for wireless mobile devices”. In: IEEE Transactions on Instrumentation and Measurement 57.9 (2008), pp. 1894–1907.