(454a) Agent-Based Control of Spatially Distributed Chemical Reactor Networks

Large-scale spatially distributed systems provide a unique and difficult control challenge because of their nonlinearity, spatial distribution and generally high order. The control structure for these systems tend to be both discrete and distributed as well and contain discrete and continuous elements. A layered control structure interfaced with complex arrays of sensors and actuators provides a flexible supervision and control system that can deal with local and global challenges. Traditionally, research on control of nonlinear distributed processes has focused on distributed parameter systems involving mathematically complex model reduction and controller synthesis methodologies. So-called hybrid control systems combine process dynamics and discrete control elements through the use of multiple linear models at different operating points. One alternative approach is based on a hierarchical agent-based system with local and global control structures that has been demonstrated on a network of interconnected continuous stirred tank reactors (CSTRs). Reactor networks exhibit highly complex behavior, with multiple steady state operating regimes, and have a large pool of candidates for manipulated variables. We use autocatalytic reactions in these networks to formulate surrogates for predator-prey, virus propagation in a distributed population, multiple species of animals that rely on the same resources, or chemical manufacturing problems.

Recent work on multiple reactor configurations with cubic autocatalytic reactions has demonstrated a rich spectrum of static and dynamic behavior. The topography of interconnected CSTR networks has been shown to drastically affect the steady state bifurcation structure of the system. Spatial inhomogeneity of the network can be increased by increasing the number of reactors in the network as well as manipulating the interconnection flow rates of the network. It has also been shown that the number of stable and unstable steady states increase with the inhomogeneity of the network. Controlling the spatial distribution of autocatalytic species in a network of reactors requires simultaneous manipulation of interconnection flow rates within the system. In a single reactor configuration, only one autocatalytic species is able to survive at a time in a stable manner. Furthermore, numerical experiments suggest that individual CSTRs in networks are capable of hosting only a single dominant species, while other competing species may be present only in trace quantities. Consequently, if the control objective calls for one species to be replaced with another, a nonlinear control scheme must be used.

For a single CSTR with competing autocatalytic species, the reactor residence time must first be modified such that the undesirable species is washed out of the system, and then set to an appropriate value that is favorable to the existence of the desired species. This concept can be extended to systems with many reactors to effectively control the spatial distribution of autocatalytic species in the network. However, the control problem becomes complex because each CSTR has a feed and exit stream, as well as multiple interconnections to its neighbor(s). Each manipulated variable (interconnection flow rates, for example) requires an actuator and control structure, along with the appropriate number of sensors for each reactor.

Intelligent supervisory knowledge-based control systems have been implemented to control a distributed process with changing operating conditions in an adaptive manner. The operation of highly nonlinear systems like autocatalytic replicator networks may benefit from evolutionary self-organizing control because the optimal operating regime and the required control strategies may not be known a priori. Agent-based control systems provide the capability for localized and global control strategies that are both reactive in controlling disturbances and proactive in searching for better operational solutions. This paper proposes an adaptive agent-based control system for a CSTR network. Finding and maintaining operating states that are both stable and efficient is an ongoing research challenge since the complex non-linear interactions between reactors often leads to undesirable or even unpredicted behavior. What is needed are adaptive control systems that can self-organize into productive patterns and self-correct in the face of unexpected deviations.

Multi-agent control system architectures have several properties that make them particularly attractive for use in supervising large, complex systems. The first, and usually most important in critical systems, is a high level of reliability. Modularity, scalability and adaptability are also attractive features of multi-agent systems. The adaptive and self-regulatory nature of agent systems has only recently been investigated for solving control problems that are normally solved with traditional methods.

The agent design procedure used is a derivative of recent agent design methodologies based on the concept of the agent-services-acquaintance model and the application to manufacturing control. The goal of the design process is to develop an agent based control system for physically distributed industrial processes. Comprehensive studies of the physical process domain provide information regarding the processes' expected normal operating conditions, types of faults and disturbances that may occur, and control strategies. Required agent types and roles are identified based on the requirements for controlling the physical system. After the agent model is specified, the services model can be derived. The services model describes all of the services and responsibilities provided by each agent. Two important distinctions in the agent responsibilities for process control applications are liveness responsibilities and safety responsibilities, the later of which play a critical role in operation of real-life process operations. Finally, the acquaintance model describes how the various agent types communicate with each other.

The architecture consists of several sub-systems, each of which are highly modularized. At the process level, network elements such as reactors and valves interface with the higher-level agents via low-level agents. Each reactor is monitored by an observation agent that is responsible for sampling data requested by other agents as well as storing the data in a history for some specified time. The interconnection flow rates are manipulated by actuation agents that receive commands from control arbitration agents. The next layer in the control hierarchy is the local decision layer. Local decision agents are responsible for monitoring control functions and proactively improving the overall performance of the network based on the control objectives of the individual agents and the reactor network as a whole. Due to the number of control responsibilities of decision agents, each agent may use sub-agents.

Although information may be exchanged between agents, these interactions are local and inherently limit the amount and quality that can propagate through the agent control structures. A global knowledge representation serves as an environment for indirect communication between agents. This concept builds upon the hierarchical structuring of the control system by adding a mechanism for communication and reinforcement of ideas. The information in the knowledge space is divided into categories including local control objectives, control heuristics, and data-based models.

Information exchange occurs indirectly between agents because agents asynchronously read/write information to/from the knowledge space. For example, a particular agent may discover a local control strategy that works particularly well in meeting an objective set by a supervisor. This strategy is cataloged in the knowledge space by the originating agent. Other agents may read this strategy from the knowledge space and implement it to satisfy their particular control objective. The value of the strategy is then rated by the agents that adopt this new strategy such that its value relative to others is promoted. Similarly, outdated information in the knowledge space continuously decreases in value and eventually may be deleted from the knowledge space.

The reactor network model and agent-based control system is implemented with the open source agent modeling and simulation environment RePast. The RePast toolkit is a java-based framework for agent simulation and provides features such as an event scheduler and visualization tools. The control agents created with RePast interact with virtual representations of the physical reactor network. The virtual network objects map the states of the physical system to objects that can be manipulated by the control objects. The interface between the physical network to the agent environment can take the form of a data acquisition system in the case of a real world control application, or in this case, a simulation of a chemical reactor network. The ordinary differential equations that describe the autocatalytic reactions in each CSTR are solved numerically using the CVODE solver. The solver code is written in C and linked with RePast via the Java Native Interface (JNI).

The performance of the agent-based control architecture is demonstrated in a case study to control the distribution of autocatalytic species in a network of 49 (7x7 grid) reactors hosting three autocatalytic species using the interaction flow rates as the manipulated variables. The spatial distribution of species determines the overall product composition that exits the network. By manipulating the spatial configuration of the autocatalytic species residing in the network, one can produce multiple grades of product by simply reconfiguring the connectivity of the network.