(12b) Semi-Centralized Multi-Agent Rl for Irrigation Scheduling

With growing concerns over freshwater scarcity, there is a pressing need for the agricultural sector to conserve water resources while maximizing crop yield [1]. Traditional irrigation scheduling methods fall short in terms of efficiency, mainly because they fail to address the inherent combinatorial nature of the scheduling task. Mixed-integer optimal control techniques have been identified as an effective approach to address the combinatorial nature of the irrigation scheduling problem [2]. However, solving mixed-integer optimal control problems remains challenging, despite significant advances in optimization algorithms and computing power. The complexity of applying mixed-integer optimal control to provide irrigation schedules for large-scale agricultural fields, which are characterized by significant spatial-variability, alongside the need to obtain irrigation schedules in a timely manner, necessitates the development of efficient solution strategies. Within this context, multi-agent reinforcement learning (MARL) has emerged as a promising approach to handle the complexities of mixed-integer optimal control challenges, offering a new paradigm for addressing the irrigation scheduling problem.

This study introduces a semi-centralized multi-agent reinforcement learning (SCMARL) approach to address the challenge of mixed-integer optimal control-based irrigation scheduling in large-scale agricultural fields, delineated into distinct management zones (MZs) due to their spatial variability. The SCMARL framework, which combines the strengths of a centralized agent and decentralized agents, is hierarchical in nature, with a coordinator agent at the top level and local agents at the lower level. The coordinator agent makes daily `yes/no' irrigation decisions based on field-wide observations from all the MZs, which are then communicated to local agents. These local agents are tasked with determining the optimal daily irrigation depths for specific management zones, utilizing both the coordinator agent's decision and local observations. A comparison between the SCMARL method and a Fully Decentralized Multi-agent Reinforcement Learning approach is presented, highlighting the superior performance of the SCMARL approach in terms of water savings (4.0%) and improved irrigation water use efficiency (6.3%).

References

[1]. United Nations World Water Assessment Programme. The United Nations World Water Development Report 2018: Nature-Based Solutions for Water, 2018

[2]. Agyeman, B.T., Nouri, M., Appels, W.M., Liu, J. and Shah, S.L., 2024. Learning-based multi-agent MPC for irrigation scheduling. Control Engineering Practice, 147, p.105908.