Application of Reinforcement Learning Algorithm to Minimize the Dosage of Insulin Infusion

Abstract

The control of insulin infusion pumps has been receiving attention from various sectors since artificial intelligence (AI) has been successfully applied to many optimal control problems. Data-driven control has become more popular with help of AI without dynamic model structure, but a white-box control would be more efficient if there exist an identified model structure. In the case of insulin pump control design, traditional mathematical glucose-insulin models will be very useful to find an optimal insulin infusion profile by applying AI technique. The objective of insulin pump control in this study is to minimize the total amount of insulin dosage while maintaining the plasma glucose concentration level within a healthy range by controlling the profile of insulin infusion rate. Reinforcement learning is used to address this dynamic glucose-insulin interaction problem with a goal of achieving minimum insulin dosage. Reinforcement learning in this study finds an optimal insulin infusion profile with minimum dosage that stabilizes a plasma glucose concentration from an initial hyperglycemic state to a basal level in a reasonable amount of time. Since diabetes is known as a chronic disease, minimal insulin dosage control may be important to decrease potential insulin resistance and play an important role in the long-term treatment of diabetes patients. An insulin infusion profile will next be generated in the presence of a meal disturbance to understand the ability of a proposed reinforcement learning algorithm to be insensitive to disturbances in glucose and act as a normal human pancreas. Numerical examples are provided of glucose-insulin dynamics models, and the proposed insulin infusion profiles are verified by comparing them with two optimal insulin infusion programs suggested by Fisher's study and a normal physiological plasma insulin response to a meal, respectively.