An influent responsive control strategy with machine learning: Q-learning based optimization method for a biological phosphorus removal system.

Journal: Chemosphere
PMID: 31252361

Abstract

Biological phosphorus removal (BPR) is an economical and sustainable processes for the removal of phosphorus (P) from wastewater, achieved by recirculating activated sludge through anaerobic and aerobic (An/Ae) processes. However, few studies have systematically analyzed the optimal hydraulic retention times (HRTs) in anaerobic and aerobic reactions, or whether these are the most appropriate control strategies. In this study, a novel optimization methodology using an improved Q-learning (QL) algorithm was developed, to optimize An/Ae HRTs in a BPR system. A framework for QL-based BPR control strategies was established and the improved Q function, Q(s,s)=Q(s,s)+k·[R(s,s)+γ·maxaQ(s,s)-Q(s,s)] was derived. Based on the improved Q function and the state transition matrices obtained under different HRT step-lengths, the optimum combinations of HRTs in An/Ae processes in any BPR system could be obtained, in terms of the ordered pair combinations of the . Model verification was performed by applying six different influent chemical oxygen demand (COD) concentrations, varying from 150 to 600 mg L and influent P concentrations, varying from 12 to 30 mg L. Superior and stable effluent qualities were observed with the optimal control strategies. This indicates that the proposed novel QL-based BPR model performed properly and the derived Q functions successfully realized real-time modelling, with stable optimal control strategies under fluctuant influent loads during wastewater treatment processes.

Authors

  • Ji-Wei Pang
    State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, PR China.
  • Shan-Shan Yang
    State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, PR China. Electronic address: shanshanyang@hit.edu.cn.
  • Lei He
    Guangxi Medical University, Nanning 530021; State Key Laboratory of Pathogen and Biosecurity, Beijing 100071, China.
  • Yi-Di Chen
    State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, PR China.
  • Guang-Li Cao
    State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, PR China.
  • Lei Zhao
    Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China.
  • Xin-Yu Wang
    State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, PR China.
  • Nan-Qi Ren
    State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150090, China.

Keywords

External Resources

Popular Topics
Recent Journals