Deploying Foundation Model Powered Agent Services: A Survey
Journal:
arXiv
Published Date:
Dec 18, 2024
Abstract
Foundation model (FM) powered agent services are regarded as a promising
solution to develop intelligent and personalized applications for advancing
toward Artificial General Intelligence (AGI). To achieve high reliability and
scalability in deploying these agent services, it is essential to
collaboratively optimize computational and communication resources, thereby
ensuring effective resource allocation and seamless service delivery. In
pursuit of this vision, this paper proposes a unified framework aimed at
providing a comprehensive survey on deploying FM-based agent services across
heterogeneous devices, with the emphasis on the integration of model and
resource optimization to establish a robust infrastructure for these services.
Particularly, this paper begins with exploring various low-level optimization
strategies during inference and studies approaches that enhance system
scalability, such as parallelism techniques and resource scaling methods. The
paper then discusses several prominent FMs and investigates research efforts
focused on inference acceleration, including techniques such as model
compression and token reduction. Moreover, the paper also investigates critical
components for constructing agent services and highlights notable intelligent
applications. Finally, the paper presents potential research directions for
developing real-time agent services with high Quality of Service (QoS).
