Generative Physical AI in Vision: A Survey
Journal:
arXiv
Published Date:
Jan 19, 2025
Abstract
Generative Artificial Intelligence (AI) has rapidly advanced the field of
computer vision by enabling machines to create and interpret visual data with
unprecedented sophistication. This transformation builds upon a foundation of
generative models to produce realistic images, videos, and 3D/4D content.
Conventional generative models primarily focus on visual fidelity while often
neglecting the physical plausibility of the generated content. This gap limits
their effectiveness in applications that require adherence to real-world
physical laws, such as robotics, autonomous systems, and scientific
simulations. As generative models evolve to increasingly integrate physical
realism and dynamic simulation, their potential to function as "world
simulators" expands. Therefore, the field of physics-aware generation in
computer vision is rapidly growing, calling for a comprehensive survey to
provide a structured analysis of current efforts. To serve this purpose, the
survey presents a systematic review, categorizing methods based on how they
incorporate physical knowledge, either through explicit simulation or implicit
learning. It also analyzes key paradigms, discusses evaluation protocols, and
identifies future research directions. By offering a comprehensive overview,
this survey aims to help future developments in physically grounded generation
for computer vision. The reviewed papers are summarized at
https://tinyurl.com/Physics-Aware-Generation.
