MaintaAvatar: A Maintainable Avatar Based on Neural Radiance Fields by Continual Learning
Journal:
arXiv
Published Date:
Feb 4, 2025
Abstract
The generation of a virtual digital avatar is a crucial research topic in the
field of computer vision. Many existing works utilize Neural Radiance Fields
(NeRF) to address this issue and have achieved impressive results. However,
previous works assume the images of the training person are available and fixed
while the appearances and poses of a subject could constantly change and
increase in real-world scenarios. How to update the human avatar but also
maintain the ability to render the old appearance of the person is a practical
challenge. One trivial solution is to combine the existing virtual avatar
models based on NeRF with continual learning methods. However, there are some
critical issues in this approach: learning new appearances and poses can cause
the model to forget past information, which in turn leads to a degradation in
the rendering quality of past appearances, especially color bleeding issues,
and incorrect human body poses. In this work, we propose a maintainable avatar
(MaintaAvatar) based on neural radiance fields by continual learning, which
resolves the issues by utilizing a Global-Local Joint Storage Module and a Pose
Distillation Module. Overall, our model requires only limited data collection
to quickly fine-tune the model while avoiding catastrophic forgetting, thus
achieving a maintainable virtual avatar. The experimental results validate the
effectiveness of our MaintaAvatar model.
