InstGenIE: Generative Image Editing Made Efficient with Mask-aware Caching and Scheduling
Journal:
arXiv
Published Date:
May 27, 2025
Abstract
Generative image editing using diffusion models has become a prevalent
application in today's AI cloud services. In production environments, image
editing typically involves a mask that specifies the regions of an image
template to be edited. The use of masks provides direct control over the
editing process and introduces sparsity in the model inference. In this paper,
we present InstGenIE, a system that efficiently serves image editing requests.
The key insight behind InstGenIE is that image editing only modifies the masked
regions of image templates while preserving the original content in the
unmasked areas. Driven by this insight, InstGenIE judiciously skips redundant
computations associated with the unmasked areas by reusing cached intermediate
activations from previous inferences. To mitigate the high cache loading
overhead, InstGenIE employs a bubble-free pipeline scheme that overlaps
computation with cache loading. Additionally, to reduce queuing latency in
online serving while improving the GPU utilization, InstGenIE proposes a novel
continuous batching strategy for diffusion model serving, allowing newly
arrived requests to join the running batch in just one step of denoising
computation, without waiting for the entire batch to complete. As heterogeneous
masks induce imbalanced loads, InstGenIE also develops a load balancing
strategy that takes into account the loads of both computation and cache
loading. Collectively, InstGenIE outperforms state-of-the-art diffusion serving
systems for image editing, achieving up to 3x higher throughput and reducing
average request latency by up to 14.7x while ensuring image quality.
