DynoStore: A wide-area distribution system for the management of data over heterogeneous storage
Journal:
arXiv
Published Date:
Jul 1, 2025
Abstract
Data distribution across different facilities offers benefits such as
enhanced resource utilization, increased resilience through replication, and
improved performance by processing data near its source. However, managing such
data is challenging due to heterogeneous access protocols, disparate
authentication models, and the lack of a unified coordination framework. This
paper presents DynoStore, a system that manages data across heterogeneous
storage systems. At the core of DynoStore are data containers, an abstraction
that provides standardized interfaces for seamless data management,
irrespective of the underlying storage systems. Multiple data container
connections create a cohesive wide-area storage network, ensuring resilience
using erasure coding policies. Furthermore, a load-balancing algorithm ensures
equitable and efficient utilization of storage resources. We evaluate DynoStore
using benchmarks and real-world case studies, including the management of
medical and satellite data across geographically distributed environments. Our
results demonstrate a 10\% performance improvement compared to centralized
cloud-hosted systems while maintaining competitive performance with
state-of-the-art solutions such as Redis and IPFS. DynoStore also exhibits
superior fault tolerance, withstanding more failures than traditional systems.
