• 中国期刊全文数据库
  • 中国学术期刊综合评价数据库
  • 中国科技论文与引文数据库
  • 中国核心期刊(遴选)数据库
陆华成, 王勇, 李志珂. 基于集群拓扑和存储池感知的Ceph数据均衡方法[J]. 桂林电子科技大学学报, xxxx, x(x): 1-8. DOI: 10.3969/1673-808X.202314
引用本文: 陆华成, 王勇, 李志珂. 基于集群拓扑和存储池感知的Ceph数据均衡方法[J]. 桂林电子科技大学学报, xxxx, x(x): 1-8. DOI: 10.3969/1673-808X.202314
LU Huacheng, WANG Yong, LI Zhike. Ceph data balancing method based on cluster topology and storage pool awareness[J]. Journal of Guilin University of Electronic Technology, xxxx, x(x): 1-8. DOI: 10.3969/1673-808X.202314
Citation: LU Huacheng, WANG Yong, LI Zhike. Ceph data balancing method based on cluster topology and storage pool awareness[J]. Journal of Guilin University of Electronic Technology, xxxx, x(x): 1-8. DOI: 10.3969/1673-808X.202314

基于集群拓扑和存储池感知的Ceph数据均衡方法

Ceph data balancing method based on cluster topology and storage pool awareness

  • 摘要: 分布式存储系统采用低成本的设备提供可扩展、灵活的数据存储服务,大大降低了系统建设成本。然而,在Ceph存储系统中设备容量的异构性和存储池的多样性会导致副本数据分布不平衡,对系统的性能和可靠性带来了新的挑战。针对该问题,提出一种基于集群拓扑和存储池感知的数据均衡方法,在优化存储系统的均衡度和性能的同时,避免存储容量较小的设备承担过多的工作负载,也降低大存储容量设备的资源浪费。首先设计动态全局权重平衡器DGWBalancer,综合考虑集群拓扑结构、集群存储使用率、设备存储利用率和存储池等多个因素,通过贪心算法获得相应数据分布节点选择策略,动态全局地调整集群中各种设备以及存储池中存储设备的权重,使数据更加合理、均衡地分布到各种设备上,达到提升云存储服务的可靠存储利用率和性能的目的。实验结果表明,与Ceph现有的mgr balancer相比,DGWBalancer在数据均衡方面能够取得更好的效果,提高了350%的均衡度和13.5%的集群可靠存储利用率;在性能方面,提高了10%的吞吐量和17%的IOPS。

     

    Abstract: The distributed storage system uses low-cost equipment to provide scalable and flexible data storage services, which greatly reduces the system construction cost. However, the heterogeneity of device capacity and the diversity of storage pool in Ceph storage system will lead to the unbalanced distribution of replica data, which brings new challenges to the performance and reliability of the system. To solve this problem, a data balancing method based on cluster topology and storage pool awareness is proposed. While optimizing the balance and performance of the storage system, it avoids the equipment with small storage capacity from bearing too much workload, and also reduces the resource waste of the equipment with large storage capacity. First, the dynamic global weight balancer DGWBalancer is designed, which comprehensively considers the cluster topology, cluster storage utilization, device storage utilization and storage pool, and obtains the corresponding data distribution node selection strategy through greedy algorithm, dynamically and globally adjusts the weight of various devices in the cluster and storage devices in the storage pool, so that the data can be more reasonably and evenly distributed to various devices, Achieve the purpose of improving the reliable storage utilization and performance of cloud storage services. The experimental results show that compared with Ceph's existing mgr balancer, DGWBalancer can achieve better results in data balancing, improving the balance degree by 350% and the cluster reliable storage utilization rate by 13.5%; In terms of performance, it improved throughput by 10% and IOPS by 17%.

     

/

返回文章
返回