Abstract:
The processing power of traditional standalone systems is no longer sufficient to handle the increasing amount of data in the era of big data. Large scale distributed key value (K/V) storage systems have become one of the core infrastructure for enterprises in the new era. The distributed consensus protocol in the K/V service system is the key to ensuring data consistency and high availability in a distributed environment. The development of modern hardware storage technology and high-speed network technology has put forward higher requirements for the performance of K/V systems, and the consensus protocol layer has gradually become a bottleneck that constrains the system performance. In response to the high response speed requirements of K/V storage services, this paper proposes a high-performance consensus protocol for K/V service systems based on the reliable and available distributed fault-tolerant consensus protocol (Raft) , improved Raft, IRaft: the replication state machine model of the Raft protocol is extended, and based on the commutativity among instructions of different keys, the total order relation of state machine operations is relaxed to a partial order relation through instruction dependency analysis, so that operations without dependency relationships can be swapped. By designing a Pending Order Pool, Partitioned Log, and a fast completion and regular completion decision structure, batch submission of independent instructions is achieved, supporting the parallel execution of independent instructions in upper level applications. By conducting multiple comparative experiments on K/V storage systems based on Raft and IRaft protocols, the experimental results show that the system based on IRaft protocol generally has better performance.