A Linearizable (Strongly Consistent) Distributed Key/Value Store with RAFT Distributed Consensus + On-Disk Hash Table + Data Partitioning with Consistent Hashing

Architecture

On-Disk Hash Table (Supports all CRUD operations)

The hash table is stored in a single file. The key is hashed to a cell in the index which stores the address of where the data is in the data section. Hash collisions are resolved with a logical linked list. Each row in the data section is a node in the linked list which points to the address of the next node.

Raft Leader Election

Leader election among nodes with ports: 3001, 3002, 3003

Node with port 3001 looses election:

Node with port 3003 looses election:

Node with port 3002 wins election, becomes leader and sends heartbeats:

Automatic Failover - Electing a new Leader

Leader with port 3002 goes down and follower with port 3003 is first to start and win the election:

To further improve the database, the following features can be added:

• Security, Encryption, and Authentication
• Service Discovery (ZooKeeper/Etcd)
• More Efficient File Reading/Writing
• Rigorous Distributed Testing
• Data Compression
• Data Integrity/Checksums