SuperMassive – Fast durable, in-memory, distributed key-value database

Hacker News

Share card

SuperMassive – Fast durable, in-memory, distributed key-value database

Hey hn! I hope you're all doing well. I’d like to share a new open-source database I’ve designed and written called SuperMassive. SuperMassive is a scalable, distributed, in-memory key-value database designed from the ground up to allow for high concurrency, fast writes, fault tolerance, and durability while remaining simple to use and efficiently scalable. The idea for SuperMassive comes from its name.. I wanted to build a key-value database that can scale infinitely, remain durable, be self-healing, consistent, and blazing fast. I also wanted to simplify sharding and replication compared to existing distributed KV stores and their protocols. SuperMassive is built to be simple yet powerful. It consists of just one multiplatform supported binary that can run in multiple modes(cluster, primary node, or replica). Nodes function as shards of your data. The design emphasizes minimal configuration, high performance, and automatic failover. While SuperMassive is still in its early stages (v1.0.2b), I’d love to hear your thoughts on the design, its architecture, and any feedback you may have :D Features~~ ~ Highly scalable Scale horizontally with ease. Simply add more nodes to the cluster. ~ Distributed Data is distributed across multiple nodes in a sharded fashion. ~ Robust Health Checking System Health checks are performed on all nodes, if any node is marked unhealthy we will try to recover it. ~ Smart Data Distribution uses a sequence-based round-robin approach for distributing writes across primary nodes. This ensures that all primary nodes get an equal share of writes. ~ Automatic Fail-over Automatic fail-over of primary nodes on write failure. If a primary node is unavailable for a write, we go to the next available primary node. ~ Parallel Read Operations Read operations are performed in parallel. ~ Consistency Management Timestamp-based version control to handle conflicts. The most recent value is always returned, the rest are deleted. ~ Fault-tolerant Replication and fail-over are supported. If a node goes down, the cluster will continue to function. ~ Self-healing Automatic data recovery. A node can recover from a journal. A node replica can recover from a primary node via a check point like algorithm. ~ Cluster Authentication with basic like auth. ~ Simple Protocol Simple protocol PUT, GET, DEL, INCR, DECR, REGX, STAT, RCNF, PING. ~ Async Node Journal Operations are written to a journal asynchronously. This allows for fast writes and recovery. ~ Multi-platform Linux, Windows, MacOS https://supermassivedb.com https://github.com/supermassivedb/supermassive - Alex