Why an Open Source JSON-LD Database

Hacker News

Share card

Why an Open Source JSON-LD Database

We are pleased to launch a public preview of an open source JSON-LD database which combines the simplicity of a document db with the analytical power of semantic graph. We hope it is approachable for most any developer while having the capabilities, when needed, for data interoperability, embedded cell-level security (ReBAC style), rich shape/schema restrictions, data provenance (time travel), and fact inferencing. Why did our team at Fluree build this? Data is increasingly critical for great decision making, AI, and more. The way we typically store and manage data, mainly as an application state store, makes it challenging to leverage data strategically. The popular approaches to leveraging data outside of an app are difficult to scale and have limited effectiveness (e.g. copy/paste of stale, un-permissioned data into a lake/ warehouse, custom python scripts for feature enrichment, ETL tools). We believe a shift in view for data management from application-centric to data-centric can address the root issue and reduce the needs of the additional layers. If all data was semantically interoperable (composable), available in a network where data was current, and data has more native capabilities around security and inferencing, we can address the foundational challenges – and is what we set out to build. Semantically interoperable data is not a new thing, semantic web/knowledge graph users enjoy many great database products, but the focus has been analytics and inferencing, and often run on a laptop or isolated server for an analyst – plus, they are considered to have a steep learning curve. Having a platform that is optimized for operational data and minimizing the learning curve, while retaining the power, are the largest obstacles we had to overcome. Beyond semantics, data that can be dynamically shared and collaborated on in a network is where we think real data leverage emerges, but data owners need programmatic policy control to agree to participate. Controlling time also becomes critical, as reproducing results across a federated network is impossible without it. We built programmability into the data layer to address all of these needs, and shift our thinking from a “database” being a mutable thing, to an immutable thing - where every update to a database creates a new immutable database, much like every “git” commit creates a new immutable version of your source code. We are still early on our journey, but would love any feedback, thoughts, help (we are open source). We hope our vision aligns with many and the product resonates. Feel free to register for an account on our cloud version, and reserve your data namespace in what we’d like to become a valuable and trusted data network.