About
LinkML-Store is an early effort to provide a unifying storage layer over multiple different backends, unified via LinkML schemas.
The overall goals are to provide:
Make it easier to work with data in different forms (tabular, JSON, columnar, RDF)
Expressive validation at scale, including full referential integrity validation
Ability to mix and match different backends (e.g. DuckDB, MongoDB, Solr, ChromaDB, HDF5)
Composability of different search indexes, including LLM textual embeddings
LAMP-like stack for LinkML
Installation
pip install linkml-store
Quickstart
See the tutorials
Data Model
A
Client
provides a top-level interface over one or more databases.A
Database
consists of one or more possibly heterogeneous collections.A
Collection
is a queryable set of objects of a similar type.A
Indexer
creates indexes over collections to enable efficient searching.
Adapters
The current backends are:
DuckDB
(supports read and write)MongoDB
(supports read and write)Solr
(write not yet supported)ChromaDB
(pre-alpha)HDF5
(pre-alpha)
Indexing
This frameworks also allows composable indexes. Currently two indexers are supported:
SimpleIndexer
Simple native trigram methodLLMIndexer
LLM text embedding
Metadata and Configuration
ClientConfig
provides a structure for configuring the client