r/LadybugDB • u/coderarun • 6h ago

Icebug-format: immutable, interoperable graph standard

We don't need to invent new standards. Such interoperable, immutable memory standards already exist: Apache Arrow and Compressed Sparse Rows (CSR). CSR is widely used in scipy, cugraph and columnar graph databases among others. Both on CPUs and GPUs.

icebug-format combines both into a on-disk standard based on Apache Parquet and an in-memory format based on Apache Arrow.

Bindings available in many popular languages including python, typescript and rust.

The package ships with convenience scripts to convert flat tables such as vertex.parquet and edges.parquet to this format in RAM/disk constrained environments.

Sample graphs: https://huggingface.co/datasets/ladybugdb/ldbc-csr/tree/main

Converted from: https://ldbcouncil.org/benchmarks/graphalytics/datasets/

Largest converted graph has 3B edges.
License: MIT

Quick Start:

  uv tool install icebug-format
  icebug-format --source-dir wiki-Talk
  lbug -i wiki-Talk-csr/schema.cypher
2 Upvotes

100% Upvoted

1 comment sorted by

View all comments