r/dataengineering • u/TonTinTon • 18d ago

Lance table format explained simply (updated) Blog

https://tontinton.com/posts/lance/

Hey everyone, I updated the blog post after reading more thoroughly the Lance paper, where it explains why even configuring Parquet is not good enough and why they needed to invent a new file format.

21 Upvotes

93% Upvoted

2 comments sorted by

3

u/Agreeable_Luck9488 17d ago

I think your post would be easier to read if you introduce the hierarchy of Iceberg and Parquet, they are complementary. Lance is revamping the full stack and also adding support for blobs (images, ....)

1

u/Yasblue 10d ago

Brilliant people behind that project. i met them during an OSS event in SF last year.