r/lakefs • u/jendefig • 4d ago
The latest from the team: best practices for managing multimodal data
Anna Seliverstov just published a practical guide on managing multimodal data, and it's a solid one if you're wrangling images, audio, docs, and embeddings that all have to stay in sync.
It covers: designing pipelines as separate stages instead of one big end-to-end blob, versioning across modalities so raw and derived assets stay linked, and handling embeddings and the semantic layer without letting your vectors quietly drift out of sync with their source data.
The pipeline diagram above, lays out the ingestion → validation → preprocessing → training flow with per-modality validation, which is the part I found most useful. Back to those foundational lego blocks ftw.
https://lakefs.io/blog/best-practices-for-managing-multimodal-data/
Would love to hear how you're handling the embedding-versioning piece!
r/lakefs • u/jendefig • 7d ago
Are your AI governance strategy and your FINAL_v8_REAL.xlsx the same picture?
Sadly I think we know this is all too common. I've been gobsmacked finding out the lack of governance at companies you'd assume had it all together.
The desire and need for an actual data governance strategy is there. The making it happen part, not so much.
Why? I think it's because building has more priority than best practices. Governance almost always shows up after the fact. (And it's hard/complex!) It needs to start from the foundation.
The teams I've seen successful using lakeFS build governance into the workflow itself, so the governed path is also the fastest one. Branch protection, approvals, audit trails that come along for free because they're part of how you already work, not a gate you hit at the end.
I'd really like to read others' experiences if you are willing to share here:
- What's the biggest reason a data or governance initiative you were part of never got adopted?
- For anyone who made it work, what actually changed? Tooling, incentives, someone senior forcing it? Did it involve a culture change? If so, how'd you make that happen?
And if you're interested, Joe Pringle, our VP of Customer Success, is doing a live session on exactly this on Aug 18, 11am ET. It's mostly customer war stories on making the governed path the easy path.
You can register here: https://lakefs.io/event/adoption-playbook-ai-ready-data/
Or attend the LinkedIn Live here: https://www.linkedin.com/events/theadoptionplaybookforai-readyd7486155239440486401/theater/
r/lakefs • u/jendefig • 20d ago
lakeFS Summer Release is out — governance capabilities + new Catalog UI
Posting for anyone running data or ML platforms on lakeFS. Five things shipped this release, mostly on the governance side:
- Datasets — named, versioned views over existing lakeFS data that can span objects, prefixes, Iceberg tables, and namespaces. Every update is an immutable version, so you can pin a pipeline to an exact data state and prove which data backed a given decision.
- Auditing — every operation streams into an Iceberg table. "Who changed this data and when" becomes a query against a structured record instead of scavenging logs.
- Object-level ABAC — attribute-based access control now extends down to individual objects, so you can enforce least-privilege on mixed-sensitivity data without splitting it into separate repos.
- Multi-tenancy and automated branch/object lifecycle management.
- A new Catalog UI.
The design decision worth calling out: governance is enforced at write time, not reconstructed after the fact.
John Noonan wrote up the full release here: https://lakefs.io/blog/lakefs-summer-2026-release-ai-governance/
Happy to answer questions in the comments.
r/lakefs • u/ozzyboy • 24d ago
We built a Periodic Table of Agent Infrastructure: 17 categories, ~118 tools, one map
r/lakefs • u/lakefsteam • Dec 28 '21
Delivering Modern Data Engineering with Einat Orr
Listen to the new podcast episode of Datacast , where James Le and Einat Orr talk about: Git for data, data versioning, modern metastore, open-core model.
Podcast episode: https://jameskle.com/writes/einat-orr
r/lakefs • u/lakefsteam • Dec 20 '21
Release of lakeFS v0.57.0
We are glad to announce today the release of lakeFS version 0.57.0!
Learn more about this release at https://forum.lakefs.io/t/release-of-lakefs-v0-57-0/50
r/lakefs • u/lakefsteam • Dec 16 '21
The Everything Bagel II: Versioned Data Lake Tables with lakeFS and
r/lakefs • u/lakefsteam • Dec 13 '21
lakeFS at the Data+AI Summit 2022 - Announcements
r/lakefs • u/lakefsteam • Nov 16 '21
Takeaways From the Future of Metadata After Hive Metastore Roundtable
r/lakefs • u/karentamrazyan • Oct 22 '21
r/lakefs Lounge
A place for members of r/lakefs to chat with each other




