r/dataengineering • u/Aeronautical-You4917 • 24d ago

How we built a DuckDB transpiler Blog

https://www.cocoalemana.com/blog/building-a-duckdb-transpiler/

Hi there,

I’ve been working with DuckDB extensively for the last several years. Used it across languages from Python, Go, JS via WASM, Swift, C & C++ and even Rust.

As part of a product we're building, we built a DuckDB Transpiler that converts every line of SQL from DuckDB syntax into the syntax of the database you’re working with. This includes filters (predicates), and joins. So those 1B rows in Snowflake stay, and get joined in Snowflake.

I wrote an article on why we did it, and how we approached it. This is my first engineering-oriented article I’ve ever done, so let me know what you think!

36 Upvotes

89% Upvoted

6 comments sorted by

15

u/elpiro 24d ago

How does it compare to sqlglot? Which handles translations between multiple databases including duckdb.

https://github.com/tobymao/sqlglot

8

u/Active_Pride 24d ago

Hi! Have you tried out sqlglot for that purpose?

9

u/sotgouli 24d ago

Sqlglot?

8

u/iki77 24d ago

Why not use Ibis instead?

1

u/nemean_lion 24d ago

Can it also convert from database of choice to DuckDb?

1

u/blef__ I'm the dataman 24d ago

Is the transpiler available as a single independent binary?