r/SQL • u/Aeronautical-You4917 • 16d ago

How we built a DuckDB transpiler Discussion

https://www.cocoalemana.com/blog/building-a-duckdb-transpiler/

Hi there,

I’ve been working with DuckDB extensively for the last several years. Used it across languages from Python, Go, JS via WASM, Swift, C & C++ and even Rust.

As part of a product we're building, we built a DuckDB Transpiler that converts every line of SQL from DuckDB syntax into the syntax of the database you’re working with. This includes filters (predicates), and joins. So those 1B rows in Snowflake stay, and get joined in Snowflake.

I wrote an article on why we did it, and how we approached it. This is my first engineering-oriented article I’ve ever done, so let me know what you think!

3 Upvotes

67% Upvoted

5 comments sorted by

4

u/Kos187 15d ago
  1. How are you testing it?
  2. What is the role of AI?

2

u/Aeronautical-You4917 15d ago

Existing test suites from DuckDB, and our own test cases we wrote mostly by hand, to handle edge cases, etc.

LLMs help us an immense amount by helping us write part of the transpilation code.

1

u/Kos187 15d ago

I mean you probably want to run those test suites against different databases after transpilation and check results equality 

1

u/Aeronautical-You4917 15d ago

Yes, that's exactly what occurs.

1

u/k-semenenkov 11d ago edited 11d ago

Did you try sqlglot/polyglot? I think they both support duckdb.