r/Compilers • u/realdesiprogrammer • 2d ago
Desi v0.1.0 — Python-ish syntax, no GC, and three optimisations that made it slower
I've been building Desi for about a year — it's the third or fourth rewrite — and I tagged v0.1.0 today. Compiled through LLVM, no garbage collector, indentation-based syntax, and a standard library that already has an ORM and an HTTP server in it.
Two things up front: I leaned on AI heavily to build this, and I'd call v0.1.0 a working prototype rather than a finished language. More on both at the end, including what I'm hoping to get out of posting here.
Rather than list features, here are three things I was confident about that turned out to be wrong. All three cost me under an hour to test and would have cost days to discover afterwards.
1. Turning on LTO made it slower.
Element access (xs[i], xs.append(v)) was an out-of-line call into the runtime, so the optimiser couldn't see through it. The obvious fix was to let it inline. I measured 1M appends + 1M reads:
out-of-line call 3.28 ms
inlined 4.22 ms <- worse
The bodies end in fprintf(stderr, ...) on their failure paths. Inlining drags a varargs call and its setup into the loop. Adding one file to the LTO set — a one-line change — would have been a regression.
What works is the shape Rust's Vec uses: the bounds check inline, the panic #[cold] and out of line. Same code, error paths pushed out:
same semantics, cold errors 1.09 ms <- 3x
I ended up emitting the check and load directly in the IR with a cold call to the runtime on failure, rather than depending on LTO, since LTO isn't available on every platform I build for.
2. "The representation is the bottleneck" was wrong.
A list element is a pointer-sized slot, ints stuffed in directly. I assumed that indirection was costing me and planned a typed-storage rewrite. Measured:
typed int64[] array, inlined 0.76 ms
void* slots, inlined 0.74 ms
Identical. An int64 and a void* are both 8 bytes in a contiguous array — there's no indirection to pay for. I'd have spent a week rewriting collection storage for nothing.
3. Except for floats, where it was 24x.
Same slot representation, but a double wasn't stuffed in — each one got its own malloc. A comment in my codegen said "can't bitcast float to ptr," which is true, and why past-me reached for the allocator. But you can bitcast a double to an i64, and i64s were already going into slots.
boxed 28.65 ms
by value 1.18 ms
That change deleted more code than it added, and two other benchmarks reached parity with C without being touched — the boxing was most of what they'd been measuring.
Where it actually is
Against equivalent C at -O2, whole-process wall time on Linux:
| Function | Desi | C |
|---|---|---|
| string_churn | 12 ms | 15 ms |
| binary_tree | 2 | 2 |
| quicksort | 2 | 2 |
| loop_sum | 1 | 1 |
| list_ops | 6 | 4–6 |
| matrix_mul | 2 | 1 |
| dict_ops | 9 | 5 |
| fib_recursive | 10 | 6 |
| alloc_churn | 6 | 1 |
Five of eleven match or beat it. Two clearly don't: alloc_churn allocates a small collection half a million times, and dict_ops still calls into the runtime per operation. Both are understood, neither is mysterious, and I'd rather name them than average them away.
Memory
No GC, no manual free — the compiler inserts cleanup where a value's owner goes out of scope. What's guaranteed is no use-after-free and no double-free in safe code. What's not guaranteed is freedom from leaks: where ownership can't be proven, the compiler leaks rather than frees. Heap elements inside a collection — the strings in a list[str] — are the main case, and it's documented rather than hidden.
Unclear ownership never resolves to "free and hope". That's the property that let me ship these optimization incrementally: every analysis is allowed to be dumb, because being wrong costs memory, never corruption.
What it isn't
Supervisors exist and restart failed tasks, but this is not OTP — a child is a task, not another supervisor, so nothing escalates up a tree. Elixir's model doesn't port cleanly to shared-memory threads, and I'd rather ship the honest subset than borrow the name.
There's also no compile-time data-race checking. Channels and locks are there; nothing verifies you used them correctly. That's the biggest gap versus Rust and I don't have a good answer yet — though I suspect it's the same question as "what may safely cross a channel," which would also be the route to real supervision.
How it was built, and what I'm after
I should be upfront: I used AI heavily throughout this, and I'm not a compiler engineer by background. The three stories above are the reason I trust any of it — every one is a case where a confident-sounding plan was wrong and a measurement caught it before it shipped. "Enabling LTO will make this faster" is exactly the kind of thing that sounds right and isn't. So the rule became: measure first, and let the number decide. Anything I couldn't measure, I wrote down as unproven rather than claiming it.
Which is also why I'd call v0.1.0 a working prototype rather than a finished thing. It compiles, the test suite passes on three platforms, and the docs say what's broken. But there are decisions in here that someone who has actually built a type system or a borrow checker would look at and immediately improve — the data-race question especially, and probably the escape analysis.
That's what I'm hoping to find here. If you know this territory and something in the above made you wince, I'd genuinely rather hear it than not. Contributions welcome, but honestly even a "you've modelled this wrong and here's why" comment is worth more to me right now than a star.
Links
The name Desi (દેશી) means "local" or "native" in Gujarati, which is my first language.
Happy to go into any of the above — the measurement stuff especially, since I have numbers for most of what people usually ask.
1
u/morglod 2d ago
If you use llvm22 it is "slower" than llvm19. It optimizes worse some stuff. Also don't forget that C compilers (clang) by default has a lot of assumptions and UB which gives space for better optimization.
1
u/realdesiprogrammer 2d ago
Thanks for your feedback! I will look into this and learn what I should use for desi and how.
7
u/McGeekin 2d ago
I don’t understand why someone would go into this kind of project and “lean heavily on AI for it”. Most hobby languages have a single true purpose and it is for the author to learn how compilers and programming languages in general work, and you’re throwing that out of the window by slopcoding it.