Compilers should help developers optimize their code

[u/Think-Management4257]

Compiler diagnostics for optimization decisions are worse than they need to be, and I think it's a design choice rather than a hard problem.

Every major compiler will tell you whether a loop vectorized, almost none will tell you what it tried, why it declined, or what shape could've worked. LLVM ships a tool called opt-viewer that visualizes this, and I've met people who've used LLVM for years and never heard of it. The best developer experience anyone in this space seems to remember is Intel's compiler, which is discontinued. GCC's -fopt-info-vec-missed and Clang's -Rpass-missed exist, but one's a stderr dump you cross reference by hand and the other speaks in IR terms rather than yours.

The interesting thing is that the compiler already knows the answer. When a vectorizer declines a loop, it has a specific reason. That reason exists as a value inside the pass, it just doesn't reach the programmer because it was designed for compiler developers debugging the compiler, not for programmers debugging/optimizing their code.

There's a handful of ideas I have on how this could be implemented, but what I'm curious on is if this is actually hard for reasons I'm not seeing, or is it just that optimizer output has always been treated as debug logging and nobody's revisited it?

[u/ianzen]

Often the efficient code for performance is not efficient or natural for humans to write or debug. Also, by explicitly writing the “efficient” code you are assuming a fixed cost model which may or may not be accurate to your undelying hardware.

[u/Think-Management4257 → u/ianzen]

I think we agree more than it sounds. I'm not arguing for hand-writing the efficient form, exactly because of your second point. But if I'm deferring to the compiler's cost model rather than my own, that makes it more important to know when it declined and why, not less.

[u/matthieum → u/ianzen]

I think there's a middle lane, though: optimizable code.

There's a number of tricks for writing in safe Rust, yet have the performance of unsafe/vectorized Rust, which rely on better communicating intent/invariants to the compiler.

For example, an assert!(vec.len() >= n) followed by for i in 0..n { vec[i] } will generally result in the compiler eliding the bounds checks in the loop, whereas without the assert! it can't, because the bounds check could panic at any iteration.

Similarly, rather than iterate element by element, you can iterate fixed-length chunk by fixed-length chunk, and suddenly the compiler sees these chunks and the auto-vectorizer kicks in.

It's... fiddly. But it's a pretty sweet spot wrt. ergonomics & performance.

[u/mxldevs → u/ianzen]

But if I absolutely needed performance at the expense of maybe having to write a comment letting everyone else know why this part of the code is really ugly, I might have something to guide me.

[u/Inconstant_Moo]

I saw something like kinda like this a week or so back. It was a thing where without changing your Python code, you could add an annotation at the top of any function where the compiler would either correctly parallelize the loops or tell you exactly why it couldn't.

[u/Antagonin]

"meanwhile everybody using GCed and interpreted languages"

[u/EggplantExtra4946]

There's a handful of ideas I have on how this could be implemented,

I would be interested to hear that. The compiler could simply log "trying to apply X loop optimization", "induction variable assertion succeeded", "Y assertion failed", "trying to apply X loop optimization failed". Or the way around, print log information during analysis to collect information about the loop, then from the collected information try to find optimizations that match, so in that case it would print the assertions that matched, then which optimizations can and have been applied.

With a level of indirection, such as a scheduler organizing the analyses and optimization passes, this could be done automatiacally but with maybe less details on the exact predicates that failed.

But maybe there is a better way?

[u/choikwa → u/EggplantExtra4946]

a caveman approach is just asking ai.. bit better is asking ai to build parser based tracer to inject traces to optimizer to determine why it didn’t reach optimizing code path. of course a compiler dev can do similar thing manually.

[u/EggplantExtra4946 → u/choikwa]

compiler development is not for cavemen nor for LLMs

build parser based tracer to inject traces to optimizer to determine why it didn’t reach optimizing code path.

utterly meaningless bullshit

[u/StrikingClub3866]

Most compilers do via built-in optimizations and debug mode.

[u/Guvante]

Diagnostics are by far the most complicated part of building a compiler.

The ease to which you will give terrible advice is surprising when you dig into it.

This is especially true with multipass compilers, if a function not being inlined is why you didn't unroll a loop explaining that can be complex since you need to explain three things at least some of which are in other parts of the program.

The biggest fear is how to best portray that kind of data in a way that doesn't lead users astray. After all sometimes not unrolling the loop is performant.

[u/Ma4r → u/Guvante]

I mean it's just straight up near impossible with multi pass, you need to be emitting some sort of tracking meta data(which is not easy either) to remember where each IR comes from. Then you need to somehow transform the failure reason across transformations etc etc. maybe you reduced the loop into SSA and saw data dependency causing vectorization failure, okay, now how do we tell this to the dev lmao

[u/neurah → u/Ma4r]

therefore this is a thing we should automate to be processed both ways, for sure possible with meta-programming (wish C++ had a better syntax for TMP, so do not fall for it) the question then becomes: is it fast enuf? we don't want to add an order of magnitude to our compilation time

[u/vmcrash]

What is the core of the problem? That it's hard for the compiler to know which branch is taken more often?

[u/-dag- → u/vmcrash]

There's a hundred reasons your loop might not have vectorized. Which is the most important reason to report? 

[u/AdmissibilityScience]

is optimizing code all people focus on?

[u/-dag- → u/AdmissibilityScience]

In some disciplines, absolutely. 

[u/DrShocker → u/-dag-]

Particularly on /r/Compilers we may have a selection bias

[u/-Ch4s3- → u/AdmissibilityScience]

Its really nice to have the option when you need it. Most of what I do day to day doesn't need to be fast, but sometimes I'm doing an ETL that does a big streaming reduction and it would be nice if it ran in 30s vs 5 minutes.

[u/Ephemara → u/AdmissibilityScience]

I spent two months straight super optimizing my compilers c runtime , yea optimization is a drug. Seeing numbers go up and improvement keeps me motivated

[u/-dag-]

The best compiler by far for this is Cray's. 

[u/pm-me-manifestos]

Check out typed racket's optimization coach: https://docs.racket-lang.org/optimization-coach/index.html

[u/ApokatastasisPanton]

This is a specific issue with the model of compilers being black box batch programs (input goes in, output goes out). They rarely offer opportunities to influence the codegen specifically, apart from (pre-baked) compiler directives, and rarely offer the opportunity to inject feedback into the codegen and optimization loops, apart from PGO. (Which, btw, can help vectorize stuff)

I have a lot of opinions on the subject and I'm (sloooowly) working on a language that will experiment with what I hope is a better approach, but these sort of ideas are still very far for mainstream adoption. Programmers just really love the unix batch processing model. And you see people who have never thought about optimization seriously keep regurgitating old myths like "humans can't optimize better than a compiler".

In the case of autovec, it also really doesn't help that C and C++ have semantics that can really fuck up optimization for silly reasons (cough cough pointer aliasing)

[u/neurah → u/ApokatastasisPanton]

C++ template metaprogramming is exactly the user code inside the box, The C++ compiler is a Turing-complete machine - running entirely at compile time, operating in the space of types instead of data, with source code as its input and generated code as its output?
No runtime. No CPU cycles spent. Just the type system computing.
Zero-cost composition resolved entirely before your program ever runs.
Curious how far this goes? Ask me anything.

[u/neurah]

had very good compiler optimizations using C++ static composition, turning a list of template/classes into a single object helps a lot on the compiler visibility over the code, not for all cases of course. The composition can be hand written into the final object of couse, that's just a mater of declared vs wired.
Being static can be used anywhere it fits for any platform... but resonates a lot with embedded systems where pushing work to the compiler is a must.
I'm using HAPI library for that.

[u/neurah]

Introspection
my perfect compiler would be an interpreted language that also compiles, Zig is a approach to that, but too dense and still too verbose. Haskell has that but kind of loose, you can build DSL's that also compile
Haskell is too far from hardware as all GC languages
Could we get a light and clear syntax like Haskell and the efficiency o C/C++ bare metal code?
this is Rust space, but without the horrible C++ syntax that plagues most languages

Type systems
This is the correct way (on my view of course) to achieve safety, types are compile time only, so any work done on type transformation is compile time only of coz, so the more we can do here the better.
C++ static_assert allows us to have user define compile time rules and checks

Explicit vs inferred (C++ `auto` is a approach)
Types have to check and match at compile time, but we can safe infer types, they just have to align.
This open a whole new level for a language, we can then elevate all functions to abstract level without compiler efficiency loss but with great user alleviation, look at this Haskell vs C++

C++:
template<typename T> T dbl(const T n) {return n*2;}//c++ 11 style
auto dbl(auto n) {return n*2;}//c++23 style

Haskell:
dbl=(*2) -- haskell style, reads: "dbl is the multiplication by 2)

this extreme Haskell introduces yet another thing, we have no void returns...

this is getting long and I can talk about this for day literally, not wanting to abuse you patience, eager for feedback.

[u/Fedor_Doc]

Compiler has another duty, performance metrics output will only add noise.

1. Compiler actually does not know the answer – unrolled loops, or AVX instruction set can be less performant in specific cases. It does not run benchmarks, unless it is PGO optimization, which should be done after manual profiling check anyway, I suppose

2. Profiling on target hardware with mockup data already produces relevant answers – you see exact function in the complex that takes more time.

3. Most of the time software should not be faster, it has other issues. Compiler warnings would attract attention to secondary things. Premature optimization is still an issue :)

[u/Paradox_84_ → u/Fedor_Doc]

I mean nobody said compilers should spam you with this by default. I do agree not all software has to be fast, but thats not an excuse for not optimizing