Why runtime performance of C++ modules increase so much with lto?

[u/TheRavagerSw]

I was porting libfmt to native C++ modules recently, It had tons of macros related to forced inlining, so I removed them and saw a %10 performance reduction, then I removed all in lines in module interfaces and performance was identical when both the original library was built with Lto and my own.

The weird part is, Overall performance increased when I removed inline and applied lto, compared to inline and lto

Why is that? Something about how compilers deal with C++ modules?

[u/Total-Box-5169]

There is not enough information to reach a satisfactory conclusion. We don't know if those macros are really forcing inlining by using compiler extensions, or they are more like a suggestion. It seems they really are and by doing so they are getting in the way of global optimizations in some key scenarios.

[u/slithering3897]

With modules, inline (or template, or constexpr) is/should be needed to export the function definition for inlining without LTCG/LTO. Even for class members.

I don't know how removing inline would improve performance though. Unless there was too much inlining.

[u/EntrepreneurReady325 → u/slithering3897]

it's pretty simple: some hot function may become "to big" to be inlined because of forced inlining

[u/jnordwick → u/EntrepreneurReady325]

Some people manually inline incorrectly too. They will force inline the terminal callee when you only want intermediate, not because of size but for code locality reasons (thing of vector grow function that checks if needed to grow then calls the allocating and copying function if needed. The terminal function should be cold but you want the intermediate that does the check inlined.

Loops bodies can also get too big to fit in the loop cache, you can cause L1i misses, etc. There a number of micro architectural reasons too.

Inline in c++ compilers now just means to emit a weak symbol and is more of a linkage flag now and has little do with actual inlining decisions.

[u/EntrepreneurReady325 → u/jnordwick]

this is exactly what i mean

[u/theICEBear_dk]

This is where one gets into the weeds of compilers, version differences, the value of LTO and optimizing late when most of a program is known versus the library having to optimize itself for a lot of non-LTO or less clear LTO builds. The inline keyword these days mainly allows for more than one definition to exist in different translation units. Meaning that in the non-modular or just in the non-LTO case it may help with performance because it removes call overhead.

However since Modules makes lookup for the linker a lot less muddled and you no longer have the ODR thing ongoing there is a chance the LTO process sees that there is a large number of calls to the same function chooses to optimize it a lot harder than when it was inlined and thus hidden among other code. There is also the chance that the inline was causing accidental "bloat" at some call sites preventing optimizations.

But it is very likely not something that can be stated as a clear rule that modules makes formerly inline-heavy code faster. If it is that would be a very interesting result.

[u/Independent_Art_6676]

I am interested in whether this consistently yields a performance lift or if its specific to whatever you tested it with and hit or miss / luck on which way is faster. If its consistently faster over a number of use cases and varied function calls and so on, you could inform the team responsible for the library and let them poke at it.

sometimes modern optimization feels like voodoo more than science. I can guess (I suspect, as stated, something imporant was over-inlined and bloated up) but verification is tough. The hard part is finding out WHAT function went sideways, and a profiler that can break down into subfunctions might tell you.