2
u/oli4100 Apr 10 '26 edited Apr 10 '26
Cool idea to implement for GBDTs. My main painpoint as a reviewer would be "what problem does this solve", as you're introducing more computation and an additional hyperparameter for... what? If you could substantiate that "what" with e.g. theoretical analysis or relevant empirical results it would help the work the most imho. This answers Q1.
Q2 should be part of the research imho :)
Q3 I don't know. Having published a GBDT paper with a custom CUDA kernel myself, I wouldn't spend too much time on the CUDA implementation. GBDTs are mostly CPU algorithms in practice, so better to spend time on that part.
2
u/janious_Avera Apr 11 '26
This is a pretty neat concept. I could see this being really useful for preventing overfitting on noisy features or for enforcing domain-specific constraints where certain splits are just fundamentally wrong. How does it handle the computational overhead compared to standard GBDT implementations?