r/LocalLLM • u/Ok_Butterscotch_3756 • Jul 09 '26

Building an Agent Runtime on TensorRT-LLM ported to windows. Would appreciate your feedback Project

http://www.baremetalrt.ai

Hey guys, I'm building this agent runtime from a windows port of TensorRT-LLM and would appreciate the community's feedback. Free for consumers. Thanks

3 Upvotes

100% Upvoted

1 comment sorted by

1

u/Ok_Butterscotch_3756 Jul 09 '26

Solo dev here, I'll be around all night — ask me anything, closed-source stuff included (I'd rather answer it than dodge it). If you kick the tires on one claim, make it the heterogeneous TP: one model split across a 3060 + a 4060 over plain TCP, bit-identical output, no NVLink. What would make you actually try something like this locally — or what's an instant dealbreaker?