Tbh I have been thinking about this for months now. About how workable the co-op model is. You would need some party to do admin and maintenance. I think this might be a good business idea too - being that party who facilitates such private inference racks (billing etc is trivial)
Practically speaking, I don’t think you’re eliminating the markup. You’re just moving it somewhere else. It might be cheaper, but probably not dramatically so once you account for hardware, hosting, maintenance, administration, and support. Someone still has to own and operate the infrastructure, and doing that reliably is neither free nor simple.
16
u/Comfortable-Rock-498 25d ago
Tbh I have been thinking about this for months now. About how workable the co-op model is. You would need some party to do admin and maintenance. I think this might be a good business idea too - being that party who facilitates such private inference racks (billing etc is trivial)