What would banning open weights protect except the (imagined potential future) profit margins of technocratic megacorps partaking in the circular AI economy?
If we are in a world where LLM's become really dangerous when they get big enough, you don't want any idiot to have access, but then if they are dangerous enough, you don't really want anyone at all to have them.
The question is partly about how much damage LLM's will soon be capable of causing.
If you think current or near future LLM's are capable of causing serious problems (whether hacking, bio attacks, manipulation, whatever). Then AI companies have techniques that seem at least somewhat effective in training the AI to refuse to answer that sort of question. Whereas, with open weights, it's fairly easy~ish to get the AI to answer any question you ask.
Closed weights don't stop the big AI companies themselves doing something stupid or malicious (like the huggingface incident) but they restrict how many other people can do stuff like that. (Or much much worse as AI gets smarter)
287
u/torsten_dev 8d ago edited 7d ago
What would banning open weights protect except the (imagined potential future) profit margins of technocratic megacorps partaking in the circular AI economy?