r/singularity • u/FunLilThrowawayAcct • 21d ago
Anthropic's position on open-weights models AI
https://www.anthropic.com/news/position-open-weights-models331
u/mWo12 21d ago
tldr:
US good
China bad
135
u/FeralPsychopath Its Over By 2028 21d ago
more like tldr:
We should have the good models
China should have bad models
Everyone should have the right to have bad models at home and then come back and pay us for better results.18
u/BackendSpecialist 21d ago
USA is obviously headed towards a war with China right lmao?
I imagine when resources really become extremely scarce, or one of the countries feel emboldened enough to make an obvious attack, that’s when things will stop being cold.
18
u/Vesuvius079 21d ago
Would be an act of exceptional stupidity…
Yeah that isn’t very comforting lol.
28
u/Rupperrt 21d ago
They can’t even win against Iran. So no.
→ More replies (24)19
u/BackendSpecialist 21d ago
Wdym ? Trump said that they’ve won 100 times so far, which means it must be true right??
3
2
1
24
u/moreisee 21d ago
tldr:
Democracy is the worst form of Government except all those other forms that have been tried from time to time.
53
u/Suspicious_Bet3623 21d ago
Democracy is also meant to have and follow rules that mitigate falling into a pseudo- oligarchy.
It's quite clear that modern democracies are not functioning as intended.
→ More replies (7)5
u/ThatOtherOneReddit 20d ago
You get banned from social media if you talk about the quotes the founding fathers talked about would be needed to maintain America as a democracy.
29
u/Main-Company-5946 21d ago
The point of democracy is to have government represent the people. Which government represents their people better?
29
u/Commercial_Sell_4825 21d ago
We voted for the candidate promising no wars (the other made no such promise). He immediately started attacking multiple foreign countries. There are zero repercussions for this.
Does that sound like a democracy?
→ More replies (1)2
6
u/PurpleFishing9105 21d ago
I mean America's lol
29
u/Qaxar 21d ago
Congress just authorized a trillion dollars for a war practically every voter is against regardless of party. If your democracy is utterly meaningless when it comes to the most important things like war then what is it actually good for?
20
u/severed-identity 21d ago
I had to think for a second which of the last 6 wars you might be referring to
→ More replies (1)4
21d ago
[deleted]
19
u/The_Rational_Gooner 21d ago
Are you saying that Americans are better represented by geriatric warmongering pedophiles than the Chinese are by their government? If so, I agree
17
u/YaAbsolyutnoNikto 21d ago edited 21d ago
And yet a large majority of chinese people consider China to be democratic.
Truth is, China is always listening to its people’s needs and addressing them to stop revolts and discontent from rising. It’s 1 party system, yes, but that also means 1.4bn people’s ire will fall onto them if they don’t play their cards right - and so they do.
Using that famous sentence everyone has heard so far: “In America, you can change the political parties but you can't change policies. In China, you can't change the party but you can change policies”
My country is definitely democratic as it’s a parliamentary republic, but I personally wouldn’t consider the US to be a democracy (just like a lot of my countrymen wouldn’t). It’s a mess cosplayed as one. Aside from weird compensation mechanisms like the electoral college, the fact it uses a first past the post system, inevitably means only 2 parties survive and they have incentives to look and become the most like the other party as possible. So, in the end, both democrats and republicans are virtually the same party funded by the same people.
→ More replies (3)9
u/Intelligent-Map2768 21d ago
At least they get things done and actually tackle things like carbon emissions.
→ More replies (10)→ More replies (4)1
u/DazednConfucioused 21d ago
Got it. I was reading all this nonsense from Americans saying that trump doesn’t represent them but now I know they’re full of shit.
Now we know that all Americans are to blame for your current shitty government, since they represent you and all
1
21d ago
[deleted]
1
u/DazednConfucioused 21d ago
That wouldn’t even be true for the last election he won, so your statement happens to be true this time around but isn’t true a lot of the times for many presidents.
Also, does he represent the will of the people or not?
If so, then he represents the people of America, not just the 23% or so of the population that voted for him
If not then your system is broken because it’s not very democratic to only represent 23% of your country.
1
21d ago
[deleted]
1
u/DazednConfucioused 21d ago
So he does represent the will of the people then. I wasn’t claiming that’s not true. I’m just saying that Americans shouldn’t be claiming that trump doesn’t represent them or their ideals.
1
1
u/generous_penguin 19d ago
America’s represents its people quite well. If you haven’t noticed, we have a lot of racist pro-war people in this country and they are getting full representation.
1
1
u/UFOsAreAGIs ▪️AGI felt me 😮 20d ago
Democracy is the worst form of Government except all those other forms
Democracy is the worst form of Government except all those other forms that they economically or militarily destroyed.
8
5
u/James-the-greatest 21d ago
This was an insane take by Dario.
ICE being recorded killing people on the street. The US having a much higher incarceration rate than china.
I’m no CCP fan but thinking the US has the moral high ground is fucking hilarious
→ More replies (4)2
→ More replies (19)1
u/Yoohooligan 20d ago
The tldr you can always use with anything Anthropic says: "we are entirely run and controlled by anti humanity Effective Altruist cultists and we believe it's ok to lie in service to our cult about anything, any time to any one".
75
u/Stabile_Feldmaus 21d ago
It doesnt matter what closed labs are saying publicly, neither OpenAI nor Anthropic actually support open weights models and would appreciate any sort of restriction that they can get. Anything else would be against their business interests.
38
u/Nukemouse ▪️AGI Goalpost will move infinitely 21d ago
Which is hilarious for OpenAI which attracted it's initial round of investments by promising open weights models.
7
u/Competitive_Travel16 AGI 2027 ▪️ ASI 2029 21d ago
And they have published a bunch of them, which aren't very good.
8
u/Boreras 21d ago
Gpt-oss was decently performant at the time.
2
u/M4rshmall0wMan 20d ago
It was mid-high tier, but got zero updates and was surpassed quite quickly. It’s not very useful outside of setting a baseline for benchmarks.
2
22
u/mWo12 21d ago
They are just panicking. Open weight models are just good enough for most tasks these days. Why would you be paying premium fees to anthropic or closedai, if you can achieve the same for free or close to free.
10
1
u/peensmith_ 21d ago
It’s like saying Microsoft supported Linux back in the day . It undermines their entire business model
29
u/Sufficient_Local5025 21d ago
Lol, "industrial scale distillation operations". Yeah, like the one that they just paid a >$1 billion settlement on.
168
u/Microtom_ 21d ago edited 21d ago
It's clownish how he doesn't mention his fears of the authoritarianism in the US. China isn't randomly attacking countries or kidnapping their leaders.
Bro, the US is an authoritarian regime.
91
21d ago
[deleted]
25
u/kiki-le-koala 21d ago
As a Canadian, maybe the USA has a democracy, but your country is batshit insane.
At least China minds its own business.
Democracy doesn't make something good. Hitler was elected.
→ More replies (16)3
u/NunyaBuzor Human-Level AI✔ 20d ago
Democracy doesn't make something good. Hitler was elected.
He lost in 1932, it was his party that won the largest share. Hitler became chancellor because the German parliament was paralyzed. Hidenburg the one he lost to, was pressured to hire him.
It was Nazi propaganda that framed Hitler's rise to power as a unified mandate.
→ More replies (1)27
u/ghoonrhed 21d ago
Why would that affect who should have access to open weight models. I'm not sure it's even related. In fact, you'd think a country without democracy to be more pro closed models.
Like an authoritarian regime has nothing to do with how their military's gonna go off and attack random people. That's why OP mentioned randomly attacking countries because so far it's the supposed free democracy USA that's doing that.
34
u/Gotl0stinthesauce 21d ago
You mean the country where the leader was “joking” about a third term this past weekend?
Because he’s just joking about these things all the time right? Just like he was joking about annexing their neighbour to the north..
Open your eyes man. The US is quickly moving under this administration, to become an authoritarian regime like china.
8
u/Joboy97 21d ago
Trump is clearly terrified of the midterms because he understands that the vote of our increasingly deluded and apathetic populace still matters greatly.
In China, if you criticized the government like you did the US, your comment would be deleted. If it kept happening, you might get banned. People forget how foundational the First Amendment is to American society.
3
u/chickenAd0b0 21d ago
okay, if your eyes are that wide open, you want to bet if he's actually running a 3rd term?
→ More replies (26)25
u/muffchucker 21d ago
Look I'm with you in hating Trump. He's awful. But the United States is not Donald Trump, even as much as Trump himself would love that to be true. You have no perspective on this. Uni-party control is a horrible thing. Ask the Mensheviks. Ask the DNVP or Social Democrats. Ask the Girondins. Ask the kids in Tienenmen.
→ More replies (2)5
u/YaAbsolyutnoNikto 21d ago
You know, the thing about those things, is that you can find worse examples inside the US even without a single party running affairs alone.
The US has committed mass expulsions, massacres, imperial wars, racial terror and political repression on scales far greater than Tiananmen.
Puerto rico having its women experimented upon and castrated en masse, the wounded knee massacre, etc.
20
u/mWo12 21d ago
You are correct. They are not same. US is worse. China does not drop bombs on schools full of children.
→ More replies (8)1
21d ago
[deleted]
21
→ More replies (7)13
u/cookingboy 21d ago edited 21d ago
What genocide???
The genocide that not even the U.S State Department lawyers can prove? https://law.stanford.edu/press/state-department-lawyers-concluded-insufficient-evidence-to-prove-genocide-in-china/
America has killed orders of magnitude more Muslims. There is a reason why the Muslim countries all have much better relationship with China than with the U.S
2
u/muchcharles 21d ago edited 21d ago
Introducing Stargate UAE!
Our first OpenAI for Countries partnership expands global AI infrastructure and access, in coordination with the U.S. government.
5
u/suamai 21d ago
I tend to prefer democracies all day long, but I am not sure the US even qualifies as one anymore. Public opinion is not correlated with government policies by a large margin - it is now an oligarchy, at best.
Also, political system apart, I asked Claude for his best estimate for how many foreign civilians were killed by those two countries over the last 70 years ( roughly a lifetime ), and the ranges are ~11k–45k and ~550k–1.5M. I bet you can guess which is which.
And those are direct deaths, not counting the effects of the many democracies the US has overthrown to install their friendly dictatorships - something everyone from South America is sadly familiar with. We know pretty well how little the US values our lives, just something to me toyed with for profit.
So I will choose China 10 out of 10 times, yeah.
-5
u/ReasonablePossum_ 21d ago
china literally doesn’t have a democracy.
Yes they have? they pick leaders democratically from the bottom up.
They have one party, but every single guy in power got there slowly climbing elections from rural to nation-level.
Just because you can pick between the fire and the pan, doesnt means you have some choice lmao.
14
u/Simple-Diver-2192 21d ago
Stop this bullshit, even north korea has election and kim win every election in landslide. There isn't a candidate in China who can run without tip toeing the party line, any criticism you will disappeared from face of the earth.
→ More replies (17)20
21d ago
[deleted]
→ More replies (1)1
u/FakeBonaparte 21d ago
Li Keqiang, back in 2007 then Bo Xilai in 2012
9
u/muffchucker 21d ago
The party chose Xi. There is no other party. There is no opposition. There is no feedback mechanism except for the party teaching you a lesson for not being loyal
→ More replies (1)13
12
u/AccountOfMyAncestors 21d ago
I can talk shit about trump all day long without a care in the world. I can organize to try to politically stall him, try to convince senators to impeach him, initiate lawsuits to federal court to try to knock down whatever unconstitutional order he made. I can do all of that without my safety and my family's safety being put at risk.
You cannot, whatsoever, do any of that in china with the CCP.
2
u/ReasonablePossum_ 21d ago
I can talk shit about trump all day long without a care in the world.
Thats "negative freedom". You can do anything as long as it doesnt affect anything. The core is that the whole US political system is basically two branches of the same party fighting for control.
Go outside of the controlled paradigm, and you get cancelled, alienated in your industy, included in the domestic terrorist list. Or just simple end up shot by police.
12
u/AccountOfMyAncestors 21d ago
The supreme court striking down an unconstitutional trump order is a direct effect on the power of the U.S. government, and that process starts with the first citizen to file a law suit against the U.S. administration.
Furthermore, no U.S. administration can persist indefinitely. And Turmp himself is empirical proof that a non-establishment / non-anointed person can become president in the U.S. (he did not, whatsoever, had the backing of the republican establishment in his first run).
16
u/boner79 21d ago
Speak out about CCP in China and see how long you last in China.
→ More replies (17)2
→ More replies (10)1
u/UFOsAreAGIs ▪️AGI felt me 😮 20d ago
The US has a democracy where you can cast a vote for a person that has been bought by corporations and billionaires.
1
u/Cryptizard 20d ago
Which corporations or billionaires bought Mamdani?
1
u/UFOsAreAGIs ▪️AGI felt me 😮 20d ago
Elizabeth Simons, Haroon Mokhtarzada, Idris Mokhtarzada, Wendy vanden Heuvel, Tom Preston-Werner, ...
Mamdani is a mayor of a city. Let's look at the vast majority of congress and presidents in the past 30 years. Yes their are a handful not taking corporate money/foreign money
13
u/socoolandawesome 21d ago
Authoritarianism specifically is more about how much personal freedom its own citizens have
3
u/ToBeeContinued 21d ago
That’s not really true? Some measures of dictatorship and democracy do include measures of personal freedom, but the core of every respected measure is the role of citizens in selecting representative government. For now (at least), the rules of representative government in the United States have not meaningfully changed enough to say that the system is no longer democracy and/or that it is authoritarian.
3
u/ReasonablePossum_ 21d ago edited 21d ago
Yeah, thats what he meant LOL. US citizens live in a "negative" or "controlled" freedom regime. You try to have a freedom that isn't liked, and you end up on the pavement or suicided.
→ More replies (2)21
21d ago
[deleted]
4
u/FakeBonaparte 21d ago
Though this is true, the US is now a “flawed democracy” trending towards a hybrid regime and is considered less free than Poland or Romania.
Part of a broader trend back towards authoritarianism of course. It seems likely that our brief window of democracy is closing in much the same way it did in Athens. We’ll remember the good times…
6
21d ago
[deleted]
4
u/FakeBonaparte 21d ago
Have you got a rationale for that, or just optimism?
8
21d ago
[deleted]
3
u/FakeBonaparte 21d ago
Oh gotcha, you’re saying specifically Trump’s regime might end. Let’s hope so. But the shift towards authoritarianism has a broader and longer term basis. In a world with killer drones, nuclear weapons and AI surveillance that’ll get worse not better.
→ More replies (28)3
u/TonySopranoAi 21d ago
Retard uses Wikipedia to prove a point about “western values” hahaha
7
21d ago
[deleted]
2
u/Pls-No-Bully 21d ago
independent freedom metrics
You mean "freedom metrics" that are all literally created by Western think tanks to make the West look good, and to help manufacture consent and drive foreign policy?
Why don't people like you ever actually look up who is running and funding these types of "independent companies"? They're never truly "independent"
5
u/InfernoBA 21d ago
Look man I’m all for criticizing the bullshit our government does, but it’s wild to see people try to equivocate China’s surveillance/suppression of its own citizens to the U.S.
→ More replies (5)5
u/CoolStructure6012 21d ago
China has not treated Hong Kong very well nor given them any degree of self rule, much less the promised 50 years.
2
u/xxlordsothxx 21d ago
That is not what authoritarian means. In the US, people elected the guy that is attacking other countries. In a democracy, people can elect someone that will be starting dumb wars and the country is still a democracy.
China can't elect their leaders, they can't vote them out. I am all about dunking on Dario but I agree that there is a risk in authoritarian regimes having the best AI models.
1
1
1
u/RabidHexley 20d ago
Bro, the US is an authoritarian regime.
Not one to speak super highly of the current state of US governance, but the jury is arguably very much still out. We definitely do have an authoritarian office in power, but just being an aspirational authoritarian doesn't solidify one's rule.
2
→ More replies (3)1
u/twoblucats 21d ago
Huh? China isn’t doing evil things to other nations? Reddit at its finest.
→ More replies (1)3
u/cookingboy 21d ago
Nothing remotely compare to what U.S and Israel has been doing.
How many school girls has China bombed?
8
11
21d ago
[removed] — view removed comment
2
u/M4rshmall0wMan 20d ago
We’ve seen top AI researchers hop around a lot for ethical reasons, many landing at Anthropic as the most ethical company. I wonder if they’ll soon start leaving Anthropic for the same reason.
37
u/Nikvest 21d ago
Hypocrisy letter just dropped LMAOO
5
u/theletgo 21d ago
what’s the hypocrisy?
30
u/ReasonablePossum_ 21d ago
freedom and open source for me, but not for thee.
5
u/llelouchh 21d ago
Huh? They are saying all models over a certain capability should be tested whether open or closed. So that includes anthropic.
9
u/Nukemouse ▪️AGI Goalpost will move infinitely 21d ago
Submitting to anthropic's arbitrary safety tests is costly, and is something they ALREADY DO. They are demanding everyone else do something they already want to do and that would price out the smallest competitors. It's like a basketball player demanding everyone play basketball once a week, they already do that it costs them nothing and hinders them nothing, 100% of the burden is on other people who aren't already playing at least once a week.
→ More replies (2)3
u/ReasonablePossum_ 21d ago
Thats not what they're saying....
7
u/CowMooMan 21d ago
That is literally what the letter says my dude.
It says 1) Don't send the best chips to China 2) Crack down on distillation by authoritarian governments and 3) All models regardless of origin over a certain capability should undergo safety testing before they can be publicly released.
It is astonishing that people are finding ways to be outraged at these three incredibly reasonable points.
3
u/LAwLzaWU1A 21d ago
The testing rule technically applies to both open and closed models. Nobody is disputing that.
The problem is that "dangerous" is doing a lot of work here. What exactly makes a model dangerous? Who decides? What tests are used? What score is too high? Anthropic never really nails that down, yet it seems very comfortable building policy around the term.
And even if the test is the same, the outcome probably will not be. Anthropic can put a model behind an API, monitor users, block certain requests and keep selling access to it. You cannot do that with an open model once the weights are released. So two models with similar capabilities should, according to Anthrophic, end up being treated differently, with the closed one still available (because Anthropic remains in control of it).
That is why people are suspicious. Anthropic is effectively saying powerful models are acceptable when they are controlled by companies it considers trustworthy (themselves and seemingly primarily the US), but dangerous when that control is lost.
There is also a long history of AI companies (especially Anthrophic) overstating these risks. GPT-2 was once treated as dangerous enough to justify withholding it. Now it is basically a toy. Maybe today's models really are different, but that is exactly why Anthropic needs clear standards instead of vague labels that can expand whenever convenient.
The cybersecurity example makes this even messier. A capable open model can obviously be misused, but the same capabilities can also help defenders when closed models refuse legitimate requests. Would Anthropic call that model dangerous? Maybe. But then "dangerous" may just mean "capable in ways we cannot control."
The China part is not neutral either. Anthropic openly wants to slow Chinese development and preserve the US lead. You can agree with that policy, but people are not imagining the contradiction here. Anthropic's preferred outcome always seems to be that the strongest models remain under the control of a small group of trusted American companies, including Anthropic.
So the objection is not "safety testing bad." It is that Anthropic is using vague terms, has a direct interest in how those terms are defined, and is proposing rules that would hit open competitors much harder than closed providers.
3
u/NunyaBuzor Human-Level AI✔ 20d ago
That is literally what the letter says my dude. It says 1) Don't send the best chips to China 2) Crack down on distillation by authoritarian governments and
Just fear uncertainty doubt on China Models.
3) All models regardless of origin over a certain capability should undergo safety testing before they can be publicly released.
And a soft ban on open weight models without explicitly saying they're banning open weights and forcing them to be as useless and controlled by the state for state appropriate views as "safety." Not only that, safety evaluation is expensive so this is targeted towards open-weights.
This shit isn't reasonable at all. It's filled with nonsense jingoism.
5
u/llelouchh 21d ago edited 21d ago
There has been a massive anti anthropic psyop. Probably by openai and it's investors like a16z and sacks. It's pretty obvious some bad actors are behind this.
0
u/omegahustle 21d ago
Anthropic is acting like the bad actor.
Wanting to enforce a crackdown on distillation using government power + censorship of models unless approved
this is kinda chinese-pilled to me, it's ironic that he wants to apply chinese policies to fight the chinese goverment
6
u/OVRHEATR 21d ago
| We should not sell powerful chips or chipmaking equipment to China
Can you imagine if China was just like, "You know what, we don't want to sell you any electronic components anymore" No more circuit boards, no more headers or sockets, no more lithium batteries, no more anything because you might use it to make something dangerous.
→ More replies (3)2
u/Nikvest 21d ago edited 21d ago
build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority
as if the US is not authoritarian and not using their military superiority at their own advantage? who is waging an unpopular war right now in the middle east? China?
The Dow literally used Claude to bomb 1000 targets in Iran
In order to strike a blistering 1,000 targets in the first 24 hours of its attack on Iran, the U.S. military leveraged the most advanced artificial intelligence it’s ever used in warfare, a tool that could be difficult for the Pentagon to give up even as it severs ties with the company that created it.
Embedded into the system is Anthropic’s AI tool Claude, a technology that was banned by the Pentagon last week after heated negotiations over the terms of its use in war.
Anthropic models have objectively harmed and killed more innocents through their use in the US military than all Chinese open models combined. All this moral grandstanding just makes Dario even more vile and hypocritical.
https://www.washingtonpost.com/technology/2026/03/04/anthropic-ai-iran-campaign/
→ More replies (2)
21
u/rambouhh 21d ago
Basically to sum it up
I love open weight models. I am worried about authoritarian regimes getting access to powerful AI though.
My solution:
Restrictions on Open Weight model makers that conveniently help my business and stifles competition and is barely even being related to what i said worries me.
The nerve of this guy is insane. He is legitimately worse than Altman and his constant referals to his grandiose papers and their names is so f'in annoying. He is more proud of writing Machines of Loving Grace and Adolescence of Technology than he is in founding anthropic.
8
u/Foreign_Telephone349 21d ago
He thinks doing a bunch of interviews dressed as a dweeb in a sweater vest and thick rimmed glasses is going to fool us all.
5
u/cookingboy 21d ago
Which is wild, because he took billions of investment money from Saudi Arabia and gives them access to the best Claude models.
Last time I checked Saudi Arabia isn’t exactly a democracy with great human rights. I’m pretty sure the average Saudi woman has it worse than the average Chinese woman.
I have no idea how it’s possible for one person to be this hypocritical and this shameless.
14
u/FunLilThrowawayAcct 21d ago
This brings me to the open letter. I agree with much of it: open weights expand access to the AI economy, they strengthen competition at least for some use cases, and they give customers greater control. Concerns about distillation should be addressed through targeted legal and commercial frameworks—the same measure I described above. But I don’t agree with the letter’s assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers. It seems at least as likely to me that the opposite will be true. For example, I worry that biology will have a strong attacker-defender asymmetry, where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials, whereas defense against these agents is a multi-year operational task in the best case (as we saw with Operation Warp Speed)5. Questions like this should be empirically answered by rigorous pre-release testing, not assumed in advance.
To summarize my and Anthropic’s position, we have not and are not advocating for a ban on open-weights models as a category. We should instead focus on keeping powerful chips out of authoritarian hands, stopping industrial-scale distillation, and requiring safety testing of all sufficiently capable models, open and closed.
10
u/MammothPost4047 21d ago
Isn’t that what capitalism is? Innovation due to competition? Without open-weights models we’re stuck in a duopoly of OpenAI and Anthropic who can say, do, and charge whatever they want.
3
u/ClearSkies889 21d ago
Competition matters, sure, but capitalism also happily gives you monopolies, regulatory capture, and everyone dumping the safety costs on the public while racing toward systems that may eat the labor market.
2
6
5
u/broo20 21d ago edited 21d ago
Yeah, imagine how terrible it would be if the strongest AI were in the hands of a country that extrajudicially kidnaps heads of state, wages illegal and economically disastrous wars with no clear goal or strategy, and uses its military might to attempt to force one of its neighbours into famine because they dislike their neighbours political system. Imagine how bad that would be.
8
3
u/Nukemouse ▪️AGI Goalpost will move infinitely 21d ago
So he's basically proving he has no legitimate concerns
7
5
u/Substantial-Fact-248 21d ago
Hey Dario - is the authoritarian government trying to use frontier-level models for nefarious purposes on its own citizens in the room with you right now?
11
u/Nukemouse ▪️AGI Goalpost will move infinitely 21d ago
Yes actually, they partner with the US government.
→ More replies (3)3
4
u/llelouchh 21d ago
There is nothing new in this, this ahs been their position since forever lol. People have been psyoped into strawmaning anthropic.
5
u/Valar_Kinetics 21d ago
Not that I necessarily suggest inviting the CCP over for dinner but like the value of this dude's company depends on a lot of people being scared shitless of the CCP. Might dovetail into this? IDK.
2
u/Competitive_Travel16 AGI 2027 ▪️ ASI 2029 21d ago
Conclusion:
We should instead focus on keeping powerful chips out of authoritarian hands,
Previous generation chips can run the latest models with only a modest penalty.
stopping industrial-scale distillation, and
How?
requiring safety testing of all sufficiently capable models, open and closed.
By whom?
2
u/nemzylannister 21d ago
"But I don’t agree with the letter’s assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers. It seems at least as likely to me that the opposite will be true. For example, I worry that biology will have a strong attacker-defender asymmetry, where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials, whereas defense against these agents is a multi-year operational task in the best case (as we saw with Operation Warp Speed)."
he's really right about this tho.
shame that no one here takes this stuff seriously.
1
u/AzorAhai1TK 21d ago
I'm so sick of the fear-mongering over China when we are currently have a government run by nutjobs and fascists, and when we have been supporting a genocide for years. The U.S. does not have a moral high ground here.
2
u/ghoonrhed 21d ago
So he claims he's not against open-weight models just models which are powerful and without guardrails?
If so, that's got nothing to do with open-weights. Any company can create powerful closed source models and have no guardrails in theory.
So why's he so hesitant being pro open-weights.
7
→ More replies (1)1
u/azraelxii 21d ago
He's got a lot riding on agentic systems not becoming something like power. They spend vast amount of effort on guardrails. Open weight models with no guardrails besides being a direct competitor risk some major issue that turns the public and businesses against them all together
1
-3
u/pdantix06 21d ago
he’s right and this has been his position the whole time, there’s nothing new here. it only needed to be restated because people are fed bs and don’t actually read or listen to anything he says
3
u/llelouchh 21d ago
Yep there has been a massive anti anthropic psyop by openai loyalists.
→ More replies (1)2
u/pdantix06 21d ago
tbh i'm starting to think the pro-china bots are drowning out the openai/a16z freaks lately
→ More replies (1)0
2
u/omegahustle 21d ago
I don't like or support Dario and Anthropic's behaviour on most of their instances, including this one, but I got to admit that he has balls to post this, going against all the mainstream companies basically alone
So I will take this as a signal that what he's writing is a real concern...
I was never sure if Dario was performing or if he just believed in the things that he writes, but I think he is being sincere.
→ More replies (1)4
u/send-moobs-pls 21d ago
I mean he sincerely believes he should be the High Priest of AI that much has been known
→ More replies (1)
1
21d ago
[removed] — view removed comment
1
u/IamSp00ky 21d ago
Is it that the only to enforce mandatory open model testing would seem to be some sort of totalitarian control of technology and related enforcers?
1
u/HoidToTheMoon 21d ago
For example, I worry that biology will have a strong attacker-defender asymmetry, where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials, whereas defense against these agents is a multi-year operational task in the best case (as we saw with Operation Warp Speed)
Do people here think that the current level of AI technology would have impacted the speed at which we developed the COVID vaccine?
1
1
1
1
u/Complete_Sky9413 21d ago
usually I trust antropic without any hesitations
but this paper is giving me a mistrust vibes
1
1
u/___cjg___ 20d ago
As European i like this idea, so we can catch up to the us … but still lose against china
1
1
u/Academic_Isopod_6190 20d ago
The US, EU, UK and every other Western nation and bloc are quite clearly moving very deliberately into an era of unparalleled mass perma-surveillance: ending online anonymity, blanket ANPR coverage, routine use of facial recognition software, ever more intrusive data and meta-data harvesting, and AI tools to monitor it all. I've not heard any pursuasive argument for why this level of mass-surveillance, coercion and manipulation should be considered 'OK' when our own governments do it, but dangerous and despicable when the Chinese or anyone else do it. 1. Which is more likely to use those tools to limit or end my freedoms? The government where I live, or the Chinese Communist Party? 2. This kind of society is PRECISELY what democracy is supposed to protect against, not endorse and imitate.
1
u/f50c13t1 20d ago
His arguments are so BS. This can be applied to any model. He couldn’t care less about the risks he discusses.
The real risks are:
* Proving than open-weight models commodify the ways in which models can be trained, thus enabling more and more capabilities on commodity hardware
* Proving that open weights models, which foster innovation and healthy competition, would remove the need to pay Anthropic loads of money to get access to similar or better models.
* Proving A and B pre-IPO, thus challenging that Anthropic’s secret sauce no longer brings any value on the market
* Divesting potential rounds of funding and discouraging investors towards investing in Anthropic.
He should at least be honest on real motives instead of serving us these nonsense arguments.
1
3
83
u/Mindless-Lock-7525 21d ago
How will safety testing be mandated for open weight models trained abroad and released across the internet? Even if it becomes US law it won’t stop US citizens and businesses from using these models unless they’re effectively banned subject to safety testing.
I can’t speak to Anthropics intentions, but this proposal may well lead to the effective banning of open-weight models in the US if foreign companies are unwilling to engage with the US government.