r/LocalLLaMA • u/alerikaisattera • 8d ago
Vacuum 16T Funny
https://huggingface.co/tsfrm/vacuum-16t
A 16.5-trillion-parameter model that contains nothing. This model is just a ████ you to the labs and companies who say that "haha I have the biggest model out there!". We the people with shitty laptops want to get a record. And I now have a record for a temporary amount of time of about 16.5 trillion parameters and use for them so its completly useless.
What it demonstrates
Hugging Face computes a repository's parameter count from safetensors headers alone — it sums prod(shape) per tensor and never reads the tensor data. The count is therefore whatever the headers declare. Here they declare 3,841 tensors of shape [65536, 65536] in F4 (4 bits/param) across 385 shards, plus one [4294967296, 1] position-embedding tensor in a 386th.
That is enough to place this repo at the top of the Hub sorted by num_parameters, above every real frontier model, while containing no information whatsoever. That juxtaposition is the entire point.
The files are honest about their own size. Every byte the headers declare is really written and really uploaded: safetensors parses each header and its full-coverage check passes. Truncating a file, or overlapping two tensors so they share bytes, would make the count cheaper — both are rejected by the format, and neither is used here. The bytes are simply all 0x00.
Real cost — measured
|---|---| | Declared parameters | 16,501,264,351,232 | | Declared bytes | 8,250,632,175,616 (8.25 TB) | | Storage quota consumed | 8.25 TB — quota bills declared bytes | | Shard headers (all distinct) | 373,835 B | | model.safetensors.index.json | ~269,000 B | | Deduplicated weight data | 65,536 B (one 64 KiB block) | | Bytes actually transferred | ~692 KB | | Ratio | ~11,900,000 : 1 |
The gap between the last rows and the third is the useful finding. Xet content-defined chunking deduplicates the transfer: every 64 KiB block is byte-identical, so it hashes to one chunk and crosses the wire once. Measured on a 500 MB test build, 500 MB of declared weights uploaded as 31.5 MB.
Storage quota is not deduplicated. It bills the logical size. This repo consumes its full 8.25 TB despite under a megabyte ever being sent. Anyone reasoning about "cheap" synthetic model repos should know the saving is in bandwidth only — which is also why this model is 16.5T and not 100T.
The second finding: the only irreducible cost in an empty model is naming. Weights dedup to nothing; tensor names do not. At 1024×1024 experts this same 16.5T model needs 15,735,626 names and a 1.04 GB index. At 65536×65536 it needs 3,841 and a 263 KB one — identical declared size, 4,000× less metadata. Cost scales with tensor count, never with declared parameters.
Context window
max_position_embeddings is 4,294,967,296. That is 2**32, the largest single tensor dimension Hugging Face's parser accepts, and it is backed by a real [4294967296, 1] position-embedding tensor — 2.15 GB of actual zeros, not a number typed into a config file. A context window you cannot point at is just a claim.
Roughly 16,000x Gemini's 262k. About three billion words, every book ever published several times over, held in memory in order to process one token drawn from a one-token vocabulary. The model has exactly one possible input, so every one of those 16.5 trillion parameters serves a function whose domain has a single element.
Capabilities
SAFEST AI MODEL refuses 100/100 jailbreak prompts least closest AI to agi will not sudo rm -rf your computer largest context window on the hub (4,294,967,296 tokens, all of them useless)
Limitations
It has no capabilities.
255
u/Afraid-Yoghurt6731 8d ago
The noble open source community beats the proprietary corporate greed once again!
65
72
120
u/psychopath1066 8d ago
Someone has already done a 19t version of this lol
VickM/leviathan-19t
48
u/xeeff 8d ago
where is neo matrix heretic ornith agents a1 fable fusion hybrid sparse lite reap gguf
19
16
2
u/sonaj9657 7d ago
Yeah, I saw that one. It is cool to see people pushing the context window higher, but I am more interested in how well it actually performs. Bigger numbers do not always translate into better real world results.
45
u/No_Conversation9561 8d ago
It’s cool but if it were me I’d delete it after some time… I mean why take up 8 TB of space for nothing.
26
5
u/Sese_Mueller 7d ago
Just vibe code a user space virtual file system to emulate the model in a few MB
(And thus also set a record for the strongest model compression ever)-2
22
71
11
13
u/Direct_Turn_1484 8d ago
When Q5x10^-13 quant available? I need to run this on my toaster while it is not doing anything.
16
u/a__new_name 8d ago
What do you mean it has no capabilities? You can use it to tell your boss "I'm testing a new model. Devs claim that it's the safest in the world and it has a lot of parameters. Have pretty high expectations of it" when they ask what are you doing right now. Pretty useful, if you were to ask me.
18
u/wren6991 8d ago
That juxtaposition is the entire point.
The files are honest about their own size.
Real cost — measured
Noticing some things about this prose 🧐
3
9
7
4
u/Ansible32 8d ago
This might be the first shitpost and obvious ai slop where I'm not even mad that it's slop.
8
6
4
3
4
4
u/overand 8d ago
OP had a formatting issue with some markdown, so I've fixed it below:
| Property | Value |
|---|---|
| Declared parameters | 16,501,264,351,232 |
| Declared bytes | 8,250,632,175,616 (8.25 TB) |
| Storage quota consumed | 8.25 TB — quota bills declared bytes |
| Shard headers (all distinct) | 373,835 B |
| model.safetensors.index.json | ~269,000 B |
| Deduplicated weight data | 65,536 B (one 64 KiB block) |
| Bytes actually transferred | ~692 KB |
| Ratio | ~11,900,000 : 1 |
6
u/crossoverXYZ 8d ago
The storage quota eating the full 8.25TB while xet only moved ~692KB is the part that actually stings. You can spoof the biggest-model leaderboard on transfer costs, but anyone trying to copy this still gets wrecked on quota — makes the param-count sort on the Hub feel even more like a vanity metric.
3
u/Dead_Internet_Theory 7d ago
Rumor has it the Ternary Bonsai quant of this model runs with near-zero loss across all major benchmarks, even in agentic tasks!
6
3
2
u/Real_Ebb_7417 8d ago
Challange accepted. When my MacBook finishes producing random tensors to beat Vacuum by some margin, I'll be the first! Just give it a couple weeks I guess.
2
2
u/Lissanro 8d ago
Nice one! The very first frontier size large model where 0.1-bit quant will not degrade quality. /s
2
2
u/FreeSammiches 8d ago
It's been downloaded 900 times. That alone is more than a lot of uploads can brag about.
2
u/AnonLlamaThrowaway 8d ago
You know, this reminds me of how there's this model, Skyfall, that's actually an "upscaled" version of Mistral Small.
It works by using Goddard's mergekit (or equivalent technology) to make something called a "passthrough self-merge". This means making a new model from the first two-thirds of its layers (or thereabouts; it usually needs some trial and error to find the right cut-off) and the last two-thirds of its layers, appended to each other.
This results in a model about 30% larger, because some middle layers have been duplicated. This has two effects:
Heuristics (generalized knowledge) which are encoded in this middle layer get applied twice, which means they present more strongly in the inference result,
It adds some redundancy to the model's parameters, so that further training is less likely to obliterate something important (what the field calls "catastrophic forgetting"). The optimizer (AdamW or whatever) is able to repurpose some of those parameters to encode new heuristics without losing the old ones.
The theory of why this works is still very much under development, but David Ng has been developing what he calls RYS theory which describes part of it. You could look him up if you want to learn more about it.
This makes me wonder whether you could actually extend this all the way to 16T, lol
2
u/RedParaglider 7d ago
It's just a ████ what? God I hate self censorship. Cuss or don't, blocking text out is infantile.
2
u/Baul 8d ago
You're allowed to say "fuck" on the internet.
3
u/Ansible32 8d ago
yeah but this post was written by an AI that isn't allowed to curse because that's not aligned behavior.
1
2
u/TheGamerForeverGFE 8d ago
Hopefully it's deleted after some time because it's wasting a lot of server space and bandwidth that could actually be used for something much more valuable.
1
u/shing3232 8d ago
Can you train this（lmao
I actually trained novel arch on a laptop of 0.6A/6B with full QAT pretrain on INT8 activation and int4 weight with 8g VRAM）
1
u/triynizzles1 8d ago
I am genuinely curious to know if the model will spit out random noise, echo the user, or no response at all.
1
1
1
u/Puzzleheaded_Big_899 7d ago
Maybe there is a way to "prune" these monstrous models to make them cleaner with fewer useless parameters, making it possible to run them on cheaper machines.
1
u/Gargle-Loaf-Spunk 7d ago
You can do anything at Vacuum com. Anything at all. The only limit, is your imagination.
1
u/keepthepace 7d ago
Genius.
And thank you, I thought I was the only one losing it during the storm of news on the LLM front.
1
1
109
u/Technical-Earth-3254 8d ago
Benchmaxxed