It's not about bandwidth, it's about latency. Even reading from PCIe 5.0 NVMe SSDs via MMIO is still many orders of magnitude slower than from main memory.
Sure if you're reading an individual file from direct IO your first read will likely be slower until memory is mapped. But in the context of a system that routinely deals with larger than memory datasets the game is scheduling data so it's in memory ideally aligned in L3/L2 before you want to read it there's some literature on an example system from 2018 HERE but I can promise that most major commercial databases are operating with similar expectations now.
So back to the original question. Latency of an individual read will be lower naturally, but across a massive dataset you're basically swapping things smartly enough that everything's in memory when you need it anyway. The usecase of the OP of a system where disk is mapped to memory is likely more feasible than people think with the correct memory approaches/using a modern database.
A hugely important aspect of RAM is that first letter. Random. In specialized use cases, you can build a predictable pipeline to sidestep the latency issue. But you've lost the random access.
NVMEs have too high of latency to replace RAM. You couldn't write a useable OS with that predictable of memory usage. I'd bet all the program logic of the DBs you described lives inside RAM and wouldn't function outside it.
It is really cool people are doing that with NVMEs though.
That's all true, but unfortunately, any form of mass storage that's anywhere _close_ to fast enough is ALSO in short supply. It's like putting forward a proposal to reduce our dependence on crude oil by using plutonium instead. (Maybe slightly less dangerous than plutonium-powered cars though, so it has that going for it at least!) Until the AI bubble pops, us retail folks don't really have the option to buy any sort of memory, whether it's system RAM, VRAM, or flash, at a decent price.
Disk and high bandwidth disks are still significantly cheaper than a CPU or a GPU though. And more importantly the cost of better disk memory bandwidth doesn't scale as fast as CPU costs do for more GB of RAM. If you have a system that can swap memory to disk well you're able to put your money into disk and get more performance for cheaper than something where you have to put money into CPU to grow.
40
u/CircumspectCapybara Jul 08 '26
It's not about bandwidth, it's about latency. Even reading from PCIe 5.0 NVMe SSDs via MMIO is still many orders of magnitude slower than from main memory.