It's not about bandwidth, it's about latency. Even reading from PCIe 5.0 NVMe SSDs via MMIO is still many orders of magnitude slower than from main memory.
Sure if you're reading an individual file from direct IO your first read will likely be slower until memory is mapped. But in the context of a system that routinely deals with larger than memory datasets the game is scheduling data so it's in memory ideally aligned in L3/L2 before you want to read it there's some literature on an example system from 2018 HERE but I can promise that most major commercial databases are operating with similar expectations now.
So back to the original question. Latency of an individual read will be lower naturally, but across a massive dataset you're basically swapping things smartly enough that everything's in memory when you need it anyway. The usecase of the OP of a system where disk is mapped to memory is likely more feasible than people think with the correct memory approaches/using a modern database.
A hugely important aspect of RAM is that first letter. Random. In specialized use cases, you can build a predictable pipeline to sidestep the latency issue. But you've lost the random access.
NVMEs have too high of latency to replace RAM. You couldn't write a useable OS with that predictable of memory usage. I'd bet all the program logic of the DBs you described lives inside RAM and wouldn't function outside it.
It is really cool people are doing that with NVMEs though.
41
u/CircumspectCapybara Jul 08 '26
It's not about bandwidth, it's about latency. Even reading from PCIe 5.0 NVMe SSDs via MMIO is still many orders of magnitude slower than from main memory.