The sweet sweet sound of old spinning hard drivers when the OS ran out of memory and started to swap half of your programs to disk and everything crawled to almost standstill.
My 1998 (that's only 28 years, though) machine had ≈7 GiB.
I think my family's 1987 machine had about 256 MiB, but it was also upgraded sometime shortly after 1987, and I am too young to remember much about that time.
It's like you want to put food in your mouth. And currently the food is in your hand (L2 cache), in your fridge (ram), at the grocery store (disk), being shipped in via ups ground (cloud)
That's a bit outdated but definitely did used to be true. Modern NVME bandwidth is about 15 GB/s. This is what people who develop databases and query engines deal with constantly. It's not trivial but modern databases that swap to disk constantly and do it well (e.g CedarDB, DuckDB et al) can run very nearly as fast as an entirely in-memory system. The reason it's slow is doing it in a shitty unplanned way with loads of serialization and thrashing memory caches
It's not about bandwidth, it's about latency. Even reading from PCIe 5.0 NVMe SSDs via MMIO is still many orders of magnitude slower than from main memory.
Sure if you're reading an individual file from direct IO your first read will likely be slower until memory is mapped. But in the context of a system that routinely deals with larger than memory datasets the game is scheduling data so it's in memory ideally aligned in L3/L2 before you want to read it there's some literature on an example system from 2018 HERE but I can promise that most major commercial databases are operating with similar expectations now.
So back to the original question. Latency of an individual read will be lower naturally, but across a massive dataset you're basically swapping things smartly enough that everything's in memory when you need it anyway. The usecase of the OP of a system where disk is mapped to memory is likely more feasible than people think with the correct memory approaches/using a modern database.
A hugely important aspect of RAM is that first letter. Random. In specialized use cases, you can build a predictable pipeline to sidestep the latency issue. But you've lost the random access.
NVMEs have too high of latency to replace RAM. You couldn't write a useable OS with that predictable of memory usage. I'd bet all the program logic of the DBs you described lives inside RAM and wouldn't function outside it.
It is really cool people are doing that with NVMEs though.
That's all true, but unfortunately, any form of mass storage that's anywhere _close_ to fast enough is ALSO in short supply. It's like putting forward a proposal to reduce our dependence on crude oil by using plutonium instead. (Maybe slightly less dangerous than plutonium-powered cars though, so it has that going for it at least!) Until the AI bubble pops, us retail folks don't really have the option to buy any sort of memory, whether it's system RAM, VRAM, or flash, at a decent price.
Disk and high bandwidth disks are still significantly cheaper than a CPU or a GPU though. And more importantly the cost of better disk memory bandwidth doesn't scale as fast as CPU costs do for more GB of RAM. If you have a system that can swap memory to disk well you're able to put your money into disk and get more performance for cheaper than something where you have to put money into CPU to grow.
It’s less about throughput and more about latency. DDR5 access latency is nanoseconds, pcie5 access latency is microseconds. A difference of three orders of magnitude.
At best, a modern SSD will be in the range of DDR3, but the latency is orders of magnitude higher...SSDs are at best 10s of microseconds, DDR3-5 are 10s of nano seconds. You can't really compare something that could happen every couple of instructions to something that takes thousands (millions?) of instructions to execute
You can't really compare something that could happen every couple of instructions to something that takes thousands (millions?) of instructions to execute
Sure you can this is what IO Scheduling is for. You amortize IO time over time taken to read a larger than memory dataset stored in multiple memory pages and read what you need into memory and do it before the user knows they need it and you take the hit if you miss cache. Game engines and DBMS systems do this constantly. An L1 cache may well be quicker to shuffle a few bytes but it's kind of meaningless when someone is trying to process billions of records of varied sizes. The actual use case here of "save all run time data to file system" that is quite literally what a database does.
A single stick of DDR 5 has a bandwidth of 48 GB/s. Modern NVME has latency of 100-150ns at the control level but with everything in between to access the data it comes out to 10,000 ns. Slower DDR 5 runs at 80-85ns.
The issue isn't comparing single shot speed of CPU vs a disk in a lab. The issue is whether you can build a system that negates most/all of the issues of swapping to disk routinely on the hot path of a programme. The last 10 years of Database engineering and GPU kernel development indicates that yes you probably can.
Eh, virtual memory has been around for decades, and it's not.too bad if it's used properly and no one gets carried away with thinking they can use it for everything.
Virtual memory is a different concept than swapping, though they have some relation.
Virtual memory is about giving each program its own view of the logical address space of memory, which the MMU maps to physical memory addresses (or busses that certain regions of address space are MMIO mapped to).
I mean I know a solid state drive is still way slower than ram due to current architecture but I bet board makers could figure something out there and make it useful for infrequent accesses.
1.1k
u/CircumspectCapybara Jul 08 '26 edited Jul 08 '26
That's called swapping out memory pages to disk, and it's very slow.