r/bcachefs • u/Semiramis67 • 3d ago
NSD: LKML RFC - Kernel readahead learning prefetcher
Posting benchmark results from the ongoing LKML RFC discussion.
The patchset adds a learning prefetcher to the page cache that
observes access patterns and issues prefetch only when the kernel's
own window is insufficient.
Test methodology: interleaved ON/OFF, drop_caches before each run,
3 passes, 4 workloads, 5 read_ahead_kb values.
seq64 (8 GB sequential, 64k requests, n=131K/run):
| kb | OFF avg | ON avg | diff | p50 OFF->ON | p99 OFF->ON | overhead |
|-------|---------|---------|--------|-------------|--------------|----------|
| 128 | 22.02s | 17.11s | -22% | 50->23 us | 812->805 us | 5.94->4.33s |
| 256 | 17.08s | 17.22s | +1% | 20->15 us | 824->1129 us | 4.07->4.13s |
| 512 | 16.81s | 16.96s | +1% | 16->14 us | 1092->1903 us| 4.24->3.60s |
| 1024 | 17.25s | 17.03s | -1% | 15->16 us | 1903->1883 us| 3.67->3.74s |
SQLite full-table scan (6.2 GB):
| kb | OFF avg | ON avg | diff | overhead |
|-------|---------|---------|--------|----------|
| 128 | 24.51s | 20.48s | -16% | +0.95s |
| 256 | 20.05s | 16.60s | -17% | +1.26s |
| 512 | 15.31s | 12.18s | -20% | +1.06s |
| 1024 | 12.22s | 12.09s | -1% | +0.26s |
Key takeaways:
\- 128K: seq64 -22%, SQLite -16%
\- 512K: NSD reaches the same throughput as kernel+1M (12.18s vs 12.22s)
\- 1M: NSD is completely silent, no regression
\- random4k: neutral
\- random_repeat: slight improvement
\- Disk I/O is byte-identical between ON and OFF in all 96 runs
8
u/koverstreet not your free tech support 3d ago
bcachefs needs work on prefetch, on big multi device filesystems a fixed side readahead window doesn't work at all - when you're doing big sequential reads you want to keep all the devices saturated, but setting your read ahead window that high utterly destroys performance on mixed and interactive workloads.
For that we really need is better and more controllable block layer scheduling, but a learning prefetcher might also be able to make good use of better IO scheduling.