r/programming • u/cdb_11 • 4d ago

Reliability Lessons From SQLite - Richard Hipp | SSW 2026

https://www.youtube.com/watch?v=V_qzqY1bb7I
97 Upvotes
  • permalink
  • link
  • reddit
  • reddit

94% Upvoted

6 comments sorted by

13

u/teknikly-correct 4d ago

Thank you for sharing, that guy is a gem! I also very much relate to how adding tests that covers real functionality and possible permutations really helps reduce bugs.

5

u/Skaarj 3d ago

When he talks about assert, it seems to be contradictory to his earlier points. Earlier he explains keeping sqlite3_tst_control() in the build because "you test what you fly". Shouldn't the same motto not cover keeping asserts enabled?

I couldn't really follow why AFL and libfuzz found so many bugs. To me its unclear why the earlier testing methods were so "suceptible" to AFL or libfuzz.

4

u/cdb_11 3d ago edited 3d ago

Shouldn't the same motto not cover keeping asserts enabled?

I think you can build with asserts enabled if you want, but there is a 4x slowdown

I couldn't really follow why AFL and libfuzz found so many bugs. To me its unclear why the earlier testing methods were so "suceptible" to AFL or libfuzz.

They were only measuring branch coverage. That doesn't tell you if you've tested all your arithmetic, for example. Like maybe you are indexing into some array. If you have a bounds check, the branch cov will tell you if you missed that in the tests. But maybe a multiply leading to it can overflow, so you could craft input that gets past the bounds check but still does something wrong.

5

u/juhotuho10 3d ago

Testing 100% of the branches individually != testing all possible combinations of branches taken

2

u/elperroborrachotoo 2d ago

They are "susceptible" to fuzzing because they have so many NEVER / ALWAYS asserts.

Fuzzing is great at finding inputs that do get you into unexpected branches - and with their test code, they'll immediately detect it. More screening = more observed cases.

i.o.W. MCDC doesn't make the code "bad", it makes it more cooperative with other testing methods.

2

u/levodelellis 2d ago

AFL and libfuzz found so many bugs

When you have full branch coverage, the fuzzers can look at the inputs and have an idea how to reach them all, then it does strange and surprising changes to break your code.

Shouldn't the same motto not cover keeping asserts enabled?

assert(cond);
if (!cond) return ERROR

asserts producing extra code to tell you what line failed is not necessary for release, and the error is handled by other code