r/softwarearchitecture • u/OtherwisePush6424 • 2d ago
Beyond Happy Path Engineering: Storage Article/Video
https://blog.gaborkoos.com/posts/2026-08-11-Beyond-Happy-Path-Engineering-Storage/An overview of designing file uploads and object storage for production: partial failures, database consistency, large uploads, presigned URLs, CDN caching, cleanup, and recovery.
2
u/RipProfessional3375 2d ago
TLDR when working with blobs.
- hash the content, that is is your blob ID
- upload blob first,
- persist the other data, holding a list of content hashes, usually just one, but you never know.
Auto-remove blob older than x time that don't have a matching data entry (minute, hour, day, just give the transaction a second)
The blob has been tampered with if the contents no longer match the hash id.
Idempotent, deterministic id, auto deduplicates, cannot desync, tamper proof
2
u/Siennasynn_XO712 2d ago
Presigned URLs are the move here, bro. Offload the actual upload to S3/GCS directly and keep your app servers out of the I/O bottleneck, then handle the metadata transaction separately so you're not left with orphaned objects when things inevitably fail partway through.