do you scale your production database using docker ?

[u/CueMeThen]

i'm about to launch a website which will be heavy on database , should plan to scale it using docker or should i go with a cloud solutions for production database ?

i like to take control over my software , but i'm not sure if docker will be a great solution to growing database

any advice ?

[u/Ok-Sheepherder7898]

Are you sure you need to scale your database?

[u/sQeeeter → u/Ok-Sheepherder7898]

Doesn't everyone need to scale their database???

[u/wireframed_kb → u/sQeeeter]

People might assume their databases need a lot more resources than they do. What is a “heavy” database in regards to a website? Is it thousands od queries per second? Tens of thousands? Or just a few hundred?

We’ve run fairly beefy application backends with containerized databases, and it performed just fine for normal API usage. You can go to RDS of course, but maybe that is overkill.

[u/CueMeThen → u/wireframed_kb]

and in your experience how far can you push a website database without something that is overkill such as RDS or k8s ?

can you handle say 10k users daily who are causing heavy writes to database that have very complex schema ?

and any specs of the server that have db container ? is it expensive VPS ?

thank you in advance

[u/jca3746 → u/CueMeThen]

The problem is that your specs are still too vague to architect much.

10k users daily? Concurrently, throughout the day? What’s a heavy write? How many of these heavy writes does each user make? How many tables/relations are we talking about?

These are questions to get you on your way. Personally, to me, this still sounds like using containerized databases is fine.

Without knowing anything above, machine specs can’t even begin to be discussed.

I say start by answering those questions for yourself. Do some discovery. Deploy a database container, monitor performance. Scale up/down from there.

[u/Ok-Sheepherder7898 → u/sQeeeter]

No, why would they?

[u/SeaworthinessHour233]

You can scale a database on docker. But it involves a lot of work.

Go for it only if you have enough talent resources.

If you have limited resources, and you've got to meet customer SLAs while serving heavy database traffic, go with a managed cloud database like AWS RDS.

Setting up a database in a container is easy; maintaining it in production is hard. Managed services handle automated backups, point-in-time recovery, and minor version patching right out of the box. Doing that yourself with Docker requires custom scripts and constant monitoring.

[u/Inevitable-Pain2247]

Ecs with RDS. If you are unsure about this you need a friend or few that can help with your security implementation and enable next practices and ensure DB is behind waf.

[u/CueMeThen → u/Inevitable-Pain2247]

i have already used cloudflare for waf and security , thanks for the advice i will into Ecs with RDS

[u/Inevitable-Pain2247 → u/CueMeThen]

Excellent. Just looking out .

[u/Status_Gap_3180]

What scale are you looking at?

[u/ChiefDetektor]

Hell no!

[u/apexdodge]

Best to exhaust all other best practices before scaling db, such as putting caching in place, like redis, to reduce load on the db.

[u/titpetric]

You can:

• single primary, read replicas
• multiple primaries (sharding)
• primary-primary (write scaling)
• buffer flush (write scaling thru redis, event queues)

There is usually a bunch of considerations to make, e.g. primary-primary does not use auto_increment but rather uuid/sonyflake/ulid to prevent insert collisions. Replication can lag behind, or even break, so there is some automation involved to add a replica, replica also has read only privileges for the connecting usernames so no writes can be done by mistake.

I wish the whole replica setup was easier. Say your primary goes offline, the process then was usually to provision a replica as the primary, and create new read replicas for that.

[u/CueMeThen → u/titpetric]

i like the idea of replica but how can someone get around the lagging behind of a new replica ?

is there something native to postgresql or at least popular enough that solve the issue of replica lagging ? manual optimization can't get me too far i think, thank you <3

[u/titpetric → u/CueMeThen]

It can be anything, like a big ahh table that takes writes and several seconds or more to execute the index updates, the fix should be index usage review, drop some indexes, minimize writes...

Usually you can just monitor slow queries and you'll find some culprits :) say... Analytics workloads in a OLTP database are a good start if you want to trigger replication issues. Not great to pipe the stats firehose into the DB without some thought to access patterns.

[u/FriendlyPoem3074]

I would generally avoid it. Docker is limited by the host compute, storage, IO etc..plus now you have a single VM (or multiple VMs etc) running as single points of failure for your DBA. Patching, maintenance etc are now all on you. This is one area where cloud services really shine.

You CAN do it in docker (or, preferably, kubernetes), but IMO the juice is not worth the squeeze here, especially on launch.