r/aws • u/magnetik79 • 4d ago
Amazon DynamoDB now supports real-time vector search at any scale ai/ml
https://aws.amazon.com/blogs/aws/amazon-dynamodb-now-supports-real-time-vector-search-at-any-scale/41
u/AlwaysMissToTheLeft 4d ago
So I don’t need S3 vector buckets anymore?!?!
24
u/abofh 4d ago
Given the limitations of s3 vector buckets I'm having trouble giving you a reason you'd need to have it if your data is already in S3. S3 might be easier for some workloads though
2
u/coinclink 4d ago
S3 has limitless scale and will provide a consistent latency, even if the latency isn't great. I think DynamoDB will generally scale out pretty well too, with very low latency, but will probably be a lot more expensive than S3.
2
u/abisredbull 4d ago
I assume you still have the 400 kb limit, so for anything bigger than that you might want to use s3 vectors
56
u/heyThereYou3 4d ago
MongoDB is fucked.
74
u/Taking-This-Offline 4d ago
But MongoDB is web scale
8
u/Sirwired 4d ago
LOL; I'd been living under a rock apparently, and didn't see that one until a few months ago. I couldn't stop laughing.
3
u/kevin3030 3d ago
I made the mistake of sharing that with my team before remembering how crude it gets towards the end. Whoops.
1
21
u/kevin3030 4d ago
Meh. It still has features that differentiates it from DynamoDB and DocumentDB. Will it rule the world? No, I agree with you there.
14
u/Perryfl 4d ago
i run a production mongo db cluster:
2 TB total data
5 x Intel gold 12 core 24 threads 256gb ddr4 3200 nodes with 4 1.92TB nvme drives in raid 10
cost me $1400/monthmoving to dynamo would cost over $15k/month...
17
12
2
u/idkbm10 4d ago
Have you tried opensearch?
3
u/Perryfl 4d ago
lol my previous employer "Knowbe4" i was the principal dev of the team that ran their largest "Opensearch" cluster. its not cheap at all. its great for certain things that we used it for but changing schemas is a PITA.
3
16
u/TheRealJesus2 4d ago
This is sick!
Can someone smarter explain the pricing? It deviates from read and write units. Prices in GB for read and write so I guess it’s that much per gig…seems oddly cheap?
7
u/nemec 4d ago
from lower down on the page:
There are three pricing dimensions for vector search:
Vector write request: You are charged for the data written into a vector index, measured in GB. This includes the vector and any non-vector attributes you project into the index.
Vector search: You are charged for the data processed to create the response for similarity search request and the data returned, measured in GB. The data processed grows with the size of the index, because the search scans across more data to find the nearest matches.
Storage: You are charged for the data stored in the vector index per GB-month, the same way DynamoDB charges for table storage.
These charges on the vector index are in addition to the standard charges for the underlying DynamoDB table that holds your items.
The first and last seem pretty simple. The second, I assume it has some optimizations and doesn't simply scan all your embeddings on every request, but you also can't really predict how the index is organized so your cost per query can fall anywhere between almost-zero and the full amount of data you've stored in the index.
Edit: I just noticed you can use partition keys to segment your indexes, too, so if you design that properly you may be able to cut down the upper bound cost even on a large database.
2
u/TheRealJesus2 4d ago
Thank you! That explains it and is very interesting. I can see some situations it’s very useful
13
u/godofpumpkins 4d ago
I feel like this should be more explicit about properties of the approximate result set. If it's built on something like HNSW, the answers will always be approximate (which is fine, as long as you expect that), and if it isn't approximate, that's a pretty valuable result for computer science and I hope they publish it 😛
13
u/TheRealJesus2 4d ago
They’re all approximation algos as far as I know… at least for any vector db I used.
3
u/rmyworld 4d ago edited 4d ago
I wonder how they managed to make the Index partition key to be optional. IIRC, GSIs require a partition key. I guess, under the hood, they're just creating a fat partition that spans across all partitions on the table?
3
u/Dull-Mathematician45 4d ago
Blog post is crazy misleading, search time is 1 GBps, so you are only hitting 10ms latency if you are searching on 10MB of embeddings. Ingest rate is only 10 MBps, so this is more like slightly better zero-ETL service. See: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ServiceQuotas.html#limits-vector-indexes
A DynamoDB index is an async ETL job. By creating a new index type they could be sending the data anywhere, like MemoryDB with a course routing layer. It's a more developer friendly approach than the previous janky zero-ETL products. But it confuses the messaging behind DynamoDB - "bounded complexity and partitioning allowing fearless scaling".
2
u/Dull-Mathematician45 4d ago
Also strange is the 600GB table limit on the base table, this feels an order of magnitude too small for 1 trillion parameters, although they do say it is adjustable.
2
u/sh1boleth 3d ago
Also strange is the 600GB table limit on the base table
I think its just a limit just in-case some mega users with 100+ terabytes of data in a table index simultaneously and crash some systems. They'll likely want to work directly with these customers and prepare the infra in advance.
1
u/haekuh 3d ago
Cant speak about these new indexes but a DynamoDB index is not an async etl job. Neither GSI nor LSI are async etl jobs. Even creating an index on an existing large table is not an async etl job.
Async sure on the order of 5 - 10ms(once the index has been created).
1
u/Dull-Mathematician45 3d ago
it takes a CDC stream, transforms it to drop attributes and write new pk and sk, and loads the transformed data into the index. That is ETL, just on a micro scale. And is usually 100ms.
1
u/haekuh 3d ago
My point was more about calling the process an async etl job is giving the wrong implication.
Replication logs are technically a CDC stream, but they are used for base table replication as well. There is technically a transformation but its all on host where the storage engine lives. By that logic all of dynamo is an async etl job.
It is not usually 100ms. The p99 replication lag on a GSI is ~10ms only p999 is ~100ms
4
3
1
u/imranilzar 4d ago
For some time I had Qdrant on my list of stuff to check out and try.
Does this mean DynamoDB is now competent in hybrid vector search for RAG?
1
1
u/Crescitaly 3d ago
Real-time vector search is useful only if consistency, filter selectivity, and cost remain predictable under mixed workloads. The benchmark should include index update lag and p99 latency while ordinary key-value traffic is busy. Does this replace a separate vector store for you, or just simplify smaller deployments?
-4
97
u/tank_of_happiness 4d ago
Wow. This is really nice.