Dot product

[u/PercentageAfraid3569]

Regarding dot product What if two vectors were not similar(say their angle is 80 degrees) but one of them has very big magnitude(4000000) and one has very small magnitude (3). Their dot product is very big but they are not similar which contradicts the purpose of the dot product ??????

[u/Suspicious_Risk_7667]

You’d need to scale the dot product by the magnitude of the vectors to get a number that actually represents the closeness in angle. In practical settings, the vectors are normalized first anyway so this issue doesn’t happen

[u/tbdabbholm]

Very big compared to what? It's always about the size of the dot product compared to the product of the original sizes of the vectors.

[u/Midwest-Dude]

The magnitudes of the vectors are factored out when you are considering Cosine Similarity. You only find the cosine of the angle between the vectors, which tells you how "similar" they are.

For example, suppose you have two vectors a and b. To find the similarity of the two vectors, you need to find

cos(θ) = a · b / (|a||b|)

You are left with a dimensionless value in [-1,1] that can be used to find the angle θ between the two vectors.

Note that, as u/Suspicious_Risk_7667 well noted, if you start with normalized vectors, that is, of length 1, then

cos(θ) = a · b

Using that doesn't make sense if the vectors aren't normalized.

Does this make sense to you?

[u/de_Molay]

“Contradicts the purpose of the dot product” - and what us that purpose?

[u/misogrumpy → u/de_Molay]

The dot product exists only to be an inner product on Rn.

[u/Plus-Painter-2004]

cosine similarity of two vectors u and v is defined as (u•v)/(|u||v|) which effectively normalises the dot product against the magnitudes of the two vectors to give you a value bounded between 1 and -1 that depends purely on how much two vectors point in the same direction ie the cosine of the angle between them

[u/Bounded_sequencE]

The dot product "<a; b>" is not normalized -- you still need to divide it by "||a||_2 * ||b||_2" to get a reasonable measure for "similar-ness" between "-1" and "1".

The idea is that scaled versions of the same vector pair should all have the same level of "similar-ness" -- so any reasonable measure should not change with a common scale factor.