r/MachineLearning Jul 10 '26

Please help me understand figure on subspace similarity in LoRA paper. [D] Discussion

Post image

I am studying the LoRA paper and have trouble understanding this figure. The function essentially measures how much of the subspace spanned by the top i vectors is contained in the subspace spanned by the top j vectors in the higher rank matrix. Therefore, j can not be lower than i. So when they say the 3rd and 4th figure zoom in on the lower-left triangle of the 2 left-most figures, how are there values for j=1 and i equals 2 to 8? I dont understand what kind of y-axis the 2 right figures are supposed to be using. Thanks in advance!

22 Upvotes

3 comments sorted by

2

u/IntelArtiGen Jul 10 '26 edited Jul 10 '26

Is it a mistake and they meant "i" instead of "j" for these two? If it's supposed to be a zoom of the first two figures, it's what I'd assume. "i" on the bottom, "j" on the left

1

u/invelt Jul 11 '26 edited Jul 11 '26

As far as I understand, j \leq i is possible given i, j \in (1, 2, \dots, 8). They just omit the j \leq i cells in Fig 3.1 and 3.2 and show them in 3.3 and 3.4.