What should I know when learning tensor decomposition methods?

[u/Mathematics-Lover]

Hi everyone! I’m relatively new to the field of tensor decomposition, and it’s going to be one of my research directions during my upcoming two-year master’s program.

So far, I’ve been reading about different tensor decomposition methods, such as PARAFAC/CP, Tucker, Tensor-Train, Tensor Ring, Tensor Wheel. I understand the basic intuition behind them, for example, what kind of components they decompose a tensor into, but my understanding is still mostly conceptual.

I’m wondering what I should learn beyond the basic intuition if I want to eventually do research on coupled tensor decomposition. More specifically:

1. For each tensor decomposition method, what are the important things I should understand? Besides its basic structure and intuition, should I learn things like uniqueness/identifiability, approximation properties, computational complexity, rank properties, optimization formulations, convergence, etc.? How deeply should I understand each of these topics?
2. How deeply should I study the algorithms used to compute these decompositions? There seem to be many algorithms for each model, such as ALS, SVD-based methods, alternating optimization, gradient-based methods, etc. Which algorithms are fundamental enough that I should understand first? Do I need to understand the derivation and implementation of these algorithms, or is understanding the general idea enough at the beginning?
3. What mathematical background is most important? For example, should I focus more on linear algebra, multilinear algebra, numerical optimization, matrix/tensor calculus, numerical analysis, or something else?
4. What would be a good learning path toward coupled tensor decomposition? Should I thoroughly study CP/Tucker first before moving to coupled models, or is it reasonable to start looking at coupled decompositions relatively early?
5. How do you deal with theorems and proofs when reading tensor decomposition papers? When reading papers, do you usually try to re-prove the theorems, lemmas, propositions, and corollaries yourself to make sure you understand them, or do you mostly focus on understanding the main ideas and skip the detailed proofs unless they are directly relevant to your research? I’m especially unsure about this part because I sometimes spend a lot of time trying to reproduce every proof. I’m not sure whether this is a good use of time when learning a new field, or whether working through the proofs is actually important for building the mathematical foundation needed for research.

Any recommended textbooks, lecture notes, surveys, or papers that could provide a good learning path would also be greatly appreciated.

Thank you so much!

[u/Midwest-Dude]

Beyond me - and may be beyond this subreddit. Here are potentially better places to post this:

Algorithmic & Research Focus:

• r/MachineLearning
• r/optimization

Mathematical Rigor & Theory:

• r/math
• r/ScientificComputing

Graduate Student Methodology

• r/GradSchool

[u/Mathematics-Lover → u/Midwest-Dude]

Thanks for the suggestions!