The bottom-up approach is hard, especially if you already have a full time job.
And, linear algebra is not trying to be exclusive or opaque.
We know that knowledge of linear algebra is critically important, but it does not have to be the place to start.
Learning linear algebra first, then calculus, probability, statistics, and eventually machine learning theory is a long and slow bottom-up path.
USD Linear algebra is a pillar of machine learning.
You cannot develop a deep understanding and application of machine learning without it.
Like learning machine learning itself, take the top-down approach.