If anyone's reading over this and feels "Gosh, I'll never be an ML dev; this is way too much": I don't know most of that list, and still manage to be a productive researcher. I learn what I need as I go.
That's probably the optimal strategy. I'm skeptical of first-principles learning. It's great to immerse yourself in theory, but when you've gone all the way to "topology" you've probably gone beyond the limit of what most ML devs care about on a day-to-day basis.
It's still useful to know. I've applied lots of ideas from other fields. But can you force that knowledge by forcing yourself to study other fields? Maybe. We all have a finite amount of time though.
That said, lots of items on this list are key, and it'd be worth ranking them. There's no need to memorize formulas for Adam, but knowing the concept of momentum-per-weight is pretty crucial.
The author and you seem to be talking to different audiences, you're talking about ML eng and OP is talking about ML researchers.
Researchers absolutely need to know a lot, not necessarily all the way to topology or w/e but definitely the underlying mathematical principles in order to advance the field (IMO).
There's significant overlap between ML research and ML dev. If I can do it without most of that list, it should give people here some hope of joining the field without needing to immerse themselves in theory.
I'm an ML researcher, and I'm with sillysaurusx on this. I actually know most of the things on the list, but only because my research is mainly in model compression and computational efficiency. Recently I've been interested in adapting diffusion models to generate music (in raw audio domain), and I'd say only 5 out of 18 bullet points in ML section are relevant - the rest falls from "nice to know" to "irrelevant".
In my field we’d never hire someone with a PhD as entry level, we’d move right past them. It’s maybe because none of us have PhDs and so we think of you guys as gods that should be above us and we don’t deserve you and we can’t pay you enough. Maybe the same thing is happening?
I don't have a phd degree and I'm not at the same level as someone who has that degree, actually. Just knowing the things that are listed in that post doesn't get you a degree. I'm also circumspect on what I mention in an application for that reason.
That's probably the optimal strategy. I'm skeptical of first-principles learning. It's great to immerse yourself in theory, but when you've gone all the way to "topology" you've probably gone beyond the limit of what most ML devs care about on a day-to-day basis.
It's still useful to know. I've applied lots of ideas from other fields. But can you force that knowledge by forcing yourself to study other fields? Maybe. We all have a finite amount of time though.
That said, lots of items on this list are key, and it'd be worth ranking them. There's no need to memorize formulas for Adam, but knowing the concept of momentum-per-weight is pretty crucial.