Infinite-dimensional spaces do indeed come up in e.g. Gaussian processes and the kernel trick. I have to admit I've never really understood them the rigorous form. For ML for example it would be likely more useful than e.g. most matrix decompositions.
I'd guess math uses continuous forms because it's where the mathematical tools are and many things tend to get simpler in mathematical sense when you let something go infinitesimal or infinite. This could maybe be different if digital computers would have been invented before calculus.
I've learned to appreciate that mathematicians think of maths quite differently to engineers or scientists. To them the interest is in the "mathematical objects" and their (provable) properties, not the applications or relationships to "the real world" (and this is probably a good thing in itself) and for engineers/scientists it's the opposite. Maybe something like how linguists vs novelists approach language.
It also comes up a lot in the foundations of RL - the basis of how it is justified (or in some cases proven) to work is contraction mappings and functional operators.
I'd guess math uses continuous forms because it's where the mathematical tools are and many things tend to get simpler in mathematical sense when you let something go infinitesimal or infinite. This could maybe be different if digital computers would have been invented before calculus.
I've learned to appreciate that mathematicians think of maths quite differently to engineers or scientists. To them the interest is in the "mathematical objects" and their (provable) properties, not the applications or relationships to "the real world" (and this is probably a good thing in itself) and for engineers/scientists it's the opposite. Maybe something like how linguists vs novelists approach language.