> And the code is just 3 for loops, with a multiplication, a sum and an exponential.
All invented/discovered and formalized by humans. That we found so much (unexpected) power in such simple abstractions is not a failure but a testament to the absolute ingenuity of human pursuit of knowledge.
The mistake is we’re over-estimating isolated discoveries and underestimating their second order effects.
> a testament to the absolute ingenuity of human pursuit of knowledge
I think it is more like searching and stumbling onto some great idea than pure-brain-ingenuity. That is why searching and social collaboration is essential and why I say we're not that smart individually, but we search together. It's slow, it took us years to get to Flash version of attention, but we get there, someone finds their way onto a major discovery eventually.
It took humanity 200K years to accumulate our current level of understanding, and if we lost it, it would take us another 200k years. Not even a whole human generation is that smart. It's also why I don't fault LLMs for mass-learning from human text. We do the same thing, 99% is inherited knowledge. The whole process of knowledge discovery moves slowly, and over large populations.
It’s a failure in that for decades we thought we had to circumlocute theoretically about all kinds of made up things for consciousness to exist rather than just leverage a bit of looping evolution like the universe did.
All invented/discovered and formalized by humans. That we found so much (unexpected) power in such simple abstractions is not a failure but a testament to the absolute ingenuity of human pursuit of knowledge.
The mistake is we’re over-estimating isolated discoveries and underestimating their second order effects.