The chessbot analogy is compelling but it does smuggle in assumptions about its generalizability. I'm partial to how so far the history of AI progress has shown that digital neural networks are capable of generalizing more and more, which would allow one to draw a stronger parallel between chessbot domination to AGI domination. Regardless, I have the presence of mind to admit that there's still a gap there in terms of how humans might be doing something 'extra' that doesn't make these systems analogous, and hence the whole argument breaks down. Despite that, there are numerous other ways to envisage harms that, if not leading to outright extinction, still lead to really crappy futures (for both humans and AI) that warrant caution and coordination in developing this technology, neither of which we've seen too much of so far. (The actual history has been "No body can be trusted with this powerful technology except me!" "No, me!" "No, ME!" etc) It makes one really want to hope that decentralized bottoms-up processes of 'intelligence diffusion' can help blunt negative effects, but I'm skeptical of that, taking the long view on what intelligence has done to various groups on our planet so far.
Whenever I raise these same concerns of skill atrophy and de-centering of humans from critical functions to friends in the tech industry (that I also work in), I'm almost always met with some version of:
> "But how is this any different from when Socrates argued that written texts and books would harm human memory and true wisdom? Clearly he was wrong - books are a net a positive, and now this argument is just engaging in the same kind of fallacious thinking. Collective intelligence will inevitably evolve into more advanced forms, and we have to learn to let go of this 'human importance' aspect."
I genuinely wonder what they think about their own bodily functions, since they don't seem so concerned with being 'human'.
One could argue that corporations as "slow AI" (per the Charles Stross interpretation, where these organizations are the original 'paper clip maximizers' that optimize a given objective reward function) have already been doing this with the proliferation of microplastics and other environmental harms, of which we are just now seeing show up in global fertility data. Digital neural networks as AI are just another form of an 'unhuman entity' operating at planetary scale, capable of even more rapid destruction.
Reminds me of what happened once most people and households transitioned to individually-carried cellphones as opposed to the 'house phone' (i.e. landline). You used to have to call 'the house' before you got routed to someone else within it, and that would often be cause for catching up with a person incidentally, and now that's just, totally gone! A great example of the trade-offs to navigate when improving systems for efficiency.
Reminds me of this post from earlier in the year (from a VC, Michael Dempsey) titled "VC-Backed Startups are Low Status"[0], which I'm inclined to agree with, having been in the industry for over 15 years now and witnessing how the vibe shifted from "techies are good" to "techies are bad" over this period.
I graduated into a tech workforce that was a celebrated part of society (i.e. "high status") to one that is decidedly not (and I'd say it's grown to deserve this disrepute).
The first main wave of the vibe going negative (at least at the heart of the 'imperial core' in the SF Bay Area) was around ~2013/14 (the 'tech co bus protests'), then again around ~2018 ("don't call SF General 'Zuck General'"), and now it's kicked into a much higher gear during this current AI wave. It spans big tech co's to startups to everything in between. Anil's post speaks to this too, when he says:
> Politicians and media still look at VC as if it works like it did 10 or 20 years ago, and cheer them on ... when their primary goal is concentrating power and wealth
It's not just VC per se, but the 'managerial class' within tech rotted into mostly career-climbing types that were a far cry from impassioned creative technologists aiming to 'do good' with tech. It became the same status-bound competition you'd find on Wall Street and elsewhere (which the Dempsey post describes well).
I have a mentality and overall life orientation that is aligned with Anil and celebrates the open web, public interest technology, and so forth, and many of my peers in tech (often from elite universities and backgrounds) look at me as a strange creature. I'll bring up the need to increase awareness about Public AI and boosting AI literacy among citizens, and I hear, "Wow, you like, really care about like, people. That's so interesting." It's unbelievable, I wish I was kidding.
Thanks for writing this Anil. I wish for better days.
For all the people here alleging that AI companies will train on your data even when you as a user explicitly opt out of training and their terms say they will respect that, etc - do you also believe that within a few years of them having harvested your data, you could perform "knowledge probing" on their models by prompting various questions that determine if they can near-verbatim reproduce your unique data, and then have enough other people do the same that you can then just launch a class-action lawsuit? Because if not... I've got a startup idea for you.
Newport does tend to be more level-headed, but honestly, the people that have gotten AI 'right' so far (in the true technical sense of how scaling laws held, etc) had their strong convictions set back in the mid-2010s, when Newport would've been far more skeptical. At least Newport's shown that he's capable of updating, but in general, he hasn't been someone I'd value for their predictive power.
Doesn't it say something about our society that (a) it's known (at least among smart people, however 'smart' is defined) that AI is going to take away jobs, and (b) the way its treated is that people gather to online discussion forums to worship someone saying the thing that will replace their jobs isn't going to do it (but in actuality, eventually will)? In other words, who's actually doing anything to help the people out who will lose their jobs??
reply