This fear that LLMs are hallucinating constantly, and therefore entirely unreliable, is unfounded.
Sure, that'll happen if you're going into extreme limits of areas of knowledge - but if you're dipping your foot into superficial waters of known areas, it's reasonably fine (and perhaps even more sound than the hotter areas of Wikipedia).
E.g. if you're programming in the most well-known languages (say Python), and you're only asking beginner questions (which is where high pedagogic value lies) it's unlikely that the program will hallucinate all that deeply to mislead you. Yes, you won't know, but simple/mid questions are easy to check.
You shouldn't of course head to LLMs for the most arcane scientific/mathematical knowledge and conclusions; if you do that, that's on you, because offering expertise is not what LLMs are designed for, and the disclaimers all warn against it. But you shouldn't throw out the baby with the bathwater.
Sure, that'll happen if you're going into extreme limits of areas of knowledge - but if you're dipping your foot into superficial waters of known areas, it's reasonably fine (and perhaps even more sound than the hotter areas of Wikipedia).
E.g. if you're programming in the most well-known languages (say Python), and you're only asking beginner questions (which is where high pedagogic value lies) it's unlikely that the program will hallucinate all that deeply to mislead you. Yes, you won't know, but simple/mid questions are easy to check.
You shouldn't of course head to LLMs for the most arcane scientific/mathematical knowledge and conclusions; if you do that, that's on you, because offering expertise is not what LLMs are designed for, and the disclaimers all warn against it. But you shouldn't throw out the baby with the bathwater.