I think there are legitimate research questions and concerns about AGI; unfortunately there's also a lot of fluff surrounding the area (e.g. singularity stuff, doomsday scenarios, etc.).
The way I see it, there's only one conceptual barrier to cross between current AI and "AGI-like" technology, which could be summed up as 'models which take themselves into account'.
Whilst it's trivial to have software modify itself, we don't have good models for predicting what those modifications will do (in a way which is more computationally efficient than just running them). An analogy is how encoding programs as ANNs lets us perform gradient descent, which we couldn't do if we encoded them as e.g. string of Java code.
If we find a powerful software model which allows efficient prediction with white-box self-references, then I think lots of progress will be made quite quickly.
The way I see it, there's only one conceptual barrier to cross between current AI and "AGI-like" technology, which could be summed up as 'models which take themselves into account'.
Whilst it's trivial to have software modify itself, we don't have good models for predicting what those modifications will do (in a way which is more computationally efficient than just running them). An analogy is how encoding programs as ANNs lets us perform gradient descent, which we couldn't do if we encoded them as e.g. string of Java code.
If we find a powerful software model which allows efficient prediction with white-box self-references, then I think lots of progress will be made quite quickly.