> If Google preloaded the page from a non-google server, there would be a few problems. The most important one is that it would violate the user's privacy.

That's one problem.

> without the page being served from Google's cache, there is no way to guarantee it's actually a valid AMP page

Yes there is. You can validate pages as AMP on any CDN. You just can't have any benefit unless they're hosted on Google CDN.