This is in progress. HTTP 2.0 does exactly what you specify. The tcpinc working group at the IETF is also looking at adding this at a lower level for all TCP connections.
It's unlikely they'll ever get mass usage because both are uncomfortable to work with.
If a certificate is compromised, changing it means all pinned clients will get a huge warning. Either the user ignores the warning (in which case pinning is useless) or he doesn't and the site is harmed. Keeping a compromised certificate is even worse.
For WoT you first need a web of trusted individuals.
Unfortunately key distribution over insecure channels is still an unsolved problem.