I remember reading about a tool Instagram used for SSH management, but that might not be a "best practice". (http://instagram-engineering.tumblr.com/post/11399488246/sim...)