I have seen performance issues and outright broken behavior with Twitter's hashbang ajax loading scheme. In that respect, regular users will care (they just won't necessarily know what is causing the issue).