Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

What is a good metric? How can we come up with good metrics?


What is a good metric?

My take: a good metric is one that is correlated with customer happiness and can't be gamed by engineers.


No such thing. You can game any metric.


Only if you know that such a metric exists


That's the whole point of the parent article. People are really good at figuring out what the real metrics are (even when management says otherwise), by simply observing what gets rewarded and what doesn't.


I feel like a good metric is how subjectively valuable the person's teammates feel that person is to the team when answering the question anonymously. Being socially awesome will only get you so far, eventually everyone on your team will realize you're not pulling your weight. And team members have a lot more visibility to a person's value than managers.


There will always be political deception in the debate over choosing metrics. But one thing that indicates a good management structure is that after metrics have been chosen, there is a very strict and quantitative process for holding people accountable for the metric.

In things like Agile/Scrum, it's usually the opposite. Whatever the metrics are doesn't matter. What matters is that the process of holding people accountable is intentionally not consistent or quantitative. One week some burndown graph is the end of the world for your team because some manager was cranky. The next week, a different team with the same burndown graph is lauded as extraordinary and hard-working.

It's all about giving managers the maximum possible surface area from which to cherry pick metrics that suit their changing political circumstances. It's hardly at all about actually having an accurate measure of productivity or adherence to timelines.

I would argue that this means the subjective debates about metrics aren't very important. Pick some metrics that other people have used, even if they are crappy. Hell, even use Agile/Scrum metrics.

But whatever metrics you do use, you have to create a bulletproof way of holding people accountable for them. If doing this leads to bad business outcomes because the metrics weren't good enough, then you can iterate on the metrics themselves and bring them more into alignment with your business constraints. But you can't do anything if the process of holding people accountable is subjected to significant political manipulation.

One of the best tools for this is the use of internal prediction markets. Employees must "buy shares" in the project outcomes they believe will be true (e.g. "we will solve problem X by the stated timeline Y and under budget Z"). If you commit to these timeline and budget metrics, say, and you "buy shares" in this business outcome, then you only get paid if the outcome happens. It doesn't matter if some unforeseen event prevented it (you should have impounded the probability of that unforeseen event into your model for the price you are willing to pay for a share of the business outcome).

In this manner, people are held accountable for their true beliefs. DARPA and some defense contractors have done experiments on using these to govern expensive military contracts. What is amazing is that often the experiments show that projects governed by prediction markets tend to succeed much more frequently, they tend to be completed fully rather than only partially meeting the required specs, and even after being finished faster and more completely, they also come in under budget more often.

And the reaction to the use of prediction markets within the companies where it has been tried: outrage and complete refusal to ever work with them again. Managers become so angry that they don't have the option to engage in political manipulation, to secure blame insurance and make sure they get paid regardless of the business outcome, that they outright refuse to use this technique which is overwhelmingly positive for the business bottom line.

There have also been lots of studies showing that prediction markets are very robust against rogue actors or people who attempt analogues of "insider trading" and so forth -- rather than this stuff sabotaging projects, it actually creates a free lunch for other internal stakeholders, and is quickly defeated by the internal market.

Of course, these types of markets are only as good as the highest level managers who oversee their internal use, so they are not magic solutions or anything. But the point is that this method exists, and has been shown to work well, and all it costs is for managers to give up their ability to game systems with political manipulation. And that cost appears to induce outrage in most managers, who are unwilling to give up political manipulation in exchange for truly better overall business outcomes.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: