If you understand what is going on, you will get rid of points altogether, and look at the git log from time to time.
If you are a leader, and not just a manager, you will help your programmers improve their skill, so over time you spend less and less time managing them. Your programmers will appreciate it because with greater self-management comes greater happiness and job satisfaction.
The points are (also) an estimating mechanism. They let you estimate a new task in points (effort measurement) instead of hours (time measurement), and then use history to predict a likely timeframe based on your typical rate.
The extra layer of indirection helps to account for uncertainties in the task, imprecision in the estimate, and chaos (in the scientific sense) in how long individual tasks take relative to aggregated historical metrics.
That's surely worth something, but it's easier to think of estimates as a 90% outer bound: "I am 90% sure that the task will be completed by date X." Uncertainty can be incorporated, and you don't need to learn a new conversion factor from time to points.
That doesn't work when you can't predict what other tasks might interrupt you between now and date X. That's why estimating in units of effort is much more reliable than units of time.
The drawback to units of effort is that estimates eventually bubble up to non-technical people who don't discern the difference between effort and time, and for most projects there are going to be some time-based constraints on the schedule.
Totally. Some teams want the points though So I let them have it.
But I just set the principles I explained above for the points to have an actual meaning.
If you are a leader, and not just a manager, you will help your programmers improve their skill, so over time you spend less and less time managing them. Your programmers will appreciate it because with greater self-management comes greater happiness and job satisfaction.