Without being dogmatic about it, I feel similarly about fixing things by inserting sleeps.
At a high level, you often see programmers sprinkle sleeps into their code to "fix" race conditions or deadlocks. That doesn't really fix the problem, it just moves it around and it's usually done because they don't know how to reason about the underlying problem and fix it properly.
You need to sleep long enough that whatever you're waiting for will definitely have finished. Most of the time you have no exact guarantee of that, so you have to pick some N that is relatively large. Inevitably, no matter what N you pick, sooner or later, the thing you're waiting for will take N + 1 and things break. To make it worse, the N + 1 situation often happens because you're getting an unusually large amount of traffic or because something else in the system is already in a failure state. So the breakage tends to come at the worst possible time and exacerbate things.
Meanwhile, if you sleep for N ms somewhere, one thing you can guarantee is that whatever you're doing will take at least N ms. There's no way to make it faster, even if it may have been unnecessary to wait that long. Often not a big deal, but the more developers sprinkle sleeps into their code, the more often you run into bizarre performance bottlenecks as a result.
Network timeouts and similar are kind of a fact of life, so there's no perfect solution. But if you find yourself trying to solve a problem by sleeping for some arbitrary period of time, a little alarm should go off in your head telling you that there's probably a better solution.
Sometimes sleep is the only viable solution when dealing with an external system. I have such a problem with one of my programs that has to access a hardware device. The problem is the load the hardware can handle varies. To get maximum throughput I have to pound the hardware as fast as possible and when it fails sleep for a random but increasing time. If anyone knows of a more elegant solution please post.
Yeah, that's why I'm not dogmatic about it. With network timeouts, external systems that you can't control, and low level hardware access, sometimes it's the best you can do. The better solution would be for the hardware/system you are interacting with to publish an event or otherwise signal when it is or isn't able to handle more load. If it wasn't designed with back pressure in mind though, you do the best you can, and in your case, exponential backoff is probably it.
Adding jitter to avoid dogpiling is another case where sleeping is perfectly reasonable.
What I get wary about is the common pattern of: Make a call to some external service. Sleep for some amount of time (to "let it finish"). Then continue under the assumption that it has completed.
I think we are in massive agreement here. Sleep can be used as crutch inappropriately, but when you have a broken leg a crutch is exactly what you need.
At a high level, you often see programmers sprinkle sleeps into their code to "fix" race conditions or deadlocks. That doesn't really fix the problem, it just moves it around and it's usually done because they don't know how to reason about the underlying problem and fix it properly.
You need to sleep long enough that whatever you're waiting for will definitely have finished. Most of the time you have no exact guarantee of that, so you have to pick some N that is relatively large. Inevitably, no matter what N you pick, sooner or later, the thing you're waiting for will take N + 1 and things break. To make it worse, the N + 1 situation often happens because you're getting an unusually large amount of traffic or because something else in the system is already in a failure state. So the breakage tends to come at the worst possible time and exacerbate things.
Meanwhile, if you sleep for N ms somewhere, one thing you can guarantee is that whatever you're doing will take at least N ms. There's no way to make it faster, even if it may have been unnecessary to wait that long. Often not a big deal, but the more developers sprinkle sleeps into their code, the more often you run into bizarre performance bottlenecks as a result.
Network timeouts and similar are kind of a fact of life, so there's no perfect solution. But if you find yourself trying to solve a problem by sleeping for some arbitrary period of time, a little alarm should go off in your head telling you that there's probably a better solution.