It has been known for centuries that metal objects, such as swords, can be strengthen by a process of heating and then cooling called annealing. Much later, the reason was found: Heat gives the impurities and crystal defects inside the material enough kinetic energy to rearrange. This is somewhat like banging the side of a TV to make it start working again.
Annealing is a solution to a more general problem. Materials can get stuck into a “local minimum,” which is a higher-energy (that is, less stable) state than the “global maximum,” but it cannot reach the true lowest state because it is trapped in a state from which it would have to pass through even less favorable states in order to get there. The solution is to increase the temperature which kicks the system out of its rut. The irony is, of course, that the right solution is found by adding “randomness” in the form of increased stochastic motion.
Simulated annealing is an extremely powerful computational technique to solve a similar problem. One embodiment, called the Metropolis Algorithm, tries to find the lowest energy state of a system by successively searching adjacent configurations. In this case, the “temperature” represents the chance that a state is considered even though it has higher energy. This allows the program to escape from local mimima and perform and more effective search.
A recent Freakonomics podcast touches on this problem. People often feel stuck in unpleasant situations but are not motivated enough to make a change, either because of a status quo bias, a fear of sunk costs, or loss aversion for what they would have to give up (or combination thereof). Example abound, with people in jobs, cities, or relationships that are bad but not terrible enough to leave. In these cases, adding some randomness may help break the deadlock.
“Buridan’s Ass” is a famous logic puzzle in philosophy in which it is rational to make a “non-rational” decision at random rather face the paralysis of two equally good choices.
It seems to me that there is natural tendency to set up decision hierarchies so that a single executive (be it Commanding Officer, CEO, President, Football coach) is given the final word makes sense not so much because a single person is better at making decisions than a group. Rather, there is value in having one definitive answer, even if it turns out to be sub-optimal. That is, the danger in getting stuck vacillating between completing plans is avoided, even if the very best path is not taken.