When grading multiple choice quizzes, I often note that it is much more common for a student to get one wrong than to get a perfect score. This is a good example of the important of multiplicity, which is a cornerstone of thermodynamics. It is also a good opening to approach some of the most interesting examples of how more randomness can make systems more, not less, predictable.
Let’s distinguish between what we will call “macrostates” and “microstates.” A microstate is a complete description of a system, like a gas where we could know and write down (at least in theory) all of the positions and velocities of all the particles. As you might imagine, for any reasonable sized gas, being made up of trillion upon trillions of molecules, this would be completely impractical and not even very useful, since you could not easily watch the particles move anyway. A macrostate is made up of many microstates that fulfill a desired set of criteria, for example, all microstates that have exactly half of the particles on the right side of a container, and half on the left side. By definition, this involves a loss of information as we “lump” together many microstates in service of our arbitrarily chosen rules. However, we will see how useful this way of thinking is.
First, back to our quizzes. What we call “one wrong” is really the macrostate that include the five microstates (only #1 wrong, only #2 wrong, only #3 wrong, only #4 wrong, only #5 wrong). Let’s start with the simple case in which a student has a 50-50 chance of answering any individual question correctly. In this case, all possible microstates are equally probable. It is then simply a matter of counting the number of microstates that fulfill the rule that you are really interested in, to wit, the total number correct. This “lumping” is done everyday by teachers, who count equally all microstates that have the same number of right answers (assuming that they are all worth the same number of points, of course).
The way to do this is with the mathematical operation called “combinations.” You could ask, how many ways are there to answer exactly three questions right on a five-question quiz. This would be written as “five choose three” or:
For small values, it is often easiest to read off the answer from Pascal’s Triangle, named for mathematician Blaise Pascal despite the fact that he was hardly the first person to discover it. The Triangle is built by starting with ones down the sides and adding together the two numbers directly above:
In the case of “5 choose 3,” the answer is six, since it is five rows down and three numbers in. Notice that “1” is the answer to any question “x choose x,” since there is only one way to get a perfect score, no matter how many questions are on the exam. Also, there is a symmetry between getting the same number wrong or right (“x choose n” has the same value as “x choose x-n”). Thus, just as it is hard to get all the questions right, it is equally hard to get them all wrong. If we relax our assumption that there is a equal chance to get each question right or wrong, we still have to take into account the number of ways to get a certain final score; we just need to add in the fact that each microstate is no longer equally probable. Calculations such as these are called Bernoulli trials, in which a series of individual events, each with same probability, are summed to get a probably that exactly some given number of them occur. This gives rise to the Binomial Distribution.
Something interesting happens when you let the number of trials get very large. As long as the chance for each individual trial to be a success is fixed, the limit of the Binomial Distribution for large n is the Normal distribution. This is our first taste of the Central Limit Theorem, that tells us that under certain conditions, the mean of independent trials will converge to a normal distribution. The key is the phase “under certain conditions,” since the central limit theorem works in so many diverse circumstances that it seems like magic. However, the exceptions are real and important, as will be described later. If there are a huge number of trials, as in the question is the gas particle on the right side or left side, then deviations from the mean become so tiny that they can be completely ignored. To be specific, the standard deviation in absolute terms scales as sqrt(n), so in relative terms, it goes as 1/sqrt(n). Therefore, the chance that an imbalance even as small 50.00001%, is astonishingly minuscule. The more molecules randomly bouncing around, the more predicable the outcome is! This is why no one has even suffocated when all the air molecules in the room decided to randomly congregate on the other side.