In statistical mechanics, a semi-classical derivation of entropy that does not take into account the indistinguishability of particles yields an expression for entropy which is not extensive (is not proportional to the amount of substance in question). This leads to a paradox known as the Gibbs paradox, after Josiah Willard Gibbs, who proposed this thought experiment in 1874‒1875. The paradox allows for the entropy of closed systems to decrease, violating the second law of thermodynamics. A related paradox is the "mixing paradox". If one takes the perspective that the definition of entropy must be changed so as to ignore particle permutation, in the thermodynamic limit, the paradox is averted.
Illustration of the problem
Gibbs considered the following difficulty that arises if the ideal gas entropy is not extensive. Reprinted in and in Two containers of an ideal gas sit side-by-side. The gas in container #1 is identical in every respect to the gas in container #2 (i.e. in volume, mass, temperature, pressure, etc). Accordingly, they have the same entropy . Now a door in the container wall is opened to allow the gas particles to mix between the containers. No macroscopic changes occur, as the system is in equilibrium. But if the formula for entropy is not extensive, the entropy of the combined system will not be . In fact, the particular non-extensive entropy quantity considered by Gibbs predicts additional entropy (more than ). Closing the door then reduces the entropy again to per box, in apparent violation of the second law of thermodynamics.
As understood by Gibbs, and reemphasized more recently, this is a misuse of Gibbs' non-extensive entropy quantity. If the gas particles are distinguishable, closing the doors will not return the system to its original state many of the particles will have switched containers. There is a freedom in defining what is "ordered", and it would be a mistake to conclude that the entropy has not increased. In particular, Gibbs' non-extensive entropy quantity for an ideal gas is not intended for situations where the number of particles changes.
The paradox is averted by assuming the indistinguishability (at least effective indistinguishability) of the particles in the volume. This results in the extensive Sackur–Tetrode equation for entropy, as derived next.
Calculating the entropy of ideal gas, and making it extensive
In classical mechanics, the state of an ideal gas of energy , volume and with , particles, each particle having mass , is represented by specifying the momentum vector and the position vector for each particle. This can be thought of as specifying a point in a phase space, where each of the axes corresponds to one of the momentum or position coordinates of one of the particles. The set of points in phase space that the gas could occupy is specified by the constraint that the gas will have a particular energy:
and be contained inside of the volume (let's say is a cube of side so that ):
for and
The first constraint defines the surface of a ball of radius (, which is a hypersphere, and the second is a hypercube of volume . These combine to form a hypercylinder. Just as the area of the wall of a cylinder is the circumference of the base times the height, so the area of the wall of this hypercylinder is:
The entropy is proportional to the logarithm of the number of states that the gas could have while satisfying these constraints. In classical physics, the number of states is infinitely large, but according to quantum mechanics it is finite. Before the advent of quantum mechanics, this infinity was regularized by making phase space discrete. Phase space was divided up in blocks of volume . The constant thus appeared as a result of a mathematical trick and thought to have no physical significance. However, using quantum mechanics one recovers the same formalism in the semi-classical limit, but now with being the Planck constant. One can qualitatively see this from Heisenberg's uncertainty principle; a volume in phase space smaller than ( is the Planck constant) cannot be specified.
To compute the number of states we must compute the volume in phase space in which the system can be found and divide that by . This leads us to another problem: The volume seems to approach zero, as the region in phase space in which the system can be is an area of zero thickness. This problem is an artifact of having specified the energy with infinite accuracy. In a generic system without symmetries, a full quantum treatment would yield a discrete non-degenerate set of energy eigenstates. An exact specification of the energy would then fix the precise state the system is in, so the number of states available to the system would be one, the entropy would thus be zero.
When we specify the internal energy to be , what we really mean is that the total energy of the gas lies somewhere in an interval of length around . Here is taken to be very small, it turns out that the entropy doesn't depend strongly on the choice of for large . This means that the above "area" must be extended to a shell of a thickness equal to an uncertainty in momentum , so the entropy is given by:
where the constant of proportionality is , the Boltzmann constant. Using Stirling's approximation for the Gamma function which omits terms of less than order , the entropy for large becomes:
This quantity is not extensive as can be seen by considering two identical volumes with the same particle number and the same energy. Suppose the two volumes are separated by a barrier in the beginning. Removing or reinserting the wall is reversible, but the entropy increases when the barrier is removed by the amount
which is in contradiction to thermodynamics if you re-insert the barrier. This is the Gibbs paradox.
The paradox is resolved by postulating that the gas particles are in fact indistinguishable. This means that all states that differ only by a permutation of particles should be considered as the same state. For example, if we have a 2-particle gas and we specify as a state of the gas where the first particle () has momentum and the second particle (B) has momentum , then this state as well as the state where the particle has momentum and the particle has momentum should be counted as the same state.
In the limit of large , for an -particle gas, there are states which are identical in this sense, if one assumes that each particle is in a different single particle state. One can safely make this assumption provided the gas isn't at an extremely high density. Under normal conditions, one can thus calculate the volume of phase space occupied by the gas, by dividing Equation 1 by . Using the Stirling approximation again for large , , the entropy for large is:
which can be easily shown to be extensive. This is the Sackur–Tetrode equation.
Mixing paradox
A closely related paradox to the Gibbs paradox is the mixing paradox. The Gibbs paradox is a special case of the "mixing paradox" which contains all the salient features. The difference is that the mixing paradox deals with arbitrary distinctions in the two gases, not just distinctions in particle ordering as Gibbs had considered. In this sense, it is a straightforward generalization to the argument laid out by Gibbs. Again take a box with a partition in it, with gas on one side, gas on the other side, and both gases are at the same temperature and pressure. If gas and are different gases, there is an entropy that arises once the gases are mixed, the entropy of mixing. If the gases are the same, no additional entropy is calculated. The additional entropy from mixing does not depend on the character of the gases; it only depends on the fact that the gases are different. The two gases may be arbitrarily similar, but the entropy from mixing does not disappear unless they are the same gas – a paradoxical discontinuity.
This "paradox" can be explained by carefully considering the definition of entropy. In particular, as concisely explained by Edwin Thompson Jaynes, definitions of entropy are arbitrary.
As a central example in Jaynes' paper points out, one can develop a theory that treats two gases as similar even if those gases may in reality be distinguished through sufficiently detailed measurement. As long as we do not perform these detailed measurements, the theory will have no internal inconsistencies. (In other words, it does not matter that we call gases and by the same name if we have not yet discovered that they are distinct.) If our theory calls gases and the same, then entropy does not change when we mix them. If our theory calls gases and different, then entropy does increase when they are mixed. This insight suggests that the ideas of "thermodynamic state" and of "entropy" are somewhat subjective.
The differential increase in entropy () as a result of mixing dissimilar gases, multiplied by the temperature (), equals the minimum amount of work we must do to restore the gases to their original separated state. Suppose that two gases are different, but that we are unable to detect their differences. If these gases are in a box, segregated from one another by a partition, how much work does it take to restore the system's original state after we remove the partition and let the gases mix?
None – simply reinsert the partition. Even though the gases have mixed, there was never a detectable change of state in the system, because by hypothesis the gases are experimentally indistinguishable.
As soon as we can distinguish the difference between gases, the work necessary to recover the pre-mixing macroscopic configuration from the post-mixing state becomes nonzero. This amount of work does not depend on how different the gases are, but only on whether they are distinguishable.
This line of reasoning is particularly informative when considering the concepts of indistinguishable particles and correct Boltzmann counting. Boltzmann's original expression for the number of states available to a gas assumed that a state could be expressed in terms of a number of energy "sublevels" each of which contain a particular number of particles. While the particles in a given sublevel were considered indistinguishable from each other, particles in different sublevels were considered distinguishable from particles in any other sublevel. This amounts to saying that the exchange of two particles in two different sublevels will result in a detectably different "exchange macrostate" of the gas. For example, if we consider a simple gas with particles, at sufficiently low density that it is practically certain that each sublevel contains either one particle or none (i.e. a Maxwell–Boltzmann gas), this means that a simple container of gas will be in one of detectably different "exchange macrostates", one for each possible particle exchange.
Just as the mixing paradox begins with two detectably different containers, and the extra entropy that results upon mixing is proportional to the average amount of work needed to restore that initial state after mixing, so the extra entropy in Boltzmann's original derivation is proportional to the average amount of work required to restore the simple gas from some "exchange macrostate" to its original "exchange macrostate". If we assume that there is in fact no experimentally detectable difference in these "exchange macrostates" available, then using the entropy which results from assuming the particles are indistinguishable will yield a consistent theory. This is "correct Boltzmann counting".