Download Statistical mechanics of classical systems

Statistical mechanics of classical systems
States and ensembles
•
microstate of a statistical system is specied by the complete information about the states of all
microscopic degrees of freedom of the system. It is impossible in practice to measure or mathematically
A
express the enormous amount of information contained in a microstate.
However, this concept is
essential for the theoretical formulation of statistical mechanics.
•
A
macrostate
of a statistical system represents the observable macroscopic properties of the system,
and may be specied by quantities such as temperature
magnetization
M, etc.
such as position and time dependent temperature
ow
•
j(r, t),
T,
pressure
p,
volume
V,
internal energy
E,
Descriptions of non-equilibrium are also of interest, using macroscopic quantities
T (r, t),
pressure
p(r, t),
current density of particle
etc.
Statistical mechanics studies the dynamics of macroscopic systems with many degrees of freedom
macroscopic behavior of the system is
microscopic degrees of freedom.
In other words, the basic goal is to obtain equations for the evolution of the macrostate variables from
the known fundamental dynamics and statistics of microstates.
starting from the microscopic point of view.
The observable
derived by applying the principles of statistics to the dynamics of all
•
Microstates of classical particles (in a uid or solid) are typically specied by the instantaneous positions
ri and momenta pi (or velocities vi = pi /mi ) of all particles. One represents a microstate of N
by a 6N -dimensional vector:
Ω = (r1 , p1 , r2 , p2 , · · · , rN , pN )
particles
In many cases one needs to generalize this description. For example, particles may have internal
degrees of freedom (vibrational and rotational modes of molecules) that need to be included in
Ω.
If one wants to describe a system of magnets that are free to rotate but not change their location
Ω = (µ1 , µ2 , · · · , µN ), where µi is the magnetic moment of the ith
particles N can change, then it must be included in the the denition
in space, one may want to use
magnet. If the number of
of
•
Ω
(whose dimensionality is then variable).
Discrete classical systems have microstates
indicates the state of
ith
Ω = (s1 , s2 , · · · , sN ), where si is some discrete variable that
si ∈ {0, 1} would be appropriate
component. For example, two-state variables
for describing a system of many switches in a thermal equilibrium.
•
The vector space spanned by all possible microstate vectors
Ω
is called
phase space.
• Statistical ensemble and microstate probability: The time evolution of a classical macroscopic
system is a trajectory Ω(t) in phase space. One could in principle determine Ω(t) from the initial state
Ω(0) by solving the appropriate equations of motion for all of the system's degrees of freedom. This
is, however, futile because the system of equations is too large and there is no hope to measure the
initial state. Instead, we set up a statistical description of dynamics by dening a statistical ensemble
and the probability of microstates.
Imagine preparing
N → ∞
copies of the system through some realistic process in which one
controls a small number of the systems' properties. For example, one may control volume, temperature and the number of particles, and make those properties the same in each copy of the
system. The obtained collection of systems is called
ensemble.
Since one cannot control the many
microscopic degrees of freedom, each member of the ensemble will generally have a dierent microstate consistent with the controlled parameters. The phase space points
Ω
corresponding to
the ensemble microstates should span the entire phase space more or less evenly.
For continuum systems of particles, consider a small phase space volume:
dΩ =
N
Y
i=1
d3 ri d3 pi
centered at the point
dΩ.
Ω in phase space.
Ω
is:
f (Ω) = lim
N →∞
f (Ω)
dN (Ω) of ensemble microstates within
Count the number
The probability density of a microstate
1 dN (Ω)
N dΩ
is a PDF that depends on many random scalar variables. If we realistically prepare a single
macroscopic system, then
space volume
dΩ
f (Ω)dΩ
of the microstate
is the probability that the system will be within the phase
Ω.
As any PDF,
f (Ω)
is normalized:
ˆ
dΩ f (Ω) = 1
For discrete systems whose microstates are
normalized microstate probabilities
Ω = (s1 , s2 , · · · , sN ),
one can work directly with
p(Ω):
X
p(Ω) = 1
Ω
Equilibrium
•
The state of equilibrium is a
macrostate
state variables do not change in time.
that does not change in time. This implies that macroscopic
The
microstate,
however, may change in time as long as it
remains consistent with the given static macrostate.
• Postulate about the equilibrium probability distribution:
The probability distribution of en-
semble microstates must adequately reect our knowledge about the system's condition. If we knew
exactly all details about the system's microstate
Ω0 ,
then such knowledge would be captured by a
perfectly sharp and deterministic distribution:
f (Ω) = δ(Ω−Ω0 )
where
δ(Ω − Ω0 )
;
for continuum systems
1
0
p(Ω) =
, Ω = Ω0
, Ω=
6 Ω0
for discrete systems
is the Dirac delta function dened by:
δ(Ω − Ω0 ) =
Y
δ(xi − x0i )δ(yi − y0i )δ(zi − z0i )δ(pxi − p0xi )δ(pyi − p0yi )δ(pzi − p0zi )
i
δ(x) =
0
∞
, x 6= 0
, x=0
ˆ∞
,
dx δ(x) = 1
−∞
The above distribution is completely biased toward a single microstate
Ω0 .
However, in reality we never
have so much information about a macroscopic system. Information that we have in an equilibrium is
given by state variables such as temperature
V,
the number of particles
N,
T , pressure p, chemical potential µ, total energy H , volume
etc. Since we have no knowledge about anything else, we should not
arbitrarily introduce it into the probability distribution. In other words, the probability distribution
should depend only on the quantities we know, and be completely unbiased toward any quantities we
do not know - microstate in particular:
f (Ω) = f T, p, µ . . . ; H(Ω), V (Ω), N (Ω) . . .
p(Ω) = p T, p, µ . . . ; H(Ω), V (Ω), N (Ω) . . .
for continuum systems in equilibrium
for discrete systems in equilibrium
We anticipated that extensive state variables (like total energy
crostate
Ω,
but required that the PDF
f
or probability
p
H)
depend explicitly on the mi-
depend on
Ω
only implicitly through
macroscopic state variables. Knowing nothing about microstates, all microstates with the same
energy, etc. appear equally likely.
The postulated equilibrium probability distribution must be tested in experiments. In practice,
all predictions of this postulate match the experiments because there are so many
corresponding to the
same macrostate.
looks completely chaotic.
realizes the observed
microstates
The detailed time evolution from one microstate to another
One indeed cannot tell anything about which particular
macrostate
microstate
at any instant of observation (i.e. there is no signicant bias).
The implicit dependence of the equilibrium PDF on total energy,
f (Ω) = f H(Ω) ,
can be
rigorously derived from the equations of motion. We will show this later.
We used the notation
H
for total energy to emphasize its dierence from internal energy
former is a random state variable
average value
E = hHi.
H(Ω)
E.
The
that depends on the microstate, while the latter is the
Also, the total energy expressed as a function
H(Ω) of the system's state
is called Hamiltonian in mechanics.
• Microcanonical ensemble
represents perfectly isolated systems in equilibrium. The total energy
E
of an isolated system is conserved, so that the PDF or probability is:
f Ω = Cδ H(Ω)−E
where
C, C 0
0
p(Ω) = C δH(Ω),H ≡ C
;
continuum
are normalization constants,
0
1
0
, H(Ω) = E
, H(Ω) =
6 E
δ(x) is the Dirac delta function, and δx,y
discrete
is Kronecker symbol
(the discrete analogue of Dirac function). The system's volume and number of particles are also xed
(but we emphasize only the PDF's dependence on energy).
• Canonical ensemble
represents the systems in equilibrium at given temperature
T.
The canonical
PDF or probability is:
f H(Ω) = Ce−βH(Ω)
where
C, C 0
β
β = 1/kB T ).
are normalization constants, and
energy (we will show later that
p(Ω) = C 0 e−βH(Ω)
;
continuum
discrete
is a temperature-dependent factor with units of inverse
This PDF is known as Boltzmann distribution.
The
system exchanges energy with its environment (so it is not conserved), but keeps its volume and
particle number xed.
One can derive the canonical PDF in the following manner.
in contact and thermal equilibrium with each other.
ensemble
EA (T )
and
B
T
Consider two systems
A
A
f
and
B
as a member of a canonical
as a member of a separate but equivalent canonical ensemble
There is a unique functional form of the PDF
by temperature
Regard
EB (T ).
for all canonical ensembles. It is parametrized
and cannot explicitly depend on the microstate (since we have equilibrium).
It can, however, depend on a few extensive microstate functions such as system's total energy
H,
N , etc. Consequently, the PDFs for nding the systems A
f (T ; HA , VA , NA . . . ) and f (T ; HB , VB , NB . . . ) respectively.
Now consider the joint system A + B . Since energy can be exchanged between its parts, A + B is
another equilibrium system belonging to the same type of canonical ensemble EA+B (T ). The PDF
of A + B is therefore f (T ; HA + HB , VA + VB , NA + NB . . . ). However, the degrees of freedom that
belong to A are completely independent from the degrees of freedom that belong to B (particles
are not exchanged between A and B ). The properties of probability then ensure:
and
volume
B
V,
number of particles
in particular microstates are
f (T ; HA + HB , VA + VB , NA + NB . . . )dΩA dΩB =
f (T ; HA , VA , NA . . . )dΩA × f (T ; HB , VB , NB . . . )dΩB
Cancel the phase space volume factors
dΩ
and take the logarithm:
log f (T ; HA + HB , VA + VB , NA + NB . . . ) = log f (T ; HA , VA , NA . . . ) + log f (T ; HB , VB , NB . . . )
The most general functional form that satises this equation is:
log f (T ; H, V, N . . . ) = −βH − γV − νN − · · ·
β, γ, ν
where the coecients
are parameters that characterize the established equilibrium in which
energy, forces, particles, etc. can be exchanged with the environment. Specically, the need to
introduce the parameter
β
β
can be taken as evidence that temperature exists as a state variable;
is some function of temperature, xed only by the precise denition of temperature.
consume
γV + νN + · · ·
into the normalization constant
C,
If we
then:
f (T ; H, V, N . . . ) = C(T ; V, N . . . ) × e−βH
• Grand canonical ensemble
T and in
µ (the systems exchange both energy and particles
represents the systems in equilibrium at given temperature
contact with a particle reservoir at chemical potential
with their environment). The grand canonical PDF or probability is:
−β H(Ω)−µN
f H(Ω), N = Ce
where
C, C 0
;
continuum
N
are normalization constants,
0 −β H(Ω)−µN
p(Ω) = C e
is the number of particles, and
β
discrete
is a temperature-
dependent factor with units of inverse energy.
N is kept
ν = −β 0 µ, where
The proof is analogous to that for the canonical distribution, but the dependence on
f = C × e−βH−νN .
outside of the normalization constant:
β0
has the same units as
we identify
0
β = β,
β,
Then we may write
0
ω : f = Ce−βH+β µN . Lastly,
E = H − µN . This is justied
in order to obtain the correct units for
so the exponent becomes proportional to
by the fact that chemical potential
µ
is potential energy of a particle that has been taken from
the particle reservoir into the system.
It is a real potential energy because it can be returned
to the reservoir at any time by returning the particle. Therefore, we may regard
E
as the true
total energy of the system, which includes the potential energy with respect to the reservoir. The
system with energy
E
is still in thermal equilibrium, so it should have a canonical PDF (β
• Density and number of states:
0
= β ).
The equilibrium probability must be normalized over the entire
phase space:
ˆ
dΩ f H(Ω) = 1
;
continuum
X p H(Ω) = 1
discrete
Ω
Given that equilibrium probability distributions depend only on energy, we can reorganize the normalization conditions as integrals/sums over energy. The weight function of energy in these and similar
integrals/sums is the density/number of states.
In continuum systems, we substitute the identity
ˆ
f H(Ω) = dH δ H − H(Ω) f (H)
in the normalization condition:
ˆ
dΩ f H(Ω) =
ˆ
ˆ
dH
dΩ δ H(Ω) − H f (H) =
ˆ
ω(H) =
The function
ω(H)
is the
ˆ
dΩ δ H(Ω) − H
density of microstates
at energy
H ; ω(H)dH is the phase space volume
(H, H + dH).
occupied by the microstates whose energy lies in the interval
dH ω(H)f (H) = 1
In discrete systems, we substitute the identity
X
p H(Ω) =
δH,H(Ω) p(H)
H
in the normalization condition:
XX
X X
p H(Ω) =
δH(Ω),H p(H) =
Ω(H)p(H) = 1
H
Ω
H
Ω
Ω(H) =
X
δH(Ω),H
Ω
The function
Ω(H)
is the
number of microstates
whose energy is
• Partition function: The normalization of the canonical
ˆ
ˆ
dH ω(H)f (H) = C dH ω(H)e−βH = 1
⇒
where
ω(H)
H.
ensemble PDF yields:
C=
1
Z
ˆ
,
dH ω(H)e−βH
Z=
Z is the so called partition function. Exactly the same
C 0 is obtained for discrete canonical distributions:
X
X
X
1
Ω(H)p(H) = C 0
Ω(H)e−βH = 1
⇒
C0 =
, Z=
Ω(H)e−βH
Z
is the density of states, and
form
of the normalization constant
H
H
H
Partition function depends on various state variables, which are then constrained and related to
the internal energy by the normalization condition.
Single and multi-particle probability distributions
•
Ω = (ω 1 , ω 2 . . . ω N ) in terms of the
ω i of individual degrees of freedom. If the system is a uid or solid of N particles, we
ω i = (ri , pi ), but more generally these could be other continuous or discrete quantities.
For the sake of generality, let us represent the microstate vector
state vectors
would use
Referring to a single degree of freedom as
as:
particle, we dene the single-particle PDF or probability
ˆ
f1 (ω i ) =
dΩ0 f (Ω0 )δ(ω 0i − ω i )
(continuous)
;
p1 (ω i ) =
X
p(Ω0 ) δω0i ωi
(discrete)
Ω0
using the microstate PDF
or Kronecker symbol
δxy
f (Ω)
1
only if
tion/summation to only those microstates
tions
f1
and
p1
p(Ω). Note that we used Dirac function δ(x − y)
x = y , otherwise zero) to constrain the integraΩ0 whose ith particle's state is xed to ω i . The func-
or probability
(equal to
capture the statistics of a single particle.
functions do not depend on the particle index
i,
e.g.
If all particles are identical, these
f1 (ω i ) ≡ f1 (ω).
Similarly, we dene the two-particle PDF or probability:
ˆ
f2 (ω i , ω j )
=
p2 (ω i , ω j )
=
dΩ0 f (Ω0 )δ(ω 0i − ω i )δ(ω 0j − ω j )
X
p(Ω0 ) δω0i ωi δω0j ωj (discrete)
(continuous)
Ω0
and so on. Such multi-particle functions contain information about correlations between particles.
Specically,
g2 (ω i , ω j ) = f2 (ω i , ω j ) − f1 (ω i )f1 (ω j )
is a two-body correlation function that equals zero only if the particles are uncorrelated (according
to the fundamental property of probability for independent random variables).
Assuming that
all particles are identical, the multi-particle distribution and correlation functions depend on the
coordinates of unspecied particles, as in
f2 (ω, ω 0 ).
28
•
As long as the system has a macroscopically large number
interested in few-particle
by
nN
(rather than many-particle
averaging over the degrees of freedom of a single system.
1 dN (ω)
N →∞ N
dω
where
dN (ω) is
dω from
volume
Proof:
1 dN (ω, ω 0 )
N →∞ N 2
dωdω 0
···
the number of particles (in a single system) whose coordinates are within a small
ω , and dN (ω, ω 0 ) is the number
dωdω 0 from the joint state (ω, ω 0 ).
the state
within a small volume
In other words,
f2 (ω, ω 0 ) = lim
,
f1 (ω) = lim
N of identical particles, and we are only
n ∼ N ) PDFs fn , we can obtain the PDFs
of particle pairs whose coordinates are
Consider the original single-particle PDF and substitute in its denition the dening
formula for the objective microstate PDF
ˆ
f 1 (ω) =
f (Ω):
ˆ
1 dN (ω)
1 dN (Ω0 )
= lim
N →∞ N
N →∞ N
dΩ0
dω
dΩ0 f (Ω0 )δ(ω 0i − ω) =
When the number
dN (Ω0 )
dΩ0 δ(ω 0i − ω) × lim
of ensemble members in the microstate
constrained phase space volume, one gets the number
a particle in the given state
ω.
Since
dN (ω)
dN (ω)
Ω0
is integrated over the
of ensemble members that contain
is obtained by a single counting experiment on
the entire ensemble, it can be regarded as a binomially-distributed random variable.
hdN (ω)i
Its mean
N P (ω), where P (ω) is the probability that a single ensemble member
the state ω . We can alternatively extract P (ω) from a single system by
is, then, equal to
will have a particle in
applying the denition of objective probability on the ensemble of particles in a given single
system of interest:
P (ω) = lim
N →∞
dN (ω)
N
Therefore:
f10 (ω)
1 dN (ω)
= lim
→
N →∞ N
dω
1 dN (ω)
N dω
=
1 N P (ω)
dN (ω)
= lim
= f1 (ω)
N
→∞
N dω
N dω
The equality indicated by the arrow can be regarded as a consequence of central limit theorem: the
random quantity
dN /N
is an average whose distribution converges to the Gaussian distribution
with the same mean and vanishing variance (of the order of
1/N ). Using analogous arguments,
fn0 = fn , and also generalize to
one can prove the equivalence between any multi-particle PDFs
discrete cases.
Fluctuations in equilibrium
•
Instantaneous total energy
H
of a macroscopic system is a random variable (except in the microcanon-
E = hHi.
σH
of
H
other words, the statistical uctuations of
H
are negligible, so all ensembles that describe the same
ical ensemble). The standard deviation
macrostate
(given by
E)
i
is a random variable with some nite mean
and standard deviation
(particles are equivalent even if they are correlated by interactions). The total energy
sum of
N
In
are equivalent.
Proof: A particle energy
σ
is negligible in comparison to its mean
individual particle energies
H=
N
X
i
,
i ,
e.g.:
i =
i=1
p2i
1X
+ U (ri ) +
V (rj − ri )
2m
2
j6=i
H is:
*N +
N
X
X
E = hHi =
i =
hi i = N Since all particles are equivalent, the average of
i=1
29
i=1
H
is the
and the variance of
H
is:
*
2
σH
N
X
2
= h(H − E) i =
!2 +
(i − )
=
*N
X
i=1
= N σ2 +
X
+
2
(i − ) +
i=1
X
(i − )(j − )
i6=j
h(i − )(j − )i
i6=j
Any correlations between particles must be local if interactions have short range (Coulomb interactions in plasmas can be included because screening makes them eectively short-ranged).
In other words, a particle may be correlated with other particles in its vicinity, but not with
the macroscopic majority of particles far away. Therefore, we may express the last term on the
right-hand-side in
X
2
σH
as:
h(i − )(j − )i =
X near
Xi
i
i6=j
h(i − )(j − )i +
from i
X farX
i
j6=i
= Nφ +
from i
X farX
i
h(i − )(j − )i
j
h(i − )ih(j − )i
j
= Nφ
We merely separated the sum over the second particle
that are near the rst particle
are near
i
i
j
can interact with it and contribute the amount
of particles
i
(the value of
φ
into the contribution from particles
and the contribution from far-away particles. The ones that
Nφ
proportional the total number
N
is not important). The far-away particles are uncorrelated with i, so
we can use the property of independent random variables that the average of their product equals
the product of their averages. Then, by denition of
we conclude that the full variance of
H
= hi i,
the average of
i − vanishes, and
is proportional to the number of particles:
2
σH
= N (σ2 + φ)
It can be now clearly seen that the standard deviation of
σH = const ×
√
N E = const × N
in the
N →∞
σH ≡ 0.
It can be shown using the same argument that all central moments
p
√
n
h(H − E)n i ∝ n N
N.
h(H − E)n i
in systems
Therefore, all measures of uctuations
are small in comparison to the mean
E ∝ N.
The same theorem applies also to the uctuations of other extensive quantities, such as the number
of particles
•
is much smaller than its mean,
The theorem trivially applies to the microcanonical ensemble, where
with short-range correlations are proportional to
H
limit.
N.
Few body PDF's
fn
(for
N n)
are the same in all types of equilibrium ensembles (microcanonical,
canonical, etc.) that correspond to the same macrostate. In this sense (much stricter than previously),
all ensembles are equivalent.
Proof: Consider
N
identical particles and focus on
ˆ
f1 (ω) =
f1 (ω):
ˆ
0
0
dΩ f (Ω
)δ(ω 01
− ω) =
dΩ0 f (Ω0 )
N
1 X
δ(ω 0i − ω)
N i=1
f (Ω0 ) ≡ f (H) depends only on the energy H(Ω0 ) in equilibrium, and must
E = hH(Ω0 )i xed by the macrostate specication. The ensemble type deter-
The microstate PDF
produce the mean
mines the system's boundary condition (interaction with the environment)
30
in a given macrostate.
Thus, only the uctuations of
H
(the variance and higher-order cumulants) may depend on the
type of ensemble. Having this in mind, it is useful to rewrite
1
N
f1 (ω) =
ˆ
dΩ0 f (Ω0 )
N
X
δ(ω 0i − ω) =
i=1
This looks like the average
hX(H, ω)i
ˆ
1
N
f1
ˆ
dH f (H) ×
as an integral over energy:
N
X
dΩ0 δ H(Ω0 ) − H
δ(ω 0i − ω)
i=1
of the random quantity
ˆ
X(H, ω) =
N
X
dΩ0 δ H(Ω0 ) − H
δ(ω 0i − ω)
i=1
that depends on random total energy H and the given single-particle state ω . By construction,
X(H, ω) is proportional to the average number of particles at ω in the microstates with total
energy H . This is a local property of the system's dynamics (mathematically encoded in H(Ω),
macroscopic total
energy per particle
the Hamiltonian) and the density of states, but expressed as a function of the
energy. Small changes of
= H/N
do aect
H
X;
have very little eect on
only small changes of
X:
dk X
∼ O(N 0 )
dk
dk X
1 dk X
= k k
k
dH
N d
⇒
We can exploit this feature in the expansion of
f1 (ω) = hX(H, ω)i
in terms of central moments
h(H − E)k i:
∞
X
1 dk X(E, ω)
1 d2 X(E, ω)
k
hX(H, ω)i =
h(H
−
E)
i
=
X(E,
ω)
+
Var(H) + · · ·
k!
dE k
2
dE 2
k=0
This expansion follows from the Taylor expansion of
average. We proved before that Var(H)
dk X/dE k ∝ 1/N k
∝N
under the ensemble
N ).
Using
we nd:
f1 (ω) = hX(H, ω)i = X(E, ω) + C2
Therefore,
hX(H, ω)i about H = E
(all central moments are proportional to
1
1
N →∞
N + C3 3 N + · · · −−−−→ X(E, ω)
N2
N
f1 (ω) equals the variable X evaluated at the mean energy H = E in the N → ∞ limit.
f1 is identical to the microcanonical ensemble calculation, where total
This way of calculating
energy is xed. In other ensembles, energy uctuations produce corrections (the terms involving
constants
C2 , C3 . . . ),
but all of them vanish).
The proof for other multi-particle PDFs
when
X
n
with
becomes comparable to
E
N
fn
proceeds in the same steps. It breaks down, however,
(because the assumption about the changes of the generalized
fails).
Entropy
•
The statistical formula for entropy consistent with laws of thermodynamics is:
ˆ
S = −kB
dΩ f (Ω) log f (Ω)∆Ω
(continuum)
S = −kB
;
X
p(Ω) log p(Ω)
(discrete)
Ω
In continuum systems,
f (Ω)dΩ → f (Ω)∆Ω inside the logarithm
∆Ω is the elementary phase
analogy with discrete systems. Then,
is microstate probability
p(Ω)
in
space volume that cannot contain
more than one microstate.
Proof: Entropy
S
is dened in thermodynamics as an
independent
transfer of heat in a reversible change from one equilibrium state
ˆB
SB − SA =
A
31
dQ
T
A
state variable related to the
to another
B:
If the system is unable to do any work, then, by conservation of energy, the amount of transferred
heat equals the change of internal energy:
ˆB
SB − SA =
dE T dW =0
A
rd law of thermodynamics to deduce the statistical denition of
We will use this relationship and 3
entropy. Without loss of generality, we will work with a discrete ensemble and seek an expression
of the form:
S = hSi =
X
p(Ω)S(Ω)
Ω
which properly extracts entropy as a state variable (ensemble average) from the distribution of
S(Ω). We ought to nd S(Ω) that is
H(Ω), because entropy is an independent state
some physical quantity
fundamentally independent of
microstate energy
variable. Consider a process in
which the internal energy of the system is changed at
for the system to do work
xed temperature
and
without any ability
(an example is Joules' experiment with a freely expanding gas that has
nothing to push and do work on). Based on this process and the thermodynamic denition of
entropy, we nd:
SB − SA =
This is a type of an equation of
related as above
E
1
(EB − EA )
T
state: S and E
only in equilibrium
⇒
S = const +
E
T
are independent at the microscopic level, but
under the conditions we xed. Let us express internal energy
as a desired ensemble average:
S = const +
1X
p(Ω)H(Ω)
T
Ω
and then try to eliminate the reference to the microstate energy in the sum:
S
=
=
1 X
1 X
p(Ω) log e−βH(Ω) = const −
p(Ω) log Zp(Ω)
βT
βT
Ω
Ω
X
1
0
const −
p(Ω) log p(Ω)
βT
const
−
Ω
We temporarily used the canonical probability distribution
p(Ω) =
1 −βH(Ω)
e
Z
and the fact that partition function
Z
,
Z=
X
e−βH(Ω)
Ω
0
is a constant whose logarithm we may absorb into const .
However, any probability distribution xed by the macrostate (i.e.
any type of ensemble) is
acceptable, because all ensembles are statistically equivalent. We would like to identify:
S(Ω) → −
1
log p(Ω)
βT
but this is not adequate because we need a truly microscopic denition of
S
that makes no
reference to macrostate quantities such as temperature. The problem is eliminated only if
βT
is
constant. In other words,
1
kB T
is fundamentally required by the units of β (βE is dimensionless) and the fact that entropy is
an independent state variable. We will later show that β = 1/kB T by studying the statistical
β∝
mechanics of an ideal gas - this is a matter of how exactly temperature is dened. Hence, the
meaning of the physical quantity
S(Ω) that microscopically relates to entropy is derived from the
Ω in the ensemble of microstates prepared from the same
fraction of occurrences of the microstate
macrostate.
32
rd law of thermodynamics.
We will now prove that the above constant should be zero using 3
Canonical distribution (which makes a reference to temperature) shows that the system is certainly
in its lowest energy state
p(Ω)
=
P
H = H0
T = 0:
at
1
1
1
T →0
e−βH = P −H/k T e−H/kB T −−−→ −H /k T e−H/kB T
B
e−βH
e
e 0 B
T →0
= e−(H−H0 )/kB T −−−→ δH(Ω),H0
because the partition function in the denominator becomes dominated by the smallest energy
in the spectrum, i.e.
Z ≈e−H0 /kB T .
few lowest energy states. If the lowest energy state
S
X
Ω0
is unique, then
const
0
− kB
=
const
0
− kB × 1 × log(1) = const0 → 0
Ω
const
0
= 0.
p(Ω) → δH(Ω),H0 = δΩ,Ω0 :
X
T →0
p(Ω) log p(Ω) −−−→ const0 − kB
δΩ,Ω0 log(δΩ,Ω0 )
=
Since entropy vanishes at
H0
All natural systems have a unique lowest energy state, or a
T = 0
Ω
rd law of thermodynamics, we nd that
according to the 3
If, more generally, there are a few minimum energy states, then entropy dened this
way will be nite at
T = 0, but of the order one (hence
N as an extensive quantity.
negligible), instead of being proportional
to the number of particles
•
E , entropy
S = kB log Ω(E)
In discrete microcanonical ensemble at total energy
where
Ω(E)
equals:
is the number of microstates at that energy.
Proof: The microcanonical probability distribution specied by internal energy
p(Ω) =
where
1
δH(Ω),E
Ω(E)
,
Ω(E) =
X
E
is:
δH(Ω),E
Ω
Ω(E) is the number of microstates at energy E .
Hence, entropy in a discrete microcanonical
ensemble becomes:
S
=
const
0
− kB
X
p(Ω) log p(Ω)
Ω
δH(Ω),E
Ω(E)
Ω
kB X
1
= const0 −
δH(Ω),E log
Ω(E)
Ω(E)
Ω
kB
1
0
= const −
× Ω(E) log
Ω(E)
Ω(E)
0
= const + kB log Ω(E)
=
const
0
−
kB X
δH(Ω),E log
Ω(E)
Note that the Kronecker symbol inside the logarithm (2nd line) has no eect because the only time
it matters is when the same Kronecker symbol outside of the logarithm is equal to one. Then, the
logarithm is essentially independent of
Ω and can be pulled out of the sum. The isolated sum over
Ω(E), which is canceled by the normalization of the
microstates is just the number of microstates
probability. The nal result is that entropy is the logarithm of the number of microstates at given
energy
E,
rd law of thermodynamics says that
up to a constant. 3
33
S=0
at zero temperature.
•
nd law of ther-
Having the statistical denition of entropy, it is possible to microscopically derive the 2
modynamics. Any system constrained to be in a given macrostate still has many accessible microstates.
Out of equilibrium, there could be some bias in the statistical distribution of microstates. For example,
one could prepare a gas by putting all of its particles in a small volume at a corner of a large box.
Among all existing microstates for the gas in the box, only those with particles conned to the corner
are initially likely. The gas will, however, quickly spread through the entire box. In this process, the
entropy
S = kB log Ω(E)
grows because the number of available microstates of the gas with conserved energy
E
grows with
the volume that the gas occupies. As the system evolves from a less to a more likely (i.e. chaotic)
nd law of thermodynamics.
condition, its entropy increases - which is one way to state 2
•
The laws of nature are invariant under time-reversal. For any possible process, the one unfolding in
exactly the opposite order is also possible according to the equation of motion. So, if one drops an
egg on the oor and the egg breaks, it is possible in principle to simultaneously impart on every atom
exactly the opposite velocity from the one it had at the instant of fall - the pieces of the broken egg
would y upward, reassemble and land back to one's hand. Such a reversed process is never observed,
even though it is possible.
nd law of thermodynamics, now interpreted
This is a reection of the 2
statistically. Every macroscopic system always evolves to more probable and more featureless states.
This is the most likely scenario, and when the number of degrees of freedom is huge, the probability
of this is huge. The reversed process is just statistically extremely unlikely. In that sense, there is an
arrow of time, the causal passage of time from past to future.
•
Entropy dened by
S∝−
X
p(Ω) log p(Ω)
Ω
is a mathematical concept, an abstract property of a probability distribution. It expresses the overall
uncertainty of measurement outcomes, or the amount of information we lack about the described
statistical system. We could have postulated this formula based solely on its mathematical meaning,
and then derived all equilibrium distributions from the requirement that entropy (uncertainty) be
maximized. The following statements have rigorous mathematical proofs:
Among all possible ensembles of a statistical system (not just equilibrium ones) with xed energy,
the one with maximum entropy is
Among all possible ensembles of a statistical system with xed temperature, the one with maximum entropy is
canonical.
Among all probability distributions dened on a set of possible measurement outcomes, the one
with maximum entropy is
uniform.
Among all probability distributions dened on
maximum entropy is
microcanonical.
exponential.
Among all probability distributions dened on
σ 2 = Var(x),
x ∈ (0, ∞)
with xed mean
µ = hxi,
the one with
x ∈ (−∞, ∞) with xed mean µ = hxi and variance
the one with maximum entropy is
Gaussian.