Extremal Principles in Classical, Statistical and Quantum Mechanics

guest post by Mike Stay

The table in John’s post on quantropy shows that energy and action are analogous:

Statics Dynamics
statistical mechanics quantum mechanics
probabilities amplitudes
Boltzmann distribution Feynman sum over histories
energy action
temperature Planck’s constant times i
entropy quantropy
free energy free action

However, this seems to be part of a bigger picture that includes at least entropy as analogous to both of those, too. I think that just about any quantity defined by an integral over a path would behave similarly.

I see four broad areas to consider, based on a temperature parameter:

  1. T = 0: statics, or “least quantity”
  2. Real T > 0: statistical mechanics
  3. Imaginary T: a thermal ensemble gets replaced by a quantum superposition
  4. Complex T: ensembles of quantum systems, as in nuclear magnetic resonance

I’m not going to get into the last of these in what follows.

1. “Least quantity”

Lagrangian of a classical particle

K is kinetic energy, i.e. the “action density” due to motion.

V is potential energy, i.e. minus the “action density” due to position.

The action is then:

\displaystyle \begin{array}{rcl} A &=& \int (K-V) \, d t \\ & = & \int \left[m\left(\frac{d q(t)}{d t}^2 - V(q(t)\right)\right] d t \end{array}

where m is the particle’s mass. We get the principle of least action by setting \delta A = 0.

“Static” systems related by a Wick rotation

  1. Substitute q(s = iz) for q(t) to get a “springy” static system.

    In John’s homework problem A Spring in Imaginary Time, he guided students through a Wick-rotation-like process that transforms the Lagrangian above into the Hamiltonian of a springy system. (I say “springy” because it’s not exactly the Hamiltonian for a hanging spring: here each infinitesimal piece of the spring is at a fixed horizontal position and is free to move only vertically.)

    \kappa is the potential energy density due to stretching.

    \upsilon is the potential energy density due to position.

    We then have

    \displaystyle \begin{array}{rcl}\int(\kappa-\upsilon) dz & = & \int\left[k\left(\frac{dq(iz)}{dz}\right)^2 - \upsilon(q(iz))\right] dz\\ & = & -i\int\left[-k\left(\frac{dq(iz)}{diz}\right)^2 - \upsilon(q(iz))\right] diz\\ & = & i \int\left[k\left(\frac{dq(iz)}{diz}\right)^2 + \upsilon(q(iz))\right] diz \end{array}

    or letting s = iz,

    \displaystyle \begin{array}{rcl} & = & i\int\left[k\left(\frac{dq(s)}{ds}\right)^2 + \upsilon(q(s))\right] ds\\ & = & iE \end{array}

    where E is the potential energy of the spring. We get the principle of least energy by setting \delta E = 0.

  2. Substitute q(β = iz) for q(t) to get a thermometer
    system.

    We can repeat the process above, but use inverse temperature, or “coolness”, instead of time. Note that this is still a statics problem at heart! We’ll introduce another temperature below when we allow for multiple possible q‘s.

    K is the potential energy due to rate of change of q with respect to \beta. (This has to do with the thermal expansion coefficient: if we fix length of the thermometer and then cool it, we get “stretching” potential energy.)

    V is any extra potential energy due to q.

    \displaystyle \begin{array}{rcl}\int(K-V) dz & = & \int\left[k\left(\frac{dq(iz)}{dz}\right)^2 - V(q(iz))\right] dz\\ & = & -i\int\left[-k\left(\frac{dq(iz)}{diz}\right)^2 - V(q(iz))\right] diz\\ & = & i \int\left[k\left(\frac{dq(iz)}{diz}\right)^2 + V(q(iz))\right] diz \end{array}

    or letting \beta = iz,

    \displaystyle \begin{array}{rcl} & = & i\int\left[k\left(\frac{dq(\beta)}{d\beta}\right)^2 + V(q(\beta))\right] d\beta\\ & = & iS_1\end{array}

    where S_1 is the entropy lost as the thermometer is cooled. We get the principle of “least entropy lost” by setting \delta S_1 = 0.

  3. Substitute q(T₁ = iz) for q(t).

    We can repeat the process above, but use temperature instead of time. We get a system whose heat capacity is governed by a function q(T) and its derivative. We’re trying to find the best function q, the most efficient way to raise the temperature of the system.

    C is the heat capacity (= entropy) proportional to (dq/dT_1)^2.

    V is the heat capacity due to q.

    \displaystyle \begin{array}{rcl}\int(C-V) dz & = & \int\left[k\left(\frac{dq(iz)}{dz}\right)^2 - V(q(iz))\right] dz\\ & = & -i\int\left[-k\left(\frac{dq(iz)}{diz}\right)^2 - V(q(iz))\right] diz\\ & = & i \int\left[k\left(\frac{dq(iz)}{diz}\right)^2 + V(q(iz))\right] diz \end{array}

    or letting T_1 = iz,

    \displaystyle \begin{array}{rcl} & = & i\int\left[k\left(\frac{dq(T_1)}{dT_1}\right)^2 + V(q(T_1))\right] dT_1\\ & = & iE \end{array}

    where E is the energy required to raise the
    temperature. We again get the principle of least energy by setting \delta E = 0.

2. Statistical mechanics

Here we allow lots of possible q‘s, then maximize entropy subject to constraints using the Lagrange multiplier trick.

Statistical mechanics of a particle

For the statistical mechanics of a particle, we choose a real measure a_x on the set of paths. For simplicity, we assume the set is finite.

Normalize so \sum a_x = 1.

Define entropy to be S = - \sum a_x \ln a_x.

Our problem is to choose a_x to minimize the “free action” F = A - \lambda S, or, what’s equivalent, to maximize S subject to a constraint on A.

To make units match, λ must have units of action, so it’s some multiple of ℏ. Replace λ by ℏλ so the free action is

F = A - \hbar\lambda\, S.

The distribution that minimizes the free action is the Gibbs distribution a_x = \exp(-A/\hbar\lambda) / Z, where Z is the usual partition function.

However, there are other observables of a path, like the position q_{1/2} at the halfway point; given another constraint on the average value of q_{1/2} over all paths, we get a distribution like

\displaystyle a_x = \exp(-\left[A + pq_{1/2}\right]/\hbar\lambda) / Z.

The conjugate variable to that position is a momentum: in order to get from the starting point to the given point in the allotted time, the particle has to have the corresponding momentum.

dA = \hbar\lambda\, dS - p\, dq.

Other examples from Wick rotation

  1. Introduce a temperature T [Kelvins] that perturbs the spring.

    We minimize the free energy F = E - kT\, S, i.e. maximize the entropy S subject to a constraint on the expected energy

    \langle E\rangle = \sum a_x E_x.

    We get the measure a_x = \exp(-E_x/kT) / Z.

    Other observables about the spring’s path give conjugate variables whose product is energy. Given constraint on the average position of the spring at the halfway point, we get a conjugate force: pulling the spring out of equilibrium requires a force.

    dE = kT\, dS - F\, dq.

  2. Statistical ensemble of thermometers with ensemble temperature T₂ [unitless].

    We minimize the “free entropy” F = S_1 - T_2S_2, i.e. we maximize the entropy S_2 subject to a constraint on the expected entropy lost

    \langle S_1\rangle = \sum a_x S_{1,x}.

    We get the measure a_x = \exp(-S_{1,x}/T_2) / Z.

    Given a constraint on the average position at the halfway point, we get a conjugate inverse length r that tells how much entropy is lost when the thermometer shrinks by dq.

    dS_1 = T_2\, dS_2 - r\, dq.

  3. Statistical ensemble of functions q with ensemble temperature T₂ [Kelvins].

    We minimize the free energy F = E - kT_2\, S, i.e. we maximize the entropy S subject to a constraint on the expected energy

    \displaystyle \langle E\rangle = \sum a_x E_x.

    We get the measure a_x = \exp(-E_x/kT_2) / Z.

    Again, a constraint on the position would give a conjugate force. It’s a little harder to see how here, but given a non-optimal function q(T), we have an extra energy cost due to inefficiency that’s analogous to the stretching potential energy when pulling a spring out of equilibrium.

3. Thermo to quantum via Wick rotation of Lagrange multiplier

We allow a complex-valued measure a as John did in the article on quantropy. We pick a logarithm for each a_x and assume they don’t go through zero as we vary them. We also choose an imaginary Lagrange multiplier.

Normalize so \sum a_x = 1.

Define quantropy Q = - \sum a_x \ln a_x.

Find a stationary point of the free action F = A - \hbar\lambda\, Q.

We get a_x = \exp(-A_x/\hbar\lambda). If \lambda = -i, we get Feynman’s sum over histories. Surely something like the two-slit experiment considers histories with a constraint on position at a particular time, and we get a conjugate momentum?

A Quantum Version of Entropy

Again allow complex-valued a_x. However, this time normalize these by setting \sum |a_x|^2 = 1.

Define a quantum version of entropy S = - \sum |a_x|^2 \ln |a_x|^2.

  1. Allow quantum superposition of perturbed springs.

    \langle E\rangle = \sum |a_x|^2 E_x. Get a_x = \exp(-E_x/kT) / Z. If T = -i\hbar/tk, we get the evolution of the quantum state |q\rangle under the given Hamiltonian for a time t.

  2. Allow quantum superpositions of thermometers.

    \langle S_1\rangle = \sum |a_x|^2 S_{1,x}. Get a_x = \exp(-S_{1,x}/T_2) / Z. If T_2 = -i, we get something like a sum over histories, but with a different normalization condition that converges because our set of paths is finite.

  3. Allow quantum superposition of systems.

    \langle E \rangle = \sum |a_x|^2 E_x. Get a_x =\exp(-E_x/kT_2) / Z. If T_2 = -i\hbar/tk, we get the result of “Measure E, then heat the superposition T₁ degrees in a time much less than t seconds, then wait t seconds.” Different functions q in the superposition change the heat capacity differently and thus the systems end up at different energies.

So to sum up, there’s at least a three-way analogy between action, energy, and entropy depending on what you’re integrating over. You get a kind of “statics” if you extremize the integral by varying the path; by allowing multiple paths and constraints on observables, you get conjugate variables and “free” quantities that you want to minimize; and by taking the temperature to be imaginary, you get quantum systems.

This entry was posted on Friday, January 13th, 2012 at 5:05 pm and is filed under physics, probability. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

7 Responses to Extremal Principles in Classical, Statistical and Quantum Mechanics

  1. John Baez says:
    13 January, 2012 at 5:10 pm

    In case you’re wondering, this is a better formatted, somewhat edited version of Mike’s comment on my post about quantropy.

    Reply
  2. Douglas Summers Stay says:
    16 January, 2012 at 9:37 am

    This is too advanced for me. Maybe you could do a new version with more explanation and fewer equations? I want to know things like this: what is new here? Why is this exciting? how could this be used? What new insights does this give into everyday things? What aspects of the analogy exist on one side but are unnamed on the other, so are new discoveries about the other side? Basically a Scientific American style article.

    Reply
    • John Baez says:
      16 January, 2012 at 10:33 am

      Douglas wrote:

      Maybe you could do a new version with more explanation and fewer equations?

      I know you were talking to your brother Mike, but you’ve just nudged me into saying a bit myself…

      What aspects of the analogy exist on one side but are unnamed on the other, so are new discoveries about the other side?

      The first thing to note is that Mike is not talking about an analogy between two sides, but at least four sides, and perhaps more. The main four are:

      • In classical dynamics, a system like a thrown rock will trace out the path that minimizes action.

      • In classical statics, a system like a hanging spring traces out the path that minimizes energy.

      • In thermal statics, a system in thermal equilibrium like a box of gas will occupy different states with different probabilities in a way that maximizes entropy.

      • In quantum dynamics, a system like a thrown quantum rock will trace out different paths with different amplitudes in a way that makes the first derivative of quantropy zero.

      The analogy between the first two was explained (not in Scientific American fashion) in my problem set A Spring in Imaginary Time. The concept of quantropy is described in my blog post Quantropy.

      Infinitely more famous than either of these are other aspects of the analogy between thermal statics and quantum dynamics: people use this analogy to do things like calculate the mass of the proton from first principles. If anyone was really going to write a Scientific American article on this material, that would be the place to start. It’s incredibly important, but I’ve never seen anyone try to explain it to ordinary folks.

      Mike is trying to push these analogies further than I’d done. So far I only understand certain portions of what he’s written. The portions I understand seem right, but I can’t vouch for the whole thing yet.

      Here are some things I don’t understand:

      Substitute q(β = iz) for q(t) to get a thermometer system.

      We can repeat the process above, but use inverse temperature, or “coolness”, instead of time. Note that this is still a statics problem at heart! We’ll introduce another temperature below when we allow for multiple possible q‘s.

      K is the potential energy due to rate of change of q with respect to \beta. (This has to do with the thermal expansion coefficient: if we fix length of the thermometer and then cool it, we get “stretching” potential energy.)

      I don’t understand what’s going on here. Mike seems to be treating the length of a thermometer as a function of temperature, or more precisely the reciprocal of temperature β. But I don’t understand the assumptions in this game—like what laws of physics we’re assuming, or what’s a function of what.

      Substitute q(T₁ = iz) for q(t).

      We can repeat the process above, but use temperature instead of time. We get a system whose heat capacity is governed by a function q(T) and its derivative. We’re trying to find the best function q, the most efficient way to raise the temperature of the system.

      C is the heat capacity (= entropy) proportional to (dq/dT_1)^2.

      V is the heat capacity due to q.

      Again, I don’t understand what’s going on here. And now I don’t even have a mental image (like a thermometer) to cling to.

      Allow quantum superpositions of thermometers.

      \langle S_1\rangle = \sum |a_x|^2 S_{1,x}.

      Get a_x = \exp(-S_{1,x}/T_2) / Z. If T_2 = -i, we get something like a sum over histories, but with a different normalization condition that converges because our set of paths is finite.

      Ditto.

      Allow quantum superposition of systems.

      \langle E \rangle = \sum |a_x|^2 E_x.

      Get a_x =\exp(-E_x/kT_2) / Z. If T_2 = -i\hbar/tk, we get the result of “Measure E, then heat the superposition T₁ degrees in a time much less than t seconds, then wait t seconds.” Different functions q in the superposition change the heat capacity differently and thus the systems end up at different energies.

      Ditto. I’m afraid I need more help to understand these four items.

      Reply
      • Mike Stay says:
        20 January, 2012 at 7:33 am

        In each example, I’m just doing a formal replacement and trying to come up with a plausible story as to why such a system might arise. I thought it was OK to do that, since it’s not particularly clear to me how you’d build your “springy” system, either.

        Let’s just talk about the second example I gave, the “thermometer system”. There are composite materials like SiC/Al fibers whose thermal expansion coefficient \alpha depends on the coolness \beta. (http://www.waset.org/journals/waset/v78/v78-179.pdf) A rod made out of this stuff has an equilibrium length q(\beta) that’s also a function of the coolness, given by integrating \alpha(T) dT. If we fix the length of the rod and then cool it, it will have some stretching potential energy K(\beta). We can also toss in some other kind of potential energy V that depends on q(\beta) just to make the analogy closer. The entropy lost upon cooling it is the integral of the stretching potential energy plus the extra potential energy. Minimizing this entropy while keeping the initial and final equilibrium lengths fixed becomes a materials science question: which composite produces the best q(\beta)?

        Reply
  3. Douglas Summers-Stay says:
    16 January, 2012 at 7:34 pm

    Thanks, that was very helpful.

    Reply
  4. Luisberis Velazquez says:
    21 March, 2013 at 3:42 pm

    There exist a deep analogy between statistical mechanics and quantum mechanics. In particular, both statistical theories support the existence of complementary quantities and non-commuting operators, where the entropy appears as a counterpart of classical action. In my opinion, this connection strongly suggests the development of extremal principles for nonequilibrium statistical mechanics, where the notion of complementarity would play a relevant role. More details about this subject can be found in my recent paper
    http://dx.doi.org/10.1016/j.aop.2012.03.002

    Reply
  5. Vincent says:
    1 February, 2017 at 9:12 pm

    Reading this great post, this paper by Vladimir Garcia-Morales came to my mind “Quantum Mechanics and the Principle of Least Radix Economy”

    https://arxiv.org/abs/1401.0963

    Reply

You can use Markdown or HTML in your comments. You can also use LaTeX, like this: $latex E = m c^2 $. The word 'latex' comes right after the first dollar sign, with a space after it.

Fill in your details below or click an icon to log in:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out /  Change )

Google photo

You are commenting using your Google account. ( Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.