Classical Mechanics versus Thermodynamics (Part 3)

There’s a fascinating analogy between classical mechanics and thermodynamics, which I last talked about in 2012:

• Classical mechanics versus thermodynamics (part 1).
• Classical mechanics versus thermodynamics (part 2).

I’ve figured out more about it, and today I’m giving a talk about it in the physics colloquium at the University of British Columbia. It’s a colloquium talk that’s supposed to be accessible for upper-level undergraduates, so I’ll spend a lot of time reviewing the basics… which is good, I think.

I don’t know if the talk will be recorded, but you can see my slides here, and I’ll base this blog article on them.

Hamilton’s equations versus the Maxwell relations

Why do Hamilton’s equations in classical mechanics:

look so much like the Maxwell relations in thermodynamics?

William Rowan Hamilton discovered his equations describing classical mechanics in terms of energy around 1827. By 1834 he had also introduced Hamilton’s principal function, which I’ll explain later.

James Clerk Maxwell is most famous for his equations describing electromagnetism, perfected in 1865. But he also worked on thermodynamics, and discovered the ‘Maxwell relations’ in 1871.

Hamilton’s equations describe how the position and momentum of a particle on a line change with time if we know the energy or Hamiltonian :

Two of the Maxwell relations connect the volume , entropy , pressure and temperature of a system in thermodynamic equilibrium:

Using this change of variables:

Hamilton’s equations:

become these relations:

These are almost like two of the Maxwell relations! But in thermodynamics we always use partial derivatives:

and we say which variables are held constant:

If we write Hamilton’s equations in the same style as the Maxwell relations, they look funny:

Can this possibly be right?

Yes! When we work out the analogy between classical mechanics and thermodynamics we’ll see why.

We can get Maxwell’s relations starting from this: the internal energy of a system in equilibrium depends on its entropy and volume

Temperature and pressure are derivatives of

Maxwell’s relations follow from the fact that mixed partial derivatives commute! For example:

To get Hamilton’s equations the same way, we need a function of the particle’s position and time such that

Then we’ll get Hamilton’s equations from the fact that mixed partial derivatives commute!

The trick is to let be ‘Hamilton’s principal function’. So let’s define that. First, the action of a particle’s path is

where is the Lagrangian:

The particle always takes a path from to that’s a critical point of the action. We can derive Hamilton’s equations from this fact.

Let’s assume this critical point is a minimum. Then the least action for any path from to is called Hamilton’s principal function

A beautiful fact: if we differentiate Hamilton’s principal function, we get back the energy and momentum :

You can prove these equations using

which implies that

where we integrate along the minimizing path. (It’s not as trivial as it may look, but you can do it.)

Now let’s fix a starting-point for our particle, and say its path ends at any old point . Think of Hamilton’s principal function as a function of just :

Then the particle’s momentum and energy when it reaches are:

This is just what we wanted. Hamilton’s equations now follow from the fact that mixed partial derivatives commute!

So, we have this analogy between classical mechanics and thermodynamics:

Classical mechanics	Thermodynamics
action:	internal energy:
position:	entropy:
momentum:	temperature:
time:	volume:
energy:	pressure:

What’s really going on in this analogy? It’s not really the matchup of variables that matters most—it’s something a bit more abstract. Let’s dig deeper.

I said we could get Maxwell’s relations from the fact that mixed partials commute, and gave one example:

But to get the other Maxwell relations we need to differentiate other functions—and there are four of them!

• : internal energy
• : Helmholtz free energy
• : enthalpy
• : Gibbs free energy

They’re important, but memorizing all the facts about them has annoyed students of thermodynamics for over a century. Is there some other way to get the Maxwell relations? Yes!

In 1958 David Ritchie explained how we can get all four Maxwell relations from one equation! Jaynes also explained how in some unpublished notes for a book. Here’s how it works.

Start here:

Integrate around a loop :

so

This says the heat added to a system equals the work it does in this cycle

Green’s theorem implies that if a loop encloses a region

Similarly

But we know these are equal!

So, we get

for any region enclosed by a loop. And this in turn implies

In fact, all of Maxwell’s relations are hidden in this one equation!

Mathematicians call something like a 2-form and write it as . It’s an ‘oriented area element’, so

Now, starting from

We can choose any coordinates and get

(Yes, this is mathematically allowed!)

If we take we get

and thus

We can actually cancel some factors and get one of the Maxwell relations:

(Yes, this is mathematically justified!)

Let’s try another one. If we take we get

Cancelling some factors here we get another of the Maxwell relations:

Other choices of give the other two Maxwell relations.

In short, Maxwell’s relations all follow from one simple equation:

Similarly, Hamilton’s equations follow from this equation:

All calculations work in exactly the same way!

By the way, we can get these equations efficiently using the identity and the product rule for :

Now let’s change viewpoint slightly and temporarily treat and as independent from and So, let’s start with with coordinates . Then this 2-form on :

is called a symplectic structure.

Choosing the internal energy function , we get this 2-dimensional surface of equilibrium states:

Since

we know

for any region in the surface , since on this surface and our old argument applies.

This fact encodes the Maxwell relations! Physically it says: for any cycle on the surface of equilibrium states, the heat flow in equals the work done.

Similarly, in classical mechanics we can start with with coordinates , treating and as independent from and This 2-form on :

is a symplectic structure. Hamilton’s principal function defines a 2d surface

We have for any region in this surface And this fact encodes Hamilton’s equations!

Summary

In thermodynamics, any 2d region in the surface of equilibrium states has

This is equivalent to the Maxwell relations.

In classical mechanics, any 2d region in the surface of allowed 4-tuples for particle trajectories through a single point has

This is equivalent to Hamilton’s equations.

These facts generalize when we add extra degrees of freedom, e.g. the particle number in thermodynamics:

or more dimensions of space in classical mechanics:

We get a vector space with a 2-form on it, and a Lagrangian submanifold : that is, a n-dimensional submanifold such that

for any 2d region

This is more evidence for Alan Weinstein’s “symplectic creed”:

EVERYTHING IS A LAGRANGIAN SUBMANIFOLD

As a spinoff, we get two extra Hamilton’s equations for a point particle on a line! They look weird, but I’m sure they’re correct for trajectories that go through a specific arbitrary spacetime point

Maxwell’s Relations (Part 3)

In Part 2 we saw a very efficient formulation of Maxwell’s relations, from which we can easily derive their usual form. Now let’s talk more about the meaning of the Maxwell relations—both their physical meaning and their mathematical meaning. For the physical meaning, I’ll draw again from Ritchie’s paper:

• David J. Ritchie, A simple method for deriving Maxwell’s relations, American Journal of Physics 36 (1958), 760–760.

but Emily Roach pointed out that much of this can also be found in the third chapter of Jaynes’ unpublished book:

• E. T. Jaynes, Thermodynamics, Chapter 3: Plausible reasoning.

First I’ll do the case of 2 dimensions, and then the case of n dimensions. In the 2d case I’ll talk like a physicist and use notation from thermodynamics. We’ll learn the Maxwell relations have these meanings, apart from their obvious ones:

• In any thermodynamic cycle, the heat absorbed by a system equals the work it does.

• In any thermodynamic cycle, energy is conserved.

• Any region in the surface of equilibrium states has the same area in pressure-volume coordinates as it does in temperature-entropy coordinates.

In the n-dimensional case I’ll use notation that mathematicians will like better, and also introduce the language of symplectic geometry. This will give a general, abstract statement of the Maxwell relations:

• The manifold of equilibrium states is a Lagrangian submanifold of a symplectic manifold.

Don’t worry—I’ll explain it!

Maxwell’s relations in 2 dimensions

Suppose we have a physical system whose internal energy is a smooth function of its entropy and volume So, we have a smooth function on the plane:

and we call the coordinates on this plane

Next we introduce two more variables called temperature and pressure So, we’ll give coordinates But, there’s a 2-dimensional surface where these extra variables are given by the usual formulas in thermodynamics:

We call this the surface of equilibrium states.

By the equations that and obey on the surface of equilibrium states, the following equation holds on this surface:

So, if is any loop on this surface, we have

where the total change in is zero because the loop ends where it starts! We thus have

In thermodynamics this has a nice meaning: the left side is the heat absorbed by a system as its state moves around the loop while the right side is the work done by this system. So, the equation says the heat absorbed by a system as it carries out a cycle is equal to the work it does.

We’ll soon see that this equation contains, hidden within it, all four Maxwell relations. And it’s a statement of conservation of energy! By the way, it’s perfectly obvious that energy is conserved in a cycle: our loop takes us from a point where has some value back to the same point, where it has the same value.

So how do we get Maxwell’s relations out of this blithering triviality? There’s a slow way and a fast way. Since the slow way provides extra insight I’ll do that first.

Suppose the loop bounds some 2-dimensional region in the surface Then by Green’s theorem we have

That is, the heat absorbed in the cycle is just the area of the region in coordinates. But Green’s theorem also says

That is, the work done in the cycle, which is minus the left hand side, is minus the area of the region in coordinates!

That’s nice to know. But since we’ve seen these are equal,

for every region in the surface

Well, at least I’ve shown it for every region bounded by a loop! But every region can be chopped up into small regions that are bounded by loops, so the equation is really true for any region in the surface

We express this fact by saying that equals when these 2-forms are restricted to the surface We write it like this:

Now, last time we saw how to quickly get from this equation to all four Maxwell relations!

(Back then I didn’t write the symbol because I was implicitly working on this surface without telling you. More precisely, I was working on the plane but we can identify the surface with that plane using the coordinates.)

So, given what we did last time, we are done! The equation

expresses both conservation of energy and all four Maxwell equations—in a very compressed, beautiful form!

Maxwell’s relations in n dimensions

Now we can generalize everything to n dimensions. Suppose we have a smooth function

Write the coordinates on as

and write the corresponding coordinates on the cotangent bundle as

There is a submanifold of where equals the partial derivative of with respect to

Let’s call this submanifold since it’s the same one we’ve seen before (except for that annoying little minus sign in the definition of pressure):

In applications to thermodynamics, this is the manifold of equilibrium states. But we’re doing math here, and this math has many applications. It’s this generality that makes the subject especially interesting to me.

Now, started out life as a function on but it lifts to a function on and we have

By the definition of we have

so

The 1-form on the right-hand side here:

is called the tautological 1-form on Its exterior derivative

is called the symplectic structure on this cotangent bundle. Both of these are a big deal in classical mechanics, but here we are seeing them in thermodynamics! And the point of all this stuff is that we’ve seen

Taking of both sides and using , we get

or in other words

And this is a very distilled statement of Maxwell’s relations!

Why? Well, in the 2d case we discussed earlier, the tautological 1-form is

thanks to the annoying minus sign in the definition of pressure. Thus, the symplectic structure is

and the fact that the symplectic structure vanishes when restricted to is just our old friend

As we’ve seen, this equation contains all of Maxwell’s relations.

In the n-dimensional case, is an n-dimensional submanifold of the 2n-dimensional symplectic manifold In general, if an n-dimensional submanifold of a 2n-dimensional symplectic manifold has the property that the symplectic structure vanishes when restricted to we call a Lagrangian submanifold.

So, one efficient abstract statement of Maxwell’s relations is:

The manifold of equilibrium states is a Lagrangian submanifold.

It takes a while to see the value of this statement, and I won’t try to explain it here. Instead, read Weinstein’s introduction to symplectic geometry:

• Alan Weinstein, Symplectic geometry, Bulletin of the American Mathematical Society 5 (1981), 1–13.

You’ll see here an introduction to Lagrangian submanifolds and an explanation of the “symplectic creed”:

EVERYTHING IS A LAGRANGIAN SUBMANIFOLD.
— Alan Weinstein

The restatement of Maxwell’s relations in terms of Lagrangian submanifolds is just another piece of evidence for this!

---

• Part 1: a proof of Maxwell’s relations using commuting partial derivatives.

• Part 2: a proof of Maxwell’s relations using 2-forms.

• Part 3: the physical meaning of Maxwell’s relations, and their formulation in terms of symplectic geometry.

For how Maxwell’s relations are connected to Hamilton’s equations, see this post:

• Classical mechanics versus thermodynamics.

Maxwell’s Relations (Part 2)

The Maxwell relations are some very general identities about the partial derivatives of functions of several variables. You don’t need to understand anything about thermodyamics to understand them, but they’re used a lot in that subject, so discussions of them tend to use notation from that subject.

Last time I went through a standard way to derive these relations for a function of two variables. Now I want to give a better derivation, which I found here:

• David J. Ritchie, A simple method for deriving Maxwell’s relations, American Journal of Physics 36 (1958), 760–760.

This paper is just one page long, and I can’t really improve on it, but I’ll work through the ideas in more detail. It again covers only the case of a function of two variables, and I won’t try to go beyond that case now—maybe later.

So, remember the setup. We have a smooth function on the plane

We call the coordinates on the plane and and we give the partial derivatives of funny names:

None of these funny names or the minus sign has any effect on the actual math involved; they’re just traditional in thermodynamics. So, mathematicians, please forgive me! If I ever generalize to the n-variable case, I’ll switch to more reasonable notation.

We instantly get this:

and since we get

so

Believe it or not, this simple relation contains all four of Maxwell’s relations within it!

To see this, note that both sides are smooth 2-forms on the plane. Now, the space of 2-forms at any one point of the plane is a 1-dimensional vector space. So, we can divide any 2-form at a point by any nonzero 2-form at that point and get a real number.

In particular, suppose and are functions on the plane such that at some point. Then we can divide both sides of the above equation by and get

at this point. We can now get the four Maxwell relations simply by making different choices of and We’ll choose them to be either or The argument will only work if so I’ll always assume that. The argument works the same way each time so I’ll go faster after the first time.

The first relation

Take and and substitute them into the above equation. We get

or

Next we use a little fact about differential forms and partial derivatives to simplify both sides:

and similarly

If you were scarred as a youth when plausible-looking manipulations with partial derivatives turned out to be unjustified, you might be worried about this—and rightly so! Later I’ll show how to justify the kind of ‘cancellation’ we’re doing here. But anyway, it instantly gives us the first Maxwell relation:

The second relation

This time we take Substituting this into our general formula

we get

and doing the same sort of ‘cancellation’ as last time, this gives the second Maxwell relation:

The third relation

This time we take Substituting this into our general formula

we get

which gives the third Maxwell relation:

The fourth relation

This time we take Substituting this into our general formula

we get

or

giving the fourth Maxwell relation:

You can check that other choices of and don’t give additional relations of the same form.

Determinants

So, we’ve see that all four Maxwell relations follow quickly from the equation

if we can do ‘cancellations’ in expressions like this:

when one of the functions equals one of the functions This works whenever Let’s see why!

First of all, by the inverse function theorem, if at some point in the plane, the functions and serve as coordinates in some neighborhood in that point. In this case we have

Yes: the ratio of 2-forms is just the Jacobian of the map sending to This is clear if you know that 2-forms are ‘area elements’ and the Jacobian is a ratio of area elements. But you can also prove it by a quick calculation:

and thus

so the ratio is the desired determinant.

How does this help? Well, take

and now suppose that either or equals either or For example, suppose Then we can do a ‘cancellation’ like this:

or to make it clear that the partial derivatives are being done in the coordinate system:

This justifies all our calculations earlier.

Conclusions

So, we’ve seen that all four Maxwell relations are unified in a much simpler equation:

which follows instantly from

This is a big step forward compared to the proof I gave last time, which, at least as I presented it, required cleverly guessing a bunch of auxiliary functions—even though these auxiliary functions turn out to be incredibly important in their own right.

So, we should not stop here: we should think hard about the physical and mathematical meaning of the equation

And Ritchie does this in his paper! But I will talk about that next time.

---

• Part 1: a proof of Maxwell’s relations using commuting partial derivatives.

• Part 2: a proof of Maxwell’s relations using 2-forms.

• Part 3: the physical meaning of Maxwell’s relations, and their formulation in terms of symplectic geometry.

For how Maxwell’s relations are connected to Hamilton’s equations, see this post:

• Classical mechanics versus thermodynamics.

Maxwell’s Relations (Part 1)

The Maxwell relations are equations that show up whenever we have a smooth convex function

They say that the mixed partial derivatives of commute, but also that the mixed partial derivatives of various functions constructed from commute. So in a sense they are completely trivial except for the way we construct these various functions! Nonetheless they are very important in physics, because they give generally valid relations between the derivatives of thermodynamically interesting quantities.

For example, here is one of the Maxwell relations that people often use when studying a thermodynamic system like a cylinder of gas or a beaker of liquid:

where:

• is the entropy of the system
• is its temperature
• is its volume
• is its pressure

There are also three more Maxwell relations involving these quantities. They all have the same general look, though half contain minus signs and half don’t. So, they’re quite tiresome to memorize. It’s more interesting to figure out what’s really going on here!

I already gave one story about what’s going on: we start with a function , which happens in this case to be a function of and called the internal energy of our system. If we say its mixed partial derivatives commute:

we get the Maxwell relation I wrote above, simply by using the standard definitions of and as partial derivatives of We can then build a bunch of other functions from and and write down other equations saying that the mixed partial derivatives of these functions commute. This gives us all the Maxwell relations.

Let me show you in detail how this works, but without explaining why I’m choosing the particular ‘bunch of other functions’ that I’ll use. There is a way to explain that using the concept of Legendre transform. But next time I’ll give a very different approach to the Maxwell relations, based on this paper:

• David J. Ritchie, A simple method for deriving Maxwell’s relations, American Journal of Physics 36 (1958), 760–760.

This sidesteps the whole business of choosing a ‘bunch of other functions’, and I think it’s very nice.

What follows is a bit mind-numbing, so let me tell you what to pay attention to. I’ll do the same general sort of calculation four times, first starting with any convex smooth function and then with three other functions built from that one. The only clever part is how we choose those other functions.

The first relation

We start with any smooth convex function of two real variables and write its differential as follows:

This says

These are the usual definitions of the temperature and pressure in terms of the internal energy but you don’t need to know anything about those concepts to follow all the calculations to come. In particular, from the pure mathematical viewpoint the minus sign is merely a stupid convention.

The commuting of mixed partials implies

giving the first of the Maxwell relations:

The second relation

Next, let’s define the Helmholtz free energy

Taking its differential we get

so if we think of as a function of we get

The commuting of mixed partials implies

giving the second of the Maxwell relations:

The third relation

Copying what we did to get the second relation, let’s define the enthalpy

Taking its differential we get

so if we think of as a function of we get

The commuting of mixed partials implies

giving the third of the Maxwell relations:

The fourth relation

Combining the last two tricks, let’s define the Gibbs free energy

Taking its differential we get

so if we think of as a function of we get

The commuting of mixed partials implies

giving the fourth and final Maxwell relation:

Conclusions

I hope you’re a bit unsatisfied, for two main reasons.

The first question you should have is this: why did we chose these four functions to work with:

The pattern of signs is not really significant here: if we hadn’t followed tradition and stuck a minus sign in our definition of here:

everything would look more systematic, and we’d use these four functions:

This would make it easier to guess how everything works if instead of a function we started with a function We could write this as

and define a bunch of functions

called ‘conjugate quantities’.

Then, we could get different functions by starting with and subtracting off products of the form where ranges over some subset of

Then we could take mixed partial derivatives of these functions, note that the mixed partial derivatives commute, and get a bunch of Maxwell relations — maybe

of them! (Sanity check: when is 2 this is indeed 4.)

The second question you should have is: how did I sneakily switch from thinking of as a function of and to thinking of as a function of and and so on? How could I get away with this? I believe the answer to this involves the concept of Legendre transform, which works well when is convex.

Answering these questions well might get us into a bit of contact geometry. That would be nice. But instead of answering these questions next time, I’ll talk about Ritchie’s approach to deriving Maxwell’s equations, which seems to sidestep the two questions I just raised!

Just for fun

Finally: students of thermodynamics are often forced to memorize the Maxwell relations. They sometimes use the thermodynamic square:

where for some idiotic reason the for pressure is written lower-case—perhaps so you can mix it up with momentum!

If you like puzzles, maybe you can figure out how the thermodynamic square works if you stare at it along with the four Maxwell equations I derived:

Again, it probably be easier if we hadn’t stuck a minus sign into the definition of pressure. If you get stuck, click on the link.

Unfortunately the thermodynamic square is itself so hard to memorize that students resort to a mnemonic for that! Sometimes they say “Good Physicists Have Studied Under Very Fine Teachers” which gives the letters “GPHSUVFT” that you see as you go around the square.

I will never try to remember any of these mnemonics. But I wonder if there’s something deeper going on here. So:

Puzzle. How does the thermodynamic square generalize if is a function of 3 variables instead of 2? How about more variables?

---

• Part 1: a proof of Maxwell’s relations using commuting partial derivatives.

• Part 2: a proof of Maxwell’s relations using 2-forms.

• Part 3: the physical meaning of Maxwell’s relations, and their formulation in terms of symplectic geometry.

For how Maxwell’s relations are connected to Hamilton’s equations, see this post:

• Classical mechanics versus thermodynamics.

The Cyclic Identity for Partial Derivatives

 

As an undergrad I learned a lot about partial derivatives in physics classes. But they told us rules as needed, without proving them. This rule completely freaked me out. If derivatives are kinda like fractions, shouldn’t this equal 1?

Let me show you why it’s -1.

First, consider an example:

This example shows that the identity is not crazy. But in fact it
holds the key to the general proof! Since is a coordinate system we can assume without loss of generality that . At any point we can approximate to first order as for some constants . But for derivatives the constant doesn’t matter, so we can assume it’s zero.

Then just compute!

There’s also a proof using differential forms that you might like
better. You can see it here, along with an application to
thermodynamics:

But this still leaves us yearning for more intuition — and for me, at least, a more symmetrical, conceptual proof. Over on Twitter, someone named
Postdoc/cake provided some intuition using the same example from thermodynamics:

Using physics intuition to get the minus sign:

1. increasing temperature at const volume = more pressure (gas pushes out more)
2. increasing temperature at const pressure = increasing volume (ditto)
3. increasing pressure at const temperature = decreasing volume (you push in more)

Jules Jacobs gave the symmetrical, conceptual proof that I was dreaming of:

As I’d hoped, the minus signs come from the anticommutativity of the wedge product of 1-forms, e.g.

Since the space of 2-forms at a point in the plane is 1-dimensional, we can divide them. In fact a ratio like

is just the Jacobian of the tranformation from coordinates to coordinates. We also need that these ratios obey the rule

where are nonzero 2-forms at a point in the plane. This seems obvious, but you need to check it. It’s not hard. But to put it in fancy language, it follows from the fact that nonzero 2-forms at a point in the plane are a ‘torsor’! I explain that idea here:

• Torsors made easy.

Torsors are widespread in physics, and the nonzero vectors in any 1-dimensional real vector space form a torsor for the multiplicative group of nonzero real numbers.

Jules Jacob’s proof is more sophisticated than my simple argument, but it’s very pretty, and it generalizes to higher dimensions in ways that’d be hard to guess otherwise.

Information Geometry (Part 21)

Last time I ended with a formula for the ‘Gibbs distribution’: the probability distribution that maximizes entropy subject to constraints on the expected values of some observables.

This formula is well-known, but I’d like to derive it here. My argument won’t be up to the highest standards of rigor: I’ll do a bunch of computations, and it would take more work to state conditions under which these computations are justified. But even a nonrigorous approach is worthwhile, since the computations will give us more than the mere formula for the Gibbs distribution.

I’ll start by reminding you of what I claimed last time. I’ll state it in a way that removes all unnecessary distractions, so go back to Part 20 if you want more explanation.

The Gibbs distribution

Take a measure space with measure Suppose there is a probability distribution on that maximizes the entropy

subject to the requirement that some integrable functions on have expected values equal to some chosen list of numbers

(Unlike last time, now I’m writing and with superscripts rather than subscripts, because I’ll be using the Einstein summation convention: I’ll sum over any repeated index that appears once as a a superscript and once as a subscript.)

Furthermore, suppose depends smoothly on I’ll call it to indicate its dependence on Then, I claim is the so-called Gibbs distribution

where

and

is the entropy of

Let’s show this is true!

Finding the Gibbs distribution

So, we are trying to find a probability distribution that maximizes entropy subject to these constraints:

We can solve this problem using Lagrange multipliers. We need one Lagrange multiplier, say for each of the above constraints. But it’s easiest if we start by letting range over all of that is, the space of all integrable functions on Then, because we want to be a probability distribution, we need to impose one extra constraint

To do this we need an extra Lagrange multiplier, say

So, that’s what we’ll do! We’ll look for critical points of this function on

Here I’m using some tricks to keep things short. First, I’m dropping the dummy variable x which appeared in all of the integrals we had: I’m leaving it implicit. Second, all my integrals are over so I won’t say that. And third, I’m using the Einstein summation convention, so there’s a sum over i implicit here.

Okay, now let’s do the variational derivative required to find a critical point of this function. When I was a math major taking physics classes, the way physicists did variational derivatives seemed like black magic to me. Then I spent months reading how mathematicians rigorously justified these techniques. I don’t feel like a massive digression into this right now, so I’ll just do the calculations—and if they seem like black magic, I’m sorry!

We need to find obeying

or in other words

First we need to simplify this expression. The only part that takes any work, if you know how to do variational derivatives, is the first term. Since the derivative of is we have

The second and third terms are easy, so we get

       

Thus, we need to solve this equation:

That’s easy to do:

Good! It’s starting to look like the Gibbs distribution!

We now need to choose the Lagrange multipliers and to make the constraints hold. To satisfy this constraint

we must choose so that

or in other words

Plugging this into our earlier formula

we get this:

Great! Even more like the Gibbs distribution!

By the way, you must have noticed the “1” that showed up here:

It buzzed around like an annoying fly in the otherwise beautiful calculation, but eventually went away. This is the same irksome “1” that showed up in Part 19. Someday I’d like to say a bit more about it.

Now, where were we? We were trying to show that

minimizes entropy subject to our constraints. So far we’ve shown

is a critical point. It’s clear that

so really is a probability distribution. We should show it actually maximizes entropy subject to our constraints, but I will skip that. Given that, will be our claimed Gibbs distribution if we can show

This is interesting! It’s saying our Lagrange multipliers actually equal the so-called conjugate variables given by

where is the entropy of

There are two ways to show this: the easy way and the hard way. The easy way is to reflect on the meaning of Lagrange multipliers, and I’ll sketch that way first. The hard way is to use brute force: just compute and show it equals This is a good test of our computational muscle—but more importantly, it will help us discover some interesting facts about the Gibbs distribution.

The easy way

Consider a simple Lagrange multiplier problem where you’re trying to find a critical point of a smooth function

subject to the constraint

for some smooth function

and constant c. (The function f here has nothing to do with the f in the previous sections.) To answer this we introduce a Lagrange multiplier and seek points where

This works because the above equation says

Geometrically this means we’re at a point where the gradient of points at right angles to the level surface of

Thus, to first order we can’t change by moving along the level surface of

But also, if we start at a point where

and we begin moving in any direction, the function will change at a rate equal to times the rate of change of . That’s just what the equation says! And this fact gives a conceptual meaning to the Lagrange multiplier

Our situation is more complicated, since our functions are defined on the infinite-dimensional space and we have an n-tuple of constraints with an n-tuple of Lagrange multipliers. But the same principle holds.

So, when we are at a solution of our constrained entropy-maximization problem, and we start moving the point by changing the value of the ith constraint, namely the rate at which the entropy changes will be times the rate of change of So, we have

But this is just what we needed to show!

The hard way

Here’s another way to show

We start by solving our constrained entropy-maximization problem using Lagrange multipliers. As already shown, we get

Then we’ll compute the entropy

Then we’ll differentiate this with respect to and show we get

Let’s try it! The calculation is a bit heavy, so let’s write for the so-called partition function

so that

and the entropy is

This is the sum of two terms. The first term

is times the expected value of with respect to the probability distribution all summed over But the expected value of is so we get

The second term is easier:

since integrates to 1 and the partition function doesn’t depend on

Putting together these two terms we get an interesting formula for the entropy:

This formula is one reason this brute-force approach is actually worthwhile! I’ll say more about it later.

But for now, let’s use this formula to show what we’re trying to show, namely

For starters,

where we played a little Kronecker delta game with the second term.

Now we just need to compute the third term:

Ah, you don’t know how good it feels, after years of category theory, to be doing calculations like this again!

Now we can finish the job we started:

Voilà!

Conclusions

We’ve learned the formula for the probability distribution that maximizes entropy subject to some constraints on the expected values of observables. But more importantly, we’ve seen that the anonymous Lagrange multipliers that show up in this problem are actually the partial derivatives of entropy! They equal

Thus, they are rich in meaning. From what we’ve seen earlier, they are ‘surprisals’. They are analogous to momentum in classical mechanics and have the meaning of intensive variables in thermodynamics:

	Classical Mechanics	Thermodynamics	Probability Theory
q	position	extensive variables	probabilities
p	momentum	intensive variables	surprisals
S	action	entropy	Shannon entropy

Furthermore, by showing the hard way we discovered an interesting fact. There’s a relation between the entropy and the logarithm of the partition function:

(We proved this formula with replacing but now we know those are equal.)

This formula suggests that the logarithm of the partition function is important—and it is! It’s closely related to the concept of free energy—even though ‘energy’, free or otherwise, doesn’t show up at the level of generality we’re working at now.

This formula should also remind you of the tautological 1-form on the cotangent bundle namely

It should remind you even more of the contact 1-form on the contact manifold namely

Here is a coordinate on the contact manifold that’s a kind of abstract stand-in for our entropy function

So, it’s clear there’s a lot more to say: we’re seeing hints of things here and there, but not yet the full picture.

---

For all my old posts on information geometry, go here:

• Information geometry.

Information Geometry (Part 20)

Last time we worked out an analogy between classical mechanics, thermodynamics and probability theory. The latter two look suspiciously similar:

	Classical Mechanics	Thermodynamics	Probability Theory
q	position	extensive variables	probabilities
p	momentum	intensive variables	surprisals
S	action	entropy	Shannon entropy

This is no coincidence. After all, in the subject of statistical mechanics we explain classical thermodynamics using probability theory—and entropy is revealed to be Shannon entropy (or its quantum analogue).

Now I want to make this precise.

To connect classical thermodynamics to probability theory, I’ll start by discussing ‘statistical manifolds’. I introduced the idea of a statistical manifold in Part 7: it’s a manifold equipped with a map sending each point to a probability distribution on some measure space Now I’ll say how these fit into the second column of the above chart.

Then I’ll talk about statistical manifolds of a special sort used in thermodynamics, which I’ll call ‘Gibbsian’, since they really go back to Josiah Willard Gibbs.

In a Gibbsian statistical manifold, for each the probability distribution is a ‘Gibbs distribution’. Physically, these Gibbs distributions describe thermodynamic equilibria. For example, if you specify the volume, energy and number of particles in a box of gas, there will be a Gibbs distribution describing what the particles do in thermodynamic equilibrium under these conditions. Mathematically, Gibbs distributions maximize entropy subject to some constraints specified by the point

More precisely: in a Gibbsian statistical manifold we have a list of observables whose expected values serve as coordinates for points and is the probability distribution that maximizes entropy subject to the constraint that the expected value of is We can derive most of the interesting formulas of thermodynamics starting from this!

Statistical manifolds

Let’s fix a measure space with measure A statistical manifold is then a manifold equipped with a smooth map assigning to each point a probability distribution on which I’ll call So, is a function on with

and

for all

The idea here is that the space of all probability distributions on may be too huge to understand in as much detail as we’d like, so instead we describe some of these probability distributions—a family parametrized by points of some manifold —using the map This is the basic idea behind parametric statistics.

Information geometry is the geometry of statistical manifolds. Any statistical manifold comes with a bunch of interesting geometrical structures. One is the ‘Fisher information metric’, a Riemannian metric I explained in Part 7. Another is a 1-parameter family of connections on the tangent bundle which is important in Amari’s approach to information geometry. You can read about this here:

• Hiroshi Matsuzoe, Statistical manifolds and affine differential geometry, in Advanced Studies in Pure Mathematics 57, pp. 303–321.

I don’t want to talk about it now—I just wanted to reassure you that I’m not completely ignorant of it!

I want to focus on the story I’ve been telling, which is about entropy. Our statistical manifold comes with a smooth entropy function

namely

We can use this entropy function to do many of the things we usually do in thermodynamics! For example, at any point where this function is differentiable, its differential gives a cotangent vector

which has an important physical meaning. In coordinates we have

and we call the intensive variable conjugate to For example if is energy, will be ‘coolness’: the reciprocal of temperature.

Defining this way gives a Lagrangian submanifold

of the cotangent bundle We can also get contact geometry into the game by defining a contact manifold and a Legendrian submanifold

But I’ve been talking about these ideas for the last three episodes, so I won’t say more just now! Instead, I want to throw a new idea into the pot.

Gibbsian statistical manifolds

Thermodynamics, and statistical mechanics, spend a lot of time dealing with statistical manifold of a special sort I’ll call ‘Gibbsian’. In these, each probability distribution is a ‘Gibbs distribution’, meaning that it maximizes entropy subject to certain constraints specified by the point

How does this work? For starters, an integrable function

is called a random variable, or in physics an observable. The expected value of an observable is a smooth real-valued function on our statistical manifold

given by

In other words, is a function whose value at at any point is the expected value of with respect to the probability distribution

Now, suppose our statistical manifold is n-dimensional and we have n observables Their expected values will be smooth functions on our manifold—and sometimes these functions will be a coordinate system!

This may sound rather unlikely, but it’s really not so outlandish. Indeed, if there’s a point such that the differentials of the functions are linearly independent at this point, these functions will be a coordinate system in some neighborhood of this point, by the inverse function theorem. So, we can take this neighborhood, use it as our statistical manifold, and the functions will be coordinates.

So, let’s assume the expected values of our observables give a coordinate system on Let’s call these coordinates so that

Now for the kicker: we say our statistical manifold is Gibbsian if for each point is the probability distribution that maximizes entropy subject to the above condition!

Which condition? The condition saying that

for all i. This is just the previous equation spelled out so that you can see it’s a condition on

This assumption of the entropy-maximizing nature of is a very powerful, because it implies a useful and nontrivial formula for It’s called the Gibbs distribution:

for all

Here is the intensive variable conjugate to while is the partition function: the thing we must divide by to make sure integrates to 1. In other words:

By the way, this formula may look confusing at first, since the left side depends on the point in our statistical manifold, while there’s no visible in the right side! Do you see what’s going on?

I’ll tell you: the conjugate variable sitting on the right side of the above formula, depends on Remember, we got it by taking the partial derivative of the entropy in the direction

and the evaluating this derivative at the point

But wait a minute! here is the entropy—but the entropy of what?

The entropy of of course!

So there’s something circular about our formula for To know you need to know the conjugate variables but to compute these you need to know the entropy of

This is actually okay. While circular, the formula for is still true. It’s harder to work with than you might hope. But it’s still extremely useful.

Next time I’ll prove that this formula for is true, and do a few things with it. All this material was discovered by Gibbs in the late 1800’s, and it’s lurking any good book on statistical mechanics—but not phrased in the language of statistical manifolds. The physics textbooks usually consider special cases, like a box of gas where:

• is energy, is 1/temperature.

• is volume, is –pressure/temperature.

• is the number of particles, is chemical potential / pressure.

While these special cases are important and interesting, I’d rather be general!

Technical comments

I said “Any statistical manifold comes with a bunch of interesting geometrical structures”, but in fact some conditions are required. For example, the Fisher information metric is only well-defined and nondegenerate under some conditions on the map For example, if maps every point of to the same probability distribution, the Fisher information metric will vanish.

Similarly, the entropy function is only smooth under some conditions on

Furthermore, the integral

may not converge for all values of the numbers But in my discussion of Gibbsian statistical manifolds, I was assuming that an entropy-maximizing probability distribution with

actually exists. In this case the probability distribution is also unique (almost everywhere).

---

For all my old posts on information geometry, go here:

• Information geometry.

Information Geometry (Part 19)

Last time I figured out the analogue of momentum in probability theory, but I didn’t say what it’s called. Now I will tell you—thanks to some help from Abel Jansma and Toby Bartels.

SURPRISE: it’s called SURPRISAL!

This is a well-known concept in information theory. It’s also called ‘information content‘.

Let’s see why. First, let’s remember the setup. We have a manifold

whose points are nowhere vanishing probability distributions on the set We have a function

called the Shannon entropy, defined by

For each point we define a cotangent vector by

As mentioned last time, this is the analogue of momentum in probability theory. In the second half of this post I’ll say more about exactly why. But first let’s compute it and see what it actually equals!

Let’s start with a naive calculation, acting as if the probabilities were a coordinate system on the manifold We get

so using the definition of the Shannon entropy we have

Now, the quantity is called the surprisal of the probability distribution at Intuitively, it’s a measure of how surprised you should be if an event of probability occurs. For example, if you flip a fair coin and it lands heads up, your surprisal is ln 2. If you flip 100 fair coins and they all land heads up, your surprisal is 100 times ln 2.

Of course ‘surprise’ is a psychological term, not a term from math or physics, so we shouldn’t take it too seriously here. We can derive the concept of surprisal from three axioms:

1. The surprisal of an event of probability is some function of , say
2. The less probable an event is, the larger its surprisal is:
3. The surprisal of two independent events is the sum of their surprisals:

It follows from work on Cauchy’s functional equation that must be of this form:

for some constant We shall choose the base of our logarithms, to be We had a similar freedom of choice in defining the Shannon entropy, and we will use base for both to be consistent. If we chose something else, it would change the surprisal and the Shannon entropy by the same constant factor.

So far, so good. But what about the irksome “-1” in our formula?

Luckily it turns out we can just get rid of this! The reason is that the probabilities are not really coordinates on the manifold They’re not independent: they must sum to 1. So, when we change them a little, the sum of their changes must vanish. Putting it more technically, the tangent space is not all of but just the subspace consisting of vectors whose components sum to zero:

The cotangent space is the dual of the tangent space. The dual of a subspace

is the quotient space

The cotangent space thus consists of linear functionals modulo those that vanish on vectors obeying the equation

Of course, we can identify the dual of with in the usual way, using the Euclidean inner product: a vector corresponds to the linear functional

From this, you can see that a linear functional vanishes on all vectors obeying the equation

if and only if its corresponding vector has

So, we get

In words: we can describe cotangent vectors to as lists of n numbers if we want, but we have to remember that adding the same constant to each number in the list doesn’t change the cotangent vector!

This suggests that our naive formula

is on the right track, but we’re free to get rid of the constant 1 if we want! And that’s true.

To check this rigorously, we need to show

for all We compute:

where in the second to last step we used our earlier calculation:

and in the last step we used

Back to the big picture

Now let’s take stock of where we are. We can fill in the question marks in the charts from last time, and combine those charts while we’re at it.

	Classical Mechanics	Thermodynamics	Probability Theory
q	position	extensive variables	probabilities
p	momentum	intensive variables	surprisals
S	action	entropy	Shannon entropy

What’s going on here? In classical mechanics, action is minimized (or at least the system finds a critical point of the action). In thermodynamics, entropy is maximized. In the maximum entropy approach to probability, Shannon entropy is maximized. This leads to a mathematical analogy that’s quite precise. For classical mechanics and thermodynamics, I explained it here:

• Classical mechanics versus thermodynamics (part 1).

• Classical mechanics versus thermodynamics (part 2).

These posts may give a more approachable introduction to what I’m doing now: now I’m bringing probability theory into the analogy, with a big emphasis on symplectic and contact geometry.

Let me spell out a bit of the analogy more carefully:

Classical Mechanics. In classical mechanics, we have a manifold whose points are positions of a particle. There’s an important function on this manifold: Hamilton’s principal function

What’s this? It’s basically action: is the action of the least-action path from the position at some earlier time to the position at time 0. The Hamilton–Jacobi equations say the particle’s momentum at time 0 is given by

Thermodynamics. In thermodynamics, we have a manifold whose points are equilibrium states of a system. The coordinates of a point are called extensive variables. There’s an important function on this manifold: the entropy

There is a cotangent vector at the point given by

The components of this vector are the intensive variables corresponding to the extensive variables.

Probability Theory. In probability theory, we have a manifold whose points are nowhere vanishing probability distributions on a finite set. The coordinates of a point are probabilities. There’s an important function on this manifold: the Shannon entropy

There is a cotangent vector at the point given by

The components of this vector are the surprisals corresponding to the probabilities.

In all three cases, is a symplectic manifold and imposing the constraint picks out a Lagrangian submanifold

There is also a contact manifold where the extra dimension comes with an extra coordinate that means

• action in classical mechanics,
• entropy in thermodynamics, and
• Shannon entropy in probability theory.

We can then decree that along with and these constraints pick out a Legendrian submanifold

There’s a lot more to do with these ideas, and I’ll continue next time.

---

For all my old posts on information geometry, go here:

• Information geometry.

Information Geometry (Part 18)

Last time I sketched how two related forms of geometry, symplectic and contact geometry, show up in thermodynamics. Today I want to explain how they show up in probability theory.

For some reason I haven’t seen much discussion of this! But people should have looked into this. After all, statistical mechanics explains thermodynamics in terms of probability theory, so if some mathematical structure shows up in thermodynamics it should appear in statistical mechanics… and thus ultimately in probability theory.

I just figured out how this works for symplectic and contact geometry.

Suppose a system has possible states. We’ll call these microstates, following the tradition in statistical mechanics. If you don’t know what ‘microstate’ means, don’t worry about it! But the rough idea is that if you have a macroscopic system like a rock, the precise details of what its atoms are doing are described by a microstate, and many different microstates could be indistinguishable unless you look very carefully.

We’ll call the microstates So, if you don’t want to think about physics, when I say microstate I’ll just mean an integer from 1 to n.

Next, a probability distribution assigns a real number to each microstate, and these numbers must sum to 1 and be nonnegative. So, we have though not every vector in is a probability distribution.

I’m sure you’re wondering why I’m using rather than to stand for an observable instead of a probability distribution. Am I just trying to confuse you?

No: I’m trying to set up an analogy to physics!

Last time I introduced symplectic geometry using classical mechanics. The most important example of a symplectic manifold is the cotangent bundle of a manifold A point of is a pair consisting of a point and a cotangent vector In classical mechanics the point describes the position of some physical system, while describes its momentum.

So, I’m going to set up an analogy like this:

	Classical Mechanics	Probability Theory
	position	probability distribution
	momentum	???

But what is to momentum as probability is to position?

A big clue is the appearance of symplectic geometry in thermodynamics, which I also outlined last time. We can use this to get some intuition about the analogue of momentum in probability theory.

In thermodynamics, a system has a manifold of states. (These are not the ‘microstates’ I mentioned before: we’ll see the relation later.) There is a function

describing the entropy of the system as a function of its state. There is a law of thermodynamics saying that

This equation picks out a submanifold of namely

Moreover this submanifold is Lagrangian: the symplectic structure vanishes when restricted to it:

This is very beautiful, but it goes by so fast you might almost miss it! So let’s clutter it up a bit with coordinates. We often use local coordinates on and describe a point using these coordinates, getting a point

They give rise to local coordinates on the cotangent bundle The are called extensive variables, because they are typically things that you can measure only by totalling up something over the whole system, like the energy or volume of a cylinder of gas. The are called intensive variables, because they are typically things that you can measure locally at any point, like temperature or pressure.

In these local coordinates, the symplectic structure on is the 2-form given by

The equation

serves as a law of physics that determines the intensive variables given the extensive ones when our system is in thermodynamic equilibrium. Written out using coordinates, this law says

It looks pretty bland here, but in fact it gives formulas for the temperature and pressure of a gas, and many other useful formulas in thermodynamics.

Now we are ready to see how all this plays out in probability theory! We’ll get an analogy like this, which goes hand-in-hand with our earlier one:

	Thermodynamics	Probability Theory
	extensive variables	probability distribution
	intensive variables	???

This analogy is clearer than the last, because statistical mechanics reveals that the extensive variables in thermodynamics are really just summaries of probability distributions on microstates. Furthermore, both thermodynamics and probability theory have a concept of entropy.

So, let’s take our manifold to consist of probability distributions on the set of microstates I was talking about before: the set Actually, let’s use nowhere vanishing probability distributions:

I’m requiring to ensure is a manifold, and also to make sure is differentiable: it ceases to be differentiable when one of the probabilities hits zero.

Since is a manifold, its cotangent bundle is a symplectic manifold And here’s the good news: we have a god-given entropy function

namely the Shannon entropy

So, everything I just described about thermodynamics works in the setting of plain old probability theory! Starting from our manifold and the entropy function, we get all the rest, leading up to the Lagrangian submanifold

that describes the relation between extensive and intensive variables.

For computations it helps to pick coordinates on Since the probabilities sum to 1, they aren’t independent coordinates on So, we can either pick all but one of them as coordinates, or learn how to deal with non-independent coordinates, which are already completely standard in projective geometry. Let’s do the former, just to keep things simple.

These coordinates on give rise in the usual way to coordinates and on the cotangent bundle These play the role of extensive and intensive variables, respectively, and it should be very interesting to impose the equation

where is the Shannon entropy. This picks out a Lagrangian submanifold

So, the question becomes: what does this mean? If this formula gives the analogue of momentum for probability theory, what does this analogue of momentum mean?

Here’s a preliminary answer: says how fast entropy increases as we increase the probability that our system is in the ith microstate. So if we think of nature as ‘wanting’ to maximize entropy, the quantity says how eager it is to increase the probability

Indeed, you can think of as a bit like pressure—one of the most famous intensive quantities in thermodynamics. A gas ‘wants’ to expand, and its pressure says precisely how eager it is to expand. Similarly, a probability distribution ‘wants’ to flatten out, to maximize entropy, and says how eager it is to increase the probability in order to do this.

But what can we do with this concept? And what does symplectic geometry do for probability theory?

I will start tackling these questions next time.

One thing I’ll show is that when we reduce thermodynamics to probability theory using the ideas of statistical mechanics, the appearance of symplectic geometry in thermodynamics follows from its appearance in probability theory.

Another thing I want to investigate is how other geometrical structures on the space of probability distributions, like the Fisher information metric, interact with the symplectic structure on its cotangent bundle. This will integrate symplectic geometry and information geometry.

I also want to bring contact geometry into the picture. It’s already easy to see from our work last time how this should go. We treat the entropy as an independent variable, and replace with a larger manifold having as an extra coordinate. This is a contact manifold with contact form

This contact manifold has a submanifold where we remember that entropy is a function of the probability distribution and define in terms of as usual:

And as we saw last time, is a Legendrian submanifold, meaning

But again, we want to understand what these ideas from contact geometry really do for probability theory!

---

For all my old posts on information geometry, go here:

• Information geometry.

Information Geometry (Part 17)

I’m getting back into information geometry, which is the geometry of the space of probability distributions, studied using tools from information theory. I’ve written a bunch about it already, which you can see here:

• Information geometry.

Now I’m fascinated by something new: how symplectic geometry and contact geometry show up in information geometry. But before I say anything about this, let me say a bit about how they show up in thermodynamics. This is more widely discussed, and it’s a good starting point.

Symplectic geometry was born as the geometry of phase space in classical mechanics: that is, the space of possible positions and momenta of a classical system. The simplest example of a symplectic manifold is the vector space with n position coordinates and n momentum coordinates

It turns out that symplectic manifolds are always even-dimensional, because we can always cover them with coordinate charts that look like When we change coordinates, it turns out that the splitting of coordinates into positions and momenta is somewhat arbitrary. For example, the position of a rock on a spring now may determine its momentum a while later, and vice versa. What’s not arbitrary? It’s the so-called ‘symplectic structure’:

While far from obvious at first, we know by now that the symplectic structure is exactly what needs to be preserved under valid changes of coordinates in classical mechanics! In fact, we can develop the whole formalism of classical mechanics starting from a manifold with a symplectic structure.

Symplectic geometry also shows up in thermodynamics. In thermodynamics we can start with a system in equilibrium whose state is described by some variables Its entropy will be a function of these variables, say

We can then take the partial derivatives of entropy and call them something:

These new variables are said to be ‘conjugate’ to the and they turn out to be very interesting. For example, if is energy then is ‘coolness’: the reciprocal of temperature. The coolness of a system is its change in entropy per change in energy.

Often the variables are ‘extensive’: that is, you can measure them only by looking at your whole system and totaling up some quantity. Examples are energy and volume. Then the new variables are ‘intensive’: that is, you can measure them at any one location in your system. Examples are coolness and pressure.

Now for a twist: sometimes we do not know the function ahead of time. Then we cannot define the as above. We’re forced into a different approach where we treat them as independent quantities, at least until someone tells us what is.

In this approach, we start with a space having n coordinates called and n coordinates called This is a symplectic manifold, with the symplectic struture described earlier!

But what about the entropy? We don’t yet know what it is as a function of the but we may still want to talk about it. So, we build a space having one extra coordinate in addition to the and This new coordinate stands for entropy. And this new space has an important 1-form on it:

This is called the ‘contact 1-form’.

This makes into an example of a ‘contact manifold’. Contact geometry is the odd-dimensional partner of symplectic geometry. Just as symplectic manifolds are always even-dimensional, contact manifolds are always odd-dimensional.

What is the point of the contact 1-form? Well, suppose someone tells us the function relating entropy to the coordinates Now we know that we want

and also

So, we can impose these equations, which pick out a subset of You can check that this subset, say is an n-dimensional submanifold. But even better, the contact 1-form vanishes when restricted to this submanifold:

Let’s see why! Suppose and suppose is a vector tangent to at this point . It suffices to show

Using the definition of this equation says

But on the surface we have

So, the equation we’re trying to show can be written as

But this follows from

which holds because is a function only of the coordinates

So, any formula for entropy picks out a so-called ‘Legendrian submanifold’ of that is, an n-dimensional submanifold such that the contact 1-form vanishes when restricted to this submanifold. And the idea is that this submanifold tells you everything you need to know about a thermodynamic system.

Indeed, V. I. Arnol’d says this was implicitly known to the great founder of statistical mechanics, Josiah Willard Gibbs. Arnol’d calls with coordinates energy, entropy, temperature, pressure and volume the ‘Gibbs manifold’, and he proclaims:

Gibbs’ thesis: substances are Legendrian submanifolds of the Gibbs manifold.

This is from here:

• V. I. Arnol’d, Contact geometry: the geometrical method of Gibbs’ thermodynamics, Proceedings of the Gibbs Symposium (New Haven, CT, 1989), AMS, Providence, Rhode Island, 1990.

A bit more detail

Now I want to say everything again, with a bit of extra detail, assuming more familiarity with manifolds. Above I was using with coordinates to describe the ‘extensive’ variables of a thermodynamic system. But let’s be a bit more general and use any smooth n-dimensional manifold Even if is a vector space, this viewpoint is nice because it’s manifestly coordinate-independent!

So: starting from we build the cotangent bundle A point in cotangent describes both extensive variables, namely and ‘intensive’ variables, namely a cotangent vector

The manifold has a 1-form on it called the tautological 1-form. We can describe it as follows. Given a tangent vector we have to say what is. Using the projection

we can project down to a tangent vector at the point . But the 1-form eats tangent vectors at and spits out numbers! So, we set

This is sort of mind-boggling at first, but it’s worth pondering until it makes sense. It helps to work out what looks like in local coordinates. Starting with any local coordinates on an open set of we get local coordinates on the cotangent bundle of this open set in the usual way. On this open set you then get

This is a standard calculation, which is really worth doing!

It follows that we can define a symplectic structure by

and get this formula in local coordinates:

Now, suppose we choose a smooth function

which describes the entropy. We get a 1-form , which we can think of as a map

assigning to each choice of extensive variables the pair of extensive and intensive variables where

The image of the map is a ‘Lagrangian submanifold‘ of that is, an n-dimensional submanifold such that

Lagrangian submanifolds are to symplectic geometry as Legendrian submanifolds are to contact geometry! What we’re seeing here is that if Gibbs had preferred symplectic geometry, he could have described substances as Lagrangian submanifolds rather than Legendrian submanifolds. But this approach would only keep track of the derivatives of entropy, not the actual value of the entropy function

If we prefer to keep track of the actual value of using contact geometry, we can do that. For this we add an extra dimension to and form the manifold The extra dimension represents entropy, so we’ll use as our name for the coordinate on

We can make into a contact manifold with contact 1-form

In local coordinates we get

just as we had earlier. And just as before, if we choose a smooth function describing entropy, the subset

is a Legendrian submanifold of

Okay, this concludes my lightning review of symplectic and contact geometry in thermodynamics! Next time I’ll talk about something a bit less well understood: how they show up in statistical mechanics.