Compositional Thermostatics

At the Topos Institute this summer, a group of folks started talking about thermodynamics and category theory. It probably started because Spencer Breiner and my former student Joe Moeller, both working at NIST, were talking about thermodynamics with some people there. But I’ve been interested in thermodynamics for quite a while now –and Owen Lynch, a grad student visiting from the University of Utrecht, wanted to do his master’s thesis on the subject. He’s now working with me. Sophie Libkind, David Spivak and David Jaz Myers also joined in: they’re especially interested in open systems and how they interact.

Prompted by these conversations, a subset of us eventually wrote a paper on the foundations of equilibrium thermodynamics:

• John Baez, Owen Lynch and Joe Moeller, Compositional thermostatics.

The idea here is to describe classical thermodynamics, classical statistical mechanics and quantum statistical mechanics in a unified framework based on entropy maximization. This framework can also handle ‘generalized probabilistic theories’ of the sort studied in quantum foundations—that is, theories like quantum mechanics, but more general.

To unify all these theories, we define a ‘thermostatic system’ to be any convex space of ‘states’ together with a concave function

assigning to each state an ‘entropy’.

Whenever several such systems are combined and allowed to come to equilibrium, the new equilibrium state maximizes the total entropy subject to constraints. We explain how to express this idea using an operad. Intuitively speaking, the operad we construct has as operations all possible ways of combining thermostatic systems. For example, there is an operation that combines two gases in such a way that they can exchange energy and volume, but not particles—and another operation that lets them exchange only particles, and so on.

It is crucial to use a sufficiently general concept of ‘convex space’, which need not be a convex subset of a vector space. Luckily there has been a lot of work on this, so we can just grab a good definition off the shelf:

Definition. A convex space is a set with an operation for each such that the following identities hold:

1)

2)

3)

4) for all satisfying and .

To understand these axioms, especially the last, you need to check that any vector space is a convex space with

So, these operations describe ‘convex linear combinations’.

Indeed, any subset of a vector space closed under convex linear combinations is a convex space! But there are other examples too.

In 1949, the famous mathematician Marshall Stone invented ‘barycentric algebras’. These are convex spaces satisfying one extra axiom: the cancellation axiom, which says that whenever

He proved that any barycentric algebra is isomorphic to a convex subset of a vector space. Later Walter Neumann noted that a convex space, defined as above, is isomorphic to a convex subset of a vector space if and only if the cancellation axiom holds.

Dropping the cancellation axiom has convenient formal consequences, since the resulting more general convex spaces can then be defined as algebras of a finitary commutative monad, giving the category of convex spaces very good properties.

But dropping this axiom is no mere formal nicety. In our definition of ‘thermostatic system’, we need the set of possible values of entropy to be a convex space. One obvious candidate is the set However, for a well-behaved formalism based on entropy maximization, we want the supremum of any set of entropies to be well-defined. This forces us to consider the larger set which does not obey the cancellation axiom.

But even that is not good enough! In thermodynamics you often read about the ‘heat bath‘, an idealized system that can absorb or emit an arbitrarily large amount of energy while keeping a fixed temperature. We want to treat the ‘heat bath’ as a thermostatic system on an equal footing with any other. To do this, we need to allow consider negative entropies—not because the heat bath can have negative entropy, but because it acts as an infinite reservoir of entropy, and the change in entropy from its default state can be positive or negative.

This suggests letting entropies take values in the convex space But then the requirement that any set of entropies have a supremum (including empty and unbounded sets) forces us to use the larger convex space

This does not obey the cancellation axiom, so there is no way to think of it as a convex subset of a vector space. In fact, it’s not even immediately obvious how to make it into a convex space at all! After all, what do you get when you take a nontrivial convex linear combination of and You’ll have to read our paper for the answer, and the justification.

We then define a thermostatic system to be a convex set together with a concave function

where concave means that

We give lots of examples from classical thermodynamics, classical and quantum statistical mechanics, and beyond—including our friend the ‘heat bath’.

For example, suppose is the set of probability distributions on an -element set, and suppose is the Shannon entropy

Then given two probability distributions and we have

for all So this entropy function is convex, and defines a thermostatic system. But in this example the entropy only takes nonnegative values, and is never infinite, so you need to look at other examples to see why we want to let entropy take values in

After looking at examples of thermostatic systems, we define an operad whose operations are convex-linear relations from a product of convex spaces to a single convex space. And then we prove that thermostatic systems give an algebra for this operad: that is, we can really stick together thermostatic systems in all these ways. The trick is computing the entropy function of the new composed system from the entropy functions of its parts: this is where entropy maximization comes in.

For a nice introduction to these ideas, check out Owen’s blog article:

• Owen Lynch, Compositional thermostatics, Topos Institute Blog, 9 September 2021.

And then comes the really interesting part: checking that this adequately captures many of the examples physicists have thought about!

The picture at the top of this post shows one that we discuss: two cylinders of ideal gas with a movable divider between them that’s permeable to heat. Yes, this is an operation in an operad—and if you tell us the entropy function of each cylinder of gas, our formalism will automatically compute the entropy function of the resulting combination of these two cylinders.

There are many other examples. Did you ever hear of the ‘canonical ensemble’, the ‘microcanonical ensemble’, or the ‘grand canonical ensemble’? Those are famous things in statistical mechanics. We show how our formalism recovers these.

I’m sure there’s much more to be done. But I feel happy to see modern math being put to good use: making the foundations of thermodynamics more precise. Once Vladimir Arnol’d wrote:

Every mathematician knows that it is impossible to understand any elementary course in thermodynamics.

I’m not sure our work will help with that—and indeed, it’s possible that once the mathematicians finally understand thermodynamics, physicists won’t understand what the mathematicians are talking about! But at least we’re clearly seeing some more of the mathematical structures that are hinted at, but not fully spelled out, in such an elementary course.

I expect that our work will interact nicely with Simon Willerton’s work on the Legendre transform. The Legendre transform of a concave (or convex) function is widely used in thermostatics, and Simon describes this for functions valued in using enriched profunctors:

• Simon Willerton, Enrichment and the Legendre–Fenchel transform I, The n-Category Café, April 16, 2014.

• Simon Willerton, Enrichment and the Legendre–Fenchel transform II, The n-Category Café, May 22, 2014.

He also has a paper on this, and you can see him talk about it on YouTube.

The Kuramoto–Sivashinsky Equation (Part 2)

I love the Kuramoto–Sivashinsky equation, beautifully depicted here by Thien An, because it’s one of the simplest partial differential equations that displays both chaos and a visible ‘arrow of time’. Last time I made some conjectures about it. Most notably, I conjecture that the ‘stripes’ you see above can appear or merge as time passes, but never disappear or split.

But I was quite confused for a while, because the Kuramoto–Sivashinsky equation comes in two forms. The integral form, which I spent most of my time discussing, is this:

where is a real function of two real variables, ‘time’ and ‘space’ The derivative form is this:

where is again a real function of two real variables. If is a solution of the integral form,

is a solution of the derivative form. This is easy to see: just take the derivative of the equation with in it and see what you get.

By the way, beware of this: Thien An’s wonderful movie above shows the integral form of the equation, but she calls the function

Note that if has a local maximum as a function of then will go positive and then negative. This is important because it affects the definition of ‘stripe’. Here’s what stripes look in the derivative form of the Kuramoto–Sivashinsky equation, as computed by Steve Huntsman:

As you can see, the function goes negative and then positive as we increase moving through a stripe. This means that would have a local minimum.

Another thing puzzling me was that Steve Huntsman found solutions of the derivative form where the stripes in mostly move in the same direction as time passes:

This suggests that there’s some way to take a solution where the stripes aren’t moving much, and switch to a moving frame of reference to get a solution with moving stripes. And indeed I’ll show that’s true! But I’ll also show it’s not true for the integral form of the Kuramoto–Sivashinsky equation—at least not with periodic boundary conditions, which is what we’re using here.

Galilean transformations

Galileo was the guy who first emphasized that the laws of physics look the same in a moving frame of reference, but later Maxwell explained the idea very poetically:

Our whole progress up to this point may be described as a gradual development of the doctrine of relativity of all physical phenomena. Position we must evidently acknowledge to be relative, for we cannot describe the position of a body in any terms which do not express relation… There are no landmarks in space; one portion of space is exactly like every other portion, so that we cannot tell where we are. We are, as it were, on an unruffled sea, without stars, compass, sounding, wind or tide, and we cannot tell in which direction we are going. We have no log which we can case out to take a dead reckoning by; we may compute our rate of motion with respect to the neighboring bodies, but we do not know how these bodies may be moving in space.

Before Einstein came along, a transformation into a moving frame of reference worked like this:

where is a constant, the velocity.

These are called ‘Galilean transformations’ And here’s something cool: the derivative form of the Kuramoto–Sivashinsky equation has Galilean transformations as symmetries! If is a solution, so is the boosted function

The prime here does not mean derivative: is the name of our new boosted solution. To get this boosted solution, we do the obvious coordinate transformation into a moving frame of reference, but then a bit more: we must subtract the constant

Let’s see how this works! Suppose is a solution of the derivative form of the Kuramoto–Sivashinsky equation:

Then defining

we see the derivative of has an extra term:

while its derivatives are simple:

and so on for the higher derivatives.

This lets us check that the boosted function is also a solution:

Note how subtracting the constant when we transform exactly cancels the extra term term we get when we transform the time derivative of

I got this idea from here:

• Uriel Frisch, Zhensu She and Olivier Thual, Viscoelastic behaviour of cellular solutions to the Kuramoto–Sivashinsky model, Journal of Fluid Mechanics 168 (1986), 221–240.

thanks to someone on Twitter.

Does the integral form of the Kuramoto-Sivashinsky equation also have Galilean transformations as symmetries? Yes, but there’s a certain problem with them. Let’s see how it goes.

If is a solution, then so is this boosted function:

But now the ‘fudge factors’ in the boosted function are more sneaky! We don’t just subtract a constant, as we did with the derivative form. We subtract a function that depends on both the space and time coordinates.

Let’s check. The first time derivative of works like this:

The first space derivative works like this:

The second space derivative is simpler:

and the higher space derivatives work the same way. Now suppose was a solution of the integral form of the Kuramoto-Sivashinsky equation:

We can use our formulas to check that is a solution:

The cancellations in the second step rely crucially on those sneaky fudge factors in the definition of the boosted solution I chose those fudge factors precisely to make these cancellations happen.

Periodic boundary conditions

A lot of really interesting results about the Kuramoto-Sivashinsky equation involve looking at solutions that are periodic in space. For the derivative form this means

It’s easy to see that if obeys this equation, so does its boosted version:

After all, a quick calculation shows

So for any spatially periodic solution with some stripes that are basically standing still, there’s a spatially periodic boosted version where they’re moving at velocity You can think of a spatially periodic solution as a function on the cylinder. In the boosted version, the stripes spiral around this cylinder like the stripes on a barber pole!

But for the integral form this doesn’t work! Suppose is a solution of the integral form that is periodic in space:

Now its boosted version is defined by

and this is not periodic in space:

A conserved quantity

The derivative form of the Kuramoto–Shivashinsky equation has an unexpected symmetry under Galilean transformations, or boosts, even for periodic solutions. It also has an interesting conserved quantity, namely

To see this, note that

Note that if we boost a solution, replacing by we also subtract a constant from this conserved quantity, namely So, it seems that the conserved quantity

is a way of measuring ‘how much the stripes are moving, on average’.

I’ll venture a guess that when

the stripes are stationary, on average. One reason I’m willing to guess this is that this equation has another meaning, too.

I mentioned that we can get solutions of the derivative form from solutions of the integral form by differentiating them, like this:

Does every solution of the derivative form arise this way from a solution of the integral form? Given we can define by

and we can check it obeys the integral form of the Kuramoto–Sivashinsky equation. I’ve been showing you lots of calculations and we’re probably both getting tired of them, so I won’t actually go through this check. But here’s the interesting part. Suppose we restrict attention to spatially periodic solutions! If is spatially periodic, will be so iff

since we have

where in the last step, the first integral is by definition and the second is zero because

and is periodic with period

Summary

Limiting ourselves to spatially periodic solutions, we see:

• The derivative form of the Kuramoto–Sivashinsky equation has symmetry under boosts.

• The integral form does not.

• Solutions of the integral form correspond precisely to solutions of the derivative form with

• Boosting a solution of the derivative form with

gives a solution where this quantity is not zero.

Given all this, I think the good theorems about spatially periodic solutions of the Kuramoto–Sivashinsky equation will focus on solutions of the integral form, or equivalently solutions of the differential form with

Epilogue: symmetries of the heat equation

At first I was shocked that the Kuramoto–Sivashinsky equation had Galilean transformations as symmetries, because it’s a relative of the heat equation, and the heat equation obviously doesn’t have Galilean transformations as symmetries: you can’t get a solution of the heat equation where a wave of heat moves along at constant velocity as it spreads out.

But then I remembered that the heat equation

is very similar to Schrödinger’s equation:

And Schrödinger’s equation does have Galilean transformations as symmetries, since it describes a free particle! Physicists know how they work. You don’t just replace with you also multiply it by a suitable complex phase that depends on and . That is, you multiply by the exponential of some imaginary-valued function of and

So then I realized that the heat equation does have Galilean symmetries! They work just as for Schrödinger’s equation, but now you have to multiply by the exponential of some real-valued function of and

This grows exponentially as you move in one direction in space, so people thinking about heat often don’t think about such solutions. Yes, you can get a ‘wave of heat that moves along at constant velocity as it spreads out’, but it’s like a huge tsunami wave rolling in from infinity!

I was very proud of myself for having discovered this weird Galilean invariance of the heat equation, and I posed it as a puzzle on Twitter. But then Tom Price pointed out a paper about it:

• U. Niederer, Schrödinger invariant generalized heat equations, Helvetica Physica Acta 51 (1978), 220–239.

It turns out the Galilean invariance of the heat equation, and much more, was discovered by Sophus Lie in 1882!

The so-called ‘Schrödinger group’ also includes certain fractional linear transformations of the plane. So this raises the question of whether the Kuramoto–Sivashinsky equation has even more symmetries than those in the Galilei group (translations, spatial reflections and Galilean transformations).

But it seems this too has been studied:

• Mehdi Nadjafikhah and Fatemeh Ahangari, Lie symmetry analysis of the two-dimensional generalized Kuramoto–Sivashinsky equation, Mathematical Sciences 6 (2012).

This discusses the analogue of the Kuramoto–Sivashinsky equation in two dimensions of space and one of time. It uses some techniques to compute its symmetry group, but I don’t see any sign of invariance under a big group like the ‘Schrödinger group’.

The Kuramoto–Sivashinsky Equation (Part 1)

I love this movie showing a solution of the Kuramoto–Sivashinsky equation, made by Thien An. If you haven’t seen her great math images on Twitter, check them out!

I hadn’t known about this equation, and it looked completely crazy to me at first. But it turns out to be important, because it’s one of the simplest \partial differential equations that exhibits chaotic behavior.

As the image scrolls to the left, you’re seeing how a real-valued function of two real variables changes with the passage of time. The vertical direction is ‘space’, while the horizontal direction is time,

Near the end of this post I’ll make some conjectures about the Kuramoto–Sivashinsky equation. The first one is very simple: as time passes, stripes appear and merge, but they never disappear or split.

The behavior of these stripes makes the Kuramoto–Sivashinsky equation an excellent playground for thinking about how differential equations can describe ‘things’ with some individuality, even though their solutions are just smooth functions. But to test my conjectures, I could really use help from people who are good at numerical computation or creating mathematical images!

First let me review some known stuff. You can skip this and go straight to the conjectures if you want, but some terms might not make sense.

Review

For starters, note that these stripes seem to appear out of nowhere. That’s because this system is chaotic: small ripples get amplified. This is especially true of ripples with a certain wavelength: roughly , as we’ll see later.

And yet while solutions of the Kuramoto–Sivanshinsky equation are chaotic, they have a certain repetitive character. That is, they don’t do completely novel things; they seem to keep doing the same general sort of thing. The world this equation describes has an arrow of time, but it’s ultimately rather boring compared to ours.

The reason is that all smooth solutions of the Kuramoto–Sivanshinsky equation quickly approach a certain finite-dimensional manifold of solutions, called an ‘inertial manifold’. The dynamics on the inertial manifold is chaotic. And sitting inside it is a set called an ‘attractor’, which all solutions approach. This attractor is probably a fractal. This attractor describes the complete repertoire of what you’ll see solutions do if you wait a long time.

Some mathematicians have put a lot of work into proving these things, but let’s see how much we can understand without doing anything too hard.

Written out with a bit less jargon, the Kuramoto–Sivashinky equation says

or in more compressed notation,

To understand it, first remember the heat equation:

This describes how heat spreads out. That is: if is the temperature of an iron rod at position at time , the heat equation describes how this temperature function flattens out as time passes and heat spreads.

But the Kuramoto–Sivashinsky equation more closely resembles the time-reversed heat equation

This equation describes how, running a movie of a hot iron rod backward, heat tends to bunch up rather than smear out! Small regions of different temperature, either hotter or colder than their surroundings, will tend to amplify.

This accounts for the chaotic behavior of the Kuramoto–Sivashinsky equation: small stripes emerge as if out of thin air and then grow larger. But what keeps these stripes from growing uncontrollably?

The next term in the equation helps. If we have

then very sharp spikes in tend to get damped out exponentially.

To see this, it helps to bring in a bit more muscle: Fourier series. We can easily solve the heat equation if our iron rod is the interval and we demand that its temperature is the same at both ends:

This lets us write the temperature function in terms of the functions like this:

for some functions . Then the heat equation gives

and we can easily solve these equations and get

and thus

So, each function decays exponentially as time goes on, and the so-called ‘high-frequency modes’, with big, get damped really fast due to that factor. This is why heat smears out as time goes on.

If we solve the time-reversed heat equation the same way we get

so now high-frequency modes get exponentially amplified. The time-reversed heat equation is a very unstable: if you change the initial data a little bit by adding a small amount of some high-frequency function, it will make an enormous difference as time goes by.

What keeps things from going completely out of control? The next term in the equation helps:

This is still linear so we can still solve it using Fourier series. Now we get

Since , none of the modes grows exponentially. In fact, all the modes decay exponentially except for three: . These will be constant in time. So, any solution will approach a constant as time goes on!

We can make the story more interesting if we don’t require our rod to have length . Say it has length . We can write periodic functions on the interval as linear combinations of functions where now the frequencies aren’t integers: instead

for integers . The longer our rod, the lower these frequencies can be. The rest of the math works almost the same: we get

but we have to remember . The modes with will grow exponentially, while the rest will decay exponentially or stay constant. Note that only for . So, modes with these frequencies grow exponentially. Modes with decay exponentially.

If , all the frequencies are integers times , which is bigger than , so no modes grow exponentially—and indeed all solutions approach a constant! But as you look at longer and longer rods, you get more and more modes that grow exponentially. The number of these will be roughly proportional to , though they will ‘pop into existence’ at certain specific values of .

Which exponentially growing modes grow the fastest? These are the ones that make as large as possible, so they happen near where

namely . The wavelength of a mode is , so these fastest-growing modes have wavelength close to .

In short, our equation has a certain length scale where the instability is most pronounced: temperature waves with about this wavelength grow fastest.

All this is very easy to work out in as much detail as we want, because our equation so far is linear. The full-fledged Kuramoto–Sivashinsky equation

is a lot harder. And yet some features of the linear version remain, which is why it was worth spending time on that version.

For example, I believe the stripes we see in the movie above have width roughly . Stripes of roughly this width tend to get amplified. Why don’t they keep on growing taller forever? Apparently the nonlinear term prevents it. But this is not obvious. Indeed, it’s conjectured that if you solve the Kuramoto–Sivashinsky equation starting with a bounded smooth function , the solution will remain bounded by a constant. But this has not been proved—or at least it was not proved as of 2000, when this very nice article was written:

• Encyclopedia of Mathematics, Kuramoto–Sivashinsky equation.

The inertial manifold

The most fascinating fact about the Kuramoto–Sivashinsky equation is that for any fixed length , it has a finite-dimensional manifold of solutions such that every solution approaches one of these, exponentially fast! So, while this equation really describes an infinite-dimensional dynamical system, as its solutions move closer and closer to the solutions of some finite-dimensional dynamical system. This finite-dimensional system contains all the information about the patterns we’re seeing in Thien An’s movie.

As I mentioned, the manifold is called an ‘inertial manifold’. This is a general concept in dynamical systems theory:

• Wikipedia, Inertial manifold.

To make these ideas precise we need to choose a notion of distance between two solutions at a given time. A good choice uses the norm for periodic functions on :

Functions on with finite norm form a Hilbert space called . If we start with any function in this Hilbert space we get a solution of the Kuramoto–Sivashinsky equation such that the function is in this Hilbert space at all later times . Furthermore, this function is smooth, even analytic, for all later times:

• P. Collet, J.-P. Eckmann, H. Epstein and J. Stubbe, Analyticity for the Kuramoto–Sivashinsky equation, Physica D 67 (1993), 321–326.

This smoothing property is well-known for the heat equation, but it’s much less obvious here!

This work also shows that the Kuramoto–Sivashinsky equation defines a dynamical system on the Hilbert space . And based on earlier work by other mathematicians, Temam and Wang have heroically shown that this Hilbert space contains an inertial manifold of dimension bounded by some constant times

• Roger Temam and Xiao Ming Wang, Estimates on the lowest dimension of inertial manifolds for the Kuramoto-Sivashinsky equation in the general case, Differential and Integral Equations 7 (1994), 1095–1108.

I conjecture that in reality its dimension grows roughly linearly with . Why? We’ve just seen this is true for the linearized version of the Kuramoto–Sivashinsky equation: all modes with frequency get damped exponentially, but since there’s one mode for each integer , and , these modes correspond to integers with . So, there are of these modes. In short, for the linearized Kuramoto–Sivashinsky equation the inertial manifold has dimension about .

This evidence is rather weak, since it completely ignores the nonlinearity of the Kuramoto–Sivashinsky equation. I would not be shocked if the dimension of the inertial manifold grew at some other rate than linearly with .

Sitting inside the inertial manifold is an attractor, the smallest set that all solutions approach. This is probably a fractal, since that’s true of many chaotic systems. So besides trying to estimate the dimension of the inertial manifold, which is an integer we should try to estimate the dimension of this attractor, which may not be an integer!

There have been some nice numerical experiments studying solutions of the Kuramoto–Sivashinsky equation for various values of , seeing how they get more complicated as increases. For small , every solution approaches a constant, just as in the linearized version. For larger we get periodic solutions, and as continues to increase we get period doubling and finally chaos—a typical behavior for dynamical systems. But that’s just the start. For details, read this:

• Demetrios T. Papageorgiou and Yiorgos S. Smyrlis, The route to chaos for the Kuramoto–Sivashinsky equation, Theoretical and Computational Fluid Dynamics 3 (1991), 15–42.

I’ll warn you that they use a slightly different formalism. Instead of changing the length , they keep it equal to and change the equation, like this:

for some number they call the ‘viscosity’. It’s just a different way of describing the same business, so if I had more energy I could figure out the relation between and and tell you at which length chaos first kicks in. But I won’t now. Instead, I want to make some conjectures.

Conjectures

There should be some fairly well-defined notion of a ‘stripe’ for the Kuramoto–Sivashinsky equations: you can see the stripes form and merge here, and if we can define them, we can count them and say precisely when they’re born and when they merge:

For now I will define a ‘stripe’ as follows. At any time, a solution of the Kuramoto–Sivashinsky gives a periodic function on the interval We can think of this as a function on the circle. A stripe will be a maximal closed subinterval of the circle on which This definition depends on a constant and it’s up to you to pick a value of the constant that makes my conjectures true—or at least, almost true!

So, here are the conjectures:

First, I conjecture that if is large enough, almost every non-negative solution in the inertial manifold has a finite number of stripes at any time and that while they can appear and merge as we increase they can never split or disappear.

(Here ‘almost every’ is in the usual sense of measure theory. There are certainly solutions of the Kuramoto–Sivashinsky equation that don’t have stripes that appear and merge, like constant solutions. These solutions may lie on the inertial manifold, but I’m claiming they are rare.)

I also conjecture that the time-averaged number of stripes is asymptotically proportional to as for almost every nonnegative solution on the inertial manifold. The constant of proportionality shouldn’t depend on the solution we pick, except for solutions in some set of measure zero. It will, however, depend on our precise definition of ‘stripe’, e.g. our choice of the constant

I also conjecture that there’s a well-defined time average of the rate at which new stripes form, which is also asymptotically proportional to and independent of which solution we pick, except for solutions in a set of measure zero.

I also conjecture that this rate equals the time-averaged rate at which stripes merge, while the time-averaged rate at which stripes disappear or split is zero.

These conjectures are rather bold, but of course there are various fallback positions if they fail.

How can we test these conjectures? It’s hard to explicitly describe solutions that are actually on the inertial manifold, but by definition, any solution keeps getting closer to the inertial manifold at an exponential rate. Thus, it should behave similarly to solutions that are on the inertial manifold, after we wait long enough. So, I’ll conjecture that the above properties hold not only for almost every solution on the inertial manifold, but for typical solutions that start near the inertial manifold… as long as we wait long enough when doing our time averages.

If you feel like working on this, here are some things I’d really like:

• Images like Thien An’s but with various choices of To create these, maybe start with

and run it for long enough to ‘settle down’—that is, get near the inertial manifold.

• A time-averaged count of the average number of stripes for various choices of I’m conjecturing that this is asymptotically proportional to for large

• Time-averaged counts of the rates at which stripes are born, merge, split, and die—again for various choices of I’m conjecturing that the first two are asymptotically proportional to for large and that they’re equal. I’m conjecturing that the last two are zero, or tiny.

If someone gets into this, maybe we could submit a short paper to Experimental Mathematics. I’ve been browsing papers on the Kuramoto–Sivashinsky equations, and I haven’t yet seen anything that gets into as much detail on what solutions look like as I’m trying to do here.

The arrow of time

One more thing. I forgot to emphasize that the dynamical system on the Hilbert space is not reversible: we can evolve a solution forwards in time and it will stay in this Hilbert space, but not backwards in time. This is very well-known for the heat equation; the point is that solutions get smoother as we run them forward, but when we run them backward they typically get more wild and eventually their norm blows up.

What makes this especially interesting is that the dynamical system on the inertial manifold probably is reversible. As long as this manifold is compact, it must be: any smooth vector field on a compact manifold generates a ‘flow’ that you can run forward or backward in time.

And yet, even if this flow is reversible, as I suspect it is, it doesn’t resemble its time-reversed version! It has an ‘arrow of time’ built in, since bumps are born and merge much more often than they merge and split.

So, if my guesses are right, the inertial manifold for the Kuramoto–Sivashinsky equation describes a deterministic universe where time evolution is reversible—and yet the future doesn’t look like the past, because the dynamics carry the imprint of the irreversible dynamics of the Kuramoto–Sivashinsky equation on the larger Hilbert space of all solutions.

A warning

If you want to help me, the following may be useful. I believe the stripes are ‘bumps’, that is, regions where for some positive constant That would make them easy to define. I was shocked when Steve Huntsman did some calculations and produced a picture showing a solution where

Here the stripes are not mere bumps: they are regions where, as we increase , the solution first becomes large and negative and then becomes large and positive!

After massive confusion I realized that Steve was using some MATLAB code adapted from this website:

• Mathab Lak, Test case for PDEs: Kuramoto–Sivashinksy, Crank-Nicolson/Adams-Bashforth (CNAB2) timestepping.

and this code solves a different version of the Kuramoto–Sivashinksy equations, the so-called derivative form:

If satisfies the integral form of the Kuramoto–Sivashinksy equation, which is the one I’ve been talking about all along:

then its derivative

satisfies the derivative form.

So, the two equations are related, but you have to be careful because some of their properties are quite different! For the integral form, the cross-section of a typical stripe looks very roughly like this:

but for the derivative form it looks more like this:

You can grab Steve Huntsman’s MATLAB code here, but beware: this program solves the derivative form! Many of Steve’s pictures in comments to this blog—indeed, all of them so far—show solutions of the derivative form. In particular, the phenomenon he discovered of stripes tending to move at a constant nonzero velocity seems to be special to the derivative form of the Kuramoto–Sivashinsky equation. I’ll say more about this next time.

Nonequilibrium Thermodynamics in Biology

William Cannon and I are organizing a special session on thermodynamics in biology at the American Physical Society March Meeting, which will be held in Chicago on March 14–18, 2022.

If you work on this, please submit an abstract here before October 22! Our session number is 03.01.32.

Non-equilibrium Thermodynamics in Biology: from Chemical Reaction Networks to Natural Selection

Since Lotka, physical scientists have argued that living things belong to a class of complex and orderly systems that exist not despite the second law of thermodynamics, but because of it. Life and evolution, through natural selection of dissipative structures, are based on non-equilibrium thermodynamics. The challenge is to develop an understanding of what the respective physical laws can tell us about flows of energy and matter in living systems, and about growth, death and selection. This session will address current challenges including understanding emergence, regulation and control across scales, and entropy production, from metabolism in microbes to evolving ecosystems.

We have some speakers lined up already. Eric Smith of the Santa Fe Institute will speak on “Combinatorics in evolution: from rule-based systems to the thermodynamics of selectivities”. David Sivak of Simon Fraser University will speak on “”Nonequilibrium energy and information flows in autonomous systems”.

If you have any questions, please ask.

Classical Mechanics versus Thermodynamics (Part 4)

Thanks to the precise mathematical analogy between classical mechanics and thermodynamics that we saw last time, we can define Poisson brackets in thermodynamics just as in classical mechanics. That is, we can define Poisson brackets of quantities like entropy, temperature, volume and pressure. But what does this mean?

Furthermore, we can get quantum mechanics from classical mechanics by replacing Poisson brackets by commutators. That’s a brief summary of a long story, but by now this story is pretty well understood. So, we can do the same maneuver in thermodynamics and get some new theory where thermodynamic quantities like entropy, temperature, volume and pressure no longer commute! But what does this mean physically?

The obvious guess is that we get some version of thermodynamics that takes quantum mechanics into account, but I believe this is wrong. I want to explain why, and what I think is really going on. Briefly, I think what we get is more like statistical mechanics—but with a somewhat new outlook on this subject.

In getting to this understanding, we’ll bump into a few thorny questions. They’re interesting in themselves, but I don’t want them to hold us back, because we don’t absolutely need to answer them here. So, I’ll leave them as ‘puzzles’. Maybe you can help me with them.

In the classical mechanics of a point particle on a line, I considered a 4-dimensional vector space where a point describes the position momentum time and energy of a particle. This is usually called the extended phase space, since it’s bigger than the usual phase space where a point only keeps track of position and momentum. We saw that the extended phase space naturally acquires a symplectic structure

We can define Poisson brackets of smooth functions on any symplectic manifold. Instead of reviewing the whole well-known procedure I’ll just turn the crank and tell you what it gives for the coordinate functions on the extended phase space. The Poisson brackets of position and momentum are nonzero, and similarly for time and energy. All the rest vanish. We have

so we call these pairs of functions conjugate variables.

Puzzle 1. The first equation above says that momentum generates translations of position. The second says that energy generates time translations, but in reverse, thanks to the minus sign. Why the difference? Is this a problem?

To go to quantum mechanics, we can build a noncommutative algebra generated by elements I’ll call and We require that these commute except for two, which have commutators that mimic the Poisson brackets above:

Here is Planck’s constant, or more precisely the reduced Planck constant. This has dimensions of action, making the above equations dimensionally consistent: momentum times position has dimensions of action, and so does energy multiplied by time.

Puzzle 2. Does this mean the well-known Poisson bracket equations is not dimensionally consistent? Does this cause problems in classical mechanics? If not, why not?

We can work with the algebra having these generators and relations; this is called a Weyl algebra. But physicists usually prefer to treat this algebra as consisting of operators on some Hilbert space. The Stone–von Neumann theorem says there’s a unique ‘really nice’ way to do this. I don’t want to explain what ‘really nice’ means—it’s too much of a digression here. In fact I wrote a whole book about it with Irving Segal and Zhengfang Zhou. But this article is better if you just want the basic idea:

• Wikipedia, Stone–von Neumann theorem.

The main thing I need to say is that in the ‘really nice’ situation, and are unbounded self-adjoint operators, and the spectrum of each one is the whole real line. That’s not all there is to it: more must be true. But a problem immediately shows up. This is fine for the usual position and momentum operators in quantum mechanics, but the spectrum of the Hamiltonian is usually not the whole real line! In fact we usually want its spectrum to be bounded below: that is, we don’t want to allow arbitrarily large negative energies.

As a result, in quantum mechanics we usually give up on trying to find a ‘time operator’ that obeys the relation

There is more to say about this:

• John Baez, The time-energy uncertainty relation.

But this is not what I want to talk about today; I mention it only because we’ll see a similar problem in thermodynamics, where volume is bounded below.

In any event, there’s a standard ‘really nice’ way to make and into operators in quantum mechanics: we take the Hilbert space of square-integrable functions on the line, and define

We may try to copy this in thermodynamics.

So, let’s see how it works! There are some twists.

In my explanations of classical mechanics versus thermodynamics so far, I’ve mainly been using the energy scheme, where we start with the internal energy as a function of entropy and volume and then define temperature and pressure as derivatives, so that

As we’ve seen, this makes it very interesting to consider a 4-dimensional space—I’ll again call it the extended phase space—where and are treated as independent coordinates. This vector space has a symplectic structure

and the physically allowed points in this vector space form a ‘Lagrangian submanifold’—that is, a 2-dimensional surface on which the symplectic structure vanishes:

The symplectic structure lets us define Poisson brackets of smooth functions on the extended phase space. Again I’ll just turn the crank and tell you the Poisson brackets of the coordinate functions. The math is just the same, only the names have been changed, so you should not be surprised that two of their Poisson brackets are nonzero:

and all the rest vanish.

Puzzle 3. What is the physical meaning of these Poisson brackets?

Again we may worry that these equations are dimensionally inconsistent—but let’s go straight to the ‘quantized’ version and do our worrying there!

So, we’ll build a noncommutative algebra generated by elements I’ll call and We’ll require that these commute except for two, namely the pair and the pair What relation should these obey?

A first try might be

However, these equations are not dimensionally consistent! In both cases the left side has units of energy. Entropy times temperature and volume times pressure both have units of energy, since the first is related to ‘heat’ and the second is related to ‘work’. But has units of action.

I declare this to be intolerable. We are groping around trying to understand the physical meaning of the mathematics here; dimensional analysis is a powerful guide, and if we don’t have that going for us we’ll be in serious trouble.

So, we could try

where is some constant with units of energy.

Unfortunately there is no fundamental constant with units of energy that plays an important role throughout thermodynamics. We could for example use the mass of the electron times the speed of light squared, but why in the world should thermodynamics place some special importance on this?

One important quantity with units of energy in thermodynamics is where is temperature and is Boltzmann’s constant. Boltzmann’s constant is fundamental in thermodynamics, or more precisely statistical mechanics: it has units of energy per temperature. So we might try

but unfortunately the temperature is not a constant: it’s one of the variables we’re trying to ‘quantize’! It would make more sense to try

but unfortunately these commutation relations are more complicated than the ones we had in quantum mechanics.

Luckily, the fundamental constant we want is sitting right in front of us: it’s Boltzmann’s constant. This has units of energy per temperature—or in other words, units of entropy. This suggests quitting the ‘energy scheme’ and working with the second most popular formulation of thermodynamics, the ‘entropy scheme’.

Here we start by writing the entropy of our system as a function of its internal energy and volume A simple calculation starting from then shows

It will help to make up short names for the quantities here, so I’ll define

and call the coldness, for the obvious reason. I’ll also define

and call the boldness, for no particularly good reason—I just need a name. In terms of these variables we get

This is the so-called entropy scheme. This equation implies that times has dimensions of entropy, and so does
times

If we go through the procedure we used last time, we get a 4-dimensional vector space with coordinates and symplectic structure

The physically allowed points in this vector space form a Lagrangian submanifold

which is really just the submanifold we had before, now described using new coordinates.

But let’s get to the point! We’ll build a noncommutative algebra generated by elements I’ll call and We’ll require that these commute except for two, namely the pair and the pair And what commutation relations should these obey?

We could try

This is now dimensionally consistent, since in each equation both sides have dimensions of entropy!

However, there’s another twist. Quantum mechanics is about amplitudes, while statistical mechanics is about probabilities, which are real. So I actually think I want

Let me give some evidence for this!

I will ignore the second equation for now and focus on the first. Suppose we’re doing statistical mechanics and we have a system that has probability of having internal energy We would like to define an internal energy operator and coldness operator Say we set

Then it’s easy to check that they obey the commutation relation

Moreover—and this is the exciting part—they make physical sense! If the system definitely has internal energy then vanishes except at It follows that is an eigvenvector of the internal energy operator with eigenvalue

so

On the other hand, suppose the system definitely has temperature and thus coldness Then statistical mechanics tells us that is given by the Boltzmann distribution

where is a normalizing constant called the partition function. It follows that is an eigenvector of the coldness operator with eigenvalue

so

This makes me very happy. We are seeing a nice analogy:

• Internal energy eigenstates in statistical mechanics are like position eigenstates in quantum mechanics.

• Coldness eigenstates in statistical mechanics are like momentum eigenstates in quantum mechanics—except for a missing factor of i in the exponential, which makes the coldness eigenstates decrease exponentially instead of oscillate.

We can also define a temperature operator

at least if we ignore the eigenvectors of the coldness operator with eigenvalue zero. But the coldness operator is more fundamental in this approach.

A lot of further questions and problems appear at this point, but I think I’ll stop here and tackle them later.

---

• Part 1: Hamilton’s equations versus the Maxwell relations.

• Part 2: the role of symplectic geometry.

• Part 3: a detailed analogy between classical mechanics and thermodynamics.

• Part 4: what is the analogue of quantization for thermodynamics?

Classical Mechanics versus Thermodynamics (Part 3)

There’s a fascinating analogy between classical mechanics and thermodynamics, which I last talked about in 2012:

• Classical mechanics versus thermodynamics (part 1).
• Classical mechanics versus thermodynamics (part 2).

I’ve figured out more about it, and today I’m giving a talk about it in the physics colloquium at the University of British Columbia. It’s a colloquium talk that’s supposed to be accessible for upper-level undergraduates, so I’ll spend a lot of time reviewing the basics… which is good, I think.

You can see my slides here, and I’ll base this blog article on them. You can also watch a video of my talk:

Hamilton’s equations versus the Maxwell relations

Why do Hamilton’s equations in classical mechanics:

look so much like the Maxwell relations in thermodynamics?

William Rowan Hamilton discovered his equations describing classical mechanics in terms of energy around 1827. By 1834 he had also introduced Hamilton’s principal function, which I’ll explain later.

James Clerk Maxwell is most famous for his equations describing electromagnetism, perfected in 1865. But he also worked on thermodynamics, and discovered the ‘Maxwell relations’ in 1871.

Hamilton’s equations describe how the position and momentum of a particle on a line change with time if we know the energy or Hamiltonian :

Two of the Maxwell relations connect the volume , entropy , pressure and temperature of a system in thermodynamic equilibrium:

Using this change of variables:

Hamilton’s equations:

become these relations:

These are almost like two of the Maxwell relations! But in thermodynamics we always use partial derivatives:

and we say which variables are held constant:

If we write Hamilton’s equations in the same style as the Maxwell relations, they look funny:

Can this possibly be right?

Yes! When we work out the analogy between classical mechanics and thermodynamics we’ll see why.

We can get Maxwell’s relations starting from this: the internal energy of a system in equilibrium depends on its entropy and volume

Temperature and pressure are derivatives of

Maxwell’s relations follow from the fact that mixed partial derivatives commute! For example:

To get Hamilton’s equations the same way, we need a function of the particle’s position and time such that

Then we’ll get Hamilton’s equations from the fact that mixed partial derivatives commute!

The trick is to let be ‘Hamilton’s principal function’. So let’s define that. First, the action of a particle’s path is

where is the Lagrangian:

The particle always takes a path from to that’s a critical point of the action. We can derive Hamilton’s equations from this fact.

Let’s assume this critical point is a minimum. Then the least action for any path from to is called Hamilton’s principal function

A beautiful fact: if we differentiate Hamilton’s principal function, we get back the energy and momentum :

You can prove these equations using

which implies that

where we integrate along the minimizing path. (It’s not as trivial as it may look, but you can do it.)

Now let’s fix a starting-point for our particle, and say its path ends at any old point . Think of Hamilton’s principal function as a function of just :

Then the particle’s momentum and energy when it reaches are:

This is just what we wanted. Hamilton’s equations now follow from the fact that mixed partial derivatives commute!

So, we have this analogy between classical mechanics and thermodynamics:

Classical mechanics	Thermodynamics
action:	internal energy:
position:	entropy:
momentum:	temperature:
time:	volume:
energy:	pressure:

What’s really going on in this analogy? It’s not really the match-up of variables that matters most—it’s something a bit more abstract. Let’s dig deeper.

I said we could get Maxwell’s relations from the fact that mixed partials commute, and gave one example:

But to get the other Maxwell relations we need to differentiate other functions—and there are four of them!

• : internal energy
• : Helmholtz free energy
• : enthalpy
• : Gibbs free energy

They’re important, but memorizing all the facts about them has annoyed students of thermodynamics for over a century. Is there some other way to get the Maxwell relations? Yes!

In 1958 David Ritchie explained how we can get all four Maxwell relations from one equation! Jaynes also explained how in some unpublished notes for a book. Here’s how it works.

Start here:

Integrate around a loop :

so

This says the heat added to a system equals the work it does in this cycle

Green’s theorem implies that if a loop encloses a region

Similarly

But we know these are equal!

So, we get

for any region enclosed by a loop. And this in turn implies

In fact, all of Maxwell’s relations are hidden in this one equation!

Mathematicians call something like a 2-form and write it as . It’s an ‘oriented area element’, so

Now, starting from

We can choose any coordinates and get

(Yes, this is mathematically allowed!)

If we take we get

and thus

We can actually cancel some factors and get one of the Maxwell relations:

(Yes, this is mathematically justified!)

Let’s try another one. If we take we get

Cancelling some factors here we get another of the Maxwell relations:

Other choices of give the other two Maxwell relations.

In short, Maxwell’s relations all follow from one simple equation:

Similarly, Hamilton’s equations follow from this equation:

All calculations work in exactly the same way!

By the way, we can get these equations efficiently using the identity and the product rule for :

Now let’s change viewpoint slightly and temporarily treat and as independent from and So, let’s start with with coordinates . Then this 2-form on :

is called a symplectic structure.

Choosing the internal energy function , we get this 2-dimensional surface of equilibrium states:

Since

we know

for any region in the surface , since on this surface and our old argument applies.

This fact encodes the Maxwell relations! Physically it says: for any cycle on the surface of equilibrium states, the heat flow in equals the work done.

Similarly, in classical mechanics we can start with with coordinates , treating and as independent from and This 2-form on :

is a symplectic structure. Hamilton’s principal function defines a 2d surface

We have for any region in this surface And this fact encodes Hamilton’s equations!

Summary

In thermodynamics, any 2d region in the surface of equilibrium states has

This is equivalent to the Maxwell relations.

In classical mechanics, any 2d region in the surface of allowed 4-tuples for particle trajectories through a single point has

This is equivalent to Hamilton’s equations.

These facts generalize when we add extra degrees of freedom, e.g. the particle number in thermodynamics:

or more dimensions of space in classical mechanics:

We get a vector space with a 2-form on it, and a Lagrangian submanifold : that is, a n-dimensional submanifold such that

for any 2d region

This is more evidence for Alan Weinstein’s “symplectic creed”:

EVERYTHING IS A LAGRANGIAN SUBMANIFOLD

As a spinoff, we get two extra Hamilton’s equations for a point particle on a line! They look weird, but I’m sure they’re correct for trajectories that go through a specific arbitrary spacetime point

---

• Part 1: Hamilton’s equations versus the Maxwell relations.

• Part 2: the role of symplectic geometry.

• Part 3: a detailed analogy between classical mechanics and thermodynamics.

• Part 4: what is the analogue of quantization for thermodynamics?

Maxwell’s Relations (Part 3)

In Part 2 we saw a very efficient formulation of Maxwell’s relations, from which we can easily derive their usual form. Now let’s talk more about the meaning of the Maxwell relations—both their physical meaning and their mathematical meaning. For the physical meaning, I’ll draw again from Ritchie’s paper:

• David J. Ritchie, A simple method for deriving Maxwell’s relations, American Journal of Physics 36 (1958), 760–760.

but Emily Roach pointed out that much of this can also be found in the third chapter of Jaynes’ unpublished book:

• E. T. Jaynes, Thermodynamics, Chapter 3: Plausible reasoning.

First I’ll do the case of 2 dimensions, and then the case of n dimensions. In the 2d case I’ll talk like a physicist and use notation from thermodynamics. We’ll learn the Maxwell relations have these meanings, apart from their obvious ones:

• In any thermodynamic cycle, the heat absorbed by a system equals the work it does.

• In any thermodynamic cycle, energy is conserved.

• Any region in the surface of equilibrium states has the same area in pressure-volume coordinates as it does in temperature-entropy coordinates.

In the n-dimensional case I’ll use notation that mathematicians will like better, and also introduce the language of symplectic geometry. This will give a general, abstract statement of the Maxwell relations:

• The manifold of equilibrium states is a Lagrangian submanifold of a symplectic manifold.

Don’t worry—I’ll explain it!

Maxwell’s relations in 2 dimensions

Suppose we have a physical system whose internal energy is a smooth function of its entropy and volume So, we have a smooth function on the plane:

and we call the coordinates on this plane

Next we introduce two more variables called temperature and pressure So, we’ll give coordinates But, there’s a 2-dimensional surface where these extra variables are given by the usual formulas in thermodynamics:

We call this the surface of equilibrium states.

By the equations that and obey on the surface of equilibrium states, the following equation holds on this surface:

So, if is any loop on this surface, we have

where the total change in is zero because the loop ends where it starts! We thus have

In thermodynamics this has a nice meaning: the left side is the heat absorbed by a system as its state moves around the loop while the right side is the work done by this system. So, the equation says the heat absorbed by a system as it carries out a cycle is equal to the work it does.

We’ll soon see that this equation contains, hidden within it, all four Maxwell relations. And it’s a statement of conservation of energy! By the way, it’s perfectly obvious that energy is conserved in a cycle: our loop takes us from a point where has some value back to the same point, where it has the same value.

So how do we get Maxwell’s relations out of this blithering triviality? There’s a slow way and a fast way. Since the slow way provides extra insight I’ll do that first.

Suppose the loop bounds some 2-dimensional region in the surface Then by Green’s theorem we have

That is, the heat absorbed in the cycle is just the area of the region in coordinates. But Green’s theorem also says

That is, the work done in the cycle, which is minus the left hand side, is minus the area of the region in coordinates!

That’s nice to know. But since we’ve seen these are equal,

for every region in the surface

Well, at least I’ve shown it for every region bounded by a loop! But every region can be chopped up into small regions that are bounded by loops, so the equation is really true for any region in the surface

We express this fact by saying that equals when these 2-forms are restricted to the surface We write it like this:

Now, last time we saw how to quickly get from this equation to all four Maxwell relations!

(Back then I didn’t write the symbol because I was implicitly working on this surface without telling you. More precisely, I was working on the plane but we can identify the surface with that plane using the coordinates.)

So, given what we did last time, we are done! The equation

expresses both conservation of energy and all four Maxwell equations—in a very compressed, beautiful form!

Maxwell’s relations in n dimensions

Now we can generalize everything to n dimensions. Suppose we have a smooth function

Write the coordinates on as

and write the corresponding coordinates on the cotangent bundle as

There is a submanifold of where equals the partial derivative of with respect to

Let’s call this submanifold since it’s the same one we’ve seen before (except for that annoying little minus sign in the definition of pressure):

In applications to thermodynamics, this is the manifold of equilibrium states. But we’re doing math here, and this math has many applications. It’s this generality that makes the subject especially interesting to me.

Now, started out life as a function on but it lifts to a function on and we have

By the definition of we have

so

The 1-form on the right-hand side here:

is called the tautological 1-form on Its exterior derivative

is called the symplectic structure on this cotangent bundle. Both of these are a big deal in classical mechanics, but here we are seeing them in thermodynamics! And the point of all this stuff is that we’ve seen

Taking of both sides and using , we get

or in other words

And this is a very distilled statement of Maxwell’s relations!

Why? Well, in the 2d case we discussed earlier, the tautological 1-form is

thanks to the annoying minus sign in the definition of pressure. Thus, the symplectic structure is

and the fact that the symplectic structure vanishes when restricted to is just our old friend

As we’ve seen, this equation contains all of Maxwell’s relations.

In the n-dimensional case, is an n-dimensional submanifold of the 2n-dimensional symplectic manifold In general, if an n-dimensional submanifold of a 2n-dimensional symplectic manifold has the property that the symplectic structure vanishes when restricted to we call a Lagrangian submanifold.

So, one efficient abstract statement of Maxwell’s relations is:

The manifold of equilibrium states is a Lagrangian submanifold.

It takes a while to see the value of this statement, and I won’t try to explain it here. Instead, read Weinstein’s introduction to symplectic geometry:

• Alan Weinstein, Symplectic geometry, Bulletin of the American Mathematical Society 5 (1981), 1–13.

You’ll see here an introduction to Lagrangian submanifolds and an explanation of the “symplectic creed”:

EVERYTHING IS A LAGRANGIAN SUBMANIFOLD.
— Alan Weinstein

The restatement of Maxwell’s relations in terms of Lagrangian submanifolds is just another piece of evidence for this!

---

• Part 1: a proof of Maxwell’s relations using commuting partial derivatives.

• Part 2: a proof of Maxwell’s relations using 2-forms.

• Part 3: the physical meaning of Maxwell’s relations, and their formulation in terms of symplectic geometry.

For how Maxwell’s relations are connected to Hamilton’s equations, see this post:

• Classical mechanics versus thermodynamics.

Maxwell’s Relations (Part 2)

The Maxwell relations are some very general identities about the partial derivatives of functions of several variables. You don’t need to understand anything about thermodyamics to understand them, but they’re used a lot in that subject, so discussions of them tend to use notation from that subject.

Last time I went through a standard way to derive these relations for a function of two variables. Now I want to give a better derivation, which I found here:

• David J. Ritchie, A simple method for deriving Maxwell’s relations, American Journal of Physics 36 (1958), 760–760.

This paper is just one page long, and I can’t really improve on it, but I’ll work through the ideas in more detail. It again covers only the case of a function of two variables, and I won’t try to go beyond that case now—maybe later.

So, remember the setup. We have a smooth function on the plane

We call the coordinates on the plane and and we give the partial derivatives of funny names:

None of these funny names or the minus sign has any effect on the actual math involved; they’re just traditional in thermodynamics. So, mathematicians, please forgive me! If I ever generalize to the n-variable case, I’ll switch to more reasonable notation.

We instantly get this:

and since we get

so

Believe it or not, this simple relation contains all four of Maxwell’s relations within it!

To see this, note that both sides are smooth 2-forms on the plane. Now, the space of 2-forms at any one point of the plane is a 1-dimensional vector space. So, we can divide any 2-form at a point by any nonzero 2-form at that point and get a real number.

In particular, suppose and are functions on the plane such that at some point. Then we can divide both sides of the above equation by and get

at this point. We can now get the four Maxwell relations simply by making different choices of and We’ll choose them to be either or The argument will only work if so I’ll always assume that. The argument works the same way each time so I’ll go faster after the first time.

The first relation

Take and and substitute them into the above equation. We get

or

Next we use a little fact about differential forms and partial derivatives to simplify both sides:

and similarly

If you were scarred as a youth when plausible-looking manipulations with partial derivatives turned out to be unjustified, you might be worried about this—and rightly so! Later I’ll show how to justify the kind of ‘cancellation’ we’re doing here. But anyway, it instantly gives us the first Maxwell relation:

The second relation

This time we take Substituting this into our general formula

we get

and doing the same sort of ‘cancellation’ as last time, this gives the second Maxwell relation:

The third relation

This time we take Substituting this into our general formula

we get

which gives the third Maxwell relation:

The fourth relation

This time we take Substituting this into our general formula

we get

or

giving the fourth Maxwell relation:

You can check that other choices of and don’t give additional relations of the same form.

Determinants

So, we’ve see that all four Maxwell relations follow quickly from the equation

if we can do ‘cancellations’ in expressions like this:

when one of the functions equals one of the functions This works whenever Let’s see why!

First of all, by the inverse function theorem, if at some point in the plane, the functions and serve as coordinates in some neighborhood of that point. In this case we have

Yes: the ratio of 2-forms is just the Jacobian of the map sending to This is clear if you know that 2-forms are ‘area elements’ and the Jacobian is a ratio of area elements. But you can also prove it by a quick calculation:

and thus

so the ratio is the desired determinant.

How does this help? Well, take

and now suppose that either or equals either or For example, suppose Then we can do a ‘cancellation’ like this:

or to make it clear that the partial derivatives are being done in the coordinate system:

This justifies all our calculations earlier.

Conclusions

So, we’ve seen that all four Maxwell relations are unified in a much simpler equation:

which follows instantly from

This is a big step forward compared to the proof I gave last time, which, at least as I presented it, required cleverly guessing a bunch of auxiliary functions—even though these auxiliary functions turn out to be incredibly important in their own right.

So, we should not stop here: we should think hard about the physical and mathematical meaning of the equation

And Ritchie does this in his paper! But I will talk about that next time.

---

• Part 1: a proof of Maxwell’s relations using commuting partial derivatives.

• Part 2: a proof of Maxwell’s relations using 2-forms.

• Part 3: the physical meaning of Maxwell’s relations, and their formulation in terms of symplectic geometry.

For how Maxwell’s relations are connected to Hamilton’s equations, see this post:

• Classical mechanics versus thermodynamics.

Maxwell’s Relations (Part 1)

The Maxwell relations are equations that show up whenever we have a smooth convex function

They say that the mixed partial derivatives of commute, but also that the mixed partial derivatives of various functions constructed from commute. So in a sense they are completely trivial except for the way we construct these various functions! Nonetheless they are very important in physics, because they give generally valid relations between the derivatives of thermodynamically interesting quantities.

For example, here is one of the Maxwell relations that people often use when studying a thermodynamic system like a cylinder of gas or a beaker of liquid:

where:

• is the entropy of the system
• is its temperature
• is its volume
• is its pressure

There are also three more Maxwell relations involving these quantities. They all have the same general look, though half contain minus signs and half don’t. So, they’re quite tiresome to memorize. It’s more interesting to figure out what’s really going on here!

I already gave one story about what’s going on: we start with a function , which happens in this case to be a function of and called the internal energy of our system. If we say its mixed partial derivatives commute:

we get the Maxwell relation I wrote above, simply by using the standard definitions of and as partial derivatives of We can then build a bunch of other functions from and and write down other equations saying that the mixed partial derivatives of these functions commute. This gives us all the Maxwell relations.

Let me show you in detail how this works, but without explaining why I’m choosing the particular ‘bunch of other functions’ that I’ll use. There is a way to explain that using the concept of Legendre transform. But next time I’ll give a very different approach to the Maxwell relations, based on this paper:

• David J. Ritchie, A simple method for deriving Maxwell’s relations, American Journal of Physics 36 (1958), 760–760.

This sidesteps the whole business of choosing a ‘bunch of other functions’, and I think it’s very nice.

What follows is a bit mind-numbing, so let me tell you what to pay attention to. I’ll do the same general sort of calculation four times, first starting with any convex smooth function and then with three other functions built from that one. The only clever part is how we choose those other functions.

The first relation

We start with any smooth convex function of two real variables and write its differential as follows:

This says

These are the usual definitions of the temperature and pressure in terms of the internal energy but you don’t need to know anything about those concepts to follow all the calculations to come. In particular, from the pure mathematical viewpoint the minus sign is merely a stupid convention.

The commuting of mixed partials implies

giving the first of the Maxwell relations:

The second relation

Next, let’s define the Helmholtz free energy

Taking its differential we get

so if we think of as a function of we get

The commuting of mixed partials implies

giving the second of the Maxwell relations:

The third relation

Copying what we did to get the second relation, let’s define the enthalpy

Taking its differential we get

so if we think of as a function of we get

The commuting of mixed partials implies

giving the third of the Maxwell relations:

The fourth relation

Combining the last two tricks, let’s define the Gibbs free energy

Taking its differential we get

so if we think of as a function of we get

The commuting of mixed partials implies

giving the fourth and final Maxwell relation:

Conclusions

I hope you’re a bit unsatisfied, for two main reasons.

The first question you should have is this: why did we chose these four functions to work with:

The pattern of signs is not really significant here: if we hadn’t followed tradition and stuck a minus sign in our definition of here:

everything would look more systematic, and we’d use these four functions:

This would make it easier to guess how everything works if instead of a function we started with a function We could write this as

and define a bunch of functions

called ‘conjugate quantities’.

Then, we could get different functions by starting with and subtracting off products of the form where ranges over some subset of

Then we could take mixed partial derivatives of these functions, note that the mixed partial derivatives commute, and get a bunch of Maxwell relations — maybe

of them! (Sanity check: when is 2 this is indeed 4.)

The second question you should have is: how did I sneakily switch from thinking of as a function of and to thinking of as a function of and and so on? How could I get away with this? I believe the answer to this involves the concept of Legendre transform, which works well when is convex.

Answering these questions well might get us into a bit of contact geometry. That would be nice. But instead of answering these questions next time, I’ll talk about Ritchie’s approach to deriving Maxwell’s equations, which seems to sidestep the two questions I just raised!

Just for fun

Finally: students of thermodynamics are often forced to memorize the Maxwell relations. They sometimes use the thermodynamic square:

where for some idiotic reason the for pressure is written lower-case—perhaps so you can mix it up with momentum!

If you like puzzles, maybe you can figure out how the thermodynamic square works if you stare at it along with the four Maxwell equations I derived:

Again, it probably be easier if we hadn’t stuck a minus sign into the definition of pressure. If you get stuck, click on the link.

Unfortunately the thermodynamic square is itself so hard to memorize that students resort to a mnemonic for that! Sometimes they say “Good Physicists Have Studied Under Very Fine Teachers” which gives the letters “GPHSUVFT” that you see as you go around the square.

I will never try to remember any of these mnemonics. But I wonder if there’s something deeper going on here. So:

Puzzle. How does the thermodynamic square generalize if is a function of 3 variables instead of 2? How about more variables?

---

• Part 1: a proof of Maxwell’s relations using commuting partial derivatives.

• Part 2: a proof of Maxwell’s relations using 2-forms.

• Part 3: the physical meaning of Maxwell’s relations, and their formulation in terms of symplectic geometry.

For how Maxwell’s relations are connected to Hamilton’s equations, see this post:

• Classical mechanics versus thermodynamics.

The Cyclic Identity for Partial Derivatives

 

As an undergrad I learned a lot about partial derivatives in physics classes. But they told us rules as needed, without proving them. This rule completely freaked me out. If derivatives are kinda like fractions, shouldn’t this equal 1?

Let me show you why it’s -1.

First, consider an example:

This example shows that the identity is not crazy. But in fact it
holds the key to the general proof! Since is a coordinate system we can assume without loss of generality that . At any point we can approximate to first order as for some constants . But for derivatives the constant doesn’t matter, so we can assume it’s zero.

Then just compute!

There’s also a proof using differential forms that you might like
better. You can see it here, along with an application to
thermodynamics:

But this still leaves us yearning for more intuition — and for me, at least, a more symmetrical, conceptual proof. Over on Twitter, someone named
Postdoc/cake provided some intuition using the same example from thermodynamics:

Using physics intuition to get the minus sign:

1. increasing temperature at const volume = more pressure (gas pushes out more)
2. increasing temperature at const pressure = increasing volume (ditto)
3. increasing pressure at const temperature = decreasing volume (you push in more)

Jules Jacobs gave the symmetrical, conceptual proof that I was dreaming of:

This proof is more sophisticated than my simple argument, but it’s very pretty, and it generalizes to higher dimensions in ways that’d be hard to guess otherwise.

He uses some tricks that deserve explanation. As I’d hoped, the minus signs come from the anticommutativity of the wedge product of 1-forms, e.g.

But what lets us divide quantities like this? Remember, are all functions on the plane, so are 1-forms on the plane. And since the space of 2-forms at a point in the plane is 1-dimensional, we can divide them! After all, given two vectors in a 1d vector space, the first is always some number times the second, as long as the second is nonzero. So we can define their ratio to be that number.

For Jacobs’ argument, we also need that these ratios obey rules like

But this is easy to check: whenever are vectors in a 1-dimensional vector space, division obeys the above rule. To put it in fancy language, nonzero vectors in any 1-dimensional real vector space form a ‘torsor’ for the multiplicative group of nonzero real numbers:

• John Baez, Torsors made easy.

Finally, Jules used this sort of fact:

Actually he forgot to write down this particular equation at the top of his short proof—but he wrote down three others of the same form, and they all work the same way. Why are they true?

I claim this equation is true at some point of the plane whenever are smooth functions on the plane and doesn’t vanish at that point. Let’s see why!

First of all, by the inverse function theorem, if at some point in the plane, the functions and serve as coordinates in some neighborhood of that point. In this case we can work out in terms of these coordinates, and we get

or more precisely

Thus

so

as desired!