The ultimate goal is to understand the nonequilibrium thermodynamics of open systems—systems where energy and maybe matter flows in and out. If we could understand this well enough, we could understand in detail how *life* works. That’s a difficult job! But one has to start somewhere, and this is one place to start.

We have a few papers on this subject:

• Blake Pollard, A Second Law for open Markov processes. (Blog article here.)

• John Baez, Brendan Fong and Blake Pollard, A compositional framework for Markov processes. (Blog article here.)

• Blake Pollard, Open Markov processes: A compositional perspective on non-equilibrium steady states in biology. (Blog article here.)

]]>Got it. Thank you very much.

]]>Entropy is maximized by the “flat” or “constant” probability distribution: the one with

for all So, it’s not surprising that entropy increases for any Markov process for which the flat probability distribution is an equilibrium state. This happens if and only if time evolution is given by doubly stochastic matrices: matrices of nonnegative numbers where the entries in any column *and in any row* sum to 1.

But for most Markov processes, time evolution is *not* doubly stochastic. For most Markov processes, the flat probability distribution is *not* an equilibrium.

So, for most Markov processes we need to use relative entropy. If we evolve a probability distribution according to a Markov process, its entropy *relative to any equilibrium distribution* will increase. (Blake says it decreases, but that’s just because he’s leaving the minus sign out of the definition of relative entropy. It’s just a different convention.)

This is the best way to generalize the 2nd law from doubly stochastic Markov processes to general Markov processes. I have argued that relative entropy is more fundamental than entropy, because how much information you gain when you learn something depends on what you already knew. Note that the ordinary entropy is — up to a constant factor and additive constant — the same as entropy relative to the flat probability distribution. So even ordinary entropy is relative entropy in disguise.

But the really nice thing is that when we use relative entropy, we don’t need to worry about the equilibrium state! If and are two probability distributions evolving in time according to a Markov process, the relative entropy always increases! The case where is an equilibrium solution — constant in time — is just a special case.

]]>Sorry, that’s meant to be a fact. :)

(The question mark was left behind after reformulating my comment.)

Francesco wrote:

I would like to point out that a sort of information-theoretic “second-law–like” converse also hold, i.e., that any process that makes any ensemble of initial distributions less and less distinguishable over time is necessarily Markovian?

The question mark at the end confuses me. Are you pointing out a fact, or asking a question?

]]>Call me convinced.

]]>Up to some minor fudge factors, the entropy of a probability distribution is the Kullback-Leibler divergence of relative to the uniform distribution. Every Markov process with a finite set of states has at least one equilibium distribution. For Markov processes where the equilibrium distribution is the uniform distribution, entropy increases. This is the simplest version of the Second Law. Equivalently, the Kullback-Leibler divergence of relative to the uniform distribution decreases.

For Markov processes where the equilibrium distribution is some other distribution the Kullback-Leibler divergence of relative to decreases. This is the simplest version of the Second Law that applies to all Markov processes with a finite set of states.

But in fact there’s a more powerful generalization, which is what Blake uses. For any Markov process, if we take two probability distributions and and evolve them forward in time, the Kullback-Leibler divergence of relative to decreases!

This subsumes all the results I mentioned previously. It has the great advantage that we don’t need to know an equilibrium distribution to apply it. This, I claim, is the right way to think about the Second Law for Markov processes.

]]>