In the first part of this mini-series, I describe how various ideas important in probability theory arise naturally when you start doing linear algebra using only the nonnegative real numbers.

But after writing it, I got an email from a rather famous physicist saying he got “lost at line two”. So, you’ll be happy to hear that the first part is *not a prerequisite* for the remaining parts! I wrote it just to intimidate that guy.

Tobias Fritz and I have proved a theorem characterizing the concept of relative entropy, which is also known as ‘relative information’, ‘information gain’ or—most terrifying and least helpful of all—‘Kullback-Leibler divergence’. In this second part I’ll introduce two key players in this theorem. The first, is a category where:

• an object consists of a system with finitely many states, and a probability distribution on those states

and

• a morphism consists of a deterministic ‘measurement process’ mapping states of one system to states of another, together with a ‘hypothesis’ that lets the observer guess a probability distribution of states of the system being measured, based on what they observe.

The second, is a subcategory of It has all the same objects, but only morphisms where the hypothesis is ‘optimal’. This means that if the observer measures the system many times, and uses the probability distribution of their observations together with their hypothesis to guess the probability distribution of states of the system, they *get the correct answer* (in the limit of many measurements).

In this part all I will really do is explain precisely what and are. But to whet your appetite, let me explain how we can use them to give a new characterization of relative entropy!

Suppose we have any morphism in In other words: suppose we have a deterministic measurement process, together with a hypothesis that lets the observer guess a probability distribution of states of the system being measured, based on what they observe.

Then we have *two* probability distributions on the states of the system being measured! First, the ‘true’ probability distribution. Second, the probability that the observer will guess based on their observations.

Whenever we have two probability distributions on the same set, we can compute the entropy of the first *relative to* to the second. This describes how surprised you’ll be if you discover the probability distribution is really the first, when you thought it was the second.

So: any morphism in will have a relative entropy. It will describe how surprised the observer will be when they discover the true probability distribution, given what they had guessed.

But this amount of surprise will be *zero* if their hypothesis was ‘optimal’ in the sense I described. So, the relative entropy will vanish on morphisms in

Our theorem says this fact almost characterizes the concept of relative entropy! More precisely, it says that any convex-linear lower semicontinuous functor

that vanishes on the subcategory must equal some constant times the relative entropy.

Don’t be scared! This should not make sense to you yet, since I haven’t said how I’m thinking of as a category, nor what a ‘convex-linear lower semicontinuous functor’ is, nor how relative entropy gives one. I will explain all that later. I just want you to get a vague idea of where I’m going.

Now let me explain the categories and We need to warm up a bit first.

### FinStoch

A stochastic map is different from an ordinary function, because instead of assigning a unique element of to each element of it assigns a *probability distribution on* to each element of So you should imagine it as being like a function ‘with random noise added’, so that is not a specific element of but instead has a probability of taking on different values. This is why I’m using a weird wiggly arrow to denote a stochastic map.

More formally:

**Definition.** Given finite sets and a **stochastic map** assigns a real number to each pair such that fixing any element the numbers form a probability distribution on We call **the probability of given **

In more detail:

• for all

and

• for all

Note that we can think of as a -shaped matrix of numbers. A matrix obeying the two properties above is called **stochastic**. This viewpoint is nice because it reduces the problem of composing stochastic maps to matrix multiplication. It’s easy to check that multiplying two stochastic matrices gives a stochastic matrix. So, composing stochastic maps gives a stochastic map.

We thus get a category:

**Definition.** Let be the category of finite sets and stochastic maps between them.

In case you’re wondering why I’m restricting attention to *finite* sets, it’s merely because I want to keep things simple. I don’t want to worry about whether sums or integrals converge.

### FinProb

Now take your favorite 1-element set and call it A function is just a point of But a stochastic map is something more interesting: it’s a probability distribution on

Why? Because it gives a probability distribution on for each element of but that set has just one element.

Last time I introduced the rather long-winded phrase **finite probability measure space** to mean a finite set with a probability distribution on it. But now we’ve seen a very quick way to describe such a thing within

And this gives a quick way to think about a measure-preserving function between finite probability measure spaces! It’s just a commutative triangle like this:

Note that the horizontal arrow is not wiggly. The straight arrow means it’s an honest function, not a stochastic map. But a function is a special case of a stochastic map! So it makes sense to compose a straight arrow with a wiggly arrow—and the result is, in general, a wiggly arrow. So, it makes sense to demand that this triangle commutes, and this says that the function is measure-preserving.

Let me work through the details, in case they’re not clear.

First: how is a function a special case of a stochastic map? Here’s how. If we start with a function we get a matrix of numbers

where is the Kronecker delta. So, each element gives a probability distribution that’s zero except at

Given this, we can work out what this commuting triangle really says:

If use to stand for the probability distribution that puts on and similarly for the commuting triangle says

or in other words:

or if you like:

In this situation people say is **pushed forward along **, and they say is a **measure-preserving function**.

So, we’ve used to describe another important category:

**Definition.** Let be the category of finite probability measure spaces and measure-preserving functions between them.

I can’t resist mentioning another variation:

A commuting triangle like this is a **measure-preserving stochastic map**. In other words, gives a probability measure on gives a probability measure on and is a stochastic map with

### FinStat

The category we really need for relative entropy is a bit more subtle. An object is a finite probability measure space:

but a morphism looks like this:

The whole diagram doesn’t commute, but the two equations I wrote down hold. The first equation says that is a measure-preserving function. In other words, this triangle, which we’ve seen before, commutes:

The second equation says that is the identity, or in math jargon, is a **section** for

But what does that *really mean?*

The idea is that is the set of ‘states’ of some system, while is a set of possible ‘observations’ you might make. The function is a ‘measurement process’. You ‘measure’ the system using and if the system is in the the state you get the observation The probability distribution says the probability that the system is any given state, while says the probability that you get any given observation when you do your measurement.

Note: are assuming for now that that there’s no random noise in the observation process! That’s why is a function instead of a stochastic map.

But what about That’s the fun part: describes your ‘hypothesis’ about the system’s state given a particular measurement! If you measure the system and get a result you guess it’s in the state with probability

And we don’t want this hypothesis to be really dumb: that’s what

says. You see, this equation says that

or in other words:

If you think about it, this implies unless

So, if you make an observation you will guess the system is in state with *probability zero* unless In short, you won’t make a really dumb guess about the system’s state.

Here’s how we compose morphisms:

We get a measure-preserving function and a stochastic map going back, You can check that these obey the required equations:

So, we get a category:

**Definition.** Let be the category where an object is a finite probability measure space:

a morphism is a diagram obeying these equations:

and composition is defined as above.

### FP

As we’ve just seen, a morphism in consists of a ‘measurement process’ and a ‘hypothesis’

But sometimes we’re lucky and our hypothesis is optimal, in the sense that

Conceptually, this says that if you take the probability distribution on our observations and use it to guess a probability distribution for the system’s state using our hypothesis you *get the correct answer*:

Mathematically, it says that this diagram commutes:

In other words, is a measure-preserving stochastic map.

There’s a subcategory of with all the same objects, but only these ‘optimal’ morphisms. It’s important, but the name we have for it is not very exciting:

**Definition.** Let be the subcategory of where an object is a finite probability measure space

and a morphism is a diagram obeying these equations:

Why do we call this category ? Because it’s a close relative of where a morphism, you’ll remember, looks like this:

The point is that for a morphism in the conditions on are so strong that they completely determine it *unless there are observations that happen with probability zero*—that is, unless there are with To see this, note that

actually says

for any choice of But we’ve already seen unless so the sum has just one term, and the equation says

We can solve this for so is completely determined… *unless *

This covers the case when We also can’t figure out if isn’t in the image of

So, to be utterly precise, is determined by and unless there’s an element that has Except for this special case, a morphism in is just a morphism in But in this special case, a morphism in has a little extra information: an arbitrary probability distribution on the inverse image of each point with this property.

In short, is the same as except that our observer’s ‘optimal hypothesis’ must provide a guess about the state of the system given an observation, *even in cases of observations that occur with probability zero.*

I’m going into these nitpicky details for two reasons. First, we’ll need for our characterization of relative entropy. But second, Tom Leinster *already ran into this category* in his work on entropy and category theory! He discussed it here:

• Tom Leinster, An operadic introduction to entropy.

Despite the common theme of entropy, he arrived at it from a very different starting-point.

### Conclusion

So, I hope that next time I can show you something like this:

and you’ll say “Oh, that’s a probability distribution on the states of some system!” Intuitively, you should think of the wiggly arrow as picking out a ‘random element’ of the set

I hope I can show you this:

and you’ll say “Oh, that’s a deterministic measurement process, sending a probability distribution on the states of the measured system to a probability distribution on observations!”

I hope I can show you this:

and you’ll say “Oh, that’s a deterministic measurement process, together with a hypothesis about the system’s state, given what is observed!”

And I hope I can show you this:

and you’ll say “Oh, that’s a deterministic measurement process, together with an *optimal* hypothesis about the system’s state, given what is observed!”

I don’t count on it… but I can hope.

### Postscript

And speaking of unrealistic hopes, if I were *really* optimistic I would hope you noticed that and which underlie the more fancy categories I’ve discussed today, were themselves constructed starting from linear algebra over the nonnegative numbers in Part 1. That ‘foundational’ work is not really needed for what we’re doing now. However, I like the fact that we’re ultimately getting the concept of relative entropy starting from very little: just linear algebra, using only nonnegative numbers!

Thanks John, this was admirably clear & self contained; I think I got most of it using only an elementary understanding of sets, mappings, probability distributions, and summation and composition notation (though I might be deluding myself, as I’m still fuzzy on what a category is and on exactly what a morphism is; and I still don’t think I know what it means for a diagram to ‘commute':-).

“unless there are observations that happen with probability zero—that is, unless there are “: Do you mean Y instead of X?

arch1 wrote:

Yes, thanks—I’ll fix that. Deliberately calling an element of “” would be heinous mathematical crime… here in Singapore I’d probably get caned for it.

I’m really glad you found the exposition clear! I’m warming up to write a paper about this, so I need to know what works and what doesn’t.

I’ll tell you what it means for a diagram to commute. Here’s a simple example:

A diagram commutes whenever, given two ways to get from one object to another by following a chain of arrows, those two ways are equal. In this case we have two ways to get from to There's a direct way using just and indirect way using first and then But they are equal—that's what the equation below the diagram says. If I were talking to category theorists, I could skip the equation and just say "the diagram commutes".

This pays off in situations where we have big complicated diagrams like this:

(from the proof of the ‘zig-zag lemma’ on Wikipedia) or this:

(a typical sort of thing you see on an algebraic topologist’s whiteboard, drawn by Patrick Orson.) Instead of writing down dozens of equations, you get a nice visual depiction of what’s going on, and you can learn to reason very quickly with these diagrams.

Unfortunately my post is not a great introduction to commutative diagrams, for two reasons.

First, I’m heavily using

two kinds of arrows, straight ones and wiggly ones. This is a bit nonstandard; it means my diagrams involve not just one category but two: the category of finite sets and stochastic maps (wiggly arrows), and a subcategory of that, the category of finite sets and functions (straight arrows). If you’re just getting started at this game, you’d want to start with one kind of arrow.Second, lots of my diagrams

don’tcommute! They don’t completely commute, so I have to say which equationsdohold, by writing them below the diagram, like here:As an exercise you can try to guess an equation not listed here that

wouldhold if the diagramdidcommute.Thanks for the broader overview of those diagrams! As long as we’re correcting small typos, I’m thinking that by “It’s each to check that multiplying two stochastic matrices gives a stochastic matrix,” you meant “It’s easy to check that multiplying two stochastic matrices gives a stochastic matrix.”

Yes indeed—I’ll fix that. Thanks!

Thanks John. (The two arrow types don’t cause me problems if I remind myself that, as you said, functions are special cases of stochastic maps.)

I think that the equation ‘q then s yields the same distribution on X as p does’ answers your exercise (in other words, as you imply above, if its diagram commutes a hypothesis is optimal).

Yes, that’s one of the equations that would hold if the whole diagram above commuted. There’s also another:

And this is something we really don’t want, usually. Taken together with

that would imply a very strong condition on our function

Can anyone see what this condition amounts to?

I had written:

But now I see that second case never comes up! In this situation

we can show that the function is onto. The paper I’m writing with Tobias will explain this…

In the last couple days I’ve returned to working on a paper with Tobias Fritz where we give a Bayesian characterization of the concept of ‘relative entropy’. This summer I wrote two blog articles about this paper:

• Relative Entropy (Part 1): how various structures important in probability theory arise naturally when you do linear algebra using only the nonnegative real numbers.

• Relative Entropy (Part 2): a category related to statistical inference, and how relative entropy defines a functor on this category.

But then Tobias Fritz noticed a big problem.

[…] but I don’t have the knowledge and training to develop this idea — you’d need someone like John Baez for […]

Tobias Fritz, a postdoc at the Perimeter Institute, is working with me on category-theoretic aspects of information theory. We published a paper on entropy with Tom Leinster, and we’ve got a followup on relative entropy that’s almost done. I should be working on it right this instant! But for now, read the series of posts here on Azimuth: Relative Entropy Part 1, Part 2 and Part 3. […]

Now Tobias Fritz and I have finally finished our paper on this subject:

• A Bayesian characterization of relative entropy.