*guest post by Jacob Biamonte*

This post is part of a series on what John and I like to call *Petri net field theory*. Stochastic Petri nets can be used to model everything from vending machines to chemical reactions. Chemists have proven some powerful theorems about when these systems have equilibrium states. We’re trying to bind these old ideas into our fancy framework, in hopes that quantum field theory techniques could also be useful in this deep subject. We’ll describe the general theory later; today we’ll do an example from population biology.

Those of you following this series should know that I’m the calculation bunny for this project, with John playing the role of the wolf. If I don’t work quickly, drawing diagrams and trying to keep up with John’s space-bending quasar of information, I’ll be eaten alive! It’s no joke, so please try to respond and pretend to enjoy anything you read here. This will keep me alive for longer. If I did not take notes during our meetings, lots of this stuff would have never made it here, so hope you enjoy.

#### Amoeba reproduction and competition

Here’s a stochastic Petri net:

It shows a world with one state, **amoeba**, and two transitions:

• **reproduction**, where one amoeba turns into two. Let’s call the rate constant for this transition .

• **competition**, where two amoebas battle for resources and only one survives. Let’s call the rate constant for this transition .

We are going to analyse this example in several ways. First we’ll study the *deterministic* dynamics it describes: we’ll look at its rate equation, which turns out to be the logistic equation, familiar in population biology. Then we’ll study the *stochastic* dynamics, meaning its master equation. That’s where the ideas from quantum field theory come in.

#### The rate equation

If is the population of amoebas at time , we can follow the rules explained in Part 3 and crank out this **rate equation**:

We can rewrite this as

where

What’s the meaning of and ?

• is the **carrying capacity**, that is, the maximum sustainable population the environment can support.

• is the **growth rate** describing the approximately exponential growth of population when is small.

It’s a rare treat to find such an important differential equation that can be solved by analytical methods. Let’s enjoy solving it.

We start by separating variables and integrating both sides:

We need to use partial fractions on the left side above, resulting in

and so we pick up a constant , let

and rearrange things as

so the population as a function of time becomes

At we can determine uniquely. We write and becomes

The model now becomes very intuitive. Let’s set and make a plot for various values of :

We arrive at three distinct cases:

• **equilibrium** (). The horizontal blue line corresponds to the case where the initial population exactly equals the carrying capacity. In this case the population is constant.

• **dieoff** (). The three decaying curves above the horizontal blue line correspond to cases where initial population is higher than the carrying capacity. The population dies off over time and approaches the carrying capacity.

• **growth** (). The four increasing curves below the horizontal blue line represent cases where the initial population is lower than the carrying capacity. Now the population grows over time and approaches the carrying capacity.

#### The master equation

Next, let us follow the rules explained in Part 6 to write down the master equation for our example. Remember, now we write:

where is the probability of having amoebas at time , and is a formal variable. The **master equation** says

where is an operator on formal power series called the **Hamiltonian**. To get the Hamiltonian we take each transition in our Petri net and build an operator built from creation and annihilation operators, as follows. Reproduction works like this:

while competition works like this:

Here is the annihilation operator, is the creation operator and is the number operator. Last time John explained precisely how the ‘s arise. So the theory is already in place, and we arrive at this Hamiltonian:

Remember, is the rate constant for reproduction, while is the rate constant for competition.

The master equation can be solved: it’s equivalent to

so that is constant, and so

and that’s it! We can calculate the time evolution starting from any initial probability distribution of populations. Maybe everyone is already used to this, but I find it rather remarkable.

Here’s how it works. We pick a population, say amoebas at This would mean . We then evolve this state using . We expand this operator as

This operator contains the full information for the evolution of the system. It contains the *histories* of all possible amoeba populations—an amoeba mosaic if you will. From this, we can construct amoeba Feynman diagrams.

To do this, we work out each of the terms in the expansion above. The first-order terms correspond to the Hamiltonian acting once. These are proportional to either or . The second-order terms correspond to the Hamiltonian acting twice. These are proportional to either , or . And so on.

This is where things start to get interesting! To illustrate how it works, we will consider two possibilities for the second-order terms:

**1)** We start with a lone amoeba, so . It reproduces and splits into two. In the battle of the century, the resulting amoebas compete and one dies. At the end we have:

We can draw this as a Feynman diagram:

You might find this tale grim, and you may not like the odds either. It’s true, the odds could be better, but people are worse off than amoebas! The great Japanese swordsman Miyamoto Musashi quoted the survival odds of fair sword duels as 1/3, seeing that 1/3 of the time both participants die. A remedy is to cheat, but these amoeba are competing *honestly*.

**2)** We start with two amoebas, so the initial state is . One of these amoebas splits into two. One of these then gets into an argument with the original amoeba over the Azimuth blog. The amoeba who solved all John’s puzzles survives. At the end we have

with corresponding Feynman diagram:

This should give an idea of how this all works. The exponential of the Hamiltonian gives all possible histories, and each of these can be translated into a Feynman diagram. In a future blog entry, we might explain this theory in detail.

#### An equilibrium state

We’ve seen the equilibrium solution for the rate equation; now let’s look for equilibrium solutions of the master equation. This paper:

• D. F. Anderson, G. Craciun and T.G. Kurtz, Product-form stationary distributions for deficiency zero chemical reaction networks, arXiv:0803.3042.

proves that for a large class of stochastic Petri nets, there exists an equilibrium solution of the master equation where the number of things in each state is distributed according to a Poisson distribution. Even more remarkably, these probability distributions are *independent*, so knowing how many things are in one state tells you nothing about how many are in another!

Here’s a nice quote from this paper:

The surprising aspect of the deficiency zero theorem is that the assumptions of the theorem are completely related to the network of the system whereas the conclusions of the theorem are related to the dynamical properties of the system.

The ‘deficiency zero theorem’ is a result of Feinberg, which says that for a large class of stochastic Petri nets, the rate equation has a unique equilibrium solution. Anderson showed how to use this fact to get equilibrium solutions of the master equation!

We will consider this in future posts. For now, we need to talk a bit about ‘coherent states’.

These are all over the place in quantum theory. Legend (or at least Wikipedia) has it that Erwin Schrödinger himself discovered coherent states when he was looking for states of a quantum system that look ‘as classical as possible’. Suppose you have a quantum harmonic oscillator. Then the uncertainty principle says that

where is the uncertainty in the momentum and is the uncertainty in position. Suppose we want to make as small as possible, and suppose we also want . Then we need our particle to be in a ‘coherent state’. That’s the definition. For the quantum harmonic oscillator, there’s a way to write quantum states as formal power series

where is the amplitude for having quanta of energy. A coherent state then looks like this:

where can be any complex number. Here we have omitted a constant factor necessary to normalize the state.

We can also use coherent states in classical stochastic systems like collections of amoebas! Now the coefficient of tells us the probability of having amoebas, so had better be real. And probabilities should sum to 1, so we really should normalize as follows:

Now, the probability distribution

is called a **Poisson distribution**. So, for starters you can think of a ‘coherent state’ as an over-educated way of talking about a Poisson distribution.

Let’s work out the expected number of amoebas in this Poisson distribution. In the answers to the puzzles in Part 6, we started using this abbreviation:

We also saw that the expected number of amoebas in the probability distribution is

What does this equal? Remember that . The annihilation operator is just , so

and we get

But we saw in Part 5 that is stochastic, meaning

for any Furthermore, our here has

since it’s a probability distribution. So:

The expected number of amoebas is just .

**Puzzle 1.** This calculation must be wrong if is negative: there can’t be a negative number of amoebas. What goes wrong then?

**Puzzle 2.** Use the same tricks to calculate the standard deviation of the number of amoebas in the Poisson distribution .

Now let’s return to our problem and consider the initial amoeba state

Here aren’t bothering to normalize it, because we’re going to look for equilibrium solutions to the master equation, meaning solutions where doesn’t change with time. So, we want to solve

Since this equation is linear, the normalization of doesn’t really matter.

Remember,

Let’s work this out. First consider the two terms:

and

Likewise for the terms we find

and

Here I’m using something John showed in Part 6: the product equals the ‘falling power’ .

The sum of all four terms must vanish. This happens whenever

which is satisfied for

Yipee! We’ve found an equilibrium solution, since we found a value for that makes . Even better, we’ve seen that the expected number of amoebas in this equilibrium state is

This is just the same as the equilibrium population we saw for the *rate equation*—that is, the carrying capacity for the logistic equation! That’s pretty cool, but it’s no coincidence: in fact, Anderson proved it works like this for lots of stochastic Petri nets.

I’m not sure what’s up next or what’s in store, since I’m blogging at gun point from inside a rabbit cage:

I’d imagine we’re going to work out the theory behind this example and prove the existence of equilibrium solutions for master equations more generally. One idea John had was to have me start a night shift—that way you’ll get Azimuth posts 24 hours a day.

You’re missing a “latex” on a $\Psi$ and a $\beta$. I’ll try to check the rest of the ciphering more carefully when I don’t have to be going to bed so I can get up in time for a meeting. . . nice post, though!

Fixed!

As long as you’re fixing up LaTeX, it would be nice if you tossed in some \displaystyle directives to make the fractions like e^{cz}/e^c a legible size and get the limits on the sums in the right place.

Yes, boss – done.

In general the LaTeX looks nicer on the version on my website.

had better be non-negative then, preventing the situation described in puzzle 1.

Yes – you got it.

Puzzle 2: Variance = .

Variance = .

Great! I hadn’t actually gotten up the nerve to do this one! I just knew it had to be doable.

By the way, the rate equation (first equation of the post) is not displaying:

$latex\displaystyle{ \frac{d P}{d t} = \alpha P – \beta P^2} $

Looks like the space after ‘latex’ is missing.

Thanks, fixed! I screwed it up when I inserted \displaystyle according to Mike’s suggestion.

Great post, as usual! I am enjoying this introduction to network theory very much.

An almost certainly very stupid question – sorry, it’s been ages since I touched a differential equation: how do you get that

from the master equation? I can see how is a solution for it – it follows at once from – but I am getting the feeling that I am missing something obvious but important here…

Thanks!

Hi. Starting with

or

We will pull one of John’s rabbits out and multiply this by and note that commutes with any power series in (which is the case of interest here from the expansion of ). So

and this is actually

So it’s consistent, at the price of one rabbit. The other method is to tinker with

and do the same thing, backwards.

Cheers!

Oh, I see – thanks a lot!

Because is a constant.

I looked at the referenced paper by Anderson, Craciun and Kurtz, and they have a section on the multiscale nature of reaction networks. I think this is the important pull-quote

“Within a cell, some chemical species may be present in much greater abundance than others. In addition, the rate constants k may vary over several orders of magnitude.”That is disorder in a nutshell and it plays a huge role in many practical network applications, including the ubiquitous TCP/IP networks.So for the classic Logistic equation derivation, note that the rate constant k is set as a constant. But in a multiscale network, the k can vary widely, and so getting a Logistic sigmoid as a result would be unlikely based on the classical derivation. In other words, what combination of functions could generate a

stableLogistic sigmoid? (I mean stable in the sense that the Gaussian, Cauchy, and Levy distributions are considered stable.)Until we realize that not only can the Logistic equation can be derived from the obvious but “brain-dead” separation of variables technique, but also from a truly stochastic approach using ideas such as the Maximum Entropy Principle, we are spinning our wheels. By that I mean that we live in a deterministic world when we try to solve the equations, but in many cases the only practical results come from a stochastic framing.

That’s why I think this “Green Math” work is so incredibly vital. I don’t have the high-powered math skills that you guys have, but being an engineer I hope I can pick out useful ideas to apply to renewable energy strategies. Many of the significant renewable energy sources and practical devices are maximally disordered, including wind, PV cells, geothermal, etc.

Hi Jake.

Have the quantum field theory techniques been useful here, with Azimuth?

Ginger

Hi Ginger,

The techniques appearing in quantum theory are very well developed, so our hope here is two-fold. (i) Cast these methods into new domains in hopes that we could leverage these tools to attack existing problems and (ii) Conversely, build a bridge so these important problems in network science and ecology (etc) can be addressed by others working in quantum theory!

[…] We have already seen one example of this theorem, back in Part 7. […]

I dont understand how the pertubation approach works. If we set and then the first order term generates a negative for t>1/4. The second order term gets a negative for t>2/13. Is this supposed toget fixed a higher order?

Didn’t seem to go through the first time.

I haven’t thought about this particular calculation for a long time, but yes, you have to go to high enough order to get a decent answer for large … and the power series may not actually converge for large so you may need to use resummation methods like Borel summation.

In practice, this perturbation theory works best for small There are other methods that work better for large

I’ve tried for a few and ‘s up to fourth order. Generally it seems the region in t for which I have strictly positive coefficients goes down with higher order! That is the higher order approximations becomes invalid(or less accurate, meaningless or whatever is actually going on) earlier than the lower orders. Do you have an example where this method is actually used? ( I didn’t find any in the book and I’m afraid I might have miscalculated something)

I don’t have a concrete calculation to point you to. If you can show me some calculations I could see if they look right.

Recently I’ve been thinking a bit about this stuff and come across a vexing property of the amoeba system: if you introduce a very large number as the maximal population size by simply not letting the population grow above that threshold, then the only equilibrium state is the one where all amoebas are dead. In other words: in order to obtain the Poisson distribution as an equilibrium, you *need* to have an infinite state space and allow the population to grow arbitrarily large.

How does this come about? First, note that there’s another solution of your equilibrium equation

that you’ve swept under the rug, namely ! The corresponding “coherent” state has probability 1 at a population size of 0. This is the state in which all amoebas are dead with certainty. It is obviously an equilibrium state!

Moreover, for any population that evolves for a time , there is a positive probability for the population to end up in this state by dying out completely, and then no new amoebas will ever be born again. In the terminology of Markov chains, all amoebas being dead is an absorbing state. So one may be tempted to conclude that, if one only waits long enough, then any initial population of amoebas will die out at some point! In particular, there should not be any equilibrium solution other than amoeba extinction: any finite absorbing Markov chain like this ends up in the absorbing state with probability 1.

So what’s going on here? How is it possible that there is an equilibrium distribution other than total extinction? I think the answer is that the actual Markov chain that describes the amoeba population can be arbitrarily large: for a Markov chain with infinite state space, the theorem that an absorbing Markov chain ends up in the absorbing state with probability 1 does not hold anymore.

Now here’s the question: does this mean that any initial population has probability 1 to either die out completely or grow indefinitely? (By “grow indefinitely”, I mean that the lim sup of population size is infinite. So I would allow the population to bounce back to small sizes before shooting up again, even infinitely many times.)

“does this mean that any initial population has probability 1 to either die out completely or grow indefinitely?”

When the population is 1, competition is impossible, so it can never reach 0.

Graham wrote:

Whoops, thanks for pointing that out! But if you replace “population is 0” by “population is 1” everywhere, then my comment is still valid.

But population=1 is not an equilibrium. If there is one ameoba, there will be two sometime later.

Erm, replace ‘equilibrium’ with ‘absorbing state’. It is possible – John even explicity mentioned it – that the carrying capacity is 1.

True… I have to admit that I have secretly been thinking in terms of a different model, namely one in which competition is replaced by a “death rate” per individual which is proportional to current population size. This results in a slightly different kind of Hamiltonian with which the population can die out completely. (It probably won’t arise from a Petri net, though.)

In any case, I understand now that infinite absorbing Markov chains can behave quite differently from finite absorbing ones: the latter can’t have non-trivial equilibria, while the former can! Introducing a finite population cap and then letting it tend to infinity will not detect such an equilibrium.

(In order to find it, I guess that one will have to look at the second largest eigenvalue of the transition matrix and see whether it converges to 1 as the population cap tends to infinity.)

The model with only the two transitions and looks like a good toy model. But in order for your statements here about the rate constants to make sense, I assume that you were meaning to write down

thesetwo transitions:This is still not what I had in mind with “death rate per individual proportional to population size”, since the probability of an individual to die is a constant rather than proportional to population size. But it’s an interesting model to consider, due to the phenomenon that no new individuals can be born once the whole population has died out. If the reaction rates are and , then the rate equation looks like this:

Since the deficiency is the deficiency zero theorem doesn’t apply here, and so we have to analyze the equilibria by hand. As you pointed out, we need in order for an equilibrium to exist, so let’s assume this and put for the sake of simplicity. Then

anyconstant is an equilibrium solution of the rate equation — but only $P=0$ is complex balanced, as you can see by noting that the complex $A+A$ gets produced in the case $P>0$, but never gets destroyed, thereby violating complex balance.The master equation looks like this:

I’m not quite sure what to do with this. The Anderson-Craciun-Kurtz theorem won’t tell us much, since we only have a trivial complex balanced equilibrium of the rate equation. Your sentiment that eventual extinction will occur with probability 1 sounds right to me; do you have any idea of how to prove it? It seems related to the recurrence of a one-dimensional random walk.

Side question: the Hamiltonians that appear in these master equations vaguely resemble Laplace operators on Cayley graphs, with respect to which the equilibrium distributions are harmonic functions. Is there a connection here?

By the way, some friends and I are currently studying mating systems, and we will hopefully be able to put this stuff to good use! I’m concerned about the possibility of the population dropping below 2, which for us implies guaranteed extinction. I wonder whether this means that we can’t expect our master equations to have any non-trivial equilibrium solutions.

The order of the comments here has gotten screwed up in a way I seem unable to fix. You have to look at the times comments were posted to read them in the right order! So, I’ve added links to help people navigate the conversations.

Over here, Tobias wrote:

Right.

Right. I didn’t see the “per individual” part.

Right. That seems pretty easy in this case. But if you ever get a harder example with deficiency 1 or more, you might look at the ‘deficiency one theorem’. It’s in here:

• Martin Feinberg, Chemical reaction network structure and the stability of complex isothermal reactors: I. The deficiency zero and deficiency one theorems,

Chemical Engineering Science42(1987), 2229–2268.and probably also somewhere in these great free online notes:

• Martin Feinberg,

Lectures On Reaction Networks, 1979.The deficiency one theorem is more complicated than the deficiency zero theorem, with weaker implications, but still powerful. Despite its name it applies to some cases of deficiency higher than one, too!

Right.

Right, that theorem is useless here.

I don’t know the tricks for solving this sort of problem, but some people do. There’s a whole industry of showing that random walks of various kinds will almost surely hit some set.

Continuous-time Markov processes where the Hamiltonian is a Dirichlet operator—both infinitesimal stochastic and self-adjoint—are basically generalizations of the heat equation. For example, when there are finitely many states, a Dirichlet operator is the same as the Laplace operator associated to some

weightedgraph. This is Problem 39 in Section 22.1 of the book Jacob and I are writing.But now you are talking about similar things for a situation with infinitely many states… and in some examples, for Hamiltonians that are not self-adjoint.

It sounds like in this situation you should be able to prove there aren’t any nontrivial equilibrium solutions, at least if a few reasonable conditions hold. I’d try argue something like this. Suppose that when the population goes to 1 it can never go back to 2, but has a nonzero probability per time of going to 0. Then in any equilibrium the probability of having population 1 must be zero (since otherwise the probability of having population 0 would grow).

But if there’s a nonzero probability per time of the population going from 2 to 1, this implies that in any equilibrium the probability of having population 2 must be zero ((since otherwise the probability of having population 1 would grow).

And so on.

Thanks a lot for that extensive answer!

Concerning the model with transitions and occurring at equal rates, I had asked:

I think I know how to do this now. For any population size , consider the next population size that you get as soon as a fission or death event happens. Since the rates are equal, this next population size is either or with equal probability. In this way, we obtain an old friend, namely the ordinary random walk on the integers, modulo the caveat that the walk never leaves the origin once it has arrived there. Due to the recurrence of this random walk, we end up at the origin eventually with probability one, no matter where we started out with. In other words, the above model ends up at extinction with probability one. If you like theatrical analogies, think of this as a mathematical proof showing that doomsday will strike us some day!

John wrote:

Countably many states shouldn’t be a problem — people study Laplace operators on countable graphs for example in geometric group theory. About self-adjointness, you’re right, but maybe we just need to be using the “right” scalar product in order for the Hamiltonian to become self-adjoint.

John wrote:

Beautiful! It feels good to have this nagging problem settled.

Tobias wrote:

Thanks. By the way, one way to show certain Markov processes have no equilibrium state where the probability of having population 1 vanishes is to copy the argument that if

on some region and is zero on the boundary of the region, then (under certain side conditions). The idea is that the Laplace equation

says that at any point is the average of its values on a small sphere around that point. So, if it’s zero on the boundary of the shape it should be zero everywhere. In the 1-dimensional discrete setting the Laplace equation becomes

Of course, this doesn’t prohibit the solution for but that solution isn’t ‘normalizable’. (That’s part of what I meant by ‘side conditions’.) Any normalizable solution has to be zero.

Right, there’s a theory of generalized Dirichlet operators on measure spaces, which allows you to adjust the measure and thus the inner product:

• M. Fukushima,

Dirichlet Forms and Markov Processes, North-Holland, Amsterdam, 1980.In the book, I wanted to apply this theory to the case of continuous-time Markov process with a finite space of states. If we allow ourselves to use a measure other than counting measure on we get new concepts of ‘self-adjoint’ and ‘infinitesimal stochastic’ operators on and It would be nice, and not too hard, to work out the consequences. But I never got around to it!

There’s also a further generalization of Dirichlet operators that allows them to have a skew-adjoint part as long as it’s dominated by the self-adjoint part:

• Zhi-Ming Ma and Michael Röckner,

Introduction to the Theory of (Non-Symmetric) Dirichlet Forms, Springer, Berlin, 1992.For general problems of this type, I would look at the theory of branching processes. (http://en.wikipedia.org/wiki/Branching_process) A branching process is like a very restricted kind of reaction network where the only reactions have one thing on the left.

is allowed, but no or etc is allowed. (Often branching processes are restricted to one type, but you can have multi-type ones.) As for tricks, I know two, and you’re both doing the first one already. That is, find the embedded discrete time Markov chain by ignoring the time it takes to get to the next state, and just look at transition probabilities of going from j to i at the next change. In all cases so far, j has been i-1 or i+1, but it doesn’t have to be.

The other trick I know is probability generating functions, which I bet you do know now I’ve reminded you!

One way in which branching processes can be more general than the reaction networks I’ve seen is that they can have an infinite number of reactions, eg

for every n. (Might be handy for fish which can lay millions of eggs.) This means that the process can ‘explode’, that is, make an infinite number of particles in a finite time. So a ‘non-explosion’ hypothesis is assumed. (I think finite number of reactions implies the hypothesis.) I mention this because of the technical difficulty in the recent blog post The Large-Number Limit for Reaction Networks (Part 3) about passing a limit through a derivative. I bet an exploding process would fail here! But I thought that if you looked at how the ‘non-explosion’ hypothesis is used in branching theory it might help prove the theorem.

Yes, ‘explosion’ is one of the things I worry about when studying the large-number limit, not only for passing the limit through the derivative, but also for showing that starting with a semiclassical family of states it remains semiclassical for a while as we evolve it. This amounts to showing that moments don’t grow huge or blow up. The moments of the population probably

doblow up even before the mean population blows up in a stochastic reaction network likeOver here, Tobias wrote:

Right.

It does, actually. We just need one species (for ‘amoeba’) and two transitions

with exactly the same rate constant if you want the rate equation to have equilibrium solutions. (Otherwise all solutions of the rate equation will go to infinity (if births outpace deaths) or zero (if deaths outpace births.)

I believe that when the rate constants are equal, the master equation will have the property that with probability 1, the population eventually reaches zero and never bounces back. I haven’t proved this.

Or, if you want something a bit less delicately balanced, you can do something like this:

This shows up in models of fish populations, where the last process represents fishing. For many choices of rate constants the rate equation should have a unique nonzero equilibrium solution… but also zero will be an equilibrium solution. I believe that in these cases, the master equation will have the property that the population eventually reaches zero with probability 1, and never bounces back.

Tobias Fritz wrote:

(I omit the possibility of extinction here because Graham pointed out that in this particular model death only occurs when there are at least 2 amoebas, so extinction is impossible.)

I haven’t tried to prove it, but that’s what I’d expect. I don’t find this disturbing, because the ‘peculiar’ phenomena you mention—near-total extinction, or shooting far above the equilibrium population—happen less and less frequently in the correct large-number limit. This is the limit where we keep increasing the equilibrium population while also counting the population in larger and larger units and also change the rate constants in the manner described in this paper.

If the equilibrium population is 1, it’s quite likely for an initial population with this equilibrium size to double in size. If the equilibrium population is 1 million, this is less likely to happen soon… though I believe the population is still certain to

eventuallydouble at some point.An example where extinction is a possibility: if you keep playing the lottery, eventually you either run out of money, die, or get lucky and win.

[…] A stochastic Petri net from population biology whose rate equation is the logistic equation; an equilibrium solution of the corresponding master equation, Network Theory Part 7 […]