I’d be happy to get your feedback on these slides of the talk I’m giving the day after tomorrow:

• John Baez, Diversity, entropy and thermodynamics, 6 July 2012, Exploratory Conference on the Mathematics of Biodiversity, Centre de Recerca Matemàtica, Barcelona.

Abstract: As is well known, some popular measures of biodiversity are formally identical to measures of entropy developed by Shannon, Rényi and others. This fact is part of a larger analogy between thermodynamics and the mathematics of biodiversity, which we explore here. Any probability distribution can be extended to a 1-parameter family of probability distributions where the parameter has the physical meaning of ‘temperature’. This allows us to introduce thermodynamic concepts such as energy, entropy, free energy and the partition function in any situation where a probability distribution is present—for example, the probability distribution describing the relative abundances of different species in an ecosystem. The Rényi entropy of this probability distribution is closely related to the change in free energy with temperature. We give one application of thermodynamic ideas to population dynamics, coming from the work of Marc Harper: as a population approaches an ‘evolutionary optimum’, the amount of Shannon information it has ‘left to learn’ is nonincreasing. This fact is closely related to the Second Law of Thermodynamics.

This talk is rather different than the one I’d envisaged giving! There was a lot of interest in my work on Rényi entropy and thermodynamics, because Rényi entropies—and their exponentials, called the Hill numbers—are an important measure of biodiversity. So, I decided to spend a lot of time talking about that.

The question I’d have after the last slide is, “If Hill numbers are a better biodiversity measure than the Shannon index, can we say the same thing about relative biodiversity and evolutionary optima using Hill numbers?”

(I think we can, but I don’t remember where I wrote down my calculations, so I should just derive them again.)

In principle yes, because the exponential of an entropy is a Hill number.

The question, then, is what does the exponential of relative entropy mean? What is it an effective number of? The answer’s not immediately obvious, but I have some half-formed thoughts I should develop a bit before sharing.

I wonder – How should this very cool analysis involving Evolutionary Stable Strategies be applied to “probability learning” by students and “optimal foraging” by fish as described in chapter 11 of the book by Gallistel, “The Organization of Learning”?

Students in the classroom are said to have learned the probability of a reward occurring at a site in the demonstration, when based on previous experience the student’s guesses for each trial have a frequency per site that matches the probability of the reward occurring at each site established beforehand by the researcher.

Second, schools of fish divide themselves proportionally to the probability of reward coming from a site, displaying a Nash equilibrium.

About the demonstration of probability learning, Gallistel writes “[Students] were greatly surprised to be shown when the demonstration was over that the rat’s behavior [against whom they were competing to get the most reward] was more intelligent than their own. We did not lessen their discomfiture by telling them that if the rat chose under the same conditions they did – under a correction procedure whereby on every trial it ended up knowing which side was the rewarded side – it too would match the relative frequencies of its initial side choices to the relative frequencies of payoffs.”

In Shannon terms, for each site the students experienced an uncertainty -(p)ln(p) that the reward would occur at a particular site. But on occasions when they chose that site – and then saw that the reward occurred at a different site – they also experienced an opposing uncertainty (1-p)ln(1-p) associated with the reward occurring somewhere other than the chosen site. So there is uncertainty on each trial for the reward occurring at a site, and at the same time an uncertainty that it will Not occur at that site. The idea here would be to introduce positive and negative uncertainties, perhaps as Franklin applied positive and negative numbers to electric charge.

But the rat was not given the opposing uncertainty (1-p)ln(1-p) when it chose a site that was not rewarded on the trial, and was simply operating on the uncertainty -(p)ln(p) associated with the reward occurring at the site.

Much has been written about how probability learning is not maximizing utility. (In the rigged demonstration, the rat was set up to maximize utility.) But there must be a way to describe how a group of probability learners do achieve a Nash equilibrium, as do the fish.

By the way, if the two opposing uncertainties are added instead of subtracted, the resulting curve for a two site demonstration is symmetric around 50/50, which seems more intuitive than just the Shannon term -(p)ln(p) alone. The logic seems to be that we are dealing with a corpuscular object, which conforms to the rule that it can only be in one place at a time. This reminds me of the Born rule, where the wave function might represent the possibility of a corpuscular object being at a location, while its complex conjugate might represent the possibility of its NOT being anywhere else. Logically ANDing (numerically multiplying) the two statements, we get “the possibility that it will be here AND the possibility that it will be nowhere else other than here, together, establish the probability of a corpuscular object being here.” Is this where the Born rule comes from?

Olivier Penacchio pointed out a very ambitious paper trying to use free energy and Bayesian ideas to explain how the brain works. Does anyone here know enough neurobiology to have an informed opinion about how seriously we should take this paper?

Abstract. This article reviews a free-energy formulation that advances Helmholtz’s agenda to find principles of brain function based on conservation laws and neuronal energy. It rests on advances in statistical physics, theoretical biology and machine learning to explain a remarkable range of facts about brain structure and function. We could have just scratched the surface of what this formulation offers; for example, it is becoming clear that the Bayesian brain is just one facet of the free-energy principle and that perception is an inevitable consequence of active exchange with the environment. Furthermore, one can see easily how constructs like memory, attention, value, reinforcement and salience might disclose their simple relationships within this framework.

As usual, clicking on the paper’s title gives a free version of the paper, while clicking on the journal name gives the official, stable version!

Thanks, fixed. Btw, Google seems to have gotten much worse in its behavior with PDF files. It became hard and now almost impossible to find the actual URL of the PDF file; instead it gives a long URL containing information about what Google thinks about that PDF file. I refuse to let my URL’s for links rely on Google in that way, so until they fix that or I discover an easy workaround, I’m using another web search engine.

The problem in this case arose from this bigger problem…

I have that problem when I right-click on a Google search result and try to copy the URL to the PDF. My solution is to click on the search result and view the PDF in a browser window. I copy the unobfuscated URL from the location bar at the top of the window. Not sure if this will work for you.

Sometimes that works and sometimes it doesn’t! It may depend on what country I’m in… it seems to have gotten worse lately, but maybe that’s because I’ve been in France and Spain rather than Singapore.

This really pisses me off. What’s Google trying to do, get lazy people to incorporate Google-specific data in all their links? Take over the internet somehow? Or is it just a stupid bug?

I’m starting to use another web search engine, DuckDuckGo, for PDF file searches. Anyone know other good solutions… or what Google is up to here?

You can use HTML in your comments. You can also use LaTeX, like this: $latex E = m c^2 $. The word 'latex' comes right after the first dollar sign, with a space after it. Cancel reply

You need the word 'latex' right after the first dollar sign, and it needs a space after it. Double dollar signs don't work, and other limitations apply, some described here. You can't preview comments here, but I'm happy to fix errors.

The question I’d have after the last slide is, “If Hill numbers are a better biodiversity measure than the Shannon index, can we say the same thing about relative biodiversity and evolutionary optima using Hill numbers?”

(I think we can, but I don’t remember where I wrote down my calculations, so I should just derive them again.)

In principle yes, because the exponential of an entropy is a Hill number.

The question, then, is what does the exponential of relative entropy mean? What is it an effective number of? The answer’s not immediately obvious, but I have some half-formed thoughts I should develop a bit before sharing.

How do you define temperature for a non-exponential family distribution such as a uniform distribution in [0, 1]?

I wonder – How should this very cool analysis involving Evolutionary Stable Strategies be applied to “probability learning” by students and “optimal foraging” by fish as described in chapter 11 of the book by Gallistel, “The Organization of Learning”?

Students in the classroom are said to have learned the probability of a reward occurring at a site in the demonstration, when based on previous experience the student’s guesses for each trial have a frequency per site that matches the probability of the reward occurring at each site established beforehand by the researcher.

Second, schools of fish divide themselves proportionally to the probability of reward coming from a site, displaying a Nash equilibrium.

About the demonstration of probability learning, Gallistel writes “[Students] were greatly surprised to be shown when the demonstration was over that the rat’s behavior [against whom they were competing to get the most reward] was more intelligent than their own. We did not lessen their discomfiture by telling them that if the rat chose under the same conditions they did – under a correction procedure whereby on every trial it ended up knowing which side was the rewarded side – it too would match the relative frequencies of its initial side choices to the relative frequencies of payoffs.”

In Shannon terms, for each site the students experienced an uncertainty -(p)ln(p) that the reward would occur at a particular site. But on occasions when they chose that site – and then saw that the reward occurred at a different site – they also experienced an opposing uncertainty (1-p)ln(1-p) associated with the reward occurring somewhere other than the chosen site. So there is uncertainty on each trial for the reward occurring at a site, and at the same time an uncertainty that it will Not occur at that site. The idea here would be to introduce positive and negative uncertainties, perhaps as Franklin applied positive and negative numbers to electric charge.

But the rat was not given the opposing uncertainty (1-p)ln(1-p) when it chose a site that was not rewarded on the trial, and was simply operating on the uncertainty -(p)ln(p) associated with the reward occurring at the site.

Much has been written about how probability learning is not maximizing utility. (In the rigged demonstration, the rat was set up to maximize utility.) But there must be a way to describe how a group of probability learners do achieve a Nash equilibrium, as do the fish.

By the way, if the two opposing uncertainties are added instead of subtracted, the resulting curve for a two site demonstration is symmetric around 50/50, which seems more intuitive than just the Shannon term -(p)ln(p) alone. The logic seems to be that we are dealing with a corpuscular object, which conforms to the rule that it can only be in one place at a time. This reminds me of the Born rule, where the wave function might represent the possibility of a corpuscular object being at a location, while its complex conjugate might represent the possibility of its NOT being anywhere else. Logically ANDing (numerically multiplying) the two statements, we get “the possibility that it will be here AND the possibility that it will be nowhere else other than here, together, establish the probability of a corpuscular object being here.” Is this where the Born rule comes from?

Olivier Penacchio pointed out a very ambitious paper trying to use free energy and Bayesian ideas to explain how the brain works. Does anyone here know enough neurobiology to have an informed opinion about how seriously we should take this paper?

• Karl Friston, The free-energy principle: a rough guide to the brain?,

Trends in Cognitive Science13(2009), 293 – 301.As usual, clicking on the paper’s title gives a free version of the paper, while clicking on the journal name gives the official, stable version!

The link to the free version is missing http://, so browsers interpret it wrong.

Thanks, fixed. Btw, Google seems to have gotten

muchworse in its behavior with PDF files. It became hard and now almost impossible to find the actual URL of the PDF file; instead it gives a long URL containing information about what Google thinks about that PDF file. I refuse to let my URL’s for links rely on Google in that way, so until they fix that or I discover an easy workaround, I’m using another web search engine.The problem in this case arose from this bigger problem…

I have that problem when I right-click on a Google search result and try to copy the URL to the PDF. My solution is to click on the search result and view the PDF in a browser window. I copy the unobfuscated URL from the location bar at the top of the window. Not sure if this will work for you.

Sometimes that works and sometimes it doesn’t! It may depend on what country I’m in… it seems to have gotten worse lately, but maybe that’s because I’ve been in France and Spain rather than Singapore.

This really pisses me off. What’s Google trying to do, get lazy people to incorporate Google-specific data in all their links? Take over the internet somehow? Or is it just a stupid bug?

I’m starting to use another web search engine, DuckDuckGo, for PDF file searches. Anyone know other good solutions… or what Google is up to here?

There seem to be some ways to disable this behavior in Firefox and Chrome, which I haven’t tried yet.

Thanks a lot! Of the solutions offered, this add-on for Firefox seems to be working for me.

The page says this add-on only works if you disable ‘Google Instant’, which is that feature that tries to guess what you’re typing as you type it.

[…] The Mathematics of Biodiversity (Part 5)(johncarlosbaez.wordpress.com) […]