Here is one such pair if anyone wants to verify (Renyi for all beta and p majorizes q):
(0.702035380032679, 0.25269600971504746, 0.04526861025227358)
(0.6012357521664494, 0.32989515938159686, 0.06886908845195372)

Oscar was asking if this behavior—all Rényi entropies of p bigger than of q, but p not majorizing q—is unusual. Maybe without much extra work you could figure out roughly what ‘percentage’ of pairs (p,q) display this behavior, say for probability distributions on a 3-element set?

If you’ve already written code to decide if a given pair of probability distributions displays this behavior, then you just need to

1) decide what measure you’ll use to define the ‘percentage’ of pairs of probability distributions that display a given behavior,

2) figure out how to estimate that percentage, either using Monte Carlo estimation or some other procedure.

You can think of a probability distribution on a 3-element set as a triple

such that

and

The space of these is an equilateral triangle. Since the problem involves majorization and (implicitly) convex-linear maps, the symmetries are just the obvious symmetries of the triangle. So, it makes some sense to just use the obvious measure on this triangle, which is proportional to

So for Monte Carlo, you can just randomly pick in a uniformly distributed way, then randomly pick in a uniformly distributed way, then let .

As you know, there’s also another measure on this equilateral triangle, coming from the Fisher information metric. That would give a different answer!

In a certain analogy to thermodynamics, which I used to relate Rényi entropy to free energy, would correspond to temperatures below absolute zero. At negative temperatures, a system is more likely to be found in states of high energy than states of low energy. Negative temperatures are not completely absurd, but they only make sense for systems with an upper bound on their energy—just as positive temperatures only make sense for systems with a lower bound on their energy.

For including code you surround the code by something like this:

[ sourcecode language="python"]

[ /sourcecode]

except without the space after the [. I don’t know what range of languages it takes. It seems to do something halfway reasonable if you leave the language choice out.

By the way, there are simple formulas for the Rényi entropy at and which allow us to understand exactly what’s going on at these points. Oscar used them to construct his example. At we get the min-entropy and at we get the max-entropy, which is the logarithm of the number of such that .

Since both of Oscar’s probability distributions have the same max-entropy, the curve you graphed approaches zero as just as it looks.

It approaches a certain negative value as though the eye could be fooled into thinking it approaches zero.