John D. Cook, Greg Egan, Dan Piponi and I had a fun mathematical adventure on Twitter. It started when John Cook wrote a program to compute the probability distribution of distances where and were two randomly chosen unit quaternions:
• John D. Cook, How far is xy from yx on average for quaternions?, 5 July 2018.
Three things to note before we move on:
• Click the pictures to see the source and get more information—I made none of them!
• We’ll be ‘randomly choosing’ lots of points on spheres of various dimensions. Whenever we do this, I mean that they’re chosen independently, and uniformly with respect to the unique rotation-invariant Borel measure that’s a probability measure on the sphere. In other words: nothing sneaky, just the most obvious symmetrical thing!
• We’ll be talking about lots of distances between points on the unit sphere in dimensions. Whenever we do this, I mean the Euclidean distance in , not the length of the shortest path on the sphere connecting them.
If you look at the histogram above, you’ll see the length is between 0 and 2. That’s good, since and are on the unit sphere in 4 dimensions. More interestingly, the mean looks bigger than 1. John Cook estimated it at 1.13.
Greg Egan went ahead and found that the mean is exactly
He did this by working out a formula for the probability distribution:
All this is great, but it made me wonder how surprised I should be. What’s the average distance between two points on the unit sphere in 4 dimensions, anyway?
Greg Egan worked this out too:
So, the mean distance for two randomly chosen unit quaternions is
The mean of is smaller than this. In retrospect this makes sense, since I know what quaternionic commutators are like: for example the points at the ‘north and south poles’ of the unit sphere commute with everybody. However, we can now say the mean of is exactly
times the mean of and there’s no way I could have guessed that.
While trying to get a better intuition for this, I realized that as you go to higher and higher dimensions, and you standing at the north pole of the unit sphere, the chance that a randomly chosen other point is quite near the equator gets higher and higher! That’s how high dimensions work. So, the mean value of should get closer and closer to And indeed, Greg showed that this is true:
The graphs here show the probability distributions of distances for randomly chosen pairs of points on spheres of various dimensions. As the dimension increases, the probability distribution gets more sharply peaked, and the mean gets closer to
Here’s the general formula for the distribution, with plots for n=2,…,10. The mean distance does tend to √2, and the mean of the squared distance is always exactly 2, so the variance tends to zero.
But now comes the surprising part.
Dan Piponi looked at the probability distribution of distances in the 4-dimensional case:
and somehow noticed that its moments
when n is even, are the Catalan numbers!
Now if you don’t know about moments of probability distributions you should go read about those, because they’re about a thousand times more important than anything you’ll learn here.
And if you don’t know about Catalan numbers, you should go read about those, because they’re about a thousand times more fun than anything you’ll learn here.
So, I’ll assume you know about those. How did Dan Piponi notice that the Catalan numbers
were the even moments of this probability distribution? Maybe it’s because he recently managed to get ahold of Richard Stanley’s book on Amazon for just $11 instead of its normal price of $77.
(I don’t know how that happened. Some people write 7’s that look like 1’s, but….)
Anyway, you’ll notice that this strange phenomenon is all about points on the unit sphere in 4 dimensions. It doesn’t seem to involve quaternions anymore! So I asked if something similar happens in other dimensions, maybe giving us other interesting sequences of integers.
Greg Egan figured it out, and got some striking results:
Here d is the dimension of the Euclidean space containing our unit sphere, and Egan is tabulating the nth moment of the probability distribution of distances between two randomly chosen points on that sphere. The gnarly formula on top is a general expression for this moment in terms of the gamma function.
The obvious interesting feature of this table is that only for d = 2 and d = 4 rows are all the entries integers.
But Dan made another great observation: Greg left out the rather trivial d = 1 row, and that all the entries of this row would be integers too! Even better, d = 1, 2, and 4 are the dimensions of the associative normed division algebras: the real numbers, the complex numbers and the quaternions!
This made me eager to find a proof that all the even moments of the probability distribution of distances between points on the unit sphere in are integers when is an associative normed division algebra.
The first step is to notice the significance of even moments.
First, we don’t need to choose both points on the sphere randomly: we can fix one and let the other vary. So, we can think of the distance
as a function on the sphere, or more generally a function of And when we do this we instantly notice that the square root is rather obnoxious, but all the even powers of the function are polynomials on
Then, we notice that restricting polynomials from Euclidean space to the sphere is how we get spherical harmonics, so this problem is connected to spherical harmonics and ‘harmonic analysis’. The nth moment of the probability distribution of distances between points on the unit sphere in is
where we are integrating with respect to the rotation-invariant probability measure on the sphere. We can rewrite this as an inner product in namely
where 1 is the constant function equal to 1 on the whole sphere.
We’re looking at the even moments, so let n = 2m. Now, why should
be an integer when d = 1, 2 and 4? Well, these are the cases where the sphere is a group! For d = 1,
is the multiplicative group of unit real numbers, For d = 2,
is the multiplicative group of unit complex numbers. And for d = 4,
is the multiplicative group of unit quaternions.
These are compact Lie groups, and of a compact Lie group is very nice. Any finite-dimensional representation of a compact Lie group gives a function called its character, given by
And it’s well-known that for two representations and the inner product
is an integer! In fact it’s a natural number: just the dimension of the space of intertwining operators from to So, we should try to prove that
is an integer this way. The function 1 is the character of the trivial 1-dimensional representation, so we’re okay there. What about
Well, there’s a way to take the mth tensor power of a representation : you just tensor the representation with itself times. And then you can easily show
So, if we can show is the character of a representation, we’re done: will be one as well, and the inner product
will be an integer! Great plan!
Unfortunately, is not the character of a representation.
Unless is the completely silly 0-dimensional representation we have
where is the identity element of But suppose we let be the distance of from the identity element—the natural choice of ‘north pole’ when we make our sphere into a group. Then we have
So can’t be a character. (It’s definitely not the character of the completely silly 0-dimensional representation: that’s zero.)
But there’s a well-known workaround. We can work with virtual representations, which are formal differences of representations, like this:
The character of a virtual representation is defined in the obvious way
Since the inner product of characters of two representations is a natural number, the inner product of characters of two virtual representations will be an integer. And we’ll be completely satisfied if we prove that
is an integer, since it’s obviously ≥ 0.
So, we just need to show that is the character of a virtual representation. This will easily follow if we can show itself is the character of a virtual representation: you can tensor virtual representations, and then their characters multiply.
So, let’s do it! I’ll just do the quaternionic case. I’m doing it right now, thinking out loud here. I figure I should start with a really easy representation, take its character, compare that to our function and then fix it by subtracting something.
Let be the spin-1/2 representation of which just sends every matrix in to itself. Every matrix in is conjugate to one of the form
so we can just look at those, and we have
On the other hand, we can think of as a unit quaternion, and then
where now stands for the quaternion of that name! So, its distance from 1 is
and if we square this we get
So, we’re pretty close:
In particular, this means is the character of the virtual representation
where is the 1d trivial rep and is the spin-1/2 rep.
So we’re done!
At least we’re done showing the even moments of the distance between two randomly chosen points on the 3-sphere is an integer. The 1-sphere and 0-sphere cases are similar.
But course there’s another approach! We can just calculate the darn moments and see what we get. This leads to deeper puzzles, which we have not completely solved. But I’ll talk about these next time, in Part 2.