I spent lots of time discussing this with James, and you can read bits of it written up in my Winter 2007 quantum gravity seminar notes, especially here:

• Week 12 (Jan. 30) – Classical, quantum and statistical mechanics as "matrix mechanics". In quantum mechanics we use linear algebra over the ring of complex numbers; in classical mechanics everything is formally the same, but we instead use the rig R^{min} = R ∪ {+∞} where the addition is min and the multiplication is +. As a warmup for bringing statistical mechanics into the picture – and linear algebra over yet another rig – we recall how the dynamics of particles becomes the statics of strings after Wick rotation. Blog entry.

• Week 13 (Feb. 6) – Statistical mechanics and deformation of rigs. Statistical mechanics (or better, "thermal statics") as matrix mechanics over a rig R^{T} that depends on the temperature T. As T → 0, the rig R^{T} reduces to R^{min} and thermal statics reduces to classical statics, just as quantum dynamics reduces to classical dynamics as Planck’s constant approaches zero. Tropical mathematics, idempotent analysis and Maslov dequantization. Blog entry.

Supplementary reading: Grigori L. Litvinov, The Maslov dequantization, idempotent and tropical mathematics: a very brief introduction. Also see the longer version here.

• Week 14 (Feb. 13) – An example of path-integral quantization: the free particle on a line (part 1). Blog entry.

• Week 15 (Feb. 20) – The free particle on a line (part 2). Showing the path-integral approach agrees with the Hamiltonian approach. Fourier transforms and Gaussian integrals. Blog entry.

]]>I imagine POMDP models either 1) need to be generalized to allow the agent to update its policies as time goes on, based on new data, or 2) need to be able to handle enough contingencies that any change in conditions is simply treated as another observation.

I hope you get what I mean. In case 1) we have actual *learning* with the passage of time as we run the model, while in case 2) all the learning must occur before we start running the model; the model needs to be sufficiently well-prepared to be already familiar with all situations it comes across.

Clearly approach 1) is more ambitious and more interesting. I wonder what various systems like self-driving cars actually do! Does one of Google’s cars actually learn as it drives, for example learn more about the probability that another car will cut in front of it as it drives along a particular highway you use to go do work, perhaps in a way that depends on the time of day? This could be good, but it’s probably harder to get right.

]]>Here’s the paper – http://www.cell.com/trends/cognitive-sciences/abstract/S1364-6613(13)00205-2

For POMDP-type models, based on what I understand, how does it handle varying probabilities? In an actual environment – agent interaction situation, the probability of the event is not always constant. How does that affect the optimal strategy?

]]>Since they’re describing it as a loop, I don’t think they’re claiming there’s a best starting-point for that loop. And they said ‘action’, not ‘reaction’. However, what you say is interesting.

]]>Stimulus-response organization of behavior may occur, but I have yet to find a good example. Even reflexes, when studied closely, only appear to be organized according to S-R concepts. If such S-R organization exists, then it is a highly derived trait and a rather rare exception.

]]>There was a fun talk about information theory in vision, and certainly a lot of people alluded to it. But the application of information theory people talked about most was to intercellular molecular communication!

]]>Cool! I hadn’t gotten into the discussion of hyperbolic discounting here because ‘discounting’ is not what matters to me in Tishby’s work. However, it’s nice to see people have thought about it.

]]>Tishby’s framework allows for hyperbolic discounting nicely, see http://www.dagstuhl.de/mat/Files/11/11131/11131.TishbyNaftali.Slides.pdf Information Gain Discounting slide.

]]>The book Active Vision by John M Findlay and Iain D Gilchrist, 2003, makes a strong case for vision being an active process rather than a passive one. Especially important are eye movements, which depend on what you’re trying to do, and what you’ve seen (and remembered) so far. Each saccade, you change what you see. The saccades even give you discrete time steps, more or less.

]]>