I’m working with Nate Osgood and other folks to develop better modeling tools for epidemiologists. Right now we’re trying to develop a category-based framework for agent-based models. It’s a bit tough since many different techniques are used in such models, without yet an overarching discipline—Nate likened it to the ‘wild West’. I have a lot to learn, but I thought I should start keeping notes on our conversations.
Agent-based models involve multiple ‘agents’ that have ‘states’ which change over time, often including location in some sort of ‘space’, and which interact via various ‘networks’. Our goal is to create convenient, flexible software for creating agent-based models, composing larger ones out of smaller parts, refining existing models, running the models, and viewing them as they run in many different ways.
There is already software for doing these things, such as AnyLogic, NetLogo and Repast. Nate is familiar with it, since his job is building agent-based models. But he’s dissatisfied with it for many reasons. We want to make better software by first analyzing the whole problem with the help of math. All those quoted words—‘agents’, ‘states’, ‘space’, ‘networks’—need to be clarified in a way that’s quite general but also practical.
I’m still learning about all this stuff. Here’s one thing I learned that makes me happy. The internal dynamics of agents—that is, how their states change with time—is often described using two methods:
Stock and flow diagrams are good for describing continuous quantities that evolve according to ordinary differential equations, while state diagrams are good for describing discrete quantities that evolve in steps. (In ether case the evolution could be deterministic or stochastic.)
We’ve already worked out the category theory of stock and flow diagrams, and used it to create software for working with such diagrams:
• John Baez, Xiaoyan Li, Sophie Libkind, Nathaniel D. Osgood and Eric Redekopp, A categorical framework for modeling with stock and flow diagrams, to appear in Mathematics for Public Health, Springer.
• John Baez, Xiaoyan Li, Sophie Libkind, Nathaniel Osgood and Evan Patterson, Compositional modeling with stock and flow diagrams, to appear in Proceedings of Applied Category Theory 2022.
So, it seems we should work out the category theory of state diagrams! Luckily they are similar to stock and flow diagrams, so a lot of the same math should apply: decorated cospans, the operad for undirected wiring diagrams, etc. So, it shouldn’t be a huge task.
However, there are other issues we need to deal with. For example, in a ‘hierarchical state diagrams’ there may be states within a single state. For example in the state ‘infected’ there may be states such as ‘diagnosed’ and ‘undiagnosed’, and in ‘diagnosed’ there may be many states describing whether and how someone has been treated.
Also, there may be variables defined only in a given state, whose dynamics are governed by some stock and flow diagram.
So, we ultimately need a good mechanism for building stock and flow diagrams and state diagrams hierarchically, and mixing the two. Maybe we shouldn’t even treat them as two distinct kinds of diagram, but rather as two ways of using some more general kind of diagram!
I’ll try to take one step at a time. Getting a good category theoretic treatment of state diagrams should be pretty quick.
Azimuth 
I have several thoughts about this:
0. It’s an exciting and interesting direction—not just for agent-based models, but modeling complex systems in general.
1. State diagrams are a lot like Petri nets: it might be productive to not just use decorated cospans, but actually treat the diagram as a finite presentation of a free SMC, a la Jade Master.
2. The nested state diagram should probably be interpreted as a pullback in the semantics, where:
2a. the apex of the cospan is the Petri net with two places Inside and Outside and two transitions Enter and Exit.
2b. One side of the cospan is the diagram with the nested diagram removed and just one ordinary state in its place. That leg of the cospan maps that one ordinary state to Inside, and everything else to Outside.
2c. The other side of the cospan is just the nested diagram, with everything outside removed. That leg of the cospan maps the special “initial state” to Outside, and everything else to Inside.
2d. It seems that there is a convention to implicitly assume that transitions are possible out of any “nested state”. To get a nice categorical treatment this needs to be made explicit, by adding a special “final state” which also maps to Outside, and which receives a transition from every nested state.
3. For a single semantic domain that can receive both state diagrams and stock-and-flow diagrams, I suggest looking into the formalism of Stochastic Dynamic Colored Petri Nets, which may sound quite arbitrary and Rube-Goldberg-esque, but is bisimilar to a bunch of other notions.
Thanks very much for your enthusiasm and for taking a crack at the problem! If state diagrams are secretly just Petri nets, then I’m in luck, since as you mentioned, there are these papers:
• John Baez and Jade Master, Open Petri nets, Mathematical Structures in Computer Science 30 (2020), 314–341.
• John Baez, Fabrizio Genovese, Jade Master and Michael Shulman, Categories of nets, in 2021 36th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS), IEEE, Rome, Italy, 2021, pp. 1–13.
I’ve thought a bit about colored Petri nets and hierarchical Petri nets, and a lot about stochastic Petri nets. So, if Petri nets are the way to go, I’ll be quite happy.
But I’m not sure that state diagrams are secretly just Petri nets! I’m just very ignorant about state diagrams at this point. I’ll try to learn more soon, and continue this thread with a bit more knowledge.
This paper might be of interest, proposing a formal semantics for state diagrams as colored Petri nets for model-checking purposes: https://dl.acm.org/doi/pdf/10.1007/s00165-016-0388-9
Thanks, that looks like it could be very useful to me, since I know categorical work on colored Petri nets and this paper is trying to use them as a mathematical foundation for UML state machines. (I’m not sure if those are the same as ‘state charts’ and/or ‘state diagrams’.)
Sounds like a job for hierarchical string diagrams (see here). That source also contains some hierarchical string diagram rewriting theory, which may be useful to model the dynamics.
I had a look at agent based models in economics recently for a side-project of mine, and I though the survey from Axtel and Farmer (the latter is a physicists actually) was quite good as it describes advances of ABMs in other fields too:
Click to access JEL-v2.0.pdf
There’s a very interesting survey (for me at least) on Agent-Based Macroeconomics specifically by Dawid and Delli Gatti but i don’t think that would be of general interest. I think in both papers Netlogo is mentioned a few times, among other tools.
In my opinion Multi-Agent Reinforcement Learning (MARL) and Game Theory provide the most effective theoretical (but also practical) framework for understanding Agent-Based models. Here each agent is also trying to learn -by interacting with the environment and the other agents- the policy that maximizes its expected (discounted) cumulative future reward. See for example
https://arxiv.org/abs/2011.00583
On the commercial software side, for stock and flow diagrams Simulink is the industry standard, especially in automotive and aerospace (disclaimer: I work for MathWorks, but I think this statement is accurate). Simulink is also used for MARL sometimes. For state diagrams, Stateflow also is popular but i know less about that one, though i’ve used it too. Both environments are fully hierarchical.
Looking forward to see more posts on the subject!
Actually come to think of it, it would be more accurate to say that Simulink is a signal flow diagram, which is a little different, perhaps more general, as what travels is really information (signals) and does not have to represent a quantity that must be conserved.
That’s interesting! I may want some help at some point. I get the feeling that agent-based models in economics are much more focused on learning and game theory than agent-based models in epidemiology. Nate Osgood has shown me agent-based models of COVID19 and also diseases affecting populations of deer in Canada, and the agents in these models are not particularly trying to optimize anything or learn anything. Of course that’s an oversimplification, but the spread of viruses, the treatment of patients, etc. impose a dynamics that’s quite complex and interesting even without ‘learning’.
Sure. Not sure how much I can actually help but, … yeah anytime! Next year I am going to seriously look into multi-agent RL anyway, for work purposes.
Also, yes I don’t think there’s really any “learning” in these epidemiological model, though some might feature some evolution, which is actually a form of learning.