Structured populations:
Mixed strategies
by Christoph Hauert, Version 1.0, April 2004.
 Location:
 VirtualLabs
 » Cooperation
 » Structured populations  mixed strategies
In wellmixed populations the equilibrium configuration does not depend on whether the individuals adopt pure strategies (to cooperate or to defect) or mixed strategies (to cooperate with a certain probability). All that matters is the frequency of the two behavioral patterns when averaging over many random encounters. This ambiguity is resolved in spatially structured populations where individuals interact only within a limited neighborhood. Spatial structures are modeled by arranging individuals on a lattice. The lattice geometry can be either square with a neighborhood size of N = 4 or N = 8, triangular with N = 3 or hexagonal with N = 6. Whenever a site x gets updated, the focal individual and all its neighbors compete to repopulate the site. Their reproductive success depends on their performance, i.e. their average payoff, in interactions with their respective neighbors. If the individual in x cooperates with probability p and the neighboring y with probability q, then the payoff for x amounts to P_{p} = p q R + p (1  q) S + (1  p) q T + (1  p) (1  q) P and similarly for y to P_{q} = p q R + p (1  q) S + (1  p) q T + (1  p)(1  q) P. Note that this calculation of the payoffs assumes that individuals interact frequently such that payoffs from single interactions average out. In contrast, if the performance depends only on a single interaction, then an individual with strategy p would obtain against strategy q the payoffs R, S, T or P with probabilities p q, p (1  q), (1  p) q and (1  p)(1  q), respectively. In that case the fluctuations arising from the probabilistic payoffs destroy all spatial correlations and yield the same results as wellmixed populations.
The performance P_{x} of x is then determined by averaging the payoffs from interactions with all its neighbors. The neighbor y succeeds in populating the focal site x with a probability proportional to
where k denotes a noise term which introduces an interesting form of errors since worse performing individuals may still manage to reproduce with a small probability. The offspring inherits the parental strategy p but with a small probability a mutation occurs changing the offspring strategy to p + s where s is a Gaussian distributed random variable with mean zero and small standard deviation. For small mutations, the difference in payoffs P_{y}  P_{x} becomes small and the generalization of the replicator dynamics to spatial settings as introduced for structured populations with pure strategies no longer performs well. For this reason we chose the above, slighlty more complicated rule to determine the reproductive success for mixed strategies. With this rule, small differences in payoffs are amplified  at least for reasonably small k. The remaining part of the update procedure is identical to the pure strategy case: with probability p_{x} = (1  w_{y1})(1  w_{y2}) … (1  w_{yN}) all neighbors y_{i} fail to reproduce and the focal individual succeeds in placing its own offspring in site x. Otherwise, with probability 1  p_{x}, one neighbor takes over the focal site. The relative probability for success of neighbors y is w_{y}/w where w = w_{1} + w_{2} + … + w_{N}.
Updating of the lattice can be synchronous referring to populations with discrete nonoverlapping generations or asynchronous for populations with overlapping generations in continuous time. For synchronous updates, all individuals interact and accumulate payoffs and then everbody attempts to reproduce with successrates relative to each individuals payoff within its neighborhood. For asynchronous updates only a single randomly selected focal site gets updated at a time: first the payoffs of the selected individual and all its neighbors are determined and then they compete to repopulate the focal site as outlined above.
Dynamical regimes
The following examples illustrate and highlight different relevant scenarios but at the same time they are meant as suggestions and starting points for further exploring and experimenting with the dynamics of the system. If your browser has JavaScript enabled, the following links open a new window containing a running lab that has all necessary parameters set as appropriate.
Legend  Time evolution of the propensity to cooperate in structured populations with individuals adopting mixed strategies and engaging in prisoner's dilemma and snowdrift interactions.
 

Space promotes cooperation:
 
Space inhibits cooperation:
 
Space promotes cooperation:

VirtualLab
Along the bottom of the VirtualLab applet are several buttons arranged to control the execution and the speed of the simulations. Of particular importance are the Param button and the data views popup list on top. The former opens a panel that allows to set and change various parameters concerning the game as well as the population structure, while the latter displays the simulation data in different ways.
Color code:  Maximum  Minimum  Mean 

Strategy code:  Defect  Cooperate 

Payoff code:  Low  High 

Note: In spatially structured populations the shade of grey indicates the individual's readiness to cooperate with black indicating defection and white full cooperation. Payoffs are similarly coded in shades of grey where black indicates low and white high payoffs.
Controls  
Params  Pop up panel to set various parameters. 

Views  Pop up list of different data presentations. 
Reset  Reset simulation 
Run  Start/resume simulation 
Next  Next generation 
Pause  Interrupt simulation 
Slider  Idle time between updates. On the right your CPU clock determines the update speed while on the left updates are made roughly once per second. 
Mouse  Mouse clicks on the graphics panels start, resume or stop the simulations. 
Data views  
Structure  Strategy  Snapshot of the spatial arrangement of strategies. 

Mean frequency  Time evolution of the strategy frequencies. 
Histogram  Frequency distribution of mixed strategies 
Structure  Fitness  Snapshot of the spatial distribution of payoffs. 
Mean Fitness  Time evolution of the mean payoff of each strategy together with the average population payoff. 
Histogram  Fitness  Histogram of payoffs for each strategy. 
Game parameters
The list below describes only the few parameters related to the Prisoner's Dilemma, Snowdrift and HawkDove games. Follow the link for a complete list and detailed descriptions of all other parameters such as spatial arrangements or update rules on the player and population level.
 Reward:
 reward for mutual cooperation.
 Temptation:
 temptation to defect, i.e. payoff the defector gets when matched with a cooperator.
 Sucker:
 sucker's payoff which denotes the payoff the cooperator gets when matched with a defector.
 Punishment:
 punishment for mutual defection.
 Init mean, init sdev:
 initial fractions of cooperators and defectors. If they do not add up to 100%, the values will be scaled accordingly. Setting the fraction of cooperators to 100% and of defectors to zero, then the lattice is initialized with a symmetrical configuration suitable for observing evolutionary kaleidoscopes.
 Mutation rate:
 the probability for a mutation to occur whenever an individual reproduces. This parameter is found on the Misc parameter tab.
 Mutation type, mutation sdev:
 mutations can occur either uniformly distributed in the interval [0,1] or modify the parental strategy by a Gaussian distributed random variable with mean zero and the specified standard deviation. This parameter is found on the Misc parameter tab.