Evolution of cooperation in costly institutions exhibits Red Queen and Black Queen dynamics in heterogeneous public goods

Well-mixed population

As shown in the Methods section, in a well-mixed population, the model can be described in terms of the replicator-mutator dynamics. I begin, by a case where the quality of the two resources are similar, r₁ = r₂ = r, and plot the frequency (solid blue), the average payoffs from the game (dashed red), and the amplitude of fluctuations (dotted blue) for different strategies in Fig. 1a–d. Here, the replicator dynamic is solved starting from a uniform initial condition in which all the strategies’ initial frequency is equal. The results of simulations in finite populations are in good agreement with the replicator dynamics results (see Supplementary Note 2 and Supplementary Figs. 1, 2, 3, and 4 for comparison to simulations). Throughout this manuscript I fix c = 1.

For small enhancement factors r, the dynamics settle in a fixed point where only defectors in the free institution survive. The advantage of the costly institution becomes apparent as r increases beyond r^* = 1 + c_g, such that the maximum possible payoff of the costly institution, which is achieved when nobody defects and is equal to r − 1 − c_g becomes positive. As shown in the Methods, a focal defector in a group composed of ({n}_{C}^{1}) costly cooperators and ({n}_{D}^{1}) other costly defectors (and ({n}_{C}^{2}+{n}_{D}^{2}) individuals who prefer the free resource), obtains a payoff of ({n}_{C}^{1}r/({n}_{D}^{1}+{n}_{C}^{1}+1)-{c}_{g}). This payoff becomes negative for a small enough value of ({n}_{C}^{1}) (or a large enough value of ({n}_{D}^{1})). Since groups with a small number of costly cooperators are drawn with a high probability when ({rho }_{C}^{1}) is small (these probabilities can be derived in terms of multinational coefficients, see the Methods), the average payoff of a costly defector remains negative in this regime (for instance, given at the transition ({rho }_{C}^{1}approx nu), a mutant costly defector finds itself in a group with no costly cooperator with probability ({(1-{rho }_{C}^{1})}^{g-1}approx {(1-nu )}^{g-1}), which is close to 1 for low mutation rates, and pays a pure cost of −c_g). On the other hand, a costly cooperator’s payoff is equal to (({n}_{C}^{1}+1)r/({n}_{D}^{1}+{n}_{C}^{1}+1)-{c}_{g}-1), which for small enough ({n}_{D}^{1}) becomes positive. As in this region, the frequency of costly defectors, ({rho }_{D}^{1}), is small, such group compositions occur with a high probability (at the transition, ({rho }_{D}^{1}approx nu), and thus the probability that a costly cooperator joins a group with no costly defectors is ({(1-{rho }_{D}^{1})}^{g-1}approx {(1-nu )}^{g-1}), which is close to 1 for low mutation rates). Consequently, the average payoff of costly cooperators from the game becomes positive, and thus, larger than the dominant non-costly defectors’ payoff (who receive a payoff of zero). Consequently, the frequency of costly cooperators rapidly increases at r^*. However, due to the rapid increase in the frequency of costly cooperators at r^*, the probability of formation of such mixed groups increases, and costly defectors start to appear in the system. Further increasing r in this region, the frequency of costly cooperators and costly defectors increases at the expense of non-costly defectors.

As r increases above a second threshold, cyclic fluctuations set in, and the dynamics settle in a periodic orbit. An example of this periodic orbit is presented in Fig. 2a, b. Interestingly, the average payoff of costly cooperators, costly defectors, and non-costly defectors in this region remains close to zero despite the evolution of cooperation. Although individuals constantly update their strategy to overcome others, no strategy wins in the evolution. Instead, individuals engage in a winnerless red queen dynamic. The game payoffs of costly cooperators and costly defectors fluctuate around zero (which is equal to the game payoff of non-costly defectors). The dynamics of the system in this regime resembles the frequency-dependent selection in the host-parasite evolution, coined the red queen dynamic based on the fact that no matter how much they run, all end up in the same place^53,54. On this basis, I call this periodic orbit the red queen periodic orbit.

The existence of a costly institution can facilitate the evolution of cooperation in its competing free institution too. As the amplitude of fluctuations increases, episodes where most of the individuals prefer the costly institution occur. During these episodes, ({rho }_{D}^{2}) drops to a small value. Consequently, the probability that a mutant non-costly cooperator finds itself in a group devoid of non-costly defectors ({(1-{rho }_{D}^{2})}^{g-1}), increases. In such groups, non-costly cooperators receive a payoff of r−1, which is larger than the payoff of all the other strategies and outcompete other strategies. At this point, a second periodic orbit emerges in which cooperation in both the costly and free institutions evolves. The evolution of cooperation in the free institution can, in turn, have a positive impact on cooperation in the costly institution. This is the case because above the point where cooperation in the free institution evolves, the frequency of individuals who prefer the free institution starts to increase by increasing r. This effect decreases the frequency of those who prefer the costly institution and its effective size. This decreases the mixing probability between costly cooperators and costly defectors and increases the costly cooperators’ payoffs. Consequently, a functional complementation between cooperators with different game preferences emerges, which is reminiscent of a black queen dynamics in which different types crucially depend on each other for performing vital functions^36,55. While vulnerable to defectors in their own institution, cooperators complement each other by beating defectors in their opposite institution. Synergistically thus, they can suppress defection in the population and alternately dominate the population (see Fig. 2c, d). At this stage, the game payoff of all the strategies starts to increase beyond zero. I call this periodic orbit the black queen orbit.

The picture depicted above is the typical behavior of the model for large enough values of the cost. To see this, in Fig. 3a, I plot the phase diagram of the model in the c_g − r plane. Here, the frequency of cooperators in the population, ({rho }_{C}={rho }_{C}^{1}+{rho }_{C}^{2}), is color plotted as well (see Supplementary Fig. 1 for the frequency of different strategies). Red dashed lines show the boundary of the region where the system settles in a periodic orbit. For high costs, as r increases, the system shows a series of successive cross-overs from a defective fixed point to the red queen periodic orbit, black queen periodic orbit, and finally a cooperative fixed point. On the other hand, for small costs, the system possesses a bistable region where both the red queen and black queen periodic orbits (or a partially cooperative fixed point and black queen periodic orbit to the left of the red dashed line in the bistable region) are stable, and the system shows a discontinuous transition between these two orbits. Orange circles show the lower boundary of the bistable region, below which the black queen orbit is unstable. Its upper boundary, above which the red queen orbit becomes unstable, is plotted by red squares, in Fig. 3. The transition between the two periodic orbits becomes a continuous transition at a single critical point (see Supplementary Fig. 4).

Examination of the overall cooperation in the population shows that an entrance cost has a contrasting effect on population cooperation for large and small enhancement factors. An entrance cost keeps free-riders away from a costly institution. This fact makes the relative frequency of cooperators to defectors higher in the costly institution than that in the free institution. To see that defectors are less likely to join the costly institution, I plot the difference between the probabilities that an individual in the costly institution is a cooperator and the probability that an individual in the free institution is a cooperator, (gamma ={rho }_{C}^{1}/({rho }_{C}^{1}+{rho }_{D}^{1})-{rho }_{C}^{2}/({rho }_{C}^{2}+{rho }_{D}^{2})) in Fig. 3b, where it can be seen it is always positive. Intuitively, as a costly defector’s payoff in a group with ({n}_{C}^{1}) cooperators and ({n}_{D}^{1}) other defectors is equal to (r{n}_{C}^{1}/({n}_{C}^{1}+{n}_{D}^{1})-{c}_{g}), a costly defector can reach a positive payoff only when ({n}_{C}^{1}) is large. Otherwise, costly defectors are better off hedging the risk of obtaining a negative payoff by joining the free institution, where their payoff is necessarily non-negative. Consequently, the expected number of cooperators in the costly institution, ({rho }_{C}^{1}g), sets a bound for the frequency of costly defectors. This fact increases a costly institution’s profitability, especially for small enhancement factors, and positively impacts cooperation in the population. On the other hand, for high enhancement factors, a large entrance cost is detrimental to cooperation. This is because, although the frequency of defectors in the costly institution remains close to zero, fewer individuals are willing to choose a costly institution with a high cost. This increases the effective size of the free institution and the mixing between cooperators and defectors in the free institution. Since defectors can better exploit cooperators in well-mixed groups, the increased mixing between cooperators and defectors in the free institution hinders cooperation.

As shown in the Supplementary Note 3, while the phenomenology of the model remains the same for lower mutation rates, lower mutation rates increase the size of the region where the dynamics settle in a periodic orbit (see Supplementary Fig. 5). Regarding the dependence of the dynamics on the mutation rate, an interesting case is the zero mutation rate, where selection is the sole driver of the dynamics. The time average cooperation for zero mutation rate, starting from an initial condition where all the strategies are equal, is plotted in Fig. 3c (See Supplementary Fig. 6 for the frequency of different strategies). Both the red queen (for small enhancement factors) and the black queen (for large enhancement factors) periodic orbits are observed in this case. However, for zero mutation rate, both the amplitude and period of fluctuations increase: The fluctuating dynamics go through periods where one of the surviving strategies reaches a frequency close to 1 only to be later replaced by another strategy (see Supplementary Fig. 7). The dynamics can also settle in different fixed points. For c_g = 0, depending on the enhancement factor, either cooperators or defectors in both institutions survive in equal densities. For nonzero c_g, however, only one of the strategies survives. For small enhancement factors, non-costly defectors dominate the population. For larger enhancement factors, either costly cooperators (the region marked with a dashed black line) or non-costly cooperators dominate the population.

In the Supplementary Note 2, I consider a case where the two institutions have different productivities, i.e., different enhancement factors, and show that similar phases are at work in this case (see the Supplementary Figs. 2 and 3). For instance, I show that a large entrance cost destabilizes full defection, removes the system’s bistability, and ensures the evolution of cooperation starting from all the initial compositions of the population. In addition, I study the continuous replicator dynamics and show similar phenomenology is at work in this case (see Supplementary Notes 1.4 and 4, and Supplementary Figs. 8 and 9).

Finally, I note that a similar phenomenology is at work in a context where instead of a costly and a cost-free institution, two costly institutions interact. To see this, assume institution 1 has a cost c_g and institution 2 has a cost ({c}_{g}^{0}). Without loss of generality, assume ({c}_{g} , > , {c}_{g}^{0}). Writing ({c}_{g}=({c}_{g}-{c}_{g}^{0})+{c}_{g}^{0}), it is easy to see that it is possible to absorb ({c}_{g}^{0}) in the base payoff b (as all the individual pay a cost ({c}_{g}^{0}) irrespective of their institution choice). Thus, the model is equivalent to a context where resource 2 has zero cost, resource 1 has a cost of ({c}_{g}-{c}_{g}^{0}), and all the individuals receive a shifted base payoff of (b-{c}_{g}^{0}) (see Supplementary Note 5 and Supplementary Fig. 10).

Structured population

In contrast to the well-mixed population, the model shows no bistability in a structured population, and the fate of the dynamics is independent of the initial condition. To see why this is the case, I note that in a well-mixed population, a situation where all the individuals are defectors, and randomly prefer one of the two institutions, is the worst case for the evolution of cooperation, as in this case, mutant cooperators are in a disadvantage in both institutions. However, in a structured population, starting from such an initial condition, blocks of defectors, most of whom prefer the same institution, form due to spatial fluctuations. A mutant cooperator who prefers the minority institution in these blocks obtains a high payoff and proliferates. This removes the bistability of the dynamics in a structured population.

To study the model’s behavior in a structured population, I perform simulations starting from an initial condition in which all the individuals are defectors and prefer one of the two institutions at random. The model shows similar behavior in a structured population to that in a well-mixed population. This can be seen in Fig. 4a–d, where the densities of different strategies are color plotted in the c_g − r plane (see Supplementary Note 6 and Supplementary Figs. 11 and 12 for further details). As was the case in a well-mixed population, cooperation does not evolve for too small values of r. As r increases beyond a threshold, cooperation does evolve in the costly institution but not in the free institution. In this region, for a fixed enhancement factor, an optimal cost, approximately equal to c_g = r − 1, optimizes the cooperation in the population. On the other hand, cooperation in both the costly and the free institutions evolves for large enhancement factors. In this region, increasing the cost can slightly increases defection in the free institution and have a detrimental effect on the evolution of cooperation, but not as much as it does in a well-mixed population.

Instead of periodic orbits observed in the well-mixed population, on a spatial structure the model’s dynamic is governed by the cyclic dominance of different strategies through spatiotemporal fluctuations manifested by traveling waves. In Fig. 5, I present snapshots of the population’s stationary state in different phases. In this figure, I consider a model in which individuals reproduce with a probability proportional to the exponential of their payoff, π, times a selection parameter, β, (exp (beta pi )) (see the Supplementary Note 1.3), with β = 5. The situation in the model where individuals reproduce with a probability proportional to their payoff is similar. In Fig. 5a, I have set r₁ = r₂ = 1.7, and c_g = 0.6. This phase corresponds to the red queen periodic orbit in the well-mixed population case. Here, the majority of the population are non-costly defectors. Costly cooperators experience an advantage over the former and can proliferate in the sea of non-costly defectors. However, costly cooperators are vulnerable to both costly defectors and non-costly cooperators. The former can only survive in small bands around costly cooperators, as they rapidly get replaced by non-costly defectors once they eliminate costly cooperators. This phenomenon shows that spatial competition between defectors with differing institution preferences can positively impact the evolution of cooperation. Non-costly cooperators, in turn, can survive by forming compact domains where they reap the benefit of cooperation among themselves. However, as the effect of network reciprocity is too small to promote cooperation in this region, non-costly cooperators get eliminated by non-costly defectors once costly cooperators are out of the picture. Consequently, the system’s dynamic is governed by traveling waves of costly cooperators followed by small trails of costly defectors and non-costly cooperators in a sea of non-costly defectors (see the Supplementary Video, SV.1⁵⁶, and Supplementary Note 7 for an illustration of the dynamics in this regime).

Figure 5b shows a snapshot of the population for r₁ = r₂ = 2.2. This phase corresponds to the black queen periodic orbit in a well-mixed population. In this phase, cooperators in both the costly and free institutions evolve. Cooperators are vulnerable to defectors in their institution and lose their territory to defectors of similar type. Defectors are in turn vulnerable to cooperators in their opposite institution and are replaced by them. Consequently, the dynamic of the model is governed by traveling waves of cooperators, chased by defectors of similar type, who are in turn extincted by cooperators of the opposite type. Thus, while cooperators of different types on their own either can not survive (in the case of non-costly cooperators) or are doomed to a winnerless competition with defectors (in the case of costly cooperators), they complement each other to efficiently suppress defection in the population (see the Supplementary Video, SV.2⁵⁶, for an illustration of the dynamics in this regime).

Another manifestation of functional complementation between cooperators of different types can be seen in the regime of large enhancement factors. An example of this situation is plotted in Fig. 5c. Here, r₁ = r₂ = 3.5 and c_g = 0.6. In this region, non-costly cooperators dominate the population. However, non-costly defectors can survive in small bands in the sea of non-costly cooperators. While at a disadvantage in the sea of non-costly cooperators, costly cooperators beat non-costly defectors. Consequently, small blocks of costly cooperators are formed within the bands of non-costly defectors. These blocks of costly cooperators move along the bands of non-costly defectors and purge the population from non-costly defectors. In this way, although costly cooperators exist only in small frequency, they play a constructive role in helping non-costly cooperators to dominate the population.

In summary, the analysis of the spatial patterns reveals that competition or synergistic relation between individuals with different institution preferences plays an essential role in the evolution of cooperation in the system. Defectors with different institution preferences always appear as competitors who compete over space. By eliminating each other, they play a surprisingly constructive role in the evolution of cooperation. Cooperators, on the other hand, while having direct competition over scarce sites, can also act synergistically and help the evolution of cooperation in their opposite institution since they can eliminate defectors in their opposite institution. In this way, by purging defectors with an opposite game preference, cooperators help fellow cooperators with an opposite game preference. Consequently, cooperators with different game preferences can engage in a mutualistic relation to efficiently suppress defection in the population.

Finally, as shown in the Supplementary Note 6, the spatial model shows similar phases in the case where the two public resources have heterogeneous profitability, that is, when r₁ ≠ r₂ (see the Supplementary Fig. 12).

Source: Ecology - nature.com