Experimental setup
This research was approved by the Carnegie Mellon University Committee of the Use of Human Subjects. All methods were performed in accordance with the relevant guidelines and regulations. Informed consent was obtained from all participants. Our data includes no identifying information of human participants. We conducted experiments from February to August 2021 (except for the preliminary sessions of random information; we ran the condition from June to November 2020). We preregistered the main experiment settings using AsPredicted (https://aspredicted.org/sm4k5.pdf).
A total of 2786 subjects participated in our incentivized decision-making game experiments. We recruited subjects using Amazon Mechanical Turk (MTurk)52,53. Supplementary Table 1 shows the subject demographics. Our participants interacted anonymously over the Internet using customized software playable in a browser window (available at http://breadboard.yale.edu). All participants provided explicit consent and passed a series of human verification checks and a screening test of understanding game rules and payoffs before playing the game (see SI). We prohibited subjects from participating in more than one session of the experiment by using unique identifications for each subject on MTurk.
In each session, subjects were paid a $2.00 show-up fee and a bonus depending on whether they took the appropriate disaster decision in four rounds. Furthermore, subjects earned $1.00 when they completed all four rounds. In each round, when a disaster stroke before they evacuated, the subjects earned no bonus. Otherwise, they earned a bonus of $1.00 without disaster or $0.50 with disaster by spending $0.50 for evacuation, plus $0.05 per other players who took the correct action accordingly (Supplementary Table 2). We have confirmed with prior work that the amount of evacuation cost, if any, makes no significant difference in the game’s performance23.
At the start, subjects were required to pass a series of human verification checks. They needed to pass Google’s reCAPTCHA using the “I’m not a robot” checkbox. They were also requested to answer whether they were human players. The exact question asked was: “Please select an applicable answer about you.” The options were: “I am not a bot. I am a real person.” “I am not a real person. I am a bot.” “I am anything but a human.” and “I am a computer program working for a person.” The option’s order was randomized. Only the participants who selected “I am not a bot. I am a real person.” moved to the step of informed consent.
When subjects provided explicit consent, they were asked to take a tutorial before the actual game would begin. In the tutorial, each subject separately interacted with three dummy players in two rounds of a 45-s practice game. In the actual game, some subjects would be informed in advance whether a disaster would indeed strike or not. In the practice game, while all subjects were not informed of such information in the first round, they were informed of the information in the second round. Thus, they practiced both conditions in terms of prior information on the disaster (see SI).
After the practice game, subjects were assessed for their comprehension of the game rules and payment structure using four multiple-choice questions with three options. If they failed to select the correct answer in one of the questions, they could reselect it only once through the entire test. If they failed to select the correct answer more than once, they were unable to join the actual game.
At 720 s after the tutorial beginning, a “Ready” button became visible simultaneously to all the subjects who completed the tutorial and passed the comprehension tests. The actual games started 30 s after the “Ready” button showed up. If subjects did not click the button before the game started, they were dropped. The game required a certain number of subjects. When the subjects who successfully clicked the button were more than 16, surplus subjects, randomly selected, were dropped from the game. When the number of qualified subjects was less than 12, the game did not start. As a result, subjects started the game in a group with an average size of 15.5 (s.d. = 1.1).
At the start of the actual game, we selected one subject (the “informant”) at random who was informed in advance whether a disaster would indeed strike or not. The other subjects were informed that some players had accurate information about the disaster, but they were not informed who the informant was. The exact sentence that the informants received in their game screen was “A disaster is going to strike!” when a disaster would strike or “There is no disaster.” when a disaster would not strike. The one that the other uninformed subjects received was “A disaster may or may not strike.” Then, the group had the same informant across the four rounds except for a supplement condition of random informants. In the random informant condition, an informant was randomly selected every round.
To prevent an end-of-game effect, we randomly set the game time with a normal distribution of a mean of 75 s and a standard deviation of 10 s. Prior work has confirmed that the game time is sufficient for players to communicate and make an evacuation decision23. As a result, each round ended at 75.0 s on average (s.d. = 9.5) without prior notice. In half of the sessions, a disaster struck at the end of the game. We did not inform any subjects, including the informants, when their sessions would end, the global network structure they were embedded in, or how many informants were in the game. After making their evacuation choice, subjects were informed of their success and failure along with overall results in their group. Then, subjects played another round of the evacuation game until they completed four total rounds. They had the same local network environment across four rounds except for the dynamic network condition.
Network structure and tie rewiring
In the network sessions, subjects played the game in a directed network with a random graph configuration. A certain number of ties were present at the game’s onset as the initial density was set to 0.25.
In the dynamic network conditions, subjects also could change their neighbors by making or breaking ties between rounds. In the tie-rewiring step, 40% of all the possible subject pairs were chosen at random. Thus, subjects could choose every other player at least once throughout the entire session (i.e., a set of four rounds) with a probability of about 80%. When the chosen pairs were connected, the pairs (the ties) were dissolved if the predecessor subject of the directed ties chose to break the tie. When the chosen pairs were not connected, the pairs (the ties) were newly created when the predecessor of the potential tie chose to create the tie. Subjects were not informed of the rewiring rate.
To equalize the game time, we made subjects in the independent and static network conditions wait for additional 10 s after each game round ended. Despite the adjustment, the game time was significantly longer in the dynamic network sessions than in the independent and static network sessions. The average game time is 429.5 s (s.d. = 20.2) for the independent condition; 428.8 s (s.d. = 19.0) for the static network condition; and 564.7 s (s.d. = 36.3) for the dynamic network condition.
To clarify mechanisms for dynamic networks to facilitate collective intelligence, we added one supplementary condition. In the supplementary condition, subjects were assigned to one of the 40 isomorphic networks that other subjects had developed with tie-rewiring options through the three rounds in the dynamic network condition (567 subjects in 40 groups). Network structure and other game settings (i.e., whether a disaster stroke, how long the game was, and which node was the informant) were identical to where the others played the game at the final round. However, players were different, and they had no prior experience in the game. They played the game in a network with a topology created by others ostensibly to optimize the accurate flow of information. In contrast to other conditions, subjects played only one round in the isomorphic network condition.
Signal buttons
During the game of network sessions, subjects were allowed to share information about the possibly impending “disaster” by using “Safe” and “Danger” buttons that indicated their assessment (see SI). The default node color was grey. Then, when they clicked the Safe button, their node turned blue and, after 5 s, automatically returned to grey. Likewise, the Danger button turned their node to red for 5 s. Subjects could see only the colors of neighbors to whom they were directly connected. Since the signal exchange occurred through directed connections, an individual could send, but not receive, information from another subject (and vice versa). Once subjects chose to evacuate, they could no longer send signals, and their node showed grey (the default color) for the rest of the game. The neighbors of evacuated subjects were not informed of their evacuation. We have confirmed with prior work that collective performance does not vary with the communication continuity and the evacuation visibility23. Subjects could use the Safe and Danger buttons any time unless they evacuated, or they did not have to.
Players dropping during the game
After each game round, when a player was inactive for 10 s, they were warned about being dropped. When they remained inactive after 10 s, they were dropped. When the selected informant was dropped, the session stopped at the round, and we did not use the data. Furthermore, as too many dropped players could affect the network structure and the behavioral dynamics of remaining players, we did not use the sessions where more than 25% of initial players were dropped during the game. Overall, 4 players dropped in 15 sessions; 3 players dropped in 22 sessions; 2 players dropped in 41 sessions; 1 player dropped in 44 sessions; and no player dropped in 58 sessions. The dropped players were prohibited from joining another session of this experiment.
As noted above, players took the additional tie-rewiring step every round in the dynamic network sessions. Thus, the total game time was longer in the dynamic network sessions than in the independent and static network sessions even with the adjustment. As a result, more players were dropped in the dynamic network sessions than in the independent and static network sessions. The average number of dropped players across the four rounds is 0.40 (s.d. = 0.60) for the independent condition; 1.15 (s.d. = 0.86) for the static network condition; 1.75 (s.d. = 1.19) for the dynamic network condition. Although group size could affect collective performance, we found the differences in group size small enough for our study. We have confirmed the dynamic network’s performance improvement with a comprehensive analysis controlling the effect of group size (Supplementary Table 3). Also, there was no statistically significant difference in the dropped players’ performance of the dynamic network condition, compared with the other two conditions. The rate of correct actions of dropped players is 0.456 (s.d. = 0.322) for the independent condition, 0.594 (s.d. = 0.387) for the static network condition, and 0.558 (s.d. = 0.411) for the dynamic network condition; P = 0.106 between the independent condition and the dynamic network condition; P = 0.599 between the static network condition and the dynamic network condition (Welch two-sample t test).
Analysis of signal diffusions
To examine the change in signal diffusion, we analyzed “diffusion chains” for each signal type in the network sessions. We first identified the subjects who sent a signal when their neighbors had never sent one as spontaneous “diffusion sources.” When a subject sent a signal after at least one neighbor had sent the same type of signal, we regarded the subject’s signaling (and evacuation with danger signals) as occurring in a chain of signal diffusion and the total number of the responded subjects (including the diffusion source) as the diffusion size.
We analyzed the distribution of signal diffusion chains with complementary cumulative distribution functions, measuring the fraction of diffusion chains that exhibit a given number of diffusion sizes. We found that the number of diffusions of both signals did not change across rounds. Safe-signal diffusions were more likely to occur than danger-signal diffusions regardless of whether a “disaster” would strike and how many rounds subjects played. On the other hand, the diffusion size varied greatly across rounds in disaster situations. With “disaster,” false safe signals spread further than true danger signals at the first round, but after that, warnings outperformed safe signals in terms of diffusion size. Figure 2B and Supplementary Fig. 3 scrutinize the changes in diffusion chains with their distributions.
Analysis of individual responsiveness
We analyzed how individual evacuation behavior varies with exposure to signals from neighbors54. Let
$${a}_{i}^{evacuate},, (t)=left{begin{array}{ll}1&quad text{if subject } i text{ evacuates at time } t 0&quad text{otherwise}end{array}right.$$
$${a}_{i}^{show, safe},, (t)=left{begin{array}{ll}1&quad text{if subject } i mathrm{ shows a safe signal at time } t 0&quad text{otherwise}end{array}right.$$
$${a}_{i}^{show , danger} ,, (t)=left{begin{array}{ll}1&quad text{if subject } i text{ shows a danger signal at time } t 0&quad text{otherwise}end{array}right.$$
The hazard function, or instantaneous rate of occurrence of subject (i)’s evacuation at time t, is defined as:
$${lambda }_{i},, (t)=underset{mathit{dt}to 0}{{mathrm{lim}}}frac{{mathrm{Pr}}({a}_{i}^{evacuate}=1;,, t<Tle t+dt|T>t)}{dt}$$
To model the time to evacuation, We used a Cox proportional hazards model with time-varying covariates for the number of signals, incorporating an individual actor-specific random effect55:
$${lambda }_{i} ,, left{t|{{P}_{i}, X}_{i}(t), {G}_{i},{Y}_{i}(t)right}={lambda }_{0}(t)mathrm{exp}left{{{beta }_{P}^{{prime}}{P}_{i}+beta }_{X}^{{prime}}{X}_{i}(t)+{beta }_{G}^{{prime}}{G}_{i}+{beta }_{Y}^{{prime}}{Y}_{i}(t)+{gamma }_{i}right}$$
where λ0(t) is a baseline hazard at time t.
In the model, the hazard λi(t) depends on the covariates Pi, Xi(t), Gi, and Yi(t). The covariate Pi is the vector of subject i’s experiences before the sessions; that is, the number of rounds, the number of disasters that she has experienced, and the number of disasters that she has been struck by.
The covariate Xi(t) is the vector of the number of safe signals ({x}_{i}^{safe} (t)), the number of danger signals ({x}_{i}^{danger} (t)). When subject j is a neighbor of subject i (i.e., (jin {N}_{i})), subject i is exposed to the signal of subject j, so that:
$${x}_{i}^{safe},, (t)=sum_{jin {N}_{i}}{a}_{j}^{show, safe}(t)$$
$${x}_{i}^{danger},, (t)=sum_{jin {N}_{i}}{a}_{j}^{show, danger}(t)$$
The covariate Gi is the vector of the properties of the network in which subject i is embedded, out-degree, in-degree, and a network plasticity indicator. The covariate Yi(t) is the vector of the number of the subject i’s actions of sending safe and danger signals before time t. The coefficients β are the fixed effects and γi is the random effect for individual i. We assumed that waiting times to evacuation in different actors are conditionally independent given the sequence of signals they receive from network neighbors. This model shows how the hazard of an individual’s evacuation depends on the signaling actions of others, their network position, and experience (Supplementary Table 4). We applied the same model to the first signaling behavior.
Source: Ecology - nature.com
