Multi-agent network example

In this example we describe an experiment to be modeled involving multiple agents interacting in constantly varying ways. It is a coöperative binary choice game played by multiple players whose connectivity to one another varies, in an effort to understand how their networks of interconnectedness facilitate or hinder their abilities to succeed at the game. This closely follows some work described in Gonzalez et al (2025) [1].

Games

The binary choice games are two from a spectrum of games described in Balliet et al. (2016) [2]. The games in this spectrum are played between two players conventionally referred to as Mary and John, exploring a taxonomy of dimensions of social interaction. For this example we use just two of the games at the extremes of one of the dimensions, fully independent and fully interdependent. In both games the players will each earn five points if they both “coöperate” by each picking A, and will each earn nothing if they both “defect” by by each picking B. The difference is in the outcome if one picks A and the other B. In the independence game

_images/independence.png

each player receives the five point reward whenever they pick A, regardless of the other player’s choice. In the interdependence game

_images/interdependence.png

each player receives the five point reward whenever the other play picks A, regadless of this player’s choice:

For this example weach player makes their choices with no knowledge of the reward matrices, nor any knowledge of the other player’s choices. All they know if what they themselves choose, and what rewards they receive.

In a normally played single independence game of many rounds we might expect both players to quickly settle on choosing A since that always rewards them. Naïvely we might similarly expect that in a similar version of the interdependence game for each player to essentially play randomly, since over the course of a single choice which choice they make has no effect on their reward. However there is a deep subtlety to this game where without any explicit transfer of information between the players, over the course of several moves they will be driven to both pick A. This is driven by a natural weighting of recent results over earlier ones, resulting in a roughly win-stay/lose-shift strategy, which curiously over a two move sequence can drive the players to coöperate.

Networks

For this example we are interested not in normal play between just two players, but rather small networks of players interacting with different players, and how this may modify the choices made. The networks chosen for this example are inspired by those used for a somewhat different task in Mason et al. (2008) [3] and Sloman et al. (2021) [4]. For this example we use three different networks, each with six nodes.

_images/networks.png

When we play the game at each round we will randomly select three, non-overlapping pairs of nodes, such that each node in a pair is connected in the network to the other node of that pair. Note that in the disconnected network each node is always paired with the same other node; in the ring network there are only two possible ways pairs of nodes can be connected; and in the fully connected case there are fifteen different ways the pairs of nodes can be selected. Where there are multiple ways to select pairs of nodes, at each round we randomly select a different way of pairing them, so, except in the disconnected case, at each round a node may play a different partner than it did in the previous or succeeding round.

Games and networks definitions

The Python code for defining the two games to be played is a dict mapping the names of the two games to further dicts mapping a tuple of the possible choices by the two players to a tuple of the outcomes for those two players.

GAMES = {"independence": {("A", "A"): (5, 5),
                          ("A", "B"): (5, 0),
                          ("B", "A"): (0, 5),
                          ("B", "B"): (0, 0)},
         "interdependence": {("A", "A"): (5, 5),
                             ("A", "B"): (0, 5),
                             ("B", "A"): (5, 0),
                             ("B", "B"): (0, 0)}}

The code for defining the three networks to be used is a dict mapping the names of the three networks to lists of sublists, each sublist containing three 2-tuples. These sublists enumerate each of the possible ways the various nodes can be connected.

PAIRINGS = {
    "disconnected": [[(0, 1), (2, 3), (4, 5)]],
    "ring": [[(0, 1), (2, 3), (4,5)],
             [(1, 2), (3, 4), (0, 5)]],
    "fully-connected": [[(4, 5), (2, 3), (0, 1)],
                        [(3, 5), (2, 4), (0, 1)],
                        [(3, 4), (2, 5), (0, 1)],
                        [(4, 5), (1, 3), (0, 2)],
                        [(3, 5), (1, 4), (0, 2)],
                        [(3, 4), (1, 5), (0, 2)],
                        [(4, 5), (1, 2), (0, 3)],
                        [(2, 5), (1, 4), (0, 3)],
                        [(2, 4), (1, 5), (0, 3)],
                        [(3, 5), (1, 2), (0, 4)],
                        [(2, 5), (1, 3), (0, 4)],
                        [(2, 3), (1, 5), (0, 4)],
                        [(3, 4), (1, 2), (0, 5)],
                        [(2, 4), (1, 3), (0, 5)],
                        [(2, 3), (1, 4), (0, 5)]]}

From this dict we further compute constants describing the number of pairs to be played at each round, and the number of total nodes.

PAIRS = len(list(PAIRINGS.values())[0][0])
for kind in PAIRINGS.values():
    for pairing in kind:
        assert len(pairing) == PAIRS
NODES = 2 * PAIRS

For these networks the constant PAIRS is always 3 and the constant NODES is always 6.

The model

For the model we create a PyIBL Agent for each of the nodes.

PREPOPULATED_VALUE = 6
...
agents = [Agent(str(i), default_utility=PREPOPULATED_VALUE)
          for i in range(1, NODES + 1)]

Note that the prepopulated value is chosen as 1.2 times the maximum possible payoff of 5 to facilitate early exploration.

To run the model we define a function run_one() which takes as parameters #. the list of agents defined above #. the name of the network to use #. the name of the game to play #. the number of sets of participants to simulate playing the game #. the number of rounds to place for each simulated set of participants #. a tqdm progress indicator to update after the simulation of each set of participants is completed #. a CSVWriter object into which to write details of each round #. and the name of a file into which to draw a plot of the results, or None simply show them on terminal as run_one() finishes.

def run_one(agents, network, game, participant_sets, rounds, progress, csv_writer, plot_file):
    counts = [Counter() for i in range(rounds)]
    for pp in range(participant_sets):
        for a in agents:
            a.reset()
        for r in range(rounds):
            for p in random.choice(PAIRINGS[network]):
                p = random.sample(p, k=len(p)) # counterbalance
                choices = tuple(agents[i].choose("AB") for i in p)
                counts[r].update((choices,))
                payoffs = GAMES[game][choices]
                for i, j in zip(p, count()):
                    agents[i].respond(payoffs[j])
                csv_writer.writerow((network, game, pp + 1, r + 1,
                                     p[0], choices[0], payoffs[0],
                                     p[1], choices[1], payoffs[1]))
            progress.update()

We create Counter object to capture the number of times each possible pair of choices is made at each round. We then iterate through the required number of sets of participants, first resetting all the agents. The for each round we randomly pick one of the pairings of participants as described by the netword; for the disconnected network this is always a fixed set of pairings, for the ring one of two possibiliites, and for the fully connected network one of fifteen. We iterate through the three pairs in the chosen pairings and play one game, having each of the two agents selected make a choice, computing their payoffs, and responding to the agents with those payoffs, and finally record what we have done to the CSVWriter. After all the rounds are finished we update the progress indicator.

At this point all the data for this network and game combination is generated and gathered, and we can plot the results

plt.plot(tuple(range(1, rounds + 1)),
         tuple(counts[r][("A", "A")] / (participant_sets * PAIRS) for r in range(rounds)),
         label='("A", "A")', color="green", linestyle="solid")
plt.plot(tuple(range(1, rounds + 1)),
         tuple(counts[r][("A", "B")] / (participant_sets * PAIRS)  for r in range(rounds)),
         label='("A", "B")', color="blue", linestyle="dotted")
plt.plot(tuple(range(1, rounds + 1)),
         tuple(counts[r][("B", "A")] / (participant_sets * PAIRS)  for r in range(rounds)),
         label='("B", "A")', color="violet", linestyle="dotted")
plt.plot(tuple(range(1, rounds + 1)),
         tuple(counts[r][("B", "B")] / (participant_sets * PAIRS)  for r in range(rounds)),
         label='("B", "B")', color="red", linestyle="dashed")
plt.legend()
plt.title(f"{game} game, {network} topology\n"
          f"{participant_sets} participant sets, noise={agents[0].noise}, decay={agents[0].decay}")
plt.xlim((0, rounds + 2))
plt.xticks((1, rounds / 2, rounds))
plt.xlabel("round")
plt.ylim((-0.05, 1.05))
plt.yticks((0, 0.5, 1))
plt.ylabel("fraction of pairs choosing")
if plot_file:
    plt.savefig(plot_file)
else:
    plt.show()
plt.clf()

Note that we plot one line for each of the four possible pairs of choices, showing the fraction of times that choice was made.

All that is left to do is iterate through the two games and three networks and call the run_one() function. This main function uses click to parse a few command line arguments allowing the number of sets of participants, number of rounds played by each, and noise and decay parameters of the agents to be set, though with sensible defaults if none are explicitly specified. This function also opens and closes the file into which the CSV data will be written, as well as creating the progress indicator.

@click.command()
@click.option("--rounds", "-r", default=DEFAULT_ROUNDS, type=int,
              help="The number of rounds to play")
@click.option("--participant-sets", "-p", default=DEFAULT_PARTICIPANT_SETS, type=int,
              help="The number of participant sets to simulate")
@click.option("-noise", "-s", default=DEFAULT_NOISE, type=float,
              help="The IBL activation noise to use")
@click.option("--decay", "-d", default=DEFAULT_DECAY, type=float,
              help="The IBL decay parameter to use")
def main(rounds, participant_sets, noise, decay):
    with open("results.csv", "w", newline="") as f:
        w = csv.writer(f)
        w.writerow(("network,game,participant set,round,"
                    "player one,player one move,player one payoff,"
                    "player two,player two move,player two payoff").split(","))
        agents = [Agent(str(i), default_utility=PREPOPULATED_VALUE, noise=noise, decay=decay)
                  for i in range(1, NODES + 1)]
        with tqdm(total=(len(GAMES) * len(PAIRINGS) * participant_sets * rounds)) as t:
            for n in PAIRINGS.keys():
                for g in GAMES.keys():
                    run_one(agents, n, g, participant_sets, rounds,
                            t, w, f"{n}-{g}.png")


if __name__ == '__main__':
    main()

The entire file can :download:be downloaded </_downloads/multi-agent-network.zip>`

  1# Copyright 2025 Carnegie Mellon University
  2# Example of networked binary choice games implemented with PyIBL
  3
  4import click
  5from collections import Counter
  6import csv
  7from datetime import datetime
  8from itertools import count
  9import matplotlib.pyplot as plt
 10from pyibl import Agent
 11import random
 12from tqdm import tqdm
 13
 14DEFAULT_ROUNDS = 60
 15DEFAULT_PARTICIPANT_SETS = 500
 16
 17DEFAULT_NOISE = 0.25
 18DEFAULT_DECAY = 0.5
 19
 20GAMES = {"independence": {("A", "A"): (5, 5),
 21                          ("A", "B"): (5, 0),
 22                          ("B", "A"): (0, 5),
 23                          ("B", "B"): (0, 0)},
 24         "interdependence": {("A", "A"): (5, 5),
 25                             ("A", "B"): (0, 5),
 26                             ("B", "A"): (5, 0),
 27                             ("B", "B"): (0, 0)}}
 28
 29PREPOPULATED_VALUE = 6
 30
 31PAIRINGS = {
 32    "disconnected": [[(0, 1), (2, 3), (4, 5)]],
 33    "ring": [[(0, 1), (2, 3), (4,5)],
 34             [(1, 2), (3, 4), (0, 5)]],
 35    "fully-connected": [[(4, 5), (2, 3), (0, 1)],
 36                        [(3, 5), (2, 4), (0, 1)],
 37                        [(3, 4), (2, 5), (0, 1)],
 38                        [(4, 5), (1, 3), (0, 2)],
 39                        [(3, 5), (1, 4), (0, 2)],
 40                        [(3, 4), (1, 5), (0, 2)],
 41                        [(4, 5), (1, 2), (0, 3)],
 42                        [(2, 5), (1, 4), (0, 3)],
 43                        [(2, 4), (1, 5), (0, 3)],
 44                        [(3, 5), (1, 2), (0, 4)],
 45                        [(2, 5), (1, 3), (0, 4)],
 46                        [(2, 3), (1, 5), (0, 4)],
 47                        [(3, 4), (1, 2), (0, 5)],
 48                        [(2, 4), (1, 3), (0, 5)],
 49                        [(2, 3), (1, 4), (0, 5)]]}
 50
 51PAIRS = len(list(PAIRINGS.values())[0][0])
 52for kind in PAIRINGS.values():
 53    for pairing in kind:
 54        assert len(pairing) == PAIRS
 55NODES = 2 * PAIRS
 56
 57def run_one(agents, network, game, participant_sets, rounds, progress, csv_writer, plot_file):
 58    counts = [Counter() for i in range(rounds)]
 59    for pp in range(participant_sets):
 60        for a in agents:
 61            a.reset()
 62        for r in range(rounds):
 63            for p in random.choice(PAIRINGS[network]):
 64                p = random.sample(p, k=len(p)) # counterbalance
 65                choices = tuple(agents[i].choose("AB") for i in p)
 66                counts[r].update((choices,))
 67                payoffs = GAMES[game][choices]
 68                for i, j in zip(p, count()):
 69                    agents[i].respond(payoffs[j])
 70                csv_writer.writerow((network, game, pp + 1, r + 1,
 71                                     p[0], choices[0], payoffs[0],
 72                                     p[1], choices[1], payoffs[1]))
 73            progress.update()
 74    plt.plot(tuple(range(1, rounds + 1)),
 75             tuple(counts[r][("A", "A")] / (participant_sets * PAIRS) for r in range(rounds)),
 76             label='("A", "A")', color="green", linestyle="solid")
 77    plt.plot(tuple(range(1, rounds + 1)),
 78             tuple(counts[r][("A", "B")] / (participant_sets * PAIRS)  for r in range(rounds)),
 79             label='("A", "B")', color="blue", linestyle="dotted")
 80    plt.plot(tuple(range(1, rounds + 1)),
 81             tuple(counts[r][("B", "A")] / (participant_sets * PAIRS)  for r in range(rounds)),
 82             label='("B", "A")', color="violet", linestyle="dotted")
 83    plt.plot(tuple(range(1, rounds + 1)),
 84             tuple(counts[r][("B", "B")] / (participant_sets * PAIRS)  for r in range(rounds)),
 85             label='("B", "B")', color="red", linestyle="dashed")
 86    plt.legend()
 87    plt.title(f"{game} game, {network} topology\n"
 88              f"{participant_sets} participant sets, noise={agents[0].noise}, decay={agents[0].decay}")
 89    plt.xlim((0, rounds + 2))
 90    plt.xticks((1, rounds / 2, rounds))
 91    plt.xlabel("round")
 92    plt.ylim((-0.05, 1.05))
 93    plt.yticks((0, 0.5, 1))
 94    plt.ylabel("fraction of pairs choosing")
 95    if plot_file:
 96        plt.savefig(plot_file)
 97    else:
 98        plt.show()
 99    plt.clf()
100
101
102@click.command()
103@click.option("--rounds", "-r", default=DEFAULT_ROUNDS, type=int,
104              help="The number of rounds to play")
105@click.option("--participant-sets", "-p", default=DEFAULT_PARTICIPANT_SETS, type=int,
106              help="The number of participant sets to simulate")
107@click.option("-noise", "-s", default=DEFAULT_NOISE, type=float,
108              help="The IBL activation noise to use")
109@click.option("--decay", "-d", default=DEFAULT_DECAY, type=float,
110              help="The IBL decay parameter to use")
111def main(rounds, participant_sets, noise, decay):
112    with open("results.csv", "w", newline="") as f:
113        w = csv.writer(f)
114        w.writerow(("network,game,participant set,round,"
115                    "player one,player one move,player one payoff,"
116                    "player two,player two move,player two payoff").split(","))
117        agents = [Agent(str(i), default_utility=PREPOPULATED_VALUE, noise=noise, decay=decay)
118                  for i in range(1, NODES + 1)]
119        with tqdm(total=(len(GAMES) * len(PAIRINGS) * participant_sets * rounds)) as t:
120            for n in PAIRINGS.keys():
121                for g in GAMES.keys():
122                    run_one(agents, n, g, participant_sets, rounds,
123                            t, w, f"{n}-{g}.png")
124
125
126if __name__ == '__main__':
127    main()

Results

Running the interdependence game in the disconnected network we see that each, fixed pair fairly quickly settles on mutual coöperation (both players selecting A)

_images/disconnected-interdependence.png

But as the number of connections is increased, with various pairs intermixed, it becomes harder for the players to settle on the optimal play

_images/ring-interdependence.png _images/fully-connected-interdependence.png

Running the independence game again players quickly settle on optimal play. But, since they don’t depend upon their partners’ moves in any way, as the number of interconnections in the network increases this behavior does not degrade

_images/disconnected-independence.png _images/ring-independence.png _images/fully-connected-independence.png