A taxon space is defined by a set of biological entities, each of which is in turn defined by a set of components. For example, if the entities in the taxon space are defined to be genomes the components will be genes, while if proteins are the taxa the components will be functional or structural domains. Given the taxon space and the corresponding component space we will construct an affinity network (Figure 1(a)).

We use the affinity graph definition similar to that of [10]. A biological affinity graph, , for a given set of taxa is a directed graph such that each vertex in the set uniquely corresponds to one taxon, and all edges have nonzero weights with the incoming edges to any given vertex summing to 1. (The distinction between whether the edges are incoming or outgoing is determined by the construction of the similarity matrix, to be discussed in more detail later.) An example of such a network can be seen in Figure 1(b). An edge from vertex to is present if and only if the edge weight is strictly positive. The edge weight is a measure for the affinity of member to relative to all other members of. No edge is drawn from any node to itself (i.e.,) by convention.

The network graph is constructed in a taxon-by-taxon approach. For each member of the taxon space we construct a similarity matrix. In these matrices, we compare the amino acid (or nucleotide) sequences of each component as a first order approximation. Suppose there are a total of components, , found in the members of. Then, as exhibited in Figure 2,

the similarity matrix, , of a given reference member is an matrix, where is the similarity score of component in member to member. This entry may be considered as a proxy for the mutual information between and with respect to component in that the higher the value the more similar the pair are in component. The values in the reference column (the column) will not be used in calculation and therefore we arbitrarily set for all.

As mentioned above, the edge directionality of our network graph depends on the construction of the similarity matrix. If the scores are established using the reference taxon, , as the intended parent sequence, the edges with nonzero weights will be the outgoing edges of the corresponding node, representing a likely ancestordescendent directionality. Similarly, if the scores are constructed to permit the inference that the reference taxon is a descendent sequence, the edges with nonzero weights will be the incoming edges to the reference node. If there is no obvious parent-offspring directionality, as is the case in the similarity matrix construction for our analysis, either convention may be used but only to keep track of the model solutions.

Once the edge weights are found for all, they in turn, as mentioned above, define the network matrix as shown in Figure 2. The directed network graph is then constructed according to the weight matrix and vice versa. Furthermore if the matrix is block diagonalizable as explained in Figure 2, then each (irreducible) block defines a distinct subnetwork graph, referred to as a cluster. Therefore, the construction of any directed affinity network graph is reduced to finding the weight vector n.

Our idea for the construction of the edge weights is based on the assumption that evolution seeks to maximize both conservation and diversity. First we will view the interwoven relationships of taxa as the result of all evolutionary processes by, to name a few, mutation, recombination and gene transfers (both vertical and lateral), all taking place amongst thousands of individual organisms contemporaneously in space and repeated for thousands of generations, and all driven by some particular selective forces. Second we will view that the net effect of these processes as a non-cooperative, twoplayer game in which one player, or one force, is to maximize the genetic conservation, or self-preservation, so that deleterious changes are eliminated and successful (or non-deleterious) structures and functions are passed on from one generation to the next; and the other player, or the other force, is to maintain the genetic diversity and to maximize evolutionary resources where novel structures and functions can be tried out. We will assume, for this paper and for the purpose of being a primary approximation, that the two goals are polar opposite because conservation as characterized by structural and functional similarity is negatively correlated with diversity which is characterized by the reverse. This first order approximation can also be justified by the principle of mass and energy conservation in nature. That is, genetic materials and natural resources that are devoted for conservation will not be available for divergence (or in population genetics, extinction vs. fixation). In short we will view evolution as a repeatedly played game with the aim to maximize both preservation and divergence simultaneously. We suppose that the net effect of playing this game of evolution is the closeness, or the distance, of one member to other members in the taxon space, and this effect is to be captured by the frequencies, explained below, with which a reference taxon has interacted with the other taxa.

The goal of our game-theoretic model for evolution is to find a Nash equilibrium [15-17] of the expected payoff for similarity, , by determining the affinity probability vector to maximize while simultaneously minimizing it (i.e., maximizing the diversity) by finding the novelty probability vector . The conservation-diversification dichotomy interpretation for the y and x probability vectors can be explained as follows. In the case of a pure “diversity” or “component” strategy being played, say, for and, that is, when component of the reference member is used to measure divergence it is the taxon (or taxa) having the largest similarity score that should be picked as the countering “conservation” or “taxon” strategy to maximize the similarity score. Here denotes the subset of the components that are present in the reference taxon. Thus for these, and the sum to 1. This gives the conservation interpretation of the y solutions.

On the other hand, in the case of a pure ‘conservation’ or ‘taxon’ strategy being played, say, for and, that is, when is used to measure affinity to the reference it is those component(s) having the smallest similarity score that stands out and should be picked as the countering “diversification”, or “component” strategy to minimize the similarity score. That is, these are positive and sum to 1, and permit the interpretation for divergence.

As we mentioned before, since all evolutionary processes—all kinds of genetic transfers or otherwise—take place amongst all organisms all the time, the evolutionary state we observe today would be the result of the frequencies with which all pure conservation strategies and all pure diversity strategies are played one event a time, and by our proposed game-theoretical model these frequencies are approximated by the solution y and x to the following min-max problem (see Equation (1)).

The solution to this problem exists and is exactly a Nash equilibrium point (see Appendix). The optimal expected similarity score, , is the so-called game value.

There are two different ways to find a Nash equilibria (NE) for two-player zero-sum games. One is through a dynamical play of the game to find an NE asymptotically which is modeled by the Brown-von-Neumann-Nash (BNN) system of differential equations. The other is by the simplex method in linear programming. The Appendix gives a comprehensive compilation for all fundamental results of both methods.

Here we present a mechanistic derivation of Nash’s map. Nash used this map to prove the existence of NE for all non-cooperative games (Appendix). Our derivation is extremely relevant to our game theory formulation for bioinformatics. It gives a plausible answer to the question how an NE is realized by nature. It shows that evolution or individual organisms need only be driven by their immediate, short term gain in game play payoff to reach a globally attractive Nash equilibrium. Here is an outline of the scenario, which works for not only two but for any number of player types of a game or multiple competing objectives of a process.

In the case that a game is played by large populations of all types repeatedly for a long time so that the time between consecutive plays can be blurred to view the game as played continuously and the play strategy frequency for player type-, , changes continuously, where is the mixed strategy probability or frequency vector,

. Each corresponds to the th strategy of the type- players, and can be interpreted to be the fraction of the player type-

population that uses its pure strategy. Consider it at time and a time later,. We would like to understand how changes from. We will do so probabilistically.

Let be the scalar inertia probability by which an individual of the type- population plays the same strategy with probability at time as at time. Then, is the non-inertia or kinetic probability with which an individual of the type- population chooses or adapts to a particular strategy, including the choice of playing the same strategy at time as at time because it is advantageous to do so, or because the individual organism is driven to do so due to selection. Let be the conditional play probability vector given that the play is kinetic, , then by elementary probability rules

The scalar marginal probability and the conditional probability vector are derived as follows. First we assume the advantage for type- player’s kinetic strategy switch or adaptation depends on its total (scalar) excess payoff from the current play frequency (Appendix, also [16,17]), where denotes the current play frequency for all player types with probability vector for player type-. That is, if, then all plays are of the inertia kind, , and without adaptation. Next, for the case, we assume the th kinetic frequency is , where (Appendix) is the th strategy’s excess payoff from the current play for the player type-,. That is, the strategy switch to strategy for the type- players is strictly proportional to its excess payoff against the total. As for the scalar marginal inertial probability, we assume it is a function of the total excess payoff as well as the time increment. Specifically, consider the probability equivalently in its reciprocal, which represents all fractional possible choices for each inertia choice. The fractional possible choices automatically include the inertia choice itself so that always holds. Then at, we must have this trivial boundary condition, the default inertia choice only for lack of time to adapt. Assume the fractional possible choices increase linearly for small time increment, we have where 1 represents the inertia choice itself and represents the rate of increase in the kinetic choices, which may include the choice of maintaining the same strategy play, because of its excess payoff is positive, and all other kinetic strategy adaptations. We assume the rate of the kinetic choice change is proportional to the total excess payoff, , i.e., the greater the excess payoff the more play switches in the population for a greater payoff gain. As a result, and equivalently,

. With the functional forms for and above, we have

which is exactly the Nash map ([17]) if and. From Equation (3) we also have

and hence the following equivalent system of differential equations

after a time scaling by. This type of equations was first introduced in [18] by Brown and von Neumann to compute an NE for symmetrical zero-sum games and the derivation of Equation (4) from the Nash map

was first noted in [19].

However, our derivation of the Nash map from the time evolution relationship (2) between inertia and kinetic strategy plays is new to the best of our knowledge. The derivation immediately suggests an evolutionary mechanism as to how a Nash equilibrium point may be realized or reached because the process or the game play is driven by the excess payoff at every step of the way, which can be interpreted as a mechanism for adaptation and a force of selection. In fact, let

define the total excess payoff potential, then for any two-player zero-sum game Theorem 2 of Appendix shows that always decreases if and for every solution of the BNN Equation (4). An NE is reached when there is no more excess payoff left,. It shows that as a global dissipative system, any mixed play frequency trajectory will always find a Nash equilibrium by following the down gradient of the excess potential function. That is, in their search of greater excess payoffs, the total excess payoff for the players can never increase along any time evolution of their game plays. A computational implication of this theorem is that a Nash equilibrium of any two-player zero-sum game can always be found by the iterates of the Nash map or the solution to the BNN equations for any initial strategy frequency. This result solves the important problem as to how dynamical plays of a zero-sum game driven by individual players seeking out only localized advantages can eventually and collectively find a globally stable Nash equilibrium. Figure 3 shows, for a prototypical two-player zero-sum game, the trajectories of the Nash map for a small time increment and the BNN equation converge to an NE which is a saddle point on the payoff surface of one player and a global minimum on the excess potential surface, which can be viewed as a energy function for the dynamics of the BNN equation.

With the existence problem and the search problem for NE solved, we can employ alternative and practical methods to find them. One standard procedure to solve the min-max problem (1) is to solve the following linear programming problem as reviewed in Appendix (see Equation (5)).

There are both commercially available and free packages to solve such LP problems. It is well-known that the y solution and the optimal value for the objective function to the LP problem (5) are exactly the y solution and respectively the game value to the min-max problem (1), and the shadow price or the set of Lagrange multipliers for the LP problem is exactly the x solution to the min-max problem.

To complete the outline of our method, the edge weight from to is assigned to be from a Nash equilibrium of the min-max problem for node. That is, the y solution, which obviously depends on but the dependence is suppressed for simplicity, for each node gives the th row of the network matrix W of Figure 2. The x solution vector for each, is used to define the component profile for the node.

Thus, by our game theoretic approach the edge weights of the affinity network and its corresponding component profiles are the result of both conservation and diversity being maximized. More specifically, a high edge weight in the affinity graph indicates a strong affinity between taxon pairs relative to the others, and a high row weight, of node, indicates that the reference individual, , is somewhat unique or dissimilar with respect to the component compared to the other members of the taxon space.

The game values also yield important information about the affinity network. For example, for two topologically identical clusters, it is their average game values that set them apart, which in this sense the cluster with the higher average game value is a “tighter” or a more similar subnetwork than the latter.

The importance of a Nash equilibrium lies in the property that if we change the affinity frequency vector y from its optimal, then we may find a different diversity frequency vector x so that the corresponding expected similarity score is lower than the game value. Similarly, if we change the diversity frequency vector x from its optimal, we may then find a different affinity vector y so that the corresponding expected similarity score is larger. That is, deviating from the conservation optimal distribution may give rise to a greater diversification, and deviating from the diversity optimal distribution may give rise to a greater conservation. The dynamical state of the evolution, according to our model, is literally sitting at a saddle point of the expected similarity function; and the game value is a balanced tradeoff between reproducibility and diversity, a minimally guaranteed affinity.

A protein domain is a part of a protein sequence, a structural unit, that can function and evolve almost independently of the rest of the protein. Proteins often include multiple domains. Domain shuffling [20] or domain accretion [21] is an important mechanism in protein evolution underlying the evolution of complex functions and life forms. Figure 4 is a simple example of evolution of multidomain proteins illustrating how multidomain proteins can be evolved from simple single-domain proteins. Multiple evolutionary events including duplication, loss, recombination, and divergence generate complex proteins [22,23]. As shown in Figure 4, the evolutionary process of multidomain protein families also contains network relationships. As a consequence of their com-

plex evolutionary history, a large variation exists in the numbers, types, combinations, and orders of domains among member proteins from the same family.

In order to understand relationships of proteins and their functions, it is important to incorporate domain information when we study multidomain proteins. To show how we can apply our game-theory based method to reconstruct protein networks, we studied an example of the Regulator of G-protein Signaling (RGS) protein family.

We extracted a set of 66 (RGS family protein) sequences from the mouse genome. RGS sequences were found by performing a profile hidden Markov model search in HMMER [24,25] using the Pfam [26] families PF00615 (RGS) and PF09128 (RGS-like) as query sequences and with E-value threshold 10.

This RGS sequence set was subsequently used to HMMER search against Pfam database to find other domains present in the sequences. This step tries to identify all other domains that coexist with the RGS and RGSlike domains in our RGS proteins. From the sixty six RGS proteins, fifty eight Pfam domains (including RGS and RGS-like) were identified. Next, each of the individual domain sequences from each of the RGS proteins was extracted and used as the query sequence in a blastp sequence similarity search [27] against each and every sequence in the RGS protein sequence set.

The BLAST E-value was used as distance measure between each domain and each of the RGS protein sequences, so that an E-value of 0 is expected when using the domain from a sequence to BLAST search against the sequence itself and large E-values are expected when highly diverged domain sequences are identified. If a domain is not found on some sequence, we use 2870 as the maximum distance since on average this is the maximum possible E-value using BLAST on our data search space. In the end, we obtained all the distances between every domain sequence of every RGS protein and every RGS protein sequence. For the entries in each similarity matrix we used a log-transformed score of the E-value with following [10], where is the E-value obtained for the domain query against the subject protein. With one similarity matrix as the input, the LP problem shown in (5) can be solved. The solutions to the set of LP problems provide the edge weights for the RGS protein affinity network.

In the resulting network, eight clusters were identified within this protein space. Clusters are labeled according to their average game values, in descending order. The first four clusters are exhibited in Figure 5. As mentioned before, the game values yield important information about the clusters present in the affinity network. For example, Cluster 1 and Cluster 4 each include three proteins (nodes) and are topologically identical. However, their average game values are 146.5601 and 68.3844, respectively, which in this sense Cluster 1 is a tighter or a more similar subnetwork than Cluster 4.

The domain profile across the proteins in the first four clusters in the RGS affinity network is exhibited in Figure 6. The proteins are grouped by the clusters in the network graph. Clear profile pattern differences exist between the model clusters. For many of the clusters, the proteins contain the same domain and the weights placed on these domains in the LP solution (the vector) for each of the proteins are similar. The profile also highlights domains that are unique to specific clusters. For example, in the set of clusters shown the Pfam domain PF00631.17 is hallmark to Cluster 4 because it is present in all members of Cluster 4 and not elsewhere.

For a regular phylogenetic analysis of multidomain proteins, usually only sequence information of domains shared across all member proteins (e.g., RGS domain) can be used. A phylogenetic analysis using RGS domain sequences showed phylogenetic clusters largely consistent with the network clusters our method identified (data

not shown). However a regular phylogenetic analysis cannot represent information from many other domains that are not shared, nor network relationships as our affinity networks reveal.

As an additional validation of the network clusters, we provide the information for the proteins within each cluster in Figure 7. It clearly shows that different domain architectures are represented in different clusters. Sequence divergence within the same domain type (e.g., RGS_RZ-like domain for RGS 19/20 vs. RGS 17 proteins) is also recognized in separating Clusters 1 and 2. Furthermore we note that isoforms (proteins coded in alternatively spliced transcripts derived from the same gene) of the same gene fall into the same cluster even if some domains are missing in different isoforms as shown in the beta-adrenergic receptor kinase 2 isoforms 1 and 2 in Cluster 3. Therefore, using our game-theoretic framework, we incorporated both sequence diversity and domain information and produced a valid RGS protein network.