Probability as a coherent mathematical theory is said to have been “born in the gaming rooms of the seventeenth century” in attempts to solve one or another betting problem [16]. Among the most ancient forms of gambling are card games, which developed initially in Asia but became popular in Europe after the invention of printing [17]. Depending on what one considers a distinct game, experts in the subject estimate the number of card games to be between 1000 and 10,000 [18] [19]. Most card games are conducted under the assumption that the deck in play has been initially randomized. From a practical standpoint, a deck is considered random if players are unable to predict any sequence of cards following a revealed card. (Mathematically, there is on average 1 chance in n of guessing correctly the value of any unrevealed card in a deck of n randomly distributed cards).

The standard way to mix a deck of cards randomly is to shuffle it, for which purpose the riffle shuffle is perhaps the most widely studied form. To execute a riffle shuffle, one separates (“cuts”) the deck into two piles, then interleaves the cards by dropping them alternately from each pile to reform a single deck. The process can be performed either by hand or mechanically by an auto shuffler, like the device shown in Figure 1 used to acquire some of the data reported in this paper. Clearly, a single riffle shuffle cannot randomize an ordered deck because the order of cards from each pile is maintained. Indeed, in a perfect riffle shuffle of an even-numbered deck, whereby the deck is cut exactly in half and 1 card is dropped alternately from each pile, there would be no randomization at all. Instead, the sequences of cards resulting from a series of perfect riffle shuffles cycle through a fixed number of permutations leading back to the original card order. For example, a pack of 52 cards recycles after only 8 perfect “out-shuffles” (i.e. where the top card remains on top) [20]. However, under ordinary circumstances

where shuffles are not perfect, the order of the cards from each pile is degraded with each successive riffle shuffle.

The central question comprising the card randomization problem is this: How many riffle shuffles are required to randomize a deck of cards? More accurately stated: After how many shuffles can one detect no evidence of non-randomness? Various researchers have studied this question theoretically and arrived at statistically different answers, depending on the adopted measure of randomness. In the analysis of Bayer and Diaconis [1] , the measure of randomness of the deck is the so-called variation distance (VD) [4] [21] between the probability density Q n , m of n cards shuffled m times and the uniform density U n = 1 / n ! of the permutation group S n of n distinct objects. In the limit of large n, the VD analysis predicted that

m VD ( n ) ≈ 3 2 log 2 ( n ) (1)

shuffles should adequately randomize a deck of n cards. Thus m VD ( 52 ) is about 8 - 9. According to [1] , VD quantifies the mean rate at which a gambler could expect to win against a fair house by exploiting any residual pattern of the cards. The researchers also showed that the VD between Q n , m and U n takes the form

‖ Q n , m − U n ‖ = 1 − 2 2π ∫ − ∞ − 1 / 4 3 e − t 2 2 d t ≈ 0.115 (2)

for large n with m given by Equation (1). For complete randomness, the VD would equal 0.

In a numerical analysis by Trefethen and Trefethen [2] , the adopted measure of randomness was based on the Shannon entropy of the deck in the sense of information theory [22] [23]. If p j ( j = 1 , ⋯ , n ! ) is the probability of the j^th permutation of S n , then the Shannon entropy of the deck is given by

H = − ∑ j = 1 n ! p j log 2 p j (3)

where, by completeness,

∑ j = 1 n ! p j = 1 , (4)

and the information associated with the set of probabilities { p j } was defined as

I n = log 2 ( n ! ) − H . (5)

According to [2] I n in Equation (5) quantifies the rate at which an ideally competent coder could expect to transmit information if the signals were encoded in shuffled decks of cards. In the limit of large n, the information theoretic (IT) calculation predicted that

m IT ( n ) ≈ log 2 ( n ) (6)

shuffles should adequately randomize a deck of n cards. Thus m IT ( 52 ) is about 5 - 6, in contrast to m VD ( 52 ) . The numerically obtained results of [2] were subsequently proved theoretically by another research group [24].

The structure of relation (5) provides a mathematical definition of the word “information” consistent with its general vernacular use. If there is no uncertainty in the communication of any n-symbol message based on card sequence, then p j = 1 for each permutation j. In that case H = 0 and the information I n = log 2 n ! is maximum. If, however, every message received is completely uncertain as to card order, then p j = 1 / n ! for each permutation j, and therefore, by use of Equation (4), the information I n = 0 . Alternatively [25] , physicists and other scientists usually associate the concept of information with entropy H, Equation (3). The rationale is that the greater the uncertainty (i.e. H) of a message or physical system, the more information one gains by a binary decision (or measurement) that reduces the uncertainty. In a system with perfect order, H = 0; the outcome of any measurement or decision is completely predictable, and therefore no new information is to be gained. Both definitions of information prove useful later in the paper (Section 3.4).

Although the two analyses [1] and [2] led to statistically different distributions of randomness as a function of shuffle number, they both started from the same mathematical model of shuffling, referred to as the GSR shuffle, named for Gilbert and Shannon [26] and, independently, for Reeds [27]. The GSR shuffle involves the following steps. The deck is cut roughly in half according to a binomial distribution in which the probability that a pile contains k out of n cards is 2 − n C k n where

C k n ≡ n ! k ! ( n − k ) ! (7)

is the binomial combinatorial coefficient. The two halves are then riffled together such that the probability of a card being dropped from a pile is proportional to the number of cards in the pile.

The research literature on the randomization of cards by shuffling is vast. An extensive list of references that survey the development of the problem, of which virtually all papers are theoretical analyses or numerical modeling by computer simulation, can be found in [28]. To the best of the author’s knowledge, there has been no comprehensive, systematic experimental examination of the card ordering and patterns produced by manual shuffling to test whether the results conform to the GSR model or support the published theoretical predictions.

This paper reports on an extensive set of tests by which was measured the progression toward randomness of card sequences produced in multiple riffle shuffles manually and, for comparison, by a mechanical auto shuffler.

The basic theory and experimental outcomes of the following measures of randomness are discussed in Section 2: 1) runs with respect to the mean, 2) runs up/down, 3) rank ordering, 4) serial correlation (lag 1), and 5) theory of rising sequences.

Analysis of the data by information theory is discussed in Section 3.

Conclusions are presented in Section 4.

A run is defined as a succession of similar events preceded and succeeded by a different event. For example, the sequence of 12 symbols

b b a a a b b b a b a b

contains:

2runsof a oflength1 2runsof b oflength1 1runof b oflength2 1runof a oflength3 1runof b oflength3 (8)

or a total of 7 runs. If a sequence is random, then all permutations of symbol order should have the same probability of occurrence. From this invariance principle, as applied to a sequence containing n a symbols of type a, n b symbols of type b, and n = n a + n b symbols in all, it can be deduced that [29] :

・ the mean number of runs of a of length precisely k (where k ≥ 1 ) is

r ¯ k a = n a ! n b ( n b + 1 ) ( n − k − 1 ) ! ( n a − k ) ! n ! (9)

・ the mean number of runs of a of length k or greater (i.e. inclusive runs) is

R ¯ k a = n a ! ( n b + 1 ) ( n − k ) ! ( n a − k ) ! n ! (10)

・ the mean number of total runs of both kinds is

R ¯ = R ¯ 1 a + R ¯ 1 b = n + 2 n a n b n . (11)

Expressions for r ¯ k b , R ¯ k b follow, mutatis mutandis, from Equation (9) and Equation (10). Proofs of these expressions are given in [30] [31].

Two methods were employed in this paper to generate runs of binary symbols from the experimentally recorded sequences of digital card values.

The Spearman rank correlation coefficient r S is a nonparametric measure of the association between two random variables X and Y as defined by their rank order in a sequence of n pairs [33]

r S = 1 − 6 ∑ i = 1 n D i 2 n ( n 2 − 1 ) (31)

in which

D i = r ( x i ) − r ( y i ) (32)

is the difference between the ranks assigned to samples x i and y i (When a distinction is necessary, lower case letters (e.g. x) represent realizations of the abstract random variable which is usually expressed by an upper case letter (e.g. X)).

Values of r S range from −1 to +1, respectively signifying perfect anti-correlation (i.e. reverse rankings) and perfect correlation. It is r S 2 , however, rather than r S , that has a statistical interpretation; r S 2 is a measure of the variability of the data attributable to the correlation between variables X and Y [34]. Thus a relatively high correlation coefficient such as r S = 0.7 , means that only 49% of the variability is accounted for by the association between X and Y.

For independent variables (and therefore uncorrelated ranks), the expectation value and variance are respectively

〈 r S 〉 = 0 (33)

σ r S 2 = 1 n − 1 , (34)

and the test statistic

z rank = r S − 〈 r S 〉 σ r S = r S n − 1 → N ( 0 , 1 ) (35)

follows a standard normal distribution to good approximation [35].

Applied to the shuffling of cards, the variable Y signifies the initial card sequence { y i } m = 0 = { 1 , 2 , ⋯ , 52 } and variable X signifies the card sequence { x i } m = { x 1 , ⋯ , x n } of the m^th shuffle. Since the face values of the cards range from 1 to 52, the rank of a card is equal to its face value. Therefore an equivalent, but simpler, way to perform the rank correlation test is to calculate the cross correlation of ranks

C rank ≡ ∑ j = 1 n r ( x j ) r ( y j ) = ∑ j = 1 n j r ( x j ) (36)

where the second equality in (36) pertains specifically to the sequence of cards in a deck of n cards. The expectation value and variance of C rank are respectively

〈 C rank 〉 = 1 4 n ( n + 1 ) 2 → 36517 (37)

σ rank 2 = 1 444 ( n − 1 ) n 2 ( n + 1 ) 2 σ rank → 1640.15 (38)

with numerical evaluations for n = 52 . The test statistic

z rank = C rank − 〈 C rank 〉 σ rank (39)

can be shown to be identical to that of Equation (35) [33].

Figure 3 shows a plot of 〈 z rank 〉 , i.e. Equation (35) or (39) averaged over the N sets of data for each shuffle number m for both manual (blue) and auto (red) shuffled cards. The correlation between the card sequence of the m^th shuffle and the initial card sequence (m = 0) is interpreted as statistically 0 (i.e. for | z rank | ≤ 1 ) starting at about m = 6 or 7.

Serial correlation refers to the relationship between elements of the same series separated by a fixed interval. Given a sequence of elements { x j } for j = 1 , 2 , ⋯ , n , the serial correlation coefficient lag-k is defined by [36]

ρ k = ∑ j = 1 n x j x j + k − 1 n ( ∑ j = 1 n x j ) 2 ∑ j = 1 n x j 2 − 1 n ( ∑ j = 1 n x j ) 2 (40)

where x j + k is to be replaced by x j + k − n for all values of j such that j + k > n .

For the purpose of testing correlations in card order following shuffling, the most useful serial coefficient is ρ 1 , which measures the correlations between pairs of consecutive cards. It can be shown, however, that a test based upon the simpler statistic [37]

c 1 = ∑ j = 1 n x j x j + 1 (41)

is equivalent to a test based on ρ 1 . The mean and variance of c 1 are given by the following expressions [36] [37]

〈 c 1 〉 = S 1 2 − S 2 n − 1 (42)

σ c 1 2 = S 2 2 − S 4 n − 1 + S 1 4 − 4 S 1 2 S 2 + 4 S 1 S 3 + S 2 2 − 2 S 4 ( n − 1 ) ( n − 2 ) (43)

where

S k = ∑ j = 1 n x j k . (44)

For large n, the statistic

z serial = c 1 − 〈 c 1 〉 σ c 1 (45)

follows a standard normal distribution to good approximation.

Figure 4 shows a plot of 〈 z serial 〉 , i.e. Equation (45) averaged over the N sets of data for each shuffle number m for both manually (blue) and auto (red) shuffled cards. The correlation between the card sequence of the m^th shuffle and the initial card sequence (m = 0) is interpreted as statistically 0 (i.e. for | z serial | ≤ 1 ) starting at about m = 8 (manual) and m = 16 (auto).

A rising sequence, as defined in [1] , is a maximal consecutively increasing subset of an arrangement of cards. For example, consider a hand of 8 cards with the sequence of face values: 1 6 2 3 7 8 4 5. By displaying the cards in the following way

{ x i } = { 1 2 3 4 5 6 7 8 } (46)

one sees that the hand consists of two rising sequences (1,2,3,4,5) and (6,7,8) interleaved together. Successive riffle shuffles tend to double the number of rising sequences up to a maximum number of 1 2 ( n + 1 ) in the limit of an infinite number of shuffles. Note that a rising sequence is different from an ascending sequence (i.e. a run up): 1) The elements of a run up merely ascend, but do not have to increment successively; 2) The elements of a rising sequence do not have to be contiguous (as in a run), but can be separated by other elements.

It is shown in [1] that the probability of a particular permutation following a

riffle shuffle depends only on the deck size n and the number r of rising sequences in the permutation. Specifically, the probability that the m^th riffle shuffle of an ordered deck has r rising sequences is

Q n , m ( r ) = 1 2 m n C n 2 + n − r . (47)

The mean number of rising sequences in the permutation following m shuffles is then given by

〈 r n , m 〉 = ∑ r = 1 n r E n ( r ) Q n , m ( r ) (48)

where the Eulerian number [38]

E n ( r ) = ∑ n = 1 r + 1 ( − 1 ) r − j j n C r − j n + 1 (49)

is the number of permutations containing r rising sequences. Substitution of Equation (49) into Equation (48) leads to the simpler expression [39]

〈 r n , m 〉 = 2 m − n + 1 2 m n ∑ r = 1 2 m − 1 r n . (50)

The sum of powers of an uninterrupted sequence of positive integers, such as contained in expression (50), is given by Faulhaber’s formula [40]

∑ r = 1 R r n = R n + 1 n + 1 + 1 2 R n + ∑ k = 2 n B k k ! n ! ( n − k + 1 ) ! R n − k + 1 (51)

in which B_k is a Bernoulli number, defined by the generating function [41]

t e t − 1 = t 2 ( coth ( t / 2 ) − 1 ) = ∑ k = 0 ∞ B k t k k ! (52)

and given explicitly by

B 0 = 1                     B 1 = − 1 2 B k = { 0                                                                                 k = 3 , 5 , 7 , ⋯ − ( − 1 ) k / 2 ( 2 π ) − k 2 k ! ζ ( k )               k = 2 , 4 , 6 , ⋯ (53)

where the Riemann zeta function is defined by (Ref. [41] , pp. 329-330)

ζ ( p ) = ∑ k = 1 ∞ k − p . (54)

In the limit R → ∞ , the sum in the right side of Equation (51) becomes negligible, and therefore

lim R → ∞ ∑ r = 1 R r n → R n + 1 n + 1 − 1 2 R n . (55)

Substitution of relation (55) into (50) with R = 2 m leads to the asymptotic number of rising sequences after an infinite number of riffle shuffles

lim m → ∞ 〈 r n , m 〉 = r n , ∞ = 1 2 ( n + 1 ) → 26.5 (56)

as stated without proof at the start of this section. The numerical evaluation pertains to a deck with n = 52.

The mean-square number of rising sequences for m shuffles can be calculated numerically from Equation (47) and Equation (49)

〈 r n , m 2 〉 = ∑ r = 1 n r 2 E n ( r ) Q n , m ( r ) (57)

from which follows, also numerically, the theoretical (i.e. population) variance

σ n , m 2 = 〈 r n , m 2 〉 − 〈 r n , m 〉 2 . (58)

The author was unable to determine an analytical closed-form expression for (57) or (58).

Table 2 summarizes the relevant statistics of rising sequences based on the GSR model as a function of shuffle number m for a deck of 52 cards. It is seen that about 13 shuffles are required to achieve the asymptotic result of Equation (56). In Figure 5 the theoretically predicted mean number of rising sequences

〈 r n , m 〉 is compared with the observed numbers obtained by manually and auto shuffled cards averaged over the N data sets. Several features are to be noted:

・ In contrast to the statistical behavior graphically displayed in preceding figures which showed gradual randomization with increasing shuffle number m, the mean number of rising sequences underwent a relatively abrupt transition from a non-random state to the asymptotically random state at a threshold shuffle number m = 7 or 8 for manual shuffles and m = 11 or 12 for auto shuffles.

・ For m < 4 , the three curves (theory, manual shuffle, auto shuffle) yielded virtually identical results.

For m ≥ 5 , the rising sequences due to manual shuffling were statistically coincident with theoretical predictions, whereas shuffling by machine yielded too few rising sequences at each shuffle number up to the asymptotic number m A ≈ 13 . This feature suggests one can randomize a deck better by shuffling it manually than by use of a mechanical auto shuffling device like that in Figure 1.

As discussed briefly in Section 1.1, the Shannon entropy of a set of n symbols is given by

H = − ∑ j = 1 n ! p j log 2 p j (59)

where p j ( j = 1 , ⋯ , n ! ) is the probability of the j^th permutation of the n ! total number of ways to permute the symbols. By completeness, the set of probabilities { p j } satisfies Equation (4). Multiplied by a universal physical constant (Boltzmann’s constant k B ), the Shannon entropy, usually expressed in terms of natural logarithms, provides the basis for deriving the partition function―and therefore all the thermodynamic potentials―of equilibrium statistical mechanics [42]. From a physicist’s perspective, H is a universal measure of the disorder of a system, maximum randomization occurring when all p j are equal. For a maximally randomized system of n = 52 symbols, H = log 2 n ! ≈ 225.58 bits.

Although Equation (59) yields the entropy of a sequence of n distinct uncorrelated symbols, it does not predict the entropy correctly when the permutations are constrained by rules that create correlations among the symbols. To chart the increasing disorder in a system of n cards as a function of the number m of riffle shuffles one can calculate the entropy of all configurations of a fixed number r of rising sequences and then sum that entropy over the total number of rising sequences produced in the shuffle. In this case, the relevant probability function is

p n , m ( r ) = E n ( r ) Q n , m ( r ) (60)

with expressions for E n ( r ) and Q n , m ( r ) given respectively by Equation (49) and Equation (47). This procedure leads to a much lower maximum entropy than Equation (59) because it respects the constraints imposed on possible orderings by the physical mechanism of the riffle shuffle. It has been shown that the possible outcomes to m riffle shuffles of an ordered deck are equivalent to the outcomes of cutting a deck into 2 m packets and interleaving the cards from different packets in such a way that the cards from each packet maintain their relative order among themselves [1] [39].

Figure 6 shows the variation in p n , m ( r ) as a function of r for various increasing values of m. In the limit of large m, which for all practical purposes

starts at about m = 10 (blue curve), the distribution over r is a nearly perfect Gaussian function, shown by the dashed red curve in Figure 7, centered on the asymptotic number of rising shuffles, Equation (56), marked in the figures by a vertical black dashed line. The width (i.e. standard deviation) of the Gaussian is approximately 2.10, as given in Table 2 for m ≥ 9 .

The entropy of a deck of n cards as a function of shuffle number m, calculated from the probability distribution (60), takes the form

H n , m = − ∑ r = 0 n p n , m ( r ) log 2 p n , m ( r ) (61)

and is plotted in Figure 8 for decks of size n = 14, 26, and 52. The transition between initial complete order H n , 0 = 0 and maximum disorder is sharp like a phase transition, such as exhibited by the mean number of rising sequence in Figure 5. For n = 52, the maximum entropy H 52 , m ≥ 7 ≈ 3.12 bits is reached by the 7^th shuffle.

Equation (61) yields the total entropy of a card deck subject to m riffle shuffles. However, it does not provide information on the randomization of specific card associations, which is the kind of information that serious players might rely on for advantage in competition or gambling. For this purpose, the conditional entropy of pairs of ordered sequences was determined experimentally.

Let X and Y be two discrete random variables spanning the same range of n sequential integers ( i = 1 , 2 , ⋯ , n ) with joint probability function p X Y ( x , y ) and marginal probability functions

p X ( x ) = ∑ y = 1 n p X Y ( x , y ) p Y ( y ) = ∑ x = 1 n p X Y ( x , y ) (62)

each satisfying the completeness relation. The entropy (mean uncertainty) in receipt of n transmitted symbols { x i } or { y i } is

H ( X ) = − ∑ x = 1 n p X ( x ) log 2 p X ( x ) H ( Y ) = − ∑ y = 1 n p Y ( y ) log 2 p Y ( y ) (63)

The conditional entropy of the sequence { x i } , given that the sequence { y i } is known, is defined by [43]

H ( X | Y ) = − ∑ x = 1 n p X | Y ( x | y ) log 2 p X | Y ( x | y )   p Y ( y ) , (64)

where the condition probability p X | Y ( x | y ) is

p X | Y ( x | y ) = p X Y ( x , y ) / p Y ( y ) . (65)

(See also Ref. [22] , pp 52-53.) The joint entropy of X and Y is then given by

H ( X , Y ) = H ( X ) + H ( Y | X ) = H ( Y ) + H ( X | Y ) . (66)

Equation (66) states that the entropy of a joint event, e.g. X and Y, is the entropy of the former plus the conditional entropy of the latter when the former is known. One may also define the quantity

H [ X Y ] ≡ H ( X ) − H ( X | Y ) (67)

which is the decrease in entropy of the events X when it is known that events Y have occurred. Given the preceding interpretation, the function expressed by Equation (67) is taken to represent the information provided by knowledge of the events Y [43].

In the analysis of permuted card sequences in the following two sections, the information function H [ X Y ] is one of the important quantities to be deduced experimentally. Substitution into Equation (67) of the conditional probability H ( X | Y ) from Equation (66) leads to the symmetric relation

H [ X Y ] = H ( X ) + H ( Y ) − H ( X Y ) (68)

which is particularly useful for calculation.

To apply the preceding concepts to riffle shuffles, the experimental sequences of digital card values are transformed into two sets of binary values by the following procedure, schematically shown in Figure 9.

・ Given a decimal sequence of card values { x i } for i = 1 , 2 , ⋯ , n , create the binary sequence { b j } for j = 1 , 2 , ⋯ , n − 1 defined by

b j = { 1               if x j + 1 − x j = 1 0                   otherwise (69)

・ Transform the set { b j } into the set { c k } k = 1 , 2 , ⋯ , n − 2 , where

c k = b k + 1 − 1 2 b k . (70)

Transformation (69) generates a binary sequence of 1’s and 0’s, in which the symbol 1 signifies a pair of cards in numerical order (e.g. 4,5). Transformation

(70) converts the binary sequence into a sequence of four values 1, 0, and ± 1 2 . Figure 9 illustrates the significance of the four symbols:

Panel A: c = + 1 2 signifies that a 1 follows a 1.

Panel B: c = − 1 2 signifies that a 0 follows a 1.

Panel C: c = 1 signifies that a 1 follows a 0.

Panel D: c = 0 signifies that a 0 follows a 0.

Given the set { c k } , the following four pair-association statistics

n ( 0 , 0 ) = ∑ k = 1 n − 2 I 0 ( c k ) (71)

n ( 0 , 1 ) = ∑ k = 1 n − 2 I 1 ( c k ) (72)

n ( 1 , 0 ) = ∑ k = 1 n − 2 I − 1 2 ( c k ) (73)

n ( 1 , 1 ) = ∑ k = 1 n − 2 I 1 2 ( c k ) , (74)

in which

I α ( c k ) = { 1             c k = α 0             c k ≠ α (75)

count the number of events of the kinds represented respectively by panels A, B, C, D. To summarize, the statistic n ( α , β ) is the number of events of symbol α followed by symbol β , where both symbols can take on values of 0 or 1. The statistics n ( α , β ) satisfy the sum rule

∑ α = 0 , 1 β = 0 , 1 n ( α , β ) = n − 2 → 50 (76)

evaluated numerically above for a deck of n = 52 cards.

In this information theoretic analysis, it is useful to think of the α symbols as the realizations of a “message” variable A that represents a received signal of 1’s and 0’s, whereas the β symbols are the realizations of a following “prediction” variable B that represents a predicted signal of 1’s and 0’s. For each successive shuffle of the deck, the set of conditional probabilities p ( β | α ) determines the conditional entropy H ( B | A ) , which is the uncertainty in predicting B given knowledge of A.

It is straightforward to show that the conditional probabilities p ( β | α ) can be estimated from the pair association statistics (71)-(74) as follows

p ( 0 | 0 ) = n ( 0 , 0 ) n ( 0 , 0 ) + n ( 0 , 1 ) (77)

p ( 1 | 0 ) = n ( 0 , 1 ) n ( 0 , 0 ) + n ( 0 , 1 ) (78)

p ( 0 | 1 ) = n ( 1 , 0 ) n ( 1 , 0 ) + n ( 1 , 1 ) (79)

p ( 1 | 1 ) = n ( 1 , 1 ) n ( 1 , 0 ) + n ( 1 , 1 ) (80)

in the limit of a sufficiently large number of sets of shuffles. Note that the order of symbols in the argument of p ( β | α ) signifies that event α precedes event β , which is the reverse of the order of symbols in the argument of n ( α , β ) . Regrettably, this potential for confusion is the price required to maintain conventional statistical notation.

The a priori probabilities p A ( α ) of a received symbol α are given by

p A ( α ) = 1 n − 2 ∑ β = 0 , 1 n ( α , β ) , (81)

or explicitly

p A ( 0 ) = n ( 0 , 0 ) + n ( 0 , 1 ) n − 2 ,               p A ( 1 ) = n ( 1 , 0 ) + n ( 1 , 1 ) n − 2 . (82)

Similarly, the a priori probabilities p B ( β ) of a predicted symbol β are

p B ( β ) = 1 n − 2 ∑ α = 0 , 1 n ( α , β ) , (83)

or explicitly

p B ( 0 ) = n ( 0 , 0 ) + n ( 1 , 0 ) n − 2 ,               p B ( 1 ) = n ( 0 , 1 ) + n ( 1 , 1 ) n − 2 . (84)

The joint probability p A B ( α , β ) of a received symbol α and predicted symbol β is given by

p A B ( α , β ) = p A ( α ) p ( β | α ) = n ( α , β ) n − 2 (85)

where the second equality follows from combining relations (82) and (77)-(80).

Given the probability functions constructed above, the a priori entropies of the received (A) and predicted (B) signals are

H ( A ) = − ∑ α = 0 , 1 p A ( α ) log 2 p A ( α ) (86)

H ( B ) = − ∑ β = 0 , 1 p B ( β ) log 2 p B ( β ) (87)

and the total entropy of A and B is

H ( A B ) = − ∑ α = 1 , 0 β = 1 , 0 p A B ( α , β ) log 2 p A B ( α , β )   . (88)

The information, or decrease in uncertainty of values of B as a result of knowing values of A, is then given by Equation (68)

H [ B A ] = H ( B ) − H ( B | A ) = H ( A ) + H ( B ) − H ( A B ) . (89)

The entropy and information are in units of bits (“binary digits”). In statistical physics, where natural logarithms are usually used rather than logarithms to base 2, entropy and information are in units of nats.

An experiment was performed in which an initially ordered deck of n = 52 cards was subject to N = 11 sets of M = 19 riffle shuffles per set implemented by the auto shuffler in Figure 1, thereby generating M columns of card permutations for each set such as illustrated in Table 1. The cards were shuffled mechanically, rather than manually, so that the riffle shuffles would be executed as uniformly as possible. The pair association numbers n ( α , β ) for each of the M shuffles were then averaged over the N sets to yield the mean numbers of pair associations summarized in Table 3 and plotted in Figure 10 as a function of shuffle number m.

For a completely ordered deck prior to shuffling (m = 0), there are n − 2 = 50 occurrences of α = 1 followed by β = 1 , as shown by the plot of n ( 1 , 1 ) (red curve) in Figure 10. This number drops rapidly with increasing shuffle number, becoming effectively 0 by about m ≈ 8 . Correspondingly, the occurrence of α = 0

followed by β = 0 rises rapidly with increasing m, approaching ≈48, as shown by the plot of n ( 0 , 0 ) (blue curve). The plots of n ( 0 , 1 ) (green curve) and n ( 1 , 0 ) (orange curve), which start at 0 and then fall off gradually from a maximum of about 12 at m = 1, are virtually indistinguishable.

The conditional probabilities p ( β | α ) , deduced from the pair-association statistics by means of Equations (77)-(80), are plotted as a function of m in Figure 11. In each panel, the conditional probabilities satisfy the completeness relation

∑ β = 0 , 1 p ( β | α ) = 1 (90)

for α = 0 , 1 .

The plots in panel A, which show the conditional probabilities of prediction variable β given received variable α = 0 , begin at m = 1 because there is no event 0 in a completely ordered deck (m = 0). As the shuffle number m increases, the number of ordered pairs decreases, and p ( 0 | 0 ) approaches 1 while p ( 1 | 0 ) approaches 0. In panel B, the probabilities are conditioned on a received variable 1. As the number of 0 events increase with m, it follows again that p ( 0 | 1 ) approaches 1 and p ( 1 | 1 ) approaches 0. For a gambler or competitive player, the probability p ( 1 | 1 ) is particularly useful, since it quantifies the chance of a third card in order (e.g. 1,2,3), given prior receipt of two cards in order (e.g. 1,2). Empirically, this conditional probability is seen to be about 20% at the 5^th shuffle.

The entropy H ( B ) of the prediction variable and information H ( B ) − H ( B | A ) are summarized in Table 4 and plotted in Figure 12. The plot of information (red curve) was multiplied by a factor 10 to enhance visibility. The black double arrow marks the standard deviation of the information at shuffle number m = 12. As shown by Table 4, the information at all shuffle numbers is within ± 1 standard deviation of 0.

Since the randomness of a deck of cards is ordinarily expected to increase with the number of shuffles, as shown explicitly in Figure 8 for the entropy of rising sequences, the decrease of H ( B ) with shuffle number in Figure 12 calls for an explanation. In the initially ordered deck, all sequential pairs of cards are in order, and therefore both the message variable A and prediction variable B for any pair take the value 1 (as demonstrated in Figure 9). With each successive shuffle, successive pairs of cards become less and less ordered and variables B and A increasingly take the value 0. Thus, as shown in Figure 11, the conditional probabilities become increasingly predictable as they asymptotically approach either 1 (100% chance of an ordered pair occurring) or 0 (100% chance of an ordered

pair not occurring). Consequently, pair association variables B and A are so defined that their outcomes become more certain and their entropies (i.e. uncertainties) decrease as the deck becomes increasingly randomized.

To put into perspective the empirical results of this information theoretical analysis, it is to be recalled that 1 bit of information, as initially construed by Shannon who largely created the subject of information or communication theory [22] , corresponds to the reduction of uncertainty by 1 binary-valued decision―e.g. a “yes or no” or “1 or 0”. As pointed out in Section 1, the word information carries two different meanings, both of which are relevant here:

1) Information, as ordinarily defined by scientists, is associated with uncertainty, i.e. entropy H. Thus, the decrease in entropy H ( B ) for about the first 10 shuffles, as seen in Figure 12, represents a steady loss in information as the outcome (1 or 0) becomes more predictable.

2) Information can also be construed as a measure of the reduction in uncertainty in one variable (e.g. B) as a result of knowledge of another variable (e.g. A). From this perspective, Figure 12 shows that the card order of a preceding pair provided no statistically useful information for predicting the card order of the following pair at any shuffle number m > 0. The reason is that H ( B | A ) decreased with m to the same extent as did H ( B ) .