The stock market has been known to be a volatile place ( Shiller, 1991 ; Knight, 1921 ; Baaquie, 2007 ; Fox, 2009 ). In 1900, Louis Bachelier wrote his Ph.D. thesis, The Theory of Speculation, which formulated the first mathematical theory of the stochastic processes of the market ( Bachelier et al., 2006 ). In 1965, Eugene Fama developed the Efficient Market Hypothesis (EMH), which states that financial markets are “informatively efficient” and that no one can accurately predict the market’s future by any means possible ( Fama, 1970 ). EMH complements the fact that the market is indeed on a random walk, and that the current price of stocks reflects all the available information. In 1973, Fisher Black and Myron Scholes developed the BSM model that describes the market’s behavior in pure mathematical terms ( Black & Scholes, 1973 ), which provides a theoretical framework to derive a price estimate of European call and put options. Their log-normal distribution of the returns is based on Brownian motion ( Broadbent et al., 1928 ).

The mainstream ways of describing the market have been to model the state of the market and the traders’ actions separately, with the market being treated as a physical particle or mathematical entity, and the traders are seen as a completely separate external factor that should be left alone when attempting to describe the market.

To model the behavior of traders, Amos Tversky and Daniel Kahneman formulated prospect theory ( Kahneman & Tversky, 1979 ; Kahneman, 2013 ), which attempts to factor in the “humanity” aspect of trading; arguing that traders value gains and losses differently that’s known as loss aversion theory. They proposed that losses will always carry a greater emotional impact on an individual, thus gains are generally perceived as greater probability wise.

When studying the market, it is crucial to factor in both the market itself and the participating traders involved and not separately, because it is together, both the market and traders’ overall is what makes up the movement of the market; it is the market’s volatility that hampers the participating traders’ decisive decision-making ability (traders are initially hesitant whether to buy or sell) and in turn it is the “collective effort” of all the traders (some will buy while others will sell) that eventually determine the market’s trend direction (increase or decrease). Thus, the market and traders are intertwined or “entangled” with each other, the market’s volatility affects the traders and the traders’ actions in turn determine the trend of the market and vice versa, in this continuous looping cycle, which fully reflects that the market is truly in a random walk.

We present a methodology to model both the market’s volatile movement (increase or decrease) and the participating trader’s actions (buy or sell) by utilizing the concept of the quantum principle of superposition and illustrate that it is the two “entangled” that causes the market to be in a random walk which fully conforms to the observed market movement as stated by the efficient market hypothesis ( Xin & Xin, 2021 ).

Again, established agent-based and stochastic models tend to treat the market and traders separately, however, when modeling market behavior, we have to keep in mind that the market’s “behavior” emerges from the interactions between the market and traders together as a collective whole of a complex system, essentially 1 + 1 > 2. Our quantum-like framework is a unified framework that models both the market and participating traders together as a collective whole all under complex Hilbert Space. Our framework establishes that the market’s random walk arises from the complex interactions between the traders and the external environment rather than from the simple random fluctuations that are commonly described by stochastic models.

Building off our methodology, we’ve developed an AI assistant economist (one that can aid human economists to analyze and forecast the financial market) powered by our quantum-like evolutionary algorithm that can produce a short horizon prediction (one week) of the market’s future movement by studying one month of data from the Dow Jones Index ( Xin & Xin, 2024 ).

The rest of the paper is structured as follows: Section 2 details the methodology. Section 3 is the results. Section 4 is the conclusion.

The volatility of the market and the hesitation of the traders’ actions are essentially intertwined; the uncertain nature of the market hampers traders’ decision-making ability of when to buy and sell, and in turn it is the “collective” actions of all the participating traders that determines the markets’ closing price in this ever-changing cycle between the market and traders.

To effectively model both the volatility of the market and the traders’ actions of buy and sell, the concept of quantum superposition principle ( Silverman, 2008 ; Feynman, 2015 ; Dirac, 1958 ) can be utilized; by “superposing” both the market’s states and the traders’ actions. This can be modeled as in (1) and (2).

$|Q\rangle =c_1|q_1\rangle +c_2|q_2\rangle$ (1)

where $|q_1\rangle$ denotes the market increases; $|q_2\rangle$ denotes the market decreases. ${\omega }_1={\left|c_1\right|}^2$ is the objective frequency that the market increases; ${\omega }_2={\left|c_2\right|}^2$ is the objective frequency that the market decreases.

$|A\rangle ={\mu }_1|a_1\rangle +{\mu }_2|a_2\rangle$ (2)

where $|a_1\rangle$ denotes the trader believes that the market increases; $|a_2\rangle$ denotes the trader believes that the market decreases. $p_1={\left|{\mu }_1\right|}^2$ is the trader’s degree of belief that the market increases; $p_2={\left|{\mu }_2\right|}^2$ is the trader’s degree of belief that the market decreases.

The market and all the participating traders can be described as a complex system, as (3).

$|\psi \rangle =c_1|q_1\rangle \otimes {\displaystyle \underset{i=1}{\overset{N}{\prod }}|a_1^i\rangle }+c_2|q_2\rangle \otimes {\displaystyle \underset{i=1}{\overset{N}{\prod }}|a_2^i\rangle }$ (3)

where N is the number of traders in the group. The density operator of the complex system can be described as (4).

$\begin{array}{l}{\rho }_{\text{market}+\text{traders}}=|\psi \rangle \langle \psi |\\ ={\omega }_1|q_1\rangle \langle q_1|+{\omega }_2|q_2\rangle \langle q_2|+\left[c_1{c}_2^{*}|q_2\rangle \langle q_1|\otimes {\displaystyle \underset{i=1}{\overset{N}{\prod }}\langle a_1^i|a_2^i\rangle }+\text{H}\text{.C}.\right]\end{array}$ (4)

where the third term is a non-diagonalization term that represents the superposition of the market either increasing or decreasing as well as the traders’ being unable to deduce whether the market will increase or decrease. Traders will tend to randomly “guess” that the market is increasing or decreasing; the traders’ believing whether the market is increasing or decreasing is “orthogonal”, and when the number of participating traders is very large then the expectations of all the traders for whether the market will increase or decrease are then zero as (5).

${\displaystyle \underset{i=1}{\overset{N}{\prod }}\langle a_1^i|a_2^i\rangle }\stackrel{N\to \infty }{\to }0$ (5)

(4) then becomes (6).

${\rho }_{\text{market}+\text{traders}}\stackrel{N\to \infty }{\to }{\omega }_1|q_1\rangle \langle q_1|+{\omega }_2|q_2\rangle \langle q_2|$ (6)

When there is a vast number of participating traders involved ( $N\to \infty$), the market and all the participating traders as a whole tends to be in a random walk ( ${\rho }_{\text{market}+\text{traders}}\approx {\rho }_{\text{market}}$) as outlined by the efficient market hypothesis. ${\rho }_{\text{market}}$ as (7) is the actual observed density operator of the market; where ${\omega }_1$ is the observed objective frequency that the market will increase and ${\omega }_2$ is the observed objective frequency that the market will decrease ( ${\omega }_1\approx {\omega }_2=0.5$).

${\rho }_{\text{market}}={\omega }_1|q_1\rangle \langle q_1|+{\omega }_2|q_2\rangle \langle q_2|$ (7)

We have shown above that it is the market and the participating traders as a collective whole that make the market become a random walk. It is widely acknowledged that nobody can accurately predict the future trend of the market in the long run. Now the question becomes: is it possible to produce a short-horizon forecast of the market’s future trend?

To answer this question, we’ve developed a quantum-like evolutionary algorithm to power the AI assistant economist that utilizes both the quantum superposition principle and Genetic Programming (GP) ( Holland, 1975 ; Koza, 1992, 1994 ) to produce possible short-horizon predictions of the market’s future trend by machine learning historical trading data.

For the AI assistant economist (AI agent), we can hypothesize that before the AI agent makes its decision, “believes” whether the market will increase or decrease, are “superposed simultaneously” in its “mind”, which can be described by the density operator as in (8).

${\rho }_{\text{agent}}=|A\rangle \langle A|=p_1|a_1\rangle \langle a_1|+p_2|a_2\rangle \langle a_2|+{\mu }_1{\mu }_2^{*}|a_1\rangle \langle a_2|+{\mu }_1^{*}{\mu }_2|a_2\rangle \langle a_1|$ (8)

where $p_1$ is the AI agent’s degree of belief that the market increases, $p_2$ is the AI agent’s degree of belief that the market decreases. The third and fourth terms in (8) are the “quantum interference” terms that indicate the AI agent’s “mind” is undecided on whether the market will increase or decrease, where the AI agent can “think” that the market is both increasing and decreasing.

When an AI agent actually “decides” on whether the market increases or decreases, a projection of pure state to mixed state happens in the AI agent’s “mind” as (9), which describes the decision-making process of an AI agent.

${\rho }_{\text{agent}}\stackrel{\text{Decide}}{\to }{{\rho }^{\prime }}_{\text{agent}}=p_1|a_1\rangle \langle a_1|+p_2|a_2\rangle \langle a_2|$ (9)

The decision-making process is essentially just a projection from pure state to mixed state, where GP, an algorithm based on Darwinian natural selection ( Darwin, 1859 ), is utilized to evolve a satisfactory pure state. The pure state is essentially just a 2 × 2 matrix, where (9) can be described by the matrix form represented in (10).

${\rho }_{\text{agent}}=\left[\begin{array}{cc}{\rho }_{11}& {\rho }_{12}\\ {\rho }_{21}& {\rho }_{22}\end{array}\right]\stackrel{\text{projection}}{\to }{{\rho }^{\prime }}_{\text{agent}}=\left[\begin{array}{cc}{p}_1& 0\\ 0& p_2\end{array}\right]=p_1|a_1\rangle \langle a_1|+p_2|a_2\rangle \langle a_2|$ (10a)

$|a_1\rangle =\left[\begin{array}{c}1\\ 0\end{array}\right],|a_2\rangle =\left[\begin{array}{c}0\\ 1\end{array}\right];|a_1\rangle \langle a_1|=\left[\begin{array}{cc}1& 0\\ 0& 0\end{array}\right],|a_2\rangle \langle a_2|=\left[\begin{array}{cc}0& 0\\ 0& 1\end{array}\right]$ (10b)

Because the pure density operator ${\rho }_{\text{agent}}$ is just an arbitrary 2 × 2 matrix, we can then approximately construct this density operator with the 8 most basic quantum gates as (11), leading it to become a “matrix tree” ( Xin et al., 2023 ).

$\left\{\begin{array}{c}H=\frac{1}{\sqrt{2}}\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]X=\left[\begin{array}{cc}0& 1\\ 1& 0\end{array}\right]Y=\left[\begin{array}{cc}0& -\text{i}\\ \text{i}& 0\end{array}\right]Z=\left[\begin{array}{cc}1& 0\\ 0& -1\end{array}\right]\\ S=\left[\begin{array}{cc}1& 0\\ 0& \text{i}\end{array}\right]D=\left[\begin{array}{cc}0& 1\\ -1& 0\end{array}\right]T=\left[\begin{array}{cc}1& 0\\ 0& {\text{e}}^{\text{i}\pi /4}\end{array}\right]I=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]\end{array}\right\}$ (11)

This “matrix tree” is the tree that’s constructed from the 8 basic quantum gates in (11) by the 3 operators’ addition (+), multiplication (×), and or (//); the final form that the matrix tree takes is a 2 × 2 matrix, hence the name matrix tree. After constructing an individual “matrix tree”, we can then construct a population of “matrix trees”, and then by using the fitness function as the evaluation criteria, the most satisfactory density matrix ${\rho }_{\text{agent}}$ from the population is evolved through generations of natural selection. The “matrix tree” is essentially a decision tree that guides the AI agent which strategies to “take” with corresponding actions. At any given time, the expected value under the current environment (the market is increasing or decreasing) and the corresponding actions (the AI agent “thinks” that the market is increasing or decreasing) can be represented as (12).

$\begin{array}{l}{\rho }_{\text{market}}\otimes {\rho }_{\text{agent}}\\ ={\omega }_1{p}_1{\left|\langle q_1||a_1\rangle \right|}^2+{\omega }_1{p}_2{\left|\langle q_1||a_2\rangle \right|}^2+{\omega }_2{p}_1{\left|\langle q_2||a_1\rangle \right|}^2+{\omega }_2{p}_2{\left|\langle q_2||a_2\rangle \right|}^2\end{array}$ (12)

where (12) is the composite system of the market and the AI agent. Essentially, (12) describes the four possible outcomes of every “decision” made by the AI agent; if the market is increasing or decreasing and the AI agent “thinks” or “doesn’t think so” and vice versa; when the AI agent “thinks” correctly in line with the corresponding motion of the market it’s “rewarded”, if not it’s “punished”. The expected value for the AI agent is the possible scenarios of what the outcome could be paired with the state of the market that’s being observed, as in (13). If the training data has N number of values, then the fitness function for the “matrix tree” is defined as (14), and it is the total sum of all the expected values of each “decision made” by the AI agent.

$E{V}_t=\{\begin{array}{l}{\omega }_1{p}_1,\text{market increases and AI agent}"\text{thinks}"\text{so with probaility}{p}_1\\ -{\omega }_1{p}_2,\text{market increases and AI agent doesn <msup> n <mo> ′ </mo> </msup> t}"\text{think}"\text{so with probability}{p}_2\\ -{\omega }_2{p}_1,\text{market decreases and AI agent does <msup> n <mo> ′ </mo> </msup> t}"\text{think}"\text{so with probability}{p}_1\\ {\omega }_2{p}_2,\text{market decreaes and AI agent}"\text{thinks}"\text{so with probability}{p}_2\end{array}$(13)

${\text{fitness}}_{\text{matrixTree}}={\displaystyle \underset{t=1}{\overset{N}{\sum }}E{V}_t}$ (14)

If there are M number of individuals in a population of “matrix trees”, the most satisfactory “matrix tree” is the one that possesses the maximum fitness function that can be described as in (15).

${\rho }_{\text{agent}}^{\text{output}}=\arg \underset{a}{\max }\left\{{\text{fitness}}_{\text{matrixTree}},k=1,\cdots ,M\right\}$ (15)

By learning historical data, the more rewards that are reaped, then the more accurate chance, there is of predicting the next outcome of whether the market will increase or decrease.

In this paper, we produced short-horizon forecast outcomes by studying a small sample of data. Data from October 4th, 2024 to November 1st, 2024 of the Dow Jones Industrial Average Index was used as training data; the following week of November 4th, 2024 to November 8th, 2024 was forecasted. The data was trained twice consecutively, with 6 possible forecast outcomes produced each session by 3 AI agents. From the 12 total possible forecast outcomes produced by means of majority rules, a final trend sequence of whether the market will increase or decrease is produced to analyze the future trend of the market. The fitting results of the two training sessions are shown in Figure 1 and Figure 2 . The trend sequence produced by means of majority rules from the 12 individual possible forecast outcomes is shown in Table 1 .

Using this action sequence produced, the future trend of the market and the market’s volatility can be analyzed. The actual recorded trend of the Dow Jones for the following week is listed in the DJIA Trend column, while the trend sequence that’s produced is listed in the trend sequence column, where 0 represents the AI agent “believes” that the market will increase and 1 represents the AI agent “believes” that the market will decrease.

For this action sequence produced, the predicted trend of the market will be {Increase, Increase, Increase, Decrease, Increase}, in which only the first value predicted was wrong, thus resulting in odds of 80% accuracy. The main emphasis of the 80% accuracy in this case only applies to the short forecast horizon, i.e. the next trading week of 5 data points; for a longer period of time, for example if the forecast horizon was extended to 3 months (60 data points), then it would not be possible for our algorithm to obtain odds of 80%. Essentially, for a longer forecast, the odds would still be closer to 50-50.

In this paper, we presented a methodology to describe both the volatility of the market and the participating traders’ actions together in an intertwined model. By utilizing the concept of quantum superposition principle to model both the state of the market and all the participating traders’ possible actions as a whole, we show that the market is indeed in a random walk as stated by the efficient market hypothesis. Unlike traditional methods that don’t factor in the participating traders, which treat the market as a mathematical or physical entity (particle) where calculus is needed to describe it; we are able to subtly model both the market without a statistical approach.

Since the future is inherently unpredictable, no matter what we do, there is no universal omnipotent method that can find the crystal ball to peer into and see the future, therefore, if a black swan suddenly landed, there is really nothing we can do to foresee it. Thus, in this paper and our methodology’s approach in general, we have deliberately chosen a small sample of historical data of about 20 data points to produce a short horizon forecast of 5 future points, assuming that the market will “keep” its “relative” trend of the recent past in the near future, and even in this case, our method is not 100% fail-proof—odds may not always reach 80%. However, if a black swan really does alight, our AI assistant economist won’t be able to predict this either, in which we will just have to “trust our instincts” to “throw” the dice back and hope for the best outcome.

Sincere thanks to the managing editor Joy Deng for a rare attitude of high quality. This work received no funding.