The goal of this article is to fill some potential gaps in the literature and introduce some notation that should be useful in applications. To motivate the main result, which may look abstract, we consider a concrete example. The reader who just wants to see and use the main result of this paper may jump directly to Section 3.

Let $f:R^2\to R^2$ be given by $x\left(u,t\right)=u+t$, $y\left(u,t\right)=u-t$, and $g:R^2\to R$ be given by $z=x^2+y$. Thinking of $z$ as a function of $\left(u,t\right)$ via $f$, if one wants to find the rate of change of $z$ with respect to $u$ at the point $\left(1,1\right)$, we can use the chain rule

$\frac{\partial z}{\partial u}=\frac{\partial z}{\partial x}\frac{\partial x}{\partial u}+\frac{\partial z}{\partial y}\frac{\partial y}{\partial u},$(1)

to get $\frac{\partial z}{\partial u}=2x\cdot 1+1\cdot 1=2\left(u+t\right)+1=2u+2t+1$, which evaluated at $\left(1,1\right)$ gives the answer 5. Note that found the answer without computing the function $g\circ f$, which is ${(u+t)}^2+\left(u-t\right)$, which we could have differentiated with respect to $u$ to get the same answer. The chain rule is especially useful if one wants to find the rate of change of $z$ with respect to $u$, knowing the other rates of change in Equation (1); this happens often in real life situations, where one may not know the functions $f$ and $g$ explicity, but knows the instantaneous rates of change on the right side of Equation (1). Such problems are called related rates problems.

Now suppose we are given functions $f:R^2\times R^3\to R^4\times R^5$ and $g:R^4\times R^5\to R$. We can compose the two to get the function $g\circ f:R^2\times R^3\to R$. Let $z$ denote the coordinate of the codomain $R$, and suppose we want to find the directional derivative of $z$ relative to a change in the coordinates of the $R^2$ in the domain of $g\circ f$ (keeping the coordinates of $R^3$ fixed) at some given point in the domain $R^2\times R^3$. We could, of course, write down the function $g\circ f$ in terms of the coordinates of $R^2\times R^3$ and then take partial derivatives, etc., but that is more work than needed. In this article, we derive a chain rule (formula (8) below) where the calculation can be done in a manner similar to the chain rule in the previous paragraph, thus making the calculation much easier (see Remark 3.3). Note that in the example above, we did not give formulas to define $f$ and $g$ since the number of variables is too much; the reader can easily make up an explicit example. Our chain rule is also useful for related rate problems in this context, either for computations or for theoretical use. In fact, the reason for the author to write up this article was that a colleague in the Economics department asked for a chain rule in a context similar to the one above, and neither of us could find it in the form that was needed in the literature. So it is hoped that this article fills a potential gap in the literature, and can be quoted for the sake of completeness in papers that need this version of the chain rule.

The organization of the rest of this article is as follows. In Section 2, we recall the classical setting alluded to two paragraphs above, and introduce a notation for the Jacobian matrix that may be new, and is very convenient. In Section 3, we consider a more general setting of the type discussed in the previous paragraph, and give the more general chain rule alluded to above, which involves a new object that we call a partial Jacobian matrix. Finally, in Section 4, we give the proof of our generalization of the chain rule.

Suppose $f:R^n\to R^m$ is differentiable. Let $x$ denote the vector of coordinate functions on the domain (so $x_1,\cdots ,x_n$, the components of $x$, are the coordinates of $R^n$) and let $y$ denote the vector of coordinate functions on the codomain. A standard notation for the Jacobian matrix of $f$ is $\frac{\partial \left(f_1,\cdots ,f_m\right)}{\partial \left(x_1,\cdots ,x_n\right)}$ and $J_f$; let us introduce another notation:

Definition 2.1. Let

$\frac{\text{d}y}{\text{d}x}$

denote the Jacobian matrix of $f$.

The notation above is clearly related to the earlier notation (our notation may not be new, but we have not seen it before), and also makes notational sense: the

$\left(j,i\right)$-th entry of $\frac{\text{d}y}{\text{d}x}$ is $\frac{\partial y_j}{\partial x_i}$.

Let $g:R^m\to R^k$ be differentiable, and let $z$ denote the vector of coordinate functions on the codomain. Letting $D$ denote the derivative as usual, the chain rule says that for $a\in R^n$, we have

$D\left(g\circ f\right)\left(a\right)=Dg\left(f\left(a\right)\right)\circ Df\left(a\right).$(2)

This looks nice in theory and is very convenient for theoretical purposes, but has to be unraveled in applications and may be too abstract for people working in applied areas. A more concrete restatement is the matrix version:

$J_{g\circ f}\left(a\right)=J_g\left(f\left(a\right)\right)J_f\left(a\right),$(3)

where the product on the right is matrix multiplication. Equation (3) still has to be unraveled, as we shall do soon, but first note that in the notation of Definition 2.1, the equation becomes

$\frac{\text{d}z}{\text{d}x}=\frac{\text{d}z}{\text{d}y}\frac{\text{d}y}{\text{d}x},$(4)

where again the product on the right is the matrix product. The rule looks much nicer with the new notation, and moreover, if $n=m=k=1$, then it becomes the usual chain rule in the Leibnitz notation, namely

$\frac{\text{d}z}{\text{d}x}=\frac{\text{d}z}{\text{d}y}\frac{\text{d}y}{\text{d}x}.$

As usual, for $j=1,\cdots ,m$, let $f_j$ denote the $j$-th component function of $f$; thus $f=\left(f_1,\cdots ,f_m\right)$. Recall that for $i=1,\cdots ,n$, and for $a\in R^n$, the symbol $D_i{f}_j\left(a\right)$ denotes the partial deriviative of $f_j$ at $a$ with respect to the $i$-th coordinate; thus $D_i{f}_j\left(a\right)$ is $\frac{\partial y_j}{\partial x_i}$ evaluated at $a$. The following version of the chain rule is very explicit; it is well known, and the only reason that we are stating it as a result is that it does not seem to be easy to find the statement in the literature (for example, in ( [1] , Theorem~2.9) and ( [2] , pp. 61-62), it is only stated for $k=1$, and in [3] , it is only mentioned in the proof of Proposition 3.10).

Proposition 2.2. For $i=1,\cdots ,n$, and $\ell =1,\cdots ,k$, we have for $a\in R^n$,

$D_i{\left(g\circ f\right)}_{\ell }\left(a\right)=\underset{j=1}{\overset{m}{{\displaystyle \sum }}}{D}_j{g}_{\ell }\left(f\left(a\right)\right)D_i{f}_j\left(a\right)$, and(5)

$\frac{\partial z_{\ell }}{\partial x_i}=\underset{j=1}{\overset{m}{{\displaystyle \sum }}}\frac{\partial z_{\ell }}{\partial y_j}\frac{\partial y_j}{\partial x_i}.$(6)

Proof. This follows from Equation (3), the definition of the Jacobian matrix, and the definition of matrix multiplication. □

Equation (6) above may be the most useful version of the chain rule in applications.

For $i=1,\cdots ,n$ and $j=1,\cdots ,m$, let $X_i$ and $Y_j$ each be $R^p$ for some $p$ (so for example, we could have $X_1=R^2$, $Y_2=R^5$, etc.), and let $u_i$ and $v_j$ denote the dimensions of $X_i$ and $Y_j$ respectively (so $X_i=R^{u_i}$ and $Y_j=R^{v_j}$). Let $f:X_1\times \cdots \times X_n\to Y_1\times \cdots \times Y_m$ be differentiable (regarding the domain and the codomain as finite dimensional vector spaces in the natural way). Note that if each of $X_i$ and $Y_j$ are $R$, then $X_1\times \cdots \times X_n=R^n$ and $Y_1\times \cdots \times Y_m=R^m$ as vector spaces, so we are in the classical setting of Section 2; thus our setup generalizes the classical setting. The function $f$ above can be thought of as a function whose inputs and outputs, instead of being tuples of numbers, are tuples of vectors (not necessarily in the same vector space). Such functions arise often in applied areas such as Economics and Engineering.

For $i=1,\cdots ,n$ and $j=1,\cdots ,m$, let $x_i$ denote the vector of coordinate functions on $X_i$ (so $x_{i,1},\cdots ,x_{i,u_i}$, the components of $x_i$, are the coordinates of $X_i$), and let $y_j$ denote the vector of coordinate functions on $Y_j$.

Definition 3.1. For each $i,j$, denote by

$\frac{\partial y_j}{\partial x_i}$

the matrix whose $\left(r,s\right)$-th entry for $s=1,\cdots ,u_i$ and $r=1,\cdots ,v_j$ is $\frac{\partial y_r}{\partial x_s}$; we call it a partial Jacobian matrix.

The reason for this notation and terminology is that while this object looks like the Jacobian matrix in the classical setting of Section 2, which might suggest the notation $\frac{\text{d}{y}_j}{\text{d}{x}_i}$ that was used for the Jacobian matrix therein, it plays the role of a partial dervative in the chain rule analog of (6), as we shall soon see (Equation (8) below).

For $\ell =1,\cdots ,k$, let $Z_{\ell }$ each be $R^p$ for some $p$, and let $z_{\ell }$ denote the vector of coordinate functions on $Z_{\ell }$. Let $g:Y_1\times \cdots \times Y_m\to Z_1\times \cdots \times Z_k$ be differentiable. Thinking of each domain and codomain as real vector spaces in the natural way and letting $D$ denote the derivative as usual, the chain rule again says that for $a$ in the domain of $f$,

$D\left(g\circ f\right)\left(a\right)=Dg\left(f\left(a\right)\right)\circ Df\left(a\right).$(7)

This equation is mostly of theoretical use, and is difficult to use in practice. Also, we are not distinguishing between the components of each domain (e.g., the components $X_1,\cdots ,X_n$ of the domain of $f$) in any way or treating an individual component special. A similar criticism holds for the analogs of equations (3) and (4), of which Equation (3) holds verbatim even in this situation.

We have the following version of the chain rule, which we prove in the next section:

Theorem 3.2. In the setup above, for $\ell =1,\cdots ,k$, and $i=1,\cdots ,n$, we have

$\frac{\partial z_{\ell }}{\partial x_i}=\underset{j=1}{\overset{m}{{\displaystyle \sum }}}\frac{\partial z_{\ell }}{\partial y_j}\frac{\partial y_j}{\partial x_i}.$(8)

The equation above distinguishes between the components of each domain, unlike Equation (7) above, and is the analog of Equation (6) from the classical setting, which it agrees with when each of $X_i$, $Y_j$, and $Z_{\ell }$ is $R$. Equation (8) may be the most concrete form of the chain rule in this general setting, and should be especially useful for calculations, but also for some theoretical purposes.

Remark 3.3. As an example of an application, if $u\in X_i$ is a unit vector, then the directional derivative¹ of the composite map $X_i\to X_1\times \cdots \times X_n\stackrel{g\circ f}{\to }{Z}_1\times \cdots \times Z_k\to Z_{\ell }$ (where the first map is the natural inclusion, and the last map is the natural projection) at a point $a\in X_1\times \cdots \times X_n$ in the direction of $u$ is

$\frac{\partial z_{\ell }}{\partial x_i}\cdot u=\left(\underset{j=1}{\overset{m}{{\displaystyle \sum }}}\frac{\partial z_{\ell }}{\partial y_j}\frac{\partial y_j}{\partial x_i}\right)\cdot u=\underset{j=1}{\overset{m}{{\displaystyle \sum }}}\left(\frac{\partial z_{\ell }}{\partial y_j}\frac{\partial y_j}{\partial x_i}\cdot u\right)$

evaluated at $a$ (this follows from Theorem 3.2 and ( [2] , Theorem~5.1), for example).

Next we give a more theoretical version of Theorem 3.2, for which we need a generalization of the usual partial derivative.

Definition 3.4. For each $j$, let $f_j$ denote the composite function

$X_1\times \cdots \times X_n\stackrel{f}{\to }{Y}_1\times \cdots \times Y_m\to Y_j,$

where the second map is the natural projection. Let $a$ be an element of $X_1\times \cdots \times X_n$. For each $i$, let ${\iota }_{a,i}$ denote the map $X_i\to X_1\times \cdots \times X_n$ that takes $x\in X_i$ to the element of $X_1\times \cdots \times X_n$ whose $p$-th component is the $p$-th component of $a$, except for $p=i$, when it is $x$. Denote by $D_i{f}_j\left(a\right)$ the derivative at the $i$-th component of $a$ of the composite map

$X_i\stackrel{{\iota }_{a,i}}{\to }{X}_1\times \cdots \times X_n\stackrel{f}{\to }{Y}_1\times \cdots \times Y_m\to Y_j,$

where the last map is the natural projection; we call it the $i$-th partial derivative of $f_j$ at $a$.

An alternate definition, which is perhaps more conceptual, is given after Definition 4.1 in the next section. Note that the matrix of $D_i{f}_j\left(a\right)$ in the standard bases is $\frac{\partial y_j}{\partial x_i}$ evaluated at $a$. If each of $X_i$ and $Y_j$ are $R$, then it is the usual partial derivative $D_i{f}_j\left(a\right)$; we remark that even in this situation, our conceptual definition above (or the equivalent one in the next section) in terms of maps seems to be missing in the literature (e.g., it is not there in [1] [2] , and [3] ), and could be useful for theoretical purposes (e.g., it is immediate that the partial derivative of a sum is the sum of the partial derivatives, since the partial derivative inherits that property of the derivative, from our definition). Note that in what follows, the definitions of $D_j{g}_{\ell }$ and $D_i{\left(g\circ f\right)}_{\ell }$ are analogous to that of $D_i{f}_j$.

Theorem 3.5. In the setup above, for $\ell =1,\cdots ,k$, and $i=1,\cdots ,n$, we have for $a$ in $X_1\times \cdots \times X_n$,

$D_i{\left(g\circ f\right)}_{\ell }\left(a\right)=\underset{j=1}{\overset{m}{{\displaystyle \sum }}}{D}_j{g}_{\ell }\left(f\left(a\right)\right)\circ D_i{f}_j\left(a\right).$(9)

The equation above also distinguishes between the components of each domain, unlike Equation (7) above, and is the analog of Equation (5) from the classical setting, which it agrees with when each of $X_i$, $Y_j$, and $Z_{\ell }$ is $R$.

Remark 3.6. As mentioned in the second proof of Theorem 3.5 in the next section, the proof does not use the fact that the $X_i$, $Y_j$, and $Z_k$ are finite dimensional vector spaces over the real numbers, just that they are normed vectors spaces, so in particular Theorem 3.5 applies even if any of them are normed vector spaces of infinite dimension and/or over the complex numbers (over the complex numbers, the derivative is in the Fréchet sense, for example, as in ( [4] , Section VIII.1)).

Remark 3.7. Recall that given a smooth map $f:X\to Y$ between two smooth manifolds, the derivative induces a map $df:TX\to TY$ between the tangent bundles of the manifolds. The classical chain rule translates to a chain rule for such maps. Similarly our chain rule (9) applies to maps between products of manifolds, and gives a chain rule for products of manifolds, which we now state more precisely. For each $i$, $j$, $\ell$ as above, let ${\mathcal{X}}_i$, ${\mathcal{Y}}_j$, and ${\mathcal{Z}}_{\ell }$ be manifolds² and let $f:{\mathcal{X}}_1\times \cdots \times {\mathcal{X}}_n\to {\mathcal{Y}}_1\times \cdots \times {\mathcal{Y}}_m$ and $g:{\mathcal{Y}}_1\times \cdots \times {\mathcal{Y}}_m\to {\mathcal{Z}}_1\times \cdots \times {\mathcal{Z}}_k$ be morphisms. Then the classical chain rule says that $d\left(g\circ f\right)=dg\circ df$. Now let $d_i{f}_j$ denote the map induced by ${\mathcal{X}}_i\to {\mathcal{X}}_1\times \cdots \times {\mathcal{X}}_n\stackrel{f}{\to }{\mathcal{Y}}_1\times \cdots \times {\mathcal{Y}}_m\to {\mathcal{Y}}_j$ (where the first map is the natural inclusion and the second map is the natural projection) on the corresponding tangent bundles; similar definitions apply to $d_j{g}_{\ell }$ and $d_i{\left(g\circ f\right)}_{\ell }$. Then our discussion above gives the following application and analog of Theorem 3.5: $d_i{\left(g\circ f\right)}_{\ell }={\displaystyle {\sum }_{j=1}^m{d}_j{g}_{\ell }\circ d_i{f}_j}$. In fact, by the previous remark, this will apply to certain infinite-dimensional manifolds, in particular to Banach manifolds.

We continue using the notation of the previous section. First note that Theorem 3.2 and Theorem 3.5 follow from each other by considering the basis of a linear transformation in the standard basis. We now prove Theorem 3.2; later, we shall give an independent proof of Theorem 3.5.

Let $p$ be an indexing variable for the entries of $z_{\ell }$ and $q$ an indexing variable for the entries of $x_i$. By definition, the $\left(p,q\right)$-th entry in the matrix on the left side of Equation (8) is $\frac{\partial z_{\ell ,p}}{\partial x_{i,q}}$. Now $z_{\ell ,p}$ is a function of $x_{i,q}$ via the inter-mediate variables $y_{j,r}$ for $j=1,\cdots ,m$ and $r=1,\cdots ,v_j$ for each $j$. So by the classical version (6) of the chain rule, we see that the left side of Equation (8) is

$\underset{j=1}{\overset{m}{{\displaystyle \sum }}}\underset{r=1}{\overset{v_j}{{\displaystyle \sum }}}\frac{\partial z_{\ell ,p}}{\partial y_{j,r}}\frac{\partial y_{j,r}}{\partial x_{i,q}}.$(10)

The $\left(p,q\right)$-th entry of the term $\frac{\partial z_{\ell }}{\partial y_j}\frac{\partial y_j}{\partial x_i}$ on the right side of Equation (8) is the product of the $p$-th row of $\frac{\partial z_{\ell }}{\partial y_j}$ and the $q$-th column of $\frac{\partial y_j}{\partial x_i}$. Now the $p$-th row of $\frac{\partial z_{\ell }}{\partial y_j}$ is

$\left[\begin{array}{l}\frac{\partial z_{\ell ,p}}{\partial y_{j,1}}\cdots \frac{\partial z_{\ell ,p}}{\partial y_{j,v_j}}\hfill \end{array}\right]$

and the $q$-th column of $\frac{\partial y_j}{\partial x_i}$ is

$\left[\begin{array}{c}\frac{\partial y_{j,1}}{\partial x_{i,q}}\\ ⋮\\ \frac{\partial y_{j,v_j}}{\partial x_{i,q}}\end{array}\right].$

So the product of the row and column becomes

$\underset{r=1}{\overset{v_j}{{\displaystyle \sum }}}\frac{\partial z_{\ell ,p}}{\partial y_{j,r}}\frac{\partial y_{j,r}}{\partial x_{i,q}}.$

As we sum these terms over $j$ on the right side of Equation (8), we get the expression (10), which recall is the left side of Equation (8). This proves that the two sides of Equation (8) are equal, which proves Theorem 3.2, and thus Theorem 3.5 as well.

We now give a second proof of Theorem 3.5, which gives another proof of Theorem 3.2 as well. In fact, the proof of Theorem 3.5 that we give below does not use the fact that $X_i$, $Y_j$, and $Z_{\ell }$ are finite dimensional, and so applies to normed vector spaces (even over the complex numbers) that need not be finite dimensional. Another reason for giving a second proof of Theorem 3.5 is to note how the second proof is conceptual and illustrates where the linearity of the derivative is used, while the first proof is computational and hides some of the ideas of the proof.

The following definitions will be convenient in the proof.

Definition 4.1. If $h\in X_i$, then let $h^i$ denote the vector in $X_1\times \cdots \times X_n$ whose components are all 0 except the $i$-component, which is $h$; so $h^i$ is the image of $h$ under the natural inclusion $X_i\to X_1\times \cdots \times X_n$, and $h={\displaystyle {\sum }_i{h}^i}$. Let $a\in X_1\times \cdots \times X_n$. If $W$ is a vector space and $\psi :X_1\times \cdots \times X_n\to W$ is a linear map, then define $D_i\psi \left(a\right):X_1\to W$ by

$D_i\psi \left(a\right)\left(h\right)=D\psi \left(a\right)\left(h^i\right);$(11)

it is a linear map.

Note that the definition of $D_i\psi \left(a\right)$ above is consistent with Definition 3.4: when $W=Y_j$ and $\psi =f_j$, we get $D_i\psi \left(a\right)=D_i{f}_j\left(a\right)$.

Let $h\in X_i$. Then

$\begin{array}{c}{D}_i{\left(g\circ f\right)}_{\ell }\left(a\right)\left(h\right)=D{\left(g\circ f\right)}_{\ell }\left(a\right)\left(h^i\right)\left(\text{by}\text{equation}\left(11\right)\right)\\ =\left(D{g}_{\ell }\left(f\left(a\right)\right)\circ Df\left(a\right)\right)\left(h^i\right)\left(\text{by}\text{the}\text{usual}\text{chain}\text{rule}\left(7\right)\right)\\ =D{g}_{\ell }\left(f\left(a\right)\right)\left(Df\left(a\right)\left(h^i\right)\right)\end{array}$(12)

Now the $j$-th component of $Df\left(a\right)\left(h^i\right)$ is $D{f}_j\left(a\right)\left(h^i\right)$, so

$Df\left(a\right)\left(h^i\right)=\underset{j}{{\displaystyle \sum }}{\left(D{f}_j\left(a\right)\left(h^i\right)\right)}^j$ (by the first sentence in Definition 4.1)

Putting this in Equation (12) above, we get

$\begin{array}{l}{D}_i{\left(g\circ f\right)}_{\ell }\left(a\right)\left(h\right)=D{g}_{\ell }\left(f\left(a\right)\right)\left(\underset{j}{{\displaystyle \sum }}{\left(D{f}_j\left(a\right)\left(h^i\right)\right)}^j\right)\\ =\underset{j}{{\displaystyle \sum }}D{g}_{\ell }\left(f\left(a\right)\right){\left(D{f}_j\left(a\right)\left(h^i\right)\right)}^j\left(\text{since the derivative is linear}\right)\\ =\underset{j}{{\displaystyle \sum }}{D}_j{g}_{\ell }\left(f\left(a\right)\right)\left(D{f}_j\left(a\right)\left(h^i\right)\right)\left(\text{by equation}\left(\text{11}\right)\right)\\ =\underset{j}{{\displaystyle \sum }}{D}_j{g}_{\ell }\left(f\left(a\right)\right)\left(D_i{f}_j\left(a\right)\left(h\right)\right)\left(\text{by equation}\left(\text{11}\right)\text{again}\right)\\ =\underset{j}{{\displaystyle \sum }}\left(D_j{g}_{\ell }\left(f\left(a\right)\right)\circ D_i{f}_j\left(a\right)\right)\left(h\right)\\ =\left(\underset{j}{{\displaystyle \sum }}{D}_j{g}_{\ell }\left(f\left(a\right)\right)\circ D_i{f}_j\left(a\right)\right)\left(h\right)\end{array}$

Since $h$ was arbitrary, this shows that $D_i{\left(g\circ f\right)}_{\ell }\left(a\right)={\displaystyle {\sum }_j{D}_j{g}_{\ell }\left(f\left(a\right)\right)\circ D_i{f}_j\left(a\right)}$, as needed.

We are grateful to R. Vijay Krishna for several useful suggestions on this article.

¹As defined on page 42 in ( [2] , Section 5), for example.

²We are not using $X_i$, etc., to denote the manifolds, since the $X_i$’s correspond to local charts on the manifolds.