In mathematical statistics courses, students learn that the quadratic function
There are commonly used, continuous probability distributions of one variable, such as the normal distribution and the exponential distribution. Likewise, there are discrete distributions that are well established, such as the binomial distribution and the Poisson distribution. Here, the much less routine situation in which there is a discrete component and a continuous component in a single probability distribution is addressed. These are called mixed-type probability distributions. Often, the components can be entirely separated from each other, but sometimes it might reduce the effectiveness of the probability model to do so.
An example of a mixed-type distribution is the lifetimes of electronic components. Some components may have zero lifetimes, because they are defective from the onset, giving a discrete probability component at zero ( [
Another example is the elapsed time at a stop sign at an intersection on a street. Some drivers will spend no time at the sign after stopping, because there is no cross traffic, giving the discrete component at elapsed time zero. Other drivers will linger until traffic clears, supplying the continuous component ( [
A third example of a mixed-type distribution is lifetimes in an experiment that is terminated at a predetermined time td. The complete lifetimes of those subjects still living or objects that have not yet failed cannot be known. That group produces a discrete component at time td ( [
On occasion, there might be times during an experiment, or in the course of events, at which interventions can introduce discrete components. For instance, a planned medical procedure, mass vaccination, or military campaign could be such an intervention.
There are choices for the method for proceeding. One choice is to standardize the continuous portion, so that it has a probability density. This is usually expressed with conditional distributions ( [
For a discrete distribution, sums are used to compute probabilities and expected values. For continuous distributions that have a probability density function, Riemann integration is used. All discrete, continuous, and mixed-type distributions, including continuous distributions without a probability density function, which are discussed in Section 3, are covered simultaneously with Riemann-Stieltjes integration ( [
Consider the following example.
Example 1 (mixed-type distribution). In a populous state in the USA, it has been determined by using surveys that there are three distinct types of voters for an upcoming ballot initiative. One type of voter definitely opposes the initiative and is believed to be 7/20 of the voters. These individuals are coded x = 0. Another type is definitely in favor of the initiative and is 2/5 of the voters. Those individuals are coded x = 1. The third type is not polarized on the initiative, and their degree of support x is between zero and one. They are the remaining 1/4 of the voters. For these voters, the proportion of the population is modeled with 1 / 5 + ( 1 / 10 ) x . The random variable X is an individual voter’s degree of positive support for the ballot initiative.
The 1 / 5 + ( 1 / 10 ) x was obtained by first fitting, via a smoothed histogram from the sample’s non-polarized voters, a curve that represents the number of votes as a function of degree of support. The fit was a linear function such that the intercept term is twice the slope coefficient. Then, the values 1/5 and 1/10 were determined from the requirement that the integral of the linear function from x = 0 to x = 1 must equal the remaining fraction, 1/4, of the voters.
This is a mixed-type distribution with a discrete part for X = 0 and X = 1 and a continuous part for X ∈ ( 0 , 1 ) . For every random variable X, the cumulative distribution function (cdf) is defined on ℝ by F ( x ) = Pr ( X ≤ x ) . A cdf F(x) has a jump discontinuity at x = a when Pr ( X = a ) > 0 . Any discrete cdf has at most a countable number of jump discontinuities ( [
The cumulative distribution function for Example 1 is
y = F ( x ) = { 0 x < 0 7 20 + 1 5 x + 1 20 x 2 0 ≤ x < 1 1 1 ≤ x , (1)
which is displayed in
Use the fact that any cdf can be decomposed uniquely into a convex sum of a discrete cdf and a continuous cdf giving the Jordan decomposition
F ( x ) = c 1 F d ( x ) + c 2 F c ( x ) , (2)
where c1 ≥ 0, c2 ≥ 0, c1 + c2 = 1, Fd(x) is a discrete cdf, and Fc(x) is a continuous cdf ( [
F d ( x ) = { 0 x < 0 7 15 0 ≤ x < 1 1 1 ≤ x and F c ( x ) = { 0 x < 0 4 5 x + 1 5 x 2 0 ≤ x < 1 1 1 ≤ x , (3)
which yield the probability mass function (pmf) and probability density function (pdf)
f d ( x ) = { 7 15 x = 0 8 15 x = 1 0 otherwise and f c ( x ) = { 4 5 + 2 5 x 0 < x < 1 0 otherwise , (4)
respectively, and c1 = 3/4 and c2 = 1/4. The weight c1 might be most easily computed from the jumps in the original cdf F(x). Then, Pr(X = 0) + Pr(X = 1) = 7/20 + 2/5 = 3/4, which is the divisor of each of the jumps 7/20 and 2/5 from (1), in order to obtain the discrete parts of (3) and (4) and the multiplier c1. For the continuous part, the normalizing divisor, and multiplier as well, is c2 = 1 – c1 = 1/4 or c 2 = ∫ 0 1 ( 1 / 5 + ( 1 / 10 ) t ) d t = 1 / 4 . Alternatively, obtain Fc from
F c = F − c 1 F d c 2 .
Use the property that the expectation of the function g(X) is
E ( g ( X ) ) = c 1 E d ( g ( X ) ) + c 2 E c ( g ( X ) ) .
where the expectations on the right-hand side are with respect to the similarly superscripted pmf and pdf ( [
μ = E ( X ) = 3 4 ( ( 0 ) 7 15 + ( 1 ) 8 15 ) + 1 4 ∫ 0 t t ( 4 5 + 2 5 t ) d t = 2 5 + 1 4 ( 4 5 1 2 2 + 2 5 1 3 3 ) = 8 15 .
Any cdf F(x) can be decomposed uniquely into a convex sum of a discrete cdf and a continuous cdf, as in (2). Further, the continuous component can be uniquely decomposed into an absolutely continuous component Fac(x) and a singular continuous component Fsc(x), giving the Lebesque decomposition
F ( x ) = c 1 F d ( x ) + c 2 F a c ( x ) + c 3 F s c ( x ) ,
where c1, c2 , c3 ≥ 0, c1 + c2 + c3 = 1 ( [
The Cantor set is created by an infinite process. Beginning with the closed interval [0,1], during the nth step of the process, remove the 2n−1 middle-third open intervals, each of which has length 1/3n. After doing that step, there remain 2n disjoint, closed intervals. The infinite intersection of the closed sets is the Cantor set. The sum of the lengths of the deleted intervals is one, so the Cantor set has Lebesgue measure zero. The Cantor set is the support of the Cantor distribution, whose cdf is called the devil’s staircase. This cdf fails to be differentiable at every point of the Cantor set, but its derivative is zero on the set’s complement. Thus, probabilities cannot be recovered by integrating the derivative of the cdf. The devil’s staircase has no jumps, and so it is continuous at every real number. It is singular continuous, because it assigns probability one to the Cantor set, which has Lebesgue measure, i.e. length, zero. The devil’s staircase has no discrete component and no absolutely continuous component. The Cantor distribution and the devil’s staircase appear in the probability and statistics literature ( [
The first seven omitted intervals, where the devil’s staircase has slope zero, and the accompanying values of the cdf are
F ( x ) = { 1 8 1 27 < x < 2 27 1 4 1 9 < x < 2 9 3 8 7 27 < x < 8 27 1 2 1 3 < x < 2 3 5 8 19 27 < x < 20 27 3 4 7 9 < x < 8 9 7 8 25 27 < x < 26 27
and are graphed in
A median of the random variable X, and therefore of F, is any real number m such that
Pr ( X ≤ m ) ≥ 1 / 2 and Pr ( X ≥ m ) ≥ 1 / 2 ,
or, equivalently,
Pr ( X ≤ m ) ≥ 1 / 2 and Pr ( X < m ) ≤ 1 / 2 . (5)
For Example 1, the median of X is
m = 7 − 2 = 0.646 , (6)
which is obtained from (1) by solving
7 20 + 1 5 m + 1 20 m 2 = 1 2 .
The main purpose of this section is to show that, if m is a median of the univariate random variable X and a ∈ ℝ , then E ( | X − a | ) ≥ E ( | X − m | ) , and this inequality is strict if a is not a median of X. To avoid trivialities, assume that these expectations exist as finite real numbers, which is assured by presupposing that E ( | X | ) ∈ ℝ . Mood, Graybill, and Boes ( [
The expectation E ( | X − x | ) is a convex function of x. Indeed, for c , d ∈ ℝ and t ∈ ( 0 , 1 ) , using the triangle inequality and the linearity of expectation,
E ( | X − ( t c + ( 1 − t ) d ) | ) = E ( | t ( X − c ) + ( 1 − t ) ( X − d ) | ) ≤ E ( | t ( X − c ) | + | ( 1 − t ) ( X − d ) | ) = t E ( | ( X − c ) | ) + ( 1 − t ) E ( | ( X − d ) | ) .
All convex functions are continuous ( [
For c ∈ ℝ , define
F ( c − ) = lim x → c x < c F ( x ) .
Because F ( c − ) = Pr ( X < c ) ,
Pr ( X = c ) = F ( c ) − F ( c − )
( [
F ( c + ) = lim x → c x > c F ( x ) = F ( c ) .
Thus, (5) can be expressed
F ( m – ) ≤ 1 2 and 1 2 ≤ F ( m ) . (7)
Assuming that g is a continuous positive function on the interval (c, d) and F ( c ) < F ( d – ) ,
∫ ( c , d ) g ( x ) d F ( x ) > 0 (8)
( [
∫ ( c , d ) d F ( x ) = F ( d − ) − F ( c ) .
Lemma 1. Let X be a random variable with the cumulative distribution function of F. Suppose that E ( X ) ∈ ℝ . Then, for any a and b ∈ ℝ ,
E ( | X – a | ) – E ( | X – b | ) = { 2 ∫ ( a , b ) ( x − a ) d F ( x ) + ( b − a ) ( 1 − 2 F ( b − ) ) a < b ( 9 ) 2 ∫ ( b , a ) ( a − x ) d F ( x ) + ( a − b ) ( 2 F ( b ) − 1 ) b < a ( 10 )
Proof. Observe that the assumption that E ( X ) ∈ ℝ is equivalent to E(|X|) < ∞. If c ∈ ℝ , then E ( | X − c | ) ≤ E ( | X | ) + | c | < ∞ and
E ( | X – c | ) = ∫ ( – ∞ , ∞ ) | x – c | d F ( x ) .
Consider two cases.
Case 1 (a < b). Expanding gives
E ( | X − a | ) = ∫ ( − ∞ , a ) ( a − x ) d F ( x ) + ∫ ( a , b ) ( x − a ) d F ( x ) + ∫ { b } ( b − a ) d F ( x ) + ∫ ( b , ∞ ) ( x − a ) d F ( x )
and
E ( | X − b | ) = ∫ ( − ∞ , a ) ( b − x ) d F ( x ) + ∫ { a } ( b − a ) d F ( x ) + ∫ ( a , b ) ( b − x ) d F ( x ) + ∫ ( b , ∞ ) ( x − b ) d F ( x ) .
Substituting gives
E ( | X − a | ) − E ( | X − b | ) = ∫ ( − ∞ , a ) ( a − b ) d F ( x ) + ∫ ( a , b ) ( 2 x − a − b ) d F ( x ) + ∫ ( b , ∞ ) ( x − a ) d F ( x ) + ( b − a ) ( F ( b ) − F ( b − ) ) − ( b − a ) ( F ( a ) − F ( a − ) )
= ( a − b ) F ( a − ) + ∫ ( a , b ) ( 2 ( x − a ) + a − b ) d F ( x ) + ( b − a ) ( 1 − F ( b ) ) + ( b − a ) ( F ( b ) − F ( b − ) ) − ( b − a ) ( F ( a ) − F ( a − ) ) = ( a − b ) F ( a − ) + 2 ∫ ( a , b ) ( x − a ) d F ( x ) + ( a − b ) ( F ( b − ) − F ( a ) ) + ( b − a ) ( 1 − F ( b ) ) + ( b − a ) ( F ( b ) − F ( b − ) ) ,
which yields (9).
Case 2 (b < a). Interchanging the roles of a and b in (9) gives
E ( | X − b | ) − E ( | X − a | ) = 2 ∫ ( b , a ) ( x − b ) d F ( x ) + ( a − b ) ( 1 − 2 F ( a − ) ) = 2 ∫ ( b , a ) ( ( x − a ) − ( b − a ) ) d F ( x ) + ( a − b ) ( 1 − 2 F ( a − ) ) = 2 ∫ ( b , a ) ( x − a ) d F ( x ) − 2 ( b − a ) ( F ( a − ) − F ( b ) ) + ( a − b ) ( 1 − 2 F ( a − ) ) = 2 ∫ ( b , a ) ( x − a ) d F ( x ) + ( a − b ) ( 1 − 2 F ( b ) ) .
Multiplying by –1 yields (10).
Theorem 1. Let X be a random variable. Suppose that E ( X ) ∈ ℝ and m ∈ ℝ is a median of X. Then, for any a ∈ ℝ ,
E ( | X − a | ) ≥ E ( | X − m | ) . (11)
The inequality is strict if a is not a median of X.
Proof. From the proof of Lemma 1, E ( | X − a | ) < ∞ . Set b = m in Lemma 1. The integrals in (9) and (10) are nonnegative. Inequality (11) follows, because, using (7), the second terms on the right-hand sides of (9) and (10) are also nonnegative. To show that the inequality (11) is strict when a is not a median of X, consider the two cases in Lemma 1.
Case 1 (a < m). From (7), either F(m–) < 1/2 or F(m–) = 1/2. If F(m–) < 1/2, then (m – a)(1 – 2F(m–)) > 0, so that the right-hand side of (9) is positive. If F(m–) = 1/2, then (m – a)(1 – 2F(m–)) = 0. Also, F(a) ≤ F(m–) = 1/2. Because a is not a median, F(a) ≠ 1/2 and, thus, F(a) < 1/2, F(a) < F(m–), and the integral in (9) is positive from (8).
Case 2 (m < a). From (5), either 1/2 < F(m) or 1/2 = F(m). If 1/2 < F(m), then (a – m)(2F(m) – 1) > 0, so that the right-hand side of (10) is positive. If F(m) = 1/2, then (a – m)(2F(m) – 1) = 0. Also, 1/2 = F(m) ≤ F(a–). Because a is not a median, F(a–) ≠ 1/2 and, thus, F(a–) > 1/2, F(m) < F(a–), and the integral in (10) is positive from (8).
The following examples display the function y = E ( | X − x | ) and the locations of the medians for various distributions. The graphs illustrate that y = E ( | X − x | ) is a convex and continuous function and that medians occur as single points or as all of the values in an interval.
Example 1 revisited (mixed-type distribution). For the voter preference example,
y = E ( | X − x | ) = ( 8 15 − x x < 0 1 30 ( x 3 + 6 x 2 − 9 x + 16 ) 0 ≤ x < 1 x − 8 15 1 ≤ x ,
which is displayed in
Example 2 (absolutely continuous cdf). For the exponential random variable with pdf f(x) = e−x/3/3 for x > 0 and zero otherwise,
y = E ( | X − x | ) = ∫ 0 ∞ | t − x | 1 3 e − t / 3 d t = { 3 − x x ≤ 0 x − 3 + 3 e − x / 3 x > 0 .
The expected value is μ = 3. The minimum is at the unique median m = 3ln2 ≈ 2.08.
Example 3 (discrete cdf and a single median). For the binomial distribution with n = 3 and p = 0.7, the expected value is μ = np = 2.1.
y = E ( | X − x | ) = { − x + 2.1 x < 0 − 0.946 x + 2.1 0 ≤ x < 1 − 0.568 x + 1.722 1 ≤ x < 2 0.314 x − 0.042 2 ≤ x < 3 x − 2.1 3 ≤ x .
The minimum is at the unique median m = 2.
Example 4 (discrete cdf and an interval of medians). For the binomial distribution with n = 5 and p = 0.5, the expected value is μ = np = 2.5.
y = E ( | X − x | ) = { − x + 2.5 x < 0 − 0.9375 x + 2.5 0 ≤ x < 1 − 0.625 x + 2.1875 1 ≤ x < 2 0.9375 2 ≤ x < 3 0.625 x − 0.9375 3 ≤ x < 4 0.9375 x − 2.1875 4 ≤ x < 5 x − 2.5 5 ≤ x .
Note that every number in the interval [2, 3] is a median.
Example 5 (singular continuous distribution). The Cantor distribution has mean 0.5 and median any m ∈ ( 1 / 3 , 2 / 3 ) . Because the derivation of an expression for y = E ( | X − x | ) is more complicated than the previous examples, details are presented.
During the nth step of the process that leads to the Cantor set, remove the 2n−1 middle-third open intervals, each of which has length 1/3n. After doing such a step, there remain 2n disjoint, closed intervals, which are denoted by
I k ( n ) = [ a k ( n ) , b k ( n ) ]
for k = 1 , 2 , ⋯ , 2 n , where
a 1 ( n ) < a 2 ( n ) < ⋯ < a 2 n ( n ) .
The following lemma provides a recursive formula that is used for computing ∫ I k ( n ) x d F ( x ) .
Lemma 2. Define
c 1 ( 0 ) = ∫ 0 1 x d F ( x ) and c k ( n ) = ∫ a k ( n ) b k ( n ) x d F ( x )
for n ≥ 1. Then
c 1 ( 0 ) = 1 2
and
c k ( n ) = { 1 6 c k ( n − 1 ) k = 1 , 2 , ⋯ , 2 n − 1 1 2 n − c 2 n + 1 − k ( n ) k = 2 n − 1 + 1 , 2 n − 1 + 2 , ⋯ , 2 n .
Proof. Because
F ( x 3 ) = F ( x ) 2 and F ( 1 − x ) = 1 − F ( x )
for x ∈ [ 0 , 1 ] ( [
d F ( x 3 ) = d F ( x ) 2 and d F ( 1 − x ) = − d F ( x ) .
Hence,
1 − c 1 ( 0 ) = 1 − ∫ 0 1 x d F ( x ) = 1 − ∫ 0 1 ( 1 − t ) d F ( t ) = ∫ 0 1 t d F ( t ) = c 1 ( 0 ) .
Thus,
c 1 ( 0 ) = 1 2 .
Consider n ≥ 1. For 1 ≤ k ≤ 2n−1,
a k ( n ) = 1 3 a k ( n − 1 ) , b k ( n ) = 1 3 b k ( n − 1 ) ,
and
c k ( n ) = ∫ a k ( n ) b k ( n ) x d F ( x ) = ∫ a k ( n − 1 ) b k ( n − 1 ) t 3 d F ( t 3 ) = ∫ a k ( n − 1 ) b k ( n − 1 ) t 3 d F ( t ) 2 = 1 6 c k ( n − 1 ) .
Let 2n−1 + 1 ≤ k ≤ 2n. From
a k ( n ) = 1 − b 2 n + 1 − k ( n ) , b k ( n ) = 1 − a 2 n + 1 − k ( n ) , and F ( b 2 n + 1 − k ( n ) ) − F ( a 2 n + 1 − k ( n ) ) = 1 2 n ,
it follows that
c k ( n ) = ∫ a k ( n ) b k ( n ) x d F ( x ) = ∫ a 2 n + 1 − k ( n ) b 2 n + 1 − k ( n ) ( 1 − t ) d F ( t ) = 1 2 n − c 2 n + 1 − k ( n ) .
The numerical values of the definite integrals c k ( n ) for n = 1, 2, and 3 are in
Theorem 2. Let J 1 ( n ) , J 2 ( n ) , ⋯ , J 2 n − 1 ( n ) be the disjoint open intervals that are removed during the nth step of the construction of the Cantor set, where the midpoints of these intervals are strictly increasing. For x ∈ J k ( n ) ,
y = y ( x ) = E ( | X − x | ) = 2 k − 1 − 2 n − 1 2 n − 1 x − ∑ j = 1 2 k − 1 c j ( n ) + ∑ j = 2 k 2 n c j ( n ) . (12)
Proof. For x ∈ J k ( n ) , it follows from Lemma 2 that
y = y ( x ) = E ( | X − x | ) = ∫ − ∞ ∞ | t − x | d F ( t ) = ∑ j = 1 2 n ∫ I j ( n ) | t − x | d F ( t ) = ∑ j = 1 2 k − 1 ∫ I j ( n ) ( t − x ) d F ( t ) + ∑ j = 2 k 2 n ∫ I j ( n ) ( t − x ) d F ( t ) = ∑ j = 1 2 k − 1 ( x 2 n − c j ( n ) ) + ∑ j = 2 k 2 n ( c j ( n ) − x 2 n ) = 2 k − 1 − 2 n − 1 2 n − 1 x − ∑ j = 1 2 k − 1 c j ( n ) + ∑ j = 2 k 2 n c j ( n ) .
Since the complement of the Cantor set in [0, 1] is dense in that interval, and since the value of a continuous function at any number in [0, 1] is determined by its values on a dense subset ( [
For k = 1 , 2 , ⋯ , 2 n − 1 , the graph of y on J k ( n ) is a line segment with slope
2 k − 1 − 2 n − 1 2 n − 1 .
Equation (13) and
| n | k | |
|---|---|---|
| 1 | 1 | c 1 ( 1 ) = 1 6 c 1 ( 0 ) = 1 6 1 2 = 1 12 |
| 2 | c 2 ( 1 ) = 1 2 − c 1 ( 1 ) = 1 2 − 1 12 = 5 12 | |
| 2 | 1 | c 1 ( 2 ) = 1 6 c 1 ( 1 ) = 1 6 1 12 = 1 72 |
| 2 | c 2 ( 2 ) = 1 6 c 2 ( 1 ) = 1 6 5 12 = 5 72 | |
| 3 | c 3 ( 2 ) = 1 2 2 − c 2 ( 2 ) = 1 4 − 5 72 = 13 72 | |
| 4 | c 4 ( 2 ) = 1 2 2 − c 1 ( 2 ) = 1 4 − 1 72 = 17 72 | |
| 3 | 1 | c 1 ( 3 ) = 1 6 c 1 ( 2 ) = 1 6 1 72 = 1 432 |
| 2 | c 2 ( 3 ) = 1 6 c 2 ( 2 ) = 1 6 5 72 = 5 432 | |
| 3 | c 3 ( 3 ) = 1 6 c 3 ( 2 ) = 1 6 13 72 = 13 432 | |
| 4 | c 4 ( 3 ) = 1 6 c 4 ( 2 ) = 1 6 17 12 = 17 432 | |
| 5 | c 5 ( 3 ) = 1 2 3 − c 4 ( 3 ) = 1 8 − 17 432 = 37 432 | |
| 6 | c 6 ( 3 ) = 1 2 3 − c 3 ( 3 ) = 1 8 − 13 432 = 41 432 | |
| 7 | c 7 ( 3 ) = 1 2 3 − c 2 ( 3 ) = 1 8 − 5 432 = 49 432 | |
| 8 | c 8 ( 3 ) = 1 2 3 − c 1 ( 3 ) = 1 8 − 1 432 = 53 432 |
y = E ( | X − x | ) = { 1 / 2 − x x < 0 107 / 216 − ( 3 / 4 ) x 1 / 27 < x < 2 / 27 17 / 36 − ( 1 / 2 ) x 1 / 9 < x < 2 / 9 89 / 216 − ( 1 / 4 ) x 7 / 27 < x < 8 / 27 1 / 3 1 / 3 < x < 2 / 3 ( 1 / 4 ) x + 35 / 216 19 / 27 < x < 20 / 27 ( 1 / 2 ) x − 1 / 36 7 / 9 < x < 8 / 9 ( 3 / 4 ) x − 55 / 216 25 / 27 < x < 26 / 27 x − 1 / 2 1 < x (13)
For a sample calculation, using (12) and
y = 2 ( 1 ) − 1 − 2 2 − 1 2 2 − 1 x − 1 72 + 5 72 + 13 72 + 17 72 = 17 36 − 1 2 x .
The necessary and sufficient condition that x minimizes y ( x ) = E ( | X − x | ) for medians of the distribution of X has been established under completely general conditions on the distribution of X and illustrated for various pure and mixed cumulative distribution functions, including the devil’s staircase of the Cantor distribution. We have evaluated the function y ( x ) = E ( | X − x | ) when X has the Cantor distribution, and we have also demonstrated the way in which the nature of this function depends heavily on the distribution of X in other cases. By way of contrast, the minimum value of the quadratic function E ( ( X − x ) 2 ) is Var(X), and the minimizing value of x is E(X). Its graph depends only on this mean and variance, and any two distributions for X yield graphs that are translates of each other.
The authors want to thank the anonymous referees for many insightful comments.
The authors declare no conflicts of interest regarding the publication of this paper.
Marengo, J.E. and Farnsworth, D.L. (2022) Probability Models with Discrete and Continuous Parts. Open Journal of Statistics, 12, 82-97. https://doi.org/10.4236/ojs.2022.121006