<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article  PUBLIC "-//NLM//DTD Journal Publishing DTD v3.0 20080202//EN" "http://dtd.nlm.nih.gov/publishing/3.0/journalpublishing3.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" dtd-version="3.0" xml:lang="en" article-type="research article"><front><journal-meta><journal-id journal-id-type="publisher-id">JSEA</journal-id><journal-title-group><journal-title>Journal of Software Engineering and Applications</journal-title></journal-title-group><issn pub-type="epub">1945-3116</issn><publisher><publisher-name>Scientific Research Publishing</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="doi">10.4236/jsea.2013.64A001</article-id><article-id pub-id-type="publisher-id">JSEA-30069</article-id><article-categories><subj-group subj-group-type="heading"><subject>Articles</subject></subj-group><subj-group subj-group-type="Discipline-v2"><subject>Computer Science&amp;Communications</subject></subj-group></article-categories><title-group><article-title>
 
 
  A Bootstrapping Approach for Software Reliability Measurement Based on a Discretized NHPP Model
 
</article-title></title-group><contrib-group><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>hinji</surname><given-names>Inoue</given-names></name><xref ref-type="aff" rid="aff1"><sup>1</sup></xref><xref ref-type="corresp" rid="cor1"><sup>*</sup></xref></contrib><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Shigeru</surname><given-names>Yamada</given-names></name><xref ref-type="aff" rid="aff1"><sup>1</sup></xref><xref ref-type="corresp" rid="cor1"><sup>*</sup></xref></contrib></contrib-group><aff id="aff1"><addr-line>Department of Social Management Engineering, Graduate School of Engineering, Tottori University, Tottori, Japan.</addr-line></aff><author-notes><corresp id="cor1">* E-mail:<email>ino@sse.tottori-u.ac.jp(HI)</email>;<email>yamada@sse.tottori-u.ac.jp(SY)</email>;</corresp></author-notes><pub-date pub-type="epub"><day>26</day><month>04</month><year>2013</year></pub-date><volume>06</volume><issue>04</issue><fpage>1</fpage><lpage>7</lpage><history><date date-type="received"><day>December</day>	<month>14th,</month>	<year>2012</year></date><date date-type="rev-recd"><day>January</day>	<month>17th,</month>	<year>2013</year>	</date><date date-type="accepted"><day>January</day>	<month>26th,</month>	<year>2013</year></date></history><permissions><copyright-statement>&#169; Copyright  2014 by authors and Scientific Research Publishing Inc. </copyright-statement><copyright-year>2014</copyright-year><license><license-p>This work is licensed under the Creative Commons Attribution International License (CC BY). http://creativecommons.org/licenses/by/4.0/</license-p></license></permissions><abstract><p>
 
 
     
   Discrete software reliability measurement has a proper characteristic for describing a software reliability growth process which depends on a unit of the software fault-detection period, such as the number of test runs, the number of executed test cases. This paper discusses discrete software reliability measurement based on a discretized nonhomogeneous Poisson process (NHPP) model. Especially, we use a bootstrapping method in our discrete software reliability measurement for discussing the statistical inference on parameters and software reliability assessment measures of our model. Finally we show numerical examples of interval estimations based on our bootstrapping method for the several software reliability assessment measures by using actual data.  
      
   
    
     
    
 
</p></abstract><kwd-group><kwd>Software Reliability Measurement; Discretized NHPP Model; Nonparametric Bootstrapping Method;  Regression Analysis; Bootstrap Confidence Intervals</kwd></kwd-group></article-meta></front><body><sec id="s1"><title>1. Introduction</title><p>It is very important to measure reliability of a software product with accuracy in the final stage of software development process for shipping a highly reliable software product. A software reliability growth model [1-4] is known as one of the useful mathematical tools for quantitative measurement or assessment of software reliability. Generally in an actual testing-phase, we observe a software reliability growth process, in which software faults are detected and removed and the number of faults remaining in the software system is decreasing along with the test-execution time. The software reliability growth model describes the software reliability growth process, and measures the software reliability quantitatively by using software reliability assessment measures, which are derived by the software reliability growth model. A huge number of software reliability growth models were proposed so far for accurate software reliability assessment. Especially, there are discretized nonhomogeneous Poisson process (discretized NHPP) models, which have good fitting and prediction performance in software reliability assessment [5,6] because the discretized NHPP models have consistency with fault counting data, which are obtained by collecting information on the frequency of the software failure-occurrence or the number of detected faults during each constant testing-period. Estimating parameters in the discretized NHPP model from actual data is conducted by using the regression analysis based on a regression equation derived from a difference equation of the discretized NHPP model. After the parameter estimation, software reliability assessment is performed based on the software reliability assessment measures derived from the discretized NHPP model. This approach is based on the point estimation, which is better to use when we have enough number of data.</p><p>In recent years, it is very difficult to obtain enough number of data for the point estimation method due to the quick delivery of software development. In such case, it is better to conduct interval estimation for considering the uncertainty of the estimators being related to the model parameters and software reliability assessment measures. We often use asymptotic approximation approaches [<xref ref-type="bibr" rid="scirp.30069-ref7">7</xref>] for the interval estimation. However, we have some difficulty in mathematical manipulation for conducting the interval estimation even if we use the asymptotic approximation approach. For overcoming the problem above, the bootstrap method [<xref ref-type="bibr" rid="scirp.30069-ref8">8</xref>] was proposed. The bootstrapping method is known as one of the useful Monte Carlo methods for obtaining probability distributions for estimators by a resampling method. Recently, the bootstrapping method is applied not only to software reliability analysis [9-11] but also optimal checkpoint replacement for hardware system [<xref ref-type="bibr" rid="scirp.30069-ref12">12</xref>].</p><p>In this paper, we discuss an interval estimation method for parameters and software reliability assessment measures of a discretized exponential software reliability growth model, which is one of the discretized NHPP models and has the simplest model-structure, by the bootstrapping method. And, we discuss several kinds of bootstrap confidence intervals for the interval estimations. Finally, we show numerical examples for our bootstrapping method for software reliability assessment based on the discretized exponential software reliability model and the bootstrap confidence intervals by using actual data.</p></sec><sec id="s2"><title>2. Discretized Exponential NHPP Model</title><sec id="s2_1"><title>2.1. The Model</title><p>We briefly discuss the aspect of the discretized NHPP model by showing a discretized exponential software reliability growth model [5,6], which has the simplest mathematical structure. Now we define a discrete counting process <img src="1-9301570\f2d420ae-9308-4fba-befb-c408f6557067.jpg" /> representing the cumulative number of faults detected up to n-th testing-period. And we can say that the discrete counting process <img src="1-9301570\eb676c6e-6a8f-4007-a1a4-2ff1f93d5739.jpg" /> follows a discrete-time NHPP [<xref ref-type="bibr" rid="scirp.30069-ref13">13</xref>] if the process has the following property:</p><disp-formula id="scirp.30069-formula14444"><label>(1)</label><graphic position="anchor" xlink:href="1-9301570\021779da-5a9e-45d1-847d-e93d1b6551a1.jpg"  xlink:type="simple"/></disp-formula><p>which is derived based on a continuous-time NHPP. In Equation (1), <img src="1-9301570\e2a3e841-7e55-4af5-a895-4b2d3c34f3ef.jpg" />means the proba of event<img src="1-9301570\d30044cc-2baf-4c1f-b3f2-829cad04186f.jpg" />. <img src="1-9301570\54122ad4-ed20-4b3f-86d4-c5a351e3d47c.jpg" />is a mean value function of the discrete-time NHPP. The mean value function, <img src="1-9301570\a7fb6f25-fff6-4ab4-82af-1f73c9f7c2ae.jpg" />, represents the expected cumulative number of faults detected up to nth testing-period.</p><p>The discretized exponential software reliability growth model is a discrete analog of the original (continuous-time) exponential software reliability growth model. Let <img src="1-9301570\9b50f377-8c2e-4f88-88cf-3115ade8bfaf.jpg" /> denote the mean value function following the discretized exponential software reliability growth model. The discretized exponential software reliability growth model is given as</p><disp-formula id="scirp.30069-formula14445"><label>(2)</label><graphic position="anchor" xlink:href="1-9301570\b3785600-8aff-4289-9ff0-d1b1c74ed08e.jpg"  xlink:type="simple"/></disp-formula><p>from the basic assumptions of the original exponential software reliability growth model. In Equation (2), <img src="1-9301570\06cbf182-eaa3-4a73-8f74-7b6156cbedab.jpg" />represents the constant time-interval, <img src="1-9301570\1d5eb16a-5660-40d6-801d-521d5b0f5021.jpg" />the expected total number of potential faults to be detected in an infinitely long duration or the expected initial fault content, and <img src="1-9301570\d74ac7bb-0921-4c26-8e69-5260c8bac851.jpg" /> the fault detection rate per fault. Regarding the discretization method, we use the Hirota’s bilinearization methods [<xref ref-type="bibr" rid="scirp.30069-ref14">14</xref>] for conserving the property of the continuous-time NHPP model. Solving the integrable difference equation in Equation (2), we can obtain an exact solution <img src="1-9301570\41651c81-2e85-448b-8a0a-8df1ba931ca9.jpg" /> as</p><disp-formula id="scirp.30069-formula14446"><label>(3)</label><graphic position="anchor" xlink:href="1-9301570\3b23e0f5-dfec-4df4-9532-9094a34ee50c.jpg"  xlink:type="simple"/></disp-formula><p>As<img src="1-9301570\33e8c636-1f0d-4fbd-808e-357321d8ed43.jpg" />, Equation (3) converges to an exact solution of the original continuous-time exponential software reliability growth model, which is derived by the differential equation.</p><p>The discretized exponential software reliability growth model in Equation (3) has two parameters, <img src="1-9301570\77b33c36-329e-495f-94e6-ef24eebb653a.jpg" />and<img src="1-9301570\cf668313-6e41-446e-9f5a-152db582c343.jpg" />, which have to be estimated by using actual data. The parameter estimations of <img src="1-9301570\253ea1bd-77e3-492f-a96a-ca2a30cff1c3.jpg" /> and<img src="1-9301570\b637484a-1c29-4221-b6c3-b89b42711204.jpg" />, <img src="1-9301570\ebca6b84-b0d5-43ad-94e5-3c5de51d75b7.jpg" />and<img src="1-9301570\21f8fe1a-3c43-428c-a0a9-74d0f11931a9.jpg" />, can be obtained by the following procedure using the method of least-squares. First of all, if we observed fault counting data<img src="1-9301570\c0d08114-49aa-4825-bced-355480e7001b.jpg" /><img src="1-9301570\e2b8dfd0-28b1-44d8-9f76-bdb5014828da.jpg" />, where <img src="1-9301570\9df60a14-6d6c-4683-be8a-e69df1106526.jpg" /> represents the cumulative number of faults detected up to nth testing-period, we derive the following regression equation from Equation (2):</p><disp-formula id="scirp.30069-formula14447"><label>(4)</label><graphic position="anchor" xlink:href="1-9301570\fef604c0-0252-4be0-babc-0300aaf68974.jpg"  xlink:type="simple"/></disp-formula><p>where</p><disp-formula id="scirp.30069-formula14448"><label>(5)</label><graphic position="anchor" xlink:href="1-9301570\e6a55bec-a4cf-47ae-b9e0-cd47ad0e95b3.jpg"  xlink:type="simple"/></disp-formula><p>Based on the regression analysis, we can estimate <img src="1-9301570\e30e1c23-2a63-40ac-b059-7d63d25ceccb.jpg" /> and<img src="1-9301570\907ef223-0d53-4f6e-9dff-14a295ddf2ba.jpg" />, which are the estimations of <img src="1-9301570\96100206-9abf-42ac-a526-be5a0464f9d0.jpg" /> and <img src="1-9301570\e8247d79-c6b8-42c5-b213-39e3d992e4cc.jpg" /> in Equation (4). Then, the parameter estimations, <img src="1-9301570\29e21d5c-be8c-447f-a535-30c3779b0a29.jpg" />and<img src="1-9301570\a7f9ff11-febc-476d-a96f-8541d7a26543.jpg" />, can be obtained as</p><disp-formula id="scirp.30069-formula14449"><label>(6)</label><graphic position="anchor" xlink:href="1-9301570\c5905089-70a3-46b4-b9ef-eef2abc0f70b.jpg"  xlink:type="simple"/></disp-formula><p><img src="1-9301570\e7787bab-c4e8-4b39-ba0e-35dd15226504.jpg" />in Equation (4) is independent of <img src="1-9301570\6d667ac0-feb5-40f0-9057-3920ae5ae15c.jpg" /> because <img src="1-9301570\74d97c7c-affa-423b-9686-aa26e882c488.jpg" /> is not used in calculating <img src="1-9301570\eb9ae3a9-c259-43dc-872d-df355fe5de0b.jpg" /> as showing Equation (5). Hence, we can obtain the same parameter estimates <img src="1-9301570\b1d1dac9-a56c-4380-a095-ab1546f5097e.jpg" /> and<img src="1-9301570\c46734ee-f92f-418b-a20c-c43c1a0320fd.jpg" />, respectively, when we choose any value of <img src="1-9301570\3b7cb964-c75c-459c-8e5f-f39c4a3598cc.jpg" /> [5,6,15,16].</p></sec><sec id="s2_2"><title>2.2. Software Reliability Assessment Measures</title><p>Software reliability assessment measures are useful in quantitative software reliability assessment. This paper discusses the expected number of remaining faults and the software reliability function, which are well-known software reliability assessment measures. The expected number of remaining faults, <img src="1-9301570\3c67fce5-98ea-4958-ada0-53c8b2ac71fb.jpg" />, represents the expected number of undetected faults in the software system at arbitrary testing-period. Then, we have</p><disp-formula id="scirp.30069-formula14450"><label>(7)</label><graphic position="anchor" xlink:href="1-9301570\4ff2234b-8023-4b28-b840-8191a83b5cfa.jpg"  xlink:type="simple"/></disp-formula><p>if we assume that <img src="1-9301570\2fd70913-8afe-4bc3-8ab4-10282aa14cac.jpg" /> follows a discrete-time NHPP with mean <img src="1-9301570\2686def7-88e8-4c6c-8382-6c86fd11e904.jpg" /> in Equation (3). The software reliability function, <img src="1-9301570\5b3a4a04-5a40-4b7b-a5ab-21b83db05f66.jpg" />, is defined as the probability that a software failure does not occur in the time-interval <img src="1-9301570\9d66a574-9a58-49f0-bf56-de0c07fffb32.jpg" /><img src="1-9301570\2671548e-6abb-4979-8154-e20dc1249633.jpg" /> given that the testing has been going Up to the nth testing-priod. Then, we have</p><disp-formula id="scirp.30069-formula14451"><label>(8)</label><graphic position="anchor" xlink:href="1-9301570\f115c61b-bd36-4045-8449-b2d5969edb39.jpg"  xlink:type="simple"/></disp-formula></sec></sec><sec id="s3"><title>3. Software Reliability Assessment Based on Bootstrapping Method</title><p>Ordinarily, the parameters of the discretized NHPP models are estimated by using the regression analysis based on the regression equation derived from the difference equation of the discretized NHPP models. However, it is difficult to discuss the statistical inference on software reliability assessment in the existing estimation approach because it is very difficult or complex to identify the probability distribution function for the estimators of parameter analytically. For overcoming a problems above, Kimura and Fujiwara [9,10] applied non-parametric bootstrap software reliability assessment methods for an incomplete gamma function-based software reliability growth model. Kaneishi and Dohi [<xref ref-type="bibr" rid="scirp.30069-ref11">11</xref>] discussed a parametric bootstrap method for software reliability assessment based on continuous-time NHPP models. In this paper, we apply a non-parametric bootstrap method to the discretized NHPP model for estimating model parameters and for obtaining information for the statistical inference on the parameters and software reliability assessment measures. Especially in this paper, we discuss five types of bootstrap confidence intervals for interval estimation of the model parameters and software reliability assessment measures.</p><sec id="s3_1"><title>3.1. Our Bootstrapping Method</title><p>As an example for discussing our bootstrapping method based on the discretized NHPP model, we apply the discretized exponential software reliability growth model. Our bootstrap method for software reliability assessment follows the following procedure:</p><p>Step 1: Estimate <img src="1-9301570\68985b93-378d-4b46-993e-e7c91a4a27d8.jpg" /> and <img src="1-9301570\80d68f3c-d465-4e19-a06b-1d448d1c5dcf.jpg" /> in Equation (4) based on the linear regression scheme by using fault counting data. We indicate <img src="1-9301570\42e86e78-0751-4f7e-8384-d5869988d9fa.jpg" /> and <img src="1-9301570\b3ba00e6-2a1d-4f01-8026-3c48ebaad05a.jpg" /> as <img src="1-9301570\6c603874-9020-4bf6-bb66-2fed61f749c0.jpg" /> and<img src="1-9301570\45e4a54c-1896-4e73-a43c-5589792a392a.jpg" />, respectively.</p><p>Step 2: Calculate the residual errors, <img src="1-9301570\f57ddd10-824c-48ab-949e-9efd663a9ed5.jpg" />at each observation point by</p><p><img src="1-9301570\2f85b295-e505-4ee2-924a-f9f4879371c9.jpg" /></p><p>Step 3: Construct an empirical distribution function <img src="1-9301570\54dd7a44-7e60-46e5-98bb-09ef7539620d.jpg" /> by assuming the residual errors <img src="1-9301570\3a745df1-3851-4e72-9baf-15730052f75e.jpg" /> follows the independent and identically probability distribution and putting mass <img src="1-9301570\6de2d671-8460-42c4-bafc-4b1299283e1e.jpg" /> at each ordered point</p><p><img src="1-9301570\73fa6fd4-09a2-4024-9f4a-df70437d2c8b.jpg" />.</p><p>Step 4: Set the total number of iteration <img src="1-9301570\c26a0540-6e6e-4ae1-996b-721eeefc7f4a.jpg" /> and let <img src="1-9301570\57fa7e92-2ee0-4e2e-a8ed-4d4148608ed9.jpg" /><img src="1-9301570\a614696f-dcca-4919-b949-d7ad3c54b778.jpg" /> be the iteration count.</p><p>Step 5: Generate a bootstrap sample for the residual errors,</p><p><img src="1-9301570\5bb604b5-3018-4747-8bbd-eff7d00df427.jpg" /></p><p>by sampling with replacement from<img src="1-9301570\c39e7cb1-ae76-4956-b419-340a5c289e7f.jpg" />.</p><p>Step 6: Generate a bootstrap sample for</p><p><img src="1-9301570\33cb89a8-5d0c-4c86-8d12-4620f40a4fef.jpg" /></p><p>by</p><p><img src="1-9301570\7e9fc385-145d-42e4-9412-4d84cf36315c.jpg" /></p><p>Step 7: Estimate <img src="1-9301570\2330f1d5-1d7e-4337-847e-64e5f9f7af70.jpg" /> and <img src="1-9301570\3e04792a-69a4-4eff-9f91-5701f3e80ccb.jpg" /> from the bootstrap sample<img src="1-9301570\cd5d12d7-fc87-4c4c-82fd-37b3c5e9bcbd.jpg" />.</p><p>Step 8: Calculate parameters of the discretized exponential software reliability growth model by the following equation:</p><p><img src="1-9301570\93e6e99f-cc93-48de-9f21-6a8978c4b032.jpg" /></p><p>Step 9: Calculate software reliability assessment measures.</p><p>Step 10: Let <img src="1-9301570\f25a6014-0693-46c2-851b-43a86130ac23.jpg" /> and go back to Step 5 if. b &lt; B Step 11: We have <img src="1-9301570\1d5b8fbf-034e-4ff7-8644-55c34349641f.jpg" /> samples for<img src="1-9301570\823c7f69-00b7-489a-81a4-958b740db157.jpg" />, <img src="1-9301570\f0cd6569-c905-41fd-a795-bb7f458b3612.jpg" />and a software reliability assessment measures.</p><p>We can calculate the mean and the standard deviation for the model parameters and software reliability assessment measures by the Monte Carlo approximation, respectively.</p></sec><sec id="s3_2"><title>3.2. Bootstrap Confidence Intervals</title><p>We discuss the following three typical bootstrap confidence intervals [<xref ref-type="bibr" rid="scirp.30069-ref17">17</xref>]: basic, standard normal, and percentile bootstrap confidence intervals. Further we discuss bootstrap-t and BCa methods [17,18] for deriving bootstrap confidence intervals considering with the asymmetric property and the bias and the skewness of the estimator of the parameter. Let <img src="1-9301570\323fe140-74ca-41a9-ba4a-6a7621e729c8.jpg" /> be parameter of interest.</p><p>The basic bootstrap confidence interval is derived by using the quantile of the distribution of<img src="1-9301570\591c1b7b-d543-47fc-bf0b-fed479f68fab.jpg" />, where <img src="1-9301570\b3d8dfc9-c9e3-4996-82d0-071616f8a8d0.jpg" /> is the bootstrap statistic. We can approximate the <img src="1-9301570\ee91f550-a294-4d24-8147-2467bb293531.jpg" /> and <img src="1-9301570\5fbdf27d-2caa-43db-947e-f34f7eea937f.jpg" /> quantile denoting v<sub>a</sub> and v<sub>a</sub><sub>-1</sub>, respectivelyof the distribution of <img src="1-9301570\a0bbd6d0-fdff-44fa-9723-40557b6f7beb.jpg" /> by <img src="1-9301570\017f71e7-1253-4b6c-82d8-20465640e26c.jpg" /> and<img src="1-9301570\ef835ae7-77d1-40fa-a7e5-1ab142ed67aa.jpg" />. Then,</p><p><img src="1-9301570\d48a807f-d00d-405e-b4b9-421325e5e980.jpg" /></p><p>Thus, the <img src="1-9301570\d9675101-c4df-4085-98b0-637256d7c65a.jpg" /> basic bootstrap confidence interval is given by</p><disp-formula id="scirp.30069-formula14452"><label>(9)</label><graphic position="anchor" xlink:href="1-9301570\44bebef2-8fad-4e3c-a1e0-b8034d41e51f.jpg"  xlink:type="simple"/></disp-formula><p>The standard normal bootstrap confidence interval is derived by assuming that the distribution of <img src="1-9301570\a966ab39-ab73-459d-b62a-e955fbcd0e91.jpg" /> can be approximated by the distribution of <img src="1-9301570\2230752d-43b7-4044-9925-707014834afa.jpg" /> and</p><p><img src="1-9301570\52933c5d-cb34-407a-90fc-839061270621.jpg" />. That is,</p><p><img src="1-9301570\accf9370-a320-48ed-9ba8-f566cc5943c9.jpg" />.</p><p>Thus, we have the <img src="1-9301570\3f56bea7-aa99-4ee4-abfb-79af96e54125.jpg" /> standard normal bootstrap confidence interval as</p><disp-formula id="scirp.30069-formula14453"><label>(10)</label><graphic position="anchor" xlink:href="1-9301570\5c0ed348-bcb3-4b36-9b63-eddcdaa8be5c.jpg"  xlink:type="simple"/></disp-formula><p>where <img src="1-9301570\e28811e2-09b8-478c-899f-153136f1f43f.jpg" /> is<img src="1-9301570\4cf2fda8-dd0e-423e-944f-091af9d37c08.jpg" />, which is the <img src="1-9301570\b486e4c0-8ada-4f96-8a3e-6e7a53622eec.jpg" /> quantile of the standard normal distribution. For example,</p><p><img src="1-9301570\bcdd08a3-b0dd-41e6-91fb-33e5f422c395.jpg" />.</p><p>The percentile bootstrap confidence interval is calculated from the empirical cumulative probability distribution function, which consists of the bootstrap iteration values:<img src="1-9301570\3d912089-8547-4680-b020-0fe33f7c9ffe.jpg" />. Then, the <img src="1-9301570\c6525523-4ce8-448f-8fc4-0e444747d6b2.jpg" /> percentile bootstrap confidence interval is calculated by</p><disp-formula id="scirp.30069-formula14454"><label>(11)</label><graphic position="anchor" xlink:href="1-9301570\fa718ef6-0cb9-4a99-8adc-85757dcc6d99.jpg"  xlink:type="simple"/></disp-formula><p>where <img src="1-9301570\55836770-7c86-4690-acfd-bc2e961a39b6.jpg" /> represents the <img src="1-9301570\9c0973f8-b80a-428b-8ac6-63b4ffb9e50a.jpg" /> quantile of the empirical cumulative probability distribution function.</p><p>Further, we discuss a bootstrap-t method, which enables us to take into consideration the variance of <img src="1-9301570\8622d699-706a-453e-afd6-5f1399ecc738.jpg" /> by deriving<img src="1-9301570\b80aea21-96cd-4a10-96ea-ad8ee6b7cbf9.jpg" />, where <img src="1-9301570\585083d2-a6d5-4026-982e-25e3fcdb75c1.jpg" /> is the variance of<img src="1-9301570\23e1d83a-7463-403d-b1f4-cef06af01631.jpg" />. Letting <img src="1-9301570\1b08e71d-5861-4fcc-b9de-3d638e84217d.jpg" /> and <img src="1-9301570\8773b3c9-4f3b-4555-b44b-239b6674b982.jpg" /> are the <img src="1-9301570\fe90525a-9a00-4e46-9842-52207b387d82.jpg" /> and <img src="1-9301570\8649c33e-3d92-4d93-b146-a40443475546.jpg" /> quantile of<img src="1-9301570\43476d97-fa87-4200-b7d4-b70718eeffc8.jpg" />, we have</p><p><img src="1-9301570\03448b4d-635f-431c-8224-f03e945ff45f.jpg" /></p><p>In above equation, we substitute <img src="1-9301570\fa7fe926-83a8-4309-8172-78685b752357.jpg" /> and<img src="1-9301570\2b47f5f8-5649-458b-9b91-d1ec28598fee.jpg" />, which are <img src="1-9301570\4096f1a1-2d84-4c76-8bcb-767127746eb5.jpg" /> and <img src="1-9301570\72a83df5-111a-4290-9b20-618f881d743d.jpg" /> quantile of<img src="1-9301570\dacfcaa3-3a68-4619-aced-748f1b87b3ff.jpg" />, into <img src="1-9301570\44382f89-96b6-4efb-b111-3b4c60fd6132.jpg" /> and</p><p><img src="1-9301570\38c758c3-21d2-48b6-b120-1e8bd3d7937e.jpg" />, which are the <img src="1-9301570\15daba15-3a16-4161-a84c-46a1fef5861f.jpg" /> and <img src="1-9301570\f8a43769-d887-4a73-be1b-df629feb302d.jpg" /> quantile of</p><p><img src="1-9301570\78d2bc94-a9e6-4e45-9f11-4eaaba238f40.jpg" />. Then, the <img src="1-9301570\76489090-52c1-4844-a417-92b90e1fcff2.jpg" /> bootstrap-t confidence interval is derived as</p><disp-formula id="scirp.30069-formula14455"><label>(12)</label><graphic position="anchor" xlink:href="1-9301570\9810aa76-5ba3-4af0-8856-6f1b4882ded4.jpg"  xlink:type="simple"/></disp-formula><p>In Equation (12), <img src="1-9301570\590c2157-8d60-462d-bf10-dca0efd4a94a.jpg" />is the standard deviation of <img src="1-9301570\6e3d3fcd-908f-4d28-8bf4-2920d3aaffd2.jpg" /> and <img src="1-9301570\5ca8ec59-93b0-4b49-8442-2bc1eeb7ad9b.jpg" /> is estimated by the bootstrap-t statistics<img src="1-9301570\6b1731e8-62bf-4521-b2ad-e9793e2ebc34.jpg" />.</p><p>And we also discuss a BCa method for getting better bootstrap confidence interval with the asymmetric property, the bias, and the skewness of the probability distribution of the estimator. The BCa confidence interval can be given as</p><disp-formula id="scirp.30069-formula14456"><label>(13)</label><graphic position="anchor" xlink:href="1-9301570\36c7f1b7-8a9a-42ba-9778-b27b8faaa9d2.jpg"  xlink:type="simple"/></disp-formula><p>where</p><p><img src="1-9301570\702e1842-97cc-40e8-b6f1-7423c703eec5.jpg" />.</p><p>In above equation, <img src="1-9301570\653812df-2ae4-4efe-9297-74985a959739.jpg" />, in which <img src="1-9301570\e38211c4-c6ad-456b-bf86-bc3eddd2a073.jpg" /> is the bootstrap distribution for the estimator. And <img src="1-9301570\9abf0cc6-ea3a-4cc4-b1aa-4576bd595644.jpg" /> is the acceleration constant derived as</p><p><img src="1-9301570\154774b9-9ff7-493e-813c-cb8cc9ee0210.jpg" />where <img src="1-9301570\ac178920-6ef2-410e-9381-c60315351dbf.jpg" /> is a jackknife iteration value, which is estimated by using the data, removed ith data and<img src="1-9301570\09186a97-981e-4896-b8b5-ea8cd7a115f8.jpg" />.</p></sec></sec><sec id="s4"><title>4. Numerial Examples</title><p>We show numerical examples for our bootstrap software reliability assessment method based on the discretized exponential software reliability growth model.</p><p>We apply fault counting data: <img src="1-9301570\d2714f0d-0938-4d24-916e-7e7e0bd141ea.jpg" /></p><p><img src="1-9301570\e00d7983-e679-47b5-bae3-26872ba2348f.jpg" />[<xref ref-type="bibr" rid="scirp.30069-ref1">1</xref>] and we set the total number of iteration<img src="1-9301570\9e8f2aed-1765-4f7e-9a92-085a683ae6e0.jpg" />.</p><p>We first obtain</p><p><img src="1-9301570\dad1bd32-56bc-446a-ad0d-5097dc069703.jpg" />and <img src="1-9301570\cf288da5-5330-448f-b7b8-3c6cc3497b09.jpg" /></p><p>by the linear regression scheme from the actual data. Following to our bootstrapping method, we have 2000 bootstrap samples<img src="1-9301570\bebc9da3-5840-4a64-9522-812bdccc9e49.jpg" />. Then, we obtain bootstrap samples for <img src="1-9301570\d4ba77e4-10c9-4aa0-8925-317ba82c0154.jpg" /> and<img src="1-9301570\65e89cd8-abb1-41a7-9761-95716f2d2e78.jpg" />. Figures 1 and 2 show histograms for the bootstrap samples <img src="1-9301570\9e8a02af-bd90-4fc3-a4ab-446562557501.jpg" /> and <img src="1-9301570\3ee3ef07-792d-409a-b034-061eb98e00bc.jpg" /> to see bootstrap distributions of <img src="1-9301570\93927f38-2d27-4a89-b1d7-82b54804e1b6.jpg" /> and<img src="1-9301570\ff51776c-ab45-450d-9141-59a184549ef8.jpg" />, respectively. And we have bootstrap samples for the software reliability assessment measures, such as the expected number of remaining fault at the termination time of the testing, <img src="1-9301570\369fd0df-e556-4ade-9399-bc791f671e22.jpg" />, and the software reliability, <img src="1-9301570\f233c1fc-7e72-42a9-9250-421ad292e3c6.jpg" />, respectively.</p><p>These bootstrap samples of <img src="1-9301570\ec594dd5-cdc9-439c-b119-870f967f8b40.jpg" /> and <img src="1-9301570\351d1f0e-878f-4626-b073-9befb1201f1b.jpg" /> are calculated by</p><disp-formula id="scirp.30069-formula14457"><label>, (14)</label><graphic position="anchor" xlink:href="1-9301570\44b0159f-2542-4af2-b6e7-64a2876a614f.jpg"  xlink:type="simple"/></disp-formula><disp-formula id="scirp.30069-formula14458"><label>(15)</label><graphic position="anchor" xlink:href="1-9301570\aa0bbd4d-9cbf-4904-9ccd-8416ba6a60f1.jpg"  xlink:type="simple"/></disp-formula><p>From Equations (7) and (8), respectively. Figures 3 and 4 show histograms of the bootstrap samples for <img src="1-9301570\f0c8f513-7a8e-48fa-b398-b9db30769035.jpg" /> and<img src="1-9301570\cc7f31d5-ff6c-42d2-8c7f-a1beebd063ff.jpg" />, respectively. Further, <xref ref-type="table" rid="table1">Table 1</xref> indicates the mean and the standard deviations of the estimators of<img src="1-9301570\c81e2c8a-cf5f-4fb8-adf5-8ebad21f8f4a.jpg" />, <img src="1-9301570\e75a68de-363f-473e-be77-b8e5dfbb4c60.jpg" />, <img src="1-9301570\0490a8de-acdc-4fa8-b908-7260a65f147b.jpg" />, <img src="1-9301570\5e822448-39b3-4fff-ae76-efdca3d44890.jpg" />, <img src="1-9301570\c80a1879-5952-4dae-9550-b2c494beb17e.jpg" />, and<img src="1-9301570\7cfa5b1c-18c1-4efc-b04e-289ddd790b6d.jpg" />, respectively. And, <xref ref-type="fig" rid="fig5">Figure 5</xref> shows the estimated discretized exponential software reliability growth model, <img src="1-9301570\80776b05-f8ef-40ff-b3ba-5a2eb2028964.jpg" />, in which we use the means of the bootstrap samples of <img src="1-9301570\4e429a80-e839-4b1a-8a63-17f1ab207da7.jpg" /> and <img src="1-9301570\6440084c-6579-4e36-97ff-0b061e0f68cb.jpg" /> as the point estimations, respectively. The means of <img src="1-9301570\9d927cac-56e5-4425-8567-f367a32b5e4c.jpg" /> and<img src="1-9301570\873d86f6-1768-43dd-b69a-a1af06601c9c.jpg" />, which are denoted by <img src="1-9301570\8185a455-4a74-4e59-ab68-9b7ba41a8d82.jpg" /> and<img src="1-9301570\8bfd92d7-b13e-4f82-8ecb-1a4331bc79c0.jpg" />, are calculated by</p><disp-formula id="scirp.30069-formula14459"><label>(16)</label><graphic position="anchor" xlink:href="1-9301570\1a18c7d8-f074-4cf5-a281-16614258195a.jpg"  xlink:type="simple"/></disp-formula><disp-formula id="scirp.30069-formula14460"><label>(17)</label><graphic position="anchor" xlink:href="1-9301570\281a7aeb-43b2-47b1-9c10-a3449f4d1428.jpg"  xlink:type="simple"/></disp-formula><p>respectively.</p><p>Further, <xref ref-type="table" rid="table2">Table 2</xref> shows the results of interval estimations based on the basic, standard normal, percentile, bootstrap-t, and BCa methods, respectively, with the 5% significance level<img src="1-9301570\7906457c-4525-44e0-8504-8fb05ae9cdf9.jpg" />. From <xref ref-type="table" rid="table2">Table 2</xref>, we can see that the inappropriate confidence intervals for <img src="1-9301570\3b3478e2-f572-46d8-ba20-4deebd8d40c1.jpg" /> are estimated in the basic, standard normal, and bootstrap-t confidence intervals because the number of re-</p><p><xref ref-type="table" rid="table1">Table 1</xref>. Quantities of the bootstrap distribution.</p><p><img src="1-9301570\4246a1ec-fee3-46f2-9b5d-553af85eaa9d.jpg" /></p><p>maining faults does not never take a negative value. And depending on the type of the bootstrap confidence interval, the results of interval estimations on <img src="1-9301570\31312a75-8e3e-4c70-b408-1c1fbd2079dd.jpg" /> are notably different each other. These results are caused by assuming the symmetric distributions to derive these bootstrap confidence intervals. However, the probability distribution of an estimator follows an asymmetric distribution and the approximate accuracy is influenced by the bias and the skewness of the probability distribution for the estimator generally. As we show in <xref ref-type="fig" rid="fig3">Figure 3</xref>, we can say the bootstrap distribution for <img src="1-9301570\a24074d1-2065-42bb-8d27-935527908e54.jpg" /> follows an asymmetric distribution. On the other hand, the percentile bootstrap confidence interval give us an appropriate interval estimations on <img src="1-9301570\9c711e17-6fb6-4cfd-aff1-4ab79e92fb98.jpg" /> because the interval estimation based on the percentile method is estimated based on only the bootstrap distribution, not assumed a symmetric distribution. Of course, the interval estimations based on the BCa method can be thought that we have more appropriate interval estimation on <img src="1-9301570\44646dc4-b91c-43b3-a929-8c9558cfa937.jpg" /> because the BCa method is the improved estimation method for the percentile bootstrap confidence interval.</p><p><xref ref-type="table" rid="table2">Table 2</xref>. Results of interval estimations based on bootstrap confidence intervals.</p><p><img src="1-9301570\7a6ee979-f93e-4c27-bb3e-166178b5e4ba.jpg" /></p></sec><sec id="s5"><title>5. Conclusions</title><p>This paper discussed a bootstrap software reliability assessment method based on a discretized exponential software reliability growth model. And we discussed five types of bootstrapping confidence intervals for interval estimations of model parameters and several software reliability assessment measures.</p><p>In our numerical examples, we confirmed that our bootstrap approach gives probability distributions of each parameters and software reliability assessment measures numerically even if we do not derive these probability distributions analytically, and that we can obtain useful information in software reliability assessment, such as results of interval estimations on the model parameters, the number of remaining faults, and software reliability. This approach is very useful for the case that we cannot collect enough number of data and we have to conduct interval estimation for complex estimators. However, regarding bootstrap confidence intervals, we encountered a problem that we could not get appropriate interval estimations in the basic, standard normal, and bootstrap-t confidence intervals for the number of remaining faults at the termination time of the testing. This problem was solved by using other bootstrap confidence intervals, such as the percentile and the BCa confidence intervals. In the future studies, we are going to apply our bootstrap approach for estimating optimal software release time and other practical software project management issues.</p></sec><sec id="s6"><title>6. Acknowledgements</title><p>The second author is supported in part by the Grant-inAid for Scientific Research (C), Grant No. 22510150, from the Ministry of Education, Culture.</p></sec><sec id="s7"><title>REFERENCES</title></sec></body><back><ref-list><title>References</title><ref id="scirp.30069-ref1"><label>1</label><mixed-citation publication-type="other" xlink:type="simple">J. D. Musa, “A Theory of Software Reliability and Its Application,” IEEE Transactions on Software Engineering, Vol. SE-1, No. 3, 1975, pp. 312-327.   
doi:10.1109/TSE.1975.6312856</mixed-citation></ref><ref id="scirp.30069-ref2"><label>2</label><mixed-citation publication-type="other" xlink:type="simple">A. L. Goel, “Software Reliability Models: Assumptions, Limitations, and Applicability,” IEEE Transactions on Software Engineering, Vol. SE-11, No. 12, 1985, pp. 1411-1423. doi:10.1109/TSE. 1985.232177</mixed-citation></ref><ref id="scirp.30069-ref3"><label>3</label><mixed-citation publication-type="other" xlink:type="simple">J. D. Musa, D. Iannio and K. Okumoto, “Software Reliability: Measurement, Prediction, Application,” McGrawHill, New York, 1987.</mixed-citation></ref><ref id="scirp.30069-ref4"><label>4</label><mixed-citation publication-type="other" xlink:type="simple">H. Pham, “Software Reliability,” Springer-Verlag, Singapore, 2000.</mixed-citation></ref><ref id="scirp.30069-ref5"><label>5</label><mixed-citation publication-type="other" xlink:type="simple">S. Inoue and S. Yamada, “Discrete Software Reliability Assessment with Discretized NHPP Models,” Computers &amp; Mathematics with Applications: An International Journal, Vol. 51, No. 2, 2006, pp. 161-170.    
doi:10.1016/j.camwa.2005.11.022</mixed-citation></ref><ref id="scirp.30069-ref6"><label>6</label><mixed-citation publication-type="other" xlink:type="simple">S. Inoue and S. Yamada, “Integrable Difference Equations for Software Reliability Assessment and Their Applications,” International Journal of Systems Assurance Engineering and Management, Vol. 1, No. 1, 2010, pp. 2-7. doi:10.1007/s13198-010-0005-x</mixed-citation></ref><ref id="scirp.30069-ref7"><label>7</label><mixed-citation publication-type="other" xlink:type="simple">S. Yamada and S. Osaki, “Software Reliability Growth Modeling: Models and Applications,” IEEE Transactions on Software Engineering, Vol. SE-11, No. 12, 1985, pp. 1431-1437. doi:10.1109/TSE. 1985.232179</mixed-citation></ref><ref id="scirp.30069-ref8"><label>8</label><mixed-citation publication-type="other" xlink:type="simple">B. Efron, “Bootstrap Methods: Another Look at the Jackknife,” The Annals of Statistics, Vol. 7, No. 1, 1979, pp. 1-26. doi:10.1214/aos/1176344552</mixed-citation></ref><ref id="scirp.30069-ref9"><label>9</label><mixed-citation publication-type="other" xlink:type="simple">M. Kimura, “A study on Bootstrap Confidence Intervals of Software Reliability Measures Based on an Incomplete Gamma Function Model,” In: T. Dohi and W. Y. Yun, Eds., Advanced Reliability Modeling II, World Scientific, Singapore City, 2006, pp. 419-426.</mixed-citation></ref><ref id="scirp.30069-ref10"><label>10</label><mixed-citation publication-type="other" xlink:type="simple">M. Kimura and T. Fujiwara, “A Bootstrap Software Reliability Assessment Method to Squeeze out Remaining Faults,” In: T. H. Kim and H. Adeli, Eds., Advances in Computer Science and Information Technology, SpringerVerlag, Berlin-Heidelberg, 2010, pp. 435-446.  
doi:10.1007/978-3-642-13577-4_39</mixed-citation></ref><ref id="scirp.30069-ref11"><label>11</label><mixed-citation publication-type="other" xlink:type="simple">T. Kaneishi and T. Dohi, “Parametric Bootstrapping for Assessing Software Reliability Measures,” Proceedings of the 17th IEEE Pacific Rim International Symposium on Dependable Computing, 12-14 December 2010, pp. 1-9.</mixed-citation></ref><ref id="scirp.30069-ref12"><label>12</label><mixed-citation publication-type="other" xlink:type="simple">S. Tokumoto, T. Dohi and W. Y. Yun, “Toward Development of Risk-Based Checkpointing Scheme via Parametric Bootstrapping,” Proceedings of the 2012 Workshop on Recent Advances in Software Dependability, 19 November 2012, pp. 50-55.</mixed-citation></ref><ref id="scirp.30069-ref13"><label>13</label><mixed-citation publication-type="other" xlink:type="simple">S. Yamada and S. Osaki, “Discrete Software Reliability Growth Models,” Journal of Applied Stochastic Models and Data Analysis, Vol. 1, No. 1, 1985, pp. 65-77.   
doi:10.1002/asm.3150010108</mixed-citation></ref><ref id="scirp.30069-ref14"><label>14</label><mixed-citation publication-type="other" xlink:type="simple">R. Hirota, “Nonlinear Partial Difference Equations. V. Nonlinear Equations Reducible to Linear Equations,” Journal of the Physical Society of Japan, Vol. 46, No. 1, 1979, pp. 312-319. doi:10.1143/ JPSJ.46.312</mixed-citation></ref><ref id="scirp.30069-ref15"><label>15</label><mixed-citation publication-type="other" xlink:type="simple">D. Satoh, “A Discrete Gompertz Equation and a software Reliability Growth Model,” IEICE Transactions on Information and Systems, Vol. E83-D, No. 7, 2000, pp. 1508-1513.</mixed-citation></ref><ref id="scirp.30069-ref16"><label>16</label><mixed-citation publication-type="other" xlink:type="simple">D. Satoh, “A Discrete Bass Model and Its Parameter Estimation,” Journal of the Operations Research Society of Japan, Vol. 44, No. 1, 2001, pp. 1-18.</mixed-citation></ref><ref id="scirp.30069-ref17"><label>17</label><mixed-citation publication-type="other" xlink:type="simple">M. L. Rizzo, “Statistical Computing with R,” Chapman and Hall/CRC, Boca Raton, 2008.</mixed-citation></ref><ref id="scirp.30069-ref18"><label>18</label><mixed-citation publication-type="other" xlink:type="simple">B. Efron, “Better Bootstrap Confidence Intervals,” Journal of the American Statistical Association, Vol. 82, No. 397, 1987, pp. 171-185.  
doi:10.1080/01621459.1987.10478410</mixed-citation></ref></ref-list></back></article>