<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article  PUBLIC "-//NLM//DTD Journal Publishing DTD v3.0 20080202//EN" "http://dtd.nlm.nih.gov/publishing/3.0/journalpublishing3.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" dtd-version="3.0" xml:lang="en" article-type="research article"><front><journal-meta><journal-id journal-id-type="publisher-id">JSEA</journal-id><journal-title-group><journal-title>Journal of Software Engineering and Applications</journal-title></journal-title-group><issn pub-type="epub">1945-3116</issn><publisher><publisher-name>Scientific Research Publishing</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="doi">10.4236/jsea.2014.78060</article-id><article-id pub-id-type="publisher-id">JSEA-47998</article-id><article-categories><subj-group subj-group-type="heading"><subject>Articles</subject></subj-group><subj-group subj-group-type="Discipline-v2"><subject>COMPUTER SCIENCE &amp; COMMUNICATIONS</subject><subject>ENGINEERING</subject></subj-group></article-categories><title-group><article-title>Fine-Grained Work Element Standardization for Project Effort Estimation</article-title></title-group><contrib-group><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Peraphon</surname><given-names>Sophatsathit</given-names></name><xref ref-type="aff" rid="aff1"><sub>1</sub></xref><xref ref-type="corresp" rid="cor1"><sup>*</sup></xref></contrib></contrib-group><aff id="aff1"><label>1</label><addr-line>Department of Mathematics and Computer Science, Faculty of Science, Chulalongkorn University, Bangkok, Thailand</addr-line></aff><author-notes><corresp id="cor1">* E-mail:<email>speraphon@gmail.com</email></corresp></author-notes><pub-date pub-type="epub"><day>07</day><month>07</month><year>2014</year></pub-date><volume>07</volume><issue>08</issue><fpage>655</fpage><lpage>669</lpage><history><date date-type="received"><day>6</day>	<month>May</month>	<year>2014</year></date><date date-type="rev-recd"><day>2</day>	<month>June</month>	<year>2014</year>	</date><date date-type="accepted"><day>1</day>	<month>July</month>	<year>2014</year></date></history><permissions><copyright-statement>&#169; Copyright  2014 by authors and Scientific Research Publishing Inc. </copyright-statement><copyright-year>2014</copyright-year><license><license-p>This work is licensed under the Creative Commons Attribution International License (CC BY). http://creativecommons.org/licenses/by/4.0/</license-p></license></permissions><abstract><p>
	Traditional project effort
estimation utilizes development models that span the entire project life cycle
and thus culminates estimation errors. This exploratory research is the first
attempt to break each project activity down to smaller work elements. There are
eight work elements, each of which is being defined and symbolized with visually
distinct shape. The purpose is to standardize the operations associated with
the development process in the form of a visual symbolic flow map. Hence,
developers can streamline their work systematically. Project effort estimation
can be determined based on these standard work elements that not only help
identify essential cost drivers for estimation, but also reduce latency cost to
enhance estimation efficiency. Benefits of the proposed work element scheme are
project visibility, better control for immediate pay-off and, in long term
management, standardization for software process automation.
</p></abstract><kwd-group><kwd>Project Effort Estimation</kwd><kwd> Work Element</kwd><kwd> Symbolic Flow Map</kwd><kwd> Latency Cost</kwd><kwd> Software Process  Automation</kwd></kwd-group></article-meta></front><body><sec id="s1"><title>1. Introduction</title><p>Traditional software project effort and cost estimation rely primarily on life cycle models, namely, waterfall, spiral, water-fountain, etc. The amount of effort expended, which eventually is translated into cost and hereafter may be used interchangeably, must be determined before the project commences. Hence, effort estimation is inevitably carried out in the absence of project work details. Various cost estimation techniques and models have been devised to arrive at an educated guess work. A few popular ones are SLIM model/Putnam model [<xref ref-type="bibr" rid="scirp.47998-ref1">1</xref>] , COnstructiveCOstMOdel (COCOMO 81 [<xref ref-type="bibr" rid="scirp.47998-ref2">2</xref>] and COCOMO II [<xref ref-type="bibr" rid="scirp.47998-ref3">3</xref>] ), Walston-Felix model [<xref ref-type="bibr" rid="scirp.47998-ref4">4</xref>] , Bailey-Basili model [<xref ref-type="bibr" rid="scirp.47998-ref5">5</xref>] , Albrecht-Gaffney model [<xref ref-type="bibr" rid="scirp.47998-ref6">6</xref>] , Kemerer empirical model [<xref ref-type="bibr" rid="scirp.47998-ref7">7</xref>] , and Matson, Barrett and Mellichamp model [<xref ref-type="bibr" rid="scirp.47998-ref8">8</xref>] . These models are supported by extensive researches and well-established metrics to derive practical costing formula for accurate estimation. The underlying metrics that participate in the estimation process are lines of code (LOC), function point (FP) [<xref ref-type="bibr" rid="scirp.47998-ref9">9</xref>] , use case point (UCP) [<xref ref-type="bibr" rid="scirp.47998-ref10">10</xref>] , application point (AP) [<xref ref-type="bibr" rid="scirp.47998-ref3">3</xref>] , and their variants such as source lines of code (SLOC), delivered source instructions (DSI), unadjusted use case point (UUCP), etc. Nevertheless, one of the biggest caveats is the granularity of measurement basis. As project life cycle (LC) spans a considerable period of time, smaller phase-wise breakdown is set up along well-defined development process cut-off points, or technically speaking, project milestones. Seemingly manageable as the phases are, phase-wise estimation is still uncommon and impractical to perform, hence the dominance of LC level measure. All of these classical approaches use project lifecycle span in the estimation process which inevitably incorporate estimation errors, hidden phase overhead, and phase-transition overhead. The latter two cost factors hereafter will be referred to as latency overheads.</p><p>The above effort estimation research attempts are just tip of the ice burg. Measurements and their accuracy are still subject to validation that involves numerous techniques, which unfortunately are far from maturity. Many researches are underway to arrive at reliably accurate effort estimation techniques, ranging from parametric modeling, knowledge-based modeling, case-based reasoning, statistical inferences, fuzzy logic, neural networks, and analogies [<xref ref-type="bibr" rid="scirp.47998-ref11">11</xref>] [<xref ref-type="bibr" rid="scirp.47998-ref12">12</xref>] . What follows the underlying method of these models is how well the method performs. This includes errors of measurement, variations in the data sets, and comparative performance statistics with other techniques.</p><p>This paper will explore a finer grain of project effort estimation based on a well-established measurement paradigm and emerging research endeavors. The study encompasses various viewpoints that propose a breakdown of project LC in order to perform finer grained estimate at activity level. Standard operational elements will be defined and put to use. The ultimate goal is to streamline work elements so that the amount of time (which subsequently can be converted to equivalent effort) can be systematically and accurately estimated. Nevertheless, whether or not the pros and cons of these novel viewpoints will be applicable to traditional project management, as well as new paradigms of practice such as agile, eXtreme Programming (XP), and Agile Unified Process (AUP), remain to be validated with real project implementation.</p><p>This paper is organized as follows. Section 2 recounts some principal building blocks that are applied in many literatures. Section 3 describes traditional project LC estimation, along with Industrial Engineering standard work study, that sets up the proposed fine-grained estimation approach. Section 4 elucidates preliminary comparative analyses of the proposed breakdown. Suggestions for future direction and prospectus will be discussed in Section 5. Some final thoughts are presented in the Conclusion.</p></sec><sec id="s2"><title>2. Related Work</title><p>Conventional effort estimation techniques [<xref ref-type="bibr" rid="scirp.47998-ref13">13</xref>] focus on project life cycle data to carry out the project effort and cost involved. Various estimation techniques have been devised to improve estimation accuracy. <xref ref-type="table" rid="table1">Table 1</xref> summarizes a brief predominant research work and their findings in this area.</p><p>A few emerging research endeavors have been attempted to estimate at a finer level using phase-wise project data. Breaking up in phases may uncover activities that were hidden too subjective to measure and average out by LC approach. Different project dimensions can then be taken into account to arrive at a more accurate estimation. The empirical findings usually are inapplicable for generic use depending on phase distribution [<xref ref-type="bibr" rid="scirp.47998-ref14">14</xref>] . Various user groups such as IFPUG, ISBSG (International Software Benchmarking Standards Group), are examples of domain specific estimation. Project managers (PMs) must decide what data are to be collected to suit the applicable domain. Jodpimai, Sophatsathit, and Lursinsap [<xref ref-type="bibr" rid="scirp.47998-ref15">15</xref>] explored the relationship of different project dimensions to select only a handful of relevant cost drivers as oppose to standard 16 factors in existing approaches, yet yielding similar outcome. The needs for standardizing its deliverables and development process are key factors to software products. Typical acclaimed standards and working groups are:</p><p> ISO/IEC 14143 [six parts] Information Technology—Software Measurement—Functional Size Measurement</p><p> ISO/IEC 19761:2011 Software engineering—COSMIC: A Functional Size Measurement Method</p><p> ISO/IEC 20926:2009 Software and Systems Engineering—Software Measurement—IFPUG Functional Size Measurement Method</p><p> ISO/IEC 20968:2002 Software engineering—Mk II Function Point Analysis—Counting Practices Manual</p><table-wrap id="table1"  position="float"><object-id pub-id-type="pii">Table 1</object-id><label>Table 1</label><caption><p>. A brief overview of effort estimation research work</p></caption><table><thead><tr><th align="center" valign="middle" >Year/author</th><th align="center" valign="middle" >Principal investigator</th><th align="center" valign="middle" >Techniques/metrics</th><th align="center" valign="middle" >Evaluation</th></tr></thead><tbody><tr><td align="center" valign="middle" >2012 Kocaguneli</td><td align="center" valign="middle" >Ensemble</td><td align="center" valign="middle" >Multimethods/solo method</td><td align="center" valign="middle" >MAR, MMRE, MdMRE, MMER, PRED (24), MBRE, MIBRE</td></tr><tr><td align="center" valign="middle" >2012 Dejaeger [16] </td><td align="center" valign="middle" >Data mining</td><td align="center" valign="middle" >OLS, OLS + log, OLS + BC, Robust regression, Ridge regression, Least median squares, MARS, CART, Model tree, MLP NN, RBFN, CBR, LS + SVM</td><td align="center" valign="middle" >MdMRE, PRED (0.25), Spearman’s rank correlation</td></tr><tr><td align="center" valign="middle" >2008 Port</td><td align="center" valign="middle" >Model evaluation</td><td align="center" valign="middle" >Models</td><td align="center" valign="middle" >MMRE, PRED</td></tr><tr><td align="center" valign="middle" >2008 Yang</td><td align="center" valign="middle" >Phase distribution</td><td align="center" valign="middle" >Models</td><td align="center" valign="middle" >KSLOC</td></tr><tr><td align="center" valign="middle" >2007 Li</td><td align="center" valign="middle" >Analogy</td><td align="center" valign="middle" >SIM</td><td align="center" valign="middle" >MMRE, PRED (0.25)</td></tr><tr><td align="center" valign="middle" >1994 Matson</td><td align="center" valign="middle" >Function point</td><td align="center" valign="middle" >Models</td><td align="center" valign="middle" >MRE</td></tr><tr><td align="center" valign="middle" >1987 Kemerer</td><td align="center" valign="middle" >Empirical model</td><td align="center" valign="middle" >KSLOC</td><td align="center" valign="middle" >MRE</td></tr><tr><td align="center" valign="middle" >1983 Albrecht, Gaffney</td><td align="center" valign="middle" >Function point, source lines of code</td><td align="center" valign="middle" >FP, KSLOC</td><td align="center" valign="middle" >MRE</td></tr><tr><td align="center" valign="middle" >1981/2000 Boehm</td><td align="center" valign="middle" >COCOMO</td><td align="center" valign="middle" >KDSI, KSLOC</td><td align="center" valign="middle" >LOC</td></tr><tr><td align="center" valign="middle" >1979 Albrecht</td><td align="center" valign="middle" >Function point</td><td align="center" valign="middle" >FP</td><td align="center" valign="middle" >UFP</td></tr><tr><td align="center" valign="middle" >1978 Putnam</td><td align="center" valign="middle" >SLIM</td><td align="center" valign="middle" >Norden/Rayleigh</td><td align="center" valign="middle" >Regression</td></tr></tbody></table></table-wrap><p> ISO/IEC 24570:2005 Software Engineering—NESMA Functional Size Measurement Method Version 2.1 —Definitions and Counting Guidelines for the Application of Function Point Analysis</p><p> 16326 WG—Information Technology-Software Engineering—Software Project Management Working Group (IEEE)</p><p> ERCIM Working Group Software Evolution</p><p> Project Management Institute (PMI) and Project Management Body of Knowledge (PMBOK<sup>&#174;</sup>)</p><p>While conventional estimation models take the entire LC activities into estimation consideration, finer grained activity breakdown has not been practiced in real software projects. Frederick W. Taylor [<xref ref-type="bibr" rid="scirp.47998-ref17">17</xref>] introduced principles of scientific management in 1911, and Frank B. Gilbreth [<xref ref-type="bibr" rid="scirp.47998-ref18">18</xref>] set up operational work elements that subsequently became work study standard measurement known as Therblig. This study will exploit the use of work element process chart to create fine-grained project activity elements, thereby effort estimation can be determined more accurately than current LC practice.</p><p>Such a fine-grained breakdown entails phase-wise estimation that in turn permits detailed project visibility. In so doing, this work will attempt to adapt UML style to represent project activity element. Consequently, effort estimation can be managed systematically using standard flow diagrams, activity element set up, and visibly traceable operations.</p></sec><sec id="s3"><title>3. Proposed Fine-Grained Operational Estimation Technique</title><p>Classical cost estimation employs the mixture of constituent cost factors (aka cost drivers) using COCOMO or COCOMO II to determine project cost. However, the number of parameters has been a formidable obstacle to accurate effort estimation since project inception. Several recent research findings [<xref ref-type="bibr" rid="scirp.47998-ref19">19</xref>] [<xref ref-type="bibr" rid="scirp.47998-ref20">20</xref>] advocate ensemble of multiple estimation methods, taking joint attributes into account. The lack of recognition to process variation and effort distribution pattern over project LC are predominant threats to model validity. Some shortcomings of existing approaches often result from inaccurate estimation, high risks of project failure, and ill-defined project artifacts such as workload, assignment, etc. For instance, project size is unarguably the principal estimator that is used throughout the entire project LC. If the project is broken down into finer grain of measure on phase basis, project size may no longer be the appropriate cost estimator of feasibility study and requirement analysis phases.</p><p><xref ref-type="fig" rid="fig1">Figure 1</xref> depicts an overview of simplified project work flow for a typical Software Project Management course. The prerequisite of this course is Software Engineering, thereby students will be ready to embark on application of software engineering models, techniques, process, and hands-on experience. Documents prescribing</p><fig id="fig1"><label>Figure 1</label><caption><p> Overview of simplified project work flow</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\7381722e-5f21-4a32-a330-11a7bdaa4faf.png"/></fig><p>dotted lines are basically internal information used by developers, while those on solid lines are external information open to customers. As each activity moves from one phase to the next, efforts are spent on constituent tasks to accomplish the activity. Unfortunately, the work load involved are not straightforwardly and procedurally described, measured, and collected to determine a reliable estimation. An exploratory research attempt based on well-established Motion and Time Study by Industrial Engineering has been conducted with the hope of setting up work elements in project effort estimation. <xref ref-type="fig" rid="fig2">Figure 2</xref> shows the 18 standard motion elements in Motion and Time Study.</p><p>The prime objective of this undertaking is to create visually standard symbols to represent descriptive and subjective constituent smaller tasks of project activity. In so doing, PM and team members can streamline their development process to arrive at a more objectively manageable work flow. Hence, an analogy of Therbligs has been defined in <xref ref-type="fig" rid="fig3">Figure 3</xref> to establish standard work elements involved in each activity. The first element “O” denotes operation to be performed by the team members. Subsequent elements, namely, “D”, “I”, “M”, “T”, “A”, “S”, “P” denote delay, inspection or review, meeting or discussion, movement or transfer, adjustment, storage, and planning or decision, respectively.</p><p><xref ref-type="fig" rid="fig4">Figure 4</xref> illustrates a symbolic flow map of how the activity is procedurally broken down to work elements. The illustration shows a dialog box activity that is broken down to seven smaller tasks proceeding in the following sequence: O-O-P-M-I-T-T-A-P. This symbolic flow map not only serves as a visual trace of development process work flow, but also a detailed estimation of processing time which can be straightforwardly converted to equivalent effort estimate.</p><p>Details on procedural research will be carried out as follows. First and foremost, data are collected from students’ reports using pre-designed forms. Second, missing value and outlier will be screened out to reduce large variations. Third, data are normalized on the same comparable scale. Fourth, feature selection is performed to choose only relevant features for use in the estimation. Finally, performance evaluation will be applied to gauge how the proposed approach stacks up against traditional LC approach. <xref ref-type="fig" rid="fig5">Figure 5</xref> illustrates the proposed process flow which will be elucidated in the sections that follow.</p><fig id="fig2"><label>Figure 2</label><caption><p> Standard 18 motion elements or Therbligs. Source: Wikipedia (accessed on May 16, 2014)</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\be012ce6-ba67-42bc-a15d-5dc0d2dd7971.png"/></fig><fig id="fig3"><label>Figure 3</label><caption><p> A novel standard work element definition</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\78f16254-bc5a-4882-b552-09c250d82922.png"/></fig><fig id="fig4"><label>Figure 4</label><caption><p> A symbolic flow map of an activity breakdown to work elements</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\4d69b23d-8248-48cb-820e-25f5ac381108.png"/></fig><sec id="s3_1"><title>3.1. Data Collection</title><p>The novel approach will utilize conventional waterfall model where each phase is well defined and known by all software developers. As existing standard benchmarking archives such as COCOMO81, NASA93, Desharnais, USP05, MAXWELL<sup>1</sup> are available in overall LC figures, this exploratory research will be conducted on a senior elective Software Project Management class to see how this novel scheme can be put to real use.</p><p>The process began from collecting data of the uppercase phases, i.e., requirements, specification, and design,</p><fig id="fig5"><label>Figure 5</label><caption><p> The proposed research process flow</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\253b1703-0f82-438b-9093-12566fc26b61.png"/></fig><p>and the lowercase phases, i.e., coding, testing, and implementation. At phase level, functional size measurement [<xref ref-type="bibr" rid="scirp.47998-ref20">20</xref>] such as FP and LOC are employed as high level effort estimation since effort estimation metrics are conventionally governed by these two major metrics. Moreover, FP metric can be converted to LOC via backfiring [<xref ref-type="bibr" rid="scirp.47998-ref21">21</xref>] . The information can then be used to estimate overall LC effort. The results so obtained would be used to gauge the accuracy of the proposed fine-grained measurement. <xref ref-type="table" rid="table2">Table 2</xref> depicts typical series of activities during the project LC.</p><p>An operation data sheet is set up as shown in <xref ref-type="fig" rid="fig6">Figure 6</xref> to record data pertinent to the above symbolic flow map, including idle or overhead that may or may not be prime tasks. Project related data are filled up in the header section, namely, activity, activity ID (primary key), programmer (or group leader), group ID, build, document control number, page, start date, activity duration, project name, and authorized personnel.</p><p>Measurements are perhaps the heart of this fine-grained approach. The time it takes to perform each task, by the very novel notion, can be taken directly from “standard time” plus “allowance” to arrive at operation time. This is equal to man-time measurement or, in the context of this study equivalent effort (equiv-Effort), i.e., man-hour, man-day, etc. The question is how both standard time and allowance are obtained. Since this is the first attempt ever to introduce such a novel notion, no reference values can be used as oppose to those of the Industrial Engineering. The fact that software development procedure has not been standardized and the nature of the development process varies from task to task, programmer to programmer, makes standardization virtually impossible and impractical to implement with current state-of-the-practice. Notice that each task plays a different role under certain situations. On the one hand, it may play the prime task, so some allowances such as personal, fatigue, administrative, are added to supplement the operation time. On the other hand, it may be idle or overhead that must be taken separately into account as part of inherent “allowance”. For example, discussion and technical resolution in the operation data sheet is a prime task in design, especially among the design team members. Thus, allowance should be incorporated explicitly. However, during unit test, individual spends less</p><fig id="fig6"><label>Figure 6</label><caption><p> Operation data sheet</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\d2e4f28f-b14e-4813-8d34-4e828ae52e25.png"/></fig><table-wrap id="table2"  position="float"><object-id pub-id-type="pii">Table 2</object-id><label>Table 2</label><caption><p>. Activities and metrics of project tasks</p></caption><table><thead><tr><th align="center" valign="middle" >Phase activity</th><th align="center" valign="middle" >Input</th><th align="center" valign="middle" >Output</th><th align="center" valign="middle" >Estimation metrics</th></tr></thead><tbody><tr><td align="center" valign="middle" >Software Requirements Specification (SRS)</td><td align="center" valign="middle" >no. of team members</td><td align="center" valign="middle" >Functionality</td><td align="center" valign="middle" >FP</td></tr><tr><td align="center" valign="middle" >Design</td><td align="center" valign="middle" >no. of requirement/member</td><td align="center" valign="middle" >Functional model</td><td align="center" valign="middle" >FP</td></tr><tr><td align="center" valign="middle" >Coding</td><td align="center" valign="middle" >Design document</td><td align="center" valign="middle" >Code</td><td align="center" valign="middle" >LOC</td></tr><tr><td align="center" valign="middle" >Testing</td><td align="center" valign="middle" >no. of rework</td><td align="center" valign="middle" ># errors/module</td><td align="center" valign="middle" >LOC</td></tr><tr><td align="center" valign="middle" >Implementation</td><td align="center" valign="middle" >no. of transactions from  production code</td><td align="center" valign="middle" >Deliverable Source Instruction</td><td align="center" valign="middle" >LOC</td></tr></tbody></table></table-wrap><p>discussion and technical resolution effort with other members of the team except for occasional consultation with test specialist or PM. In this case, allowance may be accounted for as percentage of the test effort which is not considered a prime task. Consequently, the actual activity operation time can be precisely used to determine the equivalent effort estimation, while idle could attribute to latency overhead. The efficiency of operation is directly obtained from utilization factor.</p><p>The above operation data are further broken down by team members to depict work load responsibility of each member (shaded area) as shown in <xref ref-type="fig" rid="fig7">Figure 7</xref>. Team members are listed on top which contains cross references to operation data sheet and other related documents. The breakdown permits individual load distribution percentage to be determined accordingly.</p><p>A trial collection of work element tallies was taken in order to gauge their distribution based on the governing activity breakdown. These statistics would serve as preliminary effectiveness of fine-grained measures, which in turn could attribute to standardization of activity measurement.</p></sec><sec id="s3_2"><title>3.2. Missing Value and Outlier</title><p>This step deals with two imposing problems in data collection, namely, missing value and outlier detection [<xref ref-type="bibr" rid="scirp.47998-ref22">22</xref>] . Missing value is a common phenomenon in project management where numerous small activities, assignments, and work products are too trivial, hard to identify or account for, or due to human errors. For example, button change, color change of error message display, or unrecorded informal discussion, etc. The efforts spent on such changes are, in many cases, insignificant to be recorded but appeared on request for change or maintenance re-</p><fig id="fig7"><label>Figure 7</label><caption><p> Work load breakdown by team members</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\272d8260-8ad6-4d7c-a32b-c1b3c23e9a25.png"/></fig><p>ports. Outlier usually results from freak accidents or exceptional situations where unusually high or low values are recorded. For example, the existence of data singularity in matrix multiplication caused an unusually difficult and extensive test effort to correct, hence the exceptional high effort outlier value.</p><p>There are several techniques to handle missing values. One of the most popular and effective techniques is k-nearest neighbor (k-NN) imputation [<xref ref-type="bibr" rid="scirp.47998-ref23">23</xref>] . The technique, as its name implies, uses k data points from the clos- est proximity of the missing value position to interpolate the most likely value. Such an imputation obviously incurs some errors if the actual values are not missing. A viable error estimation is to acquire other projects having the same “feature” value. The use of same feature value from other projects permits cross validation that fills the estimation of missing value to yield more accurate results.</p><p>Measuring errors is actually carried out by determining the Euclidean distant between the project having missing values and the ones without. The smaller the average of N measurements, the more accurate the predicted missing values.</p><p>Outlier detection is typically handled by examining the kurtosis and skewness of data distribution. Normality test is set up as the null hypothesis using z-score to determine if there exists a significant possibility that null hypothesis is accepted, i.e., normality holds with less than 0.001. This is written as p-value &lt; 0.001. On the contrary, if the null-hypothesis is rejected, the highest value is treated as the outlier and is discarded. This process is repeated until all outliers are removed.</p></sec><sec id="s3_3"><title>3.3. Data Normalization</title><p>This is a standard technique to linearly scale data of different ranges to the same scale, yet still preserves the relationship with the original values. This is done by Equation (1) as follows:</p><disp-formula id="scirp.47998-formula2019"><label>(1)</label><inline-graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\9945c96c-5ac1-4467-b46c-9f38d2d67160.png"/></disp-formula><p>where <inline-formula><inline-graphic xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\3a187c4d-c77c-4ad8-a2f6-4fcccb3b968d.png" xlink:type="simple"/></inline-formula> denotes the reference range, while x denotes individual range.</p></sec><sec id="s3_4"><title>3.4. Feature Selection</title><p>This is the most important activity of project cost estimation. Many existing estimation techniques utilize several cost factors as estimation parameters. For example, the COCOMO model [<xref ref-type="bibr" rid="scirp.47998-ref3">3</xref>] uses 17 cost drivers in the estimation process. Jodpimai, Sophatsathit, and Lursinsap [<xref ref-type="bibr" rid="scirp.47998-ref15">15</xref>] found that only a handful of cost drivers were effective factors that could derive as accurate estimation as the comparative models without employing the full-fledge parameter set. Moreover, fewer cost drivers translated into faster computation time. The findings revealed that certain features were vital cost drivers that could yield accurate estimation.</p><p>The process of acquiring good selected features is straightforward. In a typical experiment, the data so collected are divided into two non-overlapping sets, that is, a training set and a test set. The former is used in various parameter adjustments during model creation process, whereas the latter is held out for model validation process. The first step is to eliminate independent features that do not contribute or affect project effort estimation. The next step is to reduce all redundant features that are less relevant to project effort estimation. This is done by means of Pearson’s correlation. Features that result in low value will be less relevant and thus eliminated. Finally, only those features that are related to effort estimation in the form of a mathematical function will be retained [<xref ref-type="bibr" rid="scirp.47998-ref15">15</xref>] .</p><p>There are a number of models that can be used in the feature selection process, ranging from conventional COCOMO, RUP, statistical, and neural network model. The basis for effort estimation must rest on proper use of these selected features in the estimation process, thereby accurate estimation results can be obtained.</p><p>Based on standard 17 cost drivers [<xref ref-type="bibr" rid="scirp.47998-ref3">3</xref>] , a phase-wise breakdown and costing features are set up as shown in <xref ref-type="table" rid="table3">Table 3</xref> since the project sizes are relatively small, namely, uppercase and lowercase groups. The first group composes of software complexity, analyst capability, while the second group composes of execution time constraint, main storage constraint, and programmer capability. The features will be applied to their corresponding phases.</p></sec><sec id="s3_5"><title>3.5. Performance Evaluation</title><p>There are many performance evaluation methods and their corresponding metrics for effort estimation. Each method has its own applicability to gauge the model accuracy or the relationship between actual and predicted estimation results, given the set of selected features. <xref ref-type="table" rid="table4">Table 4</xref> summarizes some common methods and metrics for performance evaluation purpose.</p><p>Model evaluation is usually carried out by comparing the difference between predicted (estimated) effort y'<sub>i</sub> and actual effort y<sub>i</sub>. Effort is performed on the phase basis using related factors. For example, factors used in requirements and specification effort estimation involve FP to deal with both functional and non-functional requirements. As project requirements become materialized, size estimation via LOC is used instead since it yields more accurate outcome than that of FP. Some metrics are criticized for penalizing over-estimation (such as MRE), while others are just the opposite (such as BRE and MER). Two commonly used metrics are MMRE and PRED (0.25) since they are independent of units of measure and easy to use. However, they are found to give inconsistent results depending on properties of y'<sub>i</sub>/y<sub>i</sub> distribution. In which case, MdMRE is used to solve the outlier problem as MMRE cannot properly handle such inconsistencies. At any rate, this study adopted MMRE and PRED (0.25) accuracy metrics.</p><p>A noteworthy shortcoming of this exploratory research endeavor concerns project data. The above procedures have been implemented with industrial projects and extensive public data by the author’s research team, where the resulting outcomes are currently under investigation. The very notion of work element has just been introduced after data collection was completed. It was then put to test with classroom environment as it was deemed too novel to be adopted by real projects for the time being. The inherent shortfall of classroom setting was that</p><table-wrap id="table3"  position="float"><object-id pub-id-type="pii">Table 3</object-id><label>Table 3</label><caption><p>. Phase-wise costing features</p></caption><table><thead><tr><th align="center" valign="middle" >Group</th><th align="center" valign="middle" >Feature</th><th align="center" valign="middle" >Phase</th><th align="center" valign="middle" >Attribute/activity</th></tr></thead><tbody><tr><td align="center" valign="middle" >uppercase</td><td align="center" valign="middle" >Software complexity</td><td align="center" valign="middle" >System analysis</td><td align="center" valign="middle" >-Overview analysis</td></tr><tr><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td><td align="center" valign="middle" >System design</td><td align="center" valign="middle" >-Documentation</td></tr><tr><td align="center" valign="middle" ></td><td align="center" valign="middle" >Analyst capability</td><td align="center" valign="middle" >Architectural design</td><td align="center" valign="middle" >-Design overview</td></tr><tr><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td><td align="center" valign="middle" >Detailed design</td><td align="center" valign="middle" >-I/O design</td></tr><tr><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td><td align="center" valign="middle" >-Data/class design</td></tr><tr><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td><td align="center" valign="middle" >Coding</td><td align="center" valign="middle" >-Program/module structure</td></tr><tr><td align="center" valign="middle" >lowercase</td><td align="center" valign="middle" >Execution time</td><td align="center" valign="middle" >Testing</td><td align="center" valign="middle" >-Programming -Unit and integration testing</td></tr><tr><td align="center" valign="middle" ></td><td align="center" valign="middle" >Main storage</td><td align="center" valign="middle" >Production</td><td align="center" valign="middle" >-installation -use/implement</td></tr><tr><td align="center" valign="middle" ></td><td align="center" valign="middle" >Programmer capability</td><td align="center" valign="middle" >Coding</td><td align="center" valign="middle" >-syntactic and logical details</td></tr></tbody></table></table-wrap><table-wrap id="table4"  position="float"><object-id pub-id-type="pii">Table 4</object-id><label>Table 4</label><caption><p>. Summary of methods/metrics for performance evaluation</p></caption><table><thead><tr><th align="center" valign="middle" >Metric</th><th align="center" valign="middle" >Name</th><th align="center" valign="middle" >Ref.</th><th align="center" valign="middle" >Remark</th></tr></thead><tbody><tr><td align="center" valign="middle" >MRE</td><td align="center" valign="middle" >Magnitude of Relative Error</td><td align="center" valign="middle" >[22] </td><td align="center" valign="middle" >|y<sup>'</sup><sub>i</sub> − y<sub>i</sub>|/y<sub>i</sub></td></tr><tr><td align="center" valign="middle" >BRE</td><td align="center" valign="middle" >Balanced Relative Error</td><td align="center" valign="middle" >[24] </td><td align="center" valign="middle" >|y<sup>'</sup><sub>i</sub> − y<sub>i</sub>|/min(y<sup>'</sup><sub>i</sub>, y<sub>i</sub>)</td></tr><tr><td align="center" valign="middle" >MER</td><td align="center" valign="middle" >Magnitude of Error Relative</td><td align="center" valign="middle" >[25] </td><td align="center" valign="middle" >|y<sup>'</sup><sub>i</sub> − y<sub>i</sub>|/y<sup>'</sup><sub>i</sub></td></tr><tr><td align="center" valign="middle" >MMRE</td><td align="center" valign="middle" >Mean Magnitude of Relative Error</td><td align="center" valign="middle" >[22] </td><td align="center" valign="middle" ></td></tr><tr><td align="center" valign="middle" >PRED(1)</td><td align="center" valign="middle" >Prediction at Level l</td><td align="center" valign="middle" >[22] </td><td align="center" valign="middle" ></td></tr><tr><td align="center" valign="middle" >MdMRE</td><td align="center" valign="middle" >Median Magnitude of Relative Error</td><td align="center" valign="middle" >[26] </td><td align="center" valign="middle" >median (MRE)</td></tr><tr><td align="center" valign="middle" >Pearson’s correlation</td><td align="center" valign="middle" >Relation between two sets of data (estimated and actual)</td><td align="center" valign="middle" >[27] </td><td align="center" valign="middle" ></td></tr><tr><td align="center" valign="middle" >Friedman test</td><td align="center" valign="middle" >Non-parametric hypothesis test as an  alternative to one-way ANOVA</td><td align="center" valign="middle" >[26] </td><td align="center" valign="middle" ></td></tr><tr><td align="center" valign="middle" >Wilcoxon matched pairs signed-rank test</td><td align="center" valign="middle" >Non-parametric as an alternative to parametric paired simple t-test</td><td align="center" valign="middle" >[28] </td><td align="center" valign="middle" ></td></tr><tr><td align="center" valign="middle" >Kruskal-Wallis ANOVA test</td><td align="center" valign="middle" >Non-parametric as alternative one-way ANOVA &gt; 3 samples</td><td align="center" valign="middle" >[29] </td><td align="center" valign="middle" ></td></tr></tbody></table></table-wrap><p>project size was too small to warrant any reliable practice or significance.</p></sec></sec><sec id="s4"><title>4. Preliminary Analyses</title><p>This exploratory research sets out to compare the proposed measurement with that of traditional LC. Due to the small number of students enrolled in the class, the samples were not a good representative of any conclusive inferences. Anyhow, the above procedures were followed to ensure that pertinent data were properly collected. Each team consisted of a project manager (PM), a system analyst (SA), a programmer, and a user. Students were asked to estimate phase effort that they would expend. These estimates would be used as the target values to be compared with the fine-grained figures from operation data sheet. They also collected actual LC efforts on class assignment by role and by development phases. <xref ref-type="fig" rid="fig8">Figure 8</xref> depicts work element histogram of individual’s load distribution of the dialog box task based on the breakdown in <xref ref-type="fig" rid="fig7">Figure 7</xref>. <xref ref-type="fig" rid="fig9">Figure 9</xref> and <xref ref-type="fig" rid="fig10">Figure 10</xref> show the total project LC effort (y-axis) expended by role and by phase, respectively.</p><p><xref ref-type="table" rid="table5">Table 5</xref> summarizes individual’s load distribution based on operation data sheet. In analysis phase, the programmer played very little role in the assignment. Thus, his contribution in this phase was virtually non-existent (0.3%), while SA did most of the work in the first two phases and gradually reduced his role afterwards. The programmer made up for the loss in programming and testing, and the user participated heavily during the inception (analysis) phase and the closing (implementation) phase. <xref ref-type="fig" rid="fig11">Figure 11</xref> shows the resulting plot.</p><p><xref ref-type="table" rid="table6">Table 6</xref> and <xref ref-type="fig" rid="fig12">Figure 12</xref> show the value of MMRE and PRED (0.25) and corresponding plot of the class. The predictions from work elements of all parties fluctuated somewhat to the actual value as MMRE was moderate and PRED was not significant enough to yield any close predictions. The deviation was resulted from excessive allowances that were quite difficult to administer. This was a pioneer attempt to undertake something of this nature in Software Engineering which, on the contrary, has been well established with supporting standards in Industrial Engineering.</p><p><xref ref-type="table" rid="table7">Table 7</xref> and <xref ref-type="fig" rid="fig13">Figure 13</xref> show the result comparison of total project effort obtained from COCOMO estimation in organic mode with the actual work element count by the proposed method. The reason being organic mode was that students were familiar and experienced with project development process. Work element breakdown was additional clerical tasks that they had to do as part of project documentation. The effort discrepancies were likely precipitated from fine-grained measurements having fewer latency cost.</p><p>Perhaps the most fruitful findings came from the presence of visual operation from symbolic flow map. Every</p><fig id="fig8"><label>Figure 8</label><caption><p> Individual’s work element distribution of the dialog box task</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\75478bd5-8cc8-4357-b48e-21e8d66d2dd2.png"/></fig><fig id="fig9"><label>Figure 9</label><caption><p> Total project LC effort by role</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\bcb67ad8-893f-4698-b5b4-8212024e2a9b.png"/></fig><fig id="fig10"><label>Figure 10</label><caption><p> Total project LC effort by phase</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\743485ed-534f-49e2-a670-92ddcc3d21e7.png"/></fig><fig id="fig11"><label>Figure 11</label><caption><p> Work load distribution plot</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\9d3d294b-c46f-41b9-ac1b-e84e4cc21b0b.png"/></fig><fig id="fig12"><label>Figure 12</label><caption><p> MMRE and PRED (0.25) plots</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\0612ea4c-1dab-49f8-9b61-43ae8ae823f5.png"/></fig><fig id="fig13"><label>Figure 13</label><caption><p> Actual total work element effort VS COCOMO organic mode estimation plot</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://file.scirp.org/Html/htmlimages\4-9301916x\bf8469a8-9080-4b83-a666-1c96e093601f.png"/></fig><table-wrap id="table5"  position="float"><object-id pub-id-type="pii">Table 5</object-id><label>Table 5</label><caption><p>. Work load distribution from operation data sheet</p></caption><table><thead><tr><th align="center" valign="middle" ></th><th align="center" valign="middle" >Analysis</th><th align="center" valign="middle" >Design</th><th align="center" valign="middle" >Prog &amp; test</th><th align="center" valign="middle" >Implement</th><th align="center" valign="middle" >Total by role</th></tr></thead><tbody><tr><td align="center" valign="middle" >PM</td><td align="center" valign="middle" >71</td><td align="center" valign="middle" >106</td><td align="center" valign="middle" >20</td><td align="center" valign="middle" >45</td><td align="center" valign="middle" >242</td></tr><tr><td align="center" valign="middle" >SA</td><td align="center" valign="middle" >964</td><td align="center" valign="middle" >1375</td><td align="center" valign="middle" >215</td><td align="center" valign="middle" >124</td><td align="center" valign="middle" >2678</td></tr><tr><td align="center" valign="middle" >Programmer</td><td align="center" valign="middle" >4</td><td align="center" valign="middle" >88</td><td align="center" valign="middle" >1091</td><td align="center" valign="middle" >194</td><td align="center" valign="middle" >1377</td></tr><tr><td align="center" valign="middle" >User</td><td align="center" valign="middle" >771</td><td align="center" valign="middle" >155</td><td align="center" valign="middle" >83</td><td align="center" valign="middle" >425</td><td align="center" valign="middle" >1434</td></tr><tr><td align="center" valign="middle" >Total by phase</td><td align="center" valign="middle" >1810</td><td align="center" valign="middle" >1724</td><td align="center" valign="middle" >1409</td><td align="center" valign="middle" >788</td><td align="center" valign="middle" ></td></tr></tbody></table></table-wrap><table-wrap id="table6"  position="float"><object-id pub-id-type="pii">Table 6</object-id><label>Table 6</label><caption><p>. MMRE and PRED (0.25) values</p></caption><table><thead><tr><th align="center" valign="middle" ></th><th align="center" valign="middle" >MMRE</th><th align="center" valign="middle" >PRED (0.25)</th></tr></thead><tbody><tr><td align="center" valign="middle" >PM</td><td align="center" valign="middle" >0.49</td><td align="center" valign="middle" >0.00</td></tr><tr><td align="center" valign="middle" >SA</td><td align="center" valign="middle" >0.46</td><td align="center" valign="middle" >0.04</td></tr><tr><td align="center" valign="middle" >Programmer</td><td align="center" valign="middle" >0.56</td><td align="center" valign="middle" >0.03</td></tr><tr><td align="center" valign="middle" >User</td><td align="center" valign="middle" >0.54</td><td align="center" valign="middle" >0.04</td></tr><tr><td align="center" valign="middle" >Overall</td><td align="center" valign="middle" >0.52</td><td align="center" valign="middle" >0.03</td></tr></tbody></table></table-wrap><table-wrap id="table7"  position="float"><object-id pub-id-type="pii">Table 7</object-id><label>Table 7</label><caption><p>. Actual total work element effort VS COCOMO estimation</p></caption><table><thead><tr><th align="center" valign="middle" >Measurement</th><th align="center" valign="middle" >Effort</th></tr></thead><tbody><tr><td align="center" valign="middle" >Work elememt</td><td align="center" valign="middle" >5732.00</td></tr><tr><td align="center" valign="middle" >COCOMO Organic mode</td><td align="center" valign="middle" >6515.44</td></tr></tbody></table></table-wrap><p>member was well aware of his own assignment and the course of action with other team members. The posted symbolic flow map of each activity (which could be posted in the so called “project war room”) proved to be an extremely valuable tracking tool during a “sit down” meeting to discuss the course of action, effort spent, and possibly (which rarely happened) activity reassignment. This attributed to finer grained analysis that portrayed a more visible activity tracking, not to mention more accurate measurements which in turn lessened the estimation errors. From a practical standpoint, the risk of over/under estimation is reduced, thereby improving project estimation and bidding opportunity.</p></sec><sec id="s5"><title>5. Findings and Future Direction</title><p>This survey sets out to explore fine-grained project cost estimation, aiming at more accurate result than using traditional LC approach. Admittedly, the novelty of such an undertaking does not offer much provision for overall project management. All experimental data were improvised from classroom assignments. As mentioned earlier, performance could not be measured accurately due to the small sample size and unfamiliarity with work element notion. Fortunately, students were special group of people that possessed exceptional abilities, i.e., fast learning, flexibility, and adaptability. These attributes bring new work disciplines and culture to modern project management know-how. That is why this study embarks on a new realm of fine-grained measurement to cope with such a revolutionary transformation. The findings unveil some noteworthy results that have potential benefits to software project management.</p><p> modern software products are short-live which cannot fit into traditional LC analysis. The proposed fine- grained approach could open a new realm of exploiting development process standard to foster some forms of subjective project task measurements.</p><p> fine-grained analysis can be adapted to cope with more scrutinized investigation provided that appropriate metrics and analysis techniques are applied.</p><p>Benefits from this exploratory estimation proposal can be summarized as follows:</p><p>1) Operational work element is an inherent, not accidental characteristic of project management [<xref ref-type="bibr" rid="scirp.47998-ref30">30</xref>] . The nature of software development process irrespective of the underlying model lends itself to collecting data which are readily available. What has not been done is the breakdown of activity structure. In addition, the difficulty of data collection process is seen as disruptive and infeasible as many tasks are operationally intertwined. This makes it hard to succinctly separate.</p><p>2) Modern development paradigms are seen to be unfitted to LC estimation. As the term “phase” has been ingrained in software engineering since the inception of waterfall model and has become a stigma of software project management. A closer look into this investigation reveals that if the term is viewed as a milestone of partial work products that can be accurately measured by a well-established standard, the term can be generalized to cover all paradigms of software development. For example, some “standard” work elements in the dialog box task can be reused elsewhere by other tasks in the project, thus reducing the time and effort to set up and measure similar tasks. As work elements become standardized, development process can be streamlined and automatically generated to attain software automation.</p><p>3) Various available tools, techniques, and metrics can be tailored to fit work element operation without having to reinvent the wheel. The available software body of knowledge (SWEBOK) can be straightforwardly adapted and exploited by software PM.</p><p>4) Training to work with systematic operating procedures and assessments in this fine-grained measurement is required. Proper planning must be carried out to set up the necessary programs for all personnel involved.</p><p>5) The advent of smart phone technology offers unlimited avenues for software research and development. As such, newer development paradigms and management techniques are called for. Undoubtedly, the success of such an undertaking can be tailored to support project estimation and streamline the development process.</p></sec><sec id="s6"><title>6. Conclusions</title><p>This exploratory research proposes a novel work element standardization to streamline software development process. The ultimate goal is to reach software automation, wherein tasks can be automated succinctly. A number of benefits can be drawn from such an elaborative attempt. They are:</p><p>1) increasing operational visibility. This is perhaps the most important aspect of software project management that PM will be able to scrutinize any potential mishaps caused by activities or actions that have gone awry in each phase and transition between phases.</p><p>2) applying appropriate individual metrics to properly measure the performance of each designated phase. This will improve the estimation result as oppose to using the “one size fits all” metric to measure all activities of all phases.</p><p>3) handling overhead/latency cost effectively. As effort spent in each task depends primarily on standard work elements involved, such handling not only leverages out the latent cost within the activity and between activity transition, but also helps PM see the hidden problems, duplication of efforts, and overhead incurred within and between activity and phase transitions.</p><p>Despite numerous incomplete work element analyses due to the lack of essential body of knowledge and experience, the precursory establishment of work element definitions will hopefully serve as the forerunner of development process standardization research endeavors. It is envisioned that more accurate estimation not only can be achieved from the proposed approach, but also can be tailored to suit other development paradigms such as AUP and XP. As such, project manager will be able to efficiently manage and produce better project output that makes up for high quality software products. Future software process automation can be realized as well.</p></sec></body><back><ref-list><title>References</title><ref id="scirp.47998-ref1"><label>1</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>PUTNAM</surname><given-names> L. </given-names></name>,<etal>et al</etal>. (<year>1978</year>)<article-title>A GENERAL EMPIRICAL SOLUTION TO THE MACRO SOFTWARE SIZING AND ESTIMATING PROBLEM</article-title><source>. IEEE TRANSACTIONS ON SOFTWARE ENGINEERING</source><volume> SE-4</volume>,<fpage> 345</fpage>-<lpage>361</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1109/TSE.1978.231521</pub-id></mixed-citation></ref><ref id="scirp.47998-ref2"><label>2</label><mixed-citation publication-type="other" xlink:type="simple">BOEHM, B. (1981) SOFTWARE ENGINEERING ECONOMICS. PRENTICE HALL PTR, UPPER SADDLE RIVER.</mixed-citation></ref><ref id="scirp.47998-ref3"><label>3</label><mixed-citation publication-type="other" xlink:type="simple">BOEHM, B., ABTS, C., BROWN, A., CHULANI, S., CLARK, B., HOROWITZ, E., MADACHY, R., REIFER, D. AND STEECE, B. (2000) SOFTWARE COST ESTIMATION WITH COCOMO II. PRENTICE HALL PTR, UPPER SADDLE RIVER.</mixed-citation></ref><ref id="scirp.47998-ref4"><label>4</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>WALSTON</surname><given-names> C.E. </given-names></name>,<name name-style="western"><surname> FELIX</surname><given-names> C.P. </given-names></name>,<etal>et al</etal>. (<year>1977</year>)<article-title>A METHOD OF PROGRAMMING MEASUREMENT AND ESTIMATION</article-title><source>. IBM SYSTEMS JOURNAL</source><volume> 16</volume>,<fpage> 54</fpage>-<lpage>73</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1147/SJ.161.0054</pub-id></mixed-citation></ref><ref id="scirp.47998-ref5"><label>5</label><mixed-citation publication-type="other" xlink:type="simple">BAILEY, J.W. AND BASILI, V.R. (1981) A META-MODEL FOR SOFTWARE DEVELOPMENT RESOURCE EXPENDITURES. PROCEEDINGS OF THE 5TH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING, 107-116.</mixed-citation></ref><ref id="scirp.47998-ref6"><label>6</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>ALBRECHT</surname><given-names> A.J. </given-names></name>,<name name-style="western"><surname> GAFFNEY JR.</surname><given-names> J.E. </given-names></name>,<etal>et al</etal>. (<year>1983</year>)<article-title>SOFTWARE FUNCTION, SOURCE LINES OF CODE, AND DEVELOPMENT EFFORT PREDICTION</article-title><source>. IEEE TRANSACTIONS ON SOFTWARE ENGINEERING</source><volume> SE-9</volume>,<fpage> 639</fpage>-<lpage>648</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1109/TSE.1983.235271</pub-id></mixed-citation></ref><ref id="scirp.47998-ref7"><label>7</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>KEMERER</surname><given-names> C.F. </given-names></name>,<etal>et al</etal>. (1987)<article-title>AN EMPIRICAL VALIDATION OF SOFTWARE COST ESTIMATION MODELS</article-title><source>. IEEE TRANSACTIONS ON SOFTWARE ENGINEERING</source><volume> 30</volume>,<fpage> 416</fpage>-<lpage>429</lpage>.<pub-id pub-id-type="doi"></pub-id></mixed-citation></ref><ref id="scirp.47998-ref8"><label>8</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>MATSON</surname><given-names> J.E.</given-names></name>,<name name-style="western"><surname> BARRETT</surname><given-names> B.E. </given-names></name>,<name name-style="western"><surname> MELLICHAMP</surname><given-names> J.M. </given-names></name>,<etal>et al</etal>. (<year>1994</year>)<article-title>SOFTWARE DEVELOPMENT COST ESTIMATION USING FUNCTION POINTS</article-title><source>. IEEE TRANSACTIONS ON SOFTWARE ENGINEERING</source><volume> 20</volume>,<fpage> 275</fpage>-<lpage>287</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1109/32.277575</pub-id></mixed-citation></ref><ref id="scirp.47998-ref9"><label>9</label><mixed-citation publication-type="other" xlink:type="simple">ALBRECHT, A. (1979) MEASURING APPLICATION DEVELOPMENT PRODUCTIVITY. I. B. M. PRESS, NEW YORK, 83-92.</mixed-citation></ref><ref id="scirp.47998-ref10"><label>10</label><mixed-citation publication-type="other" xlink:type="simple">KARNER, G. (1993) RESOURCE ESTIMATION FOR OBJECTORY PROJECTS. OBJECTIVE SYSTEMS SF AB.</mixed-citation></ref><ref id="scirp.47998-ref11"><label>11</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>SHEPPERD</surname><given-names> M. </given-names></name>,<name name-style="western"><surname> SCHOFIELD</surname><given-names> C. </given-names></name>,<etal>et al</etal>. (<year>1997</year>)<article-title>ESTIMATING SOFTWARE PROJECT EFFORT USING ANALOGIES</article-title><source>. IEEE TRANSACTIONS ON SOFTWARE ENGINEERING</source><volume> 23</volume>,<fpage> 736</fpage>-<lpage>743</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1109/32.637387</pub-id></mixed-citation></ref><ref id="scirp.47998-ref12"><label>12</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>LI</surname><given-names> J.Z.</given-names></name>,<name name-style="western"><surname> RUHE</surname><given-names> G.</given-names></name>,<name name-style="western"><surname> AL-EMRAN</surname><given-names> A. </given-names></name>,<name name-style="western"><surname> RICHTER</surname><given-names> M.M. </given-names></name>,<etal>et al</etal>. (<year>2007</year>)<article-title>A FLEXIBLE METHOD FOR SOFTWARE EFFORT ESTIMATION BY ANALOGY</article-title><source>. EMPIRICAL SOFTWARE ENGINEERING</source><volume> 12</volume>,<fpage> 65</fpage>-<lpage>106</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1007/S10664-006-7552-4</pub-id></mixed-citation></ref><ref id="scirp.47998-ref13"><label>13</label><mixed-citation publication-type="other" xlink:type="simple">NASIR, M. (2006) A SURVEY OF SOFTWARE ESTIMATION TECHNIQUES AND PROJECT PLANNING PRACTICES. PROCEEDINGS OF THE SEVENTH ACIS INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING, ARTIFICIAL INTELLIGENCE, NETWORKING, AND PARALLEL/DISTRIBUTED COMPUTING (SNPD’06), LAS VEGAS, 19-20 JUNE 2006, 305-310.</mixed-citation></ref><ref id="scirp.47998-ref14"><label>14</label><mixed-citation publication-type="other" xlink:type="simple">YANG, Y., HE, M., LI, M., WANG, Q. AND BOEHM, B. (2008) PHASE DISTRIBUTION OF SOFTWARE DEVELOPMENT EFFORT. PROCEEDINGS OF THE SECOND ACM-IEEE INTERNATIONAL SYMPOSIUM ON EMPIRICAL SOFTWARE ENGINEERING AND MEASUREMENT (ESEM’08), KAISERSLAUTERN, 9-10 OCTOBER 2008, 61-69.</mixed-citation></ref><ref id="scirp.47998-ref15"><label>15</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>JODPIMAI</surname><given-names> P.</given-names></name>,<name name-style="western"><surname> SOPHATSATHIT</surname><given-names> P. </given-names></name>,<name name-style="western"><surname> LURSINSAP</surname><given-names> C. </given-names></name>,<etal>et al</etal>. (<year>2010</year>)<article-title>ESTIMATING SOFTWARE EFFORT WITH MINIMUM FEATURES USING NEURAL FUNCTIONAL APPROXIMATION</article-title><source>. PROCEEDINGS 2010 INTERNATIONAL CONFERENCE ON COMPUTATIONAL SCIENCE AND ITS APPLICATIONS (ICCSA)</source><volume> FUKUOKA</volume>,<fpage> 23</fpage>-<lpage>26 MARCH 2010</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1109/ICCSA.2010.63</pub-id></mixed-citation></ref><ref id="scirp.47998-ref16"><label>16</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>DEJAEGER</surname><given-names> K.</given-names></name>,<name name-style="western"><surname> VERBEKE</surname><given-names> W.</given-names></name>,<name name-style="western"><surname> MARTENS</surname><given-names> D. </given-names></name>,<name name-style="western"><surname> BAESENS</surname><given-names> B. </given-names></name>,<etal>et al</etal>. (<year>2012</year>)<article-title>DATA MINING TECHNIQUES FOR SOFTWARE EFFORT ESTIMATION: A COMPARATIVE STUDY</article-title><source>. IEEE TRANSACTIONS ON SOFTWARE ENGINEERING</source><volume> 38</volume>,<fpage> 375</fpage>-<lpage>397</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1109/TSE.2011.55</pub-id></mixed-citation></ref><ref id="scirp.47998-ref17"><label>17</label><mixed-citation publication-type="other" xlink:type="simple">TAYLOR, F.W. (1911) THE PRINCIPLES OF SCIENTIFIC MANAGEMENT. HARPER &amp; BROTHERS, NEW YORK AND LONDON, LCCN11010339, OCLC233134.</mixed-citation></ref><ref id="scirp.47998-ref18"><label>18</label><mixed-citation publication-type="other" xlink:type="simple">GILBRETH, F. AND GILBRETH, C.E. (1948) CHEAPER BY THE DOZEN.</mixed-citation></ref><ref id="scirp.47998-ref19"><label>19</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>KOCAGUNELI</surname><given-names> E.</given-names></name>,<name name-style="western"><surname> MENZIES</surname><given-names> T. </given-names></name>,<name name-style="western"><surname> KEUNG</surname><given-names> J.W. </given-names></name>,<etal>et al</etal>. (<year>2012</year>)<article-title>ON THE VALUE OF ENSEMBLE EFFORT ESTIMATION</article-title><source>. IEEE TRANSACTIONS ON SOFTWARE ENGINEERING</source><volume> 38</volume>,<fpage> 1403</fpage>-<lpage>1416</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1109/TSE.2011.111</pub-id></mixed-citation></ref><ref id="scirp.47998-ref20"><label>20</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>GENCEL</surname><given-names> C. </given-names></name>,<name name-style="western"><surname> DEMIRORS</surname><given-names> O. </given-names></name>,<etal>et al</etal>. (2008)<article-title>GENCEL, C. AND DEMIRORS, O.  FUNCTIONAL SIZE MEASUREMENT REVISITED</article-title><source>. ACM TRANSACTIONS ON SOFTWARE ENGINEERING AND METHODOLOGY</source><volume> 17</volume>,<fpage> 15:1</fpage>-<lpage>15:36</lpage>.<pub-id pub-id-type="doi"></pub-id></mixed-citation></ref><ref id="scirp.47998-ref21"><label>21</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>JONES</surname><given-names> C. </given-names></name>,<etal>et al</etal>. (<year>1995</year>)<article-title>BACKFIRING: CONVERTING LINES OF CODE TO FUNCTION POINTS</article-title><source>. COMPUTER</source><volume> 28</volume>,<fpage> 87</fpage>-<lpage>88</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1109/2.471193</pub-id></mixed-citation></ref><ref id="scirp.47998-ref22"><label>22</label><mixed-citation publication-type="other" xlink:type="simple">TSUNODA, M., KAKIMOTO, T., MONDEN, A. AND MATSUMOTO, K.-I. (2011) AN EMPIRICAL EVALUATION OF OUTLIER DELETION METHODS FOR ANALOGY-BASED COST ESTIMATION. PROCEEDINGS OF THE 7TH INTERNATIONAL CONFERENCE ON PREDICTIVE MODELS IN SOFTWARE ENGINEERING, BANFF, 20-21 SEPTEMBER 2011, 17:1-17:10.</mixed-citation></ref><ref id="scirp.47998-ref23"><label>23</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>KELLER</surname><given-names> J.M.</given-names></name>,<name name-style="western"><surname> GRAY</surname><given-names> M.R. </given-names></name>,<name name-style="western"><surname> GIVENS</surname><given-names> J.A. </given-names></name>,<etal>et al</etal>. (<year>1985</year>)<article-title>A FUZZY K-NEAREST NEIGHBOR ALGORITHM</article-title><source>. IEEE TRANSACTIONS ON SYSTEMS</source><volume> MAN AND CYBERNETICS</volume>,<fpage> SMC</fpage>-<lpage>15</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1109/TSMC.1985.6313426</pub-id></mixed-citation></ref><ref id="scirp.47998-ref24"><label>24</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>KITCHENHAM</surname><given-names> B.</given-names></name>,<name name-style="western"><surname> PICKARD</surname><given-names> L.</given-names></name>,<name name-style="western"><surname> MACDONELL</surname><given-names> S. </given-names></name>,<name name-style="western"><surname> SHEPPERD</surname><given-names> M. </given-names></name>,<etal>et al</etal>. (2001)<article-title>KITCHENHAM, B., PICKARD, L., MACDONELL, S. AND SHEPPERD, M.  WHAT ACCURACY STATISTICS REALLY MEASURE [SOFTWARE ESTIMATION]</article-title><source>. PROCEEDINGS IEE—SOFTWARE</source><volume> 148</volume>,<fpage> 81</fpage>-<lpage>85</lpage>.<pub-id pub-id-type="doi"></pub-id></mixed-citation></ref><ref id="scirp.47998-ref25"><label>25</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>FOSS</surname><given-names> T.</given-names></name>,<name name-style="western"><surname> STENSRUD</surname><given-names> E.</given-names></name>,<name name-style="western"><surname> KITCHENHAM</surname><given-names> B. </given-names></name>,<name name-style="western"><surname> MYRTVEIT</surname><given-names> I. </given-names></name>,<etal>et al</etal>. (<year>2003</year>)<article-title>A SIMULATION STUDY OF THE MODEL EVALUATION CRITERION MMRE</article-title><source>. IEEE TRANSACTIONS ON SOFTWARE ENGINEERING</source><volume> 29</volume>,<fpage> 985</fpage>-<lpage>995</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1109/TSE.2003.1245300</pub-id></mixed-citation></ref><ref id="scirp.47998-ref26"><label>26</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>FRIEDMAN</surname><given-names> M. </given-names></name>,<etal>et al</etal>. (<year>1937</year>)<article-title>THE USE OF RANKS TO AVOID THE ASSUMPTION OF NORMALITY IMPLICIT IN THE ANALYSIS OF VARIANCE</article-title><source>. AMERICAN STATISTICAL ASSOCIATION</source><volume> 32</volume>,<fpage> 675</fpage>-<lpage>701</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1080/01621459.1937.10503522</pub-id></mixed-citation></ref><ref id="scirp.47998-ref27"><label>27</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>ABRAN</surname><given-names> A. </given-names></name>,<name name-style="western"><surname> ROBILLARD</surname><given-names> P. </given-names></name>,<etal>et al</etal>. (<year>1996</year>)<article-title>FUNCTION POINTS ANALYSIS: AN EMPIRICAL STUDY OF ITS MEASUREMENT PROCESSES</article-title><source>. IEEE TRANSACTIONS ON SOFTWARE ENGINEERING</source><volume> 22</volume>,<fpage> 895</fpage>-<lpage>910</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1109/32.553638</pub-id></mixed-citation></ref><ref id="scirp.47998-ref28"><label>28</label><mixed-citation publication-type="other" xlink:type="simple">REDEI, G.P. (2008) ENCYCLOPEDIA OF GENETICS, GENOMICS, PROTEOMICS AND INFORMATICS. SPRINGER, HEIDELBERG.</mixed-citation></ref><ref id="scirp.47998-ref29"><label>29</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>DE VRIES</surname><given-names> D. </given-names></name>,<name name-style="western"><surname> CH</surname><given-names>ON</given-names></name>,<name name-style="western"><surname> Y. </surname><given-names>  </given-names></name>,<etal>et al</etal>. (<year>2007</year>)<article-title>ON THE FALSE-POSITIVE RATE OF STATISTICAL EQUIPMENT COMPARISONS BASED ON THE KRUSKAL-WALLIS HSTATISTIC</article-title><source>. IEEE TRANSACTIONS ON SEMICONDUCTOR MANUFACTURING</source><volume> 20</volume>,<fpage> 286</fpage>-<lpage>292</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1109/TSM.2007.901398</pub-id></mixed-citation></ref><ref id="scirp.47998-ref30"><label>30</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>BROOKS</surname><given-names> F.P. </given-names></name>,<etal>et al</etal>. (<year>1987</year>)<article-title>NO SILVER BULLET</article-title><source>. COMPUTER</source><volume> 20</volume>,<fpage> 10</fpage>-<lpage>19</lpage>.<pub-id pub-id-type="doi">HTTP://DX.DOI.ORG/10.1109/MC.1987.1663532</pub-id></mixed-citation></ref></ref-list></back></article>