<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article  PUBLIC "-//NLM//DTD Journal Publishing DTD v3.0 20080202//EN" "http://dtd.nlm.nih.gov/publishing/3.0/journalpublishing3.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" dtd-version="3.0" xml:lang="en" article-type="research article"><front><journal-meta><journal-id journal-id-type="publisher-id">JGIS</journal-id><journal-title-group><journal-title>Journal of Geographic Information System</journal-title></journal-title-group><issn pub-type="epub">2151-1950</issn><publisher><publisher-name>Scientific Research Publishing</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="doi">10.4236/jgis.2018.103013</article-id><article-id pub-id-type="publisher-id">JGIS-85336</article-id><article-categories><subj-group subj-group-type="heading"><subject>Articles</subject></subj-group><subj-group subj-group-type="Discipline-v2"><subject>Earth&amp;Environmental Sciences</subject></subj-group></article-categories><title-group><article-title>
 
 
  Evaluation of Land Use &amp; Land Cover Change Using Multi-Temporal Landsat Imagery: A Case Study Sulaimaniyah Governorate, Iraq
 
</article-title></title-group><contrib-group><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Karwan</surname><given-names>Alkaradaghi</given-names></name><xref ref-type="aff" rid="aff1"><sup>1</sup></xref></contrib><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Salahalddin</surname><given-names>S. Ali</given-names></name><xref ref-type="aff" rid="aff2"><sup>2</sup></xref></contrib><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Nadhir</surname><given-names>Al-Ansari</given-names></name><xref ref-type="aff" rid="aff3"><sup>3</sup></xref><xref ref-type="corresp" rid="cor1"><sup>*</sup></xref></contrib><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Jan</surname><given-names>Laue</given-names></name><xref ref-type="aff" rid="aff3"><sup>3</sup></xref></contrib></contrib-group><aff id="aff3"><addr-line>Lulea University of Technology, Lulea, Sweden</addr-line></aff><aff id="aff1"><addr-line>Kurdistan Institution for Strategic Studies and Scientific Research, Sulaimaniyah, Iraq</addr-line></aff><aff id="aff2"><addr-line>Department of Geology, College of Science, Sulaimani University, Sulaimaniyah, Iraq</addr-line></aff><author-notes><corresp id="cor1">* E-mail:<email>nadhir.alansari@ltu.se(NA)</email>;</corresp></author-notes><pub-date pub-type="epub"><day>15</day><month>06</month><year>2018</year></pub-date><volume>10</volume><issue>03</issue><fpage>247</fpage><lpage>260</lpage><history><date date-type="received"><day>28,</day>	<month>May</month>	<year>2018</year></date><date date-type="rev-recd"><day>16,</day>	<month>June</month>	<year>2018</year>	</date><date date-type="accepted"><day>19,</day>	<month>June</month>	<year>2018</year></date></history><permissions><copyright-statement>&#169; Copyright  2014 by authors and Scientific Research Publishing Inc. </copyright-statement><copyright-year>2014</copyright-year><license><license-p>This work is licensed under the Creative Commons Attribution International License (CC BY). http://creativecommons.org/licenses/by/4.0/</license-p></license></permissions><abstract><p>
 
 
  Land use &amp; land cover change detection in rapid growth urbanized area have been studied by many researchers and there are many works on this topic. Commonly, settlement sprawl in area depends on many factors such as eco-nomic prosperity and population growth. Iraq is one of the countries which witnessed rapid development in the settlement area. Remote sensing and geographic information system (GIS) are analytical software technologies to evaluate this familiar worldwide phenomenon. This study illustrates settlement development in Sulaimaniyah Governorate from 2001 to 2017 using Landsat satellite imageries of different periods. All images had been classified using remote sensing software in order to proceed powerful mapping of land use classification. Maximum likelihood method is used in the accurately extracted solution information from geospatial imagery. Landsat images from the study area were categorized into four different classes. These are: forest, vegetation, soil, and settlement. Change detection analysis results illustrate that in the face of an explosive demographic shift in the settlement area where the record + 8.99 percent which is equivalent to 51.80 Km
  <sup>2</sup> over a 16-year period and settlement area increasing from 3.87 percent in 2001 to 12.86 percent in 2017. Accuracy assessment model was used to evaluate (LULC) classified images. Accuracy results show an overall accuracy of 78.83% to 90.09% from 2001 to 2017 respectively while convincing results of Kappa coefficient given between substantial and almost perfect agreements. This study will help decision-makers in urban plan for future city development.
 
</p></abstract><kwd-group><kwd>Settlement Expansion</kwd><kwd> Geographic Information System (GIS)</kwd><kwd> Land Use Land Cover (LULC)</kwd><kwd> Land Use Classification</kwd><kwd> Satellite Images</kwd><kwd> Accuracy Assessment and Change Detection</kwd></kwd-group></article-meta></front><body><sec id="s1"><title>1. Introduction</title><p>The earth’s surface passes through rapid land use &amp; land cover (LULC) changes. Land use commonly refers to the modification of earth’s surface through human activities for decent urban management due to various socioeconomic activities and natural phenomena while the land cover illustrates the physical manifestation over the ground [<xref ref-type="bibr" rid="scirp.85336-ref1">1</xref>] . There is no doubt that the growth in population, socioeconomic, political and cultural, plays an important role in land cover change, thus, resulting in an observable pattern in the LULC over the time [<xref ref-type="bibr" rid="scirp.85336-ref2">2</xref>] .</p><p>Nowadays, rapid urbanization rate in most of the developing countries is on the priority of global problems, while this phenomenon has a significant impact on future environment and urban planning processes [<xref ref-type="bibr" rid="scirp.85336-ref3">3</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref4">4</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref5">5</xref>] .</p><p>Sulaimaniyah Governorate like the rest of other cities in Kurdistan region of Iraq encountered in the last 16 years rapid expansion in the urbanized area due to high population growth rate and economic prosperity [<xref ref-type="bibr" rid="scirp.85336-ref6">6</xref>] . Remote sensing and GIS technique are the main applications that attempt to use in direct desktop mapping for temporal analysis and quantification of change in LULC with better accuracy in less time and low cost and [<xref ref-type="bibr" rid="scirp.85336-ref7">7</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref8">8</xref>] .</p><p>Landsat data enhanced imageries in the world with the ability of frequent revisit. Different satellite image sources are used in previous studies in land use classification to achieve historical trends of land cover changes [<xref ref-type="bibr" rid="scirp.85336-ref9">9</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref10">10</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref11">11</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref12">12</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref13">13</xref>] . Satellite image selection in different acquisition dates affects the results according to the vegetation life cycle [<xref ref-type="bibr" rid="scirp.85336-ref11">11</xref>] , classification of satellite image over the time for the same area effect on change in class’s area according to vegetation growth and welting on the bare soil. However, all selected satellite images in this study refer to the same season over the years to avert detecting false changes in the land cover due to vegetation phenology [<xref ref-type="bibr" rid="scirp.85336-ref9">9</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref11">11</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref14">14</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref15">15</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref16">16</xref>] .</p><p>Maximum likelihood is a supervised classification method and was used in this study to detect LULC change to present how the land use has been changed from the year 2001 to 2017. Each pixel in classified Landsat images varies according to land cover changes over the time [<xref ref-type="bibr" rid="scirp.85336-ref2">2</xref>] . Frequently, selected data during imagery classification preferred in approximately within the maximum vegetation growth during the annual season [<xref ref-type="bibr" rid="scirp.85336-ref14">14</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref16">16</xref>] . Remotely sensed classified image requires accuracy assessment to prevent certainty limits for the results [<xref ref-type="bibr" rid="scirp.85336-ref17">17</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref18">18</xref>] .</p><p>Accuracy assessment widely used in classified images by comparing the Landsat image with the results within quantitative evaluation, which is impossible to be implemented from visual evaluation [<xref ref-type="bibr" rid="scirp.85336-ref15">15</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref18">18</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref19">19</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref20">20</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref21">21</xref>] . Cohen’s kappa coefficient is used in this study as a standard measure of classification accuracy in order to take into account the correct class which specified purely by chance [<xref ref-type="bibr" rid="scirp.85336-ref22">22</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref23">23</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref24">24</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref25">25</xref>] . Accuracy assessment is commonly processed in the reference Landsat images with classified images [<xref ref-type="bibr" rid="scirp.85336-ref15">15</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref18">18</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref19">19</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref26">26</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref27">27</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref28">28</xref>] .</p><p>To achieve this objective, random sample points are located within geographic information system (GIS), accuracy assessment between each of reference Landsat images and classified images implement a confusion matrix (error matrix).</p><p>Iraq witnessed successive wars leading to economic, urban and human recession affecting all regions, including the city of Sulaimaniyah. According to food security and vulnerability in an in-depth survey of Sulaimaniyah Governorate after the events in 2003, Iraq witnessed a huge economic boom; Sulaimaniyah’s economy today relies on tourism, agriculture, factories, trade and construction development [<xref ref-type="bibr" rid="scirp.85336-ref29">29</xref>] .</p><p>There is no doubt urban that development will affect the environment and global climate by losses in vegetation biomass, deforestation and land use change from areas with high probability of urban expansion [<xref ref-type="bibr" rid="scirp.85336-ref30">30</xref>] .</p><p>This study is an attempt to assess the status of LULC change with the development of settlements in Sulaimaniyah governorate to detect the rate of land use and the changes that have occurred over the past two decades using geospatial techniques.</p></sec><sec id="s2"><title>2. Study Area</title><p>Sulaimaniyah Governorate is located at the North East of Iraq <xref ref-type="fig" rid="fig1">Figure 1</xref> Sulaimaniyah city (the capital of Sulaymaniyah Governorate) is one of the three major urban cities of Kurdistan Region. Geographically, the city is bounded by mountains from the northeast and situated in lower land over encompasses an area of around 470 km<sup>2</sup> with total population of 829,245 in early 2017 according to the data achieved from Statistical Directorate in Sulaimaniyah. The study area is characterized by a distinct continental interior climate with hot summers and cold winters of the Mediterranean type with the average annual precipitation ranging from (500 to 700) mm.</p></sec><sec id="s3"><title>3. Methodology</title><sec id="s3_1"><title>3.1. Landsat Images and Classification</title><p>Landsat Thematic Mapper (TM) images acquired from the years 2001, 2007, and 2011 was used with Landsat Operational Land Imager (OLI) at a resolution of 30m for the years 2016 and 2017 in LULC classiﬁcation. <xref ref-type="table" rid="table1">Table 1</xref> represents the available Landsat imagery with a spectral band used in this study. All required satellite imagery for the study area was downloaded from the official site of USGS earth explorer. The city plan was obtained from the Sulaimaniyah municipality corporation as a shape file of study area.</p><p>In this study, the remote sensing software ENVI 5.3 was used for image processing. During the process, the imagery geometrically corrected through pre-processing calibration, which consist of atmospheric and topographic correction</p><table-wrap id="table1" ><label><xref ref-type="table" rid="table1">Table 1</xref></label><caption><title> Available Landsat imagery with spectral band resolution for Landsat Thematic Mapper (TM) and Landsat Operational Land Imager (OLI)</title></caption><table><tbody><thead><tr><th align="center" valign="middle" >Image tape</th><th align="center" valign="middle" >Acquisition date and Time</th><th align="center" valign="middle" >Bands</th><th align="center" valign="middle" >Wavelength (micrometers)</th><th align="center" valign="middle" >Resolution (meters)</th></tr></thead><tr><td align="center" valign="middle"  rowspan="7"  >Landsat 5 Thematic Mapper (TM)</td><td align="center" valign="middle"  rowspan="7"  >20/06/2001 7:13:08 24/07/2007 7:26:42 05/07/2011 7:22:01</td><td align="center" valign="middle" >Band 1-Blue</td><td align="center" valign="middle" >0.45 - 0.52</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle" >Band 2-Green</td><td align="center" valign="middle" >0.52 - 0.60</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle" >Band 3-Red</td><td align="center" valign="middle" >0.63 - 0.69</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle" >Band 4-Near Infrared (NIR)</td><td align="center" valign="middle" >0.76 - 0.90</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle" >Band 5-Shortwave Infrared (SWIR) 1</td><td align="center" valign="middle" >1.55 - 1.75</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle" >Band 6-Thermal</td><td align="center" valign="middle" >10.40 - 12.50</td><td align="center" valign="middle" >120* (30)</td></tr><tr><td align="center" valign="middle" >Band 7-Shortwave Infrared (SWIR) 2</td><td align="center" valign="middle" >2.08 - 2.35</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle"  rowspan="11"  >Landsat 8 Operational Land Imager (OLI)</td><td align="center" valign="middle"  rowspan="11"  >18/07/2016 7:32:47 19/07/2017 7:32:33</td><td align="center" valign="middle" >Band 1-Ultra Blue (coastal/aerosol)</td><td align="center" valign="middle" >0.435 - 0.451</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle" >Band 2-Blue</td><td align="center" valign="middle" >0.452 - 0.512</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle" >Band 3-Green</td><td align="center" valign="middle" >0.533 - 0.590</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle" >Band 4-Red</td><td align="center" valign="middle" >0.636 - 0.673</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle" >Band 5-Near Infrared (NIR)</td><td align="center" valign="middle" >0.851 - 0.879</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle" >Band 6-Shortwave Infrared (SWIR) 1</td><td align="center" valign="middle" >1.566 - 1.651</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle" >Band 7-Shortwave Infrared (SWIR) 2</td><td align="center" valign="middle" >2.107 - 2.294</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle" >Band 8-Panchromatic</td><td align="center" valign="middle" >0.503 - 0.676</td><td align="center" valign="middle" >15</td></tr><tr><td align="center" valign="middle" >Band 9-Cirrus</td><td align="center" valign="middle" >1.363 - 1.384</td><td align="center" valign="middle" >30</td></tr><tr><td align="center" valign="middle" >Band 10-Thermal Infrared (TIRS) 1</td><td align="center" valign="middle" >10.60 - 11.19</td><td align="center" valign="middle" >100 * (30)</td></tr><tr><td align="center" valign="middle" >Band 11-Thermal Infrared (TIRS) 2</td><td align="center" valign="middle" >11.50 - 12.51</td><td align="center" valign="middle" >100 * (30)</td></tr></tbody></table></table-wrap><p>[<xref ref-type="bibr" rid="scirp.85336-ref31">31</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref32">32</xref>] . Image-processing techniques are statistical algorithms that change the visual appearance or geometric properties of the images, These corrections are required for selected multitemporal or multisensor Landsat images to improve visibility appearance and image quality and get more accurate results [<xref ref-type="bibr" rid="scirp.85336-ref9">9</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref31">31</xref>] .</p><p>In Landsat Operational Land Imager (OLI) digital number values were converted to the surface reflectance with the fast line-of-sight atmospheric analysis of hypercubes (FLAASH) to remove atmospheric effects and create a surface reflectance image.</p><p>The pre-processed images were classified considering a supervised classification method. In this technique, the maximum likelihood was chosen which is based on Bayesian theory in estimating parameters of a probabilistic model [<xref ref-type="bibr" rid="scirp.85336-ref33">33</xref>] . Each pixel assigned for a class according to its probability. Mean vector and covariance metrics are the key component of maximum likelihood classification that can be retrieved from training data (signatures or region of interest ROI) [<xref ref-type="bibr" rid="scirp.85336-ref7">7</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref14">14</xref>] . All data were used for this study were projected to the Universal Transverse Mercator (UTM) projection system with WGS 84 reference system. <xref ref-type="fig" rid="fig2">Figure 2</xref> shows schematic workflow steps for land use land cover change LULC and accuracy assessment.</p><p>Landsat Thematic Mapper (TM) sensor was carried on Landsat 5 and images consist of six spectral bands with a spatial resolution of 30 meters for Bands 1 - 5 and 7 and one thermal band (Band 6). Landsat Operational Land Imager (OLI) and Thermal Infrared Sensor (TIRS) images consist of nine spectral bands with a spatial resolution of 30 meters for Bands 1 to 7 and Band 9 is useful for cirrus</p><p>cloud detection. The resolution for Band 8 (panchromatic) is 15 meters while thermal bands 10 and 11 resolutions are (100 * 30) meters [<xref ref-type="bibr" rid="scirp.85336-ref34">34</xref>] .</p></sec><sec id="s3_2"><title>3.2. Create Accuracy Assessment Points</title><p>To create accuracy assessment 470 points selected in classified images with the congruent Landsat images 2001, 2007, 2011, 2016 and 2017 respectively, the classified images were compared with the reference images by means of the error matrix [<xref ref-type="bibr" rid="scirp.85336-ref19">19</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref35">35</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref27">27</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref22">22</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref36">36</xref>] .</p><p>Overall accuracy was calculated using Equation (1), while Kappa coefficient was calculated using equation (2) [<xref ref-type="bibr" rid="scirp.85336-ref18">18</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref19">19</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref20">20</xref>] .</p><p>O A = ( C / A ) ∗ 100 (1)</p><p>where;</p><p>OA: overall classification accuracy;</p><p>C: number of correct points;</p><p>A: total number of reference points.</p><p>K = N ∑ i = 1 r x i i − ∑ i = 1 r ( x i + ∗ x + 1 ) N 2 − ∑ i = 1 r ( x i + ∗ x + 1 ) (2)</p><p>where;</p><p>r: number of categories (Classes);</p><p>N: total number of point (observations pixels);</p><p>x<sub>ii</sub>: number of points on row i and column i;</p><p>x<sub>i</sub><sub>+</sub>: marginal total of row i (from overall confusion/error matrix);</p><p>x<sub>+i</sub>: marginal total of column (from overall confusion/error matrix).</p></sec><sec id="s3_3"><title>3.3. Accuracy Assessment</title><p>Cohen’s Kappa coefficient (K) is a strong statistical method used to measure the agreement between predicted and observed categorizations of a dataset. This statistic method is useful, especially in predictive accuracy of classification [<xref ref-type="bibr" rid="scirp.85336-ref36">36</xref>] . Random sample points were used in post-classification for accuracy assessment, 470 random sample points are located by Arc-map software then all points labelled under their category type in the classification. The reference points were compared with the classification results at the same locations. The output point shape file or feature class that contains the random 470 points was computed in the confusion matrix to ensure that each point had a valid class asset values for the classified and ground-truth point [<xref ref-type="bibr" rid="scirp.85336-ref14">14</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref20">20</xref>] .</p><p>The producer accuracy and user accuracy for each class calculated from the matrix as well as an overall kappa index of agreement. When Kappa coefficient is equal to (1.00) it means perfect agreement and if the value close to (0.00) it means poor agreement. <xref ref-type="table" rid="table2">Table 2</xref> represents the rating criteria of Kappa statistics [<xref ref-type="bibr" rid="scirp.85336-ref33">33</xref>] [<xref ref-type="bibr" rid="scirp.85336-ref37">37</xref>] . User accuracy represents the probability that a pixel classified into a given category actually which represents that category on the ground [<xref ref-type="bibr" rid="scirp.85336-ref38">38</xref>] .</p><table-wrap id="table2" ><label><xref ref-type="table" rid="table2">Table 2</xref></label><caption><title> Fleiss kappa interpretation (Rating criteria of Kappa statistics)</title></caption><table><tbody><thead><tr><th align="center" valign="middle" >Kappa statistics</th><th align="center" valign="middle" >Interpretation</th></tr></thead><tr><td align="center" valign="middle" >&lt;0.00</td><td align="center" valign="middle" >Poor agreement</td></tr><tr><td align="center" valign="middle" >0.01 - 0.20</td><td align="center" valign="middle" >Slight agreement</td></tr><tr><td align="center" valign="middle" >0.21 - 0.40</td><td align="center" valign="middle" >Fair agreement</td></tr><tr><td align="center" valign="middle" >0.41 - 0.60</td><td align="center" valign="middle" >Moderate agreement</td></tr><tr><td align="center" valign="middle" >0.61 - 0.80</td><td align="center" valign="middle" >Substantial agreement</td></tr><tr><td align="center" valign="middle" >0.81 - 1.00</td><td align="center" valign="middle" >Almost perfect agreement</td></tr></tbody></table></table-wrap></sec></sec><sec id="s4"><title>4. Results and Discussion</title><sec id="s4_1"><title>4.1. Classification Results</title><p>Four land cover classes identified from maximum likelihood classification. These classes are forest, Settlement area, soil, vegetation and as shown in <xref ref-type="fig" rid="fig3">Figure 3</xref>.</p><p>Classification results gave the LULC raster image of the selected area, it is possible to calculate the area for each class separately according to pixel geometry and statistical distribution of pixels through the study area. The change detection in different LULC categories in the Sulaimaniyah Governorate from 2001 to 2017 as illustrated in <xref ref-type="table" rid="table3">Table 3</xref>.</p></sec><sec id="s4_2"><title>4.2. Accuracy Assessment Results</title><p>The accuracy report from the matrix of all classified images is tabulated in <xref ref-type="table" rid="table4">Table 4</xref>. An overall accuracy and kappa coefficient was obtained for the years 2001, 2001, 2011, 2016 and 2017 respectively. Various measures of accuracy assessment such as producer accuracy, user accuracy, overall accuracy and Kappa coefficient were carried out and are represented in <xref ref-type="table" rid="table2">Table 2</xref>. Kappa coefficients generally rated as substantial agreement for all classified.</p><p>Producers Accuracy (Omission Errors) The results from dividing the number of correctly classified pixels in each category on the major diagonal matrix by the number of reference pixels “known” to be of that category (the column total), This value represents how well reference pixels of the ground cover type are classified, While users Accuracy (Commission Error) computed by dividing the number of correctly classified pixels in each category by the total number of pixels that were classified in that category (the row total), This value represents the probability that a pixel classified into a given category actually represents that category on the ground.</p><p>While Kappa reflects the difference between actual agreement and the agreement expected by chance Kappa results means there is 80% better agreement than by chance alone, Different values of kappa and overall accuracy from 2001 to 2017 return to the choice of reference data, interaction between sensor and desired classification scheme and Error matrix is foundation of accuracy assessment.</p><table-wrap id="table3" ><label><xref ref-type="table" rid="table3">Table 3</xref></label><caption><title> Change detection in the four classes identified in the Sulaimaniyah Governorate from 2001 to 2017</title></caption><table><tbody><thead><tr><th align="center" valign="middle" >Classified area</th><th align="center" valign="middle" >% Area 2001</th><th align="center" valign="middle" >% Area 2007</th><th align="center" valign="middle" >% Area 2011</th><th align="center" valign="middle" >% Area 2016</th><th align="center" valign="middle" >% Area 2017</th><th align="center" valign="middle" >Change detection in%</th><th align="center" valign="middle" >Change detection in Km<sup>2</sup></th></tr></thead><tr><td align="center" valign="middle" >Settlement area</td><td align="center" valign="middle" >3.87</td><td align="center" valign="middle" >8.74</td><td align="center" valign="middle" >9.29</td><td align="center" valign="middle" >12.86</td><td align="center" valign="middle" >12.86</td><td align="center" valign="middle" >+8.99</td><td align="center" valign="middle" >+51.80</td></tr><tr><td align="center" valign="middle" >Forest</td><td align="center" valign="middle" >1.62</td><td align="center" valign="middle" >1.02</td><td align="center" valign="middle" >1.89</td><td align="center" valign="middle" >0.84</td><td align="center" valign="middle" >0.84</td><td align="center" valign="middle" >−0.78</td><td align="center" valign="middle" >−4.51</td></tr><tr><td align="center" valign="middle" >Soil</td><td align="center" valign="middle" >67.56</td><td align="center" valign="middle" >48.55</td><td align="center" valign="middle" >29.04</td><td align="center" valign="middle" >22.79</td><td align="center" valign="middle" >22.79</td><td align="center" valign="middle" >−44.77</td><td align="center" valign="middle" >−257.87</td></tr><tr><td align="center" valign="middle" >Vegetation</td><td align="center" valign="middle" >26.29</td><td align="center" valign="middle" >41.69</td><td align="center" valign="middle" >59.65</td><td align="center" valign="middle" >63.45</td><td align="center" valign="middle" >63.45</td><td align="center" valign="middle" >+37.16</td><td align="center" valign="middle" >+214.03</td></tr></tbody></table></table-wrap><table-wrap id="table4" ><label><xref ref-type="table" rid="table4">Table 4</xref></label><caption><title> Accuracy assessment of the classified images</title></caption><table><tbody><thead><tr><th align="center" valign="middle"  colspan="2"   rowspan="2"  >Classification years</th><th align="center" valign="middle"  colspan="4"  >Land use Land cover classes</th><th align="center" valign="middle"  colspan="2"  ></th></tr></thead><tr><td align="center" valign="middle" >Settlement</td><td align="center" valign="middle" >Forest</td><td align="center" valign="middle" >Bare soil</td><td align="center" valign="middle" >Vegetation</td><td align="center" valign="middle" >Overall Accuracy% (OA)</td><td align="center" valign="middle" >Kappa coefficient (K)</td></tr><tr><td align="center" valign="middle"  rowspan="2"  >2001</td><td align="center" valign="middle" >PA</td><td align="center" valign="middle" >83.3</td><td align="center" valign="middle" >75</td><td align="center" valign="middle" >77.8</td><td align="center" valign="middle" >92.8</td><td align="center" valign="middle"  rowspan="2"  >78.83</td><td align="center" valign="middle"  rowspan="2"  >0.72</td></tr><tr><td align="center" valign="middle" >UA</td><td align="center" valign="middle" >100</td><td align="center" valign="middle" >85.7</td><td align="center" valign="middle" >80.8</td><td align="center" valign="middle" >69</td></tr><tr><td align="center" valign="middle"  rowspan="2"  >2007</td><td align="center" valign="middle" >PA</td><td align="center" valign="middle" >86</td><td align="center" valign="middle" >75</td><td align="center" valign="middle" >80</td><td align="center" valign="middle" >95</td><td align="center" valign="middle"  rowspan="2"  >81.63</td><td align="center" valign="middle"  rowspan="2"  >0.76</td></tr><tr><td align="center" valign="middle" >UA</td><td align="center" valign="middle" >99.1</td><td align="center" valign="middle" >75</td><td align="center" valign="middle" >88.9</td><td align="center" valign="middle" >70</td></tr><tr><td align="center" valign="middle"  rowspan="2"  >2011</td><td align="center" valign="middle" >PA</td><td align="center" valign="middle" >90.9</td><td align="center" valign="middle" >75</td><td align="center" valign="middle" >80</td><td align="center" valign="middle" >95</td><td align="center" valign="middle"  rowspan="2"  >83.72</td><td align="center" valign="middle"  rowspan="2"  >0.78</td></tr><tr><td align="center" valign="middle" >UA</td><td align="center" valign="middle" >100</td><td align="center" valign="middle" >85.7</td><td align="center" valign="middle" >85.7</td><td align="center" valign="middle" >70</td></tr><tr><td align="center" valign="middle"  rowspan="2"  >2016</td><td align="center" valign="middle" >PA</td><td align="center" valign="middle" >90.5</td><td align="center" valign="middle" >66.7</td><td align="center" valign="middle" >83.3</td><td align="center" valign="middle" >95.2</td><td align="center" valign="middle"  rowspan="2"  >84.68</td><td align="center" valign="middle"  rowspan="2"  >0.79</td></tr><tr><td align="center" valign="middle" >UA</td><td align="center" valign="middle" >95.5</td><td align="center" valign="middle" >85.7</td><td align="center" valign="middle" >89</td><td align="center" valign="middle" >74.1</td></tr><tr><td align="center" valign="middle"  rowspan="2"  >2017</td><td align="center" valign="middle" >PA</td><td align="center" valign="middle" >100</td><td align="center" valign="middle" >87.5</td><td align="center" valign="middle" >83.3</td><td align="center" valign="middle" >100</td><td align="center" valign="middle"  rowspan="2"  >90.09</td><td align="center" valign="middle"  rowspan="2"  >0.87</td></tr><tr><td align="center" valign="middle" >UA</td><td align="center" valign="middle" >91.7</td><td align="center" valign="middle" >100</td><td align="center" valign="middle" >96</td><td align="center" valign="middle" >76.9</td></tr></tbody></table></table-wrap><p>PA: Producer Accuracy, UA: User Accuracy; OA: Overall Accuracy, K: Kappa coefficient.</p></sec><sec id="s4_3"><title>4.3. Change Detection Analysis</title><p>During the 16 years period (2001 to 2017), settlement area increased by around 9%, which represent 51.80 km<sup>2</sup>. On the other hand, bare soil decreased by 44.77% in area. Forest recorded minimum change according to deforestation in land use in the study area, while vegetation increased by over 37% in the area as represented in <xref ref-type="fig" rid="fig4">Figure 4</xref>.</p><p>Increased vegetation area refers to two factors which are settlement growth at the same duration and the difference in Landsat acquisition date, although all available images were selected from the same season while there is a difference in duration for first and last Landsat Image by around one month duration which is sufficient for vegetation growth.</p><p>Maximum likelihood classification result gave the raster image, Settlement area calculated according to the pixel geometry from <xref ref-type="table" rid="table1">Table 1</xref> and statistical pixels distribution in attribute table through the study area.</p><p>Change in settlement area between the years 2001 to 2017 increased from 3.87% area in 2001 to rich 12.87% area in 2017 as shown in <xref ref-type="fig" rid="fig5">Figure 5</xref>.</p><p>Two rapid expansions in settlement area noted during evaluation of classified images between 2001 to 2017 and 2011 to 2016 respectively which are mainly caused by economic prosperity and population growth.</p><p>Change in settlement area returns to modification of land cover due to human processes while the change in land cover is the natural occurrence on the earth surface like Forest decreased in area from 2001 to 2017 according to Sulaimaniyah government plan to increase green area around the city but the consequences forest fire reduced the rate of forest development, as shown in Land cover variation tendency in <xref ref-type="fig" rid="fig6">Figure 6</xref>.</p><p>Bare soil decreased in area from 2001 to 2017 according to modification of land cover and Settlement expansion.</p><p>Vegetation and grassland development in the area after 2001 refer to economic prosperity and Human needs through population growth.</p></sec></sec><sec id="s5"><title>5. Conclusions</title><p>In situations of rapid land use change, the classified images provide detail information to understand the land use and land cover of the study area. Settlement area in Sulaimaniyah Governorate expended significantly from the years 2001 to 2017. This expansion occurred comes at the expense of the bare soil and forest shortage in area by 257.87 and 4.51 Km<sup>2</sup> respectively. Thus, settlement recorded sprawl in the area according to the land use in bare soil and deforestation.</p><p>All satellite images were selected in this study referring to the same season over the years to produce series land use classification maps while there is a time deference in acquisition date in the first image in 2001 by 1 month earlier. Thus, vegetation appears with less area than soil from the begging; this causes false changes in the land cover due to phenology. Increased vegetation area mainly refers to two factors which are settlement development due to population growth and deferent in duration of Landsat acquisition date. However, all available images are from the same season.</p><p>The study area is affected by deforestation in 4.51 km<sup>2</sup> over total area of 470 km<sup>2</sup> between the years 2011 to 2017; this value impacts on future environment and urban management</p><p>This study will help planners and decision-makers for future development of the city.</p><p>Classification results have shown that Maximum likelihood classification is the robust technique and there are fewer chances of misclassification.</p><p>Accuracy assessment model is the significant method for validation in the land use classification. In addition, number of samples and distribution of random points are based on the density of each category in classified images. The Kappa coefficient indicates that the classification method is very well captured for the land use and land cover of the interest study area.</p><p>The main influence on land cover changes comes through the variety of interlinked factors which are the population growth and economic prosperity.</p></sec><sec id="s6"><title>Cite this paper</title><p>Alkaradaghi, K., Ali, S.S., Al-Ansari, N. and Laue, J. (2018) Evaluation of Land Use &amp; Land Cover Change Using Multi-Temporal Landsat Imagery: A Case Study Sulaimaniyah Governorate, Iraq. Journal of Geographic Information System, 10, 247-260. https://doi.org/10.4236/jgis.2018.103013</p></sec></body><back><ref-list><title>References</title><ref id="scirp.85336-ref1"><label>1</label><mixed-citation publication-type="other" xlink:type="simple">Comber, A.J. (2008) Land Use or Land Cover? Journal of Land Use Science, 3, 199-201. &lt;br&gt;https://doi.org/10.1080/17474230802465140</mixed-citation></ref><ref id="scirp.85336-ref2"><label>2</label><mixed-citation publication-type="other" xlink:type="simple">Tilahun, A. (2015) Accuracy Assessment of Land Use Land Cover Classification Using Google Earth. American Journal of Environmental Protection, 4, 193-198.  
&lt;br&gt;https://doi.org/10.11648/j.ajep.20150404.14</mixed-citation></ref><ref id="scirp.85336-ref3"><label>3</label><mixed-citation publication-type="book" xlink:type="simple">Clark, B.J.F. and Pellikka, P.K.E. (2009) Landscape Analysis Using Multiscale Segmentation and Object Orientated Classification. In: Roder and Hill, Eds., Recent Advances in Remote Sensing and Geoinformation Processing for Land Degradation Assessment, Taylor &amp; Francis Group, London, 323-342.</mixed-citation></ref><ref id="scirp.85336-ref4"><label>4</label><mixed-citation publication-type="other" xlink:type="simple">Rathcke, B. and Lacey, E.P. (1985) Phenological Patterns of Terrestrial Plants. Annual Review of Ecology and Systematics, 16, 179-214.  
&lt;br&gt;https://doi.org/10.1146/annurev.es.16.110185.001143</mixed-citation></ref><ref id="scirp.85336-ref5"><label>5</label><mixed-citation publication-type="other" xlink:type="simple">Song, C., Woodcock, C.E., Seto, K.C., Lenney, M.P. and Macomber, S.A. (2001) Classification and Change Detection Using Landsat TM Data: When and How to Correct Atmospheric Effects? Remote Sensing of Environment, 75, 230-244.  
&lt;br&gt;https://doi.org/10.1016/S0034-4257(00)00169-3</mixed-citation></ref><ref id="scirp.85336-ref6"><label>6</label><mixed-citation publication-type="other" xlink:type="simple">Wu, C., et al. (2017) Land Surface Phenology Derived from Normalized Difference Vegetation Index (NDVI) at Global FLUXNET Sites. Agricultural and Forest Meteorology, 233, 171-182. &lt;br&gt;https://doi.org/10.1016/j.agrformet.2016.11.193</mixed-citation></ref><ref id="scirp.85336-ref7"><label>7</label><mixed-citation publication-type="other" xlink:type="simple">Alexakis, D.D., Agapiou, A., Hadjimitsis, D.G. and Retalis, A. (2012) Optimizing Statistical Classification Accuracy of Satellite Remotely Sensed Imagery for Supporting Fast Flood Hydrological Analysis. Acta Geophysica, 60, 959-984.  
&lt;br&gt;https://doi.org/10.2478/s11600-012-0025-9</mixed-citation></ref><ref id="scirp.85336-ref8"><label>8</label><mixed-citation publication-type="other" xlink:type="simple">Kim, D.H., Narashiman, R., Sexton, J.O., Huang, C. and Towns-hend, J.R. (2011) Methodology to Select Phenologically Suitable Landsat Scenes for Forest Change Detection. International Geoscience and Remote Sensing Symposium (IGARSS), Vancouver, BC, 24-29 July 2011, 2613-2616.  
&lt;br&gt;https://doi.org/10.1109/IGARSS.2011.6049738</mixed-citation></ref><ref id="scirp.85336-ref9"><label>9</label><mixed-citation publication-type="other" xlink:type="simple">McIver, D.K. and Friedl, M.A. (2002) Using Prior Probabilities in Decision-Tree Classification of Remotely Sensed Data. Remote Sensing of Environment, 81, 253-261. &lt;br&gt;https://doi.org/10.1016/S0034-4257(02)00003-2</mixed-citation></ref><ref id="scirp.85336-ref10"><label>10</label><mixed-citation publication-type="other" xlink:type="simple">Hassen, E.E. and Assen, M. (2018) Land Use/Cover Dynamics and Its Drivers in Gelda Catchment, Lake Tana Watershed, Ethiopia. Environmental Systems Research, 6, 4. &lt;br&gt;https://doi.org/10.1186/s40068-017-0081-x</mixed-citation></ref><ref id="scirp.85336-ref11"><label>11</label><mixed-citation publication-type="other" xlink:type="simple">Rosenfield, G.H. and Fitzpatrick-Lins, K. (1986) A Coefficient of Agreement as a Measure of Thematic Classification Accuracy. Photogrammetric Engineering and Remote Sensing, 52, 223-227.</mixed-citation></ref><ref id="scirp.85336-ref12"><label>12</label><mixed-citation publication-type="other" xlink:type="simple">Foody, G.M. (2002) Status of Land Cover Classification Accuracy Assessment. Remote Sensing of Environment, 80, 185-201.  
&lt;br&gt;https://doi.org/10.1016/S0034-4257(01)00295-4</mixed-citation></ref><ref id="scirp.85336-ref13"><label>13</label><mixed-citation publication-type="other" xlink:type="simple">Weil, G., Lensky, I.M., Resheff, Y.S. and Levin, N. (2017) Optimizing the Timing of Unmanned Aerial Vehicle Image Acquisition for Applied Mapping of Woody Vegetation Species Using Feature Selection. Remote Sensing, 9, 1130.</mixed-citation></ref><ref id="scirp.85336-ref14"><label>14</label><mixed-citation publication-type="other" xlink:type="simple">Mosammam, H.M., Nia, J.T., Khani, H., Teymouri, A. and Kazemi, M. (2017) Monitoring Land Use Change and Measuring Urban Sprawl Based on Its Spatial Forms: The Case of Qom City. The Egyptian Journal of Remote Sensing and Space Science, 20, 103-116.</mixed-citation></ref><ref id="scirp.85336-ref15"><label>15</label><mixed-citation publication-type="other" xlink:type="simple">Barsi, J.A., Lee, K., Kvaran, G., Markham, B.L. and Pedelty, J.A. (2014) The Spectral Response of the Landsat-8 Operational Land Imager. Remote Sensing, 6, 10232-10251. &lt;br&gt;https://doi.org/10.3390/rs61010232</mixed-citation></ref><ref id="scirp.85336-ref16"><label>16</label><mixed-citation publication-type="other" xlink:type="simple">Morisette, J.T., et al. (2009) Tracking the Rhythm of the Seasons in the Face of Global Change: Phenological Research in the 21st Century. Frontiers in Ecology and the Environment, 7, 253-260. &lt;br&gt;https://doi.org/10.1890/070217</mixed-citation></ref><ref id="scirp.85336-ref17"><label>17</label><mixed-citation publication-type="other" xlink:type="simple">Tilahun, A. (2015) Accuracy Assessment of Land Use Land Cover Classification Using Google Earth. American Journal of Environmental Protection, 4, 193.  
&lt;br&gt;https://doi.org/10.11648/j.ajep.20150404.14</mixed-citation></ref><ref id="scirp.85336-ref18"><label>18</label><mixed-citation publication-type="other" xlink:type="simple">Congalton, R.G. and Green, K. (2009) Assessing the Accuracy of Remotely Sensed Data: Principles and Practices. 2nd Edition, CRC Press, Boca Raton.</mixed-citation></ref><ref id="scirp.85336-ref19"><label>19</label><mixed-citation publication-type="other" xlink:type="simple">Zhu, X. and Liu, D. (2014) Accurate Mapping of Forest Types Using Dense Seasonal Landsat Time-Series. ISPRS Journal of Photogrammetry and Remote Sensing, 96, 1-11. &lt;br&gt;https://doi.org/10.1016/j.isprsjprs.2014.06.012</mixed-citation></ref><ref id="scirp.85336-ref20"><label>20</label><mixed-citation publication-type="other" xlink:type="simple">Gashaw, T., Tulu, T., Argaw, M. and Worqlul, A.W. (2017) Evaluation and Prediction of Land Use/Land Cover Changes in the Andassa Watershed, Blue Nile Basin, Ethiopia. Environmental Systems Research, 6, 17.  
&lt;br&gt;https://doi.org/10.1186/s40068-017-0094-5</mixed-citation></ref><ref id="scirp.85336-ref21"><label>21</label><mixed-citation publication-type="other" xlink:type="simple">Congalton, R.G. (2005) Thematic and Positional Accuracy Assessment of Digital Remotely Sensed Data. Proceedings of the 7th Annual Forest Inventory and Analysis Symposium, Portland, 3-6 October 2005, 149-154.</mixed-citation></ref><ref id="scirp.85336-ref22"><label>22</label><mixed-citation publication-type="other" xlink:type="simple">Gashaw, T., Bantider, A. and Mahari, A. (2014) Evaluations of Land Use/Land Cover Changes and Land Degradation in Dera District, Ethiopia: GIS and Remote Sensing Based Analysis. International Journal of Scientific Research in Environmental Sciences, 2, 199-208.</mixed-citation></ref><ref id="scirp.85336-ref23"><label>23</label><mixed-citation publication-type="other" xlink:type="simple">Mosammam, H.M., Nia, J.T., Khani, H., Teymouri, A. and Kazemi, M. (2017) Monitoring Land Use Change and Measuring Urban Sprawl Based on Its Spatial Forms: The Case of Qom City. The Egyptian Journal of Remote Sensing and Space Sciences, 20, 103-116. &lt;br&gt;https://doi.org/10.1016/j.ejrs.2016.08.002</mixed-citation></ref><ref id="scirp.85336-ref24"><label>24</label><mixed-citation publication-type="other" xlink:type="simple">Fleiss, J.L. (1971) Measuring Nominal Scale Agreement among Many Raters. Psychological Bulletin, 76, 378-382. &lt;br&gt;https://doi.org/10.1037/h0031619</mixed-citation></ref><ref id="scirp.85336-ref25"><label>25</label><mixed-citation publication-type="other" xlink:type="simple">Landis, J.R. and Koch, G.G. (1977) The Measurement of Observer Agreement for Categorical Data. Biometrics, 33, 159. &lt;br&gt;https://doi.org/10.2307/2529310</mixed-citation></ref><ref id="scirp.85336-ref26"><label>26</label><mixed-citation publication-type="other" xlink:type="simple">Foody, G.M. (2002) Status of Land Cover Classification Accuracy Assessment. Remote Sensing of Environment, 80, 185-201.  
&lt;br&gt;https://doi.org/10.1016/S0034-4257(01)00295-4</mixed-citation></ref><ref id="scirp.85336-ref27"><label>27</label><mixed-citation publication-type="other" xlink:type="simple">Missions, L. (2013) Landsat 8 Data Product Information. 7-9.</mixed-citation></ref><ref id="scirp.85336-ref28"><label>28</label><mixed-citation publication-type="book" xlink:type="simple">Clark, B.J.F. and Pellikka, P.K.E. (2009) Landscape Analysis Using Multiscale Segmentation and Object-Orientated Classification. In: Roder, A. and Hill, J., Eds., Recent Advances in Remote Sensing and Geoinformation Processing for Land Degradation Assessment, CRC Press, Boca Raton, 323-342.</mixed-citation></ref><ref id="scirp.85336-ref29"><label>29</label><mixed-citation publication-type="other" xlink:type="simple">Planning, M.O.F., Cooperation, D. and Health, M.O.F. (2008) Comprehensive Food Security and Vulnerability Analysis in Iraq.</mixed-citation></ref><ref id="scirp.85336-ref30"><label>30</label><mixed-citation publication-type="other" xlink:type="simple">Seto, K.C. and Christensen, P. (2013) Remote Sensing Science to Inform Urban Climate Change Mitigation Strategies. Urban Climate, 3, 1-6.  
&lt;br&gt;https://doi.org/10.1016/j.uclim.2013.03.001</mixed-citation></ref><ref id="scirp.85336-ref31"><label>31</label><mixed-citation publication-type="other" xlink:type="simple">Song, C., Woodcock, C.E., Seto, K.C., Lenney, M.P. and Macomber, S.A. (2001) Classification and Change Detection Using Landsat TM Data: When and How to Correct Atmospheric Effects? Remote Sensing of Environment, 75, 230-244.  
&lt;br&gt;https://doi.org/10.1016/S0034-4257(00)00169-3</mixed-citation></ref><ref id="scirp.85336-ref32"><label>32</label><mixed-citation publication-type="other" xlink:type="simple">Schulz, J.J., Cayuela, L., Echeverria, C., Salas, J. and Rey Benayas, J.M. (2010) Monitoring Land Cover Change of the Dryland Forest Landscape of Central Chile (1975-2008). Applied Geography, 30, 436-447.  
&lt;br&gt;https://doi.org/10.1016/j.apgeog.2009.12.003</mixed-citation></ref><ref id="scirp.85336-ref33"><label>33</label><mixed-citation publication-type="other" xlink:type="simple">Rosenfield, G.H. and Fitzpatrick-Lins, K. (1986) A Coefficient of Agreement as a Measure of Thematic Classification Accuracy. Photogrammetric Engineering and Remote Sensing, 52, 223-227.</mixed-citation></ref><ref id="scirp.85336-ref34"><label>34</label><mixed-citation publication-type="other" xlink:type="simple">Barsi, J.A., Lee, K., Kvaran, G., Markham, B.L. and Pedelty, J.A. (2014) The Spectral Response of the Land-sat-8 Operational Land Imager. Remote Sensing, 6, 10232-10251. &lt;br&gt;https://doi.org/10.3390/rs61010232</mixed-citation></ref><ref id="scirp.85336-ref35"><label>35</label><mixed-citation publication-type="other" xlink:type="simple">McIver, D.K. and Friedl, M.A. (2002) Using Prior Probabilities in Decision-Tree Classification of Remotely Sensed Data. Remote Sensing of Environment, 81, 253-261. &lt;br&gt;https://doi.org/10.1016/S0034-4257(02)00003-2</mixed-citation></ref><ref id="scirp.85336-ref36"><label>36</label><mixed-citation publication-type="other" xlink:type="simple">Pontius, R.G. and Millones, M. (2011) Death to Kappa: Birth of Quantity Disagreement and Allocation Disagreement for Accuracy Assessment. International Journal of Remote Sensing, 32, 4407-4429. &lt;br&gt;https://doi.org/10.1080/01431161.2011.552923</mixed-citation></ref><ref id="scirp.85336-ref37"><label>37</label><mixed-citation publication-type="other" xlink:type="simple">Alexakis, D.D., Agapiou, A., Hadjimitsis, D.G. and Retalis, A. (2012) Optimizing Statistical Classification Accuracy of Satellite Remotely Sensed Imagery for Supporting Fast Flood Hydrological Analysis. Acta Geophysica, 60, 959-984.  
&lt;br&gt;https://doi.org/10.2478/s11600-012-0025-9</mixed-citation></ref><ref id="scirp.85336-ref38"><label>38</label><mixed-citation publication-type="other" xlink:type="simple">Rientjes, T.H.M., Perera, B.U.J., Haile, A.T., Reggiani, P. and Muthuwatta, L.P. (2011) Regionalisation for Lake Level Simulation—The Case of Lake Tana in the Upper Blue Nile, Ethiopia. Hydrology and Earth System Sciences, 15, 1167-1183.  
&lt;br&gt;https://doi.org/10.5194/hess-15-1167-2011</mixed-citation></ref></ref-list></back></article>