The Analytic Hierarchy Process (AHP) developed by Satty to provide a framework for dealing with decision-making problems and complex problems [3]. It is a widely used method by decision makers and researchers for selection or ranking of alternatives in a wide range of problem types [4]. The method is an effective and easy-to-understand tool used in complex problems both with qualitative and quantitative criteria. Combining with other linear programming techniques such as Data Envelopment Analysis, SWOT analysis is another advantage of the method [5].

The basic principles of AHP can be summarized as defining and determining the problem; decomposing the problem in a hierarchy from top through the intermediate levels; constructing a set of pair-wise comparison matrices; testing the consistency index; synthesizing the hierarchy to find out the ranks of the alternatives [6]. AHP makes use of pair-wise comparisons to simplify the judgment process with 1 - 9 ratio scale in Table 1 [7].

In each level of the pairwise comparison matrix, created by assigning the relative weights of the elements using 1 - 9 scale, the elements are compared pairwise with respect to their importance to the element in the higher level. The pairwise comparison matrixes are formed as given in Equation (1). The score of a_ij in the pairwise comparison matrix symbolizes the relative importance of the element on the row (i) over the element on column (j).

A = [ a i j ] = [ 1 a 12 … a 1 n 1 / a 12 1 … a 2 n ⋮ ⋮ 1 ⋮ 1 / a 1 n 1 / a 2 n … 1 ] (1)

Such that [ a i j ] > 0.

After all pairwise comparisons are completed then the problem turns into general process to calculating the largest eigenvalue corresponding to eigenvector to assess the Consistency Index (CI). A is the matrix, w is the eigenvector and λ_max is the largest eigenvalue of the matrix A.

A w = λ max w (2)

C I = λ max − n / n − 1 (3)

C R = C I / C R (4)

When CI is divided by the Random Consistency Index (RI) the final value must be less than 0.10 [7]. RI provided for different matrix orders in Table 2 [3].

There are two types of comparison in the AHP, relative measurement and absolute measurement. In relative measurement, each alternative is compared with other alternatives. However, in absolute measurement each alternative is compared with one ideal alternative [8]. The absolute measurement method, also called ratings, is convenient in a large number of alternatives and the number of pair wised comparisons is much less than the relative measurement [9]. Another advantage of the absolute measurement is adding or removing alternatives does not affect the old rankings [10]. Also the number of alternatives and criteria does not cause a consistency problem in the ratings model [11].

In absolute measurement, the hierarchy is developed as the same way in relative measurement. The goal of the problem exists at the top and it is followed by criteria and sub criteria hierarchically. The ratings are placed below the sub criterion. Ratings definitions may be qualitative and quantitative. For instance, a sub criterion can be decomposed into qualitative definitions such as hot, cold, low, high, far and near or can be defined quantitatively as certain numerical values or ranges as well. Each sub criterion could have different types and numbers of rating definitions. These definitions vary according to the problem studied. The definitions of the ratings must be carefully created in order to the experiences, the size of the alternatives and the knowledge of the experts to compare intensities. Alternatives do not need to be shown in the hierarchy since alternatives are not compared with each other.

In order to evaluate urban sustainability of cities a hierarchy three has been created with Super Decisions 2.8 software to apply the AHP ratings method. The hierarchy is presented in Figure 1.

The goal exists at the first level of the hierarchy that is comparing the sustainability of cities in Turkey. Since sustainability is considered with three components economic, environmental and social sustainability are presented at the second level of the hierarchy as criteria. At the third level of the hierarchy, five sub criteria under each criterion are placed.

The indicators under each criterion were determined by a three-stage elimination process. In the first part of the elimination, the literature was searched. The most frequently used indicators were listed among the urban sustainability studies and sustainable development indicator sets of international institutions. Thus, the indicators to represent the sustainability were identified. In the second

step, indicators were eliminated by statistical availability, easy to understanding, measurability and reliability principles by the experts. In the third step, the indicators took their final form by taking into consideration the basic principles of the AHP method.

In the AHP method the indicators in the hierarchy should not interact with each other [3]. Another principle is the “magic number 7: plus minus 2” rule. A rational pairwise comparison requires an effective number of criteria [12]. Therefore, five indicators under each component of sustainability were selected and pairwise compared by the experts. Selected sub-criteria were listed in Table 3. In the analysis the period before the lawful definition change of metropolitan cities were taken into consideration.

Labor force participation rate, GDP per capita, investments made with incentive certificates, population per bank branch and non-performing loans of banks were chosen as sub criteria of economic sustainability component. Labor force participation rate statistics, refers to people available for work as a percentage of total population aged 15 to 64, and were obtained from Turkish Statistical Institution (Turkstat). The latest announced income statistics for the provinces were published by Turkstat in 2001. In the following years, income statistics were published only at regional and national level. Therefore, GDP per capita statistics were calculated according to the latest available data. To do so, first the share of national level of GDP for each province was calculated according to the last statistics on the provincial basis. Then the data at the regional level were demoted to the provincial level. In the last stage, per capita values were calculated. The

investment incentive certificates data were obtained from the Ministry of Economy to represent the investment capacity of the urban areas. Population per bank branch and non-performing loans of banks statistics, which represent the financial potential of the cities, were taken from Banking Regulation and Supervision Agency.

Air pollution, wastes, forests and water represent main themes of the environmental sustainability. Five indicators selected by the experts which are particulate equivalent, wastes, abstracted water, wastewater and forest area. Wastes, abstracted water, wastewater date are published by TURKSTAT every two years. Forest reserves data were obtained from General Directorate of Forestry. Particle equivalent indicator to represent the air pollution was computed by employing PM₁₀ and SO₂ data according to the particle equivalent formula (PE).

P E = P M 10 + ( 0.54 * SO 2 ) (5)

Urban safety, citizen participation, medical doctors per person, number of students per classroom and percentage of illiterate women were chosen as five sub-criteria of social sustainability component. Indicators represent the safety and security, social inclusion, health and education topics. The urban safety sub-criteria calculated as the sum of traffic accidents occurred within a year in a province, those affected by these accidents (the number of dead and wounded) and the number of convicts entering the prison according to the crime per 100,000 people.

Since there is no general rule about the selection of experts, there are various studies implemented with different numbers of experts in the literature. Gómez-Navarro et al. [13] worked with 5 experts from the public and private sector in their study. Saaty [9] determined the weights of the indicators with 6 experts. Köne and Büke [14] studied with twenty-five experts. Ramanathan and Ganesh [15] made interviews with different interdisciplinary groups. Oddershede, Arias and Cancino [16] applied the pairwise comparison questionnaires to 50 experts. In AHP studies, the knowledge level of the consulted experts’ is more important than the number of experts. Besides, there should be a diversity of their study field [17].

In this study, the interviews of environmental sustainability dimension were made with environmental engineers. Selected experts are specialized on environmental sustainability and working in municipalities and private sector. For social and economic sustainability dimensions of the study, academicians were chosen who study specifically on social and economic sustainability topics. It was studied with five experts for each component, fifteen experts in total. Face-to-face interviews made with experts and paired comparison matrices generated for each criteria and sub criteria.

In order to make the definition of the ratings, the statistical series of each indicator were sorted from the lowest to the highest. Then they were divided into three groups such as low, medium and high. After making the rating definitions for each indicator, the standard deviations of the series of the indicators were calculated. The alternatives, which are at the boundaries, have been moved to an upper or lower group according to the direction of the indicators whether the contribution of the indicator to sustainability is negative or positive. Since the weights of the rating definitions for each indicator are different, the experts have also compared the rating definitions. These weights were calculated by the geometric mean method and the inconsistency rates were also checked. All calculations are below the critical value 0.10.

Aczel and Saaty [18] recommended the geometric mean method to evaluate the expert’s pairwise comparisons together. When number of judges is not large enough then the geometric mean can be obtained [19]. Additionally, the most widely used method in the literature is the geometric meaning method [20]. Thus, the weights are calculated by geometric mean method in this study. The weights calculated from pairwise comparisons of experts for criteria or in other words, components of sustainability are given in Table 4. According to the results, experts weighted the environmental sustainability by 44.3% as the most important component to ensure urban sustainability. Social and economic sustainability components are weighted 38.8% and 16.9% respectively. These results reveal the environmental concerns for urban sustainability on the ecological perspective.

The calculated weights of the urban sustainability sub criteria are given in Table 5. The weights reflect the substance that the experts give to each criterion for urban sustainability. GDP per capita is the highest weighted factor of the economic sustainability component. It is followed by investments with incentive certificate, labor force participation rate, non-performing loans of banks and, population per bank branch. The weights given to water abstraction per capita and wastewater per capita are the most important factors weighted 40.83% and 28.36% respectively, to consider urban environmental sustainability. Particulate equivalent, wastes and forest area are weighted comparatively least important. According to the weights calculated for the social sustainability urban safety indicator acquired the highest weight. It is followed by medical doctors per person, citizen participation, the number of students per classroom and percentage of illiterate women, respectively.