Usability evaluation is an important element to analyze usability issue in any system or applications. Usability issues are identified through evaluation conducted with users. Literature shows many usability models has been referred for conducting usability evaluation. Among the common usability evaluation model are Nelsen, QUIM, mGQM, Harrison and ISO. These are some of the usability evaluation models that have been in use for usability evaluation. However, when application is developed for specific targeted user, requirements of the user need to be incorporated into the application [7] . Application will fail to satisfy the user if the requirement is absent and make it more difficult.

Usability metrics measurement is generally used by the examiners to identify what they are going to measure. Between the current usability measures is ISO 9241 part 11, guidance on usability [8] . However, this measure has been used usually but is too common, tough to relate to explicit domains and does not assistant with any quality characteristic to the measure [9] .

The GQM method in emerging measurement metric could be experimental in. Moreover, this approach has also been requested as one of the most goal-focused and most popular methods used among several measurements approaches [8] . Although GQM was previously used to express and evaluate goals for specific projects and environments, its aim has been extended to a larger perspective including quality improvement, measure progress, and plan for project. Therefore, it is assumed that the GQM approach could also possibly be extended to measure the usability guidelines by providing metrics for usability guidelines. In combination, in this study also explains the definition phase of the GQM paradigm, proving the outputs of the first three steps of Basili’s GQM process, the hierarchy of goals, questions, and meaningful metrics [10] .

The previous study only focused on dimension and criteria measurement. But in this new research, it was extended the metrics measurements.

The tourism mobile applications currently can be said as the most proficient applications that can facilitate the explorers' movement of travelers. The users of this mobile tourism application can easily accomplish the way to places that they are not familiar with. The information gives relating to places of interest is useful and crucial for tourists [2] . The tourism mobile application deals not only the descriptive text of the information provided, but shows pictures of hotspots. In addition to that, tourists will be able to check for the facilities available surrounding certain places like entertainment and restaurants, hotels and other nearby hotspots. As disparate to printed maps that were mainly used by travelers beforehand, the mobile application may be more advantageous. It will help tourists to plan their tour ahead of time, and have an open feeling of quietness amid their occasions, as they will be increasingly arranged [11] .

Therefore, Smart tourism changes tourist information search activities. The tourism information nowadays became very flexible and several at the stages of collecting and getting data. Therefore, tourists can have easily access to all information and activity concerning via internet using different mobile applications [2] . Besides, tourists can also enjoy an experience of the tourist destinations and attraction places by using and applying mobile application. In this way, tourists they can get easily and know about various information related to tour destinations and places that are interesting (Table 1).

The above table shows the most popular and useful mobile tourism applications currently, each application has different functionalities that help users to obtain their trips concerning flights, hotels, restaurants and possible activities on destination through Google Play Store or App Store [12] . Therefore, this table shows Trip advisor is amongst one for tourism mobile application in terms of the functionalities of the application.

The main aim of the study is to propose usability evaluation metrics for tourism mobile application. As part of generating metric for the proposed metrics,

requirements of tourism were analyzed to be matched appropriately with the metrics developed earlier through a literature review. This will ensure that, metrics developed are not very general but focused on the tourism applications users need as well. To derive proper metric, QUIM and mGQM were used as basis in this part. This is to ensure identified metrics are adopted from proper and established studies in accordance with the identified requirements of the tourism towards usage of mobile applications.

The next step in the development phase is identifying the metrics for the generated usability dimensions and criteria from previous section. Two types of data for metric have been identified which are the objective and subjective data. Objective data refers to the task performance analysis and measures performance of usage while subjective data refers mainly on the user’s feedback towards usage of the applications [14] . To create the metrics for the criteria that have been generated, models such as QUIM and mGQM were analyzed and modified metrics from these researches were employed into the proposed metrics according to the criteria defined and suited for the study.

Reasons for accommodating these two models are due to the factor which allows freedom in selecting metrics according to the study and applied into the criteria that leave many opportunities to thoroughly analyses suitability of the metrics [15] , as discussed in earlier. Besides that, research having guideline selecting metrics for broader usability goals [15] [16] , and categorizes relationship among metric and criteria appropriately [16] .

The selection of the metrics for the measurement also considered the requirements of tourism that have been analyzed earlier and keeping in mind that the data to be collected is for a mobile platform which differs from desktop platform. This is matched with the criteria to be measured and finally developed the measurable metrics to gather both objective and subjective data. Short listing of selected metrics is done by analyzing the literature from literature review. User context is the most important aspect considered since the user is the tourism applications which will ensure applications are measured accordingly and meet the purpose of the study based in a mobile platform.

Besides that, metrics in literature are mostly focused on desktop application, however, few studies discuss metrics on mobile application compatibility and thus, metrics were chosen based on that perspective as well. Basically, metrics are derived in conscious to be suited with the selected dimensions and criteria that have been applied by studies in literature review.

The expert review was conducted to provide the verification of the proposed measurement of usability evaluation metrics of tourism mobile application. Expert review is one of the significant ways in detecting and removing defects [17] , thus, the study adapted this technique for verification of the proposed usability measurement of usability evaluation metrics of tourism mobile application. Through verification, all the components developed would be able to be confirmed as well as organized and presented appropriately [18] . The main aspect need to be verified in the proposed usability evaluation metrics of tourism mobile application is the usage of the appropriate subjective and objective metrics. Therefore, potential experts in the domain of usability and tourism especially researcher and academician, were identified. The expert was chosen according to the suggestion by [19] [20] .

Five (5) experts were contacted through email and willing to verify the proposed metrics which all of them are academician [21] . Contacted experts led for verification is conducted since it can significantly lead to give accurate result.

Among the five (5) experts, four (4) experts agreed on face to face meeting which were arranged and held with the experts, while one (1) expert agreeing in doing the review through online due to work hectic and unable to set appropriate meeting hour because the expert had the distance between cities that had been a hindrance. The expert’s background was listed in Table 2 below.

Besides completing the verification form provided, comments and suggestion were also given by the experts for metrics improvements. Table 3 below

P: Relevancy of the appropriateness of the metrics. *: irrelevant or unsuitable placements of the mentioned metrics.

summarizes on the verified proposed metrics components by the experts which determined the relevancy of data collection and analysis of the tourism mobile applications usability evaluation. Moreover, the components in the proposed metrics verified in align towards concept of Human Computer Interaction (HCI) and amendments done for any the components needed to be modified or retained in the finalized metrics.

Metrics were marked with symbols were considered to be dropped from the list or moved to more suitable metrics as per suggested by the expert. While metrics that were accepted for relevancy are retained unless comments expecting the metrics moved to suitable metrics. Metrics are being selected from the proposed usability evaluation metrics of tourism mobile applications based on the percentage score obtained by individual metrics as well as comments from the experts. If an item is scored at least 60% average percentage, possibility of the significant results to be produced are high and such item can be considered (Clarke & Warwick, 2001). However, results obtained from the experts shows some metrics scored less than 60% means the metrics are unsuitable and irrelevant for the proposed study. However, mostly the metrics are scored more than 60% means those metrics are suitable and appropriately developed and can produce a significant result. The comments are analyzed carefully on the appropriateness for acceptance and metrics have been modified accordingly. The experts have given comments and suggestions which helps in modifying the proposed usability evaluation metrics of tourism mobile applications. These comments from the experts have been significant contribution to the proposed metrics. Table 4 shows the comments and suggestion received from the experts.

The comments received are then taken into consideration for the modification of the proposed metrics which is discussed further in this section. The comments and suggestion from the experts were examined carefully and the proposed metrics were modified whereas some metrics were removed and redundant metrics were omitted from the proposed metrics (Table 5).

As the expert suggested, the above table shown the metrics that were removed or replaced from the measurement. Experts have also given suggestion for the measurement improvement through other criteria that are appropriate that can measure the dimensions derived. Table 6 below shows the added criteria as per suggested from the experts.