Methodology can be simplified as a plan of action where the method used brings to the desired outcome. The context of methodology as described by Crotty [1] cited in Levy [2], defines methodology as “the strategy, plan of action, process or design lying behind choice of particular methods and linking the choice and use of methods to the desired outcome”. Methodology refers to the theoretical arguments that researchers use in order to justify their research methods and design [3].

The purpose of this paper is to describe the application of mixed method in developing a cyber terrorism framework. This project has two primary goals, firstly is to discover a theory and then develop a conceptual framework that describes the phenomena by using qualitative method. Secondly is to verify the conceptual framework that describes the phenomena by quantitative method. By utilizing both techniques within the same study, the researcher is able to incorporate the strength of both methodologies and fit together the insights into a workable solution.

Knowledge management is a matter of managing analytical process to transform data into information, and information into knowledge, and knowledge into wisdom [17]. Recent research suggests that architectural knowledge, such as design decisions, is important and should be recorded alongside the architecture description [18]. In this research, the conceptual framework describing cyber terrorism is developed by using the grounded theory method. In grounded theory, the main source of data collection are interviews conducted with participants and observed by the researcher, whereby the researcher looks for common patterns among the sets of data [19]. Grounded theory is a rigorous process that involves generating a conceptual model (or theory) that is grounded in the data [20].

Egan [21] recommended five steps in the process of grounded theory research; initiating research, data selection, initiation and ongoing data collection, data analysis and concluding the research (Figure 1). Jones and Alony

[22] suggest five steps in grounded theory approach: acknowledgement of researchers’ bias, data selection, data collection, data coding & analysis and compilation of results. Likewise, the grounded theory method employed by Esteves et al. [23] composed of the following phases: research design phase, data collection phase, data analysis phase and comparison phase. Overall, the steps of grounded theory approach discussed above are more or less similar with each other. For this research, we adopted Egan’s recommendation.

The first level, initiation of the research involves the selection of an area of inquiry. The research begins with an awareness of the context of the research by reviewing literatures relevant to the area of inquiry. In the second level, data selection, involves the identification of potential data sources (individuals and organizations they represent) associated with the research questions. The respondents are chosen from various expert positions to reflect variety of expertise.

The third level is initiation of data collection from targeted respondents. Data collection is an ongoing process, whereby the researcher can return to the interviewees for clarification and elaboration besides comparing related documents or records with data collected during initiation of the research.

In the fourth level, data analysis, involves a constant comparison method for generating and analyzing data. Data analysis is defined as a process of systematically searching and arranging the interview transcripts, field notes and other materials with the objectives to increase the understanding and to enable researcher to present what have been discovered. Lastly, concluding the research, involves documenting the grounded theory based on researcher observation of the data saturation and sufficient theory which has emerged from the data. Data saturation is the evident when data collection no longer contributes to elaboration of the phenomenon being investigated.

Two research methods that are commonly used are qualitative research and quantitative research. It is suggested that when the goal of a research is to develop a conceptual model for the purpose of building theory around a particular phenomenon or process, an interpretive approach utilizing a qualitative methodology may be more appropriate [2]. On the other hand, if the primary goal of a research problem is to test the validity of a model where all the variables of influence a phenomenon or process is already known, then quantitative methodology may be appropriate [2]. In summary, qualitative research approaches seek to explore phenomena while quantitative research approaches seek to confirm hypothesis about phenomena.

This research is about exploring the context of cyber terrorism, focusing on the attributes or components of cyber terrorism. Since this project is exploratory in nature, a qualitative research by using a grounded theory method is adopted for this study. The phrase grounded theory refers to theory that is developed from a corpus of data through literatures, interviews and observations [24]. In this method, the objectives are to investigate the phenomena under study, to identify and construct attributes of the phenomena, and to develop a conceptual framework that describe the phenomena.

Once the theory discovery and conceptual framework have been developed, the next stage is to test or verify the conceptual framework. The outcome can be achieved by using quantitative method to quantify them and then applied statistical test to conduct the hypothesis test. In this method, the objectives are to evaluate the framework and to test the dynamic relationship of attributes (or components) of the framework.

In short, we believe that a mixed method approach is appropriate to be used in this study in order to accomplish both theory discovery and verification within a single research project. Theory discovery is achieved by using qualitative data to sharpen our theoretical ideas about the phenomena under investigation, while verification is achieved by using quantitative data to quantify them and then applied statistical test to conduct hypothesis test. The combination of qualitative and quantitative techniques within one research study able to incorporate the strength of both methodologies. The framework of research methodology is described below (Figure 2).

The approach discussed in this paper is well described by Srnka and Koeszegi [15] where they state that the combination of two separate studies, one collects and analyzed qualitative and quantitative data sequentially, has been acceptable to researchers in the social science. By conducting qualitative method in a preliminary stage, the objective is for the researchers to develop a conceptual framework (or to generate hypothesis) for the quantitative study. Thus, when qualitative data collection precedes the quantitative data collection, the intent is to first explore the problem under study then follow up on this study with quantitative data that are amenable to studying a large sample so that results might be inferred to a population [25]. We believe that by merging the results provides an overall picture of the research problem.

In this study, mixed method research involves both collecting and analyzing quantitative and qualitative data in sequential order. The first stage in the analysis cycle is to frame a theory or hypothesis from the literatures which can be used as initial guidance for data collection and data analysis. Once the theory or hypothesis has been generated, the researchers develop questionnaires to get in-depth understanding on the phenomena under investigation. Data collection is a semi-structured method by using in-depth interviews. Interviews are part of most

interpretive studies and as a key way of accessing the interpretations of informants in the field [26]. Since the first stage is explanatory in nature, data analyses are done by using grounded theory approach. Qualitative research includes a variety of methodological approaches and one example is grounded theory [27]. As have been mentioned earlier, the phrase grounded theory refers to theory that is developed from a corpus of data through literatures, interviews and observations.

In the next stage, quantitative approach is applied to corroborate the researcher’s initial discovery. A survey with close-ended questions format are used for data collection to determine the views and opinions of the population that represents various groups in the society. In this phase, the researcher’s goal is to test the theory or hypothesis, which finding is generalized from a larger representative sampling. Statistical analysis is performed to test the validity of the theory or hypothesis. The numerical findings help to interpret the results, where clearer interpretations of the statistical results are obtained. Figure 3 shows the summary on how the data are collected and analyzed.

All the relevant methodological issues discussed in this paper provide justification and practical approach on how the research is conducted. The methodology explained in this paper provides the researchers with the right direction and understanding in conducting the research by choosing the right research design. Research design as defined by Cooper and Schindler [27] is a plan and structure of investigation so conceived as to obtain answers to research questions. It includes an outline on what the researcher will do from initiation of the research to the final analysis of the data.

In this study, we suggest that a mixed method research approach is appropriate to be applied. The motivation factors in applying the mixed method (as opposed to a single method) are due to the following reasons. Firstly, the nature of the research is exploratory and explanatory in nature, and is grounded in theory. The goals are to discover theory and to develop a conceptual framework that describes the phenomena (qualitative method); and to test or verify the conceptual framework that describes the phenomena (quantitative method). As noted by Yauch and Steudel [13], the mixed method research complements each other, which explains the results of analyses. The qualitative research is interpretive, which allow for the discovery of new ideas and unanticipated occurrences. Qualitative research aims to achieve an in-depth understanding of a situation or a certain phenomenon [27]. Focus of the research is to explore (understand and interpret) of a situation or a certain phenomenon. On another hand, quantitative research aims to achieve precise measurement of something such as participants behavior, knowledge or opinion [27]. Focus of the research is to confirm (describe and explain) hypothesis about a situation or a certain phenomenon. Therefore, by utilizing a mixed method research, we believed it would bring conclusive findings on outcome of this study.

Secondly, mixed method approach helps to answer questions that cannot be answered by qualitative or quantitative approaches alone, thus, provide breadth and depth to the study [28]. Researchers who conduct mixed methods research are more likely to select methods and approaches with respect to their underlying research questions in ways that offer the best opportunity for answering important research questions [28]. In this research, a survey questionnaire using qualitative in-depth interviews is conducted as a way to tap into participants’ perspectives and insights. During analysis, the qualitative researcher uses content analysis of written or recorded materials drawn from participants’ expressions and observations [27] or documents review [2]. Qualitative research study via questionnaires interview, and supplement with close-ended survey systematically provide

breadth and depth to the research [12]. Quantitative research looks at the frequency [27] or any kind of research that produces findings arrived at by means of statistical procedures [2]. Through this approach, it is believed that the findings are corroborated across different approaches, thus provide greater confidence in the conclusion.

Lastly, the research framework of mixed method is chosen to apply the concept of triangulation. As noted by Paton [12], one important way to strengthen a study design is through triangulation, or the combination of methodologies in the study of a situation or a certain phenomenon. Paton [12] also argues that triangulation provides accuracy of the data gathered while Olsen [29] argues that triangulation provides validation besides deepening and widening one’s understanding. Triangulation is a term derived from navigation and surveying where different bearings are taken in order to arrive at a precise location. In this study, triangulation involves locating a true position by referring to two or more other coordinates [12]. As noted by Yauch and Steudel [13], triangulation corroborates data and obtains convergent validity. Moreover, triangulation reduces biasness besides be able to heighten the validity of the data collection and analysis.

The study on the context of cyber terrorism is quite complex as it is about threat perception which makes the concept differ from one to another. Understanding similarities and differences in perception of what constitutes cyber terrorism can provide insight on the concept of cyber terrorism. The outcome of this study serves the basis for various strategic decisions for policy and decision makers as well as useful foundation for academic research in understanding the context of cyber terrorism. Detail focus analysis can be conducted to investigate and analyze the context of cyber terrorism.

This paper provides information on how this research is conducted. It also describes the framework of research methodology, where we have recommended that the study to be conducted by using a mixed method research: qualitative data and quantitative data are collected and analyzed respectively in a separate phase. This paper also established the conceptual framework that guides the data collection and data analysis techniques. Finally, the research methodology explained in this paper provides the researchers with the right direction by choosing the right research design. The mixed method improves the rigor and explanation of the results, thus bring conclusive findings to the research outcome. By utilizing qualitative and quantitative techniques within the same study, a mixed method research should be able to incorporate the strength of both methodologies and fit together the insights into a workable solution.

The authors would like to thank the following individuals who provided valuable input to this paper: Dr. Solahuddin Shamsuddin and Zaleha Abd Rahim of CyberSecurity Malaysia. We also would like to thank the Center for Advanced Computing Technology, Faculty of Information and Communication Technology, Universiti Teknikal Malaysia Melaka (UTeM) that provided research grant for this project.