1. Introduction

JGIS

Journal of Geographic Information System

2151-1950

Scientific Research Publishing

10.4236/jgis.2023.152011

JGIS-124232

Articles

Earth&Environmental Sciences

Developing a GIS Audit Framework in the Context of Information Technology though a Reductive Model Approach

Fausta

Mbura Njiru

¹^*David

N. Siriba

¹Faith

N. Karanja

Department of Geospatial and Space Technology, University of Nairobi, Nairobi, Kenya

31032023

150219622228, February 20239, April 2023 12, April 2023

2014

This work is licensed under the Creative Commons Attribution International License (CC BY). http://creativecommons.org/licenses/by/4.0/

A GIS audit framework is necessary considering the diverse nature of GIS with regard to components, applications and industry. In practice, checklists are generated during the audit process based on specific objectives. There is no standardized list of items that can be used as a reference. The purpose of this study was to develop a GIS audit framework as a foundation for GIS audits. The framework provides that comprehensive approach to various GIS aspects during the audit process. The design builds on a developed conceptual framework where most significant categories of GIS audit parameters namely data quality, software utilization, GIS competency and procedures (work flows) were identified. The study adopted a reductive model approach to simplify the complexity associated with each category of GIS audit parameter. The resultant audit elements for each category are organized in a matrix that forms an integral part of the framework. The columns comprise audit goal, audit questions and audit subjects as indicators which are qualitatively measured. The rows comprise the parameters ( data quality, software utilization, personnel competency and procedure (workflows)). To use the framework, an auditor only needs to create an audit checklist that consists of particular parameters and indicators from the framework depending on audit objective. As part of an on-going research, the next step will involve validating the framework through a mock testing process.

GIS Audit Framework GIS Complexity Reductive Model GIS Monitoring and Evaluation Audit Checklist

1. Introduction

In the context of GIS audit, a framework is a reference for undertaking GIS audit [1] . The extent to which GIS is utilized in an organization should be audited to ensure monitoring and evaluation. Previous GIS audit frameworks are designed for specific operations and may require calibration or customization for a more meaningful result that fits users’ need [2] . This necessitates a generic framework that provides a wide range of audit aspects within various levels of GIS. The main objective of this paper was to present the design of GIS audit framework through a reductive model approach.

GIS is broad and complex [3] which poses a challenge in the design of the audit framework, considering that GIS inputs & outputs are distinct and specific to particular applications [4] . In this regard, a reductive model approach was adopted in the design of the audit framework. The approach ensures that the entire GIS is presented in terms of its components and still preserves its identity [5] . The reductive model approach addresses the complexity and multidimensionality of GIS audit [6] , by breaking down the main parameters and focusing on sub-parameters. The next section reviews the reductive model approach and its application in GIS audit, followed by a presentation of on the approach in the design of the GIS audit framework. The framework is then presented subsequently.

2. Literature Review2.1. Reductive Model Approach

Information systems are complex and comprise several integrated components [7] . The complexity makes development of audit framework demanding and difficult [8] . The audit framework parameters may not be exhaustively captured, necessitating use of strategies to simplify and reduce complexity [9] . Reductive model approaches have been applied to provide approximations related to parameters under consideration [10] . Reduction provides an explanation that decomposes complex activities and localizes components within the complex system [11] . It implies that a theory which describes phenomena at a particular set of large scale may be reduced to theories appropriate at smaller scale [12] .

2.2. Introduction to Reductionism

Reductionism is an approach that is used in many disciplines including psychology that is centered on the belief that something can be explained by breaking it down into its individual parts [13] . The idea of reductionism was introduced by a French philosopher “Rene Descartes” back in 1637, where he argued that a complex system can be explained by reducing its fundamental parts [14] .

Recent literature suggests that reductionism originates in what is called “multi-scale argument” based on the fact that numerous successful scientific models appeal to features and properties from wide range of scales [15] . The model concerns the behavior of materials that display radically different behaviors at different length of scales [16] . It is based on the assumption that any complex system is best understood by analyzing its physical parts in isolation [17] . The perspective taken by this paper is that GIS is composed of components such as data, software, hardware, people and methods [18] . While a component such as hardware is obvious, diversity or quantity of data, software, competency and procedures that professionals use are sometimes overlooked [19] . There is need for them to be analyzed and reduced in order to arrive at optimal audit elements.

2.3. Common Applications of Reduction

The reduction approach holds a significant influence and application on information system, medical science and biology [20] . In information system, understanding system behavior proceeds from bottom-up by aggregating explanations of individual components behaviors [21] . Basic concept is approximation of large scale dynamic systems [22] . It decomposes a given system into a number of subsystems for computational efficiency and design simplification [23] .

2.4. Reduction in GIS

GIS comprises components with complex elements and connections linking them [24] . This necessitates breaking down the complexity of such a system [25] . Even though there isn’t a generic approach for resolving system complexity, information systems problems are context specific [26] . Reducing complexity of GIS involves an approach that introduces GIS concept to determine what exactly need to be reduced [27] .

Not much has been done for GIS reduction but literature reveals that reduction approach has been applied to reduce complexity of GIS vector data during transmission [28] . In information system, reduction has been applied to deal with complexity where reduction mechanisms such as partition, projection and filtering are applied to reduce quantity or diversity of elements [10] . Also, attribute reduction model has been applied to provide parameter approximations for massive and complex information system datasets [29] . Figure 1 illustrated the GIS audit reduction process.

The strategy for reducing GIS audit complexity is centered on interpretation of elements that represent GIS as a system [30] . Essential elements and their relationships are understood qualitatively by applying partition as the main reduction strategy of abstracting audit elements [9] . From Figure 1, partition involves breaking down quantity and diversity of GIS audit parameters into simplified elements [31] . Four aspects: quality, functionality, performance, monitoring & evaluation are considered meta-models that correspond to general concepts for describing GIS audit [9] . The meta-models are themselves models comprising concepts that guide in abstraction of audit elements [32] .

The meta-models are explored to come up with simplified essential audit elements [9] . Audit concerns are addressed through formation of classes/categories in relation to each aspect under consideration [33] . For instance, quality elements characterize fitness for use and production of GIS products and services with quality level that fulfils requirements [34] . Abstraction focuses on essential

quality elements categorized into spatial location, relationship and attribute information that provide an effective framework for encoding geospatial data [35] .

Similarly, Functionality of a GIS software is characterized by analytical capability and suitability aspects that determines its ability to achieve desired objectives [24] . Commercial and open source GIS software has significant variation in terms of functionality [36] . Based on software platform (desktop, web, server or specialized) used, software utilization is centered on software effectiveness to the defined GIS application [37] . Abstraction of software functionality is based on elements deemed to provide primary needs to solve spatial issues categorized into mapping, spatial analytics, external environment support and customization [38] .

More so, a successful GIS performance is defined by competencies that address workforce need within a particular work setting [39] . This involves professionals with specialized skills and general competencies that influence their ability to manage GIS resources [40] . However, evolving set of responsibilities and expectations associated with GIS professionals poses a challenge on addressing the knowledge and abilities they are expected to possess [41] . Abstraction of essential GIS competency audit is therefore based on elements that support organizational needs, categorized into technical knowledge and skills, workplace competencies and personal effectiveness [39] .

In addition, monitoring & evaluation aspect provide procedures to oversight GIS implementation, datasets, technology and resources [42] . With GIS attributed to various technologies, processes and methods [18] , the complexity of monitoring and evaluating GIS is simplified with respect to GIS procedures that focus on staying up to date in line with GIS components, resources and technology [43] .

The classes/categories resulting from explored meta-models are filtered to provide reduced elements for entire model [9] . Filtering is restricted to properties or attributes explicitly related to each GIS audit parameter [44] as presented in Table 1. For example, quality involved filtering audit elements aspects with respect to accuracy or thoroughness of spatial and attributed information [45] . They reflect quality elements that facilitate compliance and consistency of GIS Data [46] . On the other hand, GIS software functionality filtration was based on those that match common users’ needs: capture, store, retrieve, manage, display, and analyze all types of spatial & non-spatial data [47] . This is with respect to GIS application need aimed at reducing the resources and costs associated with supporting bundled GIS software tools [48] . Similarly, filtration of performance involved elements of qualified and successful geospatial industry workforce with respect to sector-specific, personal effectiveness and management competencies [41] . Additionally, monitoring and evaluation were filtered with respect to basic

Table 1 Filtering criteria

Parameter	Filtering Criteria	Filtered Audit Elements
Data Quality	Compliance and consistency of GIS Data [46] .	Accuracy, Lineage, completeness, currency, coverage and consistency.
Software utilization	Software utilization that match common users’ needs aimed at reducing resources and costs associated with supporting bundled tools [48] .	Common GIS software utilization functionalities with respect to mapping, spatial intelligent, external environment support and customization.
GIS Competency	Qualified and successful Geospatial industry workforce with reference to technical knowledge and skills, sector-specific, personal effectiveness and management Competencies [41] .	Technical knowledge and skills, experience gained from working within a GIS environment, exposure to equipment and software, GIS software applications development, GIS quality control/quality assurance (QC/QA), governance and behavior competencies.
Procedures	Basic procedures for managing a progressive GIS to provide support for successful operations, data management, applications development, customer support and funding [49] .	Procedures for monitoring operations, data, technology, standards and operating procedures, stability, growth and funding of GIS.

procedures for managing a progressive GIS that support successful operations, data management, applications development, customer support and GIS funding [49] .

There isn’t a perfect solution to reducing system complexity neither is there a specific standard approach [26] . For complex systems such as GIS, context specific approach is recommended [9] . This approach starts with review of concepts underlying the system [33] . This enables understanding system dynamics to assist in analysis & identification of quantity or diversity of reduction elements [50] . Model abstraction mechanisms & strategies such as projection and partition are applied to reduce quantity or diversity of elements and their interrelations [9] . Partition focuses on breaking down the entity into pieces [31] while projection focuses on compacting the entity [51] and still preserves the physical properties after reduction.

In this paper, partition has been applied as the main strategy for reducing GIS audit parameters. The audit parameters have been broken down into simple elements that focus on pertinent audit elements. It may be difficult to understand all associated elements for a system that is constantly evolving like GIS [52] . This may render GIS as a system perceived more than the sum of broken elements, such that aggregation of broken elements may not equal to a complete GIS system. Major benefit of this reduction is that the complexity of GIS system is decomposed into smaller elements that are easier to investigate and work with for a successful audit engagement [21] .

3. GIS Audit Framework Design Methodology3.1. Stakeholders Perspective

The framework design is centered on different levels of GIS application areas within an enterprise setting. GIS users may have different views on GIS audit depending on the size or focus of the organization, hence necessitating their involvement as co-designers. For this reason, a questionnaire inform of a GIS audit checklist was prepared and shared with various GIS users in Kenya to ensure participation and ownership [53] . The questions were not structured but open-ended in which the participants were free to give their responses [54] . They gave views with regards to elements they would want audited with respect to the four categories of GIS audit parameters. These views were integrated into the reduction model with respect to the four meta-models: quality, functionality, GIS performance, monitoring & evaluation. They were filtered to form part of reduced audit elements. The choice of selected co-designers was informed by the level of uniqueness and size of their organizations, clustered into national government, county government, government land agency, private sector and private practitioner.

3.2. Design Architecture

Audit is a dynamic and complex activity comprising several inter-related activities [55] . A structured design is employed for each GIS audit parameter to capture aspects of planning, need assessment, audit execution, completion and reporting [56] . This helps in establishing overall audit strategy for engagement and development of an audit plan [57] . The main architecture of audit plan is presented in an audit process flow that outlines GIS audit requirements. The flow provides a sequence of processes that describes design components crucial for each GIS audit parameter. Each parameter is uniquely described by considering its associated parameters, key features and contribution to GIS.

3.3. Data Quality Audit

The design of data quality audit identifies data integrity constrains that detects and evaluate inconsistencies within a GIS database [58] . Topological, spatial and attribute consistencies are fundamental in defining GIS data integrity elements [59] . The roadmap to data quality audit is a well-defined data quality audit architecture as illustrated in Figure 2. The figure is a guide that organizes appropriate levels of data quality audit engagement. It demonstrates an audit process in which a GIS user starts by outlining the context of GIS application to determine required data, accuracy and suitable data verification tools [60] . Quality of both the existing and primary data should be evaluated to ensure fitness for the intended use [61] . A matrix is used to organize data quality audit priority levels. The matrix comprises rows that define Data Integrity Parameters (DIP) and columns that define Basic Audit Elements (BAE) vital for data quality audit process.

3.4. Software Utilization Audit

The methodology captures software utilization in relation to its features and functionality [62] . It takes into account software audit architecture that provides necessary steps for a GIS software audit as illustrated in Figure 3. The figure demonstrates a GIS software utilization audit based on the knowledge of GIS architecture in place. This provides an audit engagement that affirms right tools for the identified architecture. The architecture provides a software utilization matrix whose columns are designed to define major GIS functions (F1, F2…) within a GIS software ((S1, S2…). It is vital to determine a suitable software package for the preferred GIS architecture. This will help to analyze software impact in terms of usability, reliability and costs.

3.5. Personnel Competency Audit

The main architecture for GIS personnel competency is presented by a process flow as illustrated in Figure 4. It outlines critical GIS performance subjects that guide in assignment of appropriate staff who meet GIS application needs and desired output quality. The workflow comprises a matrix developed to measure performance expectation of each staff (Emp1, Emp2…) in relation to performance subjects outlined (Technical Knowledge and Skills-TKS, Experience gained from

working within a GIS Environment-EG, Exposure to Equipment & Software-EES, Behavior Competencies-BC…). This assists in coming up with a competency plan with specific action for competency development strategies. Audit subjects are guided by a series of important considerations to ensure each subject matter is clear on performance indicators for an operational GIS with respect to customer satisfaction, quality of internal processes, personnel development and revenue growth.

3.6. Procedures Audit

GIS procedure guidelines are presented by an architecture that is designed to ensure a centralized GIS management process that targets operations, data, technology, operating procedures, legal issues, stability, growth and funding of GIS.

Figure 5 defines major GIS monitoring and evaluation elements (M & E Elements) that determine whether the GIS setup is on its way to achieving its goals and objectives. These are characterized by GIS data and resources accountability, key GIS management subjects, GIS technological advancements and GIS operations.

4. GIS Audit Framework

The framework design outputs consist of design effort for each category of GIS audit parameter. Audit subjects are structured in a matrix whose columns comprise basic audit elements. The matrix is simple and the design is customizable,

flexible and open to meet future growth and opportunities. Core elements of audit matrix comprise audit goal, question, definition, methods, frequency and responsibility as illustrated in Table 2. Data quality, personnel competency and procedure parameters adopt these elements for their audit. However, software utilization parameter comprises two elements of audit matrix; functionality and percentage utilization score.

4.1. Data Quality

Data quality audit outputs are meant to enhance confidence in data integrity, compliance, availability, reliability, efficiency, effectiveness and confidentiality [63] . They are identified based on how well GIS dataset meet the criteria set for its product specification as provided by ISO 9113 geographic information quality principles [64] . The principles are based on completeness, logical consistency

Table 2 Elements of audit matrix

Element	Description
Goal	Clarifies achievements that must be obtained from specific audit
Question	Presents the question that facilitates execution of audit goal
Definition	Enables understanding audit goal’s results by elaborating what must be put in place to tell whether the results have been achieved
Methods	Outlines the means in which audit information is obtained
Frequency	Guides GIS users on the rate at which audit information should be obtained during GIS implementation
Responsibility	Describes the person responsible for audit review, analysis, report audit results and give recommendations

and positional accuracy [65] . Similarly, [66] describes seven crucial information that characterizes quality to include availability, integrity, compliance, reliability, efficiency, effectiveness and confidentiality. With respect to these principles and characteristics, spatial data quality is grouped into two components: internal quality which corresponds to the extent of similarity between data produced and ideal data that should have been produced and external quality that corresponds to the level of conformance that exists between spatial data products and user’s needs within a given context [67] . In relation to these two components, data quality audit parameters are grouped into eight elements: accuracy, lineage, currency, completeness, consistency, coverage, adequacy & reliability. The resultant data quality design framework is presented in Table 3.

4.2. Software Utilization

Software utilization is centered on a GIS software that is able to capture, store, process, query, analyze and visualize GIS data [68] . However, sifting through available choices of GIS software tools/functionalities and setting them up for evaluation could be time consuming with considerable amount of trial and error [48] . With this consideration, the design has adopted audit criteria based on common GIS users’ requirements in relation to important elements of software utilization and operational processes [69] . These elements are related to mapping, spatial intelligence, external environment support and customization [38] . The mapping element takes into account a software that is able to collect data, perform edits and manipulations that suit broad GIS application needs [69] . Spatial intelligence focus on a software that generates accurate and useful information for various intelligence decisions [38] . External environment support is based on the consideration that a GIS software is capable of exporting and importing data from different data formats & schemas [70] . Software customization considers customer preferences which are not fixed within the built GIS tools [71] .

In reality, GIS users utilize a small fraction of functionalities built within a GIS software and some are not able to interrogate components needed, or not needed, to meet user requirements [48] . A GIS user needs to evaluate software utilization

Table 3 Data quality audit framework design

	Goal	Audit Question	Audit Definition	Audit Method	Audit Frequency & Reports	Responsibility & Reporting
Accuracy	Examine level of data accuracy in line with the specified application/use	Does the user understand accuracy limitations of GIS data	Spatial/positional accuracy Temporal accuracy Thematic accuracy	Documentary analysis: ● Accuracy statement ● Accuracy report Comparison with an independent source of higher accuracy	Collection stage Prior to data use Upon Purchase Daily productions After ● Transformation ● Conversion ● Update ● Modification	GIS User GIS Program Manager
Lineage	Establish history of datasets being subjected for GIS application	Is data source and its transformations documented	Data history Data attributes Map legends Map libraries Database system Data Projection	Documentation report Spatial interactions through overlays Track history from database system	Prior to data use Upon Purchase Daily productions After ● Transformation ● Conversion ● Update ● Modification	GIS user GIS Program Manager
Currency	Determine data timelines	Is currency status appropriate for the specified application	Metadata Capture date Frequency of update Revision status Attributes value age	Validate data timeline contents with available metadata	Daily productions Prior to data use Frequent data updates and revision	GIS user GIS Program Manager
Coverage, Adequacy & Reliability	Access the data adequacy and appropriateness to meet specific needs	Is the data adequate & reliable in meeting product needs	Spatial extent Availability limitations Access conditions Temporal extent Adequate schema	Data catalogues Product catalogues Metadata catalogues Adequacy report Compare project area of interest	Daily productions Routine data checks (weekly, monthly, quarterly or annually)	GIS user GIS Program Manager
Completeness	Establish the fitness for use and wholeness of data for accurate and sufficient GIS application.	Is the data schema suitable for consumption	Completeness in ● Spatial/space ● Temporal/time ● Thematic/ attribute ● Feature ● Coverage ● Scope ● Format	Schema checks on data values and formats Complete inclusion of features & attributes Quality control Compliance testing User’s feedback Ground Truthing	Daily productions Scheduled quality control Program (weekly, monthly, quarterly or annually)	GIS user GIS Program Manager
Consistency	Assess the contradiction in the GIS database	Is there internal validity of the dataset to ensure correctness of outputs	Data adherence to standards Data adherence to logical rules of data structure, attributes and relationships	Consistency report to validate data conformance to standards Validate geometry conformance to topological rules Check attribute redundancy Check for validity of domain & data types Check validity of geometries Implement OGC and ISO standard	High frequency checks during daily productions	GIS user GIS Program Manager

to interrogate software functionalities in order to review and analyze percentage utilization within the GIS. Table 4 presents a software utilization matrix based on a design that provide a “yes” or “no” answer for each software to determine final percentage utilization.

4.3. Personnel Competency

Personnel competency audit design is founded on basic knowledge, skills and behavior required to perform their roles. The design provides a tabular model for major elements that entails auditing personnel competency within a GIS environment. It encompasses personnel competency associated with parameters in terms of technical knowledge and skills, experience gained from working within a GIS environment, exposure to GIS hardware and software, GIS software applications development, GIS quality Control and Quality Assurance, GIS governance and behavior competency. These are described in Table 5.

4.4. Procedures

The design is associated with various aspects that ensure proper procedures for monitoring and evaluation of a lasting GIS. Associated parameters are derived from a checklist of 7 elements which include GIS operations (strategies, implementation, processes, outputs & communication), Data (accurate & relevant data),

Table 4 Software utilization design

		Functionality
		Data capture, monitoring & editing	Data conversions & transformations	Geocoding: Location information transformation	Map interaction: search, retrieve , query, measure	Map display, visualization & symbolization	Geoprocessing tasks & manipulations	Spatial Query, analysis & modeling	Spatial topology overlay analysis	Vector, raster/image analysis & processing	Geostatistical analysis: interpolation/extrapolation	Data storage, organization & validation	Data integration	Map creation, presentation, Printing & publishing	Data view tools: navigate, identify, select etc.	Data import & export to various formats	Graphic & interactive display: reports, tables, maps	3D processing, analysis , modeling & visualization	Geodatabase creation & management	Programming: customizing or creating new tools	Data exchange & sharing	Installing & managing extensions and plugins	Support for external & remote connections	Support for OGC & web services	Interoperability: view data from other platforms	Operating system requirement
Software	S1
	S2
	S3

Responsibility: GIS User (Evaluate & Analyze), GIS Manager (Report), Senior Management (Act).

Table 5 Personnel competency audit framework design

	Goal	Audit Question	Audit Definition	Audit Method	Audit Frequency & Reports	Responsibility & Reporting
Technical knowledge and skills	Examine professional competency to delivery of quality products and services by GIS work team	Is there adequate knowledge and abilities required to apply technical principles in GIS job functions/roles	Performance of Critical GIS work functions Proficient operations that demonstrate understanding of work mandate	Qualitative and quantitative measure of input, output and time taken. Compare attainments expected goals and check for defects or customer satisfaction Interview	Daily productions	GIS Program Manager
Experience gained from working within a GIS environment	Review of technologically appropriate GIS work team who are well connected to the GIS industry environment for smarter and faster delivery of products and services	Does the GIS competency provide understanding, maturity, independence & confidence in GIS tasks at hand	Productive workforce, Engaged workforce and Operational effectiveness. Expertise in GIS technology exploitation through ● Training ● Professional networking ● Development programs ● Counselling	Measure results in terms of ● Faster & increased productivity ● Improved quality of products and client/customer satisfaction. Review individual’s adaption to technological developments & exploitation	Daily work experience Frequent ● Training ● Networking ● Development programs ● self-learning ● E-learning & observations	GIS Program Manager Senior management
Exposure to equipment and software	Assess GIS work team ability to keep track on rapid technological advancements on GIS equipment and software	Does GIS team stay apprised in developments & technological advancements with respect to software and equipment	Abreast of technological advancements	Measure effectiveness and constructiveness of an individual through results after tool or equipment use (accurate or erroneous results)	Daily productions Frequent Trainings	GIS Program Manager
GIS Software applications development	Assess GIS developers work team in providing basic functionality to the GIS software and applications	Can GIS competency develop new software products or customize existing tools to suit user need	Software effectiveness GIS Programming	Software performance plan Software/applications testing schedule	Frequent analysis of software performance Continuous software/ applications testing	GIS programmer GIS Developer GIS system Engineer
GIS quality Control and Quality Assurance (QC/QA)	Establish the quality of information input into the GIS and its fitness for use	Does GIS competency provide set of processes to measure and assure the quality of GIS in meeting product expectation	System integrity System data quality Quality assurance Quality control Quality outputs	QC/QA plan Adherence to existing data policies & standards Manufacture’s certificates of tools & equipment Updates & upgrades of GIS resources	Continuous checks	GIS user GIS Program Manager
GIS Governance	Assess the alignment of GIS structures and resources with expected goals by the management team	Is there a set mechanism through which GIS strategies are delivered, managed and continually optimized	Management structure Administrative and technical directions Separation of duties Staff handover plan	Staff feedback plan Staff reporting plan Housekeeping plan Activity schedule Resources inventory	Daily Management Routinely (weekly, monthly, quarterly or annually)	GIS Program Manager Senior management
Behavior Competency	Examine Organization & individual culture that ensures the system has the precise balance of team work	Does GIS team have attributes and traits that define strength for a successful GIS	Personal/Social competency Professionalism Workplace codes of ethics Customer satisfaction	Rating scale dependent on individual quality and quantity of output Peer appraisals Evaluate overall team performance Analyze customer feedback	Routinely (weekly, monthly, quarterly or annually)	GIS Program Manager Senior management

Technology (Hardware & software), GIS data standards and operating procedures, Stability, Growth and Funding. Major outcome includes proper documentation and reporting that expedites tracking of GIS progress. This will provide room to measure accrued benefits from the developed GIS. They also act as a pointer to the GIS stakeholders on whether to retain or refresh GIS strategies where need be and also develop cases for GIS improvement and investments. These are described in Table 6.

5. Results and Discussion5.1. Results

The GIS audit design framework presents the output obtained from analyzing each audit parameter. It provides a logical sequence of audit activities that should be performed within the specified audit structure. It has defined audit goals that describe what each audit element is intended to achieve. Based on these goals, the user may be guided on the appropriate parameter to use with respect to audit circumstances. A set of results that clarifies the role of each GIS audit parameter in monitoring and evaluating GIS is summarized in Appendix A1.

5.2. Discussion

The framework provides audit content with respect to GIS key areas and competences. It presents the flow of audit processes and tasks required to deliver GIS audit results. The design addresses the accountability of GIS in terms of development, implementation and sustainability. Data audit provides a structure that implements data quality as fitness for use [72] . It considers imperfections of spatial data which is mainly expressed in terms of position or topology [34] . However, the description of spatial data quality is broader than these two aspects [73] hence the design is fostered on an audit that gives room for assessment of other

Table 6 GIS procedures audit framework design

	Goal	Audit Question	Audit Definition	Audit Method	Audit Frequency & Reports	Responsibility & Reporting
GIS strategic vision & alignment	Assess effectiveness of GIS strategy in fulfilling overall system goals and objectives	Are the planned initiatives consistent with overall strategy being accomplished by the GIS	Strategic plan Implementation plan Management plan Communication plan Standard operating procedures (SOPs) Business continuity plan	Review aspects of system utilization, governance and funding to identify current or future needs and directions. Engagements and feedbacks Observations e.g. effect of GIS staff leaving or coming in SWOT analysis	Quarterly basis	GIS Program Manager Senior management
Implementation	Assess GIS adoption & implementation within specified structure	Is the GIS achieving the intended goals and objectives	Activities schedule & workflows Statements of work Competency plan	Review quality of implementation through output examination	Daily to monthly	GIS Program Manager
Communication plan	Examine enhancement in information flow & collaboration within the GIS	What strategies exist to ensure information flow	Communication plan Feedback plan	Review feedback to ensure the communication is understood by stakeholders	monthly or quarterly	GIS Program Manager
Quality of system outputs (deliverables- products and services)	Certify the value of created products and services	Do the products and services comply to quality standards for the specified GIS applications	Quality management plan Quality control manual Performance plan Quality assurance program and quality rules Existing GIS data policies and standards	Validate data for compliance by tracking uncertainties, outliers, errors & anomalies Assess client’s satisfaction from their feedback	Daily production Weekly data reviews Data quality tests	GIS Program Manager
Data Creation and Update	Establish procedures for geospatial data & information that meets desired industry standards, rules and best practices	Is the collected data fit for purpose and does it fit industry standards?	Existing geospatial data creation policies and standards Data Dictionary (Metadata & database description) Documentations Quality controls	Established procedures for data creation Data update notifications Automated data update mechanisms	Data update intervals specific for each dataset type ● Imagery ● Topographic ● Cadastre ● Elevation ● Geodetic ● Administrative boundaries ● Attribute ● Metadata	GIS Program Manager
Data modification, dissemination, accessibility	Assess the safety of GIS information from unauthorized access, modification or deletion	Is there proper data dissemination and protection method and will sensitive data remain confidential?	Data modification & dissemination plans. Document protocols for confidential data Data user identity mechanism	Automated monitoring of data movements. Data surveillance through tracking. Data access control matrix & monitoring techniques Data watermarks	Daily controlled and restricted environment	GIS Program Manager Senior management
Data redundancy checks	Assess system performance risks and losses associate with redundant data and storage requirements	Does data redundancy affect GIS performance and storage requirements	A centralized database Data catalogued and graded Data storage repository Procedures on data categorization	Prioritize system data in terms of ready for use, ready to delete and potential elimination Review data completeness, accuracy, integrity, timelines and applicability	Weekly review of system GIS spatial database	GIS Program Manager
GIS technology: components, equipment and information sharing	Monitor effectiveness of the GIS technology that meets system goals	Does the GIS catching-up with the changing GIS technology and other existing systems integration mechanism	GIS components keep pace with technology Interoperability: concepts in terms of formats & standards required for data exchange & information sharing	Examine data, technology and workflows to determine current GIS technology trends & software upgrade Established procedures for data conversion for incompatible formats	Daily operations Timely software licensing/update	GIS Program Manager
Hardware and network infrastructure	Establish adequacy in hardware and network setup that provide best performance and meet expected GIS loads	Does the hardware and network configuration provides acceptable response time and a conducive working environment	Networks, Workstations and Servers Data storage and backup options	Hardware and network performance monitoring software Use network management protocol Monitor new developments in hardware and network performance through online research or vendors	Daily monitoring of hardware/network performance in terms of ● CPU cores ● CPU speed ● RAM ● Operating System ● Hardware and network response ● Hardware and network failures ● Storage space and backup options ● Severs	GIS System Engineer Network Administrator
GIS Standard Operating Procedures (SOPs)	Assess sound work methods that produces consistent products and services with least time possible and minimal errors	Are there existing SOPs that defines GIS operational requirements and do procedures conform to existing standards	Documented procedures GIS conformance to existing policies and standards Document Management System	Check documentation of procedures Check compliance of procedures to existing standards Measure work efforts and credibility Automate document management	Annual review and update of SOPs Review of conformance to standard operating Procedures during job production	GIS Program Manager
GIS support & maintenance	Examine support for GIS as a service for excellent delivery of products and services	Are the resources in good condition to run operations of GIS	GIS implementation & operation practices Risks mitigation measures Technical supports, After service warranties & support guarantee to customers	Review efficiency of operations through reduced time loss, maximum performance, increased profitability and broadened competitive edge	Timely, monitoring, upgrades & maintenance controls	GIS Program Manager System Engineer GIS developer Network Administrator GIS System administrator GIS programmer
GIS security	Review GIS quality and safety measures	What are the overall GIS security measures in place	Data security guidelines Data Privacy guidelines Access and usage controls Physical security Potable device policy	Review of: ● System access & usage risks ● Information uniqueness ● Physical security measures ● Data storage and backups ● Monitor logs for access control violation	Daily security assessment Daily data backups	GIS user GIS Program Manager
Resource Accountability	Assess resources accountability that supports the continuity of GIS	Are the resources used wisely and of the right choice and condition?	Resources inventory, Identified knowledge, roles & functions of GIS resources. Instrument calibration Firmware updates	Assessment of physical and internal condition of GIS resources Allocate staff responsibility for GIS resources Calibration & firmware update procedures	Routine evaluation Timely maintenance, update, calibration & servicing of equipment	GIS user GIS Program Manager
Staff development programs	Examine GIS personnel to build on existing knowledge and skills	What are the existing gaps in the GIS that calls for development programs	Training & staff development programs	Review changes & improvements on ● Technical work outputs ● Software use ● Staff attitude towards the GIS Measure work performance difference especially after staff transition	Routine evaluation of gaps that may require training or development programs	GIS Program Manager Senior management
Funding	Assess sufficiency of funds to support and sustain the GIS	Is there enough fund to cater for the GIS requirements	Initial capital & operating budget Funding decisions that compromise system performance	Review of cost and benefit analysis Review of procurement procedures	Monthly (Operational budget changes as system matures)	Senior management

components that constitutes GIS data quality. It is important to evaluate quality based on data information and the intended application/use to ensure compliance. GIS data is differentiated in terms of space, time, and theme [74] . The design includes each of these dimensions, aggregated into several components of data quality. Consideration is done to include accuracy, lineage, currency, coverage (adequacy & reliability), consistency, and completeness [73] .

Software utilization is centered on optimum utilization of a GIS software without compromising GIS processes and workflow efficiencies. GIS software and analytical tools are known to be of two general types; general-purpose geocomputation platforms such as ArcGIS and QGIS and specialized geo-computation tools Landserf [75] . However, exposure to GIS software for many GIS professionals is limited to ESRI and open-source software products [48] . These software have several analytical and GIS mapping tools in terms of their capabilities and functionalities [76] . It is important to evaluate software utilization based on GIS user’s application requirements [48] . The software is procedure-oriented and the GIS user needs to know the functionalities they need to perform specific tasks [75] . These are considered with respect to basic GIS users’ utilization and operational functionalities to include: Capture, conversion, transformation, querying, analysis, geoprocessing, interpolation, display, visualization, presentation, reporting, import, export, modeling, geodatabase creation & management, data exchange & sharing, plugins/extensions management, support for external remote connections, support for external OGC & web services, interoperability, customization and operation system requirements. This is done with the consideration that some GIS software is commercial and come with maintenance and licensing requirements.

Personnel competency design provides flexible measures of a GIS competency within a GIS environment. An effective GIS is dependent on quality, experience and training of its users. The design has taken into account appropriate GIS personnel with regard to their qualities and potential [77] . Necessary steps must be taken to ensure appropriate experience, training and continuous professional programs.

The success of audit results is dependent on six (6) major GIS stakeholders who play critical role in ensuring successful GIS audit as described in Table 7. These include Senior Management, GIS Manager, System engineer, Network engineer, GIS system administrator and GIS user. At the same time, client/consumer feedback provides opinions to enrich the audit. From the results, it is evident that a GIS manager has more responsibility for analyzing and reporting GIS audits as compared to other stakeholders. S/he is more integral to the audit and presents GIS knowledge application, skills and techniques needed to evaluate system needs and expectations [78] . GIS user is more dominant in data quality audit. In GIS practice, a GIS user actively participates in the processes of inclusion, change and manipulation of data [79] . For this reason, quality becomes a central issue to verify data correctness and consistency for the application at hand. GIS

Table 7 GIS audit major stakeholders

Audit Responsibility	Data Quality	Software Utilization	GIS Competency	Procedures	Occurrence
Senior management		1	3	4	8
GIS Program Manager	3	1	7	13	24
GIS user	6	1	1	2	10
GIS system Engineer			1	1	2
GIS Developer			1	1	2
GIS programmer			1	1	2
GIS System administrator				1	1
Network Administrator				1	1

system engineer, developer and programmer have equal roles. Their audit effort is concentrated on GIS competency and procedures that define required tasks to build or customize GIS application. System administrator and network administrator offer their contribution in parameters that support, troubleshoot and maintain GIS computers, servers and networks. The senior management features more in procedures that define the rationale through which audit results are effected for a successful GIS. Table 7 is a summation of stakeholder’s involvement based on their roles.

GIS procedures: A GIS operates based on a well-designed plan and business rules unique to each organization [47] . The parameters are designed to provide a strategy that determines the success of a GIS to avoid issues experienced during operation [80] . They align to workflows with respect to GIS strategic vision and alignments, implementation, communication, data creation, data modification, data dissemination, data accessibility, data update, data redundancy checks, GIS document management, GIS technology, hardware & network infrastructure, GIS standard operating procedures, GIS support & maintenance, GIS security, resources accountability, staff development programs and GIS funding.

6. Conclusion and Further Work

This paper has presented a GIS audit framework that allows GIS users to assess their GIS set-ups based on some identified GIS audit parameters. The design scope has been described to include different levels of GIS application areas within an enterprise. It also accommodates other levels of GIS development; department, project or society GIS. The design adopted a reductive model approach that constructs GIS audit guidelines and considerations from the identified parameters. Overall design strategy and audit process are indicated for each audit parameter. Major factors have been considered to ensure proper implementation, performance and sustainability of an effective GIS. GIS users are guided on how to approach GIS audit in terms of GIS data quality, GIS software technology, GIS competency and GIS procedures/operations. The framework inputs incorporate views received from selected GIS industry players on how and what they would consider as audit parameters. The framework provides a checklist of various aspects that need to be audited during GIS audit process. The implication is that, an auditor selects a particular aspect from the framework that the audit will focus on. It is available in an excel format to enable users create/develop checklists for various audit purposes and scopes. Further work will involve validating the framework through a mock testing process to ensure that the developed framework meets the needs and requirements of GIS stakeholders.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

Cite this paper

Njiru, F.M., Siriba, D.N. and Karanja, F.N. (2023) Developing a GIS Audit Framework in the Context of Information Technology though a Reductive Model Approach. Journal of Geographic Information System, 15, 196-222. https://doi.org/10.4236/jgis.2023.152011

AppendixAppendix A1: Summary of Results

References1

Pick, J. (2005) Geographic Information Systems in Business. IGI Global, Pennsylvania. https://core.ac.uk/download/pdf/33797586.pdf https://doi.org/10.4018/978-1-59140-399-9

Johnstone, B. (2018) GIS Capability Maturity Assessment: How Is Your Organization Doing? Spatial Vision Group, Canada.

Schwabe, C. (2015) The Simplicity and Complexity of Future GIS Applications. GISSA Gauteng, Johannesburg, 12 March 2015, 1-38.

Walter, C. (2020) Future Trends in Geospatial Information Management: The Five to Ten Year Vision. United Nations Committee of Experts on Global Geospatial Information Management, London.

Sumislawska, M., Burnham, K. and Mansour, S. (2020) Model Complexity Reduction. The University of Sheffield, Sheffield.

Alagic, A., Turulja, L. and Bajgoric, N. (2021) Identification of Information System Audit Quality Factors. Journal of Forensic Accounting Profession, 1, 1-28. https://doi.org/10.2478/jfap-2021-0006

CIT (2009) Technical Guide on Information System Auditing. Commitee on Information Technology, New Delhi.

Drljaca, D. and Latinovic, B. (2009) Framework for Audit of an Information System in Practice. Journal of Information Technology and Applications, 12, 78-85.

Carvalho, J.A. (2022) Strategies to Deal with Complexity in Information Systems Development. http://www3.dsi.uminho.pt/jac/SI/zdocumentos/complexity.pdf

Shabir, M., Kanwal, R.S. and Ali, M.I. (2020) Reduction of an Information System. Soft Computing, 24, 10801-10813. https://doi.org/10.1007/s00500-019-04582-3

Bechtel, W. (2001) The Compatibility of Complex Systems and Reduction: A Case Analysis of Memory Research. Minds and Machines, 11, 483-502. https://doi.org/10.1023/A:1011803931581 https://link.springer.com/article/10.1023/A:1011803931581

United Nations (2017) Theory of Change. United Nations Development Group.

Cherry, K. (2020) How Does Reductionism Work in Psychology? https://www.verywellmind.com/reductionism-definition-and-examples-4583891

Descartes, R. (2022) The Basics of Philosophy. https://www.philosophybasics.com/philosophers_descartes.html

Franklin, A. (2021) How the Reductionist Should Respond to the Multiscale Argument, and What This Tells Us About Levels. https://philsci-archive.pitt.edu/20187/1/Levels_Multiscale.pdf

Batterman, R. (2020) The Tyranny of Scales. In: The Oxford Handbook of Philosophy of Physics, Oxford University Press, Oxford, 1-36.

Rocca, E. and Anjum, R.L. (2020) Complexity, Reductionism and the Biomedical Model. In: Anjum, R.L., Copeland, S. and Rocca, E., Eds., Rethinking Causality, Complexity and Evidence for the Unique Patient, Springer, Cham, 75-94. https://doi.org/10.1007/978-3-030-41239-5_5

Andreev, D.V. (2020) The Use of GIS Technology in Modern Conditions. IOP Conference Series: Earth and Environmental Science, 421, Article ID: 042001. https://doi.org/10.1088/1755-1315/421/4/042001

Hussain, M.R. (2016) Essential Elements of Geographic Information System (GIS). Berlin.

Mazzocchi, F. (2020) The Limits of Reductionism in Biology: What Alternatives. E-LOGOS: Electronic Journal for Philosophy, 18, 1-29. https://doi.org/10.18267/j.e-logos.301

Clymer, J.R. (2017) Role of Reductionism and Expansionism in System Design and Evaluation. INCOSE International Symposium, 4, 300-309. https://doi.org/10.1002/j.2334-5837.1994.tb01720.x

Fehr, J., Eberhard, P. and Hanss, M. (2022) Institute of Engineering and Computational Mechanics (ITM). http://info.itm.uni-stuttgart.de/courses/model_reduction/Info_Sheets/M02.pdf

Janardhanan, S. (2022) Model Order Reduction and Controller Design Techniques. https://web.iitd.ac.in/~janas/courses/material/eel879/sp_topics_01.pdf

Mierzejowska, A. and Zogala, M. (2018) The Characteristics of Geographical Information Systems in Terms of Their Current Use. Journal of Water and Land Development, 39, 101-108. https://doi.org/10.2478/jwld-2018-0064

Min, G., Suh, E.S. and Otto, K.H. (2016) System Architecture, Level of Granularity, and Structural Complexity: Analysis and Observations. Journal of Mechanical Design, 138, Article ID: 021102. https://doi.org/10.1115/1.4032091

Mardjan, M. (2020) Reducing Complexity. https://nocomplexity.com/wp-content/uploads/2020/06/reducing_complexity.pdf

Lanjewar, P. and Momin, B.F. (2016) A Review: Parameter Reduction Methods in Soft Set Theory. 2019 International Conference on Communication and Electronics Systems (ICCES), Coimbatore, 17-19 July 2019. https://doi.org/10.1109/ICCES45898.2019.9002423

Li, A.-B., Li, S.-S. and Lv, G.-N. (2012) Disguise and Reduction Methods of GIS Vector Data Based on Difference Expansion Principle. International Workshop on Information and Electronics Engineering (IWIEE): Procedia Engineering, 29, 1344-1350.

Liu, G., Feng, Y. and Yang, J. (2020) A Common Attribute Reduction Form for Information Systems. Knowledge-Based Systems, 193, Article ID: 105466. https://doi.org/10.1016/j.knosys.2019.105466

Chakrabarty, A., Mannan, S. and Cagin, T. (2021) SciencweDirect. https://www.sciencedirect.com/topics/engineering/model-reduction

Davechand, R. (2020) Geographic Information System Concepts Simplified. Geospatial World.

Nisbet, R., Elder, J.F. and Miner, G. (2009) Model Evaluation and Enhancement. In: Handbook of Statistical Analysis and Data, Academic Press, Cambridge, 285-312. https://doi.org/10.1016/B978-0-12-374765-5.00013-9 https://www.sciencedirect.com/topics/computer-science/metamodeling

Friedenthal, S., Dori, D. and Mordecai, Y. (2022) System Modeling Concepts. https://www.sebokwiki.org/wiki/System_Modeling_Concepts

Bielecka, E. and Burek, E. (2019) Spatial Data Quality and Uncertainty Publication Patterns and Trends by Bibliometric Analysis. Open Geosciences, 11, 219-235. https://doi.org/10.1515/geo-2019-0018

Chen, P. and Shi, W. (2018) Measuring the Spatial Relationship Information of Multi-Layered Vector Data. International Journal of Geo-Information, 7, Article No. 88. https://doi.org/10.3390/ijgi7030088

Unearth (n.d.) GIS Software. https://www.unearthlabs.com/gis-software

Eldrandaly

,et al. (2007)GIS Software Selection: A Multi Criteria Decision Making Approach Applied GIS 3, 1-17.

Tholiya, J.J. and Narkhed, P.G. (2017) GIS as an Efficient Tool for Smart Cities. Smart Cities: Approaches, Opportunities & Challenges, Pune, 24-25 January 2017.

DiBiase, D., Corbin, T., Fox, T., Francica, J., Green, K., Jackson, J., Jeffress, G., Jones, B., Jones, B., Mennis, J., Schuckman, K., Smith, C. and Van Sickle, J. (2010) The New Geospatial Technology Competency Model: Bringing Workforce Needs into Focus. URISA Journal, 22, 55-72.

Wikle, T.A. and Fagin, T.D. (2015) Hard and Soft Skills in Preparing GIS Professionals: Comparing Perceptions of Employers and Educators. Transactions in GIS, 19, 641-652. https://doi.org/10.1111/tgis.12126

Ploński

,et al. (2019)Geospatial Technology Competency Model—Geography Academic Competencies Geography and Tourism 1, 63-76.

Evaluating Enterprise GIS Requirements. Esri. https://www.esri.com/news/arcuser/0205/evaluating1of2.html

USC (2021) Top GIS Technologies in 2021 & Beyond. https://gis.usc.edu/blog/top-gis-technologies/

The Upwork Team (2021) What Content-Based Filtering Is and Why You Should Use It. Upwork. https://www.upwork.com/resources/what-is-content-based-filtering#:~:text=Content%2Dbased%20filtering%20is%20a,them%20to%20a%20user%20profile

Spencer, J. and Wilkes, B. (2019) Assessing Spatial Data Quality: Using Five Data Anomalies. United States Agency for International Development (USAID), Washington DC.

Herbei, M.V. (2011) Topology of Spatial Data. 15th International SGEM GeoConference on Cartography and GIS, Albena, 18-24 June 2015.

Ali, E. (2020) Geographic Information System (GIS): Definition, Development, Applications & Components.

Kraemer, C. and Payne, K.A. (2019) A Review of Open Source GIS Web Mapping and Enterprise Solutions. ITOS, United States Agency for International Development, Washington DC.

Sugarbaker, L.J. Managing an Operational GIS. https://www.geos.ed.ac.uk/~gisteac/gis_book_abridged/files/ch43.pdf

Malbon, E. and Parkhurst, J. (2022) System Dynamics Modelling and the Use of Evidence to Inform Policymaking. Policy Studies. https://doi.org/10.1080/01442872.2022.2080814

Benner, P. and Feng, L. (2015) Model Reduction for Dynamical Systems. Max Planck Institute for Dynamics of Complex Technical Systems, Magdeburg.

Wade, J. and Heydari, B. (2014) Complexity: Definition and Reduction Techniques. Stevens Institute of Technology, Hoboken.

Kujala, S. (2003) User Involvement: A Review of the Benefits and Challenges. Behaviour and Information Technology, 22, 1-16. https://doi.org/10.1080/01449290301782

Ahmad, S.O. (2012) Questionnaire and Types. https://www.researchgate.net/publication/304607255_Questionnaire_and_Types

Thompson, J. (2021) What Makes a Good Audit? Financial Reporting Council, London.

Arfianti, R. (2021) International Standards on Auditing (ISA) Implementation in New Zealand. New Zealand Institute of Chartered Accountants, Wellington.

ISA (2022) International Standard on Auditing 300 Planning an Audit of Financial Statements. https://www.ifac.org/system/files/downloads/a016-2010-iaasb-handbook-isa-300.pdf

Zhang, H., Tan, H.B.K., Zhang, L., Lin, X., Wang, X., Zhang, C. and Mei, H. (2011) Checking Enforcement of Integrity Constraints in Database Applications Based on Code Patterns. Journal of Systems and Software, 84, 2254-2265. https://doi.org/10.1016/j.jss.2011.06.044

Cockcroft, S. (1997) A Taxonomy of Spatial Data Integrity Constraints. GeoInformatica, 1, 327-343. https://doi.org/10.1023/A:1009754327059

Chandramali, E.A.G. and Wijesekera, N.T. S. (2016) Field Data Collection for Land Information Systems—A Case Study on Quantification of Resource Requirement for a GIS. Engineer: Journal of the Institution of Engineers, 45, 51-64. https://doi.org/10.4038/engineer.v45i4.6926

Inspire (n.d.) Data Quality. https://inspire.ec.europa.eu/training/data-quality

Khan, S. and Mohiuddin, K. (2018) Evaluating the Parameters of ArcGIS and QGIS for GIS Application. International Journal of Advance Research in Science and Engineering, 7, 582-594.

Rus, I. (2015) Technologies and Methods for Auditing Databases. Procedia Economics and Finance, 26, 991-999. https://doi.org/10.1016/S2212-5671(15)00921-1

Holmes, J., Agius, C. and Crompvoets, J. (2020) SO/TC211: Standardisation of Geographic Information and Geo-Informatics. 3rd International Workshop on Spatial Data Quality (SDQ 2020), Valletta, 28-29 January 2020.

ISO (2002) Geographic Information—Quality Principles. https://www.sis.se/api/document/preview/903487

Onunga, J.O. (2020) Information Systems Audit Framework to Improve the Quality of Audit in Higher Education in East Africa. Open Journal for Information Technology, 3, 53-62. https://doi.org/10.32591/coas.ojit.0302.02053o

Musungu, K. (2015) Assessing Spatial Data Quality of Participatory GIS Studies: A Case Study in Cape Town. Joint International Geoinformation Conference 2015, Kuala Lumpur, 28-30 October 2015.

Mencik, D., Pehlivan, H., Aytekin, A. and Mencik, V. (2022) Software in Geographic Information Systems and Cloud GIS Software. International Informatics Congress (IIC 2022), Batman, 17-19 February 2022.

Sangam, S.L. and Mogali, S.S. (2012) Mapping and Visualization Softwares Tools: A Review. Content Management in Networked Environment, Tumkur, 3-4 December 2012.

Esri (n.d.) ArcGis Data Interoperability. https://www.esri.com/en-us/arcgis/products/arcgis-data-interoperability/supported-formats

Maguire, D.J. GIS Customisation. https://www.geos.ed.ac.uk/~gisteac/gis_book_abridged/files/ch25.pdf

Devillers, R., Gervais, M., Bédard, Y. and Jeansoulin, R. (2002) Spatial Data Quality: From Metadata to Quality Indicators and Contextual End-User Manual. OEEPE/ISPRS Joint Workshop on Spatial Data Quality Management, Istanbul, 21-22 March 2002.

Veregin, H. (1999) Data Quality Parameters. https://www.geos.ed.ac.uk/~gisteac/gis_book_abridged/files/ch12.pdf

Qiang, Y. and Van de Weghe, N. (2019) Re-Arranging Space, Time and Scales in GIS: Alternative Models for Multi-Scale Spatio-Temporal Modeling and Analyses. International Journal of Geo-Information, 8, Article No. 72. https://doi.org/10.3390/ijgi8020072

Zhu, A.-X., Zhao, F.-H., Liang, P. and Qin, C.-Z. (2021) Next Generation of GIS: Must Be Easy. Annals of GIS, 27, 71-86. https://doi.org/10.1080/19475683.2020.1766563

WHO (2028) Comparison of Geographic Information Systems (GIS) Software. WHO, Geneva.

Satka, E. (2017) Internal and External Audit in the Function of the Management of the Trade Companies. Journal of US-China Public Administration, 14, 330-338. https://doi.org/10.17265/1548-6591/2017.06.004

Ahmed, Z.Y. (2021) GIS Project Management. International Journal of Information Engineering and Electronic Business, 2, 11-24. https://doi.org/10.21608/ijaebs.2021.44226.1004

Medeiros, G. and Holanda, M. (2019) Solutions for Data Quality in GIS and VGI: A Systematic Literature Review. In: Rocha, á., Adeli, H., Reis, L. and Costanzo, S., Eds., New Knowledge in Information Systems and Technologies. WorldCIST’19 2019. Advances in Intelligent Systems and Computing, Vol 930, Springer, Cham, 645-654. https://doi.org/10.1007/978-3-030-16181-1_61

Eldrandaly, K.A., Naguib, S.M. and Hassan, M.M. (2015) Evaluation of Critical Success Factors for GIS Implementation Using Analytic Hierarchy Process. International Journal of Computing Academic Research (IJCAR), 4, 132-143.