Nowadays, software organizations are immersed in very competitive markets. This situation challenges organizations for paying special attention to the quality of products and services offered to consumers. Moreover, organizations should take into account the selection of the best resources aimed at impacting positively into their processes, knowing beforehand that resources and processes influence the quality of their products and services.

In this direction, the ISO 25010 standard [1] states that: “the software lifecycle processes (such as the quality requirements process, design process and testing process) influence the quality of the software product and the system. The quality of resources, such as human resources, software tools and techniques used for the process, influence the process quality, and consequently, influence the product quality. Software product quality, as well as the quality of other components of a system, influences the quality of the system. The system quality has various influences (effects) depending on the contexts of use. The context of use can be defined by a set of a user, a task, and the environment”. Figure 1 illustrates these relationships as per ISO.

Consequently, software organizations that perform quality assurance activities should have a well-established quality evaluation and improvement approach in order to fulfill measurement, evaluation, analysis and change project goals. Project goals, particularly, information need goals may have specific purposes such as to understand the current quality state of an entity, to compare different entities for knowing their strengths and weaknesses, to improve the quality of an entity, to select the most suitable entity among a set of alternatives, etc.

Regarding the above, this article discusses the evaluation and improvement of an entity, particularly, a resource, using a holistic quality evaluation and improvement approach [2] . The architecture of this approach is based on two pillars, namely: 1) a quality multi-view modeling framework; and 2) ME/MEC integrated strategies. The first pillar specifies an ontology of quality views including terms such as entity category and quality focus and their influences and is determined by relationships as shown in Figure 1. This ontology allow us to represent quality views, for instance, the Resource Quality View, the Process Quality View, etc. in a more formal way than in [1] .

The second pillar provides a set of ME/MEC strategies, which helps software quality assurance (SQA) leaders to get data and information for analysis and decision-making in addition to achieve the information need goals for projects. Strategy is a frequently used and broad term, so for our purposes, we have defined it as: “principles, patterns, and particular domain concepts and framework that may be specified by a set of concrete processes, in addition to a set of appropriate methods and tools as core resources for helping to achieve a project goal” [3] . Furthermore, we have conceived that a strategy [3] [4] can be represented with three capabilities, namely: (1) the ME/MEC domain conceptual base and framework; (2) the process perspective specifications; and, (3) the method specifications. These three capabilities support the principle of being integrated [5] since, for instance, the same ME terms are consistently used for activities and methods.

Additionally, in the last decade we have earned experience in developing a couple of specific ME/MEC strategies. For instance, we have developed the GOCAME (Goal-Oriented Context-Aware Measurement and Evaluation) and SIQ in U (Strategy for Improving Quality in Use) strategies, in 2008 and 2010 respectively. These strategies were applied in several concrete evaluation and improvement projects such as documented in [4] [6] - [8] . For these ME/MEC projects, one or two quality views were considered. Also, both strategies have the three above-mentioned capabilities, which are supported in an integrated way.

On the other hand, we have envisioned the idea of packaging the earned experience in using ME/MEC strategies into strategy patterns. It is recognized that patterns have had and continue to have a significant impact in software and web engineering [9] - [11] . In a nutshell, the pattern’s main aim is to provide a general and reusable

solution to a recurrent problem. We have observed that strategy patterns can be applied to recurrent ME or MEC problems/goals of any project. As a result, we specify a set of strategy patterns in a catalog that offers flexible and tailorable solutions for evaluating and improving the quality focuses for different entities in ME/MEC projects.

There are different categories of patterns documented since early 90’s, for example, object-oriented design patterns [10] , usability and architectural patterns [12] , analysis patterns [11] [13] , among others. However, strategy patterns have recently been documented in [14] . Specifically, a ME/MEC strategy pattern includes in its structure a set of generic ME/MEC process and method specifications to be instantiated for a particular purpose and context. The patterns’ catalog and a potential recommender tool may support to the SQA leader in the selection process. This means that for a particular ME/MEC project goal (which embeds in its statement the quality focus and the quality views), the selection of the most appropriate strategy pattern to be instantiated can be eased.

This paper is motivated by the fact of showing the evaluation and improvement process made on the GOCAME strategy throughout the time by using strategy patterns. Thus, GOCAME was considered a resource from the entity category standpoint, with an evaluation focus on the quality of its three capabilities. Particularly, we will illustrate the employment of two strategy patterns, namely: GoME_1QV (Goal-oriented Measurement and Evaluation for One Quality View) and GoMEC_1QV (Goal-oriented Measurement, Evaluation and Change for One Quality View) in two periods of the timeline as depicted in Figure 2.

As above mentioned, we have developed the GOCAME strategy in 2008. So later, at early 2009, we have decided to understand the quality of its capabilities compared with other existing integrated strategies. To this aim, we first instantiated the GoME_1QV strategy pattern. Using this pattern allowed us to understand and analyze the strengths and weaknesses that GOCAME had in comparison with other strategies. In that case study [5] , the purpose of the ME information need was to “understand and compare” the “Capability Quality” from the “SQA leader” user viewpoint. We have defined the Capability Quality characteristic as the “degree to which a resource is suitable and appropriate for supporting and performing the actions when used under specified conditions”. The entity category is an “Integrated ME Strategy” whose super-category is “Resource”. Also, GOCAME and GQM⁺ Strategies [15] [16] were the two concrete entities to be assessed for the “Resource Quality” focus. This focus embraces the three required capabilities of an integrated strategy, which are represented by the “Process Capability Quality”, “Conceptual-Framework Capability Quality”, and “Methodology Capability Quality” sub-characteristics. The yielded results allowed us to understand their strengths and weaknesses. So in 2010, we saw the opportunity to plan actions for further improvements after analyzing the GOCAME weaknesses and some strengths of GQM⁺ Strategies that could be taken.

Regarding this, we have performed two cycles of improvements and re-evaluations, from middle of 2010 to early 2014, by instantiating the GoMEC_1QV strategy pattern. Improvements were achieved by performing evaluation-driven changes in the resource, i.e., in the GOCAME strategy. Therefore, the purpose of the ME information need was “Improve” from the same user viewpoint as before. Regarding the first improvement cycle attributes of the “Process Capability Quality” characteristic, which had benchmarked with low performance indicators, were changed and then re-evaluated to gauge the improvement gain. The second cycle was made as shown in Figure 2, where the major improvement was focused on attributes of the “Conceptual-Framework Capability Quality” characteristic, as we will discuss later on.

Hence, the main contribution documented in this paper is the instantiation of two strategy patterns for the evaluation and improvement of GOCAME with regard to the quality of its three capabilities.

Following this Introduction, Section 2 describes related work addressing research that deals with holistic evaluation and improvement approaches, and strategy patterns as well. Section 3 gives a background for understanding our holistic evaluation approach. Also, it shows the relations between Resource Quality View with others quality views. Section 4 discusses ME/MEC strategy patterns in general addressing the specification of GoMEC_1QV in particular. Then, Section 5 illustrates the GOCAME case study instantiating both GoME_1QV and GoMEC_1QV strategy patterns. Finally, Section 6 draws our main conclusions and outlines future work.

In this paper, we outline and exemplify a holistic quality evaluation and improvement approach whose architecture is based on two pillars, as commented above: 1) a quality multi-view modeling framework; and 2) ME/MEC integrated strategies. Additionally, a set of strategy patterns for this approach are introduced and two are instantiated with the aim of improving an organizational resource. So, we analyze below the state-of-the-art research literature with these three concerns in mind, i.e., the two integrated pillars and strategy patterns.

Regarding the first pillar of the approach, there exists research that deals with quality views and quality models. But as far as we know there is no work defining and specifying an ontology of quality views, nor an explicit glossary of terms. One of the most relevant documents previously cited is the ISO 25010 standard, in which different quality views and their “influences” and “depends on” (or “is determined by”) relationships are informally represented (see Figure 1). Also, the explicit meaning of the quality view concept is missing. Moreover, there is no clear association between a quality focus and an entity category, nor explicit definitions of the different entity categories as we do in Table 1. Rather, ISO 25010 outlines views in the context of a system quality lifecycle model, where some views can be evaluated by means of the quality model that the standard proposes.

Another initiative related to quality views is analyzed in [17] in which just the “influences” relationship between External Quality and Quality-in-Use (QinU) characteristics is determined by means of Bayesians networks, taking as reference the ISO 9126-1 [18] standard (this standard was superseded by [1] ). However, it does not discuss a holistic quality evaluation approach that links quality views with ME/MEC strategies, as we are proposing. Finally, in [7] the 2Q2U (internal/external Quality, Quality in use, actual Usability, and User experience) quality framework is proposed. This framework extends the quality models defined in [1] adding new

sub-characteristics for EQ and QinU, and considers the “influences” and “depends on” relationships for three quality views, namely: Software Product, System and System-in-Use Quality Views. But there is no explicit quality view component specified, as documented in [2] and in the next Section.

Regarding the second pillar, i.e., ME and MEC integrated strategies, there exists a couple of related work. For example, Goal Question Metric (GQM) [19] , Continuous Quality Assessment Methodology (CQA-Meth) [20] , Practical Software Measurement (PSM) [21] , and Quality Improvement Paradigm (QIP) [22] . Particularly, [15] [16] presents GQM⁺ Strategies, which is built on top of the so-called GQM strategy. Both strategies include the principle of the three integrated capabilities of a strategy [5] . But none consider the quality views’ concepts and the “influences” and “depends on” relationships, nor the ME/MEC strategy pattern idea.

Summarizing the approaches, CQA-Meth is a flexible methodology that allows the quality assessment of any software model. This methodology and its tool are part of the CQA integrated environment that can be used by companies to perform quality assessments of their own or third-party products. CQA-Meth defines the processes necessary to carry out the evaluation of UML models, and facilitate communication between the client (sponsor of the evaluation) and the evaluation team. CQA-Meth lacks an explicit conceptual framework from a terminological base standpoint. While CQA-Meth comes from the same research group who developed the FMESP (Framework for the Modeling and Evaluation of Software Processes) approach [23] which does have a conceptual framework with an ontological base as indicated by authors in [20] , the relationship among the three capabilities is not explicit at all.

Other measurement approach widely accepted in the industry that helps manage software development projects is PSM. It is an information-oriented approach that describes a software measurement process, also being part of a comprehensive management program and software development project management. PSM describes how to define and integrate measurement requirements, collect and analyze measurement data and implement the entire measurement process in an organization. PSM was one of the sources for the development of the ISO/IEC 15939standard [24] (as indicated in http://www.psmsc.com/iso.asp), and after the formal appearance of this document, PSM was updated according to this standard as well. Another related work is the QIP paradigm, whose premise is that improvement is a continuous process. This approach helps organizations to implement a continuous improvement process taking into consideration past experiences. Therefore, QIP is beneficial in mature organizations that are aware of their learning processes and experiences. QIP uses GQM for defining goals and appropriate metrics that will guide the implementation of the process. While this paradigm is consolidated as a continuous improvement approach, it does not offer an integrated guide for measurement, evaluation and change projects. Moreover, the three capabilities of most of these strategies are not well specified in an integrated way.

Lastly, regarding the third concern, a lot of research deals with patterns. There is plenty literature about design patterns [10] , analysis patterns [11] [13] , architectural and usability patterns [9] [12] , language patterns [25] , among other categories and issues. But this literature is, to the best of our knowledge, not intended to measurement, evaluation and improvement processes and stages in which quality views and ME/MEC strategy patterns could be used accordingly. For example, authors in [9] [12] define a framework that expresses relationships between Software Architecture and Usability (SAU). This proposal consists of an integrated set of design solutions that had been identified in various industry cases. However, in our opinion, a clear separation of concerns among quality views, quality models, ME/MEC integrated strategies and strategy patterns is missing.

As indicated above, the architecture of our holistic quality evaluation and improvement approach is built on two pillars. Sub-section 3.1 discusses the first pillar, i.e., the quality multi-view modeling framework, which specifies the proposed ontology of quality views and the grouping of its concepts into the quality_view component. Sub-section 3.2 analyzes, as part of the second pillar, what an integrated strategy for the purpose of evaluation and improvement is.

In Software Engineering, a well-known definition for pattern is “each pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice” [10] . That is, a pattern provides a documented and tested solution for recurring problems in similar contexts. Thus, we have observed that strategy patterns can be applied for ME/MEC projects’ recurrent problems. Strategy patterns offer flexible and tailorable solutions for evaluating and improving the quality focuses for different entities in ME/ MEC projects. The selection of a suitable pattern is made taking into account the goal and context of the project in conjunction with the intervening view or views, as seen in Figure 4.

We have specified a set of strategy patterns, following to a some extent the pattern specification template used in [10] . Our template includes the following items: 1) name: A descriptive and unique name, usually expressed in English; 2) alias: Acronym or other names for the pattern; 3) intent: Main objective for the pattern; 4) motivation (problem): Project problem/goal solved by the strategy pattern; 5) applicability: Situations in which the pattern can be applied; 6) structure (solution): Generic structure and instantiable solution that the strategy pattern offers; 7) known uses: References of real usage; 8) scenario of use: Concrete example and illustration for the instantiated pattern. In the sequel, a couple of strategy patterns are commented, describing only their intention and known uses.

GoME_1QV is a strategy pattern used to provide a solution in the instantiation of a ME strategy aimed at supporting just an understanding goal when one quality view is considered (e.g., Resource Quality View, Software Product Quality View, System Quality View, etc.). This strategy pattern must be selected when the project goal is just to understand the current situation of an entity with regard to the corresponding quality focus. The generic process of GoME_1QV consists of six activities, which are the A1-A6 gray-colored activities in Figure 7. This is the simplest pattern to be instantiated and the mostly used in ME projects we have run, e.g., in the evaluation of a mash-up application [7] and a shopping cart [4] . This pattern was also used to evaluate and compare GOCAME and GQM⁺ Strategies in 2010 (recall Figure 2). Also, its application will be commented in sub-section 5.1.

The GoMEC_1QV strategy pattern is applied when the project goal states that it is necessary not only to understand the current situation of the entity at hand but also to perform changes on it, re-evaluate it, and gauge the improvement gain achieved. It embraces eight generic activities, i.e., the A1-A8 activities in Figure 7. This pattern is instantiated for a MEC project goal considering just one quality view. We have illustrated GoMEC_1QV in a case study for improving the Usability of the Facebook app [14] , where the Usability characteristic was linked to the External Quality focus. This pattern is thoroughly specified next, and then instantiated in sub-sec- tion 5.2 for the Resource Quality focus.

GoMEC_2QV is another strategy pattern included in the catalog, which gives a solution for an improvement project goal which involves two quality views and their relations. Recall that between two quality views, the ‘influences’ and ‘depends on’ relationships can be used. This implies that one quality view plays the role of

dependent view, while the other plays the role of independent view (see these roles in Figure 5). For example, if we consider the System Quality View and the System-in-Use Quality View, these relations embrace the hypothesis that evaluating and improving the EQ focus of a system is one means for improving the QinU focus of a system in use [1] . In turn, understanding QinU problems may provide feedback for deriving External Quality attributes that if improved could impact positively in the system quality. Furthermore, we can envision valid and interesting relationships for instance between Resource Quality View and Product Quality View. That is, by evaluating and improving the resource quality can be one means for improving the Internal Quality focus of a product. For example, changes in the development team can impact positively in the architectural design. SIQinU strategy [6] is an instance of this pattern. This strategy was used as case study in a real software testing enterprise with headquarters in Beijing, which examined JIRA (www.atlassian.com/software/jira/), a commercial software defect tracking system.

Below, the GoMEC_1QV pattern is specified considering a template with the items listed above.

Name: Goal-oriented Measurement, Evaluation and Change for One Quality View

Alias: GoMEC_1QV or GOCAMEC_1QV in [14]

Intent: To provide a solution in the instantiation of a measurement, evaluation, analysis and change strategy aimed at supporting a specific improvement goal of a project when one quality view is considered.

Motivation (Problem): The purpose is to understand the current situation of a concrete entity in a specific context for a set of characteristics and attributes related to a given quality focus and then change the entity and re-evaluate it in order to gauge the improvement gain, through the systematic use of measurement, evaluation, analysis and change activities and methods.

Applicability: This pattern is applicable in MEC projects where the purpose is to understand and improve the quality focus of the evaluated entity for one quality view, such as Resource, System, System-in-Use Quality Views, among others.

Structure (Solution): The pattern structure is based on the three capabilities of an integrated strategy viz, the specification of the conceptual framework for the MEC domain, the specification of MEC process perspectives, and the specification of MEC methods. GoMEC_1QV provides a generic course of action that indicates which activities should be instantiated during project planning. It also provides method specifications for indicating how the activities should be performed. Specific methods can be instantiated during scheduling and execution phases of the project. Below, we describe the main structural aspects of the three strategy capabilities.

1) The concepts in the non-functional requirements, context, measurement, evaluation, change, and analysis components (Figure 5 and Figure 6) are defined as sub-ontologies. The included terms, attributes and relationships belong to the MEC area. In Figure 6 we show just the main ME terms. Note that ME terms in Figure 6 are also enriched with terms from a generic process ontology [3] by means of stereotypes. These concepts are used consistently in the activities, artifacts, outcomes and methods of any ME/MEC strategy.

2) The process specification is made up from different perspectives, i.e., functional which includes activities, inputs, outputs, etc.; behavioral, which includes parallelisms, iterations, etc.; organizational, which deals with agents, roles and responsibilities; and informational, which includes the structure and interrelationships of artifacts produced or consumed by activities. Considering the functional and behavioral perspective, Figure 7 depicts the generic process for this pattern. The names of the eight (A1-A8) MEC activities must be customized taking into account the concrete quality focus to be evaluated.

3) The method specification indicates how the descriptions of MEC activities must be performed. Table 3 and Table 4 exemplify three method specification templates: the first for a direct metric used as method specification for direct measurement tasks; the second for an indirect metric, used in indirect measurements; and the third for an elementary indicator, used in elementary evaluations. Note that terms in method specification templates come from the ME conceptual base. Many other method specifications can be envisioned such as task usage log files, questionnaires, aggregation methods for derived evaluation, amongst others. For change activities traditional methods such as refactoring, re-structuring, re-parameterization, document updating, among others can be specified as well.

Known uses: GoMEC_1QV was used in a MEC project devoted to improve Usability and Information Quality attributes of a shopping cart, i.e, from the System Quality View through refactoring as change method [8] . Besides, this pattern was instantiated in a MEC project for the Resource Quality View [5] .

As indicated in the Introduction Section, this paper is motivated by the fact of describing the evaluation and

improvement process made on the GOCAME strategy throughout the time. To this aim, we will illustrate the employment of two strategy patterns following the timeline shown in Figure 2.

As commented in the Introduction Section, we have developed in 2008 an integrated ME strategy so-called GOCAME which relies on three capabilities: the C-INCAMI conceptual frame work, the ME process perspective specifications, and the method specifications. At that moment, after its development, we decided to understand the quality of this resource and compare it with well-established strategies such as GQM⁺ Strategy. In Section 4, we have thoroughly discussed a strategy pattern that is able to support evaluation-driven changes. We think that strategy patterns can be conceived as a new category of patterns, likewise analysis or design patterns are dealt as categories too.

Particularly, in Section 5, we have illustrated the instantiation of the GoME_1QV and GoMEC_1QV strategy patterns for a resource quality view. This resource is the GOCAME strategy, which can be used to support SQA activities. We have specifically analyzed the improvement of the GOCAME strategy after applying two improvement cycles, which drew a global gain of 7.39 points. Basically, the gain in the Capability Quality characteristic was achieved by changing some Process Capability Quality and Conceptual-Framework Capability attributes.

For the sake of concluding, a ME/MEC strategy pattern is a way of packaging general and reusable solutions for common and recurrent measurement, evaluation and change/improvement problems or situations for specific projects’ goals. Moreover, according to the project goal and the amount of involved quality views a strategy pattern should be selected and then instantiated.

As future lines of research, we envision the development of a strategy pattern recommender system as a practical use of the quality views ontology (illustrated in sub-section 3.1) in the context of our holistic quality evaluation approach. The recommender system can be useful when an organization establishes ME/MEC project goals. Hence, taking into account the type of project goal and the amount of involved quality views, the strategy pattern recommender system will suggest the suitable pattern that fits that goal. Additionally, we started to work on a conceptual base that links ME information needs with business goals, in order to be able to develop strategies that support goals at different organizational levels, such as operational, tactical and strategic. It is important to remark that this multi-level goal feature was not assessed in this study, even though it was an existing feature in the GQM⁺ Strategy approach.

Lastly, as mentioned in sub-section 3.2 an integrated strategy simultaneously supports three capabilities, being one of them the ME conceptual base and framework. We have discussed in [3] [4] [33] the rationale for introducing or adapting some terms from ISO 15939 [24] . The terminology in [24] was widespread used in the new series of SQuaRE (Software Product Quality Requirements and Evaluation) documents and in many works such as [20] [23] [24] , to quote just a few. Considering this issue, we will review our ontology in order to evaluate the terminological adherence to SQuaRE documents.

We thank the support given by Science and Technology Agency of Argentina, in the PICT 2014-1224 project at UNLPam. Also, Belen Rivera thanks the support given by the co-funded CONICET and UNLPam grant.

Maria Fernanda Papa,Belen Rivera,Pablo Becker,Luis Olsina, (2016) Evaluation and Improvement of an Organizational Resource Applying Strategy Patterns. Journal of Software Engineering and Applications,09,128-146. doi: 10.4236/jsea.2016.94011