HealthHealth1949-4998Scientific Research Publishing10.4236/health.2015.712183Health-62102ArticlesMedicine&Healthcare Fidelity of Intervention Implementation: A Review of Instruments arahIbrahim1*SourayaSidani2Arthur Labatt School of Nursing, University of Western Ontario, London, CanadaDaphne Cockwell School of Nursing, Ryerson University, Toronto, Canada* E-mail:sibrah25@uwo.ca(AI);091220150712168716954 November 2015accepted 19 December 22 December 2015© Copyright 2014 by authors and Scientific Research Publishing Inc. 2014This work is licensed under the Creative Commons Attribution International License (CC BY). http://creativecommons.org/licenses/by/4.0/

Background: Interventions, whether simple or complex, are increasing in health care in response to the growing complexity and acuity of patient’s conditions. Monitoring the fidelity of implementing interventions is challenging. A common method to assess and monitor fidelity of intervention implementation is through a structured, reliable and valid instrument. Purpose: The purpose of this paper is to examine existing instruments measuring fidelity of intervention implementation in order to determine aspects of fidelity that have been assessed and reported on the reliability and validity of these instruments. Design: A descriptive review was conducted. Studies were included if they described and reported on the fidelity of intervention implementation instruments, their psychometric properties were published between 1980 and 2015. Methods: Data were extracted on the study characteristics, levels and aspects of fidelity and the psychometric properties, specifically the reliability and validity of the fidelity of intervention implementation instruments. Results: In total, 21 studies were included in the review. Overall results showed that some aspects and levels of fidelity of intervention implementation are included in the instruments. At the theoretical level, fidelity of intervention implementation is not accounted for majority of the studies and few explicitly reports on the use of instruments to evaluate intervention differentiation. At the operational level, interventionists’ adherence and competence are included in the instruments; however, participants’ engagement, exposure and enactment are not. The instruments demonstrate acceptable level of validity and reliability. Conclusion: Sustained focus on developing psychometrically sound instruments that account for all levels ( i.e. theoretical and operational) and aspects of fidelity of intervention implementation is imperative to strengthen the methodological literature for interventions research; and for researchers to correctly interpret research findings and to arrive at valid conclusions on the effectiveness of interventions, whether simple or complex.

Interventions Intervention Fidelity Intervention Fidelity Instrument1. Introduction

Historically, intervention fidelity received little attention in intervention research because of the assumption that interventions were delivered in a standardized and consistent manner by interventionists who strictly followed the treatment protocol and manual [1] . However, concerns over intervention fidelity arose in various fields of research, most notably in psychotherapy, as brought forth by Eysenck (1952) [2] . Eysenck critiqued the vague descriptions provided for psychotherapy treatments in the 1960s and the report on overall effectiveness, disregarding evidence on the contribution of different components and actual implementation of treatments [3] . In the 1970s, a shift in intervention research emerged with an emphasis on collecting data on the implementation of treatment. This was done as a means of ensuring the interventionists adhered to the treatment protocol and determining whether contamination took place, that is, the extent to which the experimental treatment was disseminated to the control arm of the study. This emphasis on examining the implementation of the experimental and comparison treatments occurred at a time when there was ambiguous information and descriptions of the nature and dose of treatments; this in turn, affected the ability to replicate interventions and to reach valid conclusions on their effects [4] .

Over the past three decades, there has been growing recognition of the importance to monitor and assess fidelity of implementing health interventions [5] . Intervention fidelity, also referred to as implementation fidelity and treatment integrity in the literature, is the competent and reliable delivery of an intervention as intended in the original design [6] - [8] . This ensures the intervention is carried out in the selected dose and mode to initiate the mechanisms that are responsible for producing the desired changes in the outcomes [1] .

Intervention fidelity is conceptualized at two levels: theoretical and operational. At the theoretical level, intervention fidelity is related to the process of developing and designing an intervention. It refers to the correspondence between the intervention’s active ingredients and its components and activities. The active ingredients are identified in the intervention theory as the elements that characterize the intervention and are responsible for producing the changes in outcomes. The active ingredients are reflected in the components and activities comprising the intervention. At the operational level, intervention fidelity refers to the degree to which the intervention is delivered according to the original design and plan. The interventionists’ performance in delivering the intervention and the client’s exposure, engagement, and adherence to the intervention are necessary for ensuring successful implementation of its components and activities and in turn, effectiveness in instigating the desired changes [1] .

Intervention fidelity affects the external and internal validity and statistical conclusions in intervention research [1] [9] . Specific to external validity, a clear and detailed description of the intervention’s active ingredients, components, activities and mode and dose of delivery, and protocol for carrying out the intervention allow for reproducibility of the intervention [4] . Specific to internal validity, the lack of information on the fidelity of intervention implementation impedes the ability to know whether the effects are due to the intervention itself or to a Type III error, that is, failure to implement the intervention as planned [10] . Specific to statistical conclusions, variations in the delivery of an intervention result in differences in participants’ exposure to the intervention components and dose, leading to variability in outcome achievement. Such variability inflates error variance in posttest outcomes and decreases the statistical power to detect significant effects, potentially leading to incorrect conclusions about the effectiveness of the intervention [1] [9] [11] [12] . In general, failure to monitor and assess fidelity of intervention implementation precludes researchers from concluding what was actually responsible for the significant or non-significant effects [6] [13] .

A common strategy to monitor and assess for fidelity of intervention implementation is through a structured, valid and reliable instrument. The development and use of such instruments to assess the quality of the implementation of an intervention has been widely accepted in health intervention research. Although efforts have been made to develop instruments to assess fidelity of intervention implementation, limited research has been conducted on identifying the aspects of fidelity that are captured in these measures. In this descriptive review, we examined existing instruments measuring fidelity of intervention implementation to determine aspects of fidelity that they assess, and reported on the reliability and validity of these instruments.

2. Methods2.1. Selection Criteria

Studies were included in the review if they met these criteria: original research study reporting on an instrument measuring fidelity of intervention implementation and its psychometric properties, published in the English language in peer-reviewed journals, dissertations or theses, between 1980 and 2015. The start date of 1980 provided a time period for publication of relevant papers, following the emphasis in the 1960s and 1970s on data collection, assessment, and reporting of intervention fidelity [4] .

2.2. Search Strategies

The databases used to identify the literature were: Health and Psychological Instruments (HAPI), Cumulative Index to Nursing and Allied Health Literature (CINAHL), Medline, Educational Resources Information Center (ERIC), Web of Science, Proquest Dissertations and Theses, and Google Scholar. The following keywords and Boolean operators were used to combine and refine the searches: (“fidelity” OR “integrity” OR “adherence” OR “implementation fidelity” OR “fidelity to treatment” OR “intervention fidelity” OR “intervention integrity” OR “intervention adherence”) AND (tool* OR instrument* OR questionnaire OR survey*) AND (valid* OR reliab*) AND (measure* OR evaluat* OR assess*). Also, reference lists of the selected articles were reviewed to identify additional publications.

2.3. Data Extraction

Data were extracted from full papers on study characteristics, aspects of fidelity assessed by the instruments and psychometric properties of the instruments. Information presented in relevant sections of the papers was summarized and coded by the authors. Any difference was resolved through consensus. High level of agreement (> 80%) was attained.

2.3.1. Study Characteristics

The following information specific to the study characteristics was extracted: (a) author’s last name and year of publication; (b) discipline; (c) target population; (d) type of intervention under evaluation; and (e) number of components comprising the intervention.

2.3.2. Aspects of Fidelity Assessed

Information was gathered on elements of theoretical and operational fidelity assessed and on strategies or items used to conduct the assessment. The information was derived from relevant methodological literature [1] [6] [13] .

Two strategies have been proposed to examine theoretical fidelity. The first is applied when designing the intervention and consists of generating a matrix to link the intervention’s active ingredients with its components and activities; the matrix forms the basis for developing the items for measuring the implementation of the intervention’s activities [1] . The second strategy refers to intervention differentiation [6] ; it involves the use of the items to monitor the performance of the specified activities when implementing the intervention and the non- engagement in these activities in the control or comparison group.

Operational fidelity is assessed at two levels: interventionist and participant. For the interventionists, two elements of operational fidelity are examined: adherence and competence [4] [10] [14] . Adherence refers to the degree to which the interventionists carried out the intervention in a way that is consistent with the original design and plan as delineated in the treatment protocol and manual [1] [8] [14] . To assess interventionists’ adherence, the instruments should contain a list of activities to be performed and allow documentation of the extent to which they are being followed during intervention implementation [15] . Competence refers to the extent to which the interventionists possess the skills and knowledge required to deliver the intervention [16] .

For participants, the elements of operational fidelity are: exposure, engagement and enactment. Exposure refers to the extent to which the participant is in contact with the intervention’s content. Exposure is often documented as the number of intervention sessions attended and duration of each session. Engagement is the extent to which the participants are involved in the intervention activities and captured through participants’ self-report and or interventionists’ observation of the activities completed during the intervention sessions (e.g. participation in group discussion). Adherence refers to the extent to which participants apply the activities or recommendations in the context of daily life such as exercising for 30 minutes five times per week [1] [6] .

2.3.3. Psychometric Properties

The psychometric properties of instruments measuring fidelity of intervention implementation were evaluated using the methodological framework of Streiner and Norman (2008) [17] . Reliability demonstrated the ability of an instrument to yield consistent and reproducible results [18] [19] . Three types of reliability were examined in this review: test-retest, inter-rater, and internal consistency. Validity refers to the extent to which an instrument measures what it is intended to measure [19] [20] . Two aspects of validity of the instruments were examined: construct and content.

2.4. Data Analysis

The data pertaining to the study characteristics, identification of the aspects of fidelity and the psychometric properties of the instruments were analyzed descriptively using the Statistical Package for the Social Sciences (SPSS) Version 22.

3. Results3.1. Literature Search

The literature search yielded a total of 104,143 titles and abstracts (Figure 1). All abstracts were reviewed and 104, 117 were excluded because they did not meet the selection criteria; nine articles were duplicates. A total of 20 articles were selected for full review, and of these, 18 met all selection criteria. A hand search of the reference lists of the selected articles yielded three additional articles for full review. A total of 21 publications were reviewed.

3.2. Study Characteristics

The studies were published between 1996 and 2015. The reported disciplines included: nursing (n = 4, 19%), psychology (n = 4, 19%), psychiatry (n = 3, 14.2%), social work (n = 1, 4.8%), public health (n = 1, 4.8%), behavioral medicine (n = 1, 4.8%), rehabilitation science (specifically Occupational Therapy) (n = 2, 9.5%), education (n = 3, 14.3%), and health services (n = 2, 9.5%). The target population included: persons living with addictions (n = 4, 19%), pre-mature infants (n = 1, 4.8%), children (n = 4, 19%), older adults (n = 1, 4.8%), persons living with mental health challenges (n = 5, 24%), persons living with physical health challenges (n = 2, 10%), homeless persons (n = 1, 4.8%); couples (n = 1, 4.8%), parents of lower socioeconomic status (n = 1, 4.8%), and teachers providing social-emotional competence and behavioral support (n = 1, 4.8%). The type of interventions comprised of two or more components, such as educational (i.e. provision of information on addictions, mental health, and child safety) and behavioral (i.e. targeting behaviors related to addictions) and included: psychotherapy (n = 3, 14.3%); psycho-education (n = 1, 4.8%); cognitive behavioral therapy (CBT) (n = 2, 9.5%); emotion-focused therapy (n = 1, 4.8%); educational (n = 1, 4.8%) behavioral therapy (n = 6, 28.9%);

Flow diagram of literature selection

prevention-related (i.e. hospital related functioning decline among older adults) (n = 4, 19%); and technology (n = 3, 14.3%).

3.3. Assessment of Intervention Fidelity

To examine theoretical fidelity, the authors of all studies (n = 21, 100%) generated a matrix to construct the items measuring fidelity. The active ingredients of the interventions were identified through various means, separately or in combination: experts, literature and review of the treatment protocol. In addition the content of the items was validated in majority of the studies (as reported in a later section). However, intervention differentiation was assessed in only three studies (14.3%).

Different elements of operational fidelity were represented in the instruments. Interventionists’ competency and adherence in delivering the intervention were most commonly assessed (n = 12, 57%). Specifically, interventionists’ adherence was measured by the respective instruments used in five studies (24%); interventionists’ general behavior was assessed in one study (4.8%); and the remaining three studies (14.3%) did not clearly indicate whether or not interventionists’ competency and adherence were measured. In the majority (n = 20, 95%) of the studies, participants’ exposure, engagement and enactment of the intervention were not accounted for in the instruments measuring the fidelity of intervention implementation. One study assessed participant’s engagement in the intervention through report by a third party and direct and indirect observations by the interventionist.

3.4. Psychometric Properties3.4.1. Reliability

In the majority (n = 19, 91%) of the studies, the reliability of the intervention fidelity instruments was evaluated.

Internal consistency. Of the 19 studies, 13 (68.4%) reported on internal consistency of the intervention fidelity instruments using Cronbach’s alpha (α), and the remaining four did not provide empirical evidence. The Cronbach’s alpha ranged from 0.70 - 0.72 (acceptable) for nursing interventions; 0.70 - 0.99 (acceptable to excellent) for rehabilitation science interventions; 0.47 - 0.98 (unacceptable to excellent) for psychological interventions; 0.62 - 0.95 (acceptable to excellent) for psychiatry interventions; and 0.721 - 0.91 (good-excellent) for education interventions.

Inter-rater reliability. A total of 13 studies (68.4%) reported on inter-rater reliability using different coefficients. The Krippendorff’s α coefficient was 0.70 and 0.81 (good to excellent) for social work interventions. The Intra-class Correlation Coefficient ranged from 0.35 - 0.79 (unacceptable to acceptable) for psychiatry interventions; 0.71 - 0.95 (acceptable to excellent) for psychological interventions; 0.60 - 0.74 (questionable to acceptable) for nursing interventions; and 0.99 (excellent) for rehabilitation science interventions. The value of the Cohen’s Kappa coefficient was reported at 0.69 (good) for psychological interventions; 0.72 - 0.87 (good to very good) for behavioral interventions; and 0.66 - 0.96 (good to very good) for nursing interventions. The G coefficients ranged from 0.75 - 0.87 (acceptable) for rehabilitation science and education interventions. The percent of agreement ranged from 78.8% - 98% (acceptable) for nursing and education interventions.

Test re-test reliability. None of the 21 studies reported on test re-test reliability of the intervention fidelity instruments.

3.4.2. Validity

The majority (n = 18, 86%) of the studies reported on the validity of the intervention fidelity instruments.

Construct validity. About half of the studies (11 of 18, 61.1%) reported on construct validity of the instruments measuring fidelity of implementing psychology, psychiatry, health services, education, and rehabilitation interventions. However, not all provided empirical evidence to support the claim of construct validity. Of those that did, results demonstrated the instruments were able to discriminate between the different conditions (e.g. Cognitive Therapy and Supportive Expressive Dynamic Therapy; Twelve Step Facilitation, Clinical Management, and Cognitive Behavioral Therapy). Further, six studies (33.3%) reported on the relationships between the instruments’ subscales which captured the different aspects of fidelity with other measures of interventionists’ adherence to interventions such as Twelve Step Facilitation, Clinical Management scales; the association were relatively small in magnitude (Pearson’s r = −0.29 to −0.10; 0.23 - 0.63; 0.18 - 0.36; Spearman’s rho Correlation: rs 0.14 and 0.33).

Content validity. Most studies (10 of 18, 56%) reported on content validity of the instruments. This was done through expert judgment of the content of the items (n = 4, 22.2%). Agreement among experts was quantified in terms of the Cohen’s Kappa Coefficient (ҡ) in one study, and the Content Validity Index (CVI) in three studies. Overall, the CVIs were high (82.2% - 100%) implying the majority of the experts rated the items as relevant in capturing the key ingredients of the interventions.

4. Discussion

This study represents a first attempt to examine and identify the aspects of fidelity that are represented in existing instruments measuring fidelity of intervention implementation and to report on the validity and reliability of these measures.

Overall, the findings of this review showed that majority of the instruments were developed to measure the fidelity of implementing a specific intervention. Although advantageous in explicitly representing the active ingredients, components and activities that characterize a particular intervention, specific measures have limited applicability to similar or different interventions [21] [22] . This situation creates the need to develop multiple instruments; this precludes meaningful comparisons of fidelity with which the interventions were implemented. Therefore, it would be useful to develop generic instruments, as proposed by Breitenstein et al. (2010) [21] and Di Rezze et al. (2012) [22] to assess fidelity. Generic instruments assess the adherence to the protocol of interventions that have the same theoretical underpinning, consist of similar components (such as cognitive-beha- vioral therapy), and target a particular population (e.g. substance use) or practice (e.g. nursing) [21] . The advantage of these generic instruments is that they can be broadly applied to theoretically consistent interventions and or adapted to a specific target population [22] . For example, the main components of Cognitive Behavioral Therapy (CBT) can be implemented in a similar manner for persons with insomnia, depression, and anxiety. This in turn, standardizes the assessment of the fidelity with which these interventions are delivered [21] [22] .

4.1. Assessment of Fidelity of Intervention Implementation

Although there has been a proliferation of instruments for measuring fidelity of intervention implementation, the findings of this review pointed to several gaps. Theoretical fidelity was addressed primarily when developing the instruments by following a systematic process, which strengthens the validity and utility of the instruments. The process involved the identification of the active ingredients, components and activities, as well as the generation of a matrix, which was used to operationalize the intervention and guide the statement of items. However, a few of the studies explicitly reported use of the instrument to evaluate intervention differentiation. This finding may not be surprising as theoretical fidelity is often perceived as important during the stage of intervention design more so than implementation and evaluation. Assessment of intervention differentiation should be done in future research in order to determine the extent of contamination or dissemination of the intervention under evaluation to the control or comparison group, which may account for non-significant effects.

The findings of this review indicated that not all aspects of operational fidelity were captured in the instruments. Most instruments contained items that assessed interventionists’ competence or adherence, but not both. According to Hogue et al. (1996) [23] , attainment of high levels of fidelity of intervention implementation requires assessment of adherence and competence. This is because adherence and competence are interrelated, that is, an interventionist cannot be competent in implementing an intervention without adhering to its protocol and adherence alone is not sufficient for the delivery of the intervention competently [23] - [26] . In contrast, the instruments did not capture aspects of operational fidelity pertaining to participants’ exposure, engagement and enactment. The extent to which participants carry out the intervention activities and or recommendations is equally important for the success of an intervention in producing the desired outcomes [1] [27] .

The overall findings of this review have demonstrated that although some aspects of fidelity (i.e. interventionists’ adherence and competence) have been accounted for in instruments measuring fidelity of intervention implementation instruments, not all levels (i.e. theoretical) and aspects (i.e. participant engagement, exposure, and enactment) have been captured in the existing measures. Implementation of interventions, whether simple of complex, is one that requires the actions of both participants and interventionists in order to successfully attain the intervention goals and achieve the desired changes [1] .

4.2. Psychometric Properties

The instruments measuring fidelity of intervention implementation have shown acceptable levels of reliability and validity. This finding is consistent with a narrative review of generic intervention fidelity instruments for paediatric rehabilitation [22] . Reliability testing yielded fair to excellent internal consistency and inter-rater reliability. Not all studies provided empirical evidence supporting construct validity; those that did reported significant association between theoretically related concepts, primarily the association between fidelity and outcomes. The majority of the instruments were subjected to content validity by experts in the field. However none of the instruments were examined for test re-test reliability. Test re-test reliability examines stability of instruments over time and is often conducted for measures of stable concepts [17] . The examination of test re-test reliability is not feasible or meaningful for intervention fidelity instruments because one cannot administer the same intervention to the same group of participants at two separate points in time (usually between two and 14 days).

5. Implications for Practice and Research and Conclusion

The results of this review highlight key implications for research and practice. First, the review can inform researchers and clinicians of the current reliable and valid intervention fidelity instruments to monitor and assess the implementation of theoretically similar interventions (i.e. CBT for persons with depression and anxiety). This will decrease the burden of developing new instruments [22] and build the science of interventions research [21] . Second, the review identifies gaps related to limited assessment of theoretical fidelity and non-inclusion of all aspects of operational fidelity in the instruments used to evaluate fidelity of implementing a range of health interventions. Though there are different terms used across disciplines (e.g. psychology and nursing) to refer to aspects of theoretical and operational fidelity, researchers and clinicians are recommended to account for interventionists’ adherence and competence, participants’ exposure, engagement and enactment and intervention differentiation for assessing and monitoring fidelity of intervention implementation. Frameworks describing types and aspects of intervention fidelity are available [6] [7] [13] [28] and have the potential to support the development of instruments to conduct comprehensive assessment of the fidelity with which interventions are implemented, as well the valid interpretation of findings.

There are several limitations that are noteworthy in this review. First, despite the extensive search strategy, only a limited number of papers that report on the development and use of instruments for measuring fidelity are found. Additional papers, published in languages other than English may have been missed. Second, there is variability in the reported reliability and validity of the instruments by researchers; some authors do not provide empirical evidence to support their claims, which limit information needed to inform researchers and clinicians intending to use the instrument of its psychometric prosperities.

There is a need for the development and use of objective, reliable and valid fidelity of intervention implementation instruments that capture all aspects of fidelity and enhance the validity of findings in interventions research [8] . This is because healthcare providers to implement appropriate, efficient, effective, and safe complex interventions in clinical practice, it is important for them to understand the goal, essential and non-essential elements, mode of delivery, and dose of the intervention. Such an understanding provides direction for the operationalization of the intervention, and the implementation of the intervention with fidelity to produce changes in outcomes and to improve patient health and care [29] . This review examines and identifies the aspects of fidelity that are included in fidelity of intervention implementation and are reported on the psychometric properties of the measures. Sustained focus on developing psychometrically sound instruments that account for all aspects of fidelity (i.e. theoretical and operational level) as a method to assess and monitor implementation of interventions is imperative to strengthen the methodological literature for interventions and health research and for researchers to correctly interpret research findings and arrive to valid conclusions on the interventions effectiveness.

References<sup>* </sup>

[1] Sidani, S. and Braden, C.J. (2011) Design, Evaluation, and Translation of Nursing Interventions. Wiley-Blackwell, Ames.

[2] Eysenck, H. (1952) The Effects of Psychotherapy, an Evaluation. Journal of Consulting Psychology, 16, 319-324. http://dx.doi.org/10.1037/h0063633

[3] VandenBos, G.R. (1980) Psychotherapy: Practice, Research, Policy. Sage, Beverly Hills.

[4] Moncher, F.J. and Prinz, R.J. (1991) Treatment Fidelity in Outcome Studies. Clinical Psychology Review, 11, 247-266. http://dx.doi.org/10.1016/0272-7358(91)90103-2

[5] Gearing, R.E., El-Bassel, N., Ghesquiere, A., Baldwin, S., Gillies, J. and Ngeow, E. (2011) Major Ingredients of Fidelity: A Review and Scientific Guide to Improving Quality of Intervention Research Implementation. Clinical Psychology Review, 31, 79-88. http://dx.doi.org/10.1016/j.cpr.2010.09.007

[6] Bellg, A.J., Borrelli, B., Resnick, B., Hecht, J., Minicucci, D.S., Ory, M., et al. (2004) Enhancing Treatment Fidelity in Health Behavior Change Studies: Best Practices and Recommendations from the NIH Behaviour Change Consortium. Health Psychology, 23, 443-451. http://dx.doi.org/10.1037/0278-6133.23.5.443

[7] Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J. and Balain, S. (2007) A Conceptual Framework for Implementation Fidelity. Implementation Science, 2, 40. http://dx.doi.org/10.1186/1748-5908-2-40

[8] Stein, K.F., Sargent, J.T. and Rafaels, N. (2007) Intervention Research: Establishing Fidelity of the Independent Variable in Nursing Clinical Trials. Nursing Research, 56, 54-62. http://dx.doi.org/10.1097/00006199-200701000-00007

[9] Shadish, W.R., Cook, T.D. and Campbell, D.T. (2002) Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company, Boston.

[10] Dusenbury, L., Brannigan, R., Falco, M. and Hansen, W.B. (2003) A Review of Research on Fidelity of Implementation: Implications for Drug Abuse Prevention in School Settings. Health Education Research, 18, 237-256. http://dx.doi.org/10.1093/her/18.2.237

[11] Brandt, P.A., Kirsch, S.D., Lewis, F.M. and Casey, S.M. (2004) Assessing the Strength and Integrity of an Intervention. Oncology Nursing Forum, 31, 833-837. http://dx.doi.org/10.1188/04.ONF.833-837

[12] Dumas, J.E., Lynch, A.M., Laughlin, J.E., Smith, P.E. and Prinze, R.J. (2001) Promoting Intervention Fidelity: Conceptual Issues, Methods, and Preliminary Results from the EARLY ALLIANCE Prevention Trial. American Journal of Preventive Medicine, 20, 38-47. http://dx.doi.org/10.1016/S0749-3797(00)00272-5

[13] Borrelli, B., Sepinwall, D., Ernst, D., Bellg, A.J., Czajkowski, S., Breger, R., et al. (2005) A New Tool to Assess Treatment Fidelity and Evaluation of Treatment Fidelity across 10 Years of Health Behaviour Research. Journal of Consulting and Clinical Psychology, 73, 852-860. http://dx.doi.org/10.1037/0022-006X.73.5.852

[14] Santacroce, S.J., Maccarelli, L.M. and Grey, M. (2004) Intervention Fidelity. Nursing Research, 53, 63-66. http://dx.doi.org/10.1097/00006199-200401000-00010

[15] Mowbray, C.T., Holter, M.C., Teague, G.B. and Bybee, D. (2003) Fidelity Criteria: Development, Measurement, and Validation. American Journal of Evaluation, 24, 315-340. http://dx.doi.org/10.1016/S1098-2140(03)00057-2

[16] Faiburn, C.G. and Cooper, Z. (2011) Therapist Competence, Therapy Quality and Therapist Training. Behavioral Research and Therapy, 49, 373-378. http://dx.doi.org/10.1016/j.brat.2011.03.005

[17] Streiner, D.L. and Norman, G.R. (2008) Health Measurement Scales: A Practical Guide to Their Development and Use. 4th Edition, Oxford University Press Inc., New York. http://dx.doi.org/10.1093/acprof:oso/9780199231881.001.0001

[18] DeVon, H.A., Block, M.E., Moyle-Wright, P., Ernst, D.M., Hayden, S.J., Lazzara, D.J., et al. (2007) A Psychometric Toolbox for Testing Validity and Reliability. Journal of Nursing Scholarship, 39, 155-164. http://dx.doi.org/10.1111/j.1547-5069.2007.00161.x

[19] Fayers, P.M. and Machin, D. (2007) Quality of Life. 2nd Edition, John Wiley & Sons Inc., Hoboken. http://dx.doi.org/10.1002/9780470024522

[20] Eigenmann, C.A., Colagiuri, T.C., Skinner, T.C. and Trevena, L. (2009) Are Current Psychometric Tools Suitable for Measuring Outcomes for Diabetes Education? Diabetes Medicine, 26, 425-436. http://dx.doi.org/10.1111/j.1464-5491.2009.02697.x

[21] Breitenstein, S.M., Fogg, L., Garvey, C., Hill, C., Resnick, B. and Gross, D. (2010) Measuring Implementation Fidelity in a Community-Based Parenting Intervention. Nursing Research, 59, 158-165. http://dx.doi.org/10.1097/NNR.0b013e3181dbb2e2

[22] Di Rezze, B., Law, M., Gorter, J.W., Eva, K. and Pollock, N. (2012) A Narrative Review of Generic Intervention Fidelity Measures. Physical and Occupational Therapy in Pediatrics, 32, 430-446. http://dx.doi.org/10.3109/01942638.2012.713454

[23] Hogue, A., Liddle, H.A. and Rowe, C. (1996) Treatment Adherence Process Research in Family Therapy: A Rationale and Some Practical Guidelines. Psychotherapy, 33, 332-345. http://dx.doi.org/10.1037/0033-3204.33.2.332

[24] Barber, J.P. and Crits-Christoph, P. (1996) Development of a Therapist Adherence/Competence Rating Scale for Supportive-Expressive Dynamic Psychotherapy: A Preliminary Report. Psychotherapy Research, 6, 81-94. http://dx.doi.org/10.1080/10503309612331331608

[25] Perepletchikova, F., Treat, T.A. and Kazdin, A.E. (2007) Treatment Integrity in Psychotherapy Research: Analysis of the Studies and Examination of the Associated Factors. Journal of Consulting and Clinical Psychology, 75, 829-841. http://dx.doi.org/10.1037/0022-006X.75.6.829

[26] Waltz, J., Addis, M., Koerner, K. and Jacobson, N.S. (1993) Testing the Integrity of a Psychotherapy Protocol: Assessment of Adherence and Competence. Journal of Consulting and Clinical Psychology, 61, 620-630. http://dx.doi.org/10.1037/0022-006X.61.4.620

[27] Leventhal, H. and Friedman, M.A. (2004) Does Establishing Fidelity of Treatment Help in Understanding Treatment Efficacy? Comment on Bellg et al. (2004). Health Psychology, 23, 452-456. http://dx.doi.org/10.1037/0278-6133.23.5.452

[28] Lichstein, K.L., Riedel, B.W. and Grieve, R. (1994) Fair Tests of Clinical Trials: A Treatment Implementation Model. Advances in Behaviour Research and Therapy, 16, 1-29. http://dx.doi.org/10.1016/0146-6402(94)90001-9

[29] Sidani, S. (2015) Health Intervention Research: Understanding Research Design and Methods. Sage, Thousand Oaks.

NOTESReferencesSidani, S. and Braden, C.J. (2011) Design, Evaluation, and Translation of Nursing Interventions. Wiley-Blackwell, Ames.Eysenck, H. (1952) The Effects of Psychotherapy, an Evaluation. Journal of Consulting Psychology, 16, 319-324. <BR/>http://dx.doi.org/10.1037/h0063633VandenBos, G.R. (1980) Psychotherapy: Practice, Research, Policy. Sage, Beverly Hills.Moncher, F.J. and Prinz, R.J. (1991) Treatment Fidelity in Outcome Studies. Clinical Psychology Review, 11, 247-266. <BR/>http://dx.doi.org/10.1016/0272-7358(91)90103-2Gearing, R.E., El-Bassel, N., Ghesquiere, A., Baldwin, S., Gillies, J. and Ngeow, E. (2011) Major Ingredients of Fidelity: A Review and Scientific Guide to Improving Quality of Intervention Research Implementation. Clinical Psychology Review, 31, 79-88. <BR/>http://dx.doi.org/10.1016/j.cpr.2010.09.007Bellg, A.J., Borrelli, B., Resnick, B., Hecht, J., Minicucci, D.S., Ory, M., et al. (2004) Enhancing Treatment Fidelity in Health Behavior Change Studies: Best Practices and Recommendations from the NIH Behaviour Change Consortium. Health Psychology, 23, 443-451. <BR/>http://dx.doi.org/10.1037/0278-6133.23.5.443Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J. and Balain, S. (2007) A Conceptual Framework for Implementation Fidelity. Implementation Science, 2, 40. <BR/>http://dx.doi.org/10.1186/1748-5908-2-40Stein, K.F., Sargent, J.T. and Rafaels, N. (2007) Intervention Research: Establishing Fidelity of the Independent Variable in Nursing Clinical Trials. Nursing Research, 56, 54-62. <BR/>http://dx.doi.org/10.1097/00006199-200701000-00007Shadish, W.R., Cook, T.D. and Campbell, D.T. (2002) Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company, Boston.Dusenbury, L., Brannigan, R., Falco, M. and Hansen, W.B. (2003) A Review of Research on Fidelity of Implementation: Implications for Drug Abuse Prevention in School Settings. Health Education Research, 18, 237-256. <BR/>http://dx.doi.org/10.1093/her/18.2.237Brandt, P.A., Kirsch, S.D., Lewis, F.M. and Casey, S.M. (2004) Assessing the Strength and Integrity of an Intervention. Oncology Nursing Forum, 31, 833-837. <BR/>http://dx.doi.org/10.1188/04.ONF.833-837Dumas, J.E., Lynch, A.M., Laughlin, J.E., Smith, P.E. and Prinze, R.J. (2001) Promoting Intervention Fidelity: Conceptual Issues, Methods, and Preliminary Results from the EARLY ALLIANCE Prevention Trial. American Journal of Preventive Medicine, 20, 38-47. <BR/>http://dx.doi.org/10.1016/S0749-3797(00)00272-5Borrelli, B., Sepinwall, D., Ernst, D., Bellg, A.J., Czajkowski, S., Breger, R., et al. (2005) A New Tool to Assess Treatment Fidelity and Evaluation of Treatment Fidelity across 10 Years of Health Behaviour Research. Journal of Consulting and Clinical Psychology, 73, 852-860. <BR/>http://dx.doi.org/10.1037/0022-006X.73.5.852Santacroce, S.J., Maccarelli, L.M. and Grey, M. (2004) Intervention Fidelity. Nursing Research, 53, 63-66.<BR/>http://dx.doi.org/10.1097/00006199-200401000-00010Mowbray, C.T., Holter, M.C., Teague, G.B. and Bybee, D. (2003) Fidelity Criteria: Development, Measurement, and Validation. American Journal of Evaluation, 24, 315-340. <BR/>http://dx.doi.org/10.1016/S1098-2140(03)00057-2Faiburn, C.G. and Cooper, Z. (2011) Therapist Competence, Therapy Quality and Therapist Training. Behavioral Research and Therapy, 49, 373-378. <BR/>http://dx.doi.org/10.1016/j.brat.2011.03.005Streiner, D.L. and Norman, G.R. (2008) Health Measurement Scales: A Practical Guide to Their Development and Use. 4th Edition, Oxford University Press Inc., New York. <BR/>http://dx.doi.org/10.1093/acprof:oso/9780199231881.001.0001DeVon, H.A., Block, M.E., Moyle-Wright, P., Ernst, D.M., Hayden, S.J., Lazzara, D.J., et al. (2007) A Psychometric Toolbox for Testing Validity and Reliability. Journal of Nursing Scholarship, 39, 155-164.<BR/>http://dx.doi.org/10.1111/j.1547-5069.2007.00161.xFayers, P.M. and Machin, D. (2007) Quality of Life. 2nd Edition, John Wiley & Sons Inc., Hobo-ken.<BR/>http://dx.doi.org/10.1002/9780470024522Eigenmann, C.A., Colagiuri, T.C., Skinner, T.C. and Trevena, L. (2009) Are Current Psychometric Tools Suitable for Measuring Outcomes for Diabetes Education? Diabetes Medicine, 26, 425-436.<BR/>http://dx.doi.org/10.1111/j.1464-5491.2009.02697.xBreitenstein, S.M., Fogg, L., Garvey, C., Hill, C., Resnick, B. and Gross, D. (2010) Measuring Implementation Fidelity in a Community-Based Parenting Intervention. Nursing Research, 59, 158-165.<BR/>http://dx.doi.org/10.1097/NNR.0b013e3181dbb2e2Di Rezze, B., Law, M., Gorter, J.W., Eva, K. and Pollock, N. (2012) A Narrative Review of Generic Intervention Fidelity Measures. Physical and Occupational Therapy in Pediatrics, 32, 430-446.<BR/>http://dx.doi.org/10.3109/01942638.2012.713454Hogue, A., Liddle, H.A. and Rowe, C. (1996) Treatment Adherence Process Research in Family Therapy: A Rationale and Some Practical Guidelines. Psychotherapy, 33, 332-345. <BR/>http://dx.doi.org/10.1037/0033-3204.33.2.332Barber, J.P. and Crits-Christoph, P. (1996) Development of a Therapist Adherence/Competence Rating Scale for Supportive-Expressive Dynamic Psychotherapy: A Preliminary Report. Psychotherapy Research, 6, 81-94.<BR/>http://dx.doi.org/10.1080/10503309612331331608Perepletchikova, F., Treat, T.A. and Kazdin, A.E. (2007) Treatment Integrity in Psychotherapy Research: Analysis of the Studies and Examination of the Associated Factors. Journal of Consulting and Clinical Psychology, 75, 829-841.<BR/>http://dx.doi.org/10.1037/0022-006X.75.6.829Waltz, J., Addis, M., Koerner, K. and Jacobson, N.S. (1993) Testing the Integrity of a Psychotherapy Protocol: Assessment of Adherence and Competence. Journal of Consulting and Clinical Psychology, 61, 620-630.<BR/>http://dx.doi.org/10.1037/0022-006X.61.4.620Leventhal, H. and Friedman, M.A. (2004) Does Establishing Fidelity of Treatment Help in Understanding Treatment Efficacy? Comment on Bellg et al. (2004). Health Psychology, 23, 452-456.<BR/>http://dx.doi.org/10.1037/0278-6133.23.5.452Lichstein, K.L., Riedel, B.W. and Grieve, R. (1994) Fair Tests of Clinical Trials: A Treatment Implementation Model. Advances in Behaviour Research and Therapy, 16, 1-29. <BR/>http://dx.doi.org/10.1016/0146-6402(94)90001-9Sidani, S. (2015) Health Intervention Research: Understanding Research Design and Methods. Sage, Thousand Oaks.