Present study examined the ECPARQ with four different higher order techniques (Bifactor EFA, Bifactor CFA, Bifactor ESEM, Second-order CFA) plus a simple CFA (ICM CFA), evaluating a total of 19 models. All models were tested twice, one for the ECPARQ Mother version and one for the Father version. For each version, alternative models were tested with 2, 3 and 4-factor structures. All models were tested having item 13 both in the intended factor and alternatively in the Warmth factor. This comparison was made to empirically answer in what factor the item 13 belongs. All higher order models had an adequate fit, suggesting thus the theoretical construct of the Warmth dimension of Parenting is supported for the Greek cultural context, although further investigation is necessary. ICM-CFA had equally adequate fit, thus preferable based on parsimony. All models tested having item 13 in the Warmth factor had superior fit in comparison to their counterparts with item 13 in the Indifference/Neglect factor. Internal consistency reliability was adequate.
As early as in turn of the 20th-century symbolic interaction theory (Cooley, 1902; Mead, 1934) postulated that children have a tendency more or less to view themselves as perceived by their significant others (or attachment figures in the words of Ainsworth, 1989; Bowlby, 1982; Colin, 1996 ). The significance of the quality in the child-parent relationship was also highlighted by Freud’s work (1923) and has been incorporated in numerous grounding theories about child development (Dwairy, 2010; Cullin, 2011) .
In a similar vein, Erikson’s Psychosocial Theory of Personality (Erikson, 1968) stipulates that it is mother’s sensitive care during the first year of life development that formulates confidence toward self and positive view of the world for a child. In contrast, when children perceive their parents’ behavior as neglecting, rejecting or indifferent toward them (Dwairy, 2010; Erikson, 1968; Rohner, 1975, 1986, 2004; Rohner & Rohner, 1980) , they probably feel unloved or even undeserving of receiving love, warmth, and acceptance (Rohner, 1975, 2004; Rohner & Rohner, 1980; Rohner & Khaleque, 2015) .
When young children are rejected by a significant other, they generally tend to develop a cohort of personality dispositions, termed “the acceptance-rejection syndrome” (Rohner, 2004; Rohner, Khaleque, & Cournoyer, 2012) . The expression, frequency, duration, timing, and intensity of the perceived rejection are graphically represented in the Warmth dimension of parenting which illustrates on a continuum how children associate perceived parental rejection with four behavioral expressions (Rohner, 1986; 2004) : 1) Warmth/affection, 2) Hostility/aggression, 3) Indifference/neglect, and finally 4) Undifferentiated rejection. In particular, warmth (parental acceptance) lies at one end of the continuum, representing positive feelings, love, and support expressed by parents to their children. Rejection lies at the opposite end. Depending on the type of perceived rejection, rejection can be expressed as either indifference, absence of positive feelings, judgment, psychological/physical aggression or simply neglect.
To measure perceived parental acceptance rejection, researchers use the Child Parental Acceptance-Rejection Questionnaire (Child PARQ) (Rohner & Khaleque, 2005) suitable for children older than 7 years. Child PARQ statements are tapping on the four theoretical dimensions of the Warmth dimension of parenting (Rohner, 1986, 2004; Rohner & Cournoyer, 1994) . For young children aged from 4 - 7 years, there is a newly developed variation of Child-PARQ Short Form: The Early Childhood Parental Acceptance-Rejection Questionnaire (ECPARQ; Rohner, 2012 ). The new PARQ variation came as a supplementary measurement tool in order to extend acceptance-rejection research on young children aged 4 - 7 years. ECPARQ is a tool administered to a young child respondent with the help of a facilitator. It contains 24 items tapping on the four categories of the Warmth Dimension of Parenting (Rohner, 1986) . Items are allocated as follows: 1) Warmth/affection with 8 items, 2) Hostility/Aggression with 6 items, 3) Indifference/Neglect with 6 items, and finally 4) Undifferentiated Rejection with 4 items (i.e. the warmth dimension of parenting categories).
ECPARQ has identical factor structure and items with Child PARQ-short form with two minor differences: 1) it uses an uncomplicated wording in 7 of the 24 PARQ statements; 2) it uses a game-like answering procedure (Rohner & Giotsa, 2012) in order to obtain 4-Likert point answers from the young children (see Procedure section for details). The new tool has been developed in English in two versions (Mother and Father). Both of them were translated into Greek (Giotsa & Kaminiotis, 2014) .
Generally, there is very limited empirical research on Parental Acceptance-Rejection Theory concerning early childhood (Giotsa & Kaminiotis, 2014) . Moreover, the published empirical works on ECPARQ factor structure are virtually almost non-existent. Specifically, the factorial structure of ECPARQ, Greek version has been researched by Giotsa and Kaminiotis (2014) , with Exploratory Factor Analysis (EFA), proposing a three-factor structure for both Mother and Father Versions. Reliability in this work of ranged from tolerable (α = 0.69 for father version) to significant (α = 0.82 for mother version). An unpublished CFA work (Giotsa & Kyriazos, 2017) also established a three-factor structure for ECPARQ with 1) Warmth factor; 2) an Indifference/Neglect factor; and 3) a Hostility/Aggression factor collapsed with the Undifferentiated Rejection factor for both mother and father ECPARQ versions. Internal reliability of this single-order structure with 3 factors was adequate (0.85 and 0.90 for mother and father version respectively).
However, we can draw useful information from PARQ factor structure research because ECPARQ is essentially the same questionnaire, only for a different age group. Actually, the original validity study of PARQ pointed to a two-factor structure: one factor for acceptance and one for rejection with a high factor correlation (r = 0.50; Rohner & Cournoyer, 1975; Rohner & Khaleque 2005 ) containing the four above dimensions as subfactors. This bi-dimensional acceptance-rejection structure was later confirmed in an Indian sample (Rohner & Chaki-Sircar, 1988) , an Italian sample (Comunian & Gielen, 2001) , in a Japanese sample (Yazdkhasti & Harizuka, 2006) and in multiethnic samples (Rohner & Cournoyer, 1994; Dwairy, 2010) . Crucially, this factor-subfactor structure in the above EFA studies might actually indicate the presence of a higher order structure.
However, the CFA studies on PARQ structure (Artemis & Touloumakos, 2014; Gomez & Rohner, 2011; Tsaousis, Giovazolias, & Mascha, 2012) proposed a four-factor, single-order structure for PARQ. In consequence, the higher factor structure hypothesis remains up today empirically untested, despite the fact that it is further supported by additional empirical findings.
More specifically, Rohner & Khaleque (2005) regarding the high interactor correlations found (Rohner & Cournoyer, 1975) , attempting to explain the high factor correlations of acceptance and rejection factors suggested that the two factors of acceptance and rejection may be the opposite poles of the Warmth dimension of parenting (Rohner & Khaleque, 2005) , essentially proposing that the Warmth dimension of parenting is a higher order construct for PARQ. This hypothesis was supported by high factor correlations in subsequent CFA research both on PARQ (Artemis & Touloumakos, 2014; Gomez & Rohner, 2011; Tsaousis et al., 2012) and on ECPARQ (Giotsa & Kyriazos, 2017) .
Note that for PARQ the inter-correlations between the indifference/neglect factor and Undifferentiated rejection was 0.91 to 0.98 (Artemis & Touloumakos, 2014) and for all factors from 0.48 to 0.98 for both versions (mother and father). Similarly, in another work in Greek context (Tsaousis et al., 2012) PARQ inter-correlations ranged for mother version from 0.76 to 0.98 (indifference/neglect factor with Undifferentiated rejection) and for the father version from 0.74 to 0.98 (indifference/neglect factor with Undifferentiated rejection). High factor inter-correlations are a strong indication of a higher order factor (Hammer & Tolland, 2016; Wang & Wang, 2012) . Crucially, this empirical evidence is also theoretically supported, because this higher order constructs in PARQ and ECPARQ is the Warmth Dimension of the parenting continuum (Rohner & Khaleque, 2005) .
Another issue of the empirical research on PARQ/ECPARQ factor structure is item 13 (“Pays a lot of attention to me”). It belongs to the Indifference/Neglect factor but the CFA findings suggest it should be in the Warmth factor.
Current research will attempt to verify whether ECPARQ has a higher order factor structure and if item 13 belongs to the Indifference/Neglect or the Warmth factor using all currently available higher order techniques (CFA, Bifactor CFA, Bifactor EFA, Bifactor ESEM, Second-order CFA). So the research questions of the present study are as following: 1) Has ECPARQ Mother version a hierarchical structure? (Warmth dimension of Parenting); 2) Has ECPARQ Father version a hierarchical structure? (Warmth dimension of Parenting); 3) To what factor does item 13 belong in the Mother version? 4) To what factor does item 13 belong in the Father version?
Our sampling frame consisted of 32 major cities of Greece in the following 7 districts: Sterea Ellada (18%), Epirus (36%), Macedonia (26%), Thessaly (8%), Peloponnese (7%), Ionian Islands (4%), and Thrace (1%). Sample included 1000 members of the general young children population (males = 47%, females = 53%). Participants’ age ranged from 5 to 7 years (M = 5.74 years, SD = 1.13).
This study used the short form of the Early Childhood Parental Acceptance-Rejection Questionnaire (ECPARQ) in two different versions: 1) Mother Version and 2) Father Version. The difference of the two versions is that the mother version contains 24 statements starting with the expression “My Mother” and “she” while father version contains the same 24 statements starting with “My Father” and “he”. All items are measuring perceived parental behavior on young children below the age of 7, in an age-appropriate wording. Specifically, ECPARQ was adapted for young children by simplifying items of the original Child PARQ Short form (Rohner & Khaleque, 2005) . For example, the item “Sees me as a big nuisance” of Child PARQ Short form was rephrased to “Sees me as a big problem” in ECPARQ. The questionnaire is scored in the direction of perceived rejection, on a 4 point Likert scale. To make the 4-point Likert scale (4 = almost always true, 1 = almost never true a two-step answering procedure was adopted), which was more child-friendly, (see in the Procedure section). Warmth items and Item 13 were reversed scored in all analyses.
During the answering procedure (Rohner & Giotsa, 2012) a flash-card game was invented in order to urge young children to choose whether an item was evaluated as TRUE or NOT TRUE. Each TRUE/NOT TRUE choice was represented by a flash card 10 × 15 cm. The flash card for “true” had the word “TRUE” printed on one side and the symbol “” on the other. The “not true” flash card had the word ‘‘NOT TRUE” on one side and the symbol “X” on the other. The cards were white with black letters to avoid biased answer due to color (Rohner & Giotsa, 2012) . See Procedure for details on the answering process.
First, we informed through parent’s associations the parents of young children involved in the study. Next, we described the procedure to parents asking them to complete an Informed Consent Form. Data were collected individually by oral administration of the questionnaires at the end of class. We organized administration in 2 sessions per respondent that lasted approximately 40 minutes per instrument (Mother Version and Father). At the same time, the children’s parents were filling a form containing demographics. Participation was voluntary. The option to quit the procedure if a young child felt uncomfortable for any reason was explicitly offered. No extrinsic incentives were offered for participation in the study. Data were collected anonymously and confidentiality was kept.
The ECPARQ is using two flash cards (see Materials), making the answering process more like a game (Rohner & Giotsa, 2012) . A room with a small chair and a table were chosen. In the beginning, a team member read a test statement not included in the questionnaire to familiarize the child with the answering process. Each item response was completed into 2 steps. During step 1, the child was asked: “Does your Mommy (or Daddy) do/say this.” Is it TRUE or NOT TRUE? Simultaneously the child was presented with the two flash cards. Next, during step 2, TRUE answers were followed by questions like “Does your mummy almost always do that, or does she only sometimes do that?”, to elicit an answer for Likert points 4 or 3. NOT TRUE answers, were followed by questions like “Does your mummy rarely do that, or does she almost never do that?” to receive either a 2 or 1 point Likert scale answer. At the end of the answering process usually a small sweet or a sticker was offered to the child.
The following data analyses took place: 1) Univariate and multivariate normality tests were estimated; 2) ECPARQ factor structure was examined using the many different higher order techniques (Bifactor CFA, Bifactor EFA, Bifactor ESEM, Second-order CFA); 3) Alternative ECPARQ solutions were evaluated having either 2, 3, or 4 factors for each higher-order method; 4) To compare the fit of the optimal solutions emerged from each method, the “likelihood ratio test” (2ΔLL; Satorra & Bentler, 2010 ) was applied when appropriate (for the Mother version) 5) To evaluate internal consistency reliability for the optimal model, Cronbach’s alpha coefficient (Cronbach, 1951) was used. The above analyses were implemented for both the Mother and the Father version of ECPARQ.
Although the sample was adequate to carry out the 3-faced construct validation method1 (Kyriazos, 2018) the focus of this research was not to establish the construct validity of ECPARQ but to empirically examine if there is a higher order factor structure, thus the method was not used in this study. Data were coded and analyzed with the following software: Mplus version 7 (Muthen & Muthen, 2012) and SPSS, version 25 (IBM, 2017) and Stata Version 14.2 (StataCorp, 2015) .
There were no missing data due to the presence of the facilitator implementing the answering procedure (Rohner & Giotsa, 2012) . For each ECPARQ version (Mother and Father), the sample to variable ratio was 42:1, (N = 1000). An acceptable sample-to-variable ratio can range anywhere from 10/1 (Osborne & Costello, 2004; Singh et al., 2016) to 20/1 (Schumacker & Lomax, 2016) , especially when testing instruments will less than 40 items (DeVellis, 2017) . This suggested that the emerging factor structure and factor loadings were robust; therefore, measurement is more reliable (Linley, Maltby, Wood, Osborne, & Hurling, 2009; Williams, Brown, & Onsman, 2010) . Additionally, Comrey and Lee (1992) commented that a sample of N = 1000 is excellent for factor analysis.
1For applied examples of the implementation of the method see Kyriazos et al. (2018a, 2018b, 2018c, 2018d) .
The data violated the assumptions of univariate normality. Specifically, Kolmogorov?Smirnov tests (Massey, 1951) for all items of ECPARQ mother and father versions were statistically significant (all p values < 0.001). To test for multivariate normality four test were estimated: Mardia’s multivariate kurtosis test (Mardia, 1970) , Mardia’s multivariate skewness test (Mardia, 1970) , Henze-Zirkler’s consistent test (Henze & Zirkler, 1990) and Doornik-Hansen omnibus test (Doornik & Hansen, 2008) . All four suggested a violation of multivariate normality since the null hypothesis was rejected (p values < 0.0001).
Models tested were separated into two categories: 1) CFA models without higher-order structure to use as a benchmark; 2) models with higher-order structure. A total of five-factor analysis methods used. Specifically, for the CFA models without higher-order structure, a standard CFA was used. For models with higher-order structure four methods were used: 1) Bifactor EFA; 2) Bifactor CFA; 3) Bifactor ESEM; 4) Second-order CFA. For each of the total five methods, alternative ECPARQ mother and ECPAQ father models with either 2, 3 or 4 factors were evaluated.
About CFA parameters used, MPlus (Muthen & Muthen, 2012) , as Wang & Wang (2012) comment, has robust MLR, a robust rescaling-based estimator. MLR is appropriate for non-normal distributions and unlike similar methods, it offers robust standard errors and chi-square test calculations. Considering the above properties, MLR was used as a parameter estimator for all CFA, EFA and ESEM models tested.
The goodness of fit in all methods was evaluated by the following fit measures in all techniques: The Standardized Root Mean Square Residual (SRMR), the Root Mean Square Error of Approximation (RMSEA, 90% CI), the Comparative Fit Index (CFI), the Tucker-Lewis index (TLI), and finally by the Chi-square and the Chi-square/df ratio. The acceptable fit was evaluated based on the following standards (Brown, 2015; Hu & Bentler, 1999) : RMSEA ≤ 0.06 (90% CI ≤ 0.06), SRMR ≤ 0.08, CFI ≥ 0.95 and TLI ≥ 0.95 and lastly chi-square/df ratio < 3. Comparing multiple fit indices to evaluate model fit is more reliable (Brown, 2015) .
Based on previous research alternative models tested, specifically, the two-factor models had an Acceptance (8 items of the Warmth factor) and a Rejection factor (16 items, containing all “non-warmth” items), originally proposed by Rohner & Cournoyer for PARQ (1975) . Note however that this structure had sub-factors. This bidimensional structure was validated by many others (Comunian & Gielen, 2001; Dwairy, 2010; Rohner & Chaki-Sircar, 1988; Rohner & Cournoyer, 1994; Yazdkhasti & Harizuka, 2006) . The three-factor structure was proposed by Giotsa & Kyriazos (2017) . The 3-factor structure is empirically supported by the high factor correlations between the Warmth factor and Indifference/Neglect factor (up to 0.98) for the 4-factor solutions for PARQ (Artemis & Touloumakos, 2014; Tsaousis et al., 2012) . Generally, high correlations ?exceeding the recommended cut-off value of 0.80 - 0.85 (Brown, 2015; Kline, 1998; Wang & Wang, 2012) correlations between latent factors suggest redundancy of dimensions ( Epitropaki & Martin, 2004
All CFA models were tested both with item 13 (Pays a lot of attention to me) in the original factor (Indifference/Neglect) as described in Rohner & Khaleque (2005) and in the Warmth factor.
For Mother version, among the 6 alternative models tested MODEL 4 with 3 factors and item 13 in the acceptance factor achieved an adequate fit with all fit measures in acceptable levels. Factor intercorrelations ranged from 0.573 to 0.734 (see
| Model | χ2 Value | Df | χ2/df | CFI | TLI | RMSEA | RMSEA Lower CI | RMSEA Higher CI | SRMR | Factor Correlation |
|---|---|---|---|---|---|---|---|---|---|---|
| ECPARQ MOTHER (N = 1000) | ||||||||||
| MODEL 1 2 FACTORS (Acceptance./Rejection) | 662.65 | 251 | 2.64 | 0.870 | 0.857 | 0.041 | 0.037 | 0.044 | 0.048 | 0.724 |
| MODEL 2 2 FACTORS with 13 in Acceptance | 592.55 | 251 | 2.36 | 0.892 | 0.882 | 0.037 | 0.033 | 0.041 | 0.043 | 0.694 |
| MODEL 3 3 FACTORS (W & I/N, H/A + UR) | 593.95 | 249 | 2.39 | 0.891 | 0.880 | 0.037 | 0.033 | 0.041 | 0.047 | 0.677 - 0.786 |
| MODEL 4 3 FACTORS w.13 in W & I/N, H/A + UR | 490.28 | 249 | 1.97 | 0.924 | 0.916 | 0.031 | 0.027 | 0.035 | 0.039 | 0.573 - 0.734 |
| MODEL 5 4 FACTORS Original (W, I/N, H/A, UR) | LATENT VARIABLE COVARIANCE MATRIX IS NOT POSITIVE DEFINITE. | |||||||||
| MODEL 6 4 FACTORS with 13 in W & I/N, H/A, UR | LATENT VARIABLE COVARIANCE MATRIX IS NOT POSITIVE DEFINITE. | |||||||||
| ECPARQ FATHER (N = 1000) | ||||||||||
| MODEL 1 2 FACTORS (Acceptance./Rejection) | 858.68 | 251 | 3.42 | 0.873 | 0.860 | 0.050 | 0.046 | 0.053 | 0.054 | 0.746 |
| MODEL 2 2 FACTORS with 13 in Acceptance | 733.42 | 251 | 2.92 | 0.899 | 0.889 | 0.044 | 0.040 | 0.048 | 0.046 | 0.715 |
| MODEL 3 3 FACTORS (W & I/N, H/A + UR) | 753.88 | 249 | 3.03 | 0.894 | 0.883 | 0.045 | 0.042 | 0.049 | 0.048 | 0.828 - 0.837 |
| MODEL 4 3 FACTORS w.13 in W & I/N, H/A + UR | 643.06 | 249 | 2.58 | 0.918 | 0.909 | 0.040 | 0.036 | 0.044 | 0.043 | 0.693 - 0.794 |
| MODEL 5 4 FACTORS Original (W, I/N, H/A, UR) | 744.70 | 246 | 3.03 | 0.896 | 0.883 | 0.045 | 0.042 | 0.049 | 0.047 | 0.638 - 0.986 |
| MODEL 6 4 FACTORS with 13 in W & I/N, H/A, UR | 632.87 | 246 | 2.57 | 0.919 | 0.909 | 0.040 | 0.036 | 0.044 | 0.042 | 0.637 - 0.986 |
Note: MLR robust rescaling-based estimator was used for parameters estimate in all models; W = Warmth, I/N = Indifference/Neglect, H/A = Hostility/Aggression, UR = Undifferentiated Rejection, Acceptance = Warmth, Rejection = Indifference/Neglect, Hostility/Aggression, Undifferentiated Rejection.
For Father version, among the 6 alternative models tested, two alternative models showed good fit: MODEL 4 (with 3 factors and item 13 in the acceptance factor) and MODEL 6 (with 4 factors and item 13 in the acceptance factor). These two models achieved an equally adequate fit with all fit measures either in acceptable levels (CFI and TLI) or satisfactory levels. The rest of the models showed unacceptable fit in some indices (see
The methods used to test the higher order factor structure of ECPARQ mother and father were the following: 1) Bifactor Models: Bifactor EFA, Bifactor CFA, Bifactor ESEM; and 2) Second-order CFA.
1) Bifactor Models
Bifactor analysis (Harman, 1976; Holzinger & Swineford, 1937) is another approach to higher-order factor analysis, specifying direct effects of the higher-order dimension (General factor) on the indicators (Specific factors), unlike the classical higher-order CFA method (Brown, 2015) . The benefit of the exploratory bifactor analysis method is that a specific a priori bi-factor model is not necessary. Instead, the exploratory bi-factor analysis can be the starting point for the specification of bi-factor models with other bifactor methods (Jennrich & Bentler, 2011) . In Bifactor ESEM (c.f. Reise, 2012 ) direct effects of the higher-order dimension are specified and additionally because ESEM (Asparouhov & Muthen, 2009) is an integration of EFA, CFA, and SEM, it can potentially resolve misspecifications and inflated factor loadings, inherent in CFA method as a result of forcing secondary factor loadings to be equal to zero (Marsh et al., 2014) .
Concerning the theoretical construct behind the Bifactor structure, bifactor models are most appropriate for unidimensional constructs, having at the same time smaller latent sub-factors (Brown, 2015) . The General Factor in the case of ECPARQ was the Warmth Dimension of the Parenting continuum. For the Specific factors, the structures with 2 (Acceptance-Rejection), 3 (Warmth, Hostility/Aggression, Indifference/Neglect/Undifferentiated Rejection) and 4 factors (Warmth, Hostility/Aggression, Indifference/Neglect, Undifferentiated Rejection (Rohner & Khaleque, 2005) , structures were tested because Bifactor higher order structures do not have any restriction regarding the number of the specific factors (unlike higher order CFA). In sum, 3 Bifactor EFA, 3 Bifactor CFA, and 3 Bifactor ESEM models were tested (9 total) having either 2, 3 or 4 factors. Regarding model parametrization, for the Bifactor EFA (and ESEM) models Bi-Geomin factor rotation was used (Jennrich & Bentler, 2011) . All models tested had item 13 in warmth factor.
For ECPARQ Mother version, all models except the 2-factor Bifactor EFA model (MODEL 1) had acceptable fit. Similar results emerged for the Father version of ECPARQ (see
| Bifactor Models | χ2 Value | df | χ2/df | CFI | TLI | RMSEA | RMSEA Lower CI | RMSEA Higher CI | SRMR |
|---|---|---|---|---|---|---|---|---|---|
| ECPARQ MOTHER (N = 1000) | |||||||||
| BIFACTOR EFA MODELS | |||||||||
| MODEL 1 (BIFATOR EFA) 2 FACTORS (Acc..-Rej.), & 13 in Warmth | 548.11 | 229 | 2.39 | 0.900 | 0.879 | 0.037 | 0.033 | 0.041 | 0.039 |
| MODEL 2 (BIFACTOR EFA) 3 FACTORS and item 13 in Warmth | 369.91 | 207 | 1.79 | 0.949 | 0.932 | 0.028 | 0.023 | 0.033 | 0.029 |
| MODEL 3 (BIFACTOR EFA) 4 FACTORS_(W, I/N, H/A, UR) & 13 W. | 284.51 | 186 | 1.53 | 0.969 | 0.954 | 0.023 | 0.018 | 0.028 | 0.024 |
| BIFACTOR ESEM MODELS | |||||||||
| MODEL 4 (BIFACTOR ESEM) 2 FACTORS (Acc..-Rej.), & 13 in Warmth | 369.91 | 207 | 1.79 | 0.949 | 0.932 | 0.028 | 0.023 | 0.033 | 0.029 |
| MODEL 5 (BIFACTOR ESEM) 3 FACTORS (W, I/N, H/A + UR) & 13 W. | 284.51 | 186 | 1.53 | 0.969 | 0.954 | 0.023 | 0.018 | 0.028 | 0.024 |
| MODEL 6 (BIFACTOR ESEM) 4 FACTORS (W, I/N, H/A, UR) & 13 W. | NO CONVERGENCE | ||||||||
| BIFACTOR CFA MODELS | |||||||||
| MODEL 7 (BIFACTOR CFA) 2 FACTORS (Acc..-Rej.), & 13 in Warmth | 432.48 | 228 | 1.90 | 0.936 | 0.922 | 0.030 | 0.026 | 0.034 | 0.034 |
| MODEL 8 (BIFACTOR CFA) 3 FACTORS (W, I/N, H/A + UR) & 13 W. | 357.31 | 228 | 1.57 | 0.959 | 0.951 | 0.024 | 0.019 | 0.029 | 0.030 |
| MODEL 9 (BIFACTOR CFA) 4 FACTORS (W, I/N, H/A, UR) & 13 W. | 376.80 | 228 | 1.65 | 0.953 | 0.943 | 0.026 | 0.021 | 0.030 | 0.033 |
| ECPARQ FATRHER (N = 1000) | |||||||||
| BIFACTOR EFA MODELS | |||||||||
| MODEL 1 (BIFATOR EFA) 2 FACTORS (Acc..-Rej.), & 13 in Warmth | 638.83 | 229 | 2.79 | 0.914 | 0.897 | 0.043 | 0.039 | 0.047 | 0.038 |
| MODEL 2 (BIFACTOR EFA) 3 FACTORS and item 13 in Warmth | 462.16 | 207 | 2.23 | 0.947 | 0.929 | 0.035 | 0.031 | 0.040 | 0.030 |
| MODEL 3 (BIFACTOR EFA) 4 FACTORS_(W, I/N, H/A, UR) & 13 W. | 328.50 | 186 | 1.77 | 0.970 | 0.956 | 0.028 | 0.023 | 0.033 | 0.023 |
| BIFACTOR ESEM MODELS | |||||||||
| MODEL 4 (BIFACTOR ESEM) 2 FACTORS (Acc..-Rej.), & 13 in Warmth | 462.16 | 207 | 2.23 | 0.947 | 0.929 | 0.035 | 0.031 | 0.040 | 0.030 |
| MODEL 5 (BIFACTOR ESEM) 3 FACTORS (W, I/N, H/A + UR) & 13 W. | 328.50 | 186 | 1.77 | 0.970 | 0.956 | 0.028 | 0.023 | 0.033 | 0.023 |
| MODEL 6 (BIFACTOR ESEM) 4 FACTORS (W, I/N, H/A, UR) & 13 W. | 279.86 | 166 | 1.69 | 0.976 | 0.960 | 0.026 | 0.021 | 0.032 | 0.020 |
| BIFACTOR CFA MODELS | |||||||||
| MODEL 7 (BIFACTOR CFA) 2 FACTORS (Acc..-Rej.), & 13 in Warmth | 534.50 | 228 | 2.34 | 0.936 | 0.922 | 0.037 | 0.033 | 0.041 | 0.037 |
| MODEL 8 (BIFACTOR CFA) 3 FACTORS (W, I/N, H/A + UR) & 13 W. | 445.72 | 228 | 1.95 | 0.955 | 0.945 | 0.031 | 0.027 | 0.035 | 0.035 |
| MODEL 9 (BIFACTOR CFA) 4 FACTORS (W, I/N, H/A, UR) & 13 in W. | 495.32 | 228 | 2.17 | 0.944 | 0.932 | 0.035 | 0.030 | 0.039 | 0.038 |
Note: MLR robust rescaling-based estimator was used for parameters estimate in all models; W = Warmth, I/N = Indifference/Neglect, H/A = Hostility/Aggression, UR = Undifferentiated Rejection, Acceptance = Warmth factor, Rejection = Indifference/Neglect, Hostility/Aggression, Undifferentiated Rejection factors.
2) Second Order CFA Models
Higher order factor analysis is typically carried ?out because occasionally first-order factors indicate narrow-scope constructs, interconnected with a higher and broader construct represented in factor analysis by one or more higher order factors ( Cattell , 1978;
Regarding model parametrization, MLR was used as a parameter estimator for all second order models tested. It was possible to test second-order CFA models having only a four first-order factor structure because evaluating the fit improvement of the second-order factor over the first-order factor was possible only for first-order latent factors ≥ 4 (i.e. Wang & Wang, 2012 ). Consequently, only 4-factor models were tested for second-order CFA.
Their estimates of variances and covariances for the mother version (but not for the father) generated a not positive definite covariance matrix. To correct this, the residual variance was fixed at zero because in this case, it had a small negative value and it was not significant. That could suggest that Warmth dimension of parenting continuum (the first order factor) is a perfect indicator of the four second-order factors. After this modification, the first- factor model with four factors and item 13 in Warmth factor had an acceptable fit for both versions (Mother and Father) in contrast to the original model with item 13 in Indifference/Neglect factor (see
| Second Order CFA Models | χ2 Value | df | χ2/df | CFI | TLI | RMSEA | RMSEA Lower CI | RMSEA Higher CI | SRMR |
|---|---|---|---|---|---|---|---|---|---|
| ECPARQ MOTHER (N = 1000) | |||||||||
| MODEL 1 4 FACTORS Original (W, I/N, H/A, UR) | 599.07 | 249 | 2.41 | 0.890 | 0.878 | 0.038 | 0.034 | 0.041 | 0.049 |
| MODEL 2 4 FACTORS with 13 in W & I/N, H/A, UR | 487.52 | 249 | 1.96 | 0.925 | 0.917 | 0.031 | 0.027 | 0.035 | 0.039 |
| ECPARQ FATHER (N = 1000) | |||||||||
| MODEL 1 4 FACTORS Original (W, I/N, H/A, UR) | 801.19 | 249 | 3.22 | 0.885 | 0.872 | 0.047 | 0.044 | 0.051 | 0.053 |
| MODEL 2 4 FACTORS with 13 in W & I/N, H/A, UR | 655.19 | 249 | 2.63 | 0.915 | 0.906 | 0.041 | 0.037 | 0.045 | 0.044 |
Note: MLR robust rescaling-based estimator was used for parameters estimate in all models; W = Warmth, I/N = Indifference/Neglect, H/A = Hostility/Aggression, UR = Undifferentiated Rejection; Bold indicates optimal model fit.
tested for the mother version and in
Internal Consistency reliability as measured by Cronbach’s alpha coefficient (Cronbach, 1951) was satisfactory, α = 0.87 for Mother Version and α = 0.90 for Father version of ECPARQ.
The first research question of this study was to examine if the dimensionality of the Greek Early Childhood Parental Acceptance-Rejection Questionnaire (Mother/Father Version) is hierarchical. Previous research on PARQ factorial structure
suggested that high inter-factor correlations exist among the four factors (e.g. Artemis & Touloumakos, 2014 ). A similar pattern emerged in the research on ECPARQ factor structure (e.g. Giotsa & Kyriazos, 2017 ). Highly correlated first-order factors suggest that a higher order factor may be present (Hammer & Tolland, 2016; Wang & Wang, 2012) . Therefore, the present research attempted to test whether ECPARQ Mother and Father versions have a higher order factor structure.
Actually, Rohner & Khaleque (2005) assumed that the Warmth Dimension of Parenting continuum may be the general construct beneath the first-order ECPARQ dimensions (Rohner & Khaleque, 2005) , in an effort to explain high correlation between PARQ factors, so we could extend this postulation for ECPARQ too. Another issue of the empirical research on ECPARQ structure is item 13 (Pays a lot of attention to me). It is currently placed to the Indifference/Neglect latent factor (as originally proposed by Rohner & Khaleque, 2005 ) but the CFA findings suggested that fit is better if item 13 is placed in the Warmth factor (Giotsa & Kyriazos, 2017) . In what factor item 13 belongs in the Greek context was the second research question. Consequently, the focus of the study was not to establish ECPARQ factor structure but to examine whether the ECPARQ factor structure is hierarchical or not.
The data was found to have a positive skew suggesting an overuse of the lower points of the Likert Scale, thus MLR robust rescaling estimator was used to handle non-normality (Muthen & Muthen, 2012) .
Summarizing the models tested, the study examined the ECPARQ with four different higher order techniques (Bifactor EFA, Bifactor CFA, Bifactor ESEM, Second-order CFA) and a simple CFA (no higher order structure) to establish a benchmark for comparison to the higher order solutions. Thus, a total of 5 different factor analysis techniques were used, evaluating a total of 19 models. Specifically, ECPARQ models with 2, 3 and 4 factors were tested per method. Specifically, for CFA 6 alternative ECPARQ models were tested with 2, 3 and 4-factor structures. In CFA all models were tested having item 13 both in the intended factor (Indifference/Neglect; 3 models) and alternatively in the Warmth factor (3 models). This comparison was made to empirically answer in what factor the item 13 belongs (except semantic similarity). All models were tested twice, one for the ECPARQ Mother version and one for the Father version.
The following optimal models emerged per method. In the CFA without higher order structure the 3-factor CFA model with item 13 in the Warmth factor was the optimal model. This solution had a Warmth factor with 9 items, the Hostility/Undifferentiated Rejection with 10 items, emerging by collapsing Hostility/Aggression and Undifferentiated rejection into one factor and finally Indifference/Neglect with 5 items. These factors were used as the three Specific factors for the Bifactor models tested subsequently. Note that all CFA models tested had high factor correlations (>0.64; c.f. Hammer & Toland, 2016 ).
| ECPARQ MODELS WITH HIERARCHICAL STRUCTURE | |
|---|---|
| Technique | Models tested |
| Bifactor EFA | 3 models (1 two-factor, 1 three-factor, and1 four-factor model) |
| Bifactor CFA | 3 models 1 two-factor, 1 three-factor, and1 four-factor model) |
| Bifactor ESEM | 3 models 1 two-factor, 1 three-factor, and1 four-factor model) |
| Higher-order CFA | 2 models (4-factor models with residual variance fixed at zero) 2 models (4-factor models without residual variance fixed at zero) |
| TOTAL 1 | 13 MODELS |
| ECPARQ MODELS WITHOUT HIERARCHICAL STRUCTURE | |
| CFA | 6 models |
| TOTAL 2 | 6 MODELS |
| GRAND TOTAL | 19 models for each ECPARQ version (Mother & Father) |
Next, among the 9 Bifactor models evaluated and all solutions reached acceptable fit limits. This means that ECPARQ structure is tapping both on the unidimensional construct of Warmth Dimension of parenting but this construct has also sub-dimensions that must be specified (Brown, 2015) . For the second-order CFA, a general construct beneath the first order-factors was also the Warmth dimension of parenting and the first order factors were Warmth, Hostility/Aggression, Indifference/Neglect, Undifferentiated Rejection. Note that despite that this second order CFA model showed a comparably adequate fit, the 3-factor Bifactor structures were more robust across all alternative methods, thus they are considered a more reliable. However, Bifactor in general is regarded a somewhat controversial method because it has a tendency to verify itself as more optimal in comparison to the ICM CFA corresponding models (Joshanloo, Jose, & Kielpikowski, 2017; Joshanloo & Jovanovic, 2016) . Additionally, the 3-factor Bifactor fit was not impressively better than the 3 factor ICM-CFA model. When Bifactor fit is comparable to the ICM-CFA fit, ICM-CFA is preferable as the more parsimonious solution (Howard et al., 2018; Marsh, Morin, Parker, & Kaur, 2014) .
Additionally, the 4-factor structure has two first-order factors with high factor intercorrelations (Indifference/Neglect with Hostility/Aggression). Such high correlations between the two latent factors are close to the generally recommended cut-off value of 0.80 - 0.85 (Brown, 2015; Kline, 1998) suggesting dimension redundancy (Epitropaki & Martin, 2004) . Note also that in the 4-factor second order structure, the latent variable covariance matrix (psi) was not positive definite. To overcome this, the residual variance of the model was constrained at zero because variances “by definition cannot be negative” (Muthen, 2018) . Two conditions must be true for constraining a negative residual variance at zero: 1) the residual variance must have a small and negative value and 2) it must be non-significant (c.f. Mplus, http://www.statmodel.com/). Both conditions were true in the case of ECPARQ. The zero residual variance suggested that the four first-order ECPARQ factors of Warmth, Indifference/Neglect, Hostility/Aggression, and Undifferentiated rejection were perfect indicators of the second-order factor of Warmth dimension of Parenting. Nevertheless, all the additional parametrization necessary to produce the second order ECPARQ model rendered the resulting solution somewhat dubious, and not supported by similar findings across methods.
Returning to the research questions, questions 1) and 2)―Has ECPARQ a hierarchical structure? (Warmth dimension of Parenting)―were verified because all higher order models had an adequate fit, suggesting thus the theoretical construct of the Warmth dimension of Parenting is supported. This could also explain 1) high factor inter-correlations, 2) dimensionality debates (Hammer & Tolland, 2016) . Regarding the questions 2) and 3) about the factor of item 13, all models tested (with no exception) having item 13 in the Warmth factor had superior fit in comparison to their counterparts with item 13 in the Indifference/Neglect factor. This supports the initial assumption that item 13 should be in the Warmth factor. Some could argue that, item13 is reversed so it is a measure of neglect. However, Warmth factor items are also reversed forming in essence a coldness factor (Artemis & Touloumakos, 2014) . Plus, semantic affiliation of coldness and Indifference/Neglect items causes too high correlations and raises issues of over-factoring. The preciseness of scoring is also impaired by keeping 13 in the Indifference/Neglect factor.
For all the reasons described above, the 3 factor ICM CFA (simple CFA) solution is proposed as the optimal solution for both ECPARQ versions for the Greek cultural context because it was more parsimonious than its counterpart Bifactor solutions which all showed similar results. Besides adequate sample size, further investigation is required to verify findings.
Finally, internal consistency reliability was evaluated with Cronbach’s alpha coefficient (Cronbach, 1951) . Coefficients for the total scale of both versions were adequate.
To recap, the two research questions were the following: 1) High correlations between the factors in the 2-factor, 3-factor and 4-factor ECPARQ models exist. Does this mean that a higher-order factor may be present in ECPARQ? Rohner & Khaleque (2005) proposed that this higher-order construct may be the opposite poles of the Warmth dimension of parenting. Could the Warmth dimension of parenting be the higher order construct of ECPARQ? Is this higher order structure theoretically supported? 2) Is item 13 (“Pays a lot of attention to me”) an Indifference/Neglect item or a Warmth item? Answers to the research questions were the followings: 1) ECPARQ has a hierarchical structure, because the higher order models showed better fit from the simple CFA models. 2) All CFA models with item 13 in the Warmth factor were better than their equivalent models with item 13 in the Interference/Neglect factor. Therefore, for the Greek culture, if item 13 is allocated to the warmth factor, the model fit is better. These findings are supported by similar findings for the ECPARQ factor structure in Greece (Giotsa, Theodoropoulos, & Kyriazos, 2018) .
A limitation of the present research is that the ECPARQ results must be compared with caution to the PARQ results because of their differences in the answering procedure. Future research must revalidate this newly found higher order factor structure in different samples and cultures and must corroborate the factor that item 13 belongs. Finally, the research questions of the present research should be also extended to the PARQ short factorial structure.
The authors declare no conflicts of interest regarding the publication of this paper.
Giotsa, A., & Kyriazos, T. A. (2019). Early Childhood Acceptance Rejection Questionnaire: Psychometric Properties of the Greek Version. Psychology, 10, 722-741. https://doi.org/10.4236/psych.2019.105047