TITLE:
Applied Psychometrics: The 3-Faced Construct Validation Method, a Routine for Evaluating a Factor Structure
AUTHORS:
Theodoros A. Kyriazos
KEYWORDS:
3-Faced Construct Validation Method, Validity, Reliability, Cross-Validation, Replicability, Overfitting, Over-Optimization, CFA, EFA, Factor Analysis
JOURNAL NAME:
Psychology,
Vol.9 No.8,
August
8,
2018
ABSTRACT: The “3-faced construct validation method” is a routine for establishing the validity and reliability of an existing scale when adapted in a different cultural context from the context initially developed. This routine can also be used for the initial validation of a newly developed scale. This is essentially a construct validation procedure based on a sample-splitting. The sample is randomly split into three parts, 20% for EFA, 40% for an exploratory CFA and 40% for a cross-validating CFA. The cases per variable threshold is set above 5:1, preferably above 10:1 (minimum conditions) and the first approximately 20% subsample emerges (adequate conditions) to evaluate EFA and Bifactor EFA models. Then the cases per variable threshold is set above 10:1, preferably above 20:1 (minimum conditions) and the 40% subsample emerges (adequate conditions) to examine alternative CFA models (ICM-CFA, Bifactor CFA, ESEM and Bifactor ESEM models). The optimal model(s) is cross-validated by a second CFA in yet another 40% subsample (equal-power CFA sample) as a protection against overfitting (over-optimization) to safeguard model replicability. Measurement invariance follows and is essentially another cross-check of the optimal model over the entire sample because the optimal model is used as the baseline model.