The primary aim of this study is to explore the applicability of the Careers 4.0 Competencies Framework within the context of contemporary labor market changes. More specifically, the study examines the development and initial evaluation of the Careers 4.0 Skill Assessment Tool, an ICT-based instrument designed to assess future-oriented and transversal competencies relevant to Industry 4.0 working environments. The framework also draws conceptually on Holland’s RIASEC model by attempting to connect vocational interests with broader employability competencies associated with changing labor market conditions.

To support the practical use of the framework in vocational education and training (VET) and career guidance contexts, the present study describes the development process, linguistic adaptation, and preliminary psychometric evaluation of the instrument. The broader objective is to contribute to career support practices by providing an exploratory competency profiling tool intended to support career adaptability, self-awareness, and career decision-making in rapidly changing employment environments.

To explore and refine the theoretical constructs of the Careers 4.0 Competencies Framework, an initial participatory research approach was adopted. Focus groups involving 26 stakeholders—including human resource managers (26%), career counselors (17%), entrepreneurs (13%), and other VET-related professionals—were conducted across Greece, Italy, Cyprus, and Romania ([11]; [15]). Participants evaluated the preliminary framework in terms of perceived labor market relevance and practical applicability. Their qualitative and quantitative feedback informed the refinement of the competency framework.

The Careers 4.0 Skill Assessment Tool was developed as a closed-ended self-report questionnaire using a 3-point Likert response format. During the initial development stage, an item pool of 123 statements was generated across the 18 competency domains in order to allow item selection and refinement during subsequent psychometric analyses. The relatively large initial pool was intended to ensure adequate coverage of each competency while allowing for the removal of weaker items during the validation process ([11]; [15]; [9]).

The instrument was originally developed in English and subsequently translated into Greek, Romanian, Italian, and Spanish using a back-translation procedure ([3]; [5]). The adaptation process aimed to support conceptual and linguistic equivalence across participating countries for the target group of young people aged 16 - 29 ([11]). In each country, the procedure involved three stages: initial translation by a bilingual expert, independent back-translation into English by a second expert, and final review and synthesis by a third expert. Discrepancies were discussed collectively until consensus was achieved ([15]; [4]).

The final instrument was designed to retain four items per competency domain in order to achieve adequate construct coverage while maintaining a concise format suitable for career guidance and VET settings.

The preliminary psychometric evaluation of the instrument was conducted using a cross-national convenience sample of 303 young participants from five European countries. Eligibility criteria included being between 16 and 29 years of age and providing informed consent to participate in the study. The sample comprised 137 males (45%) and 166 females (56%).

Participants were recruited from educational settings and included high school students, vocational education and training (VET) students, and university students. Country representation included Romania (n = 75), Italy (n = 75), Greece (n = 60), Spain (n = 53), and Cyprus (n = 40). The age range of participants was 16 - 29 years.

Given the exploratory nature of the study, a convenience sampling approach was adopted to obtain an initial cross-national sample suitable for the preliminary evaluation of the Careers 4.0 Skill Assessment Tool.

2.4.1. Content Validity Assessment

Content validity was initially examined using expert evaluation procedures. A panel of 15 subject-matter experts (three from each participating country) reviewed the preliminary items in terms of relevance, clarity, and potential bias using a 3-point Likert scale ([20]). Item-Level Content Validity Index (I-CVI) values were calculated, and items with values below 0.79 were considered for revision or removal ([24]).

2.4.2. Procedures for Item Analysis and Construct Evaluation

Initial item analysis was planned to evaluate the discriminative power of the statements, with the intent to eliminate items exhibiting poor construct validity or deleterious effects on scale reliability ([14]; [8]).

Exploratory construct evaluation was subsequently conducted using Principal Component Analysis (PCA) with Varimax rotation. Because the instrument was developed around 18 theoretically defined competency domains, PCA was conducted separately for each competency scale in order to examine the internal coherence of the individual constructs. Although PCA primarily represents a data reduction approach rather than a latent variable model, it was considered appropriate for this preliminary exploratory phase of instrument development. Sampling adequacy was assessed using the Kaiser-Meyer-Olkin (KMO) index ([19]) and Bartlett’s Test of Sphericity. Factor retention was guided by the examination of eigenvalues and the interpretability of the resulting factor solution. Varimax rotation was selected to facilitate a simpler and more interpretable factor structure. Items with factor loadings below 0.40, low communalities, or substantial cross-loadings (>0.30) were considered for exclusion ([14]; [8]).

2.4.3. Internal Consistency Reliability Assessment

The internal consistency of the retained scales was evaluated utilizing Cronbach’s alpha ([22]).

For practical interpretation within career counseling contexts, raw participant scores were rank-ordered and converted into deciles (0 - 10) for automated profiling. Further inferential statistical analyses, specifically independent samples t-tests and Analysis of Variance (ANOVA), were conducted to explore demographic variances based on gender and country of origin.

Based on the qualitative and quantitative feedback obtained from the expert focus groups, several theoretical constructs within the preliminary framework were revised. For example, organisational skills and work efficiency were merged into a broader “Organisation” competence, while “Green competence-environmental awareness” was added as a separate competency area. These revisions resulted in a final framework consisting of 18 competencies ([11]; [15]).

The initial item pool consisted of 123 items distributed across the 18 competency domains. Following expert panel evaluation, five items did not meet the pre-defined I-CVI threshold (I-CVI < 0.79) and were removed. The resulting 118-item version demonstrated satisfactory content validity, with Scale-Level Content Validity Index (S-CVI) values ranging from 0.95 to 0.99 and Item-Level Content Validity Index (I-CVI) values ranging from 0.80 to 1.00.

Subsequent item analysis and exploratory factor evaluation resulted in the removal of 46 additional items from the 118-item version. Items were excluded because of weak discrimination indices, low factor loadings, low communalities, or substantial cross-loadings. The retained items demonstrated discrimination indices ranging between 0.20 and 0.80 ([14]; [8]). Following this process, the final version of the Careers 4.0 Skill Assessment Tool consisted of 72 items distributed across the 18 competency scales. Each competency scale retained four items, resulting in a balanced structure across the 18 domains and a total of 72 items in the final instrument.

For the exploratory factor analysis procedures, Kaiser-Meyer-Olkin (KMO) values indicated acceptable sampling adequacy across the examined scales (range: 0.69 - 0.79), while Bartlett’s Test of Sphericity was statistically significant in all cases (p < 0.001), supporting the suitability of the data for exploratory factor analysis ([14]; [8]).

Based on the predefined item retention criteria, several items were removed due to low factor loadings, low communalities, or substantial cross-loadings. For example, items 33 and 82 were removed from the Communication scale because they demonstrated cross-loading patterns with Emotional Competence. The retained items demonstrated factor loadings ranging from 0.42 to 0.74. Overall, the exploratory analyses provided preliminary support for the internal structure of the individual competency scales ([14]). Across the competency scales, the retained factor solutions were consistent with the expected unidimensional structure of the corresponding competency domains.

The instrument demonstrated acceptable to strong internal consistency. Cronbach’s alpha coefficients ranged from 0.71 to 0.85 across 17 of the competency scales, with a marginal but acceptable alpha of 0.69 recorded for the Ethical Competence scale ([22]). Detailed psychometric information for all 18 competency scales, including factor loading ranges, removed items, and internal consistency coefficients, is presented in Table 1.

Exploratory inferential analyses indicated several demographic differences across competency scores. Male participants reported higher scores in digital skills (p = 0.009, Cohen’s d = 0.31), whereas female participants reported higher scores in emotional competence (p = 0.034, Cohen’s d = 0.26), organisation (p = 0.029, Cohen’s d = 0.27), and literacy (p = 0.005, Cohen’s d = 0.35). According to conventional benchmarks, these effect sizes were small.

In addition, Analysis of Variance (ANOVA) indicated statistically significant differences across participating countries for adaptability (η² = 0.08), communication (η² = 0.07), and environmental awareness (η² = 0.10). Tukey HSD post-hoc comparisons revealed significant differences between several country pairs, with the most consistent differences involving Italy and other participating countries. Given the exploratory nature of the study and the use of a convenience sample, these findings should be interpreted cautiously and may warrant further investigation using larger and more representative samples.

Table 1.Psychometric properties of the careers 4.0 skill assessment tool: construct validity and internal consistency.

Several limitations of the present study should be acknowledged. The exploratory construct evaluation relied on Principal Component Analysis (PCA), which primarily represents a data reduction approach rather than a latent variable modeling technique. Although PCA is commonly used during initial stages of instrument development and item reduction, future studies could further examine the internal structure of the instrument using approaches such as Principal Axis Factoring (PAF) or Confirmatory Factor Analysis (CFA). In addition, the study utilized a cross-national convenience sample of 303 participants. While the sample was considered adequate for this preliminary exploratory phase, larger and more representative samples would strengthen the generalizability and stability of the findings ([8]). Internal consistency was also examined exclusively through Cronbach’s alpha coefficients, and future research may benefit from the inclusion of additional reliability indicators such as McDonald’s Omega (ω) ([22]).

An additional limitation concerns the cross-national nature of the study. Although the instrument underwent a structured translation and back-translation procedure, measurement invariance across the different language versions was not formally tested. Consequently, cross-country comparisons should be interpreted as exploratory, as observed differences may reflect both substantive variation and potential differences in the interpretation of translated items. Future research should examine measurement invariance across language versions using larger cross-national samples.

Another limitation concerns the conceptual integration of the Careers 4.0 competencies with Holland’s RIASEC framework. Although the competencies were theoretically associated with the RIASEC typologies and ESCO classifications, the present study did not empirically examine the circumplex structure of the RIASEC model or the positioning of the competencies within this structure. Furthermore, the decile-based profile indicators generated by the tool should be interpreted as preliminary reference scores intended to support career exploration and guidance rather than formal diagnostic norms. Since the scoring system was developed using a convenience sample, future research should aim to establish broader normative data using more representative sampling procedures. Additional longitudinal studies may also help examine the relationship between competency profiles, career adaptability, and future educational or employment outcomes.