Abstract

The purpose of the study is to evaluate the quality of the teaching, training and learning materials content which is currently in use in Ethiopian federal technical and vocational training institute in order to give suggestion/recommendation on its weak and strong side. A mixed method, both a questionnaire and an observation were two of the data gathering methods employed in the study. Questions such as organization and clarity, use of visual representations, update and relevance of contents, active learning and engagement, alignment with the objectives, supervision and evaluation were distributed and fourteen qualified respondents participated. After that, SPSS Software was used to analyze the data gathered from the questionnaire and gathered information was input into the Excel program. Following that, the data was imported into SPSS Software, which was used to calculate and analyze the frequencies, means, mode and standard deviations of each individual item in order to determine the trainers’ expectations and actual needs. The analysis shows several weaknesses in their structure, engagement, relevance, and evaluation process. A significant portion of respondents remained neutral regarding the clarity and organization of the content, indicating uncertainty about its effectiveness. Overall, the findings suggest that while the training materials have a foundation, they require significant improvements to become more structured, engaging, and up-to-date.

Keywords

Training Materials, Evaluation, Quality, Vocational Education, Ethiopian TVET, SPSS Software, Analysis

Share and Cite:

1. Introduction

The Technical and Vocational Education Training System in Ethiopia focuses on addressing skills shortages in the economy by providing training for youth and young adults. Primarily, it is designed to prepare new graduates for employment opportunities. Recently, the system has also introduced short term training lasting several months to help unskilled individuals gain entry-level skills.

The TVET system in Ethiopia not only aims to fill skills gaps in the economy by training youth and young adults, but it also has initiatives at the municipal level to support the unemployed. Some companies collaborate with TVET institutions to provide specialized training, and there are non-formal TVET programs available for individuals looking to enhance their employability. Established in the 2000/01 academic year, the TVET system has been included in all the subsequent five-year development plans, starting with a focus on increasing enrolment in the SDPRP (2002-2005). This structured approach reflects a commitment to developing a skilled workforce that can meet the demands of the job market [1].

A high graduate unemployment rate is the result of job competitiveness brought about by rapid technological advancements that are changing occupational profiles and requiring people to acquire new knowledge and skills. It calls for educational and training establishments to constantly update their curricula to incorporate cutting edge technologies in order to adapt to the shifting demands of the labour market. Skills mismatch is a global issue that has led to unemployment and a skills shortage in Ethiopia, according to a large number of researches [2].

2. Teaching, Training, and Learning Materials (TTLMs)

Teaching, training and learning materials (TTLMs) are crucial for enhancing the quality and effectiveness of vocational education in Ethiopia. These materials provide the essential framework for the educational process, offering structured content, practical guidance, and standardized curricula that align with national occupational standards and industry needs. In Ethiopia, where vocational education is vital for tackling unemployment and promoting economic growth, TTLMs ensure students gain both theoretical knowledge and practical skills necessary for the workforce. They effectively connect classroom learning with real-world application, allowing learners to master tools, technologies, and techniques relevant to their fields [3].

In addition, TTLMs aid instructors by providing lesson plans, assessment resources, and teaching strategies, which in turn enhance the overall quality of instruction. However, despite their significance, several challenges, including limited availability, poor quality, and language barriers, often impede their effective use. Addressing these challenges is critical to achieving the objectives of vocational education in Ethiopia, which aims to produce a skilled and job-ready workforce capable of contributing to the nation’s industrialization and economic advancement. This study examines the quality of TTLMs in vocational education and their influence on the quality of training and learning outcomes in Ethiopia [4].

3. Importance of Vocational Education Teaching, Training and Learning Materials and Evaluation

3.1. Importance of Vocational Education Teaching, Training and Learning Materials

The importance of training materials evaluation is pivotal for improving technical and vocational education and training (TVET) in Ethiopia. This process helps to refine policies and strategies, identify skill gaps, and enhance the knowledge of trainers. By systematically assessing teaching, training and learning materials, research can significantly benefit trainees, trainers, and institutions. Effective evaluations not only contribute to improving the quality of training resources but also have a broader positive impact on the economy by ensuring that the teaching materials meet industry standards and equip learners with the necessary skills for the job market. In this way, the evaluation of textbooks and teaching resources plays a crucial role in advancing vocational education and ultimately fostering economic development in Ethiopia [5].

3.2. Evaluation

Evaluation is defined in various ways, influenced by the subject matter, methodology used, or how its results are applied. According to the Longman Dictionary of Contemporary English, the most fundamental definition of evaluation is “the act of considering something to decide how useful or valuable it is [6]”.

Training materials must be evaluated at each level of use in order to identify their shortcomings and make necessary improvements. Numerous intricate challenges and tightly linked components make up training materials review, and a number of elements influence whether a training material is successful or not [7].

According to [8] the seven goals for educational evaluations are:

1) To validate methodologies and instruments for needs assessment.

2) To verify or update available solutions.

3) To validate or update training methods.

4) To ascertain the responses of trainers.

5) To evaluate the attitudes and information that trainees have acquired.

6) To evaluate the performance of trainees.

7) To ascertain whether organizational objectives are fulfilled. Three main categories of assessment are distinguished by [9] for program development, causal knowledge (i.e., “to generate strong evidence that the intervention causes the intended outcomes”), and accountability (to funders and other stakeholders).

Material evaluation highlights the importance of determining whether specific elements are present in the materials you are assessing. Once these elements are identified, it becomes crucial to evaluate their value. This evaluation process is inherently subjective, as it involves making judgments. However, McGrath warns that in this evaluative process, one might overlook unique and innovative aspects of the materials [10].

The motivations behind evaluating training materials are diverse. One key reason is the need to select new training materials. Evaluating materials also serves to uncover the strengths and weaknesses of training materials currently in use [11]. Materials evaluation can enhance our understanding of how these resources function, ultimately contributing to acquisition theory and teaching methodologies. Moreover, this process can serve as a form of action research [12].

The “perfect material does not exist,” but the goal is to identify the most suitable one for a specific learner group. Training materials provide significant advantages for both trainees and trainers when utilized effectively in training. Training materials are a useful tool for self-directed learning, great for presenting information, a source of ideas and activities, a reference for trainees, a syllabus that reflects pre-established learning objectives, and aids in the confidence-building of less experienced trainers. One of the main benefits of reading training material is that they are psychologically necessary for trainees since they allow for the tangible measurement of their growth and accomplishment [13].

Material evaluation may be divided into three categories. The “predictive” or “pre-use” evaluation, which looks at a training materials prospective or future performance, is arguably the most popular type, they contend. The “in-use” assessment, which looks at content that is presently being used, and the “retrospective” or “post-use” (reflective) evaluation, which looks at a textbook that has been used at a particular institution, are the other forms of training material evaluation. Although training materials play a significant role in the majority of Automotive Technology courses, little research has been done on how and why instructors choose the materials they use [14].

4. Conceptual Framework Understanding Quality in Education

There is no one, widely recognized definition of quality in training; instead, it is defined from a variety of viewpoints, literatures assert that training is a process of personal transformation, excellence in training and can be defined as the participant’s empowerment and improvement. In order to improve the participant, quality entails adding value to the participant in the form of trainees’ knowledge, skills, and capacities [15].

5. Challenges of Technical and Vocational Education Training Trainers in Ethiopian

A number of obstacles stand in the way of the full implementation of competency based training in Ethiopia. These obstacles include a lack of communication and cooperation among stakeholders, a lack of competent technical and vocational education training trainers and low competency among their colleagues, inadequate facilities and infrastructure, and a total of 1403 TVET colleges and institutions in Ethiopia in 2023 of which 50.67% were private, 45.8% were public, and 3.52% were non-governmental organizations (NGO) TVET colleges. Additionally, Competency based training has not remained clearly aligned with occupational standards [16].

Competency based training is typically a top-down procedure, with parents and businesses/organizations not being adequately involved in its creation. Furthermore, trainers’ professional growth and consistent, ongoing training are severely lacking.

This policy brief focuses on the difficulties and shortcomings of TVET trainers’ professional ability, despite the fact that the issues facing TVET in Ethiopia are complex. For TVET trainers, it suggests an updated competency and qualification system. Additionally, it suggests to policymakers a few measures that would enhance TVET and adequately equip the workforce for the demands of the sector and the growing labor market. It contends that highly qualified, driven, adaptable, and innovative TVET trainers are crucial to Ethiopia’s success in meeting international standards [16].

6. Design and Methodology

A mixed method, both a questionnaire and an observation were two of the data gathering methods employed in this study. Based on the study’s goals, a questionnaire was created for this purpose. The purpose of the questionnaires was to assess more information about the expectations of the experienced trainers regarding the training materials and other pertinent information. In this study, a Likert scale (as shown in Table 1) was used, and participants were asked to select “strongly disagree, disagree, neutral, agree, or strongly agree”.

Table 1. Likert scale range weight [18].

For questions in the categories of structure or design, content, coverage, trainees’ consideration and evaluation schedule. The questionnaire contained 14 items in the training rating survey.

In this case, it was felt that an accurate representation of the trainees’ aims, concerns, interests, expectations, and views on training methodology would help with the overall training material evaluation process by providing a clearer picture of the compatibility between actual trainees’ needs and the perceived goals and objectives of the training material content [17].

6.1. Data Collection and Analysis

Questionnaires in teaching, training and learning materials were distributed to selected trainers. To collect data from the trainers, the Questionnaires was distributed with a Google forms online with a brief description of the goal and nature of the study, and if they had problems interpreting any item, they were assisted via email. Following instrument collection, it was gathered for data processing. To summarize the trainers’ opinions of the training material, the means of the trainers’ in each individual item were computed.

6.2. The Study’s Design

The main advantage of this research design is the utilization of a mixed method approach, which combines both quantitative and qualitative aspects of data collection. By using a variety of data gathering techniques, biases that could arise from depending solely on one technique were reduced. Closed-ended questionnaires were used to get quantitative data, and observation was used to gather qualitative data [19].

7. Result and Discussion

Every response was quantified in order to examine the respondents’ answers to the surveys. Grouping the items based on relevant subjects was necessary to facilitate the analysis. These subjects provided a clear picture of the responses provided to the 14 questionnaire items and were presented in an easy-to-understand and straightforward manner. After that, SPSS Software was used to analyze the data gathered from the questionnaire. Initially, the gathered information was input into the Excel program. Following that, the data was imported into SPSS Software, which was used to calculate and analyze the frequencies, means, mode and standard deviations of each individual item in order to determine the trainers’ expectations actual needs.

From Table 2 the weighted average or grand mean is the summation of all the individual means divided by number of items.

$\overline{x}=\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{x}_i}$

where, $\overline{x}$ = Grand mean, ${\displaystyle {\sum }_{i=1}^n{x}_i}$ = the summation of all means ($x_1$ , $x_2$ , ···,$x_n$ ) and n = Total number of means.

$\begin{array}{l}=\frac{3.57+3.43+3.43+3.57+3.21+3.29+3.64+3.29+2.5+2.57+2.43+2.07+2.57+2.07}{14}\\ =2.974\end{array}$

This result shows that the respondents have a low perception on the items

Table 2. The frequencies, means, mode and standard deviations of each individual item.

whose means below the grand mean (2.97) and have a high perception on the items whose means above the grand mean.

The data in Table 3 presents responses from the respondents evaluating various aspects of training materials. The responses are categorized into five levels: 1-Strongly Disagree/SDA, 2-Disagree/DA, 3-Neutral/ N, 4-Agree/A, and 5-Strongly Agree/SDA. Below is a breakdown of the key findings:

The analysis of the training materials indicates several areas that require improvement. A majority of respondents (71.4%) remained neutral regarding the clarity and transparency of the content, suggesting that while the material is somewhat organized, there is room for better structuring. Similarly, 64.3% of participants were neutral about whether the content aligns with training objectives, while 57.1% remained uncertain about its connection to the curriculum’s learning goals, highlighting the need for clearer integration. The comprehensiveness of the materials also appears to be lacking, as 71.4% of respondents neither strongly agreed nor disagreed that the materials cover a wide range of topics or provide sufficient additional resources. Only 7.1% agreed that the materials offer

Table 3. The frequency of the respondents in percentage.

broad content and useful links, indicating a need to expand topic coverage and include more supporting materials.

Furthermore, the training materials do not seem to strongly promote independent learning, as 78.6% of respondents remained neutral on this aspect, while only 21.4% agreed. While some activities are included at various skill levels, with 50% being neutral, 42.9% agreeing, and 7.1% strongly agreeing, engagement could be enhanced by incorporating more diverse and interactive elements. The use of realistic visuals, such as photographs, graphs, and diagrams, also appears to be an area where improvements could be made, as 64.3% remained neutral, while only 21.4% agreed that they effectively depict reality.

One of the most critical issues identified in the analysis is the lack of periodic updates to the training materials, with 71.4% of respondents disagreeing that the content is updated regularly. Additionally, there is a noticeable gap in evaluation and supervision, as 92.9% disagreed that there is an effective review process in place. The materials are neither assessed by industry experts, with 50% strongly disagreeing and 50% disagreeing, nor systematically evaluated by trainers, where 50% disagreed and 50% remained neutral. Similarly, the evaluation by trainees is also inadequate, with 57.1% disagreeing, 14.3% strongly disagreeing, and 28.6% remaining neutral, indicating a significant weakness in quality control.

Overall, the training materials appear to lack structure, engagement, and relevance, with outdated content and an absence of formal evaluation processes. To improve their effectiveness, it is essential to enhance their clarity, expand topic coverage, integrate modern methodologies, and establish a structured evaluation system that involves trainers, industry experts, and trainees.

An observation was also conducted regarding six category points such as organization and clarity, use of visual representations, update and relevance of contents, active learning and engagement, alignment with the objectives, supervision and evaluation.

From the observation I have conducted the areas which needs improvement are listed as follows:

1) Use of visual representations: This area needs to attach video links and supporting materials.

2) Update and relevance of contents: Most of the training materials are outdated and it should be aligned with the current situation and with the emerging technologies.

3) Active learning and engagement: Most of the trainers follow trainer center and trainees center should be exercised.

4) Alignment with the objectives: It is aligned but their relevance is not this much.

5) Supervision and evaluation: There is no supervision and evaluation and there should be scheduled supervision and evaluation.

8. Conclusion

The analysis of the training materials highlights several weaknesses in their structure, engagement, relevance, and evaluation process. A significant portion of respondents remained neutral regarding the clarity and organization of the content, indicating uncertainty about its effectiveness. Additionally, while the materials align to some extent with training objectives and curriculum goals, many respondents were unsure about their comprehensiveness. The lack of periodic updates and modern methodologies suggests that the materials may be outdated and require revision. Furthermore, the absence of a structured evaluation process whether by industry experts, trainers, or trainees raises concerns about the quality control and effectiveness of the training materials. Without a proper review system, the content risks becoming stagnant and less useful for trainees. Overall, the findings suggest that while the training materials have a foundation, they require significant improvements to become more structured, engaging, and up-to-date.

9. Recommendations

1) Improve Clarity and Structure: Enhance the organization of training materials by ensuring a logical flow from broad concepts to specific details. Clearer formatting, headings, and summaries can help trainees navigate the content more effectively.

2) Expand Topic Coverage and Resources: Include a wider range of topics and provide additional learning materials, such as supplementary readings, videos, and external references, to support independent learning.

3) Enhance Interactivity and Engagement: Integrate more diverse activities tailored to different knowledge and skill levels to encourage active participation. Visual aids like diagrams, photographs, and real-world examples should be utilized effectively.

4) Update Content Regularly: Establish a schedule for periodic content updates to reflect current industry trends, methodologies, and best practices, ensuring the materials remain relevant and useful.

5) Implement a Structured Evaluation System: Introduce a formal evaluation process involving trainers, industry experts, and trainees to assess the effectiveness of the training materials. Regular feedback collection and revision cycles should be implemented.

6) Encourage Independent Learning: Modify the materials to foster self-guided learning by incorporating interactive digital resources, self-assessment quizzes, and case studies to promote critical thinking and application.

By addressing these recommendations, the training materials can be significantly improved to better align with learning objectives, engage trainees effectively, and ensure continuous quality improvement.

Acknowledgements

First of all, I want to express my gratitude to God, the Almighty, for providing me with direction as I worked on this study. With great appreciation, I would like to express my gratitude to my supervisors Professor Wenping Zhao and Professor Wang Zhongmin for their time and assistance in helping me to complete my article.

Dual Publication

The results, tables in this manuscript have not been published elsewhere and not under consideration for publication.

Authorship

I have read the journal guideline and submitted the manuscript.

Third Party Material

All of the materials are owned by me/there is no material owned by third party in this manuscript.

Authors’ Contributions

Aregawi Gebremedhin Girmay: Reviewing literatures, data collection, analyzing, interpreting data, and writing the paper. Professor Wenping Zhao and Professor Wang Zhongmin—writing reviewing, editing, comment and suggesting. All authors have read and agreed to the published version of the manuscript.

Conflicts of Interest

The authors declare no conflicts of interest.

Conflicts of Interest

The authors declare no conflicts of interest.

References