Textbooks play a fundamental role in language teaching, providing a structured framework for educators and learners ( Prabhu, 1987 ). They serve as crucial resources for conveying knowledge and supporting teachers in achieving educational objectives ( Ridwan et al., 2021 ). Additionally, they offer a systematic approach to language instruction, fostering self-directed learning among both teachers and students ( Ellis, 1997 ). Textbook evaluation operates at three levels: pre-use, in-use, and post-use ( Cunningsworth, 1995 ; Ellis, 1997 ). Pre-use evaluation, also known as predictive evaluation, forecasts the potential performance of content for future use ( Litz, 2005 ; McGrath, 2002 ; Tomlinson, 2010 ). In-use evaluation, or retrospective evaluation, assesses materials currently in use, while post-use evaluation reflects on the effects of materials on users ( Litz, 2005 ; McGrath, 2002 ; Tomlinson, 2010 ). Cunningsworth (1995) proposed two main methods for textbook evaluation: general impression evaluation and in-depth evaluation. Studies have evaluated textbooks to determine how exercises align with syllabi and learners’ needs, identify strengths and weaknesses in textbooks, and conduct action research ( McDonough & Shaw, 1993 ; Cunningsworth, 1995 ). Others assess whether textbooks effectively convey knowledge, evaluate their productivity and value, or examine their suitability for students’ needs, teaching objectives, and methodology in specific contexts ( Sarem et al., 2013 ; Salehi et al., 2015 ; Hanifa, 2018 ). Ultimately, evaluations aim to enhance the quality of textbooks and instructional practices. Most research focuses on evaluating a series of textbooks or conducting a comparative evaluation between textbooks using a checklist ( Johar & Azia, 2019 ; Sedaghatgoftar, 2022 ), especially from the perspective of teachers.

In Malaysia, textbooks are authored by specialists and approved by the government, as in many developing countries. Evaluating textbooks in use can provide practical recommendations for curriculum development and instructional improvement.

The Common European Framework of Reference for Languages (CEFR), acting as “a common basis for the elaboration of curriculum guidelines, language syllabuses, textbooks, examinations, etc.” ( Council of Europe, 2001 ) was developed to establish international standards of textbooks, assessment, teaching and learning of language ( Uri & Aziz, 2018 ), ensuring they support a structured progression from A1 (basic user) to C2 (proficient user) proficiency levels.

Therefore, continuous evaluation and adaptation of textbooks are essential to ensure they meet CEFR standards and address the broader sociocultural dynamics of language use in a global context ( Al-Jarf & Mingazova, 2020 ; Karababa & Sarac-Suzer, 2010 ; Khodadady & Samavarchi, 2016 ). Also, in 2018, Malaysian educational institutions incorporated the CEFR structure into their English as a Second Language (ESL) curriculum which calls for the evaluation of CEFR-aligned textbooks in Malaysia ( Johar & Aziz, 2019 ; Rahim & Daghigh, 2019 ; Aripin & Yusoff, 2022 ; Shak et al., 2021 ).

According to Schmitt, Schmitt, and Clapham (2001) , vocabulary items are thought of as linguistic building blocks. And textbooks are the primary sources for learning vocabulary. Previous studies which were corpus-based studies ( Alsaif & Milton, 2012 ; Eldridge & Neufeld, 2009 ; O’Loughlin, 2012 ; Rahmat & Coxhead, 2021 ) revealed the extent of deficiencies in vocabulary loading and distribution patterns and the ignorance of publishers and textbook writers on the importance of linking usage to established wordlists, like the GSL and the Oxford 3000 (which indicate the most frequently used words in the English language. The textbooks not only broke rules in terms of frequency of use (according to wordlists), they also lacked in terms of pedagogical efficiency, like that of repetition and recycling of words which help with recall of words ( Thornbury, 2006 ).

Past studies using checklist have done and found some deficiencies in vocabulary presentation in Malaysian textbooks.

Aripin & Yusoff (2022) examined the vocabulary content in the Super Minds (Student’s Book 1) used for teaching English in Malaysian public schools. It surveyed 180 primary school teachers to assess the textbook’s content, physical, and overall quality, specifically focusing on vocabulary usage and appropriateness. Results indicated average to good ratings for content and physical quality, with moderate overall quality. A notable outcome was the development of a comprehensive rating scale for textbook evaluation, including criteria related to vocabulary usage and relevance.

Bakar & Ismail (2021) explored the presentation of vocabulary items in the Year 5 English language textbook “English Plus1” used in Malaysian schools and found the textbook does an excellent job of presenting vocabulary, but no particular approach was used to teach the new vocabulary, and there was no glossary of the new vocabulary at the end of the textbook.

Furthermore, Chujo (2004) conducted a study to compare the vocabulary levels of Japanese junior and senior high school texts, Japanese college qualification tests, English proficiency tests, and EGP, ESP, and semi-ESP college textbooks. Alsaif and Milton (2012) highlighted the issue of very little vocabulary uptake by English as a foreign language (EFL) learner in public schools in Saudi Arabia. While factors such as teaching methodology and learner motivation have been suggested to explain this, the vocabulary input received by these learners remains uninvestigated. Hussin et al. (2016) evaluated the presentation of new vocabulary items in the Form Three English language textbook used in Malaysian secondary schools. The purpose of this study is to describe the types of vocabulary tasks in EFL textbooks.

Taimoor et al. (2021) undertook a comparative assessment of English textbooks utilized in Intermediate-level education within Pakistan and Saudi Arabia. Employing corpus linguistics methodology, the research scrutinized the vocabulary content of these textbooks, specifically concentrating on corpus size, word types, and adherence to CEFR vocabulary standards. The findings revealed notable deficiencies in vocabulary coverage across both contexts, with Pakistani textbooks generally demonstrating a more expansive corpus and richer vocabulary compared to their Saudi Arabian counterparts. In addition, Yanuar (2022) conducted a study to outline the types of vocabulary tasks found in EFL textbooks, particularly focusing on “English on Sky 1” for Junior High School students. The studies’ results indicate a diverse range of task types, with varying emphasis on different cognitive skills such as analysis, knowledge acquisition, comprehension, application, and evaluation. Lower-order thinking skills, specifically synthesis and evaluation, constitute a smaller portion of the analyzed data.

Therefore, the evaluation of new vocabulary items in an in-use course book such as Form 4 in Malaysia will help users have a better understanding of the weaknesses and strengths of the vocabulary presentation of the textbook, inform future revisions of the textbook, and then contribute to the improvement of English language education in the region.

A total of 4 ESL teachers were chosen via purposive sampling technique. They are selected from two secondary schools located in Kemaman, a district in the state of Terengganu, Malaysia. The expert participants were selected based on their availability and accessibility. And the criterion for recruiting teachers in the study were as follows: firstly, they were all ESL teachers with more than two or three years of experience teaching English to secondary school students; secondly, they held a bachelor’s degree or higher in TESL or a related English field, and had experience teaching English to Form 4 students using the textbook; thirdly, they were experienced-teachers in teaching Full Blast Plus 4 textbooks and were actively involved in using the targeted textbook.

The designated textbooks of Form 4, Full Blast Plus 4 published in 2020 by MM Publications, will be analyzed. The Student’s Book is composed of eight modules that integrate the development of the four language skills (reading, listening, speaking, writing) with the presentation and practice of vocabulary and grammatical structures.

The data of the present study were collected through an adapted checklist for vocabulary items. The checklist that we adapted in this study came from Hussin et al. (2016) , who also adapted it from another instrument developed by Nimehchisalem and Mukundan (2015) . The checklist was chosen for its clarity and comprehensibility in evaluating vocabulary in textbooks. Also, a new item (Item 12: Introduction of online applications for learning vocabulary in the textbook and the reason for its inclusion) was added by the researchers. Appendix 1 shows the final version of the checklist that we used for data collection in the current study.

Furthermore, semi-structured interviews, with a high level of validity and reliability ( Goodarzi et al., 2021 ), were used to delve into participants’ beliefs, perspectives, and opinions regarding the presentation of vocabulary in the textbook. The four questions (Appendix 2) were based on a questionnaire developed by Ngu & Aziz (2019) , which are about the respondents’ perspectives on professional development, curriculum, and teaching and learning in the CEFR-aligned curriculum. The interviews were recorded and transcribed. The themes that emerged from the interviews were then analyzed to answer the research questions.

Data collection procedures were carried out in about one month (May 2023). Two secondary schools (as mentioned above) were purposefully choosing where the Full Blast Plus 4 textbook was the main course book. The adapted checklist was distributed among the 4 English teachers in PDF form on WhatsApp. Before they filled out the checklist, the researcher explained the general terms and asked for any comments on any items that may be presented in the Comments Section. The evaluators had plenty of time to respond to each question separately. Then, evaluators participated in the interviews via WhatsApp after the checklist was completed. The interview section was downloaded and transcribed.

In the first stage, the current checklist was scored on the 5-point Likert scale, ranging from “totally lacking”, “poor”, “satisfactory”, “good”, and “excellent”. In the second stage, the data collected from the semi-structured interviews with the expert participants was analyzed using thematic analysis. This method helped to identify and categorize patterns, themes, and ideas that emerged from the participants’ comments.

Table 1 presents the descriptive data for the demographic features of the evaluators (n = 4), aged between 29 and 45. All of the evaluators taught at a secondary school in Kemaman, Terengganu. They hold a bachelor’s degree in TESL. The evaluators’ teaching experience ranged from 3 to 23 years, with an average of around 11 years.

Having collected the required data, the researcher reported the data of evaluators’ perception of the new vocabulary items in Form 4 through descriptive analysis. Table 2 shows the teachers’ evaluation of each item.

As Table 2 shows, the mean score for the vocabulary load was 3.8 (items 1 and 3), which means that the vocabulary load in the Form 4 English Language textbook was at a proper linguistic level and had good distribution. Also, evaluators rated the textbook’s easy and difficult terms as remarkably balanced, and the wordlist/index of the vocabulary is clearly distributed (M = 3.5). Meanwhile, the mean of visuals and introduction of online vocabulary learning applications ranked the lowest (M = 2).

Key: 0.00 - 0.80 (Negligible usefulness), 0.81 - 1.60 (Low usefulness), 1.61 - 2.80 (Moderate usefulness), 2.81 - 3.60 (High usefulness), and 3.61 - 4.00 (Very high usefulness).

To be specific, for item 1, the majority of evaluators, with an average score of 3.8, found the load was appropriate for the students. All but one evaluator rated item 1 with a score of 4. The mean score of 3.5 for item 2 indicated that the evaluators perceived a balanced mix of easy and difficult terms in the textbook. Mardhiah and Fatin found an adequate balance, while evaluators 3 and 4 found the mix to be significantly satisfactory. Dakhi & Fitria (2019) emphasized this balance in ESL textbooks. Regarding item 3, Mardhiah, Fatin, and Fizah felt that the word load was adequately distributed across each chapter. Meanwhile, Huda ranked 3 for the distribution of vocabulary load across the whole book. They appeared satisfied with the placement of new vocabulary items because they believed students would learn and utilize the terms well in each chapter. The vocabulary load in each chapter is appropriate for the students’ level with a mean of 3.8. Also, Mardhiah and Fizah are of the opinion that the repetition of new words for delayed recall across the book (item 4) is decent. However, Fatin and Huda believed that the repetition of new words for delayed recall across the book is fine and of the highest standard. With an average score of 2.8, evaluators found moderate usefulness in contextualizing new words. While Mardhiah ranked it as 1, most evaluators agreed (ranked 3 or 4) that contextualization was adequate, supporting Rapaport (2004) and Ismail et al. (2017) . The average score of 2 on item 6 suggests the textbook’s moderate effectiveness in using visuals. Mardhiah and Huda ranked it as poor and good, respectively. This aligns with Ismail et al.’s (2017) emphasis on visuals in ESL textbooks. The topical nature of the exercises is being ranked as up to standard by the Mardhiah, Fatin, and Fizah. Three of the evaluators were satisfied whereas Fizah was very satisfied with the topical nature of the vocabulary exercises, which allowed for useful enrichment of vocabulary items ( Nation, 2001 ). The mean score for this item is 3.3. For item 8, Fizah ranked 4 and really agreed that the new lexical items (item 8) appear in each unit. The other evaluators agreed and moderately agreed; good and satisfactory. Huda and Fizah ranked item 9 (There is a specific method to teach new vocabulary) 4 and in the same opinion that there is a specific method to teach new vocabulary in the textbook; Mardhiah and Fatin ranked this criterion 2. The evaluators all concluded that the textbook satisfactorily defined the approach for teaching new terms. Moreover, Mardhiah, Fatin, and Huda rated the sentences and examples defining new vocabulary as good, while Fizah ranked it as excellent. This finding aligns with Hussin et al.’s (2016) study, which deliberated that sentences and examples that define new vocabulary presented in ESL textbooks are familiar to the learners. Regarding item 11, both Mardhiah and Fizah highly agreed that there is an index of new vocabulary at the end of the textbook. Meanwhile, Fatin and Huda ranked 3 for item 11. This result was consistent with Hussin et al. (2016) results that the index in ESL textbooks is essential to identifying new words used in each unit for both teachers and learners. What’s more, all 4 evaluators moderately believed the textbook introduces online applications for learning vocabulary, all ranking it as 2 for the last item.

Key: 0.00 - 0.80 (Negligible usefulness), 0.81 - 1.60 (Low usefulness), 1.61 - 2.80 (Moderate usefulness), 2.81 - 3.60 (High usefulness), and 3.61 - 4.00 (Very high usefulness).

Table 3 illustrates that Mardhiah found the new vocabulary items were presented in the textbook in a moderately helpful manner, which could be beneficial for students’ learning. The evaluator’s ratings ranged from 2 (low) to 4 (high), generally commending the inclusion of new vocabulary in the textbook.

Meanwhile, the other three evaluators regarded the presentation of new vocabulary items in the textbook as highly beneficial for students’ learning, assigning scores of 3 (high) and 4 (very high). They valued the approach to presenting new vocabulary in the textbook.

The vocabulary was particularly relevant to evaluators 2, 3, and 4, who possessed higher teaching experience, and they consistently rated the textbook as useful. In contrast, Mardhiah, with moderate teaching experience, found the textbook’s presentation of new vocabulary moderately valuable for students’ acquisition (M = 2.58). This evaluator likely rated the criteria as 2 (moderate) or 3 (highly useful). Overall, experienced teachers expressed high satisfaction with the textbook’s introduction of new terminology, perceiving it as easily comprehensible for students. Conversely, less experienced teachers (such as Mardhiah and Fatin) found the inclusion of new words slightly challenging for students, leading to struggles in teaching new vocabulary effectively.

Thus, the most notable aspect of the data presented in the table is the average item score of 36, indicating a mean score of 3, suggesting that evaluators considered the vocabulary highly useful (M = 3).

This data indicates that the textbook effectively introduces new vocabulary items. Furthermore, a correlation is observed between evaluators’ teaching experience and mean scores. Higher teaching experience correlates with higher mean scores. For instance, Fizah, with 23 years of experience, rated the highest scores among all evaluators, while Mardhiah, with 3 years of experience, rated the lowest scores. Additionally, English teaching experience influenced the research findings, suggesting that experienced evaluators are more attuned to student needs. Hence, collaboration among teachers of varying experience levels is crucial for enhancing vocabulary learning.

Follow-up interviews were conducted after the participants finished the checklist. The analysis of the interview led to a thematic analysis regarding the expert participants’ views on the presentation of vocabulary items in the textbook.

When asked about potential enhancements to vocabulary presentation in the textbook, the four evaluators offered distinct suggestions. Fatin agreed that the current presentation of vocabulary is good with a word index to help guide teachers on which words to cover. Visual connections with vocabulary presentation were proposed by Huda and Fizah. Huda proposed linking content to audio or video resources as an essential improvement strategy. Fizah thought matching and filling in the blanks were good activities for doing vocabulary. And more graphics or pictures can be used to enhance students’ understanding. Mardhiah further elaborated:

“I think the vocabulary can be improved if the root words are included as well since words with prefixes or suffixes may change the nature of the words. For example, ‘damage’ is a verb in the present tense but ‘damaged’ can both be a verb in the past tense or an adjective. There are some parts where it includes some notes for the root words, but sometimes the explanation is quite confusing for low-proficiency pupils. It’s also much easier to learn vocabulary when pupils can relate it with real-life context. This can be done by asking them to practice the vocabulary by re-enacting some situations in their daily life, or at least by showing them video clips or pictures.”

Also, the participants were asked to introduce the teaching methods they use or create to improve students’ vocabulary knowledge based on the lessons in the textbook. Various strategies are employed for this purpose. Primarily, reading, agreed by the 4 evaluators, serves as a key method to reinforce vocabulary understanding. Faizah also advocates for reading with context or contextual clues. Fatin highlights the importance of understanding contextual meanings to grasp word usage effectively. Additionally, Mardhiah organizes reading sessions where students reenact significant parts of texts in small groups. Vocabulary errors prompt correction, repetition, and dictionary use, which is fundamental in second language acquisition. She elaborated:

“I usually ask the pupils to re-enact the important part of the article/story if we’re having reading lessons. I’ll put them in a few small groups and each group needs to re-enact the scenes based on their own understanding first. If they make mistakes in explaining the vocabulary, I will correct them and ask them to do it again. But most of the time, I will ask my pupils to consider me their last resort. This means that they have to use their dictionary to find the meaning of the words since it’s a part of the learning tool necessary for learning a second language.”

Furthermore, the participants were asked about their overall perception of using the CEFR-aligned textbook. Most respondents agreed that these textbooks are convenient and significantly helpful resources, echoing Huda’s sentiment. Fizah also noted their usefulness as comprehensive reference materials. Also, Fatin proposed that the CEFR-aligned textbook is really helpful to pupils, especially in terms of how easy it is to use as well as the topic covered. In addition, the CEFR-aligned textbook explicitly covers all skills as well as the suitable vocabulary used for each skill.

However, Mardhiah slightly disagreed as the teacher poses challenges due to the lack of resources and exposure for some students in suburban areas, resulting in students being unfamiliar with certain textbook themes like globetrotting. She added:

“Since I’m teaching at a school in a rural area, I would say that the CEFR-aligned textbook is quite challenging for my pupils. As compared to pupils in the city with plenty of resources and exposure, some of my pupils here are not familiar with some themes/topics in the textbooks. For example, in chapter 5, they couldn’t relate to the topic of globetrotting, so there were times they got quite bored during the lesson. So as a teacher, I have to play my part in creating interesting and fun lessons.”

Therefore, it is important to create interesting and fun lessons to engage the students according to the EFL/ESL teachers by adding multi-media or role play ( Nation, 2001 ).

Finally, the participants were asked about their perception of whether the current textbook better helps develop students’ English language vocabulary compared to previous textbooks. Most of the teachers, except Mardhiah, who is a novice and unfamiliar with the previous textbooks, expressed a preference for the current textbook, citing its efficacy in vocabulary development. This finding corroborates the results of the evaluation checklist, indicating overall satisfaction among teachers, with more experienced teachers rating the textbook more favorably. Also, according to Huda, the current textbook is more detailed and relatable to the current situations. Fizah added the current textbook is more interesting. The contents chosen are much more related to students’ life and the articles used are all interesting in telling the students new things that they should know. What’s more, Fatin thought the current textbook provides knowledge from other countries’ contexts, which allows students to learn general knowledge and language use. Here are her words: Taking into consideration that English is a widely used language, it is crucial for pupils to learn the vocabulary in different contexts, i.e., outside Malaysia’s context. While the previous textbook focuses on Malaysia’s context, the current textbook gives knowledge from other countries’ contexts after the implementation of the CEFR framework in Malaysia ( Rahim & Daghigh, 2019 ). Pupils can learn general knowledge as well as the use of language itself.