Assessments have been a crucial part of the process of learning and teaching. Most of the time, this process has been measured with summative assessments but what about formative ones? Historically, the teacher chooses the appropriate assessment for his/her class. However, what do the students have to say about that? In this case study, we present the results of a Likert-type poll for five assessments applied in a General Chemistry course. Students gave their answers according to three measurable intended aspects: the cognitive, the emotional and the social. To measure the coherence of those three aspects a similarity index was created. This instrument, together with the Likert-type poll, allowed us to advise which assessment should be accepted or rejected according to the student’s point of view. From a variety of assessments, the student selected the following increasing raking of assessments: Short Multiple-Choice Questions, General Questions/Answers Out Loud and Prediction-Observation-Explanation.
The social media (SM) allow users to evaluate their peers by pressing “like”. Young people are the common users of SM, so they are habituated to be evaluated and evaluating. Even when this is a spread practice among the Z-generation, it seems such a behaviour is not extrapolated to assessments.
A discussion on how the students should be assessed has had a significant impact over the last years (Creme, 2005; Maxwell, 2012; Mayowski et al., 2018; Schneider et al., 2019; Smith, 2007; Stewart & Richardson, 2000). Should it be at the beginning, in the middle or at the end of a unit? It seems that a general conclusion applies; assessment must be a journey, not an objective (Bulwik, 2004; Chamizo, 1995; Viera et al., 2007).
Chamizo (1995, 1996) argues that if the questions that are asked on a test are for specific contents, the students will rapidly forget those contents due to there are no circumstances to overcome. In this sense, students must be challenged to think and reflect on why they are doing what they are doing. Thus, the assessment must be an integral process, not only a goal by itself.
Bulwik (2004) goes beyond Chamizo’s ideas indicating that:
Most of the evaluation methods currently applied for learning are conservative, considering that purposes and teaching methodologies have changed.
According to Bulwik,
[…] to privilege the social character of the evaluation of learning over its pedagogical nature, led to equalise assessment, examination, and qualification.
Moreover, she tops off saying:
[the previous statement led] to consider that assessments are only for rating an exam, turning the exams and tests in moments of tension as much the teacher as the students.
The construction of knowledge is dynamic; the meanings are revisited and updated, recurrently, so the student must do his/her own learning pathway (Price et al., 2017). Thus, if the students take part in the assessment, the commitment to learn is more significant (Bulwik, 2004). A shortlist of formative/summative assessments is presented in Appendix A (view supporting information, from now on S. I.).
Since the assessment is a trip and not a goal, it seems natural questioning: what are the advisable assessments to use? According to literature (Hortigüela et al., 2019; Prashanti & Ramnarayan, 2019; Quesada et al., 2019), this is a critical issue when deep/active/lifelong learning (Davari & Bahraman, 2019; Lewis, 2019; Rubenson, 2019) is the objective needed for reaching.
The main issue with assessments is it is the teacher who makes these instruments. He/she must have the expertise not only in his/her field, but also in the design of assessments. This issue depends on his/her career, so some teachers have the assessments design skills more developed than others. But, in what choices the assessments design is based? (Fischman et al., 2019; Tan, 2019). Is there any chance to involve the students asking for their opinions? We think this is an aspect that we should consider more often.
Furthermore, assessing implies not only a cognitive aspect, but an emotional and, sometimes, a social aspect, as well. When an assessment is placed, the students’ mind tries to find the answer. However, a part of the students’ feelings is also involved. They can react negatively to a particular question if they are not comfortable with the content. Moreover, they can react negatively to a specific type of assessment if they do not like it. As in social media, feelings are involved when teachers assess students.
On the other hand, socio-cognition is another aspect present in assessments, even more, if the students work on groups, and they have peer review. However, the social issue is also present when students are asked if some assessment should be applied to future freshmen. In this scenario, a freshman may think not only in his/her benefits but in the repercussion for their new classmates (next year), as well. In that regard, a parameter that could simultaneously measure the coherence among the cognition, the emotional and the social aspects of the assessment could bring attention to which ones are preferred by the students.
Giving the full range of assessments listed, this study had the purpose to analyse which assessments are preferred the most by the students in a General Chemistry course.
To reach that goal, this manuscript intent to bring some light to the following question: is there any parameter to evaluate the students’ choices with respect the way they are assessed?
To respond that question, a poll was conducted. Most of the surveys only evaluate the emotional part, but in this manuscript, the cognitive and social aspects were also evaluated. For that, the paper presents a parameter called “the similarity index”, to simultaneously measure the cognitive, the emotional, and the social aspects to quantify the students’ preferences. Thus, along this manuscript, a detailed argumentation is presented to answer the proposed question.
The manuscript is structured as follows: after this introduction, a description of the methods is presented, followed by the results and the discussion, ending with some concluding remarks.
This research was made in a group of tertiary students in an anonymous way and presents a one-year case study in a General Chemistry course where, traditionally, the students were no submitted to any formative assessment. The teacher in charge was trained in general formative assessment with no specificity for Chemistry. Thus, it was the mission of the teacher to explore the most suitable assessments for this course. To accomplish that three to five formative assessment were tested in any lesson along the year (see TableB1 and TableB2, Appendix B, S. I.).
To obtain the students’ opinions concerning assessment, they were polled (by a Likert-type poll), after the finishing of every term. Moreover, a sample of them was interviewed. To gather the Likert-type poll information in a “simple” way, a similarity index was created. This index intended to measure the coherence among the cognitive, the emotional and the social aspect based on the three questions given in the poll (see later). Thus, a high correlation (or coherence) of these three variables indicated the assessment that students appreciated the most, and a low correlation indicated the opposite thing.
According to the previous paragraph, and for the sake of clarity, the following section is ordered as follows: first, the type of lessons and the type of students is described, second, a brief review of each used assessment is given, third, the Likert-type poll details are presented together with segments of the interviews, and forth, the similarity index is presented.
Lectures were scheduled, as shown in TableB1 and TableB2 (see Appendix B, S. I.). Usually, a lesson consisted of an opening to the theme, an introductory activity, a development by exposition, and a closing involving some assessment listed in TableB1 and TableB2 (see Appendix B, S. I.). The opening of the lessons had the purpose of running the students’ minds. Nonetheless, not always was related to students’ assessment. For that, the teacher’s exposition delivered the required contents to complete the assessment.
The type of the students is described primarily for General Chemistry 1 (GM1, the first-term course), and secondly for General Chemistry 2 (GM2, the second-term course).
First-term (GM1)
The course consisted of 65 students, but since the attendance to classes is not mandatory at the University of Concepción, so an average of 60% of them attended the classes. The students that responded to the poll (see below) were 38. The course included 46% of men and 54% of women. Ages comprehended between 18 and 20 years old. The students’ origin was from Concepción city and nearby cities. This origin assured a composite sample concerning the socio-ethnic origins, the regime, and the quality of education.
Second-term (GM2)
Eighty-four students signed up in GM2. Thus, 19 students were attending the course for a second time. An average of 66% of the students participated in the classes, meaning 78 students responded to the poll (see below). The course included 42% of men and 58% of women with ages between 18 and 20 years old. The origins of the sample described in the first-term section are also valid here.
The students did not mandatorily answer formative assessments. Because of this, all the plots presented in this study were normalised. TableB1 (see Appendix B, S. I.) gathers a list of activities, lesson-by-lesson. Recalling the list shown in Appendix A (S. I.), the formative assessments used in classes were one-minute paper (OMP), short multiple-choice questions (SMCQ), directed paraphrasing (DP), Predict, Observe and Explain (POE), and general question/answer out loud (GQAOL). As was previously settled, the choice for these assessments was completely exploratory based on the following criteria made by the teacher in charge:
• OMP because it is an anonymous form to obtain students thoughts
• SMCQ because the students are used to work with them since high school
• DP because it is a challenge (the students never used it before)
• POE to test the students’ prediction ability
• GQAOL due to it is a classical and a fast way to receive feedback from the students.
One-Minute Paper
To follow the students’ progress, they answered two questions, in a piece of paper, at the end of some lessons (see TableB1, Appendix B, S. I.). The OMP assessment typically takes one minute (Angelo & Cross, 1993; Harwood, 1996; Kumar et al., 2017). The questions were:
• What did you like/learn most about the class?
• What content did you not understand?
The teacher analysed the answers and collected them into two groups: the ones they liked/learned the most and the ones they did not understand. This assessment allowed the teacher to understand what students “like” the most or learned and reinforce what was not clear (Kumar et al., 2017; Lutterodt, 2017; Stead, 2005). The next lesson the teacher gave feedback to the students helping them to realise their strengths and their weaknesses.
Short Multiple-Choice Questions
(Short) multiple-choice questions (SMCQ) are another form to follow the students’ progress (Bresnock et al., 1989; Butler, 2018; Dodd & Leal, 1988). Three SMCQ (each of them having four choices) were asked every time this evaluation was used (at the end of the lessons, see TableB1, Appendix B, S. I.). The questions tuned with the contents of the corresponding class, and the students received instant feedback through the platform socrative.com (Arriaga et al., 2017; Blackburn, 2015; Frías, Arce, & Flores-Morales, 2016; Guarascio et al., 2017; Kokina & Juras, 2017; Manning et al., 2017).
Directed Paraphrasing
In directed paraphrasing (DP) (Angelo & Cross, 1993), students must explain, in their own words, a complex concept to a regular audience. Due to the students must summarise content in their own words, they prove they have learned it (Cheung, 2016; Moran et al., 2014; Tan, 2017; Uemlianin, 2000).
As an example, for this assessment, after a class of colligative properties (class 36, TableB1, Appendix B, S. I.), students had to demonstrate why a hand cream based on sodium chloride was better for keeping hands warm in cold days than a hand cream based on glycerol. The regular audience was their mothers (not having the chance to talk to them).
Predict, Observe, and Explain
Predict, Observe and Explain (POE) is another formative learning method (Güven, 2014; Hong et al., 2014; Hsu et al., 2011; Kose & Bilen, 2012; White & Gunstone, 1992). In the present case, the students had to predict what was going to happen in a particular experiment, observe the chemical reaction and finally, explain in their own words what they watched.
For example, for the lesson 39 (see TableB2, Appendix B, S. I.), a range of acid-base chemical reactions was made as follows: a solution of sodium hydroxide 0.05 mol∙L−1 was poured into seven beakers with water and drops of different acid-base indicators. The visible spectrum range of colours appears as the water was poured. Finally, all the beakers were poured into a 1 L beaker containing 10 mL of sulfuric acid 2.5 mol∙L−1 for discolouring the solutions. At no time, the students knew what the solutions were.
The students had to follow the POE method: predict what was going to happen when the solutions were mixed, observe the colour appearance, and explain why this happened. Next, the students had to predict what was going to happen when the seven beakers were poured into the 1 L beaker containing 10 mL of sulfuric acid 2.5 mol/L, observe the discolouring and explain why.
General Question/Answer Out Loud
Asking students general questions out loud (GQAOL) is a classical form of formative evaluation (Bannert & Mengelkamp, 2008; Lancaster, 2007; Metcalfe & Xu, 2018; Wardrop, 2012). During almost every lesson this assessment was used.
The poll consisted of students’ perceptions about the activities. Their answers were anonymously, and not mandatory. Because of that, all the plots presented in this study, regarding the number of students, were normalised.
The students were asked three questions on the Likert-type scale modality (Likert, 1932; Matas, 2018; Orozco et al., 2018; Robertson, 2012).
For this poll, the Likert-type scale meaning was: 1, nothing; 2, little; 3, enough; 4, much; 5, very much.
| Question number | Question | Measured aspect |
|---|---|---|
| Q1 | How much do you think these types of evaluations helped you to understand the contents? | Cognitive |
| Q2 | How much would you recommend using the following assessment for General Chemistry freshmen students? | Social |
| Q3 | How much did you like the following evaluations? | Emotional |
A sample of three students (two men and one woman) was interviewed in a semi-structured interview regarding their experiences and opinions of the assessments. Verbal informed consent was obtained prior to the interviews.
The coherence between the cognitive, the emotional and the social aspects presented in the poll, was measured using the similarity index.
The similarity index was obtained as follows:
1) The percentage of students’ responses was plotted as a Cartesian plane plot considering the DK/NA/REF (Don’t Know, No Answer, Refusal) alternative of Likert-type scale as a value of 6 (see the left bottom corner of
2) The Cartesian plane plot of
3) The polygon area was calculated using the method of the Gauss’ determinant (Equation (1)) using a counter clockwise order for the three polygons of the Cartesian plane plot (
Area = 1 2 ( | x 1 x 2 y 1 y 2 | + | x 2 x 3 y 2 y 3 | + ⋯ + | x N x 1 y N y 1 | ) (1)
The colours of the “QX area” (being Q = question, and X = 1, 2 or 3) calculation presented below, match the colours of the Cartesian plane plot of
Q1area = 1 2 ( | 6 5 0 71 | + | 5 4 71 18 | + | 4 3 18 5 | + | 3 2 5 5 | + | 2 1 5 0 | + | 1 6 0 0 | ) = 99.0 length 2 Q2area = 1 2 ( | 6 5 0 79 | + | 5 4 79 11 | + | 4 3 11 8 | + | 3 2 8 0 | + | 2 1 0 3 | + | 1 6 3 0 | ) = 92.0 length 2 Q3area = 1 2 ( | 6 5 0 68 | + | 5 4 68 18 | + | 4 3 18 3 | + | 3 2 3 8 | + | 2 1 8 3 | + | 1 6 3 0 | ) = 99.0 length 2
4) The similarity of the Q1, Q2 and Q3 polygons were obtained by the ratio of their respective areas (Thales’ theorem) as is depicted in
5) The similarity indexes of the other formative assessments for the first and second term are in the S. I. (see TableC1 and TableC2, Appendix C).
As can be noticed, this algorithm is a simple way to measure the coherence among the mentioned aspects. For that, it is proposed as a direct and a “simple” manner to measure the people’s choices regarding polls. For this paper, it was applied to the students’ opinion regarding formative assessment.
Combining the results of the Likert-type scale with the similarity index, the situations that could emerge are three:
1) A high Likert-type scale value and a high similarity index value means that the corresponding assessment is well received by the students and should be maintained.
2) A high Likert-type scale value and a low similarity index value (or vice-versa) means that even though the assessment is well rated by the students, there is no coherence among the cognitive, the emotional and the social aspect. As a result, if it is planned to use the instrument again, it should be reformulated. Otherwise, it should be not used again.
| Q 1 area Q 2 area | Q 1 area Q 3 area | Q 2 area Q 3 area | Average | Standard deviation | Percentage of standard deviation (%) | Similarity index (%) |
|---|---|---|---|---|---|---|
| 1.08 | 1.09 | 1.01 | 1.06 | 0.0414 | 3.91 | 96.1 |
3) A low Likert-type scale value, and a low similarity index value means that the assessment should not be used again unless the meaning, the indications and the purpose were presented clearly by the teacher.
As a reminder, the aim of this paper, as is entitled, is collecting the students’ preferences through a correlation between three aspects:
• The cognitive aspect → the understanding of the contents → Q1
• The social aspect → the usage of formative evaluations for freshmen → Q2
• The emotional aspect → the students’ taste → Q3
The Likert-type scale (Carifio & Perla, 2007; Likert, 1932; Matas, 2018) was used to obtain that information.
This part of the manuscript is devoted to analyse the students’ responses to the poll regarding how they evaluated the assessment. The results for the first term, and then, for the second term are presented.
One-minute paper
For question Q2 (
For question Q3 (
Regarding cognitive (Q1) and emotional (Q3) parts, an extract of the student-3’s interview could clarify the results:
Teacher: Which one of the following assessments was the one you like the most, questions/answers out loud, one-minute paper or directed paraphrasing?
Student-3: One-minute paper might be… Yes… it might be.
Teacher: Why?
Student-3: Because one could… It’s like an instantaneity issue. […] One could realise immediately what was clear and what wasn’t. […] It’s like a… time issue.
Teacher: Okay. So, we could say that you recognise at that moment, your strengths, or weaknesses.
Student-3: Yes.
It seems from the student-3’s words that he likes activities in which he obtains rapid satisfaction since that assures him instantly a posture: right or wrong, which allows him to move forward or stop and think.
The students’ judgement leads to a similarity index of 87.9% for OMP (
Directed paraphrasing
The results for this issue are depicted in Figure2(b). Note that the percentages of responses to Q1, Q2, and Q3 are similar among the four and five Likert-type scale values (not the scenario for Figure2(a) results). Likewise, as a general trend, the alternative “sometimes” increases, and the alternative “always” decreases (see FigureC1, top, S. I.). It is important to note that another fact that influences this change, is the option DK/NA/REF, which becomes 11% of the answers (see FigureC1, top, S. I.).
Even when the sum of four- and five-values are 74%, 66%, and 74% for Q1, Q2, and Q3, respectively (see 4 and 5 entries in FigureC1, top, S. I.), the percentage of the five-value decreases regarding those of OMP. It seems the exercise of “being in someone else’s shoes” is not wholly accepted by the students. A subject submitted to these kinds of tasks must dominate the concepts, and those concepts, sometimes abstracts, must explain them in ordinary words.
| One-minute paper (a) | Directed paraphrasing (b) | General questions/answers out loud (c) |
|---|---|---|
| 87.9 | 96.4 | 96.1 |
aSee Methods and
Nevertheless, a coherence of 96.4% (see Table2 and TableC1, middle, S. I.) among the three aspects was found. It could say that a mixed result arises. Although the similarity index reached the value of 96.4%, the five-value decreases regarding OMP. This counterpoint is noted by the interviews of the student-1 and student-3:
Teacher: Among the three assessment […] the directed paraphrasing was the lowest scored. Why do you think was like that?
Student 1: We thought it wasn’t useful.
Teacher: […] Could you explain a little bit more?
Student-1: For example, for the other assessments, we could realise what was going to happen. I mean, at the end of the assessment, we had the confirmation or the disconfirmation. But for the paraphrasing… It was like “So what…?”
--------
Teacher: What did you think about this assessment?
Student-3: It was fun…
Teacher: Was it hard, was it easy?
Student-3: It was fun, but it was difficult because one is never in that situation.
[…]
Teacher: […] I don’t know if in the following courses, the teachers use paraphrases, but do you think this activity helped you?
Student-3: Gosh… I think to be more helpful; the assessment should be more repetitive along the semester.
Both students seem shocked by this unusual activity, but student-3 advises about the way to reinforce the usefulness of this activity. Thus, in chemistry subjects, directed paraphrasing must be carefully applied.
General questions/answer out loud
In Figure2(c), the results for the GQAOL activity are shown. According to
Delving into this point, this is part of the interview for student-1:
Teacher: Why do you think a large part of the class liked this assessment?
Student-1: I think when we did it, we had an epiphany: “Oh, it was for that!”
Teacher: And at the school, did the teachers ask you “Why do you think this phenomenon is like that?
Student-1: Nope.
Teacher: They don’t ask you, okay. But did you ask?
Student-1: They say, “Let’s work; let’s work!” “Follow the instructions on the guide.”
Teacher: And for you, is it important that the teacher asks you what do you think?
Student 1: I think it works a lot because immediately, one says: “Oh, now I know it!” And if you don’t, it means something is wrong.
Two things arise from this extract:
1) The students are not used to think from a question (because it is not a regular exercise at the school) and 2) questions are a significant issue for them because, through immediate feedback, it gives them a clue if they are right or wrong (Ruiz-Primo, 2011). This idea agrees with Lancaster (Lancaster, 2007) and with the results for the OMP. Thus, asking students during a lesson is a fast way to move their ideas forward.
As was described in the Methods section, during the second term, a group of 23 students were added to the students who had passed the course of GM1. Thus, a total of 84 students enrolled in GM2, but 78 answered the poll.
All the assessment used in the first term were repeated in the second one, and two more were added: POE (Kearney, 2004; Latifah et al., 2019) and short multiple-choice questions (Butler, 2018).
One-minute paper
According to
The similarity index reported in
| One-minute paper (a) | Directed paraphrasing (b) | General questions/answers out loud (c) | POE (d) | Short multiple-choice questions (e) |
|---|---|---|---|---|
| 83.1 | 81.0 | 94.2 | 98.9 | 91.0 |
aSee Methods and
part of the second-term students increased this opinion. This result emphasises feedbacking the students’ doubts during the next lesson.
Directed paraphrasing
In contrast with the DP of the first term, the DP of the second term was not well judged, 96.4% (see
It is necessary to re-evaluate the use of this assessment in future lessons or courses due to, although this instrument is widely used (Bronshteyn & Baladad, 2006; Cheung, 2016; Moran et al., 2014; Ponce et al., 2012; Tan, 2017; Uemlianin, 2000), a General Chemistry course would not be a proper context for its application (see the first term interviews of the student-1 and the student-3).
General questions/answer out loud
In this term, GQAOL were well evaluated by the students as in the first term. According to
Complementing the findings for GQAOL in the first and second term, some thoughts of the student-2 are presented:
Teacher: Why do you think GQAOL helped you?
Student-2: I think one is used to attending lessons, listening to the teacher, taking notes… however, we don’t think: “Am I understanding or not?” I think this is the moment [the class] to understand the concepts because, after that, one goes back to home and one has other subjects, and the doubt vanished… And when one is in the exam…
Teacher: “I should have asked this in class…”
Student-2: ¡Right! The moment has already passed.
From the student’ thoughts, it is possible to infer that for her, having immediate feedback, again, is extremely important (Havnes et al., 2012). Generalising an idea is the way to know if the students are learning. If not, they pay more attention (or study harder). Alternatively, they fail in the exam.
Predict, Observe, Explain
This assessment has a variety of contexts in which can be applied. In this case, POE was referred to a series of live chemical reactions. As can be seen in Figure3(d), most of the answers were for the five-value of Likert-type scale. The sum of the percentages of the four-value plus the five-value (see FigureC2, top, Appendix C, S. I.) gives the following results: 87% (Q1), 92% (Q2) and 83% (Q3). The similarity index reaches a value of 98.9% (see
Concerning the social aspect, an extract of the student-3’s interview adds the following:
Teacher: […] So, let’s move to the last assessment, POE […]. There was one which took your attention, people, the rainbow solutions…
Student 3: It was fun!
Teacher: Do you think so?
[…]
Student 3: It was a kind of fun because we could chat among the guys in the group, like “What’s in there?”
Teacher: Okay.
Student 3: “What’s the acid-base indicator?” Or “what acid is in there? What base is in there?” It was like it led to the conversation.
Teacher: Let’s say it encouraged the conversation with the others by asking why…
Student 3: Yes.
The student-3 is adding an interesting point of view. The chat among classmates is important because it forces them to discuss. For this chemical reaction, “before the magic happens”, the students had to predict what was going to occur, then, observe the colours appearance and finally, write a rational explanation for that phenomenal. The reasonable answer is not a “piece of cake” for the students. They must imagine the sub-microscopical world to explain the macro-phenomenon (change of colour). When a challenge of these characteristics is presented, it puts them in the place of a researcher (Nunez-Oviedo & Clement, 2019). In the end, this is what chemistry is all about, so this might be the reason why POE applied to experiments is highly well evaluated and should be maintained during the course.
Short multiple-choice questions
SMCQ were used at the end of eight lessons (see TableB2, Appendix B, S. I.). The results for the three questions appear in Figure3(e). As it can be seen, the dispersion of the Likert-type scale is more significant in comparison to the other activities. The similarity index reaches a value of 91.0% (see Table4). The sum of the four- and five-values for Q1, Q2, and Q3 is 72%, 64%, and 58%, respectively (see FigureC2, top, S. I.). This sum represents the lowest numbers of the four previous assessment and is due to the decrease in the five-value and the increase in the three-value (see FigureC2, top, S. I.). In any aspect, SMCQ was the fewer students’ favourite. An explanation for this result might be the occasional feedback that was given to the students. The platform Socrative.com was used for this purpose, but since the students not always were able to connect it (for different reasons), they did not receive instant feedback, and they felt disappointed. This fact is in tune with the results for GQAOL.
A correction for this assessment might be the replacement of virtual questions for questions in a piece of paper and the immediate feedback on the whiteboard, for example. Another improvement could be the application of the SMCQ not only at the end but at the beginning of the class, moving the philosophy from a simple test to a pre- and post-test. Pre- and post-test allow comparing knowledge between what the students partially knew or did not know, and what they finally learned.
A similarity index was created for measuring the coherence between three aspects, cognitive, social, and emotional, in a survey to evaluate assessments. This index gives a direct result of what intends to measure and, it might be extended to evaluate any kind of polls that uses Liker-type scale. Nevertheless, an improvement in which the starting points are those from the spider chart plot, is being carried out for future studies. Moreover, the inclusion of additional questions in the Likert-type polls covering other aspects or applying the index also by student and not only by group is advisable, and we are working on it.
This index evidenced that GQAOL, SMCQ and POE, have the highest coherence according to the students. The students believe that these assessments cognitive, social, and emotional speaking, are necessary. GQAOL is the assessment with the highest similarity index because students obtain instant feedback. Thus, this assessment must be maintained within the lessons.
SMCQ had a fractious reception according to the similarity index, although this fact might be due to a malfunction or the impossibility for accessing to the socrative.com platform. As a result, this assessment should either be eliminated from the lessons or be restructured.
POE was the assessment which obtained the top score in the Likert-type scale and the similarity index. Moreover, POE promotes socio-cognition among students. Thus, this assessment must be maintained within the lessons.
OMP was evaluated relatively well among the students. The reason for that is because this assessment takes into accounts their opinions, but anonymously. Nevertheless, OMP, according to students’ opinion, loses its purpose if they do not receive instantaneous feedback.
DP is the lowest-ranked assessment. Besides the small values in the Likert-type scale, the similarity index provides the maximum dispersion. The students do not feel comfortable with this assessment, and they do not understand how should be useful. A re-evaluation of its use must be done for other courses.
In general, the students’ opinions led, for this General Chemistry course, to the following increasing raking of favourite assessments: SMCQ, GQAOL and POE. Reaffirming the words of Chamizo (1995, 1996) and Bulwik (2004) the assessment must be a process more than a goal, where to measure the “students’ pulse” along the semester (or along the year) is possible, and not only when the test or the exam arrived. The students appreciate this matter.
Since this is a case of study, it needs validation with a broad quantity of courses, an issue that is in progress.
P.F.M. wants to thank M. Cecilia Nunez-Oviedo for her helpful comments and to the students who participated in this study. They know who they are. Thank you, guys! #RenunciaPiñera #NuevaConstitucion #ChileDespertó #BoricPresidente.
Supporting information includes four appendixes in one document:
Appendix A. List of Formative/Summative assessments.
Appendix B. List of content, activity and assessment carried out for every lesson.
Appendix C. Calculation of the similarity index.
Appendix D. References.
The author declares no conflicts of interest regarding the publication of this paper.
Flores-Morales, P. (2022). The Similarity Index: A New and Simple Algorithm to Measure Coherence in Polls. Creative Education, 13, 203-237. https://doi.org/10.4236/ce.2022.131014
• Chain notes (Keeley, 2015)
• Checklist (Baldwin & Ching, 2019; Caruso et al., 2019; Dreimuller et al., 2019; Robles, 2019)
• Directed paraphrasing (Angelo & Cross, 1993; Keeley, 2015)
• Double-entry journal (Allen, 2008; Angelo & Cross, 1993; Berthoff, 1982)
• Group and Self-assessment (Bartels & Kulgemeyer, 2019; Herrington & Sweeder, 2018; Sridharan & Boud, 2019; To & Panadero, 2019)
• In basket (del Pozo, 2012; Frederiksen et al., 1957; Schippmann et al., 1990)
• KPSI (Knowledge and Prior Study Inventory) (Quintanilla et al., 2008; Tamir & Lunetta, 1978)
• K-W-L (Know-Want-Learned) chart (Ogle, 1986)
• Learning registers (Chamizo, 1996)
• Mental/conceptual maps (Chamizo, 1995; Pendley et al., 1994)
• One-minute paper (Angelo & Cross, 1993)
• One-sentence summary (Berthoff, 1982)
• One-world journal (Angelo, 1991)
• POE (Prediction-Observation-Explanation) (Jasdilla et al., 2019; Kearney, 2004; Latifah et al., 2019; Treagust & Chong, 1995)
• Portfolios (Fosado et al., 2018; Lyons, 1999)
• Question/answer out loud (Bannert & Mengelkamp, 2008; Lancaster, 2007; Metcalfe & Xu, 2018; Wardrop, 2012)
• RSQC2 (Recall, Summarize, Question, Connect, Comment) (Angelo & Cross, 1993; Cowan & George, 2004)
• Rubrics (Angra & Gardner, 2018; Cheng & Chan, 2019; Cockett & Jackson, 2018; Tobajas et al., 2019)
• (Short) multiple-choice questions (Bresnock et al., 1989; Butler, 2018; da Silva, 2019; Dodd & Leal, 1988; Witchel et al., 2018)
• Venn diagrams (Hatzikiriakou & Metallidou, 2009; Kerr & Macosko, 2011; Wygoda & Tague, 1995)
• Words association (Sutton, 1980; Zakaluk et al., 1986)
| Class | Contenta | Activityb | Assessment |
|---|---|---|---|
| 1 | Presentation of the course | - | - |
| 2 | Fundamental concepts about matter and measurements. Classification, chemical and physical properties, and states of matter | Students, in groups, analyse substances of everyday life, smelling, tasting, and touching them | Groups discuss and point out what type of substance they have? An element, a molecule, an ionic compound, or a mixture? |
| 3 | Quantitative measurements and the metric system of unities. Scientific notation, significant figures. Dimensional analysis | Experiment: “Plastic cube with sand and water” | General question/answer out loud |
| 4 | Structure of matter and inorganic nomenclature. Early atomic models, atomic theory, and atom structure. Atomic number, atomic weight, isotopes. Periodic Table | Exposition | One-minute paper |
| 5 | Seminar 1. Lessons 2 to 4. | Exercises | |
| 6 | Ions, molecules. Empiric and molecular formulae. Nomenclature of ionic compounds, molecules, acids and bases | Exposition | One-minute paper |
| 7 | Atomic structure. The electromagnetic spectrum, black body radiation, photoelectrical effect, Planck’s quantum theory | Video: “Atomic models” | Short multiple-choice questions in socrative.com |
| 8 | Seminar 2. Lessons 6 and 7. | Exercises | |
| 9 | Line spectra of atoms. Rydberg’s equation. Bohr’s atomic model and its limitations | Exposition | Short multiple-choice questions in socrative.com |
| 10 | Atomic quantum model: de Broglie’s postulate, Heisenberg’s uncertainty principle and Schrödinger’s equation. Orbitals and quantic numbers | Analogy: “A hummingbird in the forest” for the atomic quantum model | Short questions |
| 11 | Test 1. Lessons 2 to 8 | ||
| 12 | Atomic structure. Electronic configurations of metals and ions. Valence electrons. Diamagnetism and paramagnetism | Exposition | Short multiple-choice questions in socrative.com |
| 13 | Periodic trends. Periods, groups, and Periodic Law (periodic table). Metals, non-metals, semi-metals. Representative elements, noble gases, and transition metals | Exposition | - |
| 14 | Seminar 3. Lessons 9 to 12 | Exercises | |
| 15 | Periodic table. Effective nuclear charge, atomic radii, ionic radii, ionization energy, electron affinity | Exposition | Short multiple-choice questions in socrative.com |
| 16 | Chemical properties of some chemical groups. Chemical bond. Lewis symbols, octet rule. Ionic bond, covalent bond, resonance hybrids | Exposition | - |
|---|---|---|---|
| 17 | Seminar 4. Lessons 15 and 16 | Exercises | |
| 18 | Dipole moments. Prediction of molecular geometries. Hybridisation. Molecular orbitals | Application: “Molecular shape” from Phetc | Students assign the molecular geometry to a set of molecules |
| 19 | Intermolecular forces. Dipole-dipole attraction, ion-dipole attraction, van del Waal forces, London forces, hydrogen bond | Searching on the internet the principal intermolecular forces | Short multiple-choice questions in socrative.com |
| 20 | Seminar 5. Lessons 18 and 19 | Exercises | |
| 21 | Gases. Properties of the gases. Boyle’s and Charles’s Law, Avogadro’s law. Ideal-Gas equation | Students interact with a balloon | Students answer a multiple-choice questionnaire through the socrative.com |
| 22 | Dalton’s law. Kinetic theory, Graham’s law of effusion, van der Waals equation | Exposition | - |
| 23 | Test 4. Lessons 12 to15 and 17 to 20 | ||
| 24 | Properties of liquids. Structure and properties of water. Solids, crystal structures | Exposition | One-minute paper |
| 25 | X-ray diffraction. Heating curves, phase diagrams | Exposition | - |
| 26 | Seminar 6. Lessons 21, 22, 24 and 25 | Exercises | |
| 27 | Stoichiometry. The mol. Determination of the empirical and molecular formula | Using coins to represent atoms and molecules | Short multiple-choice questions in socrative.com |
| 28 | Reactions and chemical equations. Types of chemical reactions | Exposition | - |
| 29 | Seminar 7. Lessons 27 y 28 | Exercises | |
| 30 | Stoichiometric calculations, | Video: Chemical reaction | One-minute paper |
| 31 | Limiting reactant, percentage yield | Exposition | - |
| 32 | Seminar 8. Lessons 30 and 31 | Exercises | |
| 33 | Solutions. Definition, solubility, concentration expressions | Exposition | One-minute paper |
| 34 | Exercises based on concentration expressions | Exposition | - |
| 35 | Test 5. Lessons 24 to 32 | ||
| 36 | Types of solutes (non-electrolyte, strong and weak electrolytes). Colligative properties | Experiment: “Electrical conductance” | Directed paraphrasing |
| 37 | Henry’s law. Exercises based on colligative properties | Exposition | - |
| Seminar 9. Lessons 37, 39 and 40 | Exercises | ||
| 39 | Reactions in aqueous solution. Acid-base reactions; neutralisation and titration | Experiment: “Acid-base and precipitation reactions” | POE assessment |
|---|---|---|---|
| 40 | Oxidation-reduction reactions, Oxidation-reduction titrations | Experiment: “oxidation-reduction reaction” | One-minute paper |
| 41 | Seminar 10. Lessons 37, 39 and 40 | Exercises | |
| 42 | Thermochemistry. Basic definitions. Energy, work and heat. Law zero | Demonstration: an experiment of an exothermic reaction | One-minute paper |
| 43 | First law. Enthalpy, enthalpy changes | Exposition | One-minute paper |
| 44 | Seminar 11. Lessons 41 and 42 | Exercises | |
| 45-46 | Thermochemistry. Calorimetry. Enthalpy of formation, enthalpy of reaction, the enthalpy change of combustion and enthalpy change of solution. Hess’s Law | Exposition | One-minute paper |
| 47 | Test 6. Lessons 33 to 42 | ||
aBased on the course syllabus. bAll the lessons included a lecture. Thus, when no activity has been indicated, the evaluation was based on the lecture. cPhet, interactive simulations from the University of Colorado, https://phet.colorado.edu/es/simulations/category/chemistry.
| Class | Contenta | Activityb | Assessment |
|---|---|---|---|
| 1 | Presentation of the course | - | - |
| 2 | Chemical thermodynamics. Spontaneous processes. Entropy and the second law of thermodynamics. Gibbs’ free energy. Free energy and chemical equilibrium | Exposition | - |
| 3 | Chemical kinetics. Rate of a reaction and stoichiometry. Reaction rate law, reaction order and its experimental obtention. The relation between reaction concentration and time: first-order reactions and its half-lives | The teacherintroduces the concept of the reaction rate with a pile of sheets and a stapler | General question/answer out loud |
| 4 | First-order reactions and its half-lives. Second-order reactions and its half-lives. Effects that influence a reaction-rate. Collision theory and transition state. Effect of the concentration in collisions. | Experiment: “Effervescent tablet in the water” | One-minute paper |
| 5 | Activation energy. Arrhenius’ equation. Chemical mechanisms and molecularity. Catalysis | Experiment: decomposition of hydrogen peroxide | One-minute paper |
| 6 | Seminar 1. Lessons 2 to 5 | Exercises | Students answer the exercises |
| 7 | Chemical Equilibria. The concept of equilibrium. The chemical equilibrium constant (KC). The relation between KC and KP. Multiple equilibriums | Analogy: exchange of students from one side of the classroom to the other one. | Students answer a poll |
| 8 | Test 1. Lessons 2 to 6. | ||
|---|---|---|---|
| 9 | Homogeneous and heterogeneous equilibriums. Calculations of the concentrations at equilibrium | Exposition | General question/answer out loud |
| 10 | The factors that affect chemical equilibrium. Le Chatelier’s principle. The response of equilibria to changes in conditions | Experiment: traffic light of solutions | Students answer a poll |
| 11 | Acid-Base Equilibria. Definition of acid and base. Arrhenius acids and bases. Brfnsted-Lowry acids and bases. Lewis acids and bases. The acid-base properties of water. The pH scale | Experiment: “Rainbow solutions” | POE assessment |
| 12 | Strength of acid and bases. Solutions of strong acids and bases. Solutions of weak acids and bases. | Exposition | One-minute paper |
| 13 | Buffer solutions | Exposition | One-minute paper |
| 14 | Seminar 3. Lessons 7, 9 to 12 | Exercises | Students answer he exercises |
| 15 | Salts and hydrolysis | Exposition | Short multiple-choice questions in socrative.com |
| 16 | Test 2. Lessons 7, 9 to 14 | ||
| 17 | Acid-base indicators. Strong acid-strong base titrations. Strong acid-weak base and weak acid-strong base titrations | Exposition | Short multiple-choice questions in socrative.com |
| 18 | Solubility Equilibria. Importance. Solubility and molar solubility. The solubility product (KP) | Mental experiment: Solubility equilibria in the eggshells | Directed paraphrasis |
| 19 | The common-ion effect. Predicting precipitation. Selective precipitation | Exposition | Short multiple-choice questions in socrative.com |
| 20 | Seminar 3. Lessons 15, 17 to 19 | Exercises | |
| 21 | Sulphurs precipitation. Factors that affect solubility equilibria | - | - |
| 22 | Equilibrium of complex ion formation. History overview. Chemical structure of the metal complex. Basic nomenclature of the complex ions. Complex ion equilibria | Experiment: Silver complex compound formation | One-minute paper |
| 23 | Seminar 4. Lessons 21 and 22 | Exercises | |
| 24 | Electrochemistry. Definition. Representation of redox reactions. Galvanic cells. Standard potentials | Link for the previous study: http://cienciadesofa.com/2013/07/respuestas-xix-anguilas-electricas.html | Short multiple-choice questions in socrative.com |
| 25 | Test 3. Lessons 15, 17 to 23 | ||
| 26 | Cell potential and reaction Gibbs free energy | - | - |
| 27 | The Nernst Equation. Concentration cells. Batteries | Exposition | One-minute paper |
|---|---|---|---|
| 28 | Seminar 5. Lessons 26 and 27 | Exercises | |
| 29 | Corrosion. Electrolytic cells | Experiment: Water electrolysis | Short multiple-choice questions in socrative.com |
| 30 | Test 4. Lessons 24, 26 to 28. | ||
| 31 | Applications of Electrolysis (metallurgic of some metals) | Experiment: Galvanized of a spoon | One-minute paper |
| 32 | Organic compounds.General characteristics of organic molecules. Hydrocarbons: alkanes | Videos: watching three ads involving organic substances | Short multiple-choice questions in socrative.com |
| 33 | Seminar 6. Lessons 29, 31 and 32 | Exercises | |
| 34 | Cycloalkanes. Reactions of alkanes. Alkenes and alkynes. Aromatic compounds | - | - |
| 35 | Reactions of electrophilic addition, electrophilic substitution, and electrophilic elimination | Exposition | One-minute paper |
| 36 | Alcohols, ethers, aldehydes, and ketones | - | - |
| 37 | Carboxylic acids, esters, amines, and amides | Experiment: “stink + travertine = perfume” | Short multiple-choice questions in socrative.com |
| 38 | Seminar 7. Lessons 34 to 37 | Exercises | |
| 39 | Test 5. Lessons 31 to 36 | ||
| 40 | Aromatic compounds, quirality | - | - |
| 41 | Synthetic and natural polymers | Conference of a faculty member | General question/answer out loud |
| 42 | Nucleic acids | Conference of a faculty member | General question/answer out loud |
| 43 | Amino acids and proteins | Conference of a faculty member | General question/answer out loud |
| 44 | Seminar 8. Lessons 45 and 46 | Exercises | |
| 45 | Nuclear chemistry. Radioactivity | Links for the previous study: http://www.dailymotion.com/video/x6t846 https://vimeo.com/75872397 http://www.rtve.es/alacarta/videos/centro-medico/centro-medico-09-11-15/3354227/ | Short multiple-choice questions in socrative.com |
| 46 | Nuclear decay | - | - |
|---|---|---|---|
| 47 | Seminar 10. Lessons 40 to 43 | Exercises | |
| 48 | Test 6. Lessons 40 to 46 | ||
| 49 | Fission and fusion. The impact on biology | - | - |
aBased on the course syllabus. bAll the lessons included a lecture. Thus, when no activity has been indicated, the evaluation was based on the lecture.
The similarity index was calculated, as was described in Methods. Here, a summary of the calculations for all the formative evaluations (first and second term) is presented.
First-term
| Q 1 area Q 2 area | Q 1 area Q 3 area | Q 2 area Q 3 area | Average | Standard deviation | Percentage of standard deviation | Similarity index (%) |
|---|---|---|---|---|---|---|
| 1.25 | 1.26 | 1.01 | 1.17 | 0.142 | 12.1 | 87.9 |
| Q 2 area Q 1 area | Q 3 area Q 1 area | Q 3 area Q 2 area | Average | Standard deviation | Percentage of standard deviation | Similarity index (%) |
|---|---|---|---|---|---|---|
| 1.07 | 1.09 | 1.01 | 1.06 | 0.0384 | 3.63 | 96.4 |
| Q 1 area Q 2 area | Q 1 area Q 3 area | Q 2 area Q 3 area | Average | Standard deviation | Percentage of standard deviation | Similarity index (%) |
|---|---|---|---|---|---|---|
| 1.08 | 1.09 | 1.01 | 1.06 | 0.0414 | 3.91 | 96.1 |
Second-term
| Q 2 area Q 1 area | Q 1 area Q 3 area | Q 2 area Q 3 area | Average | Standard deviation | Percentage of standard deviation | Similarity index (%) |
|---|---|---|---|---|---|---|
| 1.02 | 1.38 | 1.41 | 1.27 | 0.214 | 16.9 | 83.1 |
| Q 1 area Q 2 area | Q 1 area Q 3 area | Q 2 area Q 3 area | Average | Standard deviation | Percentage of standard deviation | Similarity index (%) |
|---|---|---|---|---|---|---|
| 1.45 | 1.83 | 1.26 | 1.51 | 0.288 | 19.0 | 81.0 |
| Q 2 area Q 1 area | Q 3 area Q 1 area | Q 2 area Q 3 area | Average | Standard deviation | Percentage of standard deviation | Similarity index (%) |
|---|---|---|---|---|---|---|
| 1.13 | 1.01 | 1.11 | 1.08 | 0.0634 | 5.85 | 94.2 |
| Q 1 area Q 2 area | Q 1 area Q 3 area | Q 2 area Q 3 area | Average | Standard deviation | Percentage of standard deviation | Similarity index (%) |
|---|---|---|---|---|---|---|
| 1.01 | 1.03 | 1.02 | 1.02 | 0.0107 | 1.05 | 98.9 |
| Q 1 area Q 2 area | Q 1 area Q 3 area | Q 2 area Q 3 area | Average | Standard deviation | Percentage of standard deviation | Similarity index (%) |
|---|---|---|---|---|---|---|
| 1.00 | 0.86 | 0.86 | 0.91 | 0.0817 | 9.03 | 91.0 |
Allen, J. (2008). More Tools for Teaching Content Literacy. Stenhouse Publishers.
Angelo, T. (1991). Classroom Research: Early Lessons from Success. New Directions for Teaching and Learning. Jossey-Bass. https://doi.org/10.1002/tl.37219914603
Angelo, T., & Cross, P. (1993). Classroom Assessment Techniques. A Handbook for College Teachers. Jossey-Bass.
Angra, A., & Gardner, S. M. (2018). The Graph Rubric: Development of a Teaching, Learning, and Research Tool. CBE-Life Sciences Education, 17, 1-18. https://doi.org/10.1187/cbe.18-01-0007
Baldwin, S. J., & Ching, Y. H. (2019). An Online Course Design Checklist: Development and Users’ Perceptions. Journal of Computing in Higher Education, 31, 156-172. https://doi.org/10.1007/s12528-018-9199-8
Bannert, M., & Mengelkamp, C. (2008). Assessment of Metacognitive Skills by Means of Instruction to Think Aloud and Reflect When Prompted. Does the Verbalisation Method Affect Learning? Metacognition and Learning, 3, 39-58. https://doi.org/10.1007/s11409-007-9009-6
Bartels, H., & Kulgemeyer, C. (2019). Explaining Physics: An Online Test for Self-Assessment and Instructor Training. European Journal of Physics, 40, 1-11. https://doi.org/10.1088/1361-6404/aaeb5e
Berthoff, A. (1982). Forming, Thinking, Writing: The Composing Imagination. Boyton/Cook.
Bresnock, A. E., Graves, P. E., & White, N. (1989). Multiple-Choice Testing—Question and Response Position. Journal of Economic Education, 20, 239-245. https://doi.org/10.1080/00220485.1989.10844626
Butler, A. C. (2018). Multiple-Choice Testing in Education: Are the Best Practices for Assessment Also Good for Learning? Journal of Applied Research in Memory and Cognition, 7, 323-331. https://doi.org/10.1016/j.jarmac.2018.07.002
Caruso, A. E., Hobart, T. R., Botash, A. S., & Germain, L. J. (2019). Can a Checklist Ameliorate Implicit Bias in Medical Education? Medical Education, 53, 1. https://doi.org/10.1111/medu.13840
Chamizo, J. A. (1995). Mapas conceptuales en la enseñanza y la evaluación de la química. Educación Química, 6, 118-124. https://doi.org/10.22201/fq.18708404e.1995.2.66719
Chamizo, J. A. (1996). Evaluación de los aprendizajes en química. Segunda parte: Registros de aprendizaje, asociación de palabras y portafolios. Educación Química, 7, 86-89. https://doi.org/10.22201/fq.18708404e.1996.2.66671
Cheng, M. W. T., & Chan, C. K. Y. (2019). An Experimental Test: Using Rubrics for Reflective Writing to Develop Reflection. Studies in Educational Evaluation, 61, 176-182. https://doi.org/10.1016/j.stueduc.2019.04.001
Cockett, A., & Jackson, C. (2018). The Use of Assessment Rubrics to Enhance Feedback in Higher Education: An Integrative Literature Review. Nurse Education Today, 69, 8-11. https://doi.org/10.1016/j.nedt.2018.06.022
Cowan, J., & George, J. (2004). A Handbook of Techniques for Formative Evaluation: Mapping the Student’s Learning Experience. Routledge Falmer.
da Silva, A. C. (2019). A Conceptual Questionnaire on Radiations: Formulation Process and Analysis of Distractors. Gondola Enseñanza y Aprendizaje de Las Ciencias, 14, 63-79. https://doi.org/10.14483/23464712.13113
del Pozo, J. A. (2012). Competencias profeionales. Herramientas de evaluación: El portafolios, la rúbrica y las pruebas situacionales. Narcea.
Dodd, D. K., & Leal, L. (1988). Answer Justification—Removing the Trick from Multiple-Choice Questions. Teaching of Psychology, 15, 37-38. https://doi.org/10.1207/s15328023top1501_8
Dreimuller, N., Schenkel, S., Stoll, M., Koch, C., Lieb, K., & Juenger, J. (2019). Development of a Checklist for Evaluating Psychiatric Reports. BMC Medical Education, 19, Article No. 121. https://doi.org/10.1186/s12909-019-1559-1
Fosado, R. E., Martínez, A., Hernández, N., & Ávila, R. (2018). The e-Portfolio as a Transversal Tool for Planning and Evaluating Autonomous Learning for Sustainable Development. Revista Iberoamericana Para La Investigación y El Desarrollo Educativo, 8, 194-215. https://doi.org/10.23913/ride.v8i16.338
Frederiksen, N., Saunders, D. R., & Wand, B. (1957). The In-Basket Test. Psychological Monographs: General and Applied, 71, 1-28. https://doi.org/10.1037/h0093706
Hatzikiriakou, K., & Metallidou, P. (2009). Teaching Deductive Reasoning to Pre-Service Teachers: Promises and Constraints. International Journal of Science and Mathematics Education, 7, 81-101. https://doi.org/10.1007/s10763-007-9113-8
Herrington, D. G., & Sweeder, R. D. (2018). Using Text Messages to Encourage Meaningful Self-Assessment outside of the Classroom. Journal of Chemical Education, 95, 2148-2154. https://doi.org/10.1021/acs.jchemed.8b00361
Jasdilla, L., Fitria, Y., & Sopandi, W. (2019). Predict Observe Explain (POE) Strategy toward Mental Model of Primary Students. Journal of Physics: Conference Series, 1157, 1-6. https://doi.org/10.1088/1742-6596/1157/2/022043
Kearney, M. (2004). Classroom Use of Multimedia-Supported Predict-Observe-Explain Tasks in a Social Constructivist Learning Environment. Research in Science Education, 34, 427-453. https://doi.org/10.1007/s11165-004-8795-y
Keeley, P. (2015). Science Formative Assessment: 75 Practical Strategies for Linking Assessment, Instruction, and Learning (2nd ed.). Corwin Press.
Kerr, W. C., & Macosko, J. C. (2011). Thermodynamic Venn Diagrams: Sorting out Forces, Fluxes, and Legendre Transforms. American Journal of Physics, 79, 950-953. https://doi.org/10.1119/1.3599177
Lancaster, E. L. (2007). Questions and Answers (Encouraging Students to Sing and Count out Loud). Clavier, 46, 48.
Latifah, S., Irwandani, I., Saregar, I., Diani, R., Fiani, O., Widayanti, W., & Deta, U. A. (2019). How the Predict-Observe-Explain (POE) Learning Strategy Remediates Students’ Misconception on Temperature and Heat Materials? Journal of Physics: Conference Series, 1171, 1-6. https://doi.org/10.1088/1742-6596/1171/1/012051
Lyons, H. (1999). How Portfolios Can Shape Emerging Practice. Educational Leadership, 56, 63-65.
Metcalfe, J., & Xu, J. (2018). Learning from One’s Own Errors and Those of Others. Bulletin & Review, 25, 402-408. https://doi.org/10.3758/s13423-017-1287-7
Ogle, D. (1986). K-M-L: A Teaching Model That Develops Active Reading of Expository Text. Reading Teacher, 39, 564-570. https://doi.org/10.1598/RT.39.6.11
Pendley, B. D., Bretz, R., & Novak, J. (1994). Concept Maps as a Tool to Assess Learning in Chemistry. Journal of Chemical Education, 71, 9-15. https://doi.org/10.1021/ed071p9
Quintanilla, M., Labarrere, A., Díaz, L., Camacho, J., Cuellar, L., & Ravanal, E. (2008). El inventario de ideas previas (KPSI) como un instrumento de regulación de los procesos de desarrollo profesional de docentes de ciencias naturales en ejercicio. Boletín de Investigación Educacional, 22, 97-114.
Robles, J. (2019). Rotation Self-Evaluation Checklist (REC): A Preceptors’ Perspective. Currents in Pharmacy Teaching and Learning, 11, 258-263. https://doi.org/10.1016/j.cptl.2018.12.005
Schippmann, J. S., Prien, E. P., & Katz, J. A. (1990). Reliability and Validity of In-Basket Performance Measures. Personnel Psychology, 43, 837-859. https://doi.org/10.1111/j.1744-6570.1990.tb00685.x
Sridharan, B., & Boud, D. (2019). The Effects of Peer Judgements on Teamwork and Self-Assessment Ability in Collaborative Group Work. Assessment & Evaluation in Higher Education, 44, 894-909. https://doi.org/10.1080/02602938.2018.1545898
Sutton, C. (1980). The Learner’s Prior Knowledge: A Critical Review of Techniques for Probing Its Organization. European Journal of Science Education, 2, 107-120. https://doi.org/10.1080/0140528800020202
Tamir, P., & Lunetta, V. N. (1978). An Analysis of Laboratory Activities in BSCS. Yellow Version. American Biology Teacher, 40, 426-428. https://doi.org/10.2307/4446325
To, J., & Panadero, E. (2019). Peer Assessment Effects on the Self-Assessment Process of First-Year Undergraduates. Assessment & Evaluation in Higher Education, 44, 920-932. https://doi.org/10.1080/02602938.2018.1548559
Tobajas, C., Molina, C. B., Quintanilla, A., Alonso-Morales, N., & Casas, J. A. (2019). Development and Application of Scoring Rubrics for Evaluating Students’ Competencies and Learning Outcomes in Chemical Engineering Experimental Courses. Education for Chemical Engineers, 26, 80-88. https://doi.org/10.1016/j.ece.2018.11.006
Treagust, D. F., & Chong, W. L. (1995). A Predict-Observe-Explain Teaching Sequence for Learning about Students’ Understanding of Heat and Expansion Liquids. Australian Science Teachers Journal, 41, 68-71.
Wardrop, J. (2012). Speaking out Loud: Muslim Women, Indian Delights and Culinary Practices in eThekwini/Durban.Social Dynamics—A Journal of African Studies, 38, 221-236. https://doi.org/10.1080/02533952.2012.717209
Witchel, H. T., Guppy, J. H., & Smith, C. F. (2018). The Self-Assessment Dilemma: An Open-Source, Ethical Method Using Matlab to Formulate Multiple-Choice Quiz Questions for Online Reinforcement. Advances in Physiology Education, 4, 697-703. https://doi.org/10.1152/advan.00081.2018
Wygoda, L., & Tague, R. (1995). Performance-Bases Chemistry: Developing Assesment Strategies in High School Chemistry. Journal of Chemical Education, 72, 909-911. https://doi.org/10.1021/ed072p909
Zakaluk, B., Samuels, J. S., & Taylor, B. (1986). A Simple Technique for Estimating Prior Knowledge: Word Association. Journal of Reading, 30, 56-60.