<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article  PUBLIC "-//NLM//DTD Journal Publishing DTD v3.0 20080202//EN" "http://dtd.nlm.nih.gov/publishing/3.0/journalpublishing3.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" dtd-version="3.0" xml:lang="en" article-type="research article"><front><journal-meta><journal-id journal-id-type="publisher-id">AM</journal-id><journal-title-group><journal-title>Applied Mathematics</journal-title></journal-title-group><issn pub-type="epub">2152-7385</issn><publisher><publisher-name>Scientific Research Publishing</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="doi">10.4236/am.2022.135029</article-id><article-id pub-id-type="publisher-id">AM-117511</article-id><article-categories><subj-group subj-group-type="heading"><subject>Articles</subject></subj-group><subj-group subj-group-type="Discipline-v2"><subject>Physics&amp;Mathematics</subject></subj-group></article-categories><title-group><article-title>
 
 
  Evaluating Pedagogical Quality of Learning Activities Using Fuzzy Evaluation Mappings: The Case of Pedagogical Games of Mathematical Proof
 
</article-title></title-group><contrib-group><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Pascal</surname><given-names>Stiefenhofer</given-names></name><xref ref-type="aff" rid="aff1"><sub>1</sub></xref><xref ref-type="corresp" rid="cor1"><sup>*</sup></xref></contrib></contrib-group><aff id="aff1"><label>1</label><addr-line>Newcastle University Business School, Newcastle upon Tyne, UK</addr-line></aff><pub-date pub-type="epub"><day>19</day><month>05</month><year>2022</year></pub-date><volume>13</volume><issue>05</issue><fpage>432</fpage><lpage>452</lpage><history><date date-type="received"><day>29,</day>	<month>March</month>	<year>2022</year></date><date date-type="rev-recd"><day>28,</day>	<month>May</month>	<year>2022</year>	</date><date date-type="accepted"><day>31,</day>	<month>May</month>	<year>2022</year></date></history><permissions><copyright-statement>&#169; Copyright  2014 by authors and Scientific Research Publishing Inc. </copyright-statement><copyright-year>2014</copyright-year><license><license-p>This work is licensed under the Creative Commons Attribution International License (CC BY). http://creativecommons.org/licenses/by/4.0/</license-p></license></permissions><abstract><p>
 
 
  This paper introduces a conceptual framework for developing pedagogical games of mathematical proof (PGMP) designed to help non-STEM students learn mathematical reasoning in a playful manner and without “fear”. Within the constructivist learning paradigm it develops an in-class learning activity where social science students learn the concept of mathematical deduction playfully using toys to construct tables from which formal proofs of propositions are derived without calculations. A PGMP pedagogy quality assurance evaluation method based on fuzzy evaluation mappings capturing learning effectiveness, learning efficiency, and learning satisfaction is introduced. Our results from an in-class experiment show that pedagogical games of mathematical proof help non-STEM students to effectively engage with mathematical reasoning playfully. The results are consistent according to a quality assurance consistency index.
 
</p></abstract><kwd-group><kwd>Pedagogy Quality Assurance</kwd><kwd> Pedagogical Games</kwd><kwd> Mathematics Anxiety</kwd><kwd> Proof</kwd><kwd> Toys</kwd></kwd-group></article-meta></front><body><sec id="s1"><title>1. Introduction</title><p>Self-efficacy is a student’s set of beliefs in her or his capacity to successfully execute a behavior necessary to produce a positive learning performance attainment [<xref ref-type="bibr" rid="scirp.117511-ref1">1</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref2">2</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref3">3</xref>]. Students with high self-efficacy beliefs allocate more effort when they encounter learning difficulties compared to students with low self-efficacy who often don’t make any notable attempt to achieve a learning goal, as they a priori belief that they will fail to achieve it [<xref ref-type="bibr" rid="scirp.117511-ref4">4</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref5">5</xref>].</p><p>A unique type of anxiety is mathematics and statistics anxiety. It is unique in this regard, as there are no widespread anxiety conditions for other specific learning content areas such as reading, writing, or history<sup>1</sup>. Mathematics and statistics anxiety is a negative emotional reaction to the manipulation of numbers and symbols that can be debilitating. It has been defined as “a feeling of tension and anxiety that interferes with the manipulation of numbers and the solving of mathematical problems in…ordinary life and academic situations [<xref ref-type="bibr" rid="scirp.117511-ref6">6</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref7">7</xref>].” Recent research shows that a key factor negatively impacting mathematics anxiety is self-efficacy beliefs toward learning mathematics [<xref ref-type="bibr" rid="scirp.117511-ref8">8</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref9">9</xref>]. This finding is in accordance with the observation that students with lower self-efficacy beliefs toward learning mathematics have higher levels of mathematics anxiety [<xref ref-type="bibr" rid="scirp.117511-ref10">10</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref11">11</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref12">12</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref13">13</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref14">14</xref>], a relation consistent across gender groups, grade-level groups, ethnic groups, and instruments measuring mathematics anxiety [<xref ref-type="bibr" rid="scirp.117511-ref15">15</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref16">16</xref>].</p><p>To help students overcome anxious feelings toward mathematics educators recently started to incorporate active learning elements in their instruction with pedagogical successes reported in attitudes towards mathematics studies [<xref ref-type="bibr" rid="scirp.117511-ref16">16</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref17">17</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref18">18</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref19">19</xref>]. Active learning in mathematics is a recent pedagogical development in response to a “call on institutions of higher education, ..., to ensure that effective active learning is incorporated into post-secondary mathematics classrooms”.<sup>2</sup> Prior to this call a land marking paper showed in a meta analysis of 225 studies that active learning improves learning and course performance in undergraduate STEM courses relative to traditional lecturing [<xref ref-type="bibr" rid="scirp.117511-ref20">20</xref>]. This research further suggests that active learning does not harm, and may further benefit, already high-achieving students. It also shows that students in classes with traditional lecturing were more likely to fail compared to students in classes with active learning. Active learning helps students effectively engage in their learning process and improve their performance attainment [<xref ref-type="bibr" rid="scirp.117511-ref21">21</xref>]. The basic premises of active learning include instructional techniques reinforcing higher-order thinking, requiring students to co-create their own knowledge through active participation in the learning process [<xref ref-type="bibr" rid="scirp.117511-ref22">22</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref23">23</xref>].</p><p>Game based learning is an emerging active learning instructional method in mathematics with several advantages [<xref ref-type="bibr" rid="scirp.117511-ref24">24</xref>]: it increases student engagement [<xref ref-type="bibr" rid="scirp.117511-ref25">25</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref26">26</xref>], enhances self-efficacy [<xref ref-type="bibr" rid="scirp.117511-ref18">18</xref>], improves student achievement [<xref ref-type="bibr" rid="scirp.117511-ref27">27</xref>], motivation and resilience [<xref ref-type="bibr" rid="scirp.117511-ref28">28</xref>]. The literature on pedagogical games supporting teaching and learning of mathematics in higher education courses is scarce, and largely focuses on online settings [<xref ref-type="bibr" rid="scirp.117511-ref29">29</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref30">30</xref>]. The game introduced in this paper, however, uses a “toy” to help students put hands on mathematics to physically flip and or rotate wooden objects to fit their counter-shape holes in a box. Students then write down their observations into a table from which mathematical proofs can be deduced. Even advanced mathematical concepts such as Abelian group properties of symmetries can be deduced from this method by playfully constructing a Cayley table. The activity considered here, enables students to derive proofs without calculations. There is currently little evidence in the literature about the effectiveness of using toys as an active learning activity to derive proofs “playfully”. This paper sets out to investigate this by developing a formal framework for constructing pedagogical games of mathematical proof and their pedagogical quality evaluations.</p><p>It is generally difficult to determine if a pedagogical game is devised with the explicit intention of meeting educational criteria, or whether researchers are appropriating existing games to test their possible educational impact [<xref ref-type="bibr" rid="scirp.117511-ref29">29</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref31">31</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref32">32</xref>]. Moreover, how would one compare the effectiveness of alternative pedagogies relative to each other? Naik (2017) suggests that the effectiveness of pedagogical games should ultimately be measured in terms of student learning outcomes [<xref ref-type="bibr" rid="scirp.117511-ref29">29</xref>]. In this paper, we consider a geometric approach based on fuzzy mappings to establish a pedagogical quality assurance system in which pedagogical effectiveness is only one dimension. We determine the quality of a pedagogical game by considering three main learning dimensions including: learning effectiveness, learning efficiency, and learning satisfaction. We utilize the concepts of fuzzy variables and metric functions to obtain a pedagogical quality assurance evaluation method.</p><p>The organization of this paper is as follows. Section two provides a formal framework for developing pedagogical games of mathematical proof. We apply this pedagogical framework in section four where we provide an example. Section three introduces the PGMP pedagogical quality assurance evaluation model using fuzzy variables. It provides a process for calculating a pedagogical quality measure supported by a consistency measure to judge pedagogical quality assurance of PGMPs. Section four provides the results of an in-class experiment utilizing both, the PGMP framework and its pedagogical evaluation model. Section five is a conclusion followed by an appendix.</p></sec><sec id="s2"><title>2. Constructing a PGMP Learning Activity</title><p>We define a pedagogical game of mathematical proof expressed as</p><p>⊨ A ( Γ ⊢ ϕ ) G . (1)</p><p>A pedagogical game of mathematical proof, hence, consists of a teaching model ⊨ A defined by a set of pedagogies A, a set of rules G defining the game structure, and the mathematical proof Γ ⊢ ϕ , consisting of a set of assumptions Γ , a true statement ϕ or set of statements Φ , and the method of deduction ⊢ . <xref ref-type="table" rid="table1"><xref ref-type="table" rid="table">Table </xref>1</xref> below summarizes the steps in the development of a PGMP as described in (1).</p><p>Implicit assumptions: Students have been introduced to basic abstract algebra and are familiar with the concepts of a group, Abelian group, and (non-)symmetry of mathematical objects such as those shown in <xref ref-type="fig" rid="fig1">Figure 1</xref>.</p><p>The theoretical model summarized in <xref ref-type="table" rid="table1"><xref ref-type="table" rid="table">Table </xref>1</xref> requires educators to define the</p><table-wrap id="table1" ><label><xref ref-type="table" rid="table1"><xref ref-type="table" rid="table">Table </xref>1</xref></label><caption><title> How to construct a PGMP</title></caption><table><tbody><thead><tr><th align="center" valign="middle" ></th><th align="center" valign="middle" >Construction of PGMP</th><th align="center" valign="middle" >Description</th></tr></thead><tr><td align="center" valign="middle" >1</td><td align="center" valign="middle" >Choose Γ ⊢ ϕ</td><td align="center" valign="middle" >Decide about the proof structure ⊢ , i.e. direct proof, proof by contradiction, etc. Then choose the assumptions Γ and statement ϕ ; which is to formulate the theorem to be proven.</td></tr><tr><td align="center" valign="middle" >2</td><td align="center" valign="middle" >Choose G over ( Γ ⊢ ϕ ) .</td><td align="center" valign="middle" >Define a set of rules describing the mathematical game G (consider single versus multi player games).</td></tr><tr><td align="center" valign="middle" >3</td><td align="center" valign="middle" >Choose ⊨ A over ( Γ ⊢ ϕ ) G</td><td align="center" valign="middle" >Choose A (could be a mix of pedagogies) to define ⊨ A . Keep in mind “mathematics without fear”.</td></tr></tbody></table></table-wrap><p>sets Γ , Φ , G , and A. We use the notation Φ to indicate a set of mathematical statements Φ : = { ϕ 1 , ϕ 2 , ⋯ , ϕ n } . For example Φ may represent a set of exercises, and ϕ i a specific statement i = 1 , 2 , ⋯ , n to be proven.</p><p>Step 1: Define Γ and Φ , and formulate ( Γ ⊢ ϕ ) as in Equation (1). We define the premises Γ . That is</p><p>Γ : = { γ 1 , ⋯ , γ 4 } , (2)</p><p>where</p><p>γ 1 : Δ is an equilateral triangle.</p><p>γ 2 : □ is a box with a whole Δ in it such that a block Δ fits in □ . See <xref ref-type="fig" rid="fig1">Figure 1</xref>.</p><p>γ 3 : There are two transformations called rotation Θ and flip Ψ .</p><p>γ 4 : There is a composition ∘ of transformations Θ and Ψ acting on the block Δ .</p><p>Next, let there be a set of statements to be proven, where each statement ϕ i corresponds to an exercise.</p><p>Φ : = { ϕ 1 , ⋯ , ϕ 6 } , (3)</p><p>where</p><p>ϕ 1 : Δ fits in □ in 6 different ways.</p><p>ϕ 2 : Δ ,   □ , Θ , Ψ and ∘ form a group ( G , ∗ ) .</p><p>ϕ 3 : Δ ,   □ , Θ and ∘ form subgroups ( S i , ∗ ) of ( G , ∗ ) , for some i = 1 , 2 , ⋯ , n .</p><p>ϕ 4 : ( S i , ∗ ) are Abelian.</p><p>ϕ 5 : Δ ,   □ , Θ , Ψ and ∘ form subgroups T i of ( G , ∗ ) , for some i = 1 , 2 , ⋯ , n .</p><p>ϕ 5 : ( T i , ∗ ) are Abelian.</p><p>ϕ 6 : ( G , ∗ ) is not Abelian.</p><p>Step 2: Choose G over ( Γ ⊢ ϕ ) . This requires to define the set of rules of the mathematical game.</p><p>G : = { g 1 , g 2 , g 3 } , (4)</p><p>where</p><p>g 1 : Θ is a right (or left) rotation of 120 degrees with fixed center in Δ .</p><p>g 2 : Ψ is a right or left flip of 180 degrees with fixed center in Δ (imagine a line from any vortex through the center of Δ ).</p><p>g 3 : ∘ is a composition of transformations acting on Δ .</p><p>The definition of the set G must be in accordance with the guidelines set out in the pedagogy A. In our case, the elements of G are supposed to support a constructivist learning approach, where students actively co-create knowledge by physically rotating and flipping a block Δ to match a hole of same shape Δ in a box □ to derive a table of all possible successful combinations of flips and rotations of Δ from which proofs are derived.</p><p>Step 3: We now define the pedagogy of our game of mathematical proof. With an increasing awareness that many undergraduates are passive during seminar sessions [<xref ref-type="bibr" rid="scirp.117511-ref33">33</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref34">34</xref>], we define a learning activity that allow students to actively engage in co-creating knowledge. Each student is provided a wooden block (an equilateral triangle) Δ and a wooden box □ with a hole of shape Δ in it. The student is then asked to use the building block to physically rotate and flip it to fit the hole and to therewith derive symmetries and present observations in form of a table from which proofs of propositions are then derived. We defined the set A which includes the following conditions.</p><p>1) Students are provided with a wooden box and a symmetric wooden object as shown in <xref ref-type="fig" rid="fig1">Figure 1</xref>.</p><p>2) Students are provided with a problem set which contains a set of instructions for each problem (Appendix).</p><p>3) Students work independently on the problem set (single player game).</p><p>4) Students are provided with 50 minutes of time to complete a set of tasks (Theorems 1 and 2 in Appendix).</p><p>5) Students can ask the instructor for definitions (loosing 5 points each).</p><p>6) Students compete with each other on time (loosing 5 points each rank) and points (each task is worth some points).</p><p>7) A total point score is calculated and appropriate reductions are made. The winner is announced to the class.</p><p>The pedagogical model ⊨ A consists of a mixed pedagogy including elements from active learning and competitive learning both based within the constructivist paradigm with students co-creating their own knowledge playfully.</p></sec><sec id="s3"><title>3. Pedagogy Quality Assurance Evaluation Method</title><p>In order to assess the usefulness of a PGMP as a learning activity we develop a pedagogy quality assurance evaluation method (PEM). We construct fuzzy variables [<xref ref-type="bibr" rid="scirp.117511-ref35">35</xref>] combined with a hierarchical model [<xref ref-type="bibr" rid="scirp.117511-ref36">36</xref>] to obtain a numerical value expressing pedagogical quality [<xref ref-type="bibr" rid="scirp.117511-ref37">37</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref38">38</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref39">39</xref>]. In our PEM, quality is defined as a weighted average of learning effectiveness, learning efficiency, and learning satisfaction as shown in <xref ref-type="fig" rid="fig2">Figure 2</xref>.</p><p>Let there be n evaluation factors represented by a set</p><p>U = { u 1 , u 2 , ⋯ , u n } , (5)</p><p>where u i is the i<sup>th</sup> evaluation factor in u ∈ U and u = ( u 1 , u 2 , ⋯ , u n ) a vector. There are m levels of appraisal grades represented by a set</p><p>V = { v 1 , v 2 , ⋯ , v m } , (6)</p><p>where v k is the k<sup>th</sup> appraisal grade in v ∈ V on a Likert scale and v = ( v 1 , v 2 , ⋯ , v m ) is a vector. v 1 represents “strongly disagree” and gradually increasing to v m representing “strongly agree”. A mapping U → V is a fuzzy evaluation mapping if for each evaluation factor u i ∈ U there is a mapping</p><p>μ Π i : U → [ 0,1 ] (7)</p><p>where Π i is a fuzzy set associated with evaluation factor u i ∈ U . Alternatively, for every u i ∈ U the mapping Π i : U → [ 0,1 ] yields a row vector π i = ( π i 1 , π i 2 , ⋯ , π i m ) , where π i k represents the fuzzy membership degree of appraisal factor i to grade k. The general fuzzy appraisal matrix [ π i k ] for all evaluation factors i = 1 , ⋯ , n and appraisal grades k = 1 , ⋯ , m is denoted by</p><p>Π ( n &#215; m ) = [ π 11 π 12 ⋯ π 1 m π 21 π 22 ⋯ π 2 m ⋮ ⋮ ⋱ ⋮ π n 1 π n 2 ⋯ π n m ] . (8)</p><p>For simplicity, we employ a triangular distribution function μ in the construction of the mapping Π characterizing the fuzzy measure values π i k for i = 1 , 2 , ⋯ , n and j = 1 , 2 , ⋯ , m [<xref ref-type="bibr" rid="scirp.117511-ref37">37</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref38">38</xref>]. In order to obtain a comprehensive</p><p>overall quality assurance evaluation vector Q ( 1 &#215; m ) = ( q 1 , q 2 , ⋯ , q m ) we construct a weight vector W ( n &#215; 1 ) by the AHP method [<xref ref-type="bibr" rid="scirp.117511-ref36">36</xref>] assigning relative weight w i to evaluation factor u i ∈ U for every i = 1 , 2 , ⋯ , n . The overall quality assurance evaluation vector Q is obtained by</p><p>Q = W Π , (9)</p><p>where</p><p>q k = min { 1 , ∑ i = 1 n     W i π i k } ,     for   k = 1 , ⋯ , m . (10)</p><p><xref ref-type="fig" rid="fig2">Figure 2</xref> outlines the overall methodology of the quality assurance evaluation process. Pedagogical quality is constructed as a measure of learning performance, defined by the degree of which students can successfully perform tasks within given time constraints, and learning satisfaction. A Post-Study System Usability Questionnaire (PSSUQ) ( [<xref ref-type="bibr" rid="scirp.117511-ref40">40</xref>], p.75-77) with respect to all three dimensions is employed to obtain survey data. The goal of this quality assurance system is to provide educators with a measure for the overall quality Q of a pedagogy.</p><p>We begin by constructing the fuzzy sets Π i from student survey (PSSUQ) and expert survey data. The later can be obtained via qualitative interview with subject experts. For simplicity, we assume a triangular distribution function μ i to construct the mapping functions characterizing fuzzy measure values [ π i k ] . We conduct a focus group with education experts to obtain the data points v i 1 , v i 2 , ⋯ , v i 8 for each evaluation factor u i in the product quality evaluation factor set U and appraisal intervals ( v i 1 , v i 2 ) , ( v i 2 , v i 3 ) , ⋯ , ( v i 7 , v i 8 ) . Then for all v &#175; i ∈ [ v i 1 , ⋯ , v i m ]</p><p>μ i ( v &#175; k ) = ( p k ( v &#175; − v k ) + 1, when   v &#175; k ∈ [ v k − 1 p k , v k ] p k ( v k − v &#175; ) + 1, when   v &#175; k ∈ [ v k , v k + 1 p k ] 0, otherwise , (11)</p><p>where each v &#175; k is some value provided by the student survey data (PSSUQ). From Equation (11) we use the values μ i ( v &#175; k ) ∑ k = 1 m     μ i k ( v i k ) for k = 1 , 2 , ⋯ , m to calculate the normalized appraisal vector Π i .</p><p>Π i = { π i 1 , π i 2 , ⋯ , π i m } , (12)</p><p>with appraisals grades v i ∈ V for evaluation factor i. Π i can then be normalized in the usual way yielding the values π i k .</p><p>Some factors u i ∈ U according to our product quality definition provided above contribute with different importance to the overall measure of pedagogical quality. We now discuss the construction of the vector W using the AHP method [<xref ref-type="bibr" rid="scirp.117511-ref41">41</xref>]. The AHP method requires experts to make pair-wise comparesons between evaluation factors A 1 , A 2 , ⋯ , A n and assigning numerical values a i j for i , j = 1 , 2 , ⋯ , n to them. This yields a square matrix</p><p>A ( n &#215; n ) = A 1     A 2       ⋯   A n A 1 A 2 ⋮ A n ( a 1 , 1 a 1 , 2 ⋯ a 1 , n a 2 , 1 a 2 , 2 ⋯ a 2 , n ⋮ ⋮ ⋱ ⋮ a n , 1 a n , 2 ⋯ a n , n ) (13)</p><p>of relative weights W 1 , W 2 , ⋯ , W n with the following properties:</p><p>1) a i j ≈ W i W j , for i , j = 1 , 2 , ⋯ , n .</p><p>2) a i i = 1 , for all i = 1 , 2 , ⋯ , n .</p><p>3) If a i j = α for α ≠ 0 , then a j i = 1 α , for i = 1 , 2 , ⋯ , n .</p><p>4) If A i is more important than A j then a i j ≅ ( W i / W j ) &gt; 1 .</p><p>These properties yield a positive definite and reciprocal matrix with 1’s on the main diagonal. The experts only need to provide data for the upper triangle of the matrix, that is L = n ( n − 1 ) / 2 data points. Property one above suggests a relation</p><p>A W = n W (14)</p><p>One can use Saaty’s method to compute W [<xref ref-type="bibr" rid="scirp.117511-ref41">41</xref>]. The final step is to check that human judgments (expert data, A) are consistent. For that purpose, we calculate a consistency index (CI) measuring how a given matrix A compares to a purely random matrix in terms of their consistency indices. Let CI be the consistency index</p><p>C I = λ max − n n − 1 (15)</p><p>where n is the size of the pair-wise comparison matrix A and λ max represents the maximum eigenvalue of the matrix. Let RI be the average random index which can be computed as shown in [<xref ref-type="bibr" rid="scirp.117511-ref41">41</xref>]. A matrix A is consistent if and only if λ max = n and that we always have λ max &gt; n when matrix A is a positive reciprocal one. If CI is less than 0.1, the numerical judgments will be considered acceptable [<xref ref-type="bibr" rid="scirp.117511-ref41">41</xref>].</p></sec><sec id="s4"><title>4. Results from an In-Class Experiment</title><p>An in-class experiment with 10 third year undergraduate economics students taking an advanced mathematics course was conducted at the University of Exeter. The student cohort attended two different one hour sessions plus 15 minutes introduction to the learning activity. In session one, they conducted the learning activity “Theorem 1” and after 7 days, session two covered learning activity “Theorem 2”<sup>3</sup>. The precises activities are discussed in section three and the appendix of this paper. Student performance was incentivized with a competitive game element, distributing points to correct answers and penalizing students with negative points who requested definitions from the tutor. A total score was determined and the best performer was announced to the cohort.</p><p>Students reported Subjective Task Time, measured on a Likert scale 1 - 10 with 1 long subjective time perception and 10 short subjective time perception. Both Subjective Task Completion and Subjective Task Time was reported in a USQ survey [<xref ref-type="bibr" rid="scirp.117511-ref40">40</xref>], with Task Completion determined by a tutor who marked the completed tasks to obtain an overall assessment score for each student. Together, Task Time and Task Completion determine a weighted measure of Learning Performance.</p><p>A focus group consisting of five mathematics education experts was conducted to determine the bounds of the metric function in Equation (11). As a rule, experts had to jointly agree about the metric bounds, which were determined in a verbal 30 minutes debate after being introduced to the specifics of the experiment and PGMP. Expert decisions were purely based on the description of the PGMP and they were not privy to student survey results. The same cohort of experts also provided numerical values (matrix (17)), based on a pairwise comparison of the variables “fun to play and usability”, “instruction quality”, and “game quality” of the PGMP to yield the matrix A in Equation (13).</p><p><xref ref-type="table" rid="table2"><xref ref-type="table" rid="table">Table </xref>2</xref> shows the individual average task success and self evaluated task time data for both sets of exercises resulting in the proofs of theorems 1 and 2. Associated with each theorem, an USQ survey collected data on task completion and task completion time for six tasks (see Appendix) [<xref ref-type="bibr" rid="scirp.117511-ref40">40</xref>]. On average task success was 8.308 (Learning Effectiveness) with associated average task time 8.6 (Learning Efficiency) for theorem 1. For theorem 2, average task success was 8.617 (Learning Effectiveness) with associated average task time of 8.057 (Learning Efficiency). average task success was higher for theorem 2, students completed the six tasks While associated with theorem 1 faster. From this table an average</p><table-wrap id="table2" ><label><xref ref-type="table" rid="table2"><xref ref-type="table" rid="table">Table </xref>2</xref></label><caption><title> Individual task success and task time for theorems 1 and 2</title></caption><table><tbody><thead><tr><th align="center" valign="middle"  rowspan="2"  >Subject</th><th align="center" valign="middle"  colspan="2"  >Theorem 1</th><th align="center" valign="middle"  colspan="2"  >Theorem 2</th></tr></thead><tr><td align="center" valign="middle" >Task Success 1</td><td align="center" valign="middle" >Task Time 1</td><td align="center" valign="middle" >Task Success 2</td><td align="center" valign="middle" >Task Time 2</td></tr><tr><td align="center" valign="middle" >1</td><td align="center" valign="middle" >7.833</td><td align="center" valign="middle" >8.286</td><td align="center" valign="middle" >8.333</td><td align="center" valign="middle" >7.857</td></tr><tr><td align="center" valign="middle" >2</td><td align="center" valign="middle" >8.33</td><td align="center" valign="middle" >8.571</td><td align="center" valign="middle" >8.667</td><td align="center" valign="middle" >7.714</td></tr><tr><td align="center" valign="middle" >3</td><td align="center" valign="middle" >8.5</td><td align="center" valign="middle" >7.714</td><td align="center" valign="middle" >9.167</td><td align="center" valign="middle" >7.571</td></tr><tr><td align="center" valign="middle" >4</td><td align="center" valign="middle" >8.667</td><td align="center" valign="middle" >8.857</td><td align="center" valign="middle" >8.500</td><td align="center" valign="middle" >8.571</td></tr><tr><td align="center" valign="middle" >5</td><td align="center" valign="middle" >7.667</td><td align="center" valign="middle" >9.0</td><td align="center" valign="middle" >8.333</td><td align="center" valign="middle" >7.857</td></tr><tr><td align="center" valign="middle" >6</td><td align="center" valign="middle" >8.333</td><td align="center" valign="middle" >9.143</td><td align="center" valign="middle" >8.167</td><td align="center" valign="middle" >8.0</td></tr><tr><td align="center" valign="middle" >7</td><td align="center" valign="middle" >9.0</td><td align="center" valign="middle" >8.714</td><td align="center" valign="middle" >9.333</td><td align="center" valign="middle" >8.571</td></tr><tr><td align="center" valign="middle" >8</td><td align="center" valign="middle" >8.5</td><td align="center" valign="middle" >9.143</td><td align="center" valign="middle" >9.333</td><td align="center" valign="middle" >8.857</td></tr><tr><td align="center" valign="middle" >9</td><td align="center" valign="middle" >8.5</td><td align="center" valign="middle" >8.143</td><td align="center" valign="middle" >8.667</td><td align="center" valign="middle" >8.143</td></tr><tr><td align="center" valign="middle" >10</td><td align="center" valign="middle" >7.667</td><td align="center" valign="middle" >8.429</td><td align="center" valign="middle" >7.667</td><td align="center" valign="middle" >7.429</td></tr></tbody></table></table-wrap><p>Learning Performance of 8.454 for theorem 1 compared to 8.337 for theorem 2 suggests that students performed better on proving theorem 1.</p><p><xref ref-type="table" rid="table3"><xref ref-type="table" rid="table">Table </xref>3</xref> reports individual subject data from the PSSUQ survey on three variables: average “fun to play and usability” (PGMP play) 4.250, average instruction quality (InstQual) 4.129, and average game quality (PGMPQual) 4.100. From this table, an overall average Learning Satisfaction value of 4.160 is obtained.</p><p>We now determine the parameters of the fuzzy evaluation mapping given in Equation (11). For simplicity, we assume a triangular metric. From a focus group consisting of five mathematics education experts we obtain the boundary values of each fuzzy variable reported in <xref ref-type="table" rid="table4"><xref ref-type="table" rid="table">Table </xref>4</xref>. The experts agree to apply the same metric boundaries for Task Success and Task Time, but propose different metric</p><table-wrap id="table3" ><label><xref ref-type="table" rid="table3"><xref ref-type="table" rid="table">Table </xref>3</xref></label><caption><title> Individual pedagogy satisfaction measured by PGMP play, InstQual, and PGMPQual</title></caption><table><tbody><thead><tr><th align="center" valign="middle"  rowspan="2"  >Subject</th><th align="center" valign="middle"  colspan="3"  >Learning Satisfaction</th></tr></thead><tr><td align="center" valign="middle" >PGMP play</td><td align="center" valign="middle" >InstQual</td><td align="center" valign="middle" >PGMPQual</td></tr><tr><td align="center" valign="middle" >1</td><td align="center" valign="middle" >4.625</td><td align="center" valign="middle" >4.571</td><td align="center" valign="middle" >4.667</td></tr><tr><td align="center" valign="middle" >2</td><td align="center" valign="middle" >4.625</td><td align="center" valign="middle" >4.571</td><td align="center" valign="middle" >4.0</td></tr><tr><td align="center" valign="middle" >3</td><td align="center" valign="middle" >4.0</td><td align="center" valign="middle" >4.0</td><td align="center" valign="middle" >4.667</td></tr><tr><td align="center" valign="middle" >4</td><td align="center" valign="middle" >4.5</td><td align="center" valign="middle" >4.143</td><td align="center" valign="middle" >3.333</td></tr><tr><td align="center" valign="middle" >5</td><td align="center" valign="middle" >4.375</td><td align="center" valign="middle" >4.0</td><td align="center" valign="middle" >4.0</td></tr><tr><td align="center" valign="middle" >6</td><td align="center" valign="middle" >3.625</td><td align="center" valign="middle" >4.0</td><td align="center" valign="middle" >3.0</td></tr><tr><td align="center" valign="middle" >7</td><td align="center" valign="middle" >4.375</td><td align="center" valign="middle" >4.143</td><td align="center" valign="middle" >5.0</td></tr><tr><td align="center" valign="middle" >8</td><td align="center" valign="middle" >4.125</td><td align="center" valign="middle" >3.714</td><td align="center" valign="middle" >4.333</td></tr><tr><td align="center" valign="middle" >9</td><td align="center" valign="middle" >4.625</td><td align="center" valign="middle" >4.143</td><td align="center" valign="middle" >3.667</td></tr><tr><td align="center" valign="middle" >10</td><td align="center" valign="middle" >3.625</td><td align="center" valign="middle" >4.0</td><td align="center" valign="middle" >4.333</td></tr></tbody></table></table-wrap><table-wrap id="table4" ><label><xref ref-type="table" rid="table4"><xref ref-type="table" rid="table">Table </xref>4</xref></label><caption><title> Expert metric evaluation matrix</title></caption><table><tbody><thead><tr><th align="center" valign="middle" ></th><th align="center" valign="middle"  colspan="2"  >PGMP Satisfaction</th><th align="center" valign="middle"  colspan="2"  >Task Success/Time</th><th align="center" valign="middle"  rowspan="2"  >Magnitude</th></tr></thead><tr><td align="center" valign="middle" ></td><td align="center" valign="middle" >low</td><td align="center" valign="middle" >high</td><td align="center" valign="middle" >low</td><td align="center" valign="middle" >high</td></tr><tr><td align="center" valign="middle" >very poor</td><td align="center" valign="middle" >1</td><td align="center" valign="middle" >2</td><td align="center" valign="middle" >1</td><td align="center" valign="middle" >2.5</td><td align="center" valign="middle" >negative</td></tr><tr><td align="center" valign="middle" >poor</td><td align="center" valign="middle" >1</td><td align="center" valign="middle" >1.7</td><td align="center" valign="middle" >1</td><td align="center" valign="middle" >2.5</td><td align="center" valign="middle" >positive</td></tr><tr><td align="center" valign="middle" ></td><td align="center" valign="middle" >1.7</td><td align="center" valign="middle" >3.5</td><td align="center" valign="middle" >2.5</td><td align="center" valign="middle" >5</td><td align="center" valign="middle" >negative</td></tr><tr><td align="center" valign="middle" >indifferent</td><td align="center" valign="middle" >2</td><td align="center" valign="middle" >3</td><td align="center" valign="middle" >2.5</td><td align="center" valign="middle" >5</td><td align="center" valign="middle" >positive</td></tr><tr><td align="center" valign="middle" ></td><td align="center" valign="middle" >3</td><td align="center" valign="middle" >4.4</td><td align="center" valign="middle" >5</td><td align="center" valign="middle" >9</td><td align="center" valign="middle" >negative</td></tr><tr><td align="center" valign="middle" >good</td><td align="center" valign="middle" >3.5</td><td align="center" valign="middle" >4</td><td align="center" valign="middle" >5</td><td align="center" valign="middle" >9</td><td align="center" valign="middle" >positive</td></tr><tr><td align="center" valign="middle" ></td><td align="center" valign="middle" >4</td><td align="center" valign="middle" >5</td><td align="center" valign="middle" >9</td><td align="center" valign="middle" >10</td><td align="center" valign="middle" >negative</td></tr><tr><td align="center" valign="middle" >very good</td><td align="center" valign="middle" >4.4</td><td align="center" valign="middle" >5</td><td align="center" valign="middle" >9</td><td align="center" valign="middle" >10</td><td align="center" valign="middle" >positive</td></tr></tbody></table></table-wrap><p>boundaries for PGMP Satisfaction. <xref ref-type="fig" rid="fig3">Figure 3</xref> graphically shows the suggested metric for PGMP Satisfaction and <xref ref-type="fig" rid="fig4">Figure 4</xref> graphically represents the metric for Task Success and Task Time and <xref ref-type="table" rid="table5"><xref ref-type="table" rid="table">Table </xref>5</xref>.</p><p>TableA1 (appendix) and Table5 report the individual fuzzy values for Task Success for Theorems I and II. TableA2 (appendix) and Table6 report the individual fuzzy values for Task Time for Theorems I and II.</p><p>From <xref ref-type="table" rid="table5"><xref ref-type="table" rid="table">Table </xref>5</xref>, <xref ref-type="table" rid="table6"><xref ref-type="table" rid="table">Table </xref>6</xref> and <xref ref-type="table" rid="table3"><xref ref-type="table" rid="table">Table </xref>3</xref> we obtain the matrix</p><p>Π = ( 0 0 0.13462 0.776883 0.88495 0 0 0.244403 0.755597 0 0 0 0.177471 0.75793 0.064599 ) . (16)</p><p>The following matrix A is constructed with input from experts:</p><p>InstQual GameQual PGMPplay InstQual         GameQual PGMPplay [ 1     1 2     2 2     1     3 1 2     1 3     1 ] (17)</p><table-wrap id="table5" ><label><xref ref-type="table" rid="table5"><xref ref-type="table" rid="table">Table </xref>5</xref></label><caption><title> Task success theorem 2</title></caption><table><tbody><thead><tr><th align="center" valign="middle"  rowspan="2"  >Subject</th><th align="center" valign="middle"  colspan="5"  >Theorem 2</th></tr></thead><tr><td align="center" valign="middle" >very poor</td><td align="center" valign="middle" >poor</td><td align="center" valign="middle" >indifferent</td><td align="center" valign="middle" >good</td><td align="center" valign="middle" >very good</td></tr><tr><td align="center" valign="middle" >1</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.1668</td><td align="center" valign="middle" >0.8666</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >2</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.0833</td><td align="center" valign="middle" >0.9168</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >3</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.833</td><td align="center" valign="middle" >0.167</td></tr><tr><td align="center" valign="middle" >4</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.125</td><td align="center" valign="middle" >0.875</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >5</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.1668</td><td align="center" valign="middle" >0.8666</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >6</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.2083</td><td align="center" valign="middle" >0.7918</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >7</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.3</td></tr><tr><td align="center" valign="middle" >8</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.3</td></tr><tr><td align="center" valign="middle" >9</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.0833</td><td align="center" valign="middle" >0.9168</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >10</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.3333</td><td align="center" valign="middle" >0.6668</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >sum</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >1.1668</td><td align="center" valign="middle" >6.7334</td><td align="center" valign="middle" >0.767</td></tr><tr><td align="center" valign="middle" >normalized sum</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.134622</td><td align="center" valign="middle" >0.776883</td><td align="center" valign="middle" >0.88495</td></tr></tbody></table></table-wrap><table-wrap id="table6" ><label><xref ref-type="table" rid="table6"><xref ref-type="table" rid="table">Table </xref>6</xref></label><caption><title> Task time theorem 2</title></caption><table><tbody><thead><tr><th align="center" valign="middle"  rowspan="2"  >Subject</th><th align="center" valign="middle"  colspan="5"  >Theorem 2</th></tr></thead><tr><td align="center" valign="middle" >very poor</td><td align="center" valign="middle" >poor</td><td align="center" valign="middle" >indifferent</td><td align="center" valign="middle" >good</td><td align="center" valign="middle" >very good</td></tr><tr><td align="center" valign="middle" >1</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.2858</td><td align="center" valign="middle" >0.7142</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >2</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.4572</td><td align="center" valign="middle" >0.6785</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >3</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.3573</td><td align="center" valign="middle" >0.6427</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >4</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.1072</td><td align="center" valign="middle" >0.8928</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >5</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.2857</td><td align="center" valign="middle" >0.7143</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >6</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.25</td><td align="center" valign="middle" >0.75</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >7</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.1073</td><td align="center" valign="middle" >0.8927</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >8</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.0358</td><td align="center" valign="middle" >0.9643</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >9</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.2143</td><td align="center" valign="middle" >0.7858</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >10</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.3927</td><td align="center" valign="middle" >0.673</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >sum</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >2.4933</td><td align="center" valign="middle" >7.7083</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >normalized sum</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.244403</td><td align="center" valign="middle" >0.755597</td><td align="center" valign="middle" >0</td></tr></tbody></table></table-wrap><p>Using the average of normalized columns (ANC) method</p><p>a ′ i = 1 n ∑ j = 1 n a i j ∑ i = 1 n a i j ,     for   i , j = 1 , 2 , ⋯ , n . (18)</p><p>where a i j are the entries of the matrix A we obtain a normalized matrix</p><table-wrap id="table7" ><label><xref ref-type="table" rid="table7"><xref ref-type="table" rid="table">Table </xref>7</xref></label><caption><title> Satisfaction evaluation</title></caption><table><tbody><thead><tr><th align="center" valign="middle"  rowspan="2"  >Subject</th><th align="center" valign="middle"  colspan="5"  >PGMP Satisfaction</th></tr></thead><tr><td align="center" valign="middle" >very poor</td><td align="center" valign="middle" >poor</td><td align="center" valign="middle" >indifferent</td><td align="center" valign="middle" >good</td><td align="center" valign="middle" >very good</td></tr><tr><td align="center" valign="middle" >1</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.368</td><td align="center" valign="middle" >0.3867</td></tr><tr><td align="center" valign="middle" >2</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.474</td><td align="center" valign="middle" >0.21</td></tr><tr><td align="center" valign="middle" >3</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.2107</td><td align="center" valign="middle" >0.895</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >4</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.1729</td><td align="center" valign="middle" >0.842</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >5</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.1729</td><td align="center" valign="middle" >0.842</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >6</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.3535</td><td align="center" valign="middle" >0.474</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >7</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.0229</td><td align="center" valign="middle" >0.632</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >8</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.2857</td><td align="center" valign="middle" >1</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >9</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.06</td><td align="center" valign="middle" >0.684</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >10</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.3607</td><td align="center" valign="middle" >0.79</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >sum</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >1.6393</td><td align="center" valign="middle" >7.001</td><td align="center" valign="middle" >0.5967</td></tr><tr><td align="center" valign="middle" >normalized sum</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.177471</td><td align="center" valign="middle" >0.75793</td><td align="center" valign="middle" >0.064599</td></tr></tbody></table></table-wrap><p>A ′ = ( 0.2857 0.2727 0.3333 0.5714 0.5454 0.5 0.1429 0.1818 0.166 ) . (19)</p><p>Averaging A ′ over rows yields a vector W ( 1 &#215; n ) defined by</p><p>w j = ∑ j = 1 n a i j n     for   i , j = 1,2, ⋯ , n . (20)</p><p>Hence W = ( 0.297,0.539,0.164 ) . We calculate the overall evaluation vector</p><p>Q = ( 0.297,0.539,0.164 ) ⋅ ( 0 0 0.13462 0.776883 0.88495 0 0 0.244403 0.755597 0 0 0 0.177471 0.75793 0.064599 ) . (21)</p><p>That is Q = ( 0,0,0.20,0.762,0.273 ) . We defuzyfy Q to obtain an overall measure of quality Q &#175; using</p><p>Q &#175; = ∑ i = 1 n b i 2 a i ∑ i = 1 n b i 2 = 0 2 &#215; 31 + 0 2 &#215; 50 + 0.2 2 &#215; 67 + 0.762 2 &#215; 82 + 0.273 2 &#215; 95 0.2 2 + 0.762 2 + 0.273 2 = 82.551 (22)</p><p>where a 1 = 31 , a 2 = 50 , a 3 = 67 , a 4 = 82 and a 5 = 95 is the appraisal vector in [<xref ref-type="bibr" rid="scirp.117511-ref38">38</xref>] representing “very poor”, “poor”, “medium”, “good”, “excellent”.</p><p>We now calculate the consistency of our result. That is</p><p>A W = ( 1 1 / 2 2 2 1 3 1 / 2 1 / 3 1 ) ( 0.2972 0.539 0.1641 ) = ( 0.8945 1.625 0.4922 ) . (23)</p><p>λ max = ∑ i = 1 3 ( A W ) i n w i = 0.8945 3 &#215; 0.2972 + 1.625 3 &#215; 0.539 + 0.4922 3 &#215; 0.1641 = 3.01 (24)</p><p>C I = 3.01 − 3 3 − 1 = 0.05. (25)</p></sec><sec id="s5"><title>5. Conclusions</title><p>This paper designs a pedagogical game of mathematical proof to help non-STEM mathematics educators to produce pedagogically effective active learning exercises within the constructivist paradigm using toys and without tedious calculations supporting anxious students in deriving mathematical proofs playfully. The purpose of this learning activity is to help students improve self-efficacy and reduce mathematics anxiety.</p><p>An in-class experiment shows that our pedagogy based on PGMPs satisfies a pedagogical quality criterion. We derive this conclusion from our Pedagogy Quality Assurance Evaluation Method, which we introduce using fuzzy variables. Our model uses Learning Effectiveness, Learning Efficiency, and Learning Satisfaction as inputs to derive a weighted measure of pedagogical quality. The advantage of this measure of learning activity evaluation is that it relies on data from both, students and experts.</p><p>The data from the experiment show that students were effective in playfully producing mathematical proofs using a hands-on approach using toys demanding them to physically rotate and flip geometric objects, and write up their observations in a table. The table, together with a set of sequential instructions effectively guided students through the reasoning required to produce a complete proof. The success was not only reflected in high performance attainment but also in the survey responses.</p><p>To improve on the robustness of our main result, future work should consider different metric functions such as trapezoids, bell-shaped, and sigmoid functions to specifically take into account other pedagogical variables. An empirical experiment with a large group of mathematics education experts would provide more accurate metrics.</p><p>Obtaining a more specific measure of our learning activity of reducing mathematics anxiety would be a valuable result for many educators. A robust measure could be obtained from a field-experiment with a much larger subject pool incorporating established instruments for measuring mathematics anxiety [<xref ref-type="bibr" rid="scirp.117511-ref23">23</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref24">24</xref>] [<xref ref-type="bibr" rid="scirp.117511-ref33">33</xref>] into our model.</p></sec><sec id="s6"><title>Acknowledgement</title><p>This work was supported by the Research-Inspired Inquiry-Led (RIIL) projects fund, as part of the Exeter Education Incubator, and by the NUBS School Research Fund.</p></sec><sec id="s7"><title>Conflicts of Interest</title><p>The author declares no conflicts of interest regarding the publication of this paper.</p></sec><sec id="s8"><title>Cite this paper</title><p>Stiefenhofer, P. (2022) Evaluating Pedagogical Quality of Learning Activities Using Fuzzy Evaluation Mappings: The Case of Pedagogical Games of Mathematical Proof. Applied Mathematics, 13, 432-452. https://doi.org/10.4236/am.2022.135029</p></sec><sec id="s9"><title>Appendix</title><p>Definition 1. A symmetry is any transformation of Δ which leaves Δ in an equivalent state (−5 pts)</p><p>Theorem 1 ϕ 1 : There are 6 symmetries.</p><p>Task 1. Label the edges of the triangle Δ with the capitals A, B, and C in counterclockwise direction. Similarly, label the edges of the corresponding hole Δ in the box □ so that A Δ matches A □ , B Δ matches B □ , and C Δ matches C □ . [20 pts]</p><p>Task 2. Check that if starting at position A Δ matches A □ (AA) a rotation Θ moves to the new position A Δ matches B □ (AB). We express this compactly as r ( A ) A B . Check that if starting at position A Δ matches A □ (AA) with fixed (AA) a flip Ψ leaves A Δ matches A □ unchanged, but moves B Δ on B □ (BB) to B Δ on C □ (BC) and C Δ on C □ (CC) to C Δ on B □ (CB). We use the notation f A to express this property. [15 pts]</p><p>Task 3. Construct a table to show all symmetries of r ( A ) A A , r ( A ) A B and r ( A ) A C only using rotations Θ and its compositions ∘ . [15 pts]</p><p>Task 4. Fix the edge (AA) and use a flip Ψ in isolation and in combination with rotations Θ to obtain the symmetries of f A . Follow this procedure to obtain the symmetries associated with f B and f C and collect them in a table. [15 pts]</p><p>Task 5. Collect all symmetries from above tables to define the set G . [15pts]</p><p>Task 6. Use the results of tasks 1 - 5 in given order to formally write up the proof of theorem 1. [20 pts]</p><p>The expected result from task 6 is summarized in the proof bellow.</p><p>Proof. Label the vertices of Δ counterclockwise starting from the top by A, B and C. Similarly on □ label the initial vertices of Δ counterclockwise starting from the top by A, B and C, such that the labeling for Δ matches the labeling on the □ , i.e., AA, BB and CC.</p><p>We introduce two operations; a counterclockwise rotation around the center of Δ and a flip of Δ at a fixed vortex (A, B, orC). Let Θ denote the counterclockwise rotation of Δ from i.e. initial A to final B such that A on Δ moves from position A to position B on the □ . This is compactly expressed as r ( A ) A B . Let Ψ be a flip with fixed vortex A such that initial B moves to final C and initial C moves to final B while A remains unchanged. We compactly express this as f A .</p><p>The composition of two operations is denoted by ∘ . Then by definition of symmetry we obtain:</p><p>r ( A ) A A = Θ ∘ Θ ∘ Θ = Θ 3 = n 0 , noaction</p><p>r ( A ) A B = Θ</p><p>r ( A ) A C = r ( A ) A B ∘ r ( A ) B C = Θ 2 .</p><p>These are the symmetries obtained from only applying rotation to Δ . Notice that there are two ways to achieve r ( A ) A A ; i) apply rotation three times, or ii) do not take any action n 0 at all. Let’s generate the symmetries associated with flipping.</p><p>f A = Ψ</p><p>f B = r ( A ) A B ∘ r ( A ) B C ∘ f A = Θ 2 ∘ Ψ</p><p>f C = r ( A ) A B ∘ f A = Θ ∘ Ψ .</p><p>We collect all symmetries in a set G = { n 0 , Θ , Θ 2 , Ψ , Θ 2 Ψ , Θ Ψ } .</p><p>Definition 2. A Latin square is an n &#215; n array filled with n different symbols, each occurring exactly once in each row and exactly once in each column. (−5 pts)</p><p>Definition 3. A group is a set G with a binary operation ∘ (−2 pts each),</p><p>∘ : G &#215; G → G .</p><p>for all a , b , c ∈ G satisfying:</p><p>1) Closure: a ∘ b ∈ G .</p><p>2) Associativity: a ∘ ( b ∘ c ) = ( a ∘ b ) ∘ c .</p><p>3) Identity element: There exists and element e ∈ G such that a ∘ e = a = e ∘ a .</p><p>4) Inverse: For every a ∈ G there exists some b ∈ G such that a ∘ b = e and b ∘ a = e .</p><p>Definition 4. A Cayley table describes the structure of a finite group by arranging all the possible products of all the group’s elements in a square table (−5 pts).</p><p>Theorem 2. ϕ 2 : Δ ,   □ , Θ , and Ψ , form a group ( G , ∘ ) .</p><p>1) Construct a table of symmetries (Cayley table) and show that it is a Latin square. [20 pts]</p><p>2) Using the definition of a group show the closure property (use the Cayley table). [15 pts]</p><p>3) Using the definition of a group demonstrate the associativity condition (use the Cayley table). [15 pts]</p><p>4) Using the definition of a group find the identity element (use the Cayley table). [15 pts]</p><p>5) Using the definition of a group show that each element has an identity (use the Caleyly table). [15 pts]</p><p>6) Write up a formal proof using the results of tasks 2 - 5. [20 pts]</p><p>Proof. To show the closure property we construct a Cayley table for ( G , ∘ ) . From TableA3 one can also see that it defines a Latin square. The closure property follows from the Latin square property. Associativity follows from the fact that the composition of bijections from a set to itself is associative. By inspection of the Cayley table we have:</p><p>n 0 ∘ n 0 = n 0 = n 0 ∘ n 0</p><p>n 0 ∘ Θ = Θ = Θ ∘ n 0</p><p>n 0 ∘ Θ 2 = Θ 2 = Θ 2 ∘ n 0</p><table-wrap id="table8" ><label><xref ref-type="table" rid="table">Table </xref>A1</label><caption><title> Task success theorem I</title></caption><table><tbody><thead><tr><th align="center" valign="middle"  rowspan="2"  >Subject</th><th align="center" valign="middle"  colspan="5"  >Theorem I</th></tr></thead><tr><td align="center" valign="middle" >very poor</td><td align="center" valign="middle" >poor</td><td align="center" valign="middle" >indifferent</td><td align="center" valign="middle" >good</td><td align="center" valign="middle" >very good</td></tr><tr><td align="center" valign="middle" >1</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.2918</td><td align="center" valign="middle" >0.7083</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >2</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.1668</td><td align="center" valign="middle" >0.8666</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >3</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.125</td><td align="center" valign="middle" >0.875</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >4</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.0833</td><td align="center" valign="middle" >0.9168</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >5</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.3333</td><td align="center" valign="middle" >0.6668</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >6</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.1668</td><td align="center" valign="middle" >0.8666</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >7</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >1</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >8</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.125</td><td align="center" valign="middle" >0.875</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >9</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.125</td><td align="center" valign="middle" >0.875</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >10</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.333</td><td align="center" valign="middle" >0.6668</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >sum</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >1.7503</td><td align="center" valign="middle" >8.3169</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >normalized sum</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0</td><td align="center" valign="middle" >0.173862</td><td align="center" valign="middle" >0.826138</td><td align="center" valign="middle" >0</td></tr></tbody></table></table-wrap><table-wrap id="table9" ><label><xref ref-type="table" rid="table">Table </xref>A2</label><caption><title> Task time evaluation theorem I</title></caption><table><tbody><thead><tr><th align="center" valign="middle"  colspan="2"   rowspan="2"  >Subject</th><th align="center" valign="middle"  colspan="8"  >Theorem I</th></tr></thead><tr><td align="center" valign="middle"  colspan="2"  >very poor</td><td align="center" valign="middle"  colspan="2"  >poor</td><td align="center" valign="middle"  colspan="2"  >indifferent</td><td align="center" valign="middle" >good</td><td align="center" valign="middle" >very good</td></tr><tr><td align="center" valign="middle"  colspan="2"  >1</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0.1785</td><td align="center" valign="middle" >0.8215</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle"  colspan="2"  >2</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0.1072</td><td align="center" valign="middle" >0.8928</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle"  colspan="2"  >3</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0.4572</td><td align="center" valign="middle" >0.6785</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle"  colspan="2"  >4</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0.0358</td><td align="center" valign="middle" >0.9643</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle"  colspan="2"  >5</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle" >1</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle"  colspan="2"  >6</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle" >0.86</td><td align="center" valign="middle" >0.14</td></tr><tr><td align="center" valign="middle"  colspan="2"  >7</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0.0715</td><td align="center" valign="middle" >0.9285</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle"  colspan="2"  >8</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle" >0.86</td><td align="center" valign="middle" >0.14</td></tr><tr><td align="center" valign="middle" >9</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0.2143</td><td align="center" valign="middle"  colspan="2"  >0.7858</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >10</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0.1428</td><td align="center" valign="middle"  colspan="2"  >0.8572</td><td align="center" valign="middle" >0</td></tr><tr><td align="center" valign="middle" >sum</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >1.2073</td><td align="center" valign="middle"  colspan="2"  >8.6486</td><td align="center" valign="middle" >0.28</td></tr><tr><td align="center" valign="middle" >normalized sum</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0</td><td align="center" valign="middle"  colspan="2"  >0.119111</td><td align="center" valign="middle"  colspan="2"  >0.853264</td><td align="center" valign="middle" >0.027625</td></tr><tr><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td><td align="center" valign="middle" ></td></tr></tbody></table></table-wrap><table-wrap id="table10" ><label><xref ref-type="table" rid="table">Table </xref>A3</label><caption><title> Cayley table for the six symmetries of Δ </title></caption><table><tbody><thead><tr><th align="center" valign="middle" >∘</th><th align="center" valign="middle" >n 0</th><th align="center" valign="middle" >Θ</th><th align="center" valign="middle" >Θ 2</th><th align="center" valign="middle" >Ψ</th><th align="center" valign="middle" >Θ 2 Ψ</th><th align="center" valign="middle" >Θ Ψ</th></tr></thead><tr><td align="center" valign="middle" >n 0</td><td align="center" valign="middle" >n 0</td><td align="center" valign="middle" >Θ</td><td align="center" valign="middle" >Θ 2</td><td align="center" valign="middle" >Ψ</td><td align="center" valign="middle" >Θ 2 f</td><td align="center" valign="middle" >Θ Ψ</td></tr><tr><td align="center" valign="middle" >Θ</td><td align="center" valign="middle" >Θ</td><td align="center" valign="middle" >Θ 2</td><td align="center" valign="middle" >n 0</td><td align="center" valign="middle" >Θ Ψ</td><td align="center" valign="middle" >Ψ</td><td align="center" valign="middle" >Θ 2 f</td></tr><tr><td align="center" valign="middle" >Θ 2</td><td align="center" valign="middle" >Θ 2</td><td align="center" valign="middle" >n 0</td><td align="center" valign="middle" >Θ</td><td align="center" valign="middle" >Θ 2 Ψ</td><td align="center" valign="middle" >Θ Ψ</td><td align="center" valign="middle" >Ψ</td></tr><tr><td align="center" valign="middle" >Ψ</td><td align="center" valign="middle" >Ψ</td><td align="center" valign="middle" >Θ 2 Ψ</td><td align="center" valign="middle" >Θ Ψ</td><td align="center" valign="middle" >n 0</td><td align="center" valign="middle" >Θ</td><td align="center" valign="middle" >Ψ</td></tr><tr><td align="center" valign="middle" >Θ 2 Ψ</td><td align="center" valign="middle" >Θ 2 f</td><td align="center" valign="middle" >Θ Ψ</td><td align="center" valign="middle" >Ψ</td><td align="center" valign="middle" >Θ 2</td><td align="center" valign="middle" >n 0</td><td align="center" valign="middle" >Θ 2</td></tr><tr><td align="center" valign="middle" >Θ Ψ</td><td align="center" valign="middle" >Θ Ψ</td><td align="center" valign="middle" >Ψ</td><td align="center" valign="middle" >Θ 2 Ψ</td><td align="center" valign="middle" >Θ</td><td align="center" valign="middle" >Θ 2</td><td align="center" valign="middle" >n 0</td></tr></tbody></table></table-wrap><p>n 0 ∘ Ψ = Ψ = Ψ ∘ n 0</p><p>n 0 ∘ Θ 2 Ψ = Θ 2 Ψ = Θ 2 Ψ ∘ n 0</p><p>n 0 ∘ Θ Ψ = Θ Ψ = Θ Ψ ∘ n 0 .</p><p>It follows that n 0 ∈ G is the identity element since for any g ∈ G n 0 ∘ g = g ∘ n 0 = g . Each of our symmetries g has an inverse g − 1 such that g ∘ g − 1 = g − 1 ∘ g = n 0 . We have</p><p>n 0 , with inverse n 0 since n 0 ∘ n 0 = n 0</p><p>Θ , with inverse Θ since Θ ∘ Θ 2 = Θ 2 ∘ Θ = n 0</p><p>Θ 2 , with inverse Θ 2 since Θ 2 ∘ Θ = Θ ∘ Θ 2 = n 0</p><p>Ψ , with inverse Ψ since Ψ ∘ Ψ = n 0</p><p>Θ 2 Ψ , with inverse Θ 2 Ψ since Θ 2 Ψ ∘ Θ 2 Ψ = n 0</p><p>Θ Ψ , with inverse Θ Ψ since Θ Ψ ∘ Θ Ψ = n 0 . &#168;</p></sec><sec id="s10"><title>NOTES</title></sec></body><back><ref-list><title>References</title><ref id="scirp.117511-ref1"><label>1</label><mixed-citation publication-type="other" xlink:type="simple">Bandura, A. (1977) Self-Efficacy: Toward a Unifying Theory of Behavioral Change. Psychological Review, 84, 191-215. https://doi.org/10.1037/0033-295X.84.2.191</mixed-citation></ref><ref id="scirp.117511-ref2"><label>2</label><mixed-citation publication-type="other" xlink:type="simple">Bandura, A. (1986) Social Foundations of Thought and Action: A Social Cognitive Theory. Prentice-Hall, Englewood Cliffs.</mixed-citation></ref><ref id="scirp.117511-ref3"><label>3</label><mixed-citation publication-type="other" xlink:type="simple">Bandura, A. (1997) Self-Efficacy: The Exercise of Control. W.H. Freeman, New York.</mixed-citation></ref><ref id="scirp.117511-ref4"><label>4</label><mixed-citation publication-type="other" xlink:type="simple">Pajares, F. (2021) Self-Efficacy Beliefs in Academic Contexts: An Outline. http://des.emory.edu/mfp/efftalk.html</mixed-citation></ref><ref id="scirp.117511-ref5"><label>5</label><mixed-citation publication-type="other" xlink:type="simple">Carey, M.P. and Forsyth, A.D. (2021) Teaching Tip Sheet: Self-Efficacy. https://www.apa.org/pi/aids/resources/education/self-efficacy</mixed-citation></ref><ref id="scirp.117511-ref6"><label>6</label><mixed-citation publication-type="other" xlink:type="simple">Ashcract, M.H. (2002) Math Anxiety: Personal, Educational, and Cognitive Consequences. Current Directions in Psychological Science, 11, 181-185. https://doi.org/10.1111/1467-8721.00196</mixed-citation></ref><ref id="scirp.117511-ref7"><label>7</label><mixed-citation publication-type="other" xlink:type="simple">Richardson, F.C. and Suinn, R.M. (1972) The Mathematics Anxiety Rating Scale: Psychometric Data. Journal of Counseling Psychology, 19, 551-554. https://doi.org/10.1037/h0033456</mixed-citation></ref><ref id="scirp.117511-ref8"><label>8</label><mixed-citation publication-type="other" xlink:type="simple">Tahamassian, K. and Moghadam, N.J. (2011) Relationship between Self-Efficacy and Symptoms of Anxiety, Depression, Worry and Social Avoidance in a Normal Sample of Students. Iranian Journal of Psychiatry and Behavioral Sciences, 5, 91-98.</mixed-citation></ref><ref id="scirp.117511-ref9"><label>9</label><mixed-citation publication-type="other" xlink:type="simple">Unlu, M., Ertekin, E. and Dilmae, B. (2017) Predicting Relationships between Mathematics Anxiety, Mathematics Teaching Anxiety, Self-Efficacy Beliefs towards Mathematics and Mathematics Teaching. International Journal of Research in Education and Science, 3, 636-645.</mixed-citation></ref><ref id="scirp.117511-ref10"><label>10</label><mixed-citation publication-type="other" xlink:type="simple">Hackett, G. and Betz, N.E. (1989) An Exploration of the Mathematics Self-Efficacy/Mathematics Performance Correspondence. Journal for Research in Mathematics Education, 20, 261-273. https://doi.org/10.2307/749515</mixed-citation></ref><ref id="scirp.117511-ref11"><label>11</label><mixed-citation publication-type="other" xlink:type="simple">Effandi, Z. and Norazahn, M.N. (2008) The Effects of Mathematics Anxiety on Matriculation Students as Related to Motivation and Achievement. Eurasia Journal of Mathematics, Science &amp; Technology Education, 4, 27-30. https://doi.org/10.12973/ejmste/75303</mixed-citation></ref><ref id="scirp.117511-ref12"><label>12</label><mixed-citation publication-type="other" xlink:type="simple">Luo, X., Wang, F. and Luo, Z. (2009) Investigation and Analysis of Mathematics Anxiety in Middle School Students. Journal of Mathematics Education, 2, 12-19.</mixed-citation></ref><ref id="scirp.117511-ref13"><label>13</label><mixed-citation publication-type="other" xlink:type="simple">Lim, E. and Seng, K. (2015) The Influence of Pre-University Student’s Mathematics Test Anxiety and Numerical Anxiety on Mathematics Achievement. International Education Studies, 8, 162-168. https://doi.org/10.5539/ies.v8n11p162</mixed-citation></ref><ref id="scirp.117511-ref14"><label>14</label><mixed-citation publication-type="other" xlink:type="simple">Alves, M., Rodrigues, C.S., Rocha, A.M. and Coutinho, C. (2016) Self-Efficacy, Mathematics—Anxiety and Perceived Importance: An Empirical Study with Portuguese Engineering Students. European Journal of Engineering Education, 41, 105-121. https://doi.org/10.1080/03043797.2015.1095159</mixed-citation></ref><ref id="scirp.117511-ref15"><label>15</label><mixed-citation publication-type="other" xlink:type="simple">Ma, X. (1999) A Meta-Analysis of the Relationship between Anxiety toward Mathematics and Achievement in Mathematics. Journal for Research in Mathematics Education, 30, 520-540. https://doi.org/10.2307/749772</mixed-citation></ref><ref id="scirp.117511-ref16"><label>16</label><mixed-citation publication-type="other" xlink:type="simple">Mavridis, A., Katmada, A. and Tsiatsos, T. (2017) Impact of Online Flexible Games on Students’ Attitude towards Mathematics. Educational Technology Research and Development, 65, 1451-1470. https://doi.org/10.1007/s11423-017-9522-5</mixed-citation></ref><ref id="scirp.117511-ref17"><label>17</label><mixed-citation publication-type="other" xlink:type="simple">Rounds Jr., J.B. and Hendel, D.D. (2018) Mathematics Anxiety and Attitudes toward Mathematics. Measurement and Evaluation in Guidance, 13, 83-89. https://doi.org/10.1080/00256307.1980.12022221</mixed-citation></ref><ref id="scirp.117511-ref18"><label>18</label><mixed-citation publication-type="other" xlink:type="simple">Ku, O., Chen, S.Y., Wu, D.H., Lao, A.C. and Chan, T.W. (2018) The Effects of Game-Based Learning on Mathematical Confidence and Performance: High Ability vs. Low Ability. Journal of Educational Technology &amp; Society, 17, 65-78.</mixed-citation></ref><ref id="scirp.117511-ref19"><label>19</label><mixed-citation publication-type="other" xlink:type="simple">Shroff, R., Ting, F. and Lam, W. (2019) Development and Validation of an Instrument to Measure Student’s Perceptions of Technology-Enabled Active Learning. Australian Journal of Educational Technology, 35, 109-127. https://doi.org/10.14742/ajet.4472</mixed-citation></ref><ref id="scirp.117511-ref20"><label>20</label><mixed-citation publication-type="other" xlink:type="simple">Freeman, S., Eddy, S., McDonough, M., Smith, M., Okoroafor, N., Jordt, H. and Wenderoth, M. (2014) Active Learning Increases Student Performance in Science, Engineering, and Mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111, 8410-8415. https://doi.org/10.1073/pnas.1319030111</mixed-citation></ref><ref id="scirp.117511-ref21"><label>21</label><mixed-citation publication-type="other" xlink:type="simple">Laursen, S.L., Hassi, M.L., Kogan, M.M., and Weston, T.J. (2014) Benefits for Women and Men of Inquiry-Based Learning in College Mathematics: A Multi-Institutional Study. Journal for Research in Mathematics Education, 45, 406-418. https://doi.org/10.5951/jresematheduc.45.4.0406</mixed-citation></ref><ref id="scirp.117511-ref22"><label>22</label><mixed-citation publication-type="other" xlink:type="simple">Kim, K., Sharma, P., Land, S.M. and Furlong, K.P. (2013) Effects of Active Learning on Enhancing Student Critical Thinking in an Undergraduate General Science Course. Innovative Higher Education, 38, 223-235. https://doi.org/10.1007/s10755-012-9236-x</mixed-citation></ref><ref id="scirp.117511-ref23"><label>23</label><mixed-citation publication-type="book" xlink:type="simple">Marton, F. (2018) Towards a Pedagogical Theory of Learning. In: Matsushita, K., Ed., Deep Active Learning, Springer, Singapore, 59-77. https://doi.org/10.1007/978-981-10-5660-4_4</mixed-citation></ref><ref id="scirp.117511-ref24"><label>24</label><mixed-citation publication-type="other" xlink:type="simple">Kerrigan, J. (2018) Active Learning Strategies for the Mathematics Classroom. College Teaching, 66, 35-36. https://doi.org/10.1080/87567555.2017.1399335</mixed-citation></ref><ref id="scirp.117511-ref25"><label>25</label><mixed-citation publication-type="other" xlink:type="simple">Sung, H.Y. and Hwang, G.J. (2018) Facilitating Effective Digital Game-Based Learning Behaviors and Learning Performances of Students Based on a Collaborative Knowledge Construction Strategy. Interactive Learning Environments, 26, 118-134. https://doi.org/10.1080/10494820.2017.1283334</mixed-citation></ref><ref id="scirp.117511-ref26"><label>26</label><mixed-citation publication-type="other" xlink:type="simple">Hwang, G.J., Wu, P.-H. and Chen, C.C. (2012) An Online Game Approach for Improving Students—Learning Performance in Web-Based Problem-Solving Activities. Computers &amp; Education, 59, 1246-1256. https://doi.org/10.1016/j.compedu.2012.05.009</mixed-citation></ref><ref id="scirp.117511-ref27"><label>27</label><mixed-citation publication-type="other" xlink:type="simple">Thou Ting, F.S., Lam, W.H. and Shroff, R.H. (2019) Active Learning via Problem-Based Collaborative Games in a Large Mathematics University Course in Hong Kong. Education Sciences, 9, Article No. 172. https://doi.org/10.3390/educsci9030172</mixed-citation></ref><ref id="scirp.117511-ref28"><label>28</label><mixed-citation publication-type="other" xlink:type="simple">Yang, J.C., Chien, K.H. and Liu, T.C. (2012) A Digital Game-Based Learning System for Energy Education: An Energy Conservation Pet. Turkish Online Journal of Educational Technology, 11, 27-37.</mixed-citation></ref><ref id="scirp.117511-ref29"><label>29</label><mixed-citation publication-type="other" xlink:type="simple">Naik, N. (2017) The Use of GBL to Teach Mathematics in Higher Education. Innovations in Education and Teaching International, 54, 238-246. https://doi.org/10.1080/14703297.2015.1108857</mixed-citation></ref><ref id="scirp.117511-ref30"><label>30</label><mixed-citation publication-type="other" xlink:type="simple">Gil-Doménech, D. and Berbegal-Mirabent, J. (2019) Stimulating Students-Engagement in Mathematics Courses in non-STEM Academic Programmes: A Game-Based Learning. Innovations in Education and Teaching International, 56, 57-65. https://doi.org/10.1080/14703297.2017.1330159</mixed-citation></ref><ref id="scirp.117511-ref31"><label>31</label><mixed-citation publication-type="other" xlink:type="simple">O-Neil, H.F., Wainess, R. and Baker, E.L. (2005) Classification of Learning Outcomes: Evidence from the Computer Games Literature. Curriculum Journal, 16, 455-474. https://doi.org/10.1080/09585170500384529</mixed-citation></ref><ref id="scirp.117511-ref32"><label>32</label><mixed-citation publication-type="other" xlink:type="simple">Perrotta, C., Featherstone, G., Aston, H. and Houghton, E. (2013) Game-Based Learning: Latest Evidence and Future Directions. NFER Research Programme: Innovation in Education, National Foundation for Educational Research, Slough.</mixed-citation></ref><ref id="scirp.117511-ref33"><label>33</label><mixed-citation publication-type="other" xlink:type="simple">Skilling, K., Bobis, J. and Martin, A.J. (2021) The “Ins and Outs” of Student Engagement in Mathematics: Shifts in Engagement Factors among High and Low Achievers. Mathematics Education Research Journal, 33, 469-493. https://doi.org/10.1007/s13394-020-00313-2</mixed-citation></ref><ref id="scirp.117511-ref34"><label>34</label><mixed-citation publication-type="other" xlink:type="simple">Mac, C., Fitzmaurice, O., Fhloinn, E.N. and O’Sullivan, C. (2013) Student Non-Engagement with Mathematics Learning Supports. Teaching Mathematics and Its Applications, 32, 191-205. https://doi.org/10.1093/teamat/hrt018</mixed-citation></ref><ref id="scirp.117511-ref35"><label>35</label><mixed-citation publication-type="other" xlink:type="simple">Zimmerman, H.J. (2011) Fuzzy Set Theory and Its Applications. Springer Science &amp; Business Media, Berlin, Heidelberg.</mixed-citation></ref><ref id="scirp.117511-ref36"><label>36</label><mixed-citation publication-type="other" xlink:type="simple">Saaty, T.L. (1977) A Scaling Method for Priorities in Hierarchical Structures. Journal of Mathematical Psychology, 15, 234-281. https://doi.org/10.1016/0022-2496(77)90033-5</mixed-citation></ref><ref id="scirp.117511-ref37"><label>37</label><mixed-citation publication-type="other" xlink:type="simple">Liu, S., Triantis, K.P. and Sarangi, S. (2010) Representing Qualitative Variables and Their Interactions with Fuzzy Logic in System Dynamics Modeling. Systems Research and Behavioral Science, 28, 245-263. https://doi.org/10.1002/sres.1064</mixed-citation></ref><ref id="scirp.117511-ref38"><label>38</label><mixed-citation publication-type="other" xlink:type="simple">Zhou, J.H., Wang, Z., Wang, Z.K. and Liu, X. (2004) Research on Software Quality Evaluation Based on Fuzzy Method. Systems Engineering and Electronics, 26, 988-991.</mixed-citation></ref><ref id="scirp.117511-ref39"><label>39</label><mixed-citation publication-type="other" xlink:type="simple">Ying-Feng, K. and Ling-Show, C. (2002) Using the Fuzzy Synthetic Decision Approach to Assess the Performance of University Teachers in Taiwan. International Journal of Management, 19, 593.</mixed-citation></ref><ref id="scirp.117511-ref40"><label>40</label><mixed-citation publication-type="other" xlink:type="simple">Lewis, J.R. (1995) IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use. International Journal of Human-Computer Interaction, 7, 57-58. https://doi.org/10.1080/10447319509526110</mixed-citation></ref><ref id="scirp.117511-ref41"><label>41</label><mixed-citation publication-type="other" xlink:type="simple">Saaty, T.L. (1990) How to Make a Decision: The Analytic Hierarchy Process. European Journal of Operations Research, 48, 9-26. https://doi.org/10.1016/0377-2217(90)90057-I</mixed-citation></ref></ref-list></back></article>