Factors Impacting K-12 Students’ Academic Achievement in Urban Areas During the Pandemic ()
1. Introduction
The COVID-19 pandemic hit the world in the middle of the 2019-20 school year, disrupting in-person education in traditional schools. Around 100,000 schools that were affected by the pandemic had to rapidly switch to remote learning around March 2020 [1]. The challenge of transitioning to online learning in K-12 settings was caused by the fact that most secondary schools are brick-and-mortar institutions that provide traditional means of in-person education. Despite the challenges of switching to remote learning in such a short timeline, educational administrators and teachers did their best to continue the educational processes and meet the basic needs of students, including attending to their nutritional and social-emotional health needs [1].
However, the National Center for Education Statistics [2] reported that 87% of public schools in the U.S. experienced negative impacts from the COVID-19 pandemic, including a delay in students’ socioemotional development. Therefore, it is important to understand K-12 school student engagement and disengagement in online learning by exploring how social presence and student engagement affect the academic achievement of K-12 students.
Even before the pandemic, there were K-12 online schools, also known as e-schools, which offered 100% distance education through online learning. According to a 2019 Snapshot study conducted by the Digital Learning Collaborative (DLC) [3], there were 310,000 students in 32 different states of the U.S. enrolled in fully online schools and there were 420,000 additional U.S. students from 23 states who took at least one online course from a state-supported virtual school. Still, the quick transitioning to remote or online learning nationwide raised some concerns about its quality.
Although the overall quality of online education during the COVID-19 pandemic had been of great concern [4], this concern existed even before the pandemic and it had been scrutinized since the early days of online schooling [5] [6]. Therefore, educators and researchers have been studying the characteristics and practices of online learning [3] [7] to find best practices that improve both teacher instruction and student learning [5] [8].
The pandemic is not over yet but in-person education in K-12 schools resumed at the beginning of the 2021-22 school year after one year of online education. Educators have already seen the impact of social isolation on students, such as social, emotional, behavioral, and academic challenges [9]. The need for social presence and student engagement for an effective online learning environment has become more evident during the pandemic [9] [10].
Although students’ positive social experiences in online classes contribute to their academic achievement, the factors that impact achievement have not been thoroughly examined in recent literature. A handful of studies have investigated the relationships between social presence and student engagement [11] [12], highlighting that both factors are critical for increasing student motivation and retention in online courses. Some scholars [13] [14] examined the effects of CoI constructs (presences) on academic achievement or final scores, reporting that social presence did not influence academic achievement or learning outcomes. Some other studies [15] reported that participation in the asynchronous online setting significantly improved students’ academic outcomes (final grades). Studies [16] [17] also reported that students who came from a low socioeconomic status were more negatively impacted by the pandemic and therefore performed lower academically than students from other socioeconomic groups.
Based on the review of recent and relevant literature, this study investigated relationships between social presence and student engagement, social presence and academic achievement, engagement and achievement, and demographics and student achievements. This study investigated the relationships of these factors because current literature has not thoroughly examined how these factors impact secondary students’ final grades.
To examine these factors, the community of inquiry (CoI) was chosen as a framework; It served as a theoretical lens in examining how the Edgenuity platform created presences through student-teacher and student-student interactions, the fostering student engagement, and effective learning experiences, which were manifested in students receiving higher grades.
2. Community of Inquiry Theoretical Framework
Virtual distant learners form online learning communities, and they communicate with each other synchronously or asynchronously to create collaborative distance education practices [18]. The complex nature of teaching and learning in online and blended learning environments calls for the need to explore and develop sound frameworks and working models in these environments [19]. Garrison et al. [20] developed the CoI theoretical framework. CoI is defined as “a group of individuals who collaboratively engage in purposeful critical discourse and reflection to construct personal meaning and confirm mutual understanding” ([18], p. 105). The key participants in a community of inquiry are students and teachers who can create worthwhile educational experiences in online learning [19]. CoI framework offers structured guidance to complex and dynamic learning communities by taking a collaborative constructivist approach to teaching and learning [18] [20]. Therefore, this framework “represents a process of creating a deep and meaningful (collaborative-constructivist) learning experience through the development of three interdependent elements: social, cognitive, and teaching presence” ([18], p. 106). Figure 1 presents the three elements and their interdependency.
The social presence component [20] in this framework refers to the degree to which participants present their personal characteristics to the community; in other words, the degree to which they can project their real selves. However, the definition of social presence has gone through revision over time. Garrison, in 2009 [21], redefined social presence as “the ability of participants to identify with the community (e.g., course of study), communicate purposefully in a trusting environment, and develop inter-personal relationships by way of protecting their
Figure 1. The community of inquiry [20].
individual personalities” (p. 352). This change stemmed from social presence being necessary not only for “establishing relationships and a sense of belonging” but also having to “support critical inquiry and the achievement of educational outcomes” ([18], p. 107). There are three categories of social presence in CoI: Personal/affective expression, open communication, and group cohesion [20] [21]. Affective expressions help establish the initial conditions of a community of inquiry and create a foundation for open communication [18]. Open communication allows critical reflection when responding to the contributions of others [18]. Affective expression and open communication contribute to the last category of social presence, known as group cohesion [18] [21]. When the community is cohesive, the quality of learning is optimal because of meaningful and collaborative exchanges [18].
Cognitive presence is the degree to which participants construct new knowledge, or meaning, out of communications [20]. Cognitive presence is at the heart of the CoI framework [21], and it is the most basic component for successful learning [20]. Nevertheless, it is the most challenging aspect to develop [22]. Garrison et al. introduces the Practical Inquiry model (Figure 2) that explains how cognitive presence can happen in a community of inquiry. There are four phases in the cycle. First, a student recognizes the issue. Second, that student gathers information related to the issue through research and interaction. Third, the student establishes a sense of gathered information and reaches a resolution. Fourth, the student tests the resolution [21].
Teaching presence is “the design, facilitation, and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes” ([23], p. 5). Teaching presence is a crucial
Figure 2. The practical inquiry model [20].
element that integrates both social and cognitive presence in order to ensure a functioning community of learners [18] [23]. There are three categories that refer to the functions of teaching presence: “Design and organization; facilitating discourse; and direct instruction” ([18], p. 111). These functions may be performed by any participant in a community of inquiry, but they are primarily considered to be the roles of a teacher [19] [20].
3. Community of Inquiry and K-12 Education
It was important to understand the theoretical framework of Community of Inquiry in order to research solutions for the online learning needs of K-12 schools in the entire nation. CoI theoretical framework was originally designed for online education in higher education settings [20]. Researchers have studied its implications in the past two decades for higher education [5] [7] [8] [18] [19] but more exploration and descriptive research is needed to apply the CoI framework to the K-12 environments [24].
Akyol et al.’s study [19] found that different contexts cause significant differences on the development of CoI framework presences (social, teaching, and cognitive). This is because “younger students tend to have less internal locus of control, fewer meta-cognitive skills, and lower self-regulation abilities, making quality interactions in virtual schools more crucial than they are with adult learners” ([25], p. 155).
Several scholars [7] [8] examined the implications of CoI in regard to higher education but more exploration and descriptive research has been needed to apply the CoI framework to K-12 environments. One main reason is that K-12 distance education is different from higher education [26]. Recent studies show that the Community of Inquiry framework is applicable to K-12 environments [7] [27]. Wei et al. [27] state that CoI presences in K-12 differ slightly from higher education. For example, K-12 students are more used to face-to-face interaction and the physical presence of other students in the same classroom.
CoI has a great potential to be a powerful framework for online learning in K-12 settings. Understanding the relationships between social presence and student engagement and their combined effects on student achievement is important because it helps to develop an understanding of student learning outcomes during the pandemic when online learning has become a necessity rather than a choice.
4. Literature Review
Since student engagement is crucial in online settings, it is important to look at studies that examine these concepts. Student engagement is central to student learning as it contributes to the improvement of student performance in online learning [28]. Kuh [29] defines student engagement as “the time and energy students devote to educationally sound activities” (p. 25). The term “student engagement” is commonly used in education settings; therefore, its meaning initially seems to be obvious. There has been disagreement, however, about what is actually included within the definition of this term [30] [31].
This study has utilized a more comprehensive definition provided by Fredricks et al. [32] that includes three dimensions of engagement: behavioral, emotional, and cognitive engagement. Behavioral engagement refers to observable signs such as participation in activities, attendance, following class procedures, and meeting instructor’s expectations. Emotional engagement refers to a student’s feelings towards their learning experience, such as interest, excitement, and willingness, as well as their social connections with their peers and teachers. Cognitive engagement refers to a student’s effort and investment in learning and comprehending complex ideas in order to master difficult skills [32].
Factors Impacting Student Achievement in Online Learning
Several studies discussed some factors that might impact academic achievement of K-12 and college students in online classes during pandemic, such as engagement and social presence. Borup et al. [25] conducted research at the Open High School of Utah (OHSU), examining two virtual high school courses. Survey responses showed that students spent a significant amount of time on interactions with course content, their instructors, and their classmates. The significant finding from Borup et al. [25] was that students’ time spent on learner-learner interaction and students’ social learner-learner interaction was significantly correlated with their grade, implying that social presence had a positive impact on final grades. Collins et al. [11] reported that communication in asynchronous classes could increase the instructor social presence and improve student engagement because it creates a sense of community.
Khammat Al-iessa et al. [33] examined the components of the CoI (e.g., social presence) in the context of an online setting that included over 300 English as a foreign language students’ online engagement in online classes. The researchers reported that there were significant relationships between the respondents’ online engagement and their perceptions of the existence of the components of CoI. However, compared to other components of CoI, social presence showed the strongest relationship, showing that it contributed more to the changes in learners’ engagement in online classes. Harrell and Wendt [34] also reported a significant difference in social presence between online students and blended students while no significant difference existed in cognitive presence and teaching presence. For instance, Shores et al. [35] examined the Black-White gaps for multiple educational outcomes that exist in some U.S. schools, drawing upon a sample that consisted of several U.S. public school districts that had 71% of the Black public-school population. Based on their findings, this study reported differences in socioeconomic status, racial and socioeconomic segregation, and racial composition variables that explained 1.65 to 6.9 times more variation for test score gaps between Black and White students. The authors made recommendations for school districts to improve the gap by considering both in-school (e.g., integrating SEL component into teaching practice) and out-of-school factors (e.g., factors related to race-based differences, such as lack of engagement and access to housing and health facilities, etc.).
5. Methods
5.1. Research Model and Procedure
The purpose of this study was to examine the relationships between social presence, engagement, and achievement of high school students in asynchronous online classes in urban charter schools during the COVID-19 pandemic. This study used quantitative research methods, an approach that helps explain the relationships that exist among variables [36]. Quantitative research is called nonexperimental when a researcher does not manipulate the conditions experienced by the participants [36]. Moreover, nonexperimental quantitative research is correlational (predictive) when it intends to describe the impact of the predictor variables on the outcome variables [36]. Therefore, it was best to use a nonexperimental correlational research design for this quantitative study.
The dependent variable was the final grade that was used for measuring actual student achievement. Independent variables were student perceptions of engagement and social presence as well as demographics such as gender, race, socioeconomic status, ESL (English as a second language) status, and SPED (special education) status. Race was recategorized as a dummy variable as “Blacks and Other” because of the nonhomogenous distribution among race categories.
5.2. Research Questions
The following research questions guided this study:
RQ1: Is there any relationship between high school students’ perceived social presence and perceived engagement in asynchronous online classes?
RQ2: Is there any relationship between high school students’ perceived social presence and student achievement (actual learning outcome) in asynchronous online classes?
RQ3: Is there any relationship between high school students’ perceived engagement and student achievement (actual learning outcome) in asynchronous online classes?
RQ4: Is there any relationship between demographics (gender, race, socioeconomic status, SPED & ESL status) and student achievement in asynchronous online classes?
5.3. Data Procedures
CSN collected perceptional data from its students in April and May of 2022 through its internal student surveys. Demographic data and final grades were combined by CSN’s assessment office when the school year ended in June 2022. Researchers obtained de-identified data for the purposes of this study. Ethics approval had been obtained from the researchers’ affiliated institution and an IRB-exemption was granted for this study.
Demographics and Data Measures
CSN houses student demographic records in its student information system (SIS). All demographic data was included in the secondary data obtained from CSN. Schools were named as School-1 through School-6. Grade levels were marked as nine through 12. Subjects were classified as English, Math, Science, Social Studies, Foreign Language, Health, and Other Subjects. Gender at birth was classified as male or female in secondary data. Race and ethnicity measures were Asian, Black, Hispanic, White, and other. In statistical analyses race was reclassified as “Black and Other” because homogeneity among races was not feasible for analysis purposes. Therefore, it was reclassified as a dummy variable, namely Raceblack. Socioeconomic status was classified as low-SES and high-SES. Students were identified as low-SES if they qualified for free or reduced school lunch per the federal school meals program and were identified as high-SES if they did not qualify for free or reduced school lunch or if they had to pay for school meals per federal income guidelines. An English as a second language (ESL) identifier was marked as “yes” or “no” based upon eligibility for ESL services. Similarly, a special education (SPED) identifier was marked as “yes” or “no” based upon eligibility for SPED services per federal guidelines.
5.4. Research Context and Sample
Setting
This study was conducted by utilizing secondary data provided by a nonprofit charter school network (CSN) that serves under-resourced communities in urban areas located in six different states in the Midwestern United States. All CSN schools are brick-and-mortar schools that house diverse populations. In the 2020-21 school year, CSN implemented remote learning due to the COVID-19 pandemic. Even though the worldwide pandemic was not over, most schools in the U.S. resumed in-person learning at the beginning of the 2021-22 school year. Following the same practice, CSN’s 13,000 students returned to in-person learning in August/September of 2021 and physically attended their traditional brick-and-mortar schools daily throughout the 2021-22 school year.
CSN serves more than 5000 students enrolled in high school (grades 9 to 12). While CSN offers a traditional education model in physical classrooms, it also provides its students with online course offerings for various reasons, including credit recovery, expanding elective course options, and maintaining mandatory course offerings in the absence of certified teachers. During the 2021-22 school year, 1800 of the high school students were enrolled in one or more online courses. CSN partners with Edgenuity to offer online courses that are asynchronous and self-paced. Edgenuity is one of the country’s major online education providers and serves more than four million students in 20,000 schools nationwide, including 20 of the 25 largest school districts [37]. Students were assigned to certain class periods and a physical classroom during the day to complete their online courses. These courses had pre-populated learning material. Students interacted with online course teachers through messaging and emailing.
Sample
Data for this study comes from 395 students in six schools (out of 1,800) who responded to a voluntary online survey conducted in Spring 2022 by CSN to learn more about students’ online learning experiences. As shown in Table 1, among 395 high school students, 124 (31.4%) were in 12th grade, 112 (28.4%) were in ninth grade, 83 (21%) were in 11th grade, and 76 (19.2%) were in 10th grade. In terms of gender (at birth), 57.7% were female and 42.3% were male. In terms of race, most students came from minority backgrounds, with the majority being Black or African American (70.9%), followed by Hispanic (18.7%), Asian (5.6%), White (4.1%), and other races or ethnicities (0.8%). In terms of socioeconomic status (SES), 93.2% of the students were identified as low-SES per federal free-reduced meals program eligibility while 6.8% were identified as high-SES, meaning they had to pay for school meals per federal income guidelines. Regarding the eligibility for various educational services, 78 students (19.7%) were eligible for English as a second language (ESL) services and 34 students (8.6%) were eligible for federal special education (SPED) services. The online subjects that students enrolled in were: 106 students (26.8%) in Math, 68 students (17.2%) in Foreign Language, 63 students (15.9%) in Health, 54 students (13.7%) in Social Studies, 42 students (10.6%) in science, 30 students (7.6%) in English, and 32 students (8.1%) in other subjects.
Social Presence Data
Perceptional social presence data in secondary data came from CSN’s student surveys conducted in Spring 2022. All survey items for social presence were
Table 1. Student descriptors.
|
No of Students |
Percent |
By school |
|
|
School 1 |
32 |
8.1 |
School 2 |
136 |
34.4 |
School 3 |
11 |
2.8 |
School 4 |
87 |
22.0 |
School 5 |
64 |
16.2 |
School 6 |
65 |
16.5 |
By grade level |
|
|
9 |
112 |
28.4 |
10 |
76 |
19.2 |
11 |
83 |
21.0 |
12 |
124 |
31.4 |
By course subjects |
|
|
1 English |
30 |
7.6 |
2 Math |
106 |
26.8 |
3 Science |
42 |
10.6 |
4 Social studies |
54 |
13.7 |
5 Foreign lang |
68 |
17.2 |
6 Health |
63 |
15.9 |
7 Other subjects |
32 |
8.1 |
By gender |
|
|
1 Male |
167 |
42.3 |
2 Female |
228 |
57.7 |
By race/ethnicity |
|
|
1 Asian |
22 |
5.6 |
2 Black |
280 |
70.9 |
3 Hispanic |
74 |
18.7 |
4 White |
16 |
4.1 |
5 Other |
3 |
.8 |
By SES* |
|
|
1 Low |
368 |
93.2 |
2 High |
27 |
6.8 |
By ESL** services eligibility |
|
|
1 No |
317 |
80.3 |
Continued
2 Yes |
78 |
19.7 |
By SPED*** services eligibility |
|
|
1 No |
361 |
91.4 |
2 Yes |
34 |
8.6 |
Total |
395 |
100.0 |
*SES: Socioeconomic status, **ESL: English as a second language, ***SPED: Special education.
measured on a five-point Likert scale which were as follows: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.
There were nine survey items that measured social presence, and they were adopted from the original CoI survey instrument developed by Arbaugh et al. [38]. They measured students’ perceptions of affective expression, open communication, and group cohesion. According to the original CoI instrument, the Cronbach’s alpha for social presence was 0.91, which indicated significant internal consistency reliability. See Appendix A for social presence survey items.
Engagement Data
Perceptional engagement data came from CSN’s student surveys conducted in Spring 2022. All survey items for engagement were measured on the same five-point Likert scale that ranged from 1 = strongly disagree to 5 = strongly agree.
There were 15 survey items that measured engagement. These items were adopted from Sun’s Engagement Scale, which was developed by Sun and Rueda [39]. They measured three different facets of engagement: emotional (Cronbach’s α: 0.88), cognitive (Cronbach’s α: 0.75), and behavioral (Cronbach’s α: 0.63) [39]. While a value of around 0.70 or greater for Cronbach’s α is widely considered desirable for reliability, some scientists find α values between 0.60 and 0.70 to be questionable but still acceptable [40] [41]. See Appendix B for engagement survey items.
Actual Student Achievement Data (Final grade)
Students’ final course grades were used to measure actual student achievement. Final grades were the cumulative points (out of 100) that students earned in their online courses. At the end of the 2021-22 school year, all final grades were automatically generated by the Edgenuity learning management system as percentages. Points for final grades came from assignments, tasks, quizzes, chapter tests, and final exams. These were all readily available in prepopulated courses.
5.5. Instruments Used and Their Validation
Structural equation modeling (SEM) was utilized to find the relationships between variables because it is a sophisticated theoretical model that allows for determining the impact of multiple variables on the outcome variable [42]. As part of the SEM analysis, confirmatory factor analysis (CFA) was performed first to ensure the validity and reliability of survey items [42]. As the second step, a model was formed and then improved by testing the statistical significance of the relationships among variables [43]. As the third and last step, a mediation analysis was utilized to discover the direct, indirect, and total effects of independent variables on student achievement [43].
Factor Analysis
Exploratory factor analysis (EFA) was not necessary for the purposes of this study. Perceptional data came from student surveys, instruments that had been pre-developed by other researchers [38] [39]. Hence, the validity and reliability of those survey instruments had already been established. However, a confirmatory factor analysis (CFA) was conducted to examine the hypothesized factor structure of the latent variables, which were social presence and engagement. Figure 3 shows the CFA model that was used to validate the measurement model of the latent construct of “Social Presence” by using AMOS 18 statistical analysis software.
Figure 3. The CFA model for social presence.
Social Presence (SP) is the mediating variable in this model (see Figure 3) and is conceptualized as a latent construct to measure the social presence of students. A second order CFA was employed to validate the main construct, Social Presence. Therefore, Social Presence (SP) was a second-order construct, which had three first-order latent constructs: Affective Expression (AE), Open Communication (OC), and Group Cohesion (GC). Each of the first-order constructs had three indicators and all of them were measured on a five-point Likert scale ranging from strongly disagree to strongly agree. The factor loading of SP on AE, OC, and GC were 0.91. 0.96 and 0.96, which were higher than the level of Malthouse`s cutoff value of 0.3 [44]. Thus, none of the indicators were removed from the model.
The goodness of fit is a statistical test to determine if sample data is accurate or somehow skewed. In this study, the goodness of fit statistics for the model indicated a good fit of the measurement model (χ2 = 59.774; x2/df = 2.989; SRMR = 0.0290; RMSEA = 0.071; CFI = 0.980; NFI = 0.970). After confirming the scales through CFA analysis, a reliability analysis for each subscale was also conducted. Cronbach’s α for Social Presence (SP) was 0.91 while Cronbach’s α for Affective Expression (AE), Open Communication (OC), and Group Cohesion (GC) were 0.77, 0.83, and 0.82 respectively. Based on Nunnally’s [45] rule of thumb of 0.70 for an acceptable Cronbach’s alpha value, all subscales were accepted as reliable.
Engagement is the other latent exogenous variable in this model and is conceptualized as a latent construct to measure the engagement of students. A second-order CFA was again employed to validate the main construct, Engagement. Therefore, engagement was a second-order construct, which had three first-order latent constructs: Behavioral Engagement (BE), Emotional Engagement (EE), and Cognitive Engagement (CE). BE had three indicators, EE had seven indicators, and CE had five indicators. All of them were measured on the same five-point Likert scale ranging from strongly disagree to strongly agree. All the factor loadings for the indicators were higher than the level of Malthouse`s cut-off value of 0.3, except for the ninth indicator (Eng9) of EE. The CFA analysis yielded a lower value for this indicator, and therefore, Eng9 was removed from the model.
The relevance of fit statistics for the model indicated a good fit of the measurement model (χ2 = 192.720; x2/df = 2.793; SRMR = 0.0379; RMSEA = 0.067; CFI = 0.964; NFI = 0.946) based on calculations via AMOS 18.
After confirming the scales through CFA analysis, a reliability analysis for each subscale was also conducted. Cronbach’s α for engagement was 0.91 while Cronbach’s α for Behavioral Engagement (BE), Emotional Engagement (EE), and Cognitive Engagement (CE) were 0.76, 0.91, and 0.87, respectively. Based on Nunnally’s [45] rule of thumb of 0.70 for an acceptable Cronbach’s alpha value, all the subscales were accepted as reliable (See Figure 4).
6. Data Analysis
Structural Equation Model (SEM)
The main purpose of this research was to investigate the effects of the exogenous variables (engagement and demographics variables) and the mediating
Figure 4. The CFA model for engagement.
variable (social presence) on the endogenous variable (student achievement), represented by the Final Grade. We utilized structural equation modeling (SEM) to analyze the data, which is defined as a powerful and complex comprehensive statistical method used to test hypotheses among latent and observed variables [46] [47]. Structural equation modeling “has become a widely used method for specifying, estimating and testing hypothesized interrelationships among substantively meaningful variables in the behavioral and social sciences” ([48], p. 499).
AMOS 18.0 and SPSS-28 software were used to perform statistical analysis. The estimated structural equation model for the conceptual model is presented in Figure 5. After confirming the measurement models of the latent variables in the first step, the hypothesized structural equation model was developed. Two exogenous latent variables, Engagement and Social Presence, were used to establish a
Figure 5. Structural equation model.
generic structural equation model. Out of these two exogenous latent variables, one of them (Social Presence) was used as a mediating variable. This generic model also included five control variables: Raceblack, Gender, SES, ESL and SPED.
The initial SEM analysis for the generic model yielded two out of five of the control variables to be insignificant. Therefore, these insignificant control variables (SES and SPED) were excluded from the model, one at a time, until the best-fitting model was reached. As a result, the variables Raceblack, Gender, and ESL were retained in the revised model. The exclusion process of the insignificant control variables developed the model fit in each step. When the final step was reached, all critical ratios were significant (at p < 0.05 level) for the remaining items in the revised model. Excluding the insignificant control variables from the generic model improved the model fit. For the next step, modification indices were examined to improve the model fit. They required the correlation of error terms to further improve the model fit. One at a time, a path between error terms was added to the model based on logical and theoretical considerations. The modification indices were re-examined until reaching the best-fitting model. The revised model yielded some improvement in the goodness of fit statistics. Table 2 provides the goodness of fit statistics for the revised SEM model. Table 3 provides the parameters estimated for the SEM model.
Table 2. Goodness of fit statistics for SEM.
Index |
Criterion |
Model |
Chi-square (x2) |
Low |
663.698 |
Degrees of Freedom (df) |
≥0.0 |
304 |
Probability |
≥0.05 |
0.000 |
Likelihood Ratio (x2/df) |
<3 |
2.183 |
Comparative Fit Index (CFI) |
>0.90 |
0.939 |
Tucker Lewis Index (TLI) |
>0.90 |
0.930 |
Normed Fit Index (NFI) |
>0.90 |
0.894 |
Root Mean Square Error of Approximation (RMSEA) |
≤0.05 |
0.05 |
Standardized RMR |
≤0.05 |
0.05 |
Hoelter’s Critical N (CN) |
>200 |
217 |
Table 3. Parameter estimates for structural equation model.
|
|
|
R.W. |
S.R.W. |
S.E. |
C.R |
P |
Social_Presence |
< - |
Raceblack |
−0.039 |
−0.004 |
0.379 |
−0.103 |
0.918 |
Social_Presence |
< - |
Engagement |
6.875 |
0.834 |
2.242 |
3.066 |
0.002 |
Social_Presence |
< - |
ESL |
0.706 |
0.061 |
0.484 |
1.46 |
0.144 |
Social_Presence |
< - |
Gendr |
−0.368 |
−0.039 |
0.362 |
−1.017 |
0.309 |
Final_Grd |
< - |
Engagement |
0.756 |
0.02 |
0.521 |
1.49 |
0.154 |
Final_Grd |
< - |
Raceblack |
−8.333 |
−0.137 |
3.032 |
−2.748 |
0.006 |
Final_Grd |
< - |
Social_Presence |
8.053 |
0.167 |
3.412 |
2.098 |
0.004 |
Final_Grd |
< - |
ESL |
−0.398 |
−0.006 |
3.464 |
−0.115 |
0.908 |
Final_Grd |
< - |
Gendr |
2.607 |
0.047 |
2.75 |
0.948 |
0.343 |
Note: Correlation significant p ≤ 0.01; RW = regression weights, SRW = standardized regression weights, SE = standard error, CR = critical ratio.
Out of two latent exogenous variables, Engagement had no statistically significant effect on the endogenous variable, Final Grade. The other exogeneous variable, Social Presence, had a positive and significant relationship with the Final Grade (β = 0.167, p ≤ 0.01). Therefore, this latent variable is employed as the mediating variable. While the Social Presence increased one standard unit, the Final Grade of the students also increased 0.16 standard units. The other statistically significant variable in the model was Raceblack. As previously mentioned, this variable was produced from the race variable as a dummy variable in which “1” indicated Black students and “0” indicated Others. Race had a statistically significant but negative effect on Final Grade (β = −0.137, p ≤ 0.01). In other words, Final Grades of Others were higher than African American or Black students’ final grades.
In the SEM model, there was one more significant relationship between two exogenous latent variables, Social Presence and Engagement (β = 0.834, p ≤ 0.01), which increased simultaneously. Since social presence also had a positive and significant effect on Final Grade, Social Presence was employed as a mediating variable in this model.
7. Results
A structural equation model was utilized to test the mediated effects. To examine whether Social Presence partially or fully mediated the impact on the endogenous variable, this variable was employed as a mediator variable. Table 4 provides the direct, indirect, and total effects of each variable in the model on the endogenous variable. If a variable has a significant direct effect and a significant indirect effect on the endogenous variable, it is concluded that the effect of this variable on the endogenous variable is partially mediated through the mediator variable. If the variable has no direct effect but has both a significant indirect and total effect on the endogenous variable, then the effect of this variable is fully mediated through the mediator variable [49] [50].
Table 4. Direct, indirect, and total effects of all variables on the endogenous variable (Final grade).
Variable |
Direct β |
Indirect β |
Total β |
Effect Type |
Social Presence |
0.167** |
NA |
0.167 |
Direct |
Engagement |
0.020 |
0.140** |
0.160** |
Mediated |
Raceblack |
−0.137** |
−0.001 |
−0.138** |
Direct |
Gender |
0.047 |
−0.007 |
−0.040 |
None |
ESL |
−0.006 |
0.010** |
0.004 |
Effect |
Note: **Correlation significant at p ≤ 0.05.
Social Presence as a mediator variable is found to have a direct and significant effect on the endogenous variable, Final Grade (β = 0.167, p ≤ 0.05). The other latent exogeneous variable, Engagement, has no direct effect, but instead has a fully mediated effect, on the endogenous variable (Indirect β = 0.140 p ≤ 0.05 and total β = 0.160, p ≤ 0.05).
Among the control variables, Raceblack has a significant but negative direct effect and a negative total effect on endogenous variable (Direct β = −0.137, p ≤ 0.05 and total β = −0.138, p ≤ 0.05). ESL, on the other hand, has only an indirect significant effect on endogenous variable (Indirect β = 0.10, p ≤ 0.05).
8. Discussion
The COVID-19 pandemic greatly impacted the routines, well-being, and learning of K-12 students [16] [51] [52]. The purpose of this study was to examine the relationships between social presence, engagement, and the achievement of high school students in asynchronous online classes in urban charter schools during the pandemic. It has been acknowledged that, in general, while “students have suffered throughout the pandemic, so too has their learning” [52]. However, it has been found that students of color, especially black and brown students, suffered more [16] [51] [52]. Similar to the findings of these studies [51], the results of this study also indicated that Black high school students received significantly lower grades in asynchronous online classes during the pandemic than the combination of all other races (β = −0.137, p ≤ 0.01). Disparities between students of color and the general population had existed even before the pandemic [35]. Hence, it was not unpredictable to have similar results when black communities were disproportionately impacted by the COVID-19 pandemic [52].
Researchers and practitioners know that “to determine all the influencing factors in a single attempt is a complex and difficult task” (Farooq et al., 2011, p.10). Social presence and engagement were found to be significantly related to each other (β = 0.834, p ≤ 0.01). Students reported that they were more socially present when they were more engaged, or they felt more engaged when they were more socially present. This finding is in alignment with Collins et al.’s [11] study that discussed the importance of text-based communication among students through online discussion boards for increasing student engagement.
While there was a significant relationship between the two, the impact of student engagement on final grades was surprisingly not statistically significant. However, when social presence was employed as a mediating factor, engagement had a fully mediated effect on students’ final grades (Indirect β = 0.140 p ≤ 0.05, and total β = 0.160, p ≤ 0.05). It can be understood from this study that students’ higher engagement resulted in higher social presence, which then resulted in higher course grades for students. The indirect impact of engagement through social presence was statistically significant, and it allowed students to get higher grades. When we look at the whole group (n = 395), social presence had a significant positive impact on students’ final grades in their online courses (β = 0.167, p ≤ 0.01). Even though Zehra et al. [53] stipulated that CoI served as a process framework for instructional methods rather than informing learning outcomes, this study’s results demonstrated that students’ higher social presence produced higher grades. This finding is consistent with Borup et al.’s [25] study that discussed the positive impact of social presence on learning outcomes of high school students. Moreover, this study extends CoI theory to K-12 settings and adds to the literature that examined the use of CoI constructs in K-12 settings [24] [34].
Looking at the mediation analysis, one more significant finding was the indirect relationship between ESL students and their final grades (Indirect β = 0.10, p ≤ 0.05).
ESL students reported that they were more socially present in online classes compared to their peers; however, this was not enough of a factor to earn significantly higher grades than their classmates. This finding draws similarity with Khammat Al-Iessa et al. [33] study that reported ESL students’ social presence contributed more to the changes in their engagement in online classes during pandemic. Therefore, increasing the social presence of ESL students in online classes is important as it can be a mediating factor to improve their learning outcomes.
9. Conclusion, Limitations, and Future Research
Previous studies that employed the CoI framework in relation to online learning examined the constructs of CoI as instructional methods in online teaching without explicitly discussing their impact on student learning outcomes. Therefore, this study provided new evidence to CoI theory by demonstrating that the students’ higher engagement in the Edgenuity platform resulted in higher social presence, which then resulted in higher course grades for students. This major finding can help improve high school students’ learning outcomes in asynchronous online classes by increasing student engagement and social presence. However, researchers recommend looking at other contributing factors for positive online learning outcomes because they might not be solely impacted by either engagement or social presence, the factors that this study examined. Nevertheless, the findings of this study can help educators address issues and create well-designed online learning environments that would produce better learning outcomes.
There were also some limitations inherent to this study. The major limitation of this study was that it was conducted with a small population of students attending a single charter school network located in the Midwestern United States. Therefore, the results cannot be easily generalized for all K-12 online settings. Moreover, secondary data was used, which came with its own limitations. Still, K-12 teachers and administrators can take the results of this study into consideration when designing and planning instructional processes to improve students’ learning outcomes, especially for African American students. The researchers also make recommendations for further research that examines the underlying factors of the relationships between social presence, engagement, and achievement of high school students of color in asynchronous online classes.
Conflicts of Interest
The authors declare no conflicts of interest.
Appendix A. Survey Items for Social Presence
CoI Survey—Social Presence Subsection [38]
9 Items; 3 Dimensions
Affective expression
1. Getting to know other course participants gave me a sense of belonging in the course.
2. I was able to form distinct impressions of some course participants.
3. Online or web-based communication is an excellent medium for social interaction.
Open communication
4. I felt comfortable conversing through the online medium.
5. I felt comfortable participating in the course discussions.
6. I felt comfortable interacting with other course participants.
Group cohesion
7. I felt comfortable disagreeing with other course participants while still maintaining a sense of trust.
8. I felt that my point of view was acknowledged by other course participants.
9. Online discussions help me to develop a sense of collaboration.
5-point Likert-type rating scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.
Survey available at
https://www.thecommunityofinquiry.org/CoISurveyDraft14b1.pdf
Appendix B. Survey Items for Engagement
Engagement Scale [39]
15 Items; 3 Dimensions
Behavioral engagement
1. I follow the rules of the online class.
2. I complete my homework on time.
3. I check my schoolwork for mistakes.
Emotional Engagement
4. I like taking the online class.
5. I feel excited by my work at the online class.
6. The online classroom is a fun place to be.
7. I am interested in the work at the online class.
8. I feel happy when taking online class.
9. I feel bored by the online class.
10. I talk with people outside of school about what I am learning in the online class.
Cognitive Engagement
11. I study at home even when I do not have a test.
12. I try to look for some course-related information on other resources such as television, journal papers, magazines, etc.
13. When I read the course materials, I ask myself questions to make sure I understand what it is about.
14. I read extra materials to learn more about things we do in the online class.
15. If I do not know about a concept when I am learning in the online class, I do something to figure it out.
Survey available at https://learninglab.uni-due.de/research-instrument/13923