Media plays a significant role in shaping public perception, especially concerning multiculturalism and representation. [15] examined representation in Australian news media through a multimodal survey combining CATI (Computer-Assisted Telephone Interviewing) and CAPI (Computer-Assisted Personal Interviewing) methodologies. Their study revealed a strong link between the perception of adequate media representation and trust in society, demonstrating how fair media portrayal fosters civic engagement, enhances a sense of belonging, and influences participation in socio-political discourse. Similarly, [15] conducted a content analysis of mainstream news and Twitter sentiment. Using Stuart Hall’s encoding or decoding model, this study assessed media portrayals of multicultural events, revealing how media narratives impact public perception and social cohesion, either promoting inclusivity or exacerbating divisions ([15]).

Beyond general perception, media also significantly shape public attitude and policy. [3] explored this by examining the media’s influence on attitudes towards issues such as disability, climate change, and economic development. Their findings illustrated how media framing can shift public opinion, particularly in cases where negative portrayals reinforce societal biases. For instance, negative depictions of individuals receiving disability benefits led to hardened societal attitudes and reduced public support for welfare programs ([3]). Similarly, [8] focused on media representation of minorities in Singapore. Their work revealed that ethnic groups, such as Malays and Indians, are often depicted in stereotypical roles, influencing public attitudes and reinforcing systematic biases. The authors suggested anchoring future research in social cognitive theory, framing theory, and cultivation theory to better understand how media representation shapes societal norms and values ([8]).

Media diversity not only reflects society but also acts as a driver of audience engagement and inclusivity. [7] examined various media forms, including film, television, and digital platforms, finding that diverse representation leads to greater audience engagement and social understanding. The Annenberg Inclusion Initiative reported that minority groups are often sidelined in media, resulting in a lack of authentic voices and stories ([7]). This study emphasised the importance of diverse media representation in fostering social cohesion and combating prejudice by incorporating inclusive narratives that challenge systematic inequalities and promote empathy among audiences.

Despite calls for diversity, implicit biases remain prevalent. [11] examined subtle racism in media, highlighting how ethnic and cultural groups are implicitly essentialised. Their study found that media narratives frequently generalise the actions or attitudes of specific groups, reinforcing stereotypes. For example, crimes committed by individuals from minority backgrounds were often framed around their ethnicity, whereas similar crimes committed by majority group members were not racialised. This implicit bias influences public perception and perpetuates systematic discrimination ([11]).

Recent work on mutuality emphasises not only the presence of diversity within institutions, but the degree of alignment between internal composition and the demographic profile of the communities they serve. In this view, representation is most consequential when there is a close correspondence between “who produces or mediates content” and “who is addressed by that content”, since misalignment can erode trust, perceived legitimacy, and engagement in key sectors such as healthcare, retail and public services. ([12])

Challenges related to diversity and community representation manifest across different media sectors, including sports, advertising, and the broader digital landscape.

Sports media coverage significantly impacts the formation of national identity. [2] analysed England and Italy’s Euro 2020 media portrayal, highlighting how national identity is selectively constructed, often marginalising women and ethnic minorities. While England’s team was praised for its diversity, Italy’s team was criticised for its lack of ethnic representation, showing how race and racialisation influence public discourse around sports ([2]).

In advertising sector, diversity representation is also crucial. [10] investigated the increasing demand for diversity in this sector, highlighting the challenges brands face in maintaining authenticity while meeting consumer expectations. Their study introduced an eight-step framework for diversity in advertising, which synthesises key insights from the literature and identifies future research directions. A critical challenge identified was the backlash brands faced when they failed to genuinely engage with diverse representation, as consumers are becoming more aware of performative inclusivity ([10]).

The rise of digital platforms and algorithmic filtering introduces further complexities. [1] examined the effect of algorithmic filtering on media diversity, particularly as audiences shift from traditional legacy media to digital platforms. Their study found that algorithmic filtering often narrows news consumption, reducing exposure to diverse viewpoints and contributing to social polarisation and misinformation. The authors proposed a framework for understanding media diversity across journalism, law, and computer science, highlighting the need for an integrated approach to media consumption analysis ([1]).

The interplay between cultural diversity, policymaking, and media representation is significant. [16] investigated how the European union’s diversity policies influence the construction of a supranational European identity and how intercultural representation is portrayed in official EU messaging. While the study found that the EU promotes “unity in diversity”, concerns were raised that its media representations sometimes reinforce racial hierarchies ([16]). These insights underscore the complex relationship between cultural diversity, policymaking, and media representation, emphasising the need for ongoing critical analysis of media narratives and their societal impacts.

Building on earlier work that applied Grey and ARIMA models to predict cohort-level diversity from short and fragmentary demographic series, we extend the logic of diversity forecasting from population baselines to mediated representation. Whereas previous studies showed that reasonably accurate forecasts are achievable under “small data” conditions for relatively stable attributes such as country of birth, our hybrid LSTM-ARIMA-Prophet framework leverages two decades of media traces and engagement metrics to model future trajectories of community visibility. In effect, the system operates as a media-focused mutuality forecaster, translating historical representation patterns into forward-looking scenarios that can be continuously updated and evaluated as new data become available. ([13]; [14])

To support comprehensive forecasting of community representation across media platforms, data was meticulously collected by scraping and aggregating from multiple sources spanning news media, social media, and entertainment platforms.

4.1.1. Data Sources and Coverage

Over a 20-year period (2004-2024), we compiled a longitudinal corpus spanning three major media domains: news, social media and entertainment. For each domain, we focused on six ethnic communities that are highly salient in Australian public discourse: African, Asian, European, Hispanic, Indigenous and Middle Eastern. The goal was not to exhaustively scrape all available content, but to construct a consistent, multi-platform panel that enables comparative analysis of representation levels and engagement depth over time.

For newsmedia, we queried the online archives of five international broadcasters (BBC, CNN, Fox News, Al Jazeera and ABC News) using combinations of community identifiers (e.g., African, Asian, Indigenous) with general diversity terms (e.g., representation, racism, inclusion). This yielded a raw set of approximately 70,000 articles. For socialmedia, we collected posts via the official APIs of Twitter/X, Facebook, YouTube, Instagram and Reddit. Monthly keyword queries mirrored the news pattern and were constrained by platform-specific rate limits; this produced 60,000 posts before filtering. For entertainment, we scraped title- and cast-level metadata from IMDb, TMDb and TVMaze, focusing on films and television series tagged with community-relevant descriptors (e.g., Aboriginal, Latinx, MiddleEastern) and released between 2004 and 2024, totalling 1500 unique titles.

To maintain linguistic coherence, all collections were restricted to content in English or containing an English translation. When APIs exposed geolocation or country-of-origin metadata, we retained only items produced in, distributed to, or explicitly referring to Australia. Each record stores the year of publication or release, the dominant community mentioned, platform identifier, and additional attributes such as title, description and sentiment scores. This design yields a multi-source panel that prioritises interpretability and temporal comparability over sheer volume, and should therefore be interpreted as a conservative,structuredsample rather than an exhaustive crawl of all possible content. (Table 1)

Table 1. Summary of datasets.

4.1.2. Cleaning, Aggregation and Final Dataset Size

The raw scrape undergoes several cleaning and normalisation steps. First, we remove exact and near duplicates across and within platforms. Text fields are lower-cased, URLs and emojis are stripped, and posts or articles with fewer than 70,000 alphabetic characters are discarded as likely noise (e.g., URLs only, single hashtags). Second, we drop records with missing or inconsistent year information and align all timestamps to calendar years. Third, where multiple communities are mentioned, we assign the record to the dominant community using a simple frequency heuristic: a community label must appear at least twice and more often than any other community’s label in the same document to be treated as the primary focus.

A naïve scrape of the selected platforms would yield millions of individual posts and articles over two decades. However, our analysis operates at the level of yearly community-platform summaries rather than individual items. For each year t, community c and platform p, we aggregate all matching records to compute representation percentages, sentiment scores and engagement metrics (Section 4.1.3). This aggregation, combined with the strict quality filters above, explains the large reduction in row count. The final analytical dataset comprises 2500+ rows, each corresponding to a unique (c, p, t) triple with non-zero activity. Thus, for example, “Indigenous - Twitter - 2015” appears as a single row whose features summarise all tweets about Indigenous communities in that year.

This two-stage reduction - from raw posts to quality-filtered records, and from records to yearly aggregates - is deliberate. It preserves the long-term temporal structure of community representation across platforms while avoiding the computational and statistical issues that arise when fitting sequence models to highly noisy, post-level data.

4.1.3. Representation Percentage and Engagement Metrics

To compare communities and platforms on a common scale, we derive two types of variables from the cleaned corpus: representation percentages and engagement metrics.

For each community c, platform p and year t, we define the representation percentage as

where N c , p , t is the number of records (articles, posts or titles) assigned to community c on platform p in year t, and N p , t is the total number of records we collected for that platform and year.

Example. If in 2015 we collect N p , t = 1000 news articles from the BBC and N = 120 contain a dominant reference to African communities, then

The engagementmetrics aim to capture not only howoften communities appear but also howtheyarediscussed. For a fixed (c, p, t) cell we let d = 1 , ⋯ , D index the associated documents. Each document d is encoded using: 1) a TF-IDF vector t d ∈ R K over the vocabulary; 2) a BERT sentence embedding b d ∈ R 768 and 3) topic scores Z d obtained via principal component analysis (PCA) of the TF-IDF matrix. Building on these, we define five scalar metrics:

TextComplexityScore(TCS) - the average variance of TF-IDF weights across documents,

which increases as language becomes richer and more varied.

SemanticDiversityScore(SDS) - one minus the average pairwise cosine similarity between BERT embeddings,

so higher values indicate that the community appears in more diverse semantic contexts.

SentimentPolarityScore(SPS) - the mean of all embedding dimensions,

which correlates with the overall positive-negative orientation of coverage.

TopicDiversityScore(TDS) - the cumulative variance explained by the first M principal components (we use M = 10),

where λ m is the proportion of TF-IDF variance captured by component m.

CommunityRepresentationIndex(CRI) - the standard deviation across embedding dimensions of the mean BERT vector,

used here as a proxy for how balanced or skewed the community’s portrayal is in that year.

Example. For Indigenous communities on Instagram in 2020 we observe D = 6000 posts. Their TF-IDF variance yields TCS = 0.09; the average pairwise cosine similarity between BERT embeddings is 0.720, giving SDS = 1 − 0.720 = 0.280 the mean BERT activation corresponds to a mildly positive sentiment with SPS = 0.012. The first ten PCA components explain TDS = 0.540 of TF-IDF variance; and the embedding-level standard deviation is CRI = 0.110. Together these illustrate how the metric suite characterises not only the frequency of Indigenous coverage but also its linguistic richness, topical breadth and tonal balance.

Given the temporal nature of the community representation data, a hybrid modelling approach was adopted to leverage the strengths of different forecasting techniques.

4.2.1. Model Selection and Customised Architecture

A Long Short-Term Memory (LSTM) neural network was selected as the foundational architecture due to its proven efficiency in capturing temporal dependencies and complex non-linear relationships in sequential datasets ([6]; [5]). This choice was informed by its enhanced capability to capture the dynamic characteristics of media representation datasets over traditional time-series forecasting methods such as ARIMA and Prophet, which were also included for comparative evaluation in later stage.

The custom-designed LSTM model architecture comprised sequentially stacked layers aimed at effectively modelling temporal patterns in historical data. See Figure 2 for an illustrated architecture.

FirstLSTMLayer(64Units): Capturing initial sequential patterns, maintaining historical context and returning sequences for further temporal pattern extraction.FirstDropoutLayer(20%rate): Introduced to combat overfitting, which is common in sequential models, by randomly deactivating neurons during training, ensuring robust learning.SecondLSTMLayer(32units): This deeper layer condenses sequence information into a context-rich vector, effectively extracting subtle, longer-term sequential patterns from historical representation data.SecondDropoutLayer(20%rate): Additional dropouts provide further regularisation and stability during training.DenseoutputLayer(1-unit,linearactivation): This final layer outputs a single numeric value representing the forecasted representation percentage for each successive year.

Crucially, the community-specific LSTM models were developed and optimised with Adam. This approach allows for fine-tuning to distinct data patterns within each community’s representation trends, yielding higher predictive accuracy compared to generalised models.

Figure 2. Detailed Custom LSTM architecture with an addition to the community-specific algorithm.

For the ARIMA model, it was configured using an auto-ARIMA method. This automated approach systematically explores various ARIMA parameter combinations and selects the optimal model based on the Akaike Information Criterion (AIC), ensuring an efficient and statistically sound fit to the data.

Finally, the Prophet model was implemented, which incorporated both weekly and yearly seasonality components, alongside the integration of holiday effects, allowing the model to capture recurring patterns and fluctuations in the media industry. A linear growth trend was assumed for the underlying time series.

4.2.2. Theoretical Alignment and Metric Influence

The conceptual Framework for understanding cultural diversity within spatial and media context, as articulated by [9] research, offers critical alignment with this methodology. Their study examined the relationship between cultural diversity (measured by language, religion, and country of birth), and key urban parameters using 2021 Australian Census data. Employing Spearman’s correlation and Simpson’s Diversity Index, they revealed nuanced spatial patterns and economic implications of diversity across Statistical Area Level 4 (SA4) regions in Australia. The multi-dimensional and data-driven nature of their approach has strongly influenced the framing of diversity metrics used in the forecasting models and engagement visualisations in this project.

Moreover, their focus on socio-spatial implications and anomalies inspired the integration of anomaly detection methods and community-specific forecasting in this study, extending their geographic analysis to a media-specific representation context.

4.2.3. Forecast Models, Training and Model Selection

We estimate threemodelfamilies for every community-platform time series: 1) a univariate ARIMA model, 2) Prophet, and 3) a univariate LSTM network. This avoids a priori assignment of specific models to specific communities and enables a data-drivenselection based on out-of-sample performance.

Pre-processingandtargetseries. For each community ccc and platform ppp, the annual representation percentage R e p % c , p , t (Section 4.1.3) forms the target series. We linearly interpolate at most one missing year per series; if more than one consecutive year is missing, the segment is excluded from forecasting and

flagged in the results. Before ARIMA and Prophet estimation, we optionally stabilise variance with a logit transform l o g i t ( x 100 ) when the series is close to 0 or 100; LSTM models operate on the raw percentage scaled to [0, 1].

Modelspecifications.

ARIMA. Orders ( p , d , q ) are selected via a grid over p , q ∈ { 0 , 1 , 2 , 3 } and d ∈ { 0 , 1 } , choosing the configuration with minimum AIC on the training window.Prophet. We include yearly seasonality and an automatic changepoint prior, with the number of changepoints set to [[FILLIN:N_CP]]. Sensitivity analysis over the changepoint prior scale ∈ {0.01, 0.1, 0.5} is conducted and the best-performing setting retained.LSTM. A single-layer LSTM with [[FILLIN:HIDDEN_UNITS]] units followed by a dense output is trained using a sliding input window of [[FILLIN:WINDOW]] years. We use Adam (η = [[FILLIN:LR]]) and early stopping with patience [[FILLIN:PATIENCE]]. All hyperparameters were fixed a priori and applied uniformly across communities to avoid overfitting by repeated search on small series.

Selectioncriteriaandreconciliation. For each community we report as the primarymodel the one that minimises MAPE on a temporally held-out test window (Section 4.2.4), with MAE, RMSE, and R2R^2R2 provided for completeness. This procedure yielded the LSTM as best for four of the six communities, while ARIMA was preferred for the comparatively smoother Asian series; Prophet was competitive only on specific runs but not dominant overall (see [[Table 5]] and [[Figure 5]]). Earlier references in the paper that seemed to “assign” models to communities reflected preliminary experiments and have been reconciled to the empirical selection reported here.

Rationalepercommunity. The observed representation trajectories help explain the winning models: Indigenous and African series exhibit non-linear drifts and occasional regime shifts, favouring LSTM’s capacity to model long-range dependencies; the Asian series shows short-memory fluctuations about a stable mean, for which parsimonious ARIMA dynamics suffice; Middle Eastern and Hispanic series show intermittent spikes (e.g., event-linked), where Prophet captured changepoints but did not consistently surpass LSTM on test error. These empirical choices will be revisited as additional years of data become available.

4.2.4. Forecasting Technique (Recursive Prediction Loop)

A recursive forecasting method was implemented, where predictions for each future year (2025-2035) were iteratively fed back into the model as inputs for subsequent predictions. This recursive forecasting approach effectively enables accurate multi-year forecasts by allowing models to capture future representation trends. See AppendixA for a more technical breakdown of the training and prediction process.

The final phase of this project involved designing and deploying a robust and user-friendly web application for forecasting media representation across diverse communities.

4.3.1. Architecture Design

A multi-tier architecture was selected to achieve scalability, maintainability, and reliability (see Figure 4). The system comprises three layers:

PresentationLayer: Utilises HTML, CSS, JavaScript, and Flask’s templating engine (Jinja2) to provide an intuitive graphical user interface (GUI) for inputting historical data and viewing prediction results visually via interactive charts using Chart.js.ApplicationLayer: Implemented using Flask, a lightweight Python web framework, responsible for managing HTTP requests, data validation, input pre-processing, invoking pre-trained models (LSTM, ARIMA, and Prophet), and formatting prediction outputs.DataLayer: Incorporates serialised machine learning models (LSTM models saved in the Keras native format; ARIMA, and Prophet models serialised using joblib). The models and relevant data files are effectively stored and retrieved as needed.

4.3.2. Technology Stack

The following technologies were chosen for their suitability for high performance and ease of deployment:

Python: Primary language for backend development and machine learning.FlaskFramework: For creating RESTful APIs and serving web pages.TensorFlowandKeras: To develop, train, and serialize custom LSTM models.StatsmodelsandProphet: For ARIMA and Prophet-based time series forecasting models.HTML/CSS/JavaScript (chart.js): To design a responsive and interactive front-end user interface.AWSEC2: For cloud infrastructure, facilitating remote hosting and continuous availability.

4.3.3. Model Serialisation and Integration

To facilitate real-time predictions, the trained forecasting models were serialised and integrated within the Flask application. LSTM models were serialised using the modern keras format for enhanced compatibility, while ARIMA and Prophet models were saved using Python’s joblib library, enabling rapid deserialisation and prediction upon request.

Deployment was performed using an AWS EC2 (Elastic Compute Cloud) instance. The process involved:

InputStep: An EC2 instance running Ubuntu Linux was provisioned, providing a stable and secure server environment.SecurityConfiguration: Firewall rules and security groups were configured to allow traffic over standard HTTP ports (5000 for Flask applications) while securing SSH access for remote administration.EnvironmentConfiguration: Dependencies were managed within a Python virtual environment (venv), ensuring that package installations and versions remain consistent and isolated.FileTransferandRemoteManagement: Project files, including Python scripts, serialized models, and HTML/CSS/JavaScript resources, were securely transferred via SSH using SCP.ApplicationExecution: The Flask application (Deployment.py) was started using the Gunicorn WSGI server in production mode, ensuring optimal performance and scalability.

4.4.1. Testing and Validation

Post-deployment, rigorous system-level testing was conducted, including:

UnitTesting: Individual modules (including API endpoints, model serialisation / deserialisation, and prediction outputs) were tested to ensure accurate and reliable functionality. IntegrationTesting: Frontend-backend communication was thoroughly tested using tools like Postman, verifying data integrity and response consistency.LoadTesting: System performance was validated under varying levels of simulated user traffic to ensure scalability and robustness.

4.4.2. Contributions and Enhancements to Deployment

Significant contributions to this deployment process included optimising the model serialisation strategy, enhancing API response structures, and streamlining the frontend-backend integration. Additionally, improvements in error handling, real-time prediction visualisations, and user interface intuitiveness substantially increased system usability and effectiveness.

This also includes the development of an automated interpretation function, which generates human-readable interpretations of numerical forecasts (e.g., “CNN” will see a 10% rise in representation of Asian communities in 2030), thereby improving interpretability and user comprehension.

Table 2. Result table for the LSTM model.

Table 3. Result table for ARIMA model.

Table 4. Result for the prophet model.

To evaluate and compare model effectiveness, the study employed standard accuracy metrics: Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and R² score. The LSTM Model demonstrated superior performance with consistently lower error metrics, indicating higher predictive accuracy compared to ARIMA and Prophet models. Specifically, the LSTM model achieved an average RMSE of approximately 1.32% (see Table 2), while ARIMA and Prophet reported average RMSE values of 2.08% and 1.76% (see Table 3), respectively. This highlights LSTM’s capability in capturing temporal patterns effectively. (see Table 4)

A comparison of the three modelling approaches across our six target communities (Table 5) makes it clear that the LSTM consistently delivers the balance of accuracy and exploratory power. For every community except Asian, the LSTM attains both the lowest AMAPE (ranging from 0.12 for indigenous to 0.21 for Asian) and the highest R² (0.89 - 0.98), reflecting its superior ability to capture nonlinear, long-range temporal patterns in representation data. The sole exception is the Asian Community, where the ARIMA model achieves an impressive AMAPE of 0.12- beating the LSTM’s 0.21-albeit with a lower R² (0.82 vs 0.91), suggesting that ARIMA can tightly fit short-term trends for this subgroup but at the cost of overall variance explained. The prophet forecast, by contrast yield the highest errors (AMAPE 0.24 - 0.36) and the weakest fits (R² 0.77 - 0.95), particularly for African and Indigenous communities. In sum, while ARIMA may be preferable when minimizing absolute forecast error on relativity stable series (as with Asian representation), the LSTM emerges as the go-to model for delivering robust, high-fidelity forecasts across the full diversity of communities.

See Table 5 below for a summary of the performance of each model by community with the combined datasets.

Table 5. Summary of model performance by community.

The analysis provided granular forecasts from 2025 to 2035 for various communities - African, Asian, European, Hispanic, Indigenous, and Middle Eastern - across distinct media categories. These predictions revealed meaningful insights into future representation dynamics. The overall predicted trends for each model are visually presented in Figures 3-5.

NewsMedia: Predictions indicate nuanced shifts. Some channels such as Fox News and CNN project declines in representation percentages for African and Middle Eastern communities by approximately 8% - 12%. Conversely, channels like BBC and Al Jazeera demonstrate positive growth trends of around 5% - 9% for Asian and Indigenous communities, respectively.

Figure 3. Predictions of community representation from 2025 to 2035 by the ARIMA model.

Figure 4. Predictions community representation from 2025 to 2035 by the LSTM model.

Figure 5. Prediction community representation from 2025 to 2035 by the Prophet model.

SocialMediaPlatforms: Predictions suggest a general upward trend across platforms like Twitter, Instagram, and YouTube, particularly benefiting Hispanic, Indigenous, and Middle Eastern communities. Notably, Instagram is expected to witness an increase in Hispanic representation by about 14% by 2035, highlighting evolving engagement patterns and shifts towards inclusive digital spaces.EntertainmentMedia: Entertainment forecasts varied significantly across sectors such as Hollywood, Bollywood, and K-Dramas. The Hollywood industry projects a moderate increase (6% - 9%) for Asian and Middle Eastern representation, whereas Bollywood is expected to enhance Indigenous and Hispanic community representation significantly (by about 12% - 15%). Korean Dramas (K-Dramas) displayed stable growth trajectories, particularly for Asian and European communities, forecasting an average increment of 9% by 2035.

To enhance the interpretability and accessibility of the predictive outcomes, an interactive web application was developed. This application visualises forecasted representation trends and allows users to run new predictions with custom inputs. These visualisations clearly depict community-specific trends and platform-specific dynamics, facilitating easier comprehension of future representation scenarios.

Here is the link to the application: http://sidmediadr.diversityatlas.io:5000/. See AppendixB for some screenshots of the deployed application.

Significant contributions were made in refining the model architecture and data pre-processing pipeline to achieve higher accuracy and robust forecasts. By integrating techniques such as enhanced data pre-processing (TF-IDF, BERT embedding), community-level modelling granularity, and feature engineering improvements, prediction accuracy increased notably from preliminary baselines. Moreover, adopting a streamlined deployment approach via Flask on AWS EC2 enhanced accessibility and user interaction with real-time forecasting capabilities.

In summary, the predictive outcomes not only validated the methodological approach but also provided actionable insights into evolving media landscapes concerning diversity and representation. These forecasts have significant implications, enabling targeted interventions by media organisations and advocacy groups aimed at achieving balanced and inclusive community representation.

To assess the depth and quality of community representation beyond frequency counts, an Engagement-Based Metric Framework was introduced. This framework is composed of five interpretable metrics designed to quantify linguistic complexity, semantic richness, affective tone, topic diversity, and representational fairness across news channels, social media, and entertainment media.

Each dataset was processed using a combination of TF-IDF vectorisation, BERT embedding, and Principal Component Analysis (PCA). The specific metrics are defined as follows (see Table 6):

TextComplexityScore: Calculated as the average variance of TF-IDF vectors across documents, capturing the lexical variability and sophistication of language used to describe the communities.SemanticDiversityScore: Derived by computing 1 minus the average cosine similarity between BERT-based sentence embedding, indicating the breadth of topics or contexts in which communities are discussed.SentimentPolarityScore: Computed as the mean of all BERT embedding values, reflecting the overall tone (positive, negative and neutral) associated with community mentions.TopicDiversityScore: Estimated by summing the top 10 PCA components’ variances from TF-IDF features, signifying the thematic focus or spread within the content. CommunityRepresentationIndex: Computed using the standard deviation across all BERT features to measure balance and inclusivity in representation.

This engagement metric framework enables a multi-dimensional evaluation that advances conventional representation analysis by integrating semantic, affective and structural dimensions into a cohesive evaluation strategy. While traditional research has focused on quantity-based representation (how often the community appears), this metric Framework integrates NLP-based semantic modelling (TF-IDF and BERT), dimensionality reduction (PCA), and affective measures (sentiment polarity). It provides a qualitative, multi-dimensional lens for assessing engagement potential, narrative diversity, and fairness of portrayal, allowing quantification of bias or narrow framing in news, detection of sentiment skew in social platforms, and uncovering of topic under-representation in entertainment content.

Table 6. Description of metrics.

The results from each dataset were visualised using bar plots (see Figure 6),

Figure 6. Engagement metrics representation.

6.1.1. Performance of Predictive Models

The comparative analysis revealed that among the three models evaluated - LSTM, ARIMA, and Prophet - the LSTM model consistently outperformed in predictive accuracy metrics (RMSE and MAE). The higher accuracy of the LSTM model can be attributed to its inherent capability to capture complex temporal dependencies and non-linear relationships, which are prevalent in the dynamic nature of media representation data. This finding aligns with existing literature that emphasises the superiority of deep learning techniques over traditional statistical methods for intricate time-series data, particularly when dealing with long-range dependencies and subtle patterns.

6.1.2. Representation Dynamics across Media Types

The predictions demonstrated significant variations in representation trends across different media platforms, underscoring unique dynamic within each domain:

NewsMedia: The observed variability in predicted representation trends among news media like the BBC, CNN, Al Jazeera, and Fox News suggests the influence of diverse editorial policies, organisational priorities, and potential structural biases. Predicted positive growth trends for Asian and Indigenous communities on channels like BBC and AI Jazeera may reflect intentional diversity policies and proactive editorial strategies. Conversely, declining trends predicted for channels like Fox News potentially indicate structural biases or gaps in inclusion strategies, identifying clear opportunities for targeted interventions to address these imbalances. SocialMediaPlatforms: The projected substantial growth of Hispanic and Indigenous communities on platforms such as Instagram and Twitter highlights the increasing influence of grassroots movements and increased digital activism. Social media’s inherently participatory and user-driven nature may explain these optimistic trends, reflecting a broader societal push toward greater inclusivity and self-representation. This underscores social media’s transformative potential as platforms for amplifying historically marginalised voices.EntertainmentMedia: Forecasts indicating increases in diverse representation across entertainment sectors, such as Hollywood, Bollywood, and K-dramas, suggest an encouraging global evolution within creative industries. The prominent predicted growth for Indigenous and Hispanic representation in Bollywood and increased Asian and Middle Eastern representation in Hollywood aligns with changing audience preferences, active advocacy from diverse groups, and growing market pressures demanding more authentic and inclusive storytelling. These shifts indicate a positive shift towards broader cultural representation in global entertainment.

6.2.1. Policy Implications

The outcomes of this research provide actionable insights for policymakers, media organisations, and advocacy groups aiming to foster inclusive representation. Specifically, identifying predicted declines or insufficient growth in the representation of certain communities offers a data-driven basis to inform targeted diversity policies, refine media guidelines, and implement proactive intervention strategies designed to address identified representation imbalances.

6.2.2. Media Industry Recommendations

Media organisations can leverage these forecasts to guide their strategic content planning, diversity hiring practices, and audience engagement strategies. For instance, news channels facing projected declines in representation might benefit from revisiting editorial policies and conducting comprehensive representation audits. Whereas entertainment media sectors forecasting positive trends can capitalise on and reinforce inclusivity through continued diverse casting, culturally nuanced storytelling, and authentic collaborations with under-represented creators.

6.2.3. Social Implications

The anticipated improvements in representation on social media platforms underline the platforms’ societal importance as drivers of cultural inclusion and representation awareness. Social media corporations should therefore continuously enhance their platforms governance, promote inclusive content creation initiatives, and implement community engagement policies to sustain positive representation trends.

This research uniquely contributes to existing literature through several critical advancements. Firstly, a novel longitudinal dataset was created through web scraping from news websites, social media APIs, and entertainment databases, spanning two decades from 2004 to 2024. This significant empirical contribution offers extensive and detailed insights into media representation patterns that were previously unavailable.

Besides, a granular community-level forecasting methodology was implemented. This involved training individual LSTM models for each specific community, providing significantly deeper and more nuanced insights than the generic or aggregated-level analyses typically presented in prior literature. Furthermore, the study incorporated advanced feature engineering techniques using Natural Language Processing (NLP). This included TF-IDF vectorisation and BERT embedding, which substantially enhanced model accuracy and interpretability by effectively capturing the textual nuances and semantic complexities within media content.

Furthermore, the development of a deployed and accessible predictive tool represents an innovative and practical approach. The Flask-based interactive forecasting application, deployed via AWS EC2, democratises access to predictive analytics for media organisations, researchers, and policymakers alike, facilitating immediate, data-driven decision-making and practical application of the research findings.

Despite significant achievements, several limitations warrant consideration for future work:

6.4.1. Limitations

While data was rigorously collected from reputable sources, source-specific coverage or algorithmic recommendations could potentially impact the completeness or neutrality of representation captured. Besides, model generalisability remains a factor; although LSTM models provided the most accurate forecasts, their reliance on historical patterns means unexpected societal shifts or unprecedented events (e.g., major policy changes, global crises) could limit predictive reliability. Thus, continuous model updates and the incorporation of real-time data streams are recommended for maintaining accuracy. Lastly, LSTM and deep learning methods, despite their high accuracy, introduce increased computational complexity and training time, which could potentially limit broader implementation in resource-constrained settings, highlighting a trade-off between model sophistication and practicality for all users.

6.4.2. Future Research Directions

Future research can build upon the current work by exploring several promising avenues. This includes pursuing enhanced multimodal analysis, which would involve combining textual, visual (e.g., image and video content analysis), and social network analysis for deeper, more holistic insights into representation dynamics. Additionally, cross-platform interplay studies could be conducted to analyse the intricate relationships between news, social media, and entertainment media in shaping community perceptions and representation, identifying areas of synergy or conflict. The development and integration of real-time adaptive forecasting models using online learning methods represent another key direction to continuously update predictions and maintain accuracy amidst rapidly changing societal dynamics and media landscapes. Finally, future work could focus on developing a system capable of predicting representation within individual article from any media platform.