Evaluating Data-Protection Mechanisms in Federated Learning: A Performance-Centric Analysis ()
1. Introduction
Federated Learning has recently been adopted in decentralised machine learning to enable multiple participants to collaboratively train a shared model without revealing or transferring their raw data to a central location. By keeping data local and exchanging only model updates or gradients, federated learning reduces many of the data-exposure risks associated with centralised training. However, federated learning is not inherently immune to leakage: adversaries can exploit shared updates or gradients to infer sensitive information about the underlying data. To close this gap, researchers have proposed a variety of data-protection mechanisms, including differential privacy, homomorphic encryption, secure multi-party computation, secure aggregation protocols, federated averaging with secure aggregation, model distillation, regularisation, and client-side data anonymisation.
These mechanisms address data exposure and entail distinct operational costs. Homomorphic encryption and secure multi-party computation have offered strong guarantees at the expense of computation and communication, thereby impacting the performance of resource-constrained devices such as smartphones and IoT devices. Statistical approaches such as differential privacy are lightweight but introduce a trade-off between the strength of protection and model utility. Secure aggregation, model distillation, and regularisation remain lightweight with their own limitations regarding dropout handling, teacher-model dependency, and the risk of underfitting.
A large body of literature evaluates these mechanisms based on the strength and formality of the protection they provide. However, less emphasis on systematic comparison on performance metrics: computational overhead, communication cost, impact on model accuracy, and scalability they impose on federated learning system that is directly useful for practitioners selecting mechanism for a given deployment setting, the paper proposed to address this gap by conducting a comparative analysis of the performance of existing data protection techniques and propose mechanism that can be adopted in order to improve the data protection performance in federated learning systems. The analysis was guided by the following questions: 1) How commonly are data protection mechanisms used in federated learning systems compared in terms of computational overhead? 2) How do existing mechanisms compare in terms of communication overhead? 3) What is the impact of each mechanism on model accuracy and utility? Moreover, 4) How well does each mechanism scale to large, heterogeneous, and resource-constrained federated learning deployments?
2. Methods
The study adopted a systematic comparative analysis of the literature; the approach was chosen because the mechanisms under review span heterogeneous implementations, hardware assumptions, and federated learning frameworks, making a single controlled experimental comparison across all the data protection mechanisms within the study’s scope impractical. The comparative approach synthesizes performance-relevant findings reported across studies and carefully evaluates each mechanism, considering the common performance criteria.
2.1. Threat Model and Protection Objective
The protection objective is to prevent adversaries from interfering with the client raw data training during the federated learning, model updates, gradients, and aggregated parameters. Three threat actors were considered in this review: 1) an honest-but-curious central server, the actor follows protocol correctly and attempts to infer client data from received updates; 2) an external eavesdropper which intercepts client-server communication; and 3) a malicious client attempting to corrupt and infer information about participants. The attacks relevant to “protection” were update and gradient leakage, model inversion, and membership inference, which determines whether a sample belongs to the client’s training set. Poisoning and backdoor attacks were referenced whenever they intersect with a mechanism’s confidentiality guarantees.
2.2. Mechanism Selection
A total of eight data-protection mechanisms were selected for analysis based on their frequent use in federated learning and their conceptual distinctness. Mechanisms analysed were differential privacy, homomorphic encryption, secure multi-party computation, secure aggregation protocols, FedAvg with secure aggregation, model distillation, regularization techniques, and client-side data anonymization. Mechanisms were selected because they represent major categories of data-protection approaches used in federated learning, such as statistical approaches (differential privacy), cryptographic approaches (homomorphic encryption, secure multi-party computation, secure aggregation), architectural/algorithmic approaches (FedAvg, model distillation, regularization), and data-level approaches (client-side anonymization). FedAvg, model distillation, and regularization are optimization methods: developed to improve convergence, communication efficiency, and generalization, while the methods can incidentally change the content of the client information being transmitted, none provide a formal confidentiality guarantee comparable to differential privacy’s (ϵ, σ) bound or cryptographic guarantees underlying homomorphic encryption, secure multi-party computation, and secure aggregation. Table 1 summarise classification of the mechanisms.
Table 1. Mechanism classification by protection type.
Mechanism |
Category |
Protection Guarantee |
Design Purpose |
Differential
Privacy |
Data protection
(statistical) |
Yes—(ϵ, σ) bound |
Bound information leakage via
calibrated noise |
Homomorphic
Encryption |
Data protection
(cryptographic) |
Yes—computational hardness |
Computation on encrypted data
(Not exposing raw data) |
Secure Multi-Party
Computation |
Data protection
(cryptographic) |
Yes—computational
hardness/information-theoretic |
Joint computation without revealing
individual inputs |
Secure Aggregation |
Data protection
(cryptographic/architectural) |
Yes, conditional on protocol and
non-collusion assumptions |
Prevent server from observing
individual plaintext updates |
Client-Side Data
Anonymization |
Data protection
(data-level) |
No formal guarantee; heuristic |
Obscure identifying features in
data prior to training |
FedAvg |
Training/optimization
method (baseline) |
None |
Efficient aggregation of local
model updates |
Model Distillation |
Training/optimization
method |
None |
Transfer knowledge from teacher to
student model; reduce communication |
Regularization |
Training/optimization
method |
None |
Improve generalization, reduce
overfitting |
2.3. Literature Sources
Sources that address data protection were identified through a literature search across Google Scholar, Semantic Scholar, PubMed, Springer Nature, Research Gate, ScienceDirect, IEEE, Scilit, ACM digital library, Wiley online library, SciSpace, National foundation (.gov), HAL open science database, open Ukrainian citation index, open review, Iniria, and nature.com from inception to March 2026. These sources were selected based on their relevance to the operational behaviour of each mechanism, with greater emphasis on the reported effects on computation time, communication overhead, model accuracy, and behaviour under scale and/or heterogeneity. Search terms combined the federated learning with mechanism-specific term and a performance-related term described in the pattern: (“federated learning”) AND (“overhead” OR “performance” OR “scalability” OR “communication cost” OR “computational cost”) The eight mechanism-specific term sets used were: “differential privacy”; “homomorphic encryption”; “secure multi-party computation” OR “SMPC”; “secure aggregation”; “FedAvg” OR “federated averaging”; “model distillation” OR “knowledge distillation”; “regularization”; and “data anonymization” OR “client-side anonymization.”
Inclusion criteria: 1) Source that address federated learning explicitly; 2) source that reports at least one of the four performance dimensions (computational overhead, communication overhead, accuracy/utility impact, or scalability) for one of the eight mechanisms, either empirically or through direct technical/theoretical analysis; 3) the source was a peer-reviewed journal or conference article, or an arXiv preprint reporting a complete technical contribution rather than a work-in-progress abstract.
Exclusion criteria: Articles addressing only federated learning without a reported performance evaluation were excluded.
Titles and abstracts for articles and reports were screened, followed by full-text eligibility assessment. A total of 100 records were identified across the four mechanism-and-performance search patterns; after de-duplication and eligibility screening, 38 unique sources met the inclusion criteria and were cited (Table 2). This is a targeted, illustrative synthesis of the literature and should be read as representative of reported performance patterns, not as a complete census of the literature.
2.4. Comparison Framework
Each mechanism was evaluated against four performance metrics identified to reflect the practical issues that are most relevant to federated learning deployment. The performance metrics considered were:
1) Computational overhead, which is the additional processing burden the mechanism places on client devices and/or the aggregation server.
2) Communication overhead, which is the additional bandwidth/number of communications rounds the mechanism requires relative to unprotected federated learning.
3) Impact on model accuracy or utility, which is the degree to which the mechanism degrades the predictive performance of the trained model.
4) Scalability, which describes how the mechanism’s overhead behaves as the number of participating clients grows and as client capability and connectivity become more heterogeneous.
Reported findings across sources were synthesized into a qualitative rating (low, moderate, high, or very high overhead; poor, moderate, or good Scalability) and avoiding the use of a single quantitative benchmark figure since the underlying studies use different models, datasets, and hardware, making it not directly numerically comparable. The qualitative synthesis was presented in (Table 2).
2.5. Rating and Coding Rule
Each mechanism was described with a qualitative overhead rating for Table 2 using a fixed coding rule applied consistently across the four performance dimensions.
1) Computational and communication overhead ratings
i) Low: No additional cryptographic computation or communication is required beyond FedAvg aggregation. Synthesized sources reported negligible overhead under approximately 10% relative to baseline.
ii) Moderate: No superlinear increase (bounded) in computation and communication (noise calibration, masking, and key exchange) with client count or model size. Synthesized sources reported overhead in the range of 10% - 50% relative to baseline or qualitatively described as “moderate” or “manageable”.
iii) High: Linear increase of intensive operations with the number of participants. Synthesized sources reported an overhead of 50% relative to baseline, and others described it as a primary practical deployment barrier.
iv) Very High: Prohibitive overhead for real-time or resource-constrained deployment in the majority of the sources analyzed for that mechanism (due to large ciphertext expansion factors in homomorphic encryption).
2) Scalability rating
i) Poor: overhead increases with client count, and the mechanism also requires a trusted central coordinator, which can become a bottleneck as the number of participants increases.
ii) Moderate: overhead grows linearly and is manageable at the cross-setting scale, but is not extensively validated at the cross-device scale in the cited sources.
iii) Good: overhead reported as sub-linear or constant in client count, or explicitly validated at a scale exceeding 100 simulated clients in the cited sources.
Resolving Conflicting Findings
Where the source analyzed reported different overhead levels for the same mechanism under different conditions (for instance in secure aggregation’s communication overhead varying with client dropout rate), Table 2 rating reflects the overhead reported under typical, stable operating conditions, low-to-moderate dropout, cross-setting, or moderately sized cross-device settings as described by the majority of cited sources for that mechanism. Conditions under which overhead diverges substantially from this typical case (for instance, high client churn, very large ciphertexts) were described in Sections 3 and 4 so that rating can reflect a central tendency rather than a best or worst-case figure.
3. Results
This section presents a summarized comparative performance profile of the eight mechanisms across the four-evaluation metrics: computation overhead, communication overhead, accuracy and utility, and scalability of the mechanisms.
3.1. Computational Overhead
The literature analysis shows a clear separation between cryptographic and non-cryptographic mechanisms in terms of computational cost. Homomorphic encryption was found to impose the highest computational burden among the mechanisms reviewed in the study due to the complexity of performing arithmetic directly on ciphertext, which is limiting on resource-constrained client devices, including smartphones and IoT devices. Secure multi-party computation incurs high computational costs due to secret sharing and garbled-circuit operations. Differential privacy, secure aggregation protocols, FedAvg, model distillation, regularization, and client-side anonymization have low-to-moderate computational overhead given that their operation of noise addition, masking, local averaging, knowledge transfer, penalty terms, and data generalization, respectively, are lightweight, which does not require many rounds of computations (Table 2).
3.2. Communication Overhead
Communication costs across the data protection mechanisms show a largely common pattern: homomorphic encryption, for instance, ranks highest when encrypted payloads are larger than plaintext model updates. Secure multi-party computation incurs high communication costs due to multiple rounds of interaction among participants. In contrast, secure aggregation protocols and FedAvg have a small communication overhead, which is attributed to key exchange and masking rounds, and/or to the size and frequency of model update transmissions. On the other hand, client-side anonymization, regularization, and model distillation have a lower communication overhead among the mechanisms reviewed, since they either transmit smaller artifacts (distilled knowledge, sparse updates) or incur no additional transmission cost (Table 2).
3.3. Impact on Model Accuracy and Utility
Regarding model accuracy and utility, data protection mechanisms show greater divergence, where differential privacy has a direct, well-documented utility trade-off with smaller privacy budgets (ϵ, σ), providing stronger data protection but degrading model performance, particularly on small and/or non-independent and identically distributed (non-IID) datasets. On client-side anonymization, adding noise and generalization obscure data patterns that are important for learning. Homomorphic encryption and secure multi-party computation have minimal direct effect on model accuracy, given that they preserve exact computation over protected data without approximation. Model distillation’s effect on accuracy depends on the quality of the teacher model, and regularization can improve generalization and, in some instances, cause underfitting if applied aggressively (Table 2).
3.4. Scalability
Table 2. Comparative performance of data-protection mechanisms in federated learning.
Mechanism |
Computational
Overhead |
Communication
Overhead |
Impact on Model
Accuracy/Utility |
Scalability |
Differential Privacy
(DP) [1]-[7] |
Low-moderate
(noise injection,
gradient clipping) |
Moderate
(noisy updates,
clipping) |
Degrades with smaller
privacy budget (ϵ); worse on
small/non-IID datasets |
Moderate; harder to
tune uniformly across
heterogeneous clients |
Homomorphic
Encryption (HE)
[8]-[12] |
Very high (complex
cryptographic
operations) |
Very high
(ciphertext far
larger than plaintext) |
Minimal direct accuracy loss,
but limited to supported
operations |
Poor in decentralized
settings; key management
does not scale well |
Secure Multi-Party
Computation (SMPC)
[5] [13]-[15] |
High (secret sharing,
garbled circuits) |
High (frequent
multi-round
interactions) |
Minimal direct accuracy loss |
Poor; overhead grows
exponentially with
participants |
Secure Aggregation
Protocols [10] [16]-[18] |
Moderate (masking/key
exchange) |
Moderate (multi-round
key sharing/unmasking) |
Minimal accuracy loss
under stable participation |
Good for large-scale
FL, but degrades with
dropout and heterogeneity |
Federated Averaging
with Secure Aggregation
(FedAvg) [19]-[24] |
Low on clients
(local computation) |
Moderate-high
(frequent large model
transmissions) |
Robust to non-IID data;
No confidentiality guarantee |
Good; widely used baseline |
Model Distillation
[25]-[31] |
Low on client
(smaller student model) |
Low (smaller updates
transmitted) |
Dependent on teacher model
quality; can lose nuance |
Good; well suited to
resource-constrained devices |
Regularization
Techniques [32]-[34] |
Low-moderate
(extra tuning cost) |
Low (can shrink update
size, for instance L1 sparsity) |
Improves generalization
but risks underfitting if
over-applied |
Good; lightweight to
deploy |
Client-Side Data
Anonymization [35] |
Low-moderate
(masking/generalization) |
Low |
Can obscure useful data
patterns, reducing accuracy |
Moderate; local computation
strains weaker devices |
Analysis shows that scalability varies widely across all data protection mechanisms. Secure multi-party computation and homomorphic encryption were found to scale poorly. Overhead grows substantially, and in secure multi-party computation grows exponentially as the number of participants increases, making decentralized key management increasingly difficult without a central trusted authority. Secure aggregation protocols scale favourably; however, their sensitivity remains to client dropout and device heterogeneity. FedAvg, model distillation, and regularization are reported as the most scalable of the mechanisms analysed, which is consistent with their design that emphasizes lightweight, resource-conscious operation, which is suited to large and heterogeneous client populations (Table 2).
4. Discussion
The results points to a consistent pattern: mechanisms that provide the strong formal or cryptographic protection: homomorphic encryption, and secure multi-party computation, which tend to impose great performance cost, while lightweight architectural and statistical mechanisms: secure aggregation, FedAvg, model distillation, and regularization offer more favourable performance profiles but achieve protection through different, generally less exhaustive, means [36]. This is not a new observation in isolation, but framing it explicitly along four performance dimensions clarifies where the trade-offs are sharpest and where they are more manageable.
4.1. Cryptographic Mechanisms: Strong Guarantees, High Cost
Homomorphic encryption and secure multi-party computation both emerge from this analysis as the least performance-friendly mechanisms, principally because their protection guarantees depend on computationally expensive operations encrypted arithmetic in the case of homomorphic encryption, and secret-sharing protocols in the case of secure multi-party computation. Both also face acute scalability problems in decentralized FL settings, where there is no single trusted authority to coordinate key management. These findings suggest that, from a purely performance-oriented standpoint, cryptographic mechanisms are best suited to smaller-scale, higher-value deployments for example, cross-silo FL among a small number of well-resourced institutional participants rather than large-scale, cross-device deployments involving thousands of resource-constrained clients.
4.2. Statistical and Architectural Mechanisms: Lower Cost,
Different Trade-Offs
Differential privacy has comparatively low computational and communication overhead, making the mechanism attractive. It introduces a direct, tunable trade-off against model utility. This makes practitioners scruple over the difficulty of tuning the privacy budget correctly under non-IID data distributions, where the noise disproportionately affects clients with smaller datasets or, in some instances, an imbalanced dataset. Secure aggregation protocols, FedAvg, model distillation, and regularization represent the most performance-favourable cluster of mechanisms analysed. Secure aggregation achieves meaningful protection at a fraction of the computational and communication cost of full cryptographic methods: homomorphic encryption and secure multi-party computation, but remains sensitive to client dropout, which is a challenge in cross-device federated learning. Model distillation and regularization introduce small overhead across the mechanisms analysed in this study and directly reduce communication costs by producing smaller distilled models or sparser regularized updates. However, they depend on auxiliary quality factors: teacher-model quality and careful hyperparameter tuning captured by overhead metrics.
4.3. Practical Implications for Mechanism Selection
The study’s findings suggested that mechanism selection in FL should be dictated by deployment settings rather than a single protection-strength metric, especially in large-scale, cross-device deployments on heterogeneous, resource-constrained clients such as mobile and IoT devices. Findings also indicate that providing better data protection, lightweight mechanisms, secure aggregation, FedAvg, model distillation, or regularization implemented individually or combined, cryptographic on the other hand, is sufficient where implementation is in smaller-scale, cross-silo deployments with well-resourced participants, where the highest level of confidentiality is paramount, communication and compute budgets are less constrained. Therefore, cryptographic mechanisms: homomorphic encryption and secure multi-party computation are viable despite their overhead. Differential privacy offers a middle path, at a cost of requiring careful, context-specific tuning of the privacy budget to avoid unacceptable utility loss.
4.4. Limitations
The analysis was based on a synthesis of the reported findings across heterogeneous studies and the qualitative ratings in Table 1 should be interpreted as directional comparisons rather than precise, universally applicable figures. Reported overheads depend heavily on implementation details, model architecture, dataset characteristics, and hardware, all of which vary across the underlying literature. In addition, several mechanisms are frequently deployed in combination for instance differential privacy being layered secure aggregation. Future work applying a controlled, common testbed across mechanisms would help validate and refine the directional findings presented in this study.
5. Conclusion
The study carried out systematic comparative analysis of eight data-protection mechanisms in Federated learning, evaluated based on their performance metrics, including computational overhead, communication overhead, model accuracy, and scalability. Study analysis found that there is a consistent trade-off pattern where cryptographic mechanisms: homomorphic encryption, secure multi-party computation, provide a strong protection at high computational, communication, and scalability cost, while secure aggregation, FedAvg, model distillation, and regularization offer more favourable performance, which is suited to large-scale and resource-constrained deployments. Differential privacy has low overhead but comes with a direct, tunable trade-off against model utility. The findings offer practical guidance for practitioners selecting data-protection mechanisms for deployment and also highlight the need for standardized, controlled benchmarking across mechanisms to complement literature-based approaches. Future work can conduct empirical evaluations of mechanisms for the hybrid combination across varied federated learning systems, including cross-device deployments, to validate and extend the directional findings of this study.
Acknowledgements
The author would like to thank the School of Informatics and Innovative Systems at Jaramogi Oginga Odinga University of Science and Technology for providing a conducive environment for conducting this research. Richard Omolo Newton Masinde for their constructive suggestions and comments.
Author Contributions
Mitende Nicholus Nyapete: Conceptualization, Data curation, Formal Analysis, Funding acquisition, Investigation, Methodology, Resources, Software, Validation, Visualization, Writing—original draft, Writing review & editing.
Richard Omolo: Supervision, Writing—review & editing.
Newton Masinde: Supervision, Writing—review & editing.