Mathematical Foundations of Federated Learning for Privacy-Preserving Analytics

Lewis Feisal Injai  ·  Masters of Engineering, Data Science & Mathematics  ·  UCLA
Capstone Project  ·  Cathay Bank, Los Angeles  ·  March 2023 - November 2023

Abstract

This paper presents a federated learning system for training fraud detection models across geographically and regulatorily distinct jurisdictions without transmitting raw customer records across borders. We make four technical contributions. First, we derive a closed-form expression for the optimal number of local gradient steps $\tau^*$ by minimising the non-IID FedAvg convergence bound over a fixed communication round budget. Second, we implement differentially private gradient aggregation via the Gaussian mechanism, with privacy composition tracked through Rényi Differential Privacy (RDP) under full client participation, without invoking the client-subsampling amplification lemma, which does not apply to this topology. Third, top-$k$ gradient sparsification reduces per-round bandwidth by 50x with bounded convergence impact. Fourth, we implement Byzantine-robust aggregation via a variance-triggered median fallback, with the detection threshold derived from sub-Gaussian tail bounds and the adversarial hiding radius formally characterised.

We evaluate on a simulated eight-client topology structured to match Cathay Bank's branch network. Without differential privacy, the federated model reaches 97.3% AUC against a centralised baseline of 98.1% and an average local-only baseline of 91.4% (macro-averaged across 8 models evaluated on the global held-out test set). Under RDP composition with noise multiplier $z = 10.0$ over $T = 80$ rounds, the conservative full-participation privacy cost is $\varepsilon \approx 4.7$ at $\delta = 10^{-5}$, at which point AUC is 96.2%. We distinguish explicitly between the theoretical subsampling-amplified regime (where $\varepsilon = 1.0$ is achievable) and the non-amplified bound applicable to small fixed-topology deployments, and recommend the latter for regulatory disclosure.

A note on scope. The primary contribution is not the fraud detection AUC. A centralised model with pooled data reaches 98.1% and is trivially better on that metric. The contribution is the simultaneous analytical treatment of four coupled constraints - aggregation frequency, communication compression, differential privacy composition, and Byzantine robustness - within a single system. Each is typically addressed in isolation in the literature. The value of this work is their joint derivation and the formal guarantees that result: convergence bounds, privacy budget, and adversarial robustness are each analytically grounded rather than empirically asserted.

1. Introduction

Financial institutions operating across multiple regulatory jurisdictions face a structural constraint in machine learning system design. The data most informative for fraud detection is distributed across geographic boundaries, yet applicable data protection law prohibits centralised pooling. In California, the Consumer Privacy Act (CCPA) restricts transfer of customer data outside established data governance frameworks; in China, the Personal Information Protection Law (PIPL) imposes localisation requirements that effectively bar cross-border aggregation. Institutions have generally responded by either accepting weaker jurisdiction-local models or constructing costly cross-border data transfer agreements that carry residual compliance exposure.

Federated learning (FL) offers a structurally different approach: clients train local models and transmit only gradient updates to a central server, with raw data never leaving any node. This satisfies CCPA and PIPL by construction. Three engineering challenges arise in any real deployment. Data heterogeneity across clients produces gradient divergence that degrades global optimisation. Gradient updates can leak private information about training records. Transmitting high-dimensional gradient vectors across a wide-area network is bandwidth-intensive. A system that addresses only one or two of these is not deployable; a regulatory auditor will ask about all three simultaneously.

Section 3 derives an analytically optimal aggregation frequency under non-IID conditions. Section 4 establishes the differential privacy framework with honest composition accounting under full client participation. Section 5 covers communication compression. Section 6 formalises the Byzantine robustness mechanism. Section 7 reports experimental results with fully specified baselines.

2. System Architecture

The system comprises $N = 8$ clients, each representing a distinct jurisdiction or branch type in Cathay Bank's network. Client $i \in [N] = \{1, \ldots, 8\}$ holds a local dataset $\mathcal{D}_i$ of transaction records. Datasets are non-overlapping and are never transmitted. A central aggregation server holds no training data; it receives gradient updates, aggregates them, and broadcasts the updated global weights.

The fraud detection model is a feedforward network ($47 \to 128 \to 128 \to 1$): an input layer of $d_{\text{in}} = 47$ features, two hidden layers of 128 units with ReLU activations, and a sigmoid output for binary classification. The input vector $x \in \mathbb{R}^{47}$ is constructed from the following explicit feature groups:

Feature GroupEncodingDimensions
Transaction amountRaw continuous value1
Log-transformed amount$\log(1 + \text{amount})$, stabilises gradient variance across regional economies1
Merchant category codesOne-hot across 12 high-risk MCC groups12
Time of daySine/cosine of hour fraction, eliminates boundary discontinuity at midnight2
Day of weekSine/cosine of weekday fraction2
Velocity: 7-day windowRolling transaction count1
Velocity: 30-day windowRolling transaction count1
Card-present indicatorBinary1
Channel: digital/alternativeBinary1
Cross-border flagBinary1
Account-level historical statistics24 summary statistics: mean, standard deviation, 25th/50th/75th percentiles of 30-day amounts; rolling dispute count; historical chargeback ratio; 16 additional long-term risk profile fields24
Total47

The cyclical encodings for time and day are deliberate: raw integer hour values create an artificial discontinuity (23:59 appears far from 00:00 in Euclidean space), which distorts gradient contributions from temporal fraud clusters. Sine/cosine pairs remove this artefact. The log-transformed amount mitigates the effect of high-value outlier transactions on gradient magnitude, which is particularly relevant when clipping norms are set uniformly across clients with different regional transaction scales.

The total parameter count follows directly from the layer dimensions:

Parameter Count
$$d = \underbrace{(47 \times 128 + 128)}_{6{,}144} + \underbrace{(128 \times 128 + 128)}_{16{,}512} + \underbrace{(128 \times 1 + 1)}_{129} = 22{,}785$$
Verified via sum(p.numel() for p in model.parameters() if p.requires_grad) in PyTorch. Each term covers the weight matrix and bias vector for one layer.

All notation used throughout the paper is collected here. $w^{(t)} \in \mathbb{R}^d$: global model weights at round $t$. $\eta > 0$: learning rate. $\tau$: local SGD steps per round. $T$: total communication rounds. $C$: gradient clipping norm. $\sigma$: Gaussian noise standard deviation. $z = \sigma/C$: noise multiplier. $\Gamma$: gradient divergence parameter. $L$: loss smoothness constant. $\lambda$: RDP order. $\sigma_g^2$: stochastic gradient variance. $\theta$: Byzantine detection threshold. Each training round proceeds as follows: the server broadcasts $w^{(t)}$ to all clients; client $i$ runs $\tau$ steps of SGD on $\mathcal{D}_i$, producing update $\Delta_i^{(t)} = w_i^{(t,\tau)} - w^{(t)}$; the server clips, perturbs, and aggregates the updates; the global model advances to $w^{(t+1)}$. The implementation used PyTorch for local training and PySyft 0.5 for federated coordination, validated on a three-node distributed setup spanning two geographic regions.

3. Convergence Analysis and Optimal Aggregation Frequency

The IID assumption does not hold here. Fraud patterns in Cathay's Hong Kong operations differ structurally from those in Los Angeles: the merchant ecosystem, card-present ratio, and transaction velocity norms produce divergent local loss landscapes. Averaging gradients trained on these different distributions pulls the global model toward conflicting local optima, a problem known as client drift. The degree of drift depends directly on how many local steps each client runs before synchronisation: more steps mean greater divergence from the global optimum before the next correction.

Standard practice is to tune $\tau$ empirically by grid search, then retune periodically as distributions shift. This approach is fragile and offers no signal about when retuning is needed. We instead derive $\tau^*$ analytically from the convergence bound.

Assumptions (following Li et al., 2020)

A1 (L-smoothness). The global loss $F: \mathbb{R}^d \to \mathbb{R}$ satisfies $\|\nabla F(w) - \nabla F(w')\| \leq L\|w - w'\|$ for all $w, w' \in \mathbb{R}^d$, with $L > 0$.

A2 (Bounded stochastic variance). Stochastic gradients are unbiased with bounded variance: $\mathbb{E}\|\nabla \ell(w; \xi) - \nabla F_i(w)\|^2 \leq \sigma_g^2$ for all $i \in [N]$ and all $w$.

A3 (Bounded gradient divergence). $\frac{1}{N}\sum_{i=1}^N \|\nabla F_i(w) - \nabla F(w)\|^2 \leq \Gamma^2$ for all $w$. The parameter $\Gamma \geq 0$ quantifies non-IID severity; $\Gamma = 0$ recovers the IID case.

A4 (Bounded initialisation gap). $F(w^{(0)}) - F^* < \infty$, where $F^* = \inf_w F(w)$.

Under A1-A4, the FedAvg convergence bound for a fixed round budget $T$ takes the form (Li et al., 2020, Theorem 3):

FedAvg Convergence Bound — Non-IID, Fixed Round Budget
$$\frac{1}{T} \sum_{t=0}^{T-1} \mathbb{E}\|\nabla F(w^{(t)})\|^2 \;\leq\; \underbrace{\frac{2[F(w^{(0)}) - F^*]}{\eta \tau T}}_{\text{(I) initialisation gap}} + \underbrace{\eta L \sigma_g^2}_{\text{(II) gradient noise}} + \underbrace{6\eta^2 L^2 \tau^2 \Gamma^2}_{\text{(III) client drift}}$$
Term (I) scales as $\tau^{-1}$: more local steps reduce the per-round initialisation penalty. Term (III) scales as $\tau^2$: more local steps amplify drift. Term (II) is independent of $\tau$. The competing signs of the $\tau$ dependence in (I) and (III) guarantee an interior minimum.

We minimise the right-hand side over $\tau$ for fixed $T$. Term (II) vanishes under differentiation, leaving:

First-Order Condition
$$\frac{\partial}{\partial \tau}\left[\frac{2[F(w^{(0)}) - F^*]}{\eta \tau T} + 6\eta^2 L^2 \tau^2 \Gamma^2\right] = -\frac{2[F(w^{(0)}) - F^*]}{\eta \tau^2 T} + 12\eta^2 L^2 \tau \Gamma^2 = 0$$
The two contributions are $-2(F_0 - F^*)/(\eta \tau^2 T)$ from term (I) and $+12\eta^2 L^2 \tau \Gamma^2$ from term (III).

Setting equal to zero and collecting $\tau$ terms:

Derivation — Solving for $\tau^*$
$$12\eta^2 L^2 \tau^3 \Gamma^2 = \frac{2[F(w^{(0)}) - F^*]}{\eta T} \;\implies\; \tau^3 = \frac{F(w^{(0)}) - F^*}{6\eta^3 L^2 T \Gamma^2}$$
The cubic in $\tau$ arises from term (I) contributing $\tau^{-2}$ under differentiation and term (III) contributing $\tau^{+1}$. Setting their sum to zero gives $\tau^3 = \text{const}$.
Optimal Local Steps
$$\tau^* = \left(\frac{F(w^{(0)}) - F^*}{6\eta^3 L^2 T \Gamma^2}\right)^{1/3}$$
$\tau^*$ decreases in $T$ (more rounds available, fewer local steps optimal per round), decreases in $\Gamma$ (more homogeneous data tolerates longer local runs), and decreases in $L$ (more sharply curved loss surfaces benefit from faster re-synchronisation). The $\eta^3$ in the denominator is consistent: factors of $\eta$ appear in the original terms (I) and (III) at orders $\eta^{-1}$ and $\eta^2$ respectively, yielding $\eta^3$ in the denominator after rearrangement.
On Budget Parameterisation

The cubic root arises only when $T$ is held fixed and the bound minimised over $\tau$. If instead the total compute $S = T\tau$ is fixed, substituting $T = S/\tau$ into term (I) gives $2(F_0 - F^*)\tau/(\eta S)$, which scales as $\tau^{+1}$. Under that constraint, both (I) and (III) are increasing in $\tau$, so the first-order condition is linear and has no interior minimum: the bound is minimised at $\tau = 1$, i.e., full synchronisation every step. The fixed-$T$ formulation reflects the actual operational constraint of this system, where the number of communication rounds is the binding resource.

For Cathay's topology, we estimate $\Gamma \approx 0.31$ from the variance of client gradients in early training rounds, $L \approx 1.8$ by finite differences of the training loss, $\eta = 0.01$, and $F(w^{(0)}) - F^* \approx 0.74$. At $T = 80$, the formula yields $\tau^* \approx 17$. The empirical grid search settled on $\tau = 20$; the analytically derived schedule produces marginally tighter terminal gradient norm reduction over the same round budget. More practically, when the fraud distribution shifts mid-deployment, $\Gamma$ can be re-estimated from the observed gradient variance distribution and $\tau^*$ updated without further search.

4. Differential Privacy

Gradient updates leak information about training records. Zhu et al. (2019) showed that individual training samples can be reconstructed with high fidelity from a single gradient update, in some architectures approaching pixel-level accuracy for image data. For transaction records, the exposure is analogous: a gradient from a fraud detection model can reveal which transactions were present in a client's training batch. This is not a distant regulatory concern; it is the category of risk that CCPA and PIPL data minimisation requirements are designed to address. We apply the Gaussian mechanism at the aggregation layer to provide a formal guarantee.

Definition 1 — (ε, δ)-Differential Privacy

A randomised mechanism $\mathcal{M}: \mathcal{D} \to \mathcal{R}$ satisfies $(\varepsilon, \delta)$-differential privacy if for all $D, D' \in \mathcal{D}$ differing in exactly one record and all measurable $S \subseteq \mathcal{R}$:

$$\Pr[\mathcal{M}(D) \in S] \;\leq\; e^{\varepsilon} \cdot \Pr[\mathcal{M}(D') \in S] + \delta$$

$\varepsilon \geq 0$ bounds the log-odds shift in output distribution due to any single record. $\delta \geq 0$ is a failure probability, set to $10^{-5}$ throughout. Smaller $\varepsilon$ implies stronger privacy.

Gradient clipping. Before transmission, client $i$ clips its gradient update $g_i$ to L2 norm $C$:

Gradient Clipping
$$\tilde{g}_i = g_i \cdot \min\!\left(1,\; \frac{C}{\|g_i\|_2}\right)$$
Clipping bounds the L2 sensitivity of the aggregation function: $\Delta_2 f = \max_{D,D'}\|f(D)-f(D')\|_2 \leq C$. We use $C = 1.0$.

We then add calibrated Gaussian noise to the clipped gradient before it leaves the client. The noise standard deviation must satisfy:

Gaussian Mechanism — Single-Round Noise Floor
$$\sigma \;\geq\; \frac{\sqrt{2\ln(1.25/\delta)} \cdot C}{\varepsilon_0}$$
This guarantees $(\varepsilon_0, \delta)$-DP for a single gradient release, where $\varepsilon_0$ is the per-round budget. The noise multiplier $z = \sigma/C$ parameterises the mechanism; we use $z = 10.0$, giving $\sigma = 10.0$ at $C = 1.0$.

Composition across rounds. Each of the $T = 80$ training rounds constitutes one query against the private data. Naive $(\varepsilon, \delta)$-DP composition gives $\varepsilon_{\text{total}} = T\varepsilon_0$, which is prohibitively loose. Rényi Differential Privacy (RDP) provides tighter composition by tracking the Rényi divergence between output distributions across mechanisms.

Definition 2 — Rényi Differential Privacy (Mironov, 2017)

$\mathcal{M}$ satisfies $(\lambda, \alpha)$-RDP if for all adjacent $D, D'$ and all $\lambda > 1$:

$$D_\lambda(\mathcal{M}(D) \| \mathcal{M}(D')) \;\leq\; \alpha$$

where $D_\lambda$ is the Rényi divergence of order $\lambda$. $T$ applications of an $(\lambda, \alpha)$-RDP mechanism compose to $(\lambda, T\alpha)$-RDP. Conversion to $(\varepsilon, \delta)$-DP: $\varepsilon(\delta) = T\alpha + \log(1/\delta)/(\lambda-1)$, minimised over $\lambda$.

For the Gaussian mechanism with noise multiplier $z$, the per-round RDP guarantee at order $\lambda$ is $\alpha(\lambda) = \lambda/(2z^2)$. After $T$ rounds under full client participation:

Total Privacy Cost — Full Participation, No Subsampling
$$\varepsilon_{\text{total}}(\delta) = \min_{\lambda > 1} \left[\frac{T\lambda}{2z^2} + \frac{\log(1/\delta)}{\lambda - 1}\right]$$
At $T = 80$, $z = 10.0$, $\delta = 10^{-5}$: the minimising order is $\lambda^* \approx 6.4$, yielding $\varepsilon_{\text{total}} \approx 4.7$. This is the operative privacy figure for regulatory disclosure.
On Subsampling Amplification

Privacy amplification by subsampling (Balle et al., 2018) can substantially reduce the effective $\varepsilon$ in FL systems where only a random fraction of clients participates per round. In that setting, the probability that any given client's data affects a round's output is reduced by the subsampling rate, tightening the privacy bound. The gain is significant: a subsampling rate of 10% can reduce effective $\varepsilon$ by roughly a factor of $\sqrt{q}$ where $q$ is the fraction sampled, which would bring $\varepsilon$ close to 1.0 at the parameters used here.

This amplification does not apply to the present system. All $N = 8$ clients participate in every round; the subsampling rate is 1.0. Invoking the amplification lemma here would be incorrect. The $\varepsilon_{\text{total}} \approx 4.7$ reported above reflects the actual, non-amplified privacy cost. In contexts where a tighter bound is required by regulation, the client pool must be enlarged to permit genuine subsampling, or the noise multiplier increased substantially (reaching $\varepsilon \approx 1.0$ over 80 rounds under full participation requires $z \gtrsim 25$, with meaningful AUC cost).

With $N = 8$ clients each adding independent noise, the noise contribution to the mean-aggregated gradient scales as $\sigma/\sqrt{N} = 10.0/\sqrt{8} \approx 3.54$, while honest gradient signal averages coherently. The signal-to-noise ratio at the aggregated update is lower than in the single-client case, but sufficient for useful model convergence at this client count.

5. Communication Efficiency

A dense float32 gradient update for the $d = 22{,}785$ parameter model requires $22{,}785 \times 4 = 91{,}140$ bytes (89 KB) per client per round, or 6.95 MB per client over 80 rounds. This is manageable for the present prototype but grows linearly with model size and round count. We apply top-$k$ gradient sparsification throughout.

Top-$k$ sparsification. Define $\text{Top}_k: \mathbb{R}^d \to \mathbb{R}^d$ as the operator retaining the $k$ entries of its argument with the largest absolute values and zeroing the rest. At $k = 0.01d \approx 227$, each client transmits 227 gradient values and their corresponding 32-bit indices:

Communication Cost — Sparse vs Dense
$$\text{Bytes}_{\text{sparse}} = k \times (4_{\text{value}} + 4_{\text{index}}) = 227 \times 8 = 1{,}816\text{ bytes} \approx 1.77\text{ KB}$$
Compression ratio: $89\text{ KB} / 1.77\text{ KB} \approx 50\times$. Over 80 rounds, total per-client transmission under sparsification is 142 KB versus 6.95 MB dense.

The convergence penalty from dropping $(1 - k/d) = 99\%$ of gradient components is bounded by an additive term proportional to $(1-k/d)$ in the convergence rate (Alistarh et al., 2017). At $k/d = 0.01$, this term is negligible relative to the leading convergence terms from Section 3. The sparsified system requires four additional rounds to reach the same loss threshold as the dense system. Sparsification is applied after clipping and noise injection, so the DP guarantee of Section 4 applies without modification by the post-processing theorem.

Dense update Sparse update (top-k, k=1%)
Communication cost: dense 91,140 bytes vs sparse 1,816 bytes per client per round — 50x reduction.

Top-k sparsification (k=0.05%) transmits only the coordinates with highest gradient magnitude. Over 80 rounds across 8 clients, total bandwidth drops from 6.95 MB to 142 KB per client.

6. Byzantine-Robust Aggregation

FedAvg with mean aggregation is vulnerable to Byzantine clients: participants that transmit arbitrary or adversarial gradient updates. A single client sending a gradient of large norm in a targeted direction can dominate the mean and steer the global model. Gradient clipping (Section 4) partially mitigates this by bounding each update's norm to $C = 1.0$, but a clipped adversarial update pointing in an adversarial direction still biases the mean persistently across rounds.

Coordinate-wise median. For each coordinate $j \in [d]$, we define the median aggregator as:

Coordinate-Wise Median
$$\hat{g}_j = \text{median}\!\left(\Delta_{1,j}^{(t)},\, \ldots,\, \Delta_{N,j}^{(t)}\right), \quad j = 1, \ldots, d$$
With at most $f < N/2$ Byzantine clients, the median satisfies $|\hat{g}_j - \bar{g}_j^{\text{honest}}| \leq \max_{i \in \text{honest}}|\Delta_{i,j}^{(t)} - \bar{g}_j^{\text{honest}}|$: no Byzantine client can shift $\hat{g}_j$ outside the range of honest values. Convergence at $\mathcal{O}(1/\sqrt{T})$ is preserved under A1-A4 (Yin et al., 2018).

The median's asymptotic variance is $\pi\sigma^2/2$ versus $\sigma^2$ for the mean under Gaussian noise (efficiency factor $\pi/2 \approx 1.57$). In the non-adversarial setting this reduces the effective gradient signal, slowing convergence per round. We avoid this cost except when Byzantine activity is detected.

Hybrid detection. Let $\mathbf{v}^{(t)} = (\|\Delta_1^{(t)}\|_2, \ldots, \|\Delta_N^{(t)}\|_2)$ be the vector of incoming update norms at round $t$. We compute rolling mean $\hat{\mu}_v$ and standard deviation $\hat{\sigma}_v$ over the preceding 10 rounds. The detection threshold is:

Byzantine Detection Threshold
$$\theta = \hat{\mu}_v + 3\hat{\sigma}_v$$
Median fallback triggers when $\max_i \|\Delta_i^{(t)}\|_2 > \theta$. Mean aggregation is used otherwise.

The $3\hat{\sigma}_v$ threshold is grounded in the sub-Gaussian tail bound. If honest gradient norms follow a sub-Gaussian distribution with parameter $\hat{\sigma}_v$, the probability that any single honest update exceeds $\hat{\mu}_v + 3\hat{\sigma}_v$ is bounded by $e^{-9/2} \approx 0.011$ per client per round. Across $N = 8$ clients and $T = 80$ rounds, the expected number of false positive triggers is at most $8 \times 80 \times 0.011 \approx 7$, or roughly 9% of rounds. Empirically, under non-adversarial conditions, we observed 6 threshold crossings over 80 rounds, consistent with this bound.

An adversary evading detection is constrained to submit updates with norm at most $\theta$. After clipping, the maximum norm is $C = 1.0$ regardless. For a single Byzantine client among $N = 8$, the worst-case perturbation to the mean-aggregated gradient is:

Bounded Impact of a Concealed Adversary
$$\left\|\frac{\Delta_{\text{adv}}}{N}\right\|_2 \;\leq\; \frac{C}{N} = \frac{1.0}{8} = 0.125 \text{ per round}$$
Over $T = 80$ rounds, the cumulative L2 perturbation to model weights is bounded by $\eta \cdot (C/N) \cdot T = 0.01 \times 0.125 \times 80 = 0.1$. This is the adversarial hiding radius: an adversary operating within it causes bounded, predictable model degradation rather than catastrophic drift. The false negative observed in simulation (adversarial norm set to $0.4C$) falls within this radius; the measured AUC impact was below 0.2 pp.

7. Experimental Setup and Results

Data. Synthetic training data was generated from distributional parameters supplied by Cathay Bank's data office - empirical feature distributions, fraud prevalence rates, and inter-branch heterogeneity statistics drawn from production records. No customer records were accessed at any stage; the data office provided statistical summaries within a bank-provisioned sandbox. The same data localisation requirements the system is designed to satisfy governed the research collaboration itself. Client partitions were produced via Dirichlet allocation with concentration parameter $\alpha = 0.5$, yielding meaningfully divergent local label distributions. Fraud prevalence was set at 1.8%. The held-out test set (20% of total) was stratified by client and class.

Baselines. We report three baselines. (1) Centralised: trained on the union of all client data with identical architecture; this is the performance ceiling, achievable only by violating data localisation requirements. (2) Local-only: each of the 8 clients trains independently with no federation; we report the macro-average AUC across all 8 models evaluated on the global stratified test set (not on local test partitions). This tests true generalisation variance rather than local memorisation. The per-client AUC range is 88.1%-94.3%; the macro-average is 91.4%. (3) Federated, empirical $\tau$: FedAvg with $\tau = 20$ determined by grid search, dense updates, no DP; this is the direct comparison for the analytical $\tau^*$ result.

Hyperparameters. $\eta = 0.01$; batch size 64; $C = 1.0$; $z = 10.0$ ($\sigma = 10.0$); $k = 227$ (1% of $d$). All federated results are macro-averaged over 5 independent seeds; standard deviation was below 0.003 AUC in all configurations.

ConfigurationAUCRoundsComm./Client$\varepsilon$
Centralised (pooled data)98.1%----None
Local-only (macro-avg, global test set)91.4%------
FedAvg, $\tau{=}20$ (empirical), dense, no DP97.3%806.95 MBNone
FedAvg, $\tau^*{=}17$ (analytical), dense, no DP97.3%806.95 MBNone
FedAvg, $\tau^*{=}17$, top-$k$ (1%), no DP97.1%84142 KBNone
FedAvg, $\tau^*{=}17$, top-$k$, DP ($z{=}10.0$)96.2%80142 KB$\approx 4.7$
All $\varepsilon$ values use RDP composition under full client participation ($N = 8$, all clients every round), $\delta = 10^{-5}$, with no subsampling amplification. The $\varepsilon \approx 4.7$ row is the appropriate figure for regulatory disclosure.
Local only FedAvg empirical τ FedAvg τ* (analytical) Full system + DP
Convergence curves: local only plateaus at 91%, empirical and analytical tau* converge identically around round 40, full DP system reaches 96.2% AUC at round 80.

Illustrative convergence trajectories consistent with the reported terminal AUC values in Table 1. Curves are schematic; the key result - that green and blue overlap, confirming τ* derived analytically matches the empirically tuned τ=20 - holds by construction from the experimental results. The DP penalty (gap between red and green) is 0.3 pp AUC. Real training logs will replace these curves in a subsequent revision.

Privacy-utility frontier. We evaluated AUC across $z \in \{5, 8, 10, 15, 20\}$ with $T = 80$ and $\delta = 10^{-5}$. Conservative RDP $\varepsilon$ values (full participation) are approximately $\{10.2, 6.0, 4.7, 3.0, 2.2\}$ respectively. AUC falls from 96.8% at $z = 5$ to 94.1% at $z = 20$. The $z = 10$ point offers meaningful noise calibration without catastrophic utility loss at this client count.

AUC vs ε Selected operating point (z=10, ε≈4.7)
Privacy-utility tradeoff: AUC drops from 96.8% to 94.1% as epsilon tightens from 10.2 to 2.2. Operating point at epsilon 4.7 yields 96.2% AUC.

Lower ε = stronger privacy guarantee. The selected point (z=10, ε≈4.7) sits at the inflection of the curve, where tighter noise calibration begins extracting substantial utility cost.

Byzantine robustness. Injecting a single adversarial client transmitting full-norm clipped updates in an adversarial direction collapses mean-only AUC to 84.1% over 80 rounds. The hybrid fallback limits the degradation to 1.2 pp (AUC 96.9%), with correct detection in 9 of 10 adversarial injection trials. The single false negative occurred at adversarial norm $0.4C$, within the theoretical hiding radius; the measured AUC impact was 0.18 pp, consistent with the cumulative perturbation bound of 0.04 in L2 norm derived in Section 6.

Aggregated gradient norm Detection threshold θ  Adversarial injection rounds
Byzantine detection: gradient norm spikes above threshold at rounds 20-22 and 50-52. Median fallback triggered each time; convergence resumes within 3-4 rounds.

Illustrative detection timeline consistent with the reported experimental outcomes: correct detection in 9 of 10 adversarial injection trials, with the single false negative at adversarial norm 0.4C. Spike magnitudes and round positions are schematic. Real gradient norm logs will replace this in a subsequent revision.

8. Limitations and Future Work

Synthetic data. All evaluations use synthetic data generated to match distributional statistics, not real transaction records. Real banking data exhibits temporal clustering, inter-client correlation, and distributional noise that smooth generative models do not reproduce. AUC figures here are likely optimistic relative to a live deployment. The analytic framework is designed to be data-agnostic, but the numerical results should not be read as production performance estimates.

Privacy accounting scope. The RDP analysis covers the gradient aggregation mechanism. It does not account for potential leakage through model inference on released global weights or through the sequence of checkpoint broadcasts over $T$ rounds. A complete end-to-end analysis would need to bound these additional surfaces.

Parameter estimation uncertainty. $\tau^*$ depends on $L$ and $\Gamma$, estimated empirically from early training dynamics. Both estimates carry uncertainty that propagates into $\tau^*$. The sensitivity of $\tau^*$ to perturbations in these estimates is not currently bounded; this is a gap in the theoretical coverage.

Sub-Gaussian assumption. The $3\hat{\sigma}_v$ Byzantine detection threshold assumes honest gradient norms are approximately sub-Gaussian, which holds when gradients are clipped and loss surfaces are smooth, but may fail in early training when gradient magnitudes are large. A distribution-free threshold via Chebyshev's inequality would be more conservative but more broadly valid: $\Pr[|X - \mu| \geq k\sigma] \leq 1/k^2$ gives a false positive rate of at most $1/9 \approx 0.11$ per client at $k = 3$, somewhat looser than the sub-Gaussian bound but requiring no tail assumptions.

Open directions include: extending the $\tau^*$ analysis to personalised FL where clients maintain mixtures of global and local parameters; deriving the joint optimum over $(T, \tau)$ when the round budget is itself a decision variable; replacing the RDP accountant with the numerically tighter PRV accountant (Gopi et al., 2021); and adapting the sparsification rate $k/d$ online based on the observed gradient magnitude distribution.

9. Conclusion

We presented a federated learning system for cross-jurisdictional fraud detection that addresses gradient divergence, privacy leakage, communication overhead, and Byzantine robustness simultaneously. The convergence schedule is analytically derived rather than empirically tuned. The privacy accounting is honest about the full-participation topology, reporting a conservative $\varepsilon \approx 4.7$ under RDP composition rather than invoking subsampling amplification that does not apply. Communication compression achieves 50x bandwidth reduction with a four-round convergence overhead. The Byzantine fallback bounds adversarial impact to 1.2 pp AUC degradation with a formally characterised hiding radius.

The full system reaches 96.2% AUC against a 98.1% centralised ceiling, with 142 KB total per-client communication over 80 rounds. Taken together, the formal guarantees in three dimensions simultaneously separate this from a demonstration: convergence bounds, privacy budget, and adversarial robustness are each analytically grounded rather than empirically asserted.

Acknowledgements

This work would not have taken the shape it did without Professor Xiaowu Dai (SCALE Lab). He opened the door to the Cathay Bank collaboration and remained a steady sounding board at every point where the engineering and the theory pulled in different directions. I am grateful for his generosity with both his network and his time.


References
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L. (2016). Deep learning with differential privacy. Proceedings of the 23rd ACM SIGSAC Conference on Computer and Communications Security (CCS).
Alistarh, D., Grubic, D., Li, J., Tomioka, R., and Vojnovic, M. (2017). QSGD: Communication-efficient SGD via gradient quantization and encoding. Advances in Neural Information Processing Systems (NeurIPS).
Balle, B., Barthe, G., and Gaboardi, M. (2018). Privacy amplification by subsampling: Tight analyses via couplings and divergences. NeurIPS.
Blanchard, P., El Mhamdi, E. M., Guerraoui, R., and Stainer, J. (2017). Machine learning with adversaries: Byzantine tolerant gradient descent. NeurIPS.
Gopi, S., Lee, Y. T., and Wutschitz, L. (2021). Numerical composition of differential privacy. NeurIPS.
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V. (2020). Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems (MLSys 2020).
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., and Agüera y Arcas, B. (2017). Communication-efficient learning of deep networks from decentralized data. AISTATS.
Mironov, I. (2017). Rényi differential privacy. 30th IEEE Computer Security Foundations Symposium (CSF).
Yin, D., Chen, Y., Kannan, R., and Bartlett, P. (2018). Byzantine-robust distributed learning: Towards optimal statistical rates. ICML.
Zhu, L., Liu, Z., and Han, S. (2019). Deep leakage from gradients. NeurIPS.