BOTH REGULARISERS
+73.6%
EWC
+71.1%
COMPARED ABOUT
+32%
REPLAY ACHIEVES
+46.1%
Imagine you're a chef who learns a new recipe every week but forgets how to make last month's dishes. You could try mental tricks to retain the old recipes (regularization), or you could just keep a small index card with key steps for each past dish and glance at it occasionally (replay). This paper tests both strategies for network intrusion detection systems that must learn new attack types sequentially — and finds the index cards do almost all the work. The committed claim: combining Elastic Weight Consolidation (EWC) or Learning without Forgetting (LwF) with a class-size-adaptive replay buffer of just 20–300 samples per previous class can partially mitigate catastrophic forgetting in class-incremental intrusion detection across three public datasets. But the authors are unusually candid — they do not claim the regularizers themselves add meaningful value beyond the replay component, because their fine-tuning baseline had no replay at all. The experimental setup uses a 1D CNN trained over nine sequential tasks, each introducing one new attack category, across UNSW-NB15, ToN-IoT, and CIC-IDS2017. On UNSW-NB15 (the tuning set), EWC+replay reaches 73.6% and LwF+replay reaches 71.1% final accuracy, versus ~32% for sequential fine-tuning alone. The transfer story is uneven: on CIC-IDS2017, LwF+replay hits 91.6%, but on ToN-IoT it drops to 46.1%. EWC+replay performs worse on both transfer datasets (61.9% and 23.7%). Hyperparameters were frozen after tuning on UNSW-NB15, which is honest but explains the cross-dataset variance. The architectural choice — a 1D CNN rather than a transformer or recurrent model — is pragmatic for tabular network flow data but limits the generality of the findings. The replay buffer is class-size-adaptive (20 to 300 samples), which is a sensible engineering choice but means the buffer size is a confounded variable across tasks. The regularization methods (EWC penalizing movement from important weights via Fisher information; LwF using knowledge distillation from the old model's soft outputs) are standard continual learning tools, neither of which was designed for this domain. The integrity picture is mixed. Using three public benchmarks is good practice, and fixing hyperparameters on one dataset before transferring is a genuine cross-validation discipline. However, all results come from a single random seed, the fine-tuning baseline lacks replay (making replay-augmented methods look better by default rather than isolating the regularizer's contribution), and the authors acknowledge both limitations plainly. The forgetting metric critique — showing it can be misleading when a model never learned the old classes in the first place — is a useful methodological contribution that may outlast the main results. The honest bottom line: this paper demonstrates that replay matters and regularizers might not, but it doesn't have the experimental controls to prove the latter claim rigorously. A replay-only baseline (buffer without EWC or LwF) is the glaring missing experiment. Without it, we cannot separate the regularizer's contribution from the buffer's. The most likely reason it wasn't run: the authors realized mid-project that replay was doing the work and pivoted to honesty rather than claim inflation — which is admirable but leaves the ladder incomplete. For practitioners building production IDS systems that must update without full retraining, the actionable takeaway is narrow but real: a small replay buffer (a few hundred samples per old class) is cheap and effective. Whether you need EWC or LwF on top of it remains genuinely open. The field needs replay-only baselines, multi-seed runs, and architecture comparisons before anyone should deploy these regularizers with confidence.
The analogy
You're a chef who learns one new recipe per week but forgets last month's dishes. You could try memory palace tricks (regularization) to retain old recipes, or you could keep a small index card with key steps for each past dish (replay buffer). This paper tests both — and the index cards do almost all the work.
Index cards
A chef keeps a few key steps per past recipe and reviews them when cooking, preventing forgetting without complex mental strategies
Replay buffer
A CNN stores 20-300 exemplars per previous attack class and replays them during training, preventing catastrophic forgetting without relying on regularization alone
Honest assessment
An honest but preliminary study that applies established continual learning methods to intrusion detection and finds — candidly — that replay buffers do the heavy lifting. The missing replay-only baseline is a critical gap that prevents the central question from being answered. The forgetting-metric critique and the cross-dataset transfer discipline are the most valuable contributions. Useful as a pilot for practitioners considering continual IDS, but not yet a basis for deployment decisions.
What Is CLAIMS?
40
/ 100
BedrockNews applies a structured analytical framework called CLAIMS to academic papers and technical research releases.
How CLAIMS worksClaim
1/5
Ladder
1/5
Architecture
3/5
Integrity
2/5
Milestone
1/5
Successor
4/5
What is this paper actually committing to?
Incremental, not novel. Applying EWC and LwF to IDS is a straightforward domain transfer of established continual learning methods. The candid admission that regularizers may not help is the most interesting contribution.
The field fight
In continual learning for sequential tasks, does regularization (EWC, LwF, SI) add value beyond simple experience replay?
Side A
Regularization provides complementary protection against forgetting that replay alone cannot match
Held by: EWC (Kirkpatrick et al.), LwF (Li & Hoiem), PackNet
Side B
Replay buffers do most of the work; regularization is overhead with marginal benefit
Held by: Experience Replay baselines, GDumb, this paper (implicitly)
Milestone tracker
Unlocks: Deployable continual IDS that updates without full retraining or catastrophic accuracy loss
Current
73.6
% accuracy
EWC+replay on UNSW-NB15
Next
90
% accuracy
All three datasets consistently
Unknown — requires replay-only baseline, multi-seed, and architecture search
Verified Facts (8)
On UNSW-NB15, replay improves the final accuracy of both regularisers to 73.6% (EWC) and 71.1% (LwF) compared to about 32%
LwF with replay achieves 46.1% and 91.6% final accuracy for ToN-IoT and CIC-IDS2017
EWC with replay achieves 23.7% and 61.9% for ToN-IoT and CIC-IDS2017
Class-size-adaptive rehearsal buffer of 20% of all previous classes (from 20 to 300 samples)
CNN with one-dimensional (1D) architecture trained over nine sequential tasks, each introducing one attack category
Hyper-parameters were selected using UNSW-NB15 and fixed for the other two datasets
Fine-tuning baseline was not replayed and all results were based on a single seed
It is not claimed that the regularisers provide any extra support over replay alone
Practical Implications (4)
A replay buffer of 20-300 samples per previous attack class is cheap to maintain and dramatically outperforms no-replay fine-tuning for sequential IDS updates
EWC and LwF hyperparameters tuned on one IDS dataset do not transfer well — expect significant accuracy drops on new network environments without re-tuning
Standard forgetting metrics can be misleading when a model never learned old classes well; practitioners should track per-class accuracy, not just aggregate forgetting scores
Before adding regularization complexity to a production IDS pipeline, test whether a simple replay buffer alone achieves comparable results
More latest news
Read all