You know how your body's immune system works? It doesn't memorize one virus and call it done. Every new pathogen triggers an adaptive response — new antibodies — but the system also retains memory B-cells so it doesn't forget last year's flu while learning this year's. The problem with current LLM defenses is they're more like a single vaccine: one-shot alignment that works until the adversary mutates. COPA is the paper that says, what if we gave the model an immune system instead? The committed claim: treating prompt injection defense as a continual learning problem — with GRPO-based preference optimization and margin-weighted experience replay — reduces attack success rates by up to 6.3x and 4.4x on average versus state-of-the-art static defenses, without catastrophic forgetting of prior attack classes or degradation of general capabilities. The core mechanism is elegant in its framing if not its novelty. COPA uses Group Relative Policy Optimization (GRPO) to incrementally update the model's preferences when new attack types appear. The key structural choice is margin-weighted experience replay: when training against new attacks, the system replays examples from previously encountered attack classes, weighted by how close the model's defense margins are to failing. This directly targets catastrophic forgetting — the classic continual learning problem where learning task B destroys performance on task A. The weighting is the clever bit: instead of uniform replay, COPA prioritizes the old attacks the model is closest to forgetting. On the ladder, COPA claims a 6.3x reduction in attack success rate at best and 4.4x on average compared to unnamed "state-of-the-art defenses." The abstract doesn't name the specific baselines or give absolute attack success rate numbers — we get multipliers but not the underlying rates. A 6.3x reduction from a 63% baseline (to 10%) is very different from a 6.3x reduction from a 6.3% baseline (to 1%). The paper also distinguishes itself from "recent lifelong alignment methods" that handle shifting user preferences but not adaptive adversaries, which positions COPA against work like CPO and online DPO variants rather than purely static defenses. The architecture sits squarely in the RLHF/preference-optimization family — specifically the GRPO branch that emerged from DeepSeek's work as an alternative to PPO. The continual learning wrapper (experience replay with margin weighting) borrows directly from the catastrophic forgetting literature that goes back to elastic weight consolidation and progressive neural networks. The combination is the contribution: nobody has applied continual preference optimization specifically to the adversarial security domain with this replay mechanism. Whether that combination constitutes a new paradigm or a well-executed mashup is the honest question. The integrity picture has gaps visible even from the abstract. We don't know the specific attack benchmarks, the model sizes tested, whether the attack streams are synthetic or drawn from real-world jailbreak datasets, or whether the baselines include the strongest current defenses like circuit breakers or representation engineering. The "lifelong prompt injection attack streams" framing suggests a synthetic evaluation protocol designed by the authors, which means they're grading their own homework to some degree. The 6.3x number is a peak result, not a median — the 4.4x average is the more honest figure. The successor experiment that's conspicuously absent: testing against an adversary that is itself adapting to COPA in real time — a true red-team/blue-team co-evolution. The paper frames the problem as adapting to evolving attacks, but the evaluation almost certainly uses a fixed stream of attack classes presented sequentially, not an adversary that observes COPA's defenses and counter-adapts. This is the hard version of the problem, and my honest read is (a) — it requires a fundamentally different experimental setup with an adaptive red-team agent, which is a different paper's worth of infrastructure.
