A Recipe Book for Fake Cyberattacks: Model-Based Authoring Keeps SOC Training Exercises Internally Consistent

Uses a structured incident model plus LLM text generation to author SOC simulation exercises that stay consistent across alerts, evidence, tasks, and assessments. Feasibility demo, not scaling proof.

Imagine you're writing a murder mystery dinner party. You've got clues scattered across rooms, timelines that need to match, a detective's notebook that must reference evidence that actually exists, and quiz questions for players that can only be answered if the right clue was planted. Now imagine doing that for a cybersecurity training exercise — where a phishing email has to spawn a ransomware chain, every SOC alert needs to reference real artifacts, and every learner task has to connect to something observable. One wrong cross-reference and the whole scenario falls apart. That's the consistency problem this paper solves with a structured model. The committed claim: a model-based authoring method where a formal incident exercise model (events, objects, evidence, alerts, timeline entries, tasks, assessment items) constrains an LLM (GLM-5.3) that generates only the natural-language scenario text, keeping the structural skeleton consistent. The paper does NOT claim the LLM is doing anything novel — the LLM fills in alert descriptions from a spec. The contribution is the model architecture and the validation pipeline that checks cross-references, ordering, and completeness. The reference exercise is a phishing-to-ransomware scenario with 8 incident events, 8 objects, 6 evidence items, 7 alerts across 5 batches, 13 timeline entries (12 learner tasks), and 5 assessment items. The validator produces true-negative outcomes on valid data and true-positive outcomes on deliberately planted inconsistencies. This is the entire evaluation. There is no learner study, no comparison to hand-authored exercises, no measurement of educational outcomes, and no second scenario. Architecturally, this sits in the model-driven engineering family applied to educational content generation — think Eclipse Modeling Framework meets prompt engineering. The LLM is a generation endpoint, not the reasoning engine. The formal model does the structural heavy lifting: event sequencing, object-evidence linking, alert batching. The validator is a separate rule-based checker, not learned. This design choice is smart — it keeps the LLM's hallucination risk contained to prose style rather than structural logic. The integrity picture is thin. Validation is self-referential: the authors wrote the model, wrote the validator, wrote the reference data, and wrote the deliberately broken variants. No external evaluator, no learner deployment, no comparison to existing SOC exercise authoring tools (CyberRange, KYPO, or even manual authoring by instructors). The true-positive/true-negative framing sounds rigorous but is testing the validator against known-bad inputs the authors themselves injected — a necessary sanity check, not an independent evaluation. The milestone question is about scaling: can this method handle multi-day, multi-analyst exercises with branching decision trees and adaptive difficulty? The current exercise is a single linear scenario. Real SOC training platforms like SANS CyberCity or IBM's X-Force Command Center run exercises with dozens of concurrent incident streams. Getting from 8 events and 13 timeline entries to exercises with 50+ events and branching paths is the gap that would make this method genuinely useful to training providers. The obvious experiment the authors did not run is a learner study — even a small one with 10-15 students completing the exercise and measuring time-on-task, error rates, and subjective quality ratings against a manually authored control. This is almost certainly a resource constraint: building a Virtual SOC prototype and authoring the model framework consumed the paper's budget. A learner evaluation is the natural next paper, and the authors likely know it.