A Public Cyber-Physical Dataset for Power Grid Intrusion Detection — And the Uncomfortable Finding That High Recall Is Easy to Game

Data4Cyber provides labeled OT attack data across six scenarios on a real DMS testbed. The real contribution: proving that alarm saturation fakes detection quality.

Imagine you're a smoke detector manufacturer who tests products by setting them off in a room full of burning candles, incense, a fog machine, and an actual fire. Every detector screams constantly — and technically none of them "miss" the fire. You'd report 100% recall and ship the product. That's the state of intrusion detection in operational technology networks for power grids, and Data4Cyber exists to expose exactly this problem. The core claim: the authors built and released a labeled cyber-physical dataset captured from a laboratory distribution management system (DMS) that synchronizes OT network traffic (Modbus/TCP and MQTT packet captures), physical process measurements, controller state, and price incentive signals — all at a fixed 1-second time base with phase-resolved labels — across one benign baseline and six distinct attack scenarios. The attacks span protocol-semantic Modbus manipulation, man-in-the-middle false-data injection, and MQTT-based price manipulation. This is a dataset paper, not an algorithm paper, and the value proposition is infrastructure: giving the community something to build on that doesn't exist publicly today. The architecture of the testbed matters. This is a self-consumption optimization DMS with photovoltaic generation, battery storage, controllable loads, and substation metering — a realistic microgrid setup, not a toy. The synchronized capture across network, process, and control layers is the structural contribution. Most existing IDS datasets give you one channel (network traffic alone, or power measurements alone). Data4Cyber gives you all three locked to the same timeline, which is what you need to build detectors that understand context rather than just pattern-match on packet anomalies. The integrity picture is mixed in instructive ways. This is a laboratory testbed, not a field deployment — so the physical dynamics are controlled and repeatable, which is good for reproducibility but limited for ecological validity. The six attack scenarios are representative but not exhaustive; the authors chose them to span the attack taxonomy (protocol manipulation, false data injection, market-layer manipulation) rather than to maximize scenario count. The baseline evaluation is honest and revealing: they show that several detectors achieve high recall simply by saturating alarms, which is the smoke-detector-in-a-fog-room problem. This is the paper's sharpest insight. On the ladder, the relevant prior art includes the IEEE 118-bus synthetic datasets, the BATADAL water network dataset, and various SCADA-specific packet capture collections. Data4Cyber's advance is the multi-layer synchronization — network + process + control + market incentives at 1 s resolution with phase-resolved attack labels. No existing public dataset provides this combination. The limitation is scale: this is one DMS topology, one optimization objective, six scenarios. It's a proof of architecture, not a comprehensive benchmark. The milestone question is straightforward: the community needs datasets at this fidelity level covering dozens of topologies, hundreds of attack variants, and real field noise. Data4Cyber is the first rung. The next concrete number to watch is whether the dataset sees adoption in 10+ independent IDS evaluations within two years, which would establish it as a de facto benchmark. The successor experiment the authors didn't run — and almost certainly couldn't in a lab — is deploying the capture architecture on a live distribution grid with real DER assets and real adversarial conditions. That's a regulatory and liability barrier, not a compute barrier. The uncomfortable takeaway for practitioners: if your IDS evaluation relies on recall alone without measuring alarm-rate economics, you're grading with a broken rubric. Data4Cyber gives the community the raw material to fix that — but only if the community actually uses process-aware, multi-channel evaluation instead of defaulting to single-channel thresholding. The dataset is the contribution; the finding about alarm saturation is the wake-up call.