You know that coworker who memorizes every fire-drill rule but freezes when actual smoke appears? That's Wazuh and OpenSearch. They have the rulebook — predefined signatures, statistical baselines — and they're excellent at recognizing things they've seen before. But hand them a login that's technically valid yet contextually weird — same user, new device, 3 AM, from a country they've never visited — and the rulebook doesn't have a page for that. This paper asks whether small instruction-tuned LLMs, the kind you can run on a single GPU, can read the room where rule engines cannot. The committed claim: a standardized instruction-based prompting framework lets off-the-shelf LLMs (7-20B parameters) classify endpoint authentication logs into normal, borderline, and anomalous categories, dramatically outperforming Wazuh rule-based detection (52.0% accuracy, 68.6% false negative rate) and OpenSearch statistical anomaly detection (49.3% accuracy, 74.5% false negative rate). Meta Llama 3.1 8B Instruct leads with 89.3% accuracy, 88.2% recall, 91.8% F1, and an 11.8% false negative rate. The borderline detection gap is the headline: Llama catches 80% of ambiguous cases vs. 20% for Wazuh and 15% for OpenSearch. The architecture is straightforward: take instruction-tuned LLMs — Llama 3.1 8B, Qwen 2.5 7B, GPT-OSS 20B — feed them structured prompts containing authentication log entries with endpoint-specific context, and ask for a severity classification. No fine-tuning, no RAG, no exotic pipeline. The method leans entirely on the pretrained instruction-following capability of these models plus careful prompt engineering. Qwen trades detection performance for speed (lowest inference latency) and reliability (100% structured-response validity). GPT-OSS shows strong classification when it produces valid responses, but that qualifier — "when valid responses were produced" — is doing heavy lifting. The integrity picture is where you need to slow down. The authors built their own controlled cybersecurity testbed to generate the evaluation dataset. This is both the paper's strength and its weakness. Strength: the data includes endpoint-specific authentication behaviors with ground-truth severity labels covering normal, borderline, and genuinely anomalous scenarios — something generic public datasets like HDFS or BGL logs don't offer for authentication. Weakness: the authors designed the scenarios, generated the data, labeled the ground truth, and then evaluated against it. Nobody else has seen this testbed. The validation is circular in the sense that the same team defined what counts as anomalous and then measured who detects it. There's no community benchmark, no independent replication, no pre-registration. The ladder comparison is honest in one direction and incomplete in another. Wazuh and OpenSearch are real production tools that SOC teams actually use — not stale academic baselines. That's good. But the comparison is unfair in an important way: Wazuh's rule-based engine was never designed to catch borderline semantic anomalies. Comparing an LLM's contextual reasoning to a regex-based SIEM on ambiguous cases is like comparing a chess engine to a calculator on chess problems. The more interesting comparison — against fine-tuned BERT/RoBERTa log classifiers, against LogBERT, against other LLM-based log anomaly detection papers — is either absent or not surfaced in the abstract. The milestone question is where this gets practical. Today we have 89% accuracy on a synthetic testbed with curated scenarios. The number that matters for production deployment is false-positive rate at scale — real enterprise environments generate millions of auth events per day, and even a 1% false positive rate means tens of thousands of alerts drowning the SOC. The paper doesn't report false positive rates prominently, and inference latency for an 8B model on every auth event is a hard constraint. The next milestone: demonstrate >95% accuracy with <0.1% false positive rate on a real enterprise auth log corpus of 1M+ events, with inference under 100ms per event. The obvious experiment not run: testing on a real production environment's logs, even a small one. The authors built an elaborate testbed instead. My read is (a) — access constraints. Getting real enterprise authentication logs with ground-truth labels is genuinely hard due to privacy, compliance, and the fact that ground truth for "was this login actually anomalous" barely exists outside of red-team exercises. The testbed is a reasonable workaround, but it means every result carries an asterisk until validated on wild data.
