Read the paper
TRIAGE-CPG: A Cross-Model Study of Budgeted Static Malware Triage with Large Language Models
on Zenodo
BEST MODEL FLAGS
+65%
MALWARE ESCALATES ANOTHER
+12%
CLEARS
+94%
BINARY. THATS
+4%
Imagine you're a building inspector who has to assess a 50-story skyscraper but can only open 40 doors. You need a strategy for which doors to open, what to look for behind each one, and when to stop looking and make a call: safe, suspicious, or condemned. The choice of inspector — cautious vs. aggressive, pattern-matching vs. checklist-driven — changes which doors get opened and what verdict comes back. That's the core mechanism of this paper: LLMs as building inspectors for malware, constrained by a door budget. The committed claim: when you give LLMs a code property graph of a decompiled Windows binary and a fixed budget of function inspections, they can produce useful benign/suspicious/malicious verdicts — but model identity is the dominant variable controlling the triage trade-off. No universal best configuration exists. The best model (unnamed in the abstract but among eight tested) achieves 0.805 strict balanced accuracy and 0.855 triage balanced accuracy after examining roughly 40 of a mean 1,000 functions per binary. That's a 4% sample rate producing actionable verdicts. The architecture is clean and deliberately LLM-agnostic. TRIAGE-CPG builds a code property graph from decompiled code, uses an LLM-free heuristic to select seed functions, then hands control to an LLM agent that expands the evidence dossier under budget. The pipeline's three-stage design — seed selection, budget-constrained expansion, verdict generation — means the LLM never sees the whole binary. It sees curated slices and must decide when it has enough evidence. This is budgeted evidence acquisition, not classification. The zero-shot protocol is identical across all eight models, which is the right call for commensurability. The ladder position is interesting but incomplete. The paper benchmarks eight models against each other but does not name a traditional static analysis baseline (YARA rules, heuristic scanners, or commercial AV engines) with comparable numbers. This is a cross-model comparison study, not a SOTA-beating claim. The honest read: the paper demonstrates that LLM-based triage is viable and characterizes its behavior, but we don't know if 0.855 tBA beats or trails what a well-tuned ClamAV or YARA ruleset would achieve on the same 600-sample test split. The absence of that comparison is conspicuous. Integrity is mixed-to-solid. The 1,000 PE binary dataset with a 600-sample test split is reasonable for a controlled study. The same zero-shot protocol across all models eliminates prompt-tuning as a confound. Code and artifacts are released via Zenodo, which is a strong signal. But there's no pre-registration, no independent replication, and the benchmark construction details matter enormously — how were the 1,000 binaries selected? Were families balanced? Were packed or obfuscated samples included? The paper's value hinges on dataset representativeness, and the abstract doesn't address this. The most useful finding isn't a single accuracy number — it's the triage policy spectrum. High-specificity models produce small, malware-rich escalation queues (fewer false alarms, more misses). Recall-oriented models escalate broadly (fewer misses, more analyst workload). Both closed-source and open-weight models appear across this spectrum. This means deployment teams can't just pick the "best" model — they must pick the model whose triage policy matches their operational appetite for false negatives vs. false positives. That's a genuinely practical insight. The obvious next experiment is scaling the function budget and measuring the accuracy curve — does 80 functions instead of 40 close the gap, or do returns flatten? The authors almost certainly have this data or could generate it cheaply, which suggests they're either saving it for a follow-on paper or the curve is uninteresting. The other missing piece is adversarial robustness: can a malware author add decoy functions that burn the LLM's budget on irrelevant code? If so, the entire triage-under-budget paradigm has a structural weakness that needs surfacing.
The analogy
You're a building inspector with a 50-story tower and permission to open only 40 doors. Your inspection strategy — which doors, what order, when to stop — determines whether you condemn or clear the building. Different inspectors with the same 40-door budget reach different verdicts, not because one is wrong but because they weight different evidence differently.
Building inspector
Opens 40 of 1,000 doors under a fixed budget, decides: safe, suspicious, or condemned based on partial evidence
LLM triage agent
Inspects ~40 of ~1,000 decompiled functions under a fixed budget, decides: benign, suspicious, or malicious based on partial code evidence
Honest assessment
A well-designed cross-model comparison that answers a question nobody had systematically asked: how do different LLMs behave as budget-constrained malware triagers? The finding that model identity drives triage policy is operationally useful. But the missing classical AV baseline is a serious omission — without it, we know LLMs can triage but not whether they should. Security teams should read this for the pipeline design and the policy-spectrum insight, not as evidence that LLMs outperform existing tools.
What Is CLAIMS?
67
/ 100
BedrockNews applies a structured analytical framework called CLAIMS to academic papers and technical research releases.
How CLAIMS worksClaim
4/5
Ladder
2/5
Architecture
4/5
Integrity
3/5
Milestone
3/5
Successor
4/5
What is this paper actually committing to?
Genuine first for systematic cross-model budgeted triage on decompiled binaries. The claim that model identity drives triage policy is novel and operationally useful, not incremental.
The field fight
Can LLMs perform operationally useful malware analysis on decompiled code, or are they still limited to toy demonstrations?
Side A
LLMs lack the precision and robustness for real-world security triage; traditional static analysis and sandboxing remain superior
Held by: Classical AV vendors, YARA community, most SOC practitioners
Side B
LLMs can interpret decompiled code at scale and provide useful triage signals, especially under budget constraints where exhaustive analysis is impossible
Held by: Google security research, this paper, LLM-for-security startups
Milestone tracker
Unlocks: Evidence that LLM triage adds value beyond existing static analysis tools in real SOC workflows
Current
0.855
tBA
best model, ~40 functions
Next
0.90+
tBA
head-to-head vs commercial AV
~6-12 months with released pipeline and standard AV comparison
Verified Facts (6)
Eight closed-source and open-weight models evaluated on 1,000 Windows PE binaries
600-sample test split with same zero-shot protocol for every model
Highest-scoring model reaches strict balanced accuracy of 0.805 and tBA of 0.855
Models analyze about 40 of a mean 1,000 functions per binary
Best model flags 65% of malware, escalates another 12%, and clears 94% of benign samples
Code and artifacts available at https://doi.org/10.5281/zenodo.22771897
Practical Implications (4)
Security teams selecting LLMs for triage must match model triage policy (high-specificity vs. recall-oriented) to their operational tolerance for false negatives vs. false positives
The released TRIAGE-CPG pipeline can be used as a reproducible benchmark for future LLM-based malware analysis work
Budget-constrained analysis of ~4% of functions is a viable paradigm for scaling LLM code review to large binaries
SOCs should not expect a universal 'best model' — the clear/flag/escalate ratio shifts with model choice and must be tuned to the queue
More latest news
Read all