
PASS5 SCORES
+81.3%
RESPECTIVELY
+93.8%
GEM5
+83.0%
GEM5
+100.0%
Imagine you're a locksmith who has to figure out how a lock works — but you can't open the case. All you can do is insert keys and listen to the clicks. That's black-box microarchitectural reverse engineering: you feed a processor carefully designed test programs, measure timing or cache behavior, and infer internal design decisions the manufacturer never published. Security researchers do this to find Spectre-class vulnerabilities. It's painstaking, manual, and demands deep expertise in both hardware design and low-level programming. ProcReverser replaces that human loop with a multi-agent LLM system. The committed claim: a multi-agent LLM pipeline can autonomously design experiments, write test programs, collect hardware measurements, and correctly identify microarchitectural properties relevant to speculative-execution attacks — achieving 81.3% first-try accuracy and 93.8% within five attempts on the XuanTie C910, and 83.0%/100.0% on the gem5 O3 simulator. This is the first system to close the full loop from hypothesis generation to measurement-based property identification without human intervention. The architecture is a multi-agent orchestration pattern — think crew-of-specialists, not monolithic model. Distinct agents handle experimental planning, test-program generation, execution, and result analysis, passing structured artifacts between stages. Critically, the system uses execution feedback: when a test program fails or measurements are ambiguous, the agents iterate on the plan. The underlying LLMs are not specified in the abstract, but the design leans on tool-use and code-generation capabilities now standard in frontier models. The compute property being exploited is the LLM's ability to translate natural-language hardware hypotheses into runnable C/assembly and then reason about numeric timing data — a task that previously required a PhD-level human. The task suite covers speculative execution, branch prediction, memory dependence, and cache behavior — the four pillars of Spectre-class attack surfaces. This is not a toy benchmark. These are the properties a real attacker or defender needs to characterize when assessing a new chip. The evaluation spans three targets: the XuanTie C910 (a real RISC-V core from Alibaba's T-Head), the gem5 O3 CPU model (a widely-used academic simulator where ground truth is known), and the SiFive P550 (a commercial RISC-V chip). The gem5 result — 100% Pass@5 — is the strongest because ground truth is fully available, but the C910 and P550 results on real silicon are what matter for practical security work. Integrity is mixed. The gem5 evaluation is essentially self-grading against a simulator whose internals are open, which is circular in the strictest sense but reasonable for validation. The C910 and P550 evaluations are closer to real-world conditions, but the task suite was designed by the authors — there's no independent, community-standard benchmark for microarchitectural reverse engineering tasks. The Pass@5 metric (best of five attempts) is generous; Pass@1 at 81-83% is the more honest headline. No code release or pre-registration is mentioned in the abstract. The milestone question is sharp: today's system characterizes known property categories on RISC-V cores. The next concrete threshold is whether such a system can discover a genuinely novel, previously-unknown microarchitectural vulnerability — not just confirm known property types. That's the difference between automated reconnaissance and automated discovery. We're at the reconnaissance stage. The gap to discovery likely requires scaling the task suite, integrating with fuzzing pipelines, and testing on x86/ARM targets where the attack surface is richer and more commercially consequential. The obvious experiment not run: x86 and ARM. Every major Spectre disclosure targeted Intel, AMD, or ARM cores. RISC-V is a valid proving ground, but the real test is whether ProcReverser generalizes to the architectures that run the world's servers, phones, and laptops. The honest read: the authors likely chose RISC-V because open-source toolchains and the gem5 model make iteration fast and ground-truth validation possible. x86 is harder, messier, and would require navigating proprietary toolchains. This is probably (a) — scope and tooling constraints — with a side of (c), saving the marquee target for the follow-up.
The analogy
Like a locksmith who can't open the case — you insert keys, listen to clicks, and infer the mechanism. ProcReverser replaces the locksmith's ear and intuition with LLM agents that design keys, listen to timing measurements, and deduce the lock's internals autonomously.
Blind locksmith
Insert keys, listen to clicks, infer internal pin arrangements without opening the case
ProcReverser
Run test programs, measure timing/cache behavior, infer microarchitectural properties without documentation
Honest assessment
ProcReverser is a genuinely novel system that automates a previously manual expert task, and the results on real silicon are credible. But the ladder is thin — there's no competing automated tool to compare against, the task suite is author-designed, and the commercially consequential targets (x86, ARM) are untested. This is a strong proof-of-concept for automated hardware security reconnaissance, not yet a tool for vulnerability discovery. Security researchers working on RISC-V or building LLM-agent toolchains should read this; everyone else should watch for the x86 follow-up.
What Is CLAIMS?
63
/ 100
BedrockNews applies a structured analytical framework called CLAIMS to academic papers and technical research releases.
How CLAIMS worksClaim
4/5
Ladder
2/5
Architecture
3/5
Integrity
3/5
Milestone
3/5
Successor
4/5
What is this paper actually committing to?
Genuine first: no prior system closes the full loop from hypothesis to measurement-based property ID autonomously. The claim is narrow (RISC-V, known property types) but honestly scoped and real.
The field fight
Can LLM-based agents replace deep manual expertise in hardware security analysis, or are they limited to pattern-matching on known vulnerability classes?
Side A
LLM agents are fancy test-case generators; real vulnerability discovery still requires human intuition and hardware expertise
Held by: Traditional hardware security community, manual RE practitioners
Side B
Autonomous agents with execution feedback can close the RE loop and eventually discover novel vulnerabilities, not just confirm known ones
Held by: ProcReverser, LLM-for-security proponents, agent-based tool builders
Milestone tracker
Unlocks: Automated vulnerability discovery pipeline that scales across chip families without per-target expert effort
Current
81-83%
Pass@1 accuracy
Known properties, RISC-V
Next
1
novel vulnerability
Discovery on x86/ARM
~2-3 years if integrated with fuzzing and tested on mainstream ISAs
Verified Facts (5)
ProcReverser achieves Pass@1 and Pass@5 scores of 81.3% and 93.8%, respectively, on C910
83.0% and 100.0% on gem5 O3
Evaluated on the XuanTie C910, the gem5 RISC-V O3 CPU model, and the SiFive P550
Covers speculative execution, branch prediction, memory dependence, and cache behavior
ProcReverser uses execution feedback to improve experimental plans and test programs
Practical Implications (3)
Security teams evaluating new RISC-V chips could use a ProcReverser-like system to rapidly characterize Spectre-class attack surfaces without deep microarchitectural expertise
Chip vendors could run automated RE against their own designs during pre-release security audits, catching speculative execution leaks before tape-out
The multi-agent feedback loop pattern is generalizable — expect similar architectures applied to firmware RE, protocol fuzzing, and other expertise-heavy security tasks
More latest news
Read all