You know how a GPS gives you the fastest route, but you actually want the one that avoids highways, passes a coffee shop, and doesn't make you parallel park? Those preferences aren't hard requirements — you'll survive without them — but ignoring them gives you a route nobody wants. That's the gap between hard-constraint optimization (the math must hold) and soft-constraint optimization (the human preferences should hold, with penalties when they don't). Most LLM-based autoformulation research has quietly treated every optimization problem as if it only has hard constraints, which is like building a GPS that only knows about road closures and speed limits but never about your preferences. ASTOR's committed claim: a grammar-guided abstract syntax tree (AST) framework that lets LLMs translate natural-language optimization problems — including soft constraints — into executable code, outperforming existing single-request and multi-agent LLM methods on the authors' new SoftOR benchmark while staying competitive on hard-constraint-only benchmarks. The key architectural move is forcing the LLM's output through a formal grammar before compilation, which eliminates an entire class of formatting and syntax errors that plague direct code-generation approaches. The ladder here matters. Prior autoformulation work — OptiMUS, chain-of-thought prompting, multi-agent pipelines — has been evaluated almost exclusively on problems with hard constraints only (NL4Opt, MAMO, and similar). ASTOR introduces SoftOR as a unified benchmark specifically for soft-constraint formulations, then shows best aggregate accuracy on SoftOR while remaining competitive on existing hard-constraint benchmarks. The honest caveat: 'competitive' on hard-constraint benchmarks means it doesn't clearly dominate there, and the SoftOR benchmark is the authors' own creation with no independent adoption yet. Architecturally, ASTOR sits in the structured-decoding family — think constrained beam search, LMQL, or grammar-masked generation — but applied at a higher abstraction level. Instead of constraining token-by-token generation, it constrains the LLM to produce a valid AST that maps to a formal grammar, then deterministically compiles that AST into optimization code (likely targeting solvers like Gurobi or CPLEX). This is inference-time only — no fine-tuning, no RL — which means it's model-agnostic and can ride improvements in base LLMs for free. The integrity picture is mixed. SoftOR is a welcome contribution, but the benchmark is self-authored and the anonymous review link suggests this is a conference submission (likely ICLR). Code and data are available via anonymous repository, which is good practice. However, the baselines compared are existing LLM methods, not hand-crafted optimization models by domain experts — the real ceiling. There's no pre-registration, and the benchmark selection was clearly designed to highlight soft-constraint capabilities where prior work is weakest. The milestone question is practical: autoformulation accuracy needs to be high enough that a planner or operations researcher trusts the output without manually verifying every constraint. We're not there yet. Current methods hover in accuracy ranges where human review is still mandatory. The threshold that matters is something like 95%+ constraint-correct formulations on realistic multi-constraint problems, which would make LLM-assisted optimization genuinely useful for non-experts. ASTOR advances toward this but the gap remains substantial. The obvious experiment not run: testing ASTOR on real-world optimization problems sourced from actual planners and managers, rather than curated benchmark problems. The authors likely didn't run this because (a) collecting real industrial problems with ground-truth formulations is expensive and slow, and (b) messy real-world problem descriptions would probably expose failure modes that clean benchmarks don't. This is understandable at the benchmarking stage but it's the experiment that would actually validate the practical motivation the paper leads with.
