Imagine you hire a brilliant new associate who aced the bar exam, memorized the entire Internal Revenue Code, and can calculate depreciation schedules in their sleep. On their first day, a client walks in with a shoebox of receipts, a vague question about selling a condo, and a common-law marriage that may or may not be legally recognized in Florida. The associate freezes — not because they lack knowledge, but because nobody told them which problem to solve first. That is the gap DAYJOB: Tax is measuring. Surge AI's latest benchmark gives AI agents 50 full-scope tax assignments — not isolated questions or pre-structured returns, but end-to-end professional engagements with messy files, ambiguous instructions, and issues the client never thought to raise. The workspace is enormous: 1,789 files per benchmark (median 30 per assignment, up to 123), including PDFs, Excel workbooks, handwritten receipt photographs, dispatch logs, and cross-border cryptocurrency records. The agent must figure out what the actual problem is, not just answer the question asked. The headline result is sobering. Claude Sonnet 5.5 leads the leaderboard at 10.8% pass rate, followed by Claude Fable 5.1 at 9.6% and Claude Opus 5.5 at 9.2%. Pass is defined strictly: 95% of rubric criteria satisfied per task, covering calculations, supporting evidence, consequential issues, and the completed deliverable. These are not hallucination failures or arithmetic mistakes — they are failures of professional judgment. In the condo example, the agent correctly evaluated two proposed sale dates but never flagged the approaching deadline after which the client would lose a $250,000 tax exclusion. In the retirement-account case, Opus built an entire recommendation around the assumption that a couple was unmarried, despite evidence in the files suggesting a potentially valid common-law marriage from Texas. What distinguishes DAYJOB from prior tax benchmarks (TaxBench, TaxCalcBench, TaxLab) is scope of responsibility. TaxBench asks whether a model can answer a hard tax question — a defined problem with a defined answer. TaxCalcBench tests return accuracy against expected values. TaxLab tests whether an agent can gather missing information from a simulated taxpayer. DAYJOB asks all of these simultaneously and adds the hardest layer: issue identification. Can the agent recognize that the client asked the wrong question, that a receipt is missing, that field dispatch logs imply multistate nexus obligations the client never mentioned? The benchmark's construction adds credibility. Tasks were designed by tax professionals with backgrounds at PwC, Deloitte, EY, Baker Tilly, and Exxon, covering corporate taxation, partnership allocations, REITs, trusts, and multistate compliance. Each assignment underwent three-layer expert review for correctness, completeness, and solvability. The grading rubrics assess not just final answers but whether the agent identified consequential issues and provided adequate supporting evidence. The practical implication is clear: AI agents can perform individual tax subtasks competently but cannot yet own an engagement. The 10.8% ceiling is not a compute limitation — it is a judgment limitation. The systems lack the professional instinct to notice what is missing, question what is assumed, and flag what is consequential. For firms evaluating AI integration into tax practice, DAYJOB provides the first benchmark that tests what actually matters: not whether the machine knows the law, but whether it can do the work. The benchmark also surfaces an interesting model-level signal. On the condo deadline problem, Opus caught the expiring eligibility window in all five attempts while Astra missed it in all five. On the marital-status question, the pattern reversed — Astra flagged the need for verification in four of five runs while Opus plowed ahead with an unverified assumption. This suggests that current models have inconsistent professional judgment profiles, strong on some issue-recognition tasks and blind on others, rather than uniformly weak.
