Imagine you're a restaurant server who's been trained to ask better questions — 'How would you like your steak prepared?' instead of 'The usual?' Your questions improve, your technique sharpens, but the customer still just points at the menu and nods. The quality of the invitation changed; the social dynamic of the interaction did not. That's what happened here: automated feedback made volunteer math tutors measurably better at inviting student thinking, but it couldn't crack the 91.5% tutor-talk-time wall. The committed claim: automated discourse feedback delivered to volunteer peer tutors on Schoolhouse.world increases specific talk moves — eliciting learner ideas, uptake of learner contributions, and learner reasoning — by 8-15%, replicated across two independent four-week SAT math bootcamps. This is a genuine RCT at real scale: 1,030 tutors, 7,443 learners, 6,508 recorded sessions. The effect is robust. But the more ambitious claim — that adding learner-facing participation feedback would amplify these gains — falls flat. Tutor talk dominance didn't budge in any condition. The architecture is straightforward: NLP classifiers detect talk moves in recorded tutoring sessions, generate automated feedback dashboards for tutors (and in treatment arms, participation statistics for learners), and the study measures discourse changes across conditions. This sits in the family of automated classroom discourse analysis tools — think the work of Demszky's own prior systems or NCTE's Teaching Effectiveness framework — applied to a volunteer peer tutoring context rather than professional instruction. The compute demands are modest; the real engineering challenge is the feedback loop design and the ecological validity of deploying it in live sessions. The ladder matters here. Prior work (Demszky & Liu, 2023; Wang et al., 2024) showed automated feedback improves professional teachers' discourse in K-12 and online settings. This paper extends that to volunteer peer tutors — a population with less training and less authority — and still finds positive effects on talk moves. That's a meaningful extension. But the null result on talk balance is the honest, load-bearing finding. The control group's 91.5% tutor talk time is staggering — this isn't a classroom, it's a monologue with occasional audience participation. None of the interventions moved that number. Integrity is solid for an education RCT. Randomization was at the tutor level across two bootcamps (replication built into the design). The null findings are reported candidly — learner-facing feedback didn't work, and the authors don't spin it. Qualitative interviews with 17 tutors and 13 learners provide genuine explanatory depth: learners reported the feedback was group-level not individual-level, some preferred listening, and peer-to-peer interaction was structurally rare. The honest reporting of what didn't work is the paper's best feature. The milestone question is where the field gets interesting. The 8-15% improvement in talk moves is real but incremental. The next meaningful threshold is reducing tutor talk time below, say, 70% — a level associated with genuine dialogic instruction in the literature. This paper shows that feedback alone won't get there. The gap requires structural intervention: session formats that force turn-taking, smaller group sizes, or real-time nudges during (not after) sessions. That's a different engineering problem. The obvious successor experiment is real-time, in-session feedback — nudging tutors mid-conversation rather than between sessions. The authors almost certainly know this; the current system processes recordings post-hoc. Building a real-time pipeline is a significant engineering lift (latency, accuracy under noise, UX of mid-session interruptions), and the Schoolhouse.world platform may not support it yet. My read: this is (a) compute/engineering constraint combined with (c) saving it for the next paper. The other missing experiment is individual-level learner feedback rather than group-level — the interviews basically told them this was the problem, and it's a natural next step they'll likely pursue.
