
Read the paper
ViTeX-Bench: Benchmarking High-Fidelity Video Scene Text Editing
on Project Page
Imagine you're a copy editor, but instead of fixing typos in a document, you need to change the text on a moving storefront sign in a movie — while the camera pans, the lighting shifts, and pedestrians walk by. You can't reshoot. You can't freeze the frame. The new text has to look like it was always there, across every single frame. That's video scene text editing, and until now, there was no standardized way to measure whether anyone was doing it well. ViTeX-Bench is a benchmark, not a model — though the authors do release a reference model alongside it. The core contribution is a dataset of 387 real-world 720p videos with text-region masks and editing instructions, split into 230 training pairs (with pipeline-generated ground truth) and 157 frozen evaluation videos. The three-axis evaluation protocol measures text correctness (does the OCR read the right characters?), visual/temporal quality (does it look good and stay consistent across frames?), and edit locality (did the edit leave the rest of the scene alone?). Thirteen metrics total, one primary per axis, with Pareto analysis of trade-offs. The claim is specific and honest: this is the first benchmark suite that provides paired real-video data and a protocol designed to directly measure whether the requested text remains correct over time. Previous resources offered limited paired data, and general video-editing metrics don't capture character-level text accuracy across frames. The authors evaluated eight baselines spanning four editing families — and the headline result is that no method simultaneously achieves accurate text, temporal stability, and scene preservation. Their reference editor, ViTeX-Edit-14B, uses motion-aligned glyph-video conditioning and achieves a CharAcc of 0.688 — the highest among video-native editors tested. It also achieves the lowest text-crop Warp Error among raw editor outputs, meaning its text is the most temporally stable. But 0.688 character accuracy means roughly one in three characters is wrong. The bar is not cleared; the bar is measured. This is the distinction that matters. The integrity setup is relatively strong for a benchmark paper. The evaluation split is frozen (157 videos), OCR calibration is performed, human evaluation supplements automated metrics, and annotation-sensitivity analysis examines how mask quality affects scores. The authors acknowledge limitations in their pipeline-generated pairs for training (they're reviewed but not perfect). Code and data are released via a project page. Notably, the paper is accepted to the NeurIPS 2026 Datasets and Evaluations Track — peer review specifically geared toward benchmark rigor. Architecturally, the reference editor sits in the conditional video diffusion family — a 14B-parameter model fine-tuned on the paired training split with glyph-video conditioning that aligns text renderings with motion. The four baseline families tested span image-based editors applied frame-by-frame, video diffusion editors, instruction-following editors, and hybrid approaches. The Pareto analysis is the most useful part: it shows that methods optimizing for text accuracy tend to sacrifice temporal consistency, and vice versa. The field fight here is whether video text editing should be treated as a solved sub-problem of general video editing or as a distinct task requiring dedicated benchmarks. ViTeX-Bench argues forcefully for the latter — general video metrics miss character-level correctness entirely. The milestone to watch is CharAcc breaking 0.90 on this benchmark while maintaining competitive temporal quality scores. That would signal readiness for real-world applications like sign translation in video localization.
The analogy
You're a copy editor, but instead of fixing typos in a document, you must change the text on a moving storefront sign in a movie — camera panning, lighting shifting, pedestrians walking by — and the new text has to look native across every frame. ViTeX-Bench is the first standardized exam for grading that skill.
Spell-checker exam
A standardized test that checks whether corrections preserve the document's formatting, grammar, and original meaning — not just whether the fix is real English
ViTeX-Bench
A standardized benchmark that checks whether video text edits preserve temporal consistency, visual quality, and scene context — not just whether the text reads correctly in one frame
Honest assessment
ViTeX-Bench is a benchmark paper, and a good one — it defines the problem, provides paired real-world data, and honestly shows that no existing method solves it. The reference editor leads but with 0.688 CharAcc, 'leading' means 'least bad.' The real value is the frozen evaluation protocol and Pareto analysis — this gives the field a shared scoreboard. The dataset at 387 videos is small but sufficient to establish baselines; scaling it will be someone else's contribution. If you work in video editing, scene text, or video localization, this is immediately useful infrastructure.
What Is CLAIMS?
73
/ 100
BedrockNews applies a structured analytical framework called CLAIMS to academic papers and technical research releases.
How CLAIMS worksClaim
4/5
Ladder
4/5
Architecture
3/5
Integrity
4/5
Milestone
3/5
Successor
4/5
What is this paper actually committing to?
Genuine first for the specific niche: no prior benchmark combined paired real-video data with three-axis evaluation for video scene text editing. Not a new method claim — a new measurement infrastructure claim.
The field fight
Can general-purpose video editing metrics adequately evaluate task-specific edits like scene text replacement, or do specialized benchmarks expose failures that general metrics miss?
Side A
General video editing metrics (FVD, CLIP, temporal consistency) are sufficient; task-specific benchmarks fragment the field
Held by: General video editing community, Instruct-Pix2Pix, TokenFlow
Side B
Task-specific edits require dedicated metrics because general metrics don't capture character-level correctness or strict locality constraints
Held by: Scene text editing community, MOSTEL, this paper
Milestone tracker
Unlocks: Automated video localization — storefront signs, product labels, subtitles rendered natively into scene surfaces
Current
0.688
CharAcc
ViTeX-Edit-14B (best video-native)
Next
0.90
CharAcc
Deployment-grade accuracy
~2-4 years given video diffusion scaling trends
Verified Facts (8)
387 real-world 720p videos with text-region masks and editing instructions
230 provide reviewed, pipeline-generated paired edits for training, and 157 form a frozen evaluation split
13 metrics, with one primary metric per axis
ViTeX-Edit-14B achieves CharAcc 0.688, the highest mean among the evaluated video-native editors
Lowest comparable text-crop Warp among raw editor outputs
Eight baselines from four editing families
Accepted to NeurIPS 2026 (Evaluations and Datasets Track)
27 pages (10-page main text), 5 figures, 12 tables
Practical Implications (4)
Video localization teams now have a benchmark to evaluate whether their text replacement pipelines actually work across frames, not just on static crops
The Pareto analysis reveals a fundamental trade-off: text accuracy vs. temporal consistency — practitioners must choose which axis to optimize for their use case
The released ViTeX-Edit-14B provides an open-source baseline that other groups can fine-tune on domain-specific data without starting from scratch
OCR calibration methodology can be reused by any team building text-correctness metrics for video
More latest news
Read all