W8 DOM-diff Bench
w8-domdiff-bench: Browser Evidence Infrastructure for Agent Development
WootzApp Research
Agent evaluation usually starts after the trajectory has been captured. We ask the earlier question: what did the recording fail to preserve?
We ran the same 100 browser-agent tasks once and kept two records of each run: screenshots, and structured page state from W8, our Chromium-based browser. Across 639 audited rubric criteria, screenshots lacked the required evidence on 4.07% of criteria and W8 on 0.31%, 92.3% fewer gaps.
- Paired browser tasks
- 100
- Audited rubric criteria
- 639
- Screenshot-side missing evidence
- 26 (4.07%)
- W8-side missing evidence
- 2 (0.31%)
Track 01 / Evidence fidelity
92.3% fewer evidence gaps than screenshot recordings.
When a required value falls outside the captured viewport, when an endpoint is hidden in a structured field, or when an important control isn't legible, a screenshot-based verifier can only reason from incomplete evidence.
Our DOM-diff Optimized representation preserved the required source evidence more often in the audited paired comparison.
Exhibit 1 — Criteria with asymmetric missing source evidence. Lower is better.
| Representation | Missing-evidence criteria | Rate |
|---|---|---|
| Screenshot recording | 26 of 639 | 4.07% |
| W8 DOM-diff Optimized | 2 of 639 | 0.31% |
Relative reduction
(26 − 2) / 26 = 92.3%
Where evidence disappeared
Each confirmed gap is assigned one primary category, so counts sum without double counting.
Exhibit 2 — Source-evidence gaps by category.
| Source-evidence gap category | Screenshot | W8 DOM |
|---|---|---|
| Exact text and values | 11 | 0 |
| URLs | 3 | 0 |
| Control states | 1 | 0 |
| Table entries | 1 | 0 |
| Label–value relationships | 0 | 1 |
| Capture omissions | 10 | 1 |
| Total | 26 | 2 |
Preserved details · Table 4 of the report
Task 19 / Exact release date
The exact release date is explicit in the structured state (“02 Sep 17:40”). The screenshot record shows only a relative date (“2 weeks ago”).
Screenshot run ↗W8 run ↗Task 24 / Complete ranked list
Structured states preserve ranks 1–30. The captured screenshot frames cover ranks 1–29.
Screenshot run ↗W8 run ↗Task 75 / Exact endpoint URL
The exact endpoint URL is explicit in the structured header. It is not visible in the captured screenshot frames.
Screenshot run ↗W8 run ↗
- 01
Record once. Each task produces a trajectory of actions and ordered browser states.
- 02
Preserve both views. Capture screenshots and DOM-diff Optimized from the same execution.
- 03
Freeze the rubric. Both verifier pathways receive the same task, action history, final answer and frozen rubric.
- 04
Audit the sources. Review whether each criterion's required information actually exists in each representation.
- 05
Count asymmetric omissions. A criterion counts against a representation only when required evidence is missing there and present in the other.
Scope of the metric
The metric covers asymmetric evidence loss. Evidence missing from both representations, agent success, human agreement with judge scores, downstream learning gains, and performance on visual tasks are separate measurements.
The audit and scoring pipeline use model-based components and manual source review. The study measures evidence preservation. Deterministic checks versus LLM graders is the next benchmark.
Engineering measurement / Separate sample
Smaller representation, richer context.
4.84 KiB
Average compact browser-state record. In a separate historical sample of 650 recorded states, DOM-diff Optimized averaged 4.84 KiB of stored text per state, versus 44.88 KiB for detailed readable DOM text and 191.32 KiB for structured snapshots.
This compares stored size per state. Inference-token usage, verifier cost, and training time are separate measurements. The sample is separate from the 100-task evidence-loss study.
Post-training
A better reward starts with a better record.
A model judge cannot recover an exact value that was never saved. A deterministic checker also needs the relevant state to have been captured in the first place.
W8's work establishes a measurable reduction in missing browser source evidence. The next engineering step is to turn that evidence into task-specific checks for observable state and outcome changes, while retaining screenshot and human/model judgment when visual or subjective criteria demand it.
- MeasuredEvidence capture
- Next engineering stepCriterion checks
- Next engineering stepTrusted rewards
- Next engineering stepEvaluations and training
Sources
Inspect the evidence. Reproduce the result.
Put your hardest agent on the record.
Bring a workflow where existing traces or screenshots fail to explain what happened. We'll compare the evidence the harness emits with what the execution environment and browser can preserve.
Discuss a benchmark runCitation
@techreport{wootzapp2026w8domdiffbench,
title = {w8-domdiff-bench: Browser Evidence Infrastructure for Agent Development},
author = {{WootzApp Research}},
institution = {WootzApp},
year = {2026},
month = oct,
url = {https://github.com/wootzapp/w8-domdiff-bench}
}