System 03 · October 2026
AI matte bench
Four soft-matte models times one hero track, scored against a roto vendor's mattes, with a deterministic gate that denies the egregious ones before a person looks.
Measured results
- Body agreement with the vendor 0.995 IoU Talent shot, 103 frames, median of four models times the hero track. Every soft model lands between 0.993 and 0.996; the differences are all at the edge.
- Edge error 23.0 px edgeMAD Mean absolute difference inside a 12 px band around the vendor's edge. The remaining gap is fine hair, the shadowed gap between arm and torso, and motion-blurred hands.
- Temporal stability 1.10 dtSSD Frame-to-frame sum of squared differences of the matte, lower is steadier. The busier shot with blurred runners behind the subject lands at 1.89.
- Gate validation 0 false denies All four vendor mattes pass. Six injected defects, an empty frame, lost legs, a two-frame blob, a one-frame hole, a 6 px pop and a missing frame, are all denied. In the real round it caught an extra runner, a wrist hole and a one-frame arm-gap pop.
- Cost per shot $0.50 to $1 and 2.5 minutes List prices for a 4.3 s shot with the four models run in parallel, against a vendor rate per frame.
Problem
Roto for talent and crowd holdouts is bought from a vendor by the frame. AI mattes are cheap and fast, but nobody could say how close they were to vendor quality, where exactly they fail, or how to keep a bad one from reaching the comp without a person checking every frame.
Method
A bench that scores any matte against the vendor's on the same frames: IoU and mean absolute difference for the body, an edge band for the silhouette, a temporal term for flicker, and far-field terms for gross extra or missing pixels. Candidates come from four soft-alpha models run on 1920 px display proxies, multiplied by a SAM 2 track of the hero so only the right person is matted, then combined as a per-pixel median. A separate gate needs no ground truth at all: it checks frame counts, empty or solid frames, area jumps, transient blobs and holes, edge pops measured only where the plate itself warps cleanly, and disagreement with the tracker.
In production
The question was never whether an AI matte looks fine on a contact sheet. It was whether the matte is close enough to what the vendor delivers to replace it on a shot, and if not, where the money is. Scoring against the vendor’s own mattes, frame for frame, answers both. On the body the answer is yes at IoU 0.995. On the edge the answer is a list: wispy hair that the vendor keeps and the AI softens into a pulled-in edge, the shadowed gap between an arm and the torso that the AI fills, and motion-blurred hands placed mid-blur.
Two findings changed the pipeline. First, soft removers matte every salient person in the frame, so a hero track from SAM 2 (an automatic box plus a centre point on frame one) has to multiply the matte before it means anything; prompted trackers returned every runner on the field. Second, check what the reference matte is of before scoring: the vendor’s holdouts were everything in front of the stands, not the coach, and a wrong assumption made good mattes look 75 percent wrong.
The gate is the part that makes any of this safe to run unattended. It is deterministic, needs no ground truth, and is validated the only honest way: the vendor’s mattes must pass, and defects injected on purpose must fail. It did, and in the real round it denied the two single-model mattes that popped on frame 30 while the median passed.
Next: hair. A local alpha model built on a video diffusion base is the candidate, and the whole chain from plate to proxy to hero track to four models to median to plate-resolution EXR to gate becomes one command.
Captures
real time, frame counter, no speed ramps-
test_gate.py live in a terminal, the last ten seconds of a 28-second run in real time: the clean reference matte passes, then the six injected defects (empty frame, lost legs, two-frame blob, torso hole, 6 px pop, missing frame) are each denied with the rule that caught them. Recorded 2026-10-15 at 3840x2088, 60 fps, timestamp and frame counter burned in.
BiRefNet v2 Dynamic 2304 and 4 more · not in delivered pixels · real time, frame counter and timestamp