Roto Brush: a SAM 2.1 video tracker (Hiera-Tiny, in Rust on candle) #1507

Idea

Add Meta's SAM 2.1 as an optional Roto Brush tracker, starting with Hiera-Tiny (38.9M parameters), ported to Rust.

Why

Roto Brush feedback (#1499, #1500) describes a crude outline and trouble with movement, colour changes, and things entering or leaving the frame. Today propagation warps the previous matte by optical flow and re-cuts a band around it, and a trained model (MobileSAM) is called on each frame without any memory of earlier frames. SAM 2 has a streaming memory built for video: it tracks an object across frames, handles occlusion, and re-acquires it when it returns.

Approach

Port Hugging Face transformers' Sam2Model / Sam2VideoModel (Apache-2.0) to Rust on candle, the way storytold/lightcraft ported SAM 3:

• A new native-only crate (not built for wasm32), forbid(unsafe_code), vendored with a NOTICE and per-file attribution.
• Reference tests against PyTorch fixtures, one per stage (Hiera encoder, neck, prompt encoder, decoder, memory encoder, memory attention), so a mismatch points at one module.
• An optional download that is SHA-256 verified and never bundled. Meta's README states the SAM 2 code and checkpoints are Apache-2.0, so it fits the registry's licence rule.
• Roto Brush integration: strokes become point prompts, tracking runs in both directions from any prompted frame, a correction stroke becomes a new prompted frame, and each frame gets a confidence value shown on the timeline.
• The existing graph-cut and matting stages stay, for edge refinement and as the fallback. With no model installed, nothing changes.
• A bench --roto harness (synthetic composites with known alpha) to measure it against the current path and MobileSAM, so the improvement is measured and not assumed.

Status

I'm looking into this and plan to work on it. Nothing is built yet; I'm starting with the reference setup and the benchmark.

Questions for maintainers

1. Is a native-only candle dependency acceptable, in its own crate behind a feature? The segment crate's README says "no ML framework"; LightCraft already uses candle.
2. Is a separate Apache-2.0-only crate fine, since the port derives from Apache-2.0 code?
3. Weights: official source with a pinned SHA-256, or a project mirror?
4. Opt-in at first, and default only if the benchmark supports it?

Not in scope

Text prompts, larger models, and a web (wasm) build of the model.