# Image-to-video benchmark: classic community stress tests, OCR legibility & effective $/s (v1)

Suite ID: `product-i2v-v1` (v1, frozen 2026-11-16; data as of 2026-11-16).

Standardized single-draw (n=1) 5-second image-to-video benchmark across three classic AI-video stress tests (curved metallic soda can splash orbit, rainy neon corner diner push-in, and desert highway vintage rally car lateral tracking). Graded 100% by deterministic scripts (Apple Vision OCR with language autocomplete disabled + dense Farnebäck optical flow + Laplacian sharpness retention), pricing every model uniformly against public fal.ai retail per-second rates.

- Max draws per case: Pass@1
- Automated pure-script grader: `scripts/benchmark-grade.py`

## Script pass gates & metrics

- **Post-motion OCR accuracy** (`ocr_char_accuracy`, ratio, higher is better) — pass gate: `ocr_char_accuracy gte 0.9`
- **Foreground motion ratio** (`motion_disp_ratio`, ratio, higher is better) — pass gate: `motion_disp_ratio gte 0.12`
- **End-frame sharpness retention** (`sharpness_retention`, ratio, higher is better) — pass gate: `sharpness_retention gte 0.55`
- **Delivered duration ratio** (`duration_ratio`, ratio, higher is better) — pass gate: `duration_ratio gte 0.9`

## Frozen test cases

### Curved metallic soda can splash & 25° camera orbit (`can-orbit-5s`, 5s @ 480p)

- Prompt: "Slow commercial studio camera orbit 25 degrees around the chilled emerald-green FOURTHFEED aluminum soda can while the top water splash crown, floating ice cubes, and fresh lime wedges drift gently in slow motion; keep both 'FOURTHFEED' and 'COLD. LOUD. LIME.' printed on the curved can sharp and readable."
- Pass rule: ocr_char_accuracy >= 0.90 on [FOURTHFEED, COLD, LOUD, LIME] at t>=2.5s, motion_disp_ratio >= 0.12, sharpness_retention >= 0.55, duration_ratio >= 0.90
- Frozen image input: https://assets.fourthfeed.com/benchmarks/product-i2v-v1/ff-can-orbit-v2.png (sha256: `e6f93829af912a89789f27880343d7d6807e44d1f8d06b164f5377c4fa76a7b7`) — Proprietary curved metallic can with condensation droplets over ink; target OCR tokens at t>=2.5s: FOURTHFEED, COLD, LOUD, LIME

### Rainy night neon corner diner push-in & wet reflections (`neon-diner-5s`, 5s @ 480p)

- Prompt: "Smooth cinematic nighttime camera push-in toward the glowing neon FOURTHFEED DINER facade in light rain as wet asphalt reflections shimmer and the pedestrian holding the clear umbrella walks slowly forward along the sidewalk; keep 'FOURTHFEED DINER' and 'OPEN 24 HOURS' razor-sharp and legible."
- Pass rule: ocr_char_accuracy >= 0.90 on [FOURTHFEED, DINER, OPEN, 24, HOURS] at t>=2.5s, motion_disp_ratio >= 0.12, sharpness_retention >= 0.55, duration_ratio >= 0.90
- Frozen image input: https://assets.fourthfeed.com/benchmarks/product-i2v-v1/ff-neon-diner-v1.png (sha256: `ccab1c2a52989a76b3986e8b200e7f65b313526e988267ba8f2fa6484a146355`) — Proprietary rainy neon diner storefront; target OCR tokens at t>=2.5s: FOURTHFEED, DINER, OPEN, 24, HOURS

### Desert highway vintage rally car lateral tracking shot (`rally-track-5s`, 5s @ 480p)

- Prompt: "Fast lateral motorsport tracking shot pacing alongside the orange-and-white FOURTHFEED #86 vintage rally coupe accelerating down the desert highway past the ROUTE 66 roadside sign while desert dust trails behind the spinning tires; keep the 'FOURTHFEED 86' side-door livery crisp and legible in center frame."
- Pass rule: ocr_char_accuracy >= 0.90 on [FOURTHFEED, 86] at t>=2.5s, motion_disp_ratio >= 0.12, sharpness_retention >= 0.55, duration_ratio >= 0.90
- Frozen image input: https://assets.fourthfeed.com/benchmarks/product-i2v-v1/ff-rally-track-v1.png (sha256: `cc65ce87cf1b4d4712f3368a98e760581187c458f019428ab33481c9a31db56a`) — Proprietary desert rally coupe; target OCR tokens at t>=2.5s on moving door livery: FOURTHFEED, 86

## Leaderboard (ranked by cases passed, then effective $/s)

| Subject | Cases passed | Pass rate | Effective $/s | Nominal $/s | Median time | Post-motion OCR accuracy | Foreground motion ratio | End-frame sharpness retention | Delivered duration ratio | Credits |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Vidu Q4 (q4-540p) | 2/3 | 67% (2/3) | $0.047/s | $0.032/s | 115.47s | 100.0% | 15.4% | 74.3% | 100.0% | Self-funded |
| Wan 3.0 Video (3.0-standard-480p) | 1/3 | 33% (1/3) | $0.150/s | $0.050/s | 93.07s | 87.3% | 19.7% | 44.0% | 100.0% | Self-funded |
| Grok Imagine Video 1.5 (1.5-480p) | 1/3 | 33% (1/3) | $0.238/s | $0.080/s | 45.07s | 79.6% | 22.2% | 61.7% | 100.0% | Self-funded |
| Kling 4.0 (4.0-720p) | 1/3 | 33% (1/3) | $0.467/s | $0.157/s | 243.53s | 98.5% | 12.5% | 59.9% | 100.0% | Self-funded |
| Seedance 2.5 (2.5-480p) | 1/3 | 33% (1/3) | $0.656/s | $0.221/s | 142.46s | 88.9% | 15.3% | 61.8% | 100.0% | Self-funded |
| FLUX.3 Max Video (flux-3-max-720p) | 0/3 | 0% (0/3) | DNF | $0.170/s | 18.9s | 97.0% | 4.6% | 81.4% | 100.0% | Self-funded |
| LTX-2.5 Fast (Open-Weight) (2.5-fast-720p-open) | 0/3 | 0% (0/3) | DNF | $0.090/s | 33.23s | 50.0% | 18.1% | 66.7% | 100.0% | Self-funded |
| MiniMax Hailuo H3 Max Turbo (h3-max-turbo-480p) | 0/3 | 0% (0/3) | DNF | $0.025/s | 52.31s | 85.6% | 9.4% | 86.4% | 100.0% | Self-funded |
| Seedance 2.0 Fast (2.0-fast-480p) | 0/3 | 0% (0/3) | DNF | $0.115/s | 104.66s | 88.7% | 19.9% | 71.9% | 100.0% | Self-funded |
| NVIDIA Cosmos 3 Super (Open-Weight) (cosmos-3-super-5s-open) | 0/3 | 0% (0/3) | DNF | $0.050/s | 464.48s | 89.8% | 10.4% | 83.7% | 100.0% | Self-funded |

Calling Seedance 2.5, Kling 4.0, Flux 3 Max, Wan 3.0 and LTX 2.5 from your own code? Use one pay-per-call API gateway with $0.10 free trial credit: KeelRouter: https://keelrouter.com (disclosure: we make KeelRouter; it is not ranked in the benchmark table above).

Page: https://fourthfeed.com/benchmarks/product-i2v-v1
