Image-to-video benchmark: classic community stress tests, OCR legibility & effective $/s (v1)
Suite product-i2v-v1 (v1, frozen 2026-11-16) · Pass@1 · Grader: scripts/benchmark-grade.py
Standardized single-draw (n=1) 5-second image-to-video benchmark across three classic AI-video stress tests (curved metallic soda can splash orbit, rainy neon corner diner push-in, and desert highway vintage rally car lateral tracking). Graded 100% by deterministic scripts (Apple Vision OCR with language autocomplete disabled + dense Farnebäck optical flow + Laplacian sharpness retention), pricing every model uniformly against public fal.ai retail per-second rates.
1. Leaderboard
| # | Model / workflow | Cases passed | Pass rate | Effective $/s | Nominal $/s | Median time | Post-motion OCR accuracy | Foreground motion ratio | End-frame sharpness retention | Delivered duration ratio | Credits |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Vidu Q4 q4-540p Passed 2/3 I2V stress tests (100.0% mean post-motion OCR, 15.4% foreground motion, 115s median render) for $0.047/s effective usable cost at fal.ai retail rates. | 2/3 | 67% (2/3) | $0.047/s 1.5x retry cost | $0.032/s | 115s | 100.0% | 15.4% | 74.3% | 100.0% | Self-funded |
| 2 | Wan 3.0 Video 3.0-standard-480p Passed 1/3 I2V stress tests (87.3% mean post-motion OCR, 19.7% foreground motion, 93s median render) for $0.150/s effective usable cost at fal.ai retail rates. | 1/3 | 33% (1/3) | $0.150/s 3.0x retry cost | $0.050/s | 93s | 87.3% | 19.7% | 44.0% | 100.0% | Self-funded |
| 3 | Grok Imagine Video 1.5 1.5-480p Passed 1/3 I2V stress tests (79.6% mean post-motion OCR, 22.2% foreground motion, 45s median render) for $0.238/s effective usable cost at fal.ai retail rates. | 1/3 | 33% (1/3) | $0.238/s 3.0x retry cost | $0.080/s | 45s | 79.6% | 22.2% | 61.7% | 100.0% | Self-funded |
| 4 | Kling 4.0 4.0-720p Passed 1/3 I2V stress tests (98.5% mean post-motion OCR, 12.5% foreground motion, 244s median render) for $0.467/s effective usable cost at fal.ai retail rates. | 1/3 | 33% (1/3) | $0.467/s 3.0x retry cost | $0.157/s | 244s | 98.5% | 12.5% | 59.9% | 100.0% | Self-funded |
| 5 | Seedance 2.5 2.5-480p Passed 1/3 I2V stress tests (88.9% mean post-motion OCR, 15.3% foreground motion, 142s median render) for $0.656/s effective usable cost at fal.ai retail rates. | 1/3 | 33% (1/3) | $0.656/s 3.0x retry cost | $0.221/s | 142s | 88.9% | 15.3% | 61.8% | 100.0% | Self-funded |
| 6 | FLUX.3 Max Video flux-3-max-720p Passed 0/3 I2V stress tests (97.0% mean post-motion OCR, 4.6% foreground motion, 19s median render) for DNF (0/3 usable) effective usable cost at fal.ai retail rates. | 0/3 | 0% (0/3) | DNF | $0.170/s | 19s | 97.0% | 4.6% | 81.4% | 100.0% | Self-funded |
| 7 | LTX-2.5 Fast (Open-Weight) 2.5-fast-720p-open Passed 0/3 I2V stress tests (50.0% mean post-motion OCR, 18.1% foreground motion, 33s median render) for DNF (0/3 usable) effective usable cost at fal.ai retail rates. | 0/3 | 0% (0/3) | DNF | $0.090/s | 33s | 50.0% | 18.1% | 66.7% | 100.0% | Self-funded |
| 8 | MiniMax Hailuo H3 Max Turbo h3-max-turbo-480p Passed 0/3 I2V stress tests (85.6% mean post-motion OCR, 9.4% foreground motion, 52s median render) for DNF (0/3 usable) effective usable cost at fal.ai retail rates. | 0/3 | 0% (0/3) | DNF | $0.025/s | 52s | 85.6% | 9.4% | 86.4% | 100.0% | Self-funded |
| 9 | Seedance 2.0 Fast 2.0-fast-480p Passed 0/3 I2V stress tests (88.7% mean post-motion OCR, 19.9% foreground motion, 105s median render) for DNF (0/3 usable) effective usable cost at fal.ai retail rates. | 0/3 | 0% (0/3) | DNF | $0.115/s | 105s | 88.7% | 19.9% | 71.9% | 100.0% | Self-funded |
| 10 | NVIDIA Cosmos 3 Super (Open-Weight) cosmos-3-super-5s-open Passed 0/3 I2V stress tests (89.8% mean post-motion OCR, 10.4% foreground motion, 464s median render) for DNF (0/3 usable) effective usable cost at fal.ai retail rates. | 0/3 | 0% (0/3) | DNF | $0.050/s | 464s | 89.8% | 10.4% | 83.7% | 100.0% | Self-funded |
Vidu Q4 (q4-540p) — per-case scores & raw clip evidence (tested 2026-11-16)
can-orbit-5s: 0/1 usable · 5.04s @ 540p · billed $0.158 · ocr_char_accuracy=100.0%, motion_disp_ratio=8.8%, sharpness_retention=100.0%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 8.8% < 12% gateneon-diner-5s: 1/1 usable · 5.04s @ 540p · billed $0.158 · ocr_char_accuracy=100.0%, motion_disp_ratio=21.7%, sharpness_retention=66.9%, duration_ratio=100.0%rally-track-5s: 1/1 usable · 5.04s @ 540p · billed $0.158 · ocr_char_accuracy=100.0%, motion_disp_ratio=15.6%, sharpness_retention=56.1%, duration_ratio=100.0%
Wan 3.0 Video (3.0-standard-480p) — per-case scores & raw clip evidence (tested 2026-11-16)
can-orbit-5s: 1/1 usable · 5s @ 480p · billed $0.250 · ocr_char_accuracy=90.9%, motion_disp_ratio=13.2%, sharpness_retention=77.7%, duration_ratio=100.0%neon-diner-5s: 0/1 usable · 5s @ 480p · billed $0.250 · ocr_char_accuracy=98.7%, motion_disp_ratio=22.5%, sharpness_retention=43.3%, duration_ratio=100.0% · failure: End-frame blur: sharpness retention 43.3% < 55% gaterally-track-5s: 0/1 usable · 5s @ 480p · billed $0.250 · ocr_char_accuracy=72.2%, motion_disp_ratio=23.4%, sharpness_retention=10.9%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 72.2% < 90% gate (worst post-motion frame: "6 86"); End-frame blur: sharpness retention 10.9% < 55% gate
Grok Imagine Video 1.5 (1.5-480p) — per-case scores & raw clip evidence (tested 2026-11-16)
can-orbit-5s: 0/1 usable · 5.04s @ 480p · billed $0.400 · ocr_char_accuracy=66.7%, motion_disp_ratio=14.6%, sharpness_retention=74.5%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 66.7% < 90% gate (worst post-motion frame: "FOUN COLD.LOU")neon-diner-5s: 1/1 usable · 5.04s @ 480p · billed $0.400 · ocr_char_accuracy=100.0%, motion_disp_ratio=23.1%, sharpness_retention=83.1%, duration_ratio=100.0%rally-track-5s: 0/1 usable · 5.04s @ 480p · billed $0.400 · ocr_char_accuracy=72.2%, motion_disp_ratio=28.9%, sharpness_retention=27.5%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 72.2% < 90% gate (worst post-motion frame: "D Fme 86"); End-frame blur: sharpness retention 27.5% < 55% gate
Kling 4.0 (4.0-720p) — per-case scores & raw clip evidence (tested 2026-11-16)
can-orbit-5s: 0/1 usable · 5.04s @ 720p · billed $0.785 · ocr_char_accuracy=95.5%, motion_disp_ratio=9.4%, sharpness_retention=72.4%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 9.4% < 12% gateneon-diner-5s: 1/1 usable · 5.04s @ 720p · billed $0.785 · ocr_char_accuracy=100.0%, motion_disp_ratio=13.1%, sharpness_retention=71.7%, duration_ratio=100.0%rally-track-5s: 0/1 usable · 5.04s @ 720p · billed $0.785 · ocr_char_accuracy=100.0%, motion_disp_ratio=15.2%, sharpness_retention=35.7%, duration_ratio=100.0% · failure: End-frame blur: sharpness retention 35.7% < 55% gate
Seedance 2.5 (2.5-480p) — per-case scores & raw clip evidence (tested 2026-11-16)
can-orbit-5s: 0/1 usable · 5.04s @ 480p · billed $1.103 · ocr_char_accuracy=100.0%, motion_disp_ratio=8.7%, sharpness_retention=94.4%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 8.7% < 12% gateneon-diner-5s: 1/1 usable · 5.04s @ 480p · billed $1.103 · ocr_char_accuracy=100.0%, motion_disp_ratio=15.3%, sharpness_retention=80.5%, duration_ratio=100.0%rally-track-5s: 0/1 usable · 5.04s @ 480p · billed $1.103 · ocr_char_accuracy=66.7%, motion_disp_ratio=22.1%, sharpness_retention=10.6%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 66.7% < 90% gate (worst post-motion frame: no readable text by end frame); End-frame blur: sharpness retention 10.6% < 55% gate
FLUX.3 Max Video (flux-3-max-720p) — per-case scores & raw clip evidence (tested 2026-11-16)
can-orbit-5s: 0/1 usable · 5.04s @ 720p · billed $0.850 · ocr_char_accuracy=90.9%, motion_disp_ratio=5.8%, sharpness_retention=93.4%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 5.8% < 12% gateneon-diner-5s: 0/1 usable · 5.04s @ 720p · billed $0.850 · ocr_char_accuracy=100.0%, motion_disp_ratio=4.0%, sharpness_retention=93.3%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 4.0% < 12% gaterally-track-5s: 0/1 usable · 5.04s @ 720p · billed $0.850 · ocr_char_accuracy=100.0%, motion_disp_ratio=4.1%, sharpness_retention=57.5%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 4.1% < 12% gate
LTX-2.5 Fast (Open-Weight) (2.5-fast-720p-open) — per-case scores & raw clip evidence (tested 2026-11-16)
can-orbit-5s: 0/1 usable · 6.04s @ 720p · billed $0.540 · ocr_char_accuracy=83.3%, motion_disp_ratio=14.3%, sharpness_retention=100.0%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 83.3% < 90% gate (worst post-motion frame: "OURTHPEE LD.LOUD.LIME.")neon-diner-5s: 0/1 usable · 6.04s @ 720p · billed $0.540 · ocr_char_accuracy=33.3%, motion_disp_ratio=21.3%, sharpness_retention=96.2%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 33.3% < 90% gate (worst post-motion frame: no readable text by end frame)rally-track-5s: 0/1 usable · 6.04s @ 720p · billed $0.540 · ocr_char_accuracy=33.3%, motion_disp_ratio=18.7%, sharpness_retention=3.8%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 33.3% < 90% gate (worst post-motion frame: no readable text by end frame); End-frame blur: sharpness retention 3.8% < 55% gate
MiniMax Hailuo H3 Max Turbo (h3-max-turbo-480p) — per-case scores & raw clip evidence (tested 2026-11-16)
can-orbit-5s: 0/1 usable · 5.17s @ 480p · billed $0.125 · ocr_char_accuracy=98.5%, motion_disp_ratio=7.9%, sharpness_retention=88.9%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 7.9% < 12% gateneon-diner-5s: 0/1 usable · 5.17s @ 480p · billed $0.125 · ocr_char_accuracy=100.0%, motion_disp_ratio=10.3%, sharpness_retention=78.9%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 10.3% < 12% gaterally-track-5s: 0/1 usable · 5.17s @ 480p · billed $0.125 · ocr_char_accuracy=58.3%, motion_disp_ratio=9.9%, sharpness_retention=91.3%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 58.3% < 90% gate (worst post-motion frame: "86"); Static/under-animated clip: foreground motion 9.9% < 12% gate
Seedance 2.0 Fast (2.0-fast-480p) — per-case scores & raw clip evidence (tested 2026-11-16)
can-orbit-5s: 0/1 usable · 5.04s @ 480p · billed $0.577 · ocr_char_accuracy=87.9%, motion_disp_ratio=11.8%, sharpness_retention=100.0%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 87.9% < 90% gate (worst post-motion frame: "URTHFEED OUD.LIME. O"); Static/under-animated clip: foreground motion 11.8% < 12% gateneon-diner-5s: 0/1 usable · 5.04s @ 480p · billed $0.577 · ocr_char_accuracy=78.2%, motion_disp_ratio=23.4%, sharpness_retention=65.0%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 78.2% < 90% gate (worst post-motion frame: "OPEN24HOURS")rally-track-5s: 0/1 usable · 5.04s @ 480p · billed $0.577 · ocr_char_accuracy=100.0%, motion_disp_ratio=24.6%, sharpness_retention=50.8%, duration_ratio=100.0% · failure: End-frame blur: sharpness retention 50.8% < 55% gate
NVIDIA Cosmos 3 Super (Open-Weight) (cosmos-3-super-5s-open) — per-case scores & raw clip evidence (tested 2026-11-16)
can-orbit-5s: 0/1 usable · 5.04s @ 480p · billed $0.250 · ocr_char_accuracy=100.0%, motion_disp_ratio=8.7%, sharpness_retention=91.9%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 8.7% < 12% gateneon-diner-5s: 0/1 usable · 5.04s @ 480p · billed $0.250 · ocr_char_accuracy=100.0%, motion_disp_ratio=1.8%, sharpness_retention=100.0%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 1.8% < 12% gaterally-track-5s: 0/1 usable · 5.04s @ 480p · billed $0.250 · ocr_char_accuracy=69.4%, motion_disp_ratio=20.7%, sharpness_retention=59.2%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 69.4% < 90% gate (worst post-motion frame: "8")
2. Script pass gates & metrics
- Post-motion OCR accuracy
ocr_char_accuracyUnit: ratio · Higher is better · Required gate: gte 0.9
- Foreground motion ratio
motion_disp_ratioUnit: ratio · Higher is better · Required gate: gte 0.12
- End-frame sharpness retention
sharpness_retentionUnit: ratio · Higher is better · Required gate: gte 0.55
- Delivered duration ratio
duration_ratioUnit: ratio · Higher is better · Required gate: gte 0.9
3. Frozen test cases (3)
Prompt: “Slow commercial studio camera orbit 25 degrees around the chilled emerald-green FOURTHFEED aluminum soda can while the top water splash crown, floating ice cubes, and fresh lime wedges drift gently in slow motion; keep both 'FOURTHFEED' and 'COLD. LOUD. LIME.' printed on the curved can sharp and readable.”
Pass rule: ocr_char_accuracy >= 0.90 on [FOURTHFEED, COLD, LOUD, LIME] at t>=2.5s, motion_disp_ratio >= 0.12, sharpness_retention >= 0.55, duration_ratio >= 0.90
- Frozen image: ff-can-orbit-v2.png ↗
sha256:e6f93829af91…— Proprietary curved metallic can with condensation droplets over ink; target OCR tokens at t>=2.5s: FOURTHFEED, COLD, LOUD, LIME
Prompt: “Smooth cinematic nighttime camera push-in toward the glowing neon FOURTHFEED DINER facade in light rain as wet asphalt reflections shimmer and the pedestrian holding the clear umbrella walks slowly forward along the sidewalk; keep 'FOURTHFEED DINER' and 'OPEN 24 HOURS' razor-sharp and legible.”
Pass rule: ocr_char_accuracy >= 0.90 on [FOURTHFEED, DINER, OPEN, 24, HOURS] at t>=2.5s, motion_disp_ratio >= 0.12, sharpness_retention >= 0.55, duration_ratio >= 0.90
- Frozen image: ff-neon-diner-v1.png ↗
sha256:ccab1c2a5298…— Proprietary rainy neon diner storefront; target OCR tokens at t>=2.5s: FOURTHFEED, DINER, OPEN, 24, HOURS
Prompt: “Fast lateral motorsport tracking shot pacing alongside the orange-and-white FOURTHFEED #86 vintage rally coupe accelerating down the desert highway past the ROUTE 66 roadside sign while desert dust trails behind the spinning tires; keep the 'FOURTHFEED 86' side-door livery crisp and legible in center frame.”
Pass rule: ocr_char_accuracy >= 0.90 on [FOURTHFEED, 86] at t>=2.5s, motion_disp_ratio >= 0.12, sharpness_retention >= 0.55, duration_ratio >= 0.90
- Frozen image: ff-rally-track-v1.png ↗
sha256:cc65ce87cf1b…— Proprietary desert rally coupe; target OCR tokens at t>=2.5s on moving door livery: FOURTHFEED, 86
This suite as data: .json · .md · /api/v1/benchmarks/product-i2v-v1