Benchmarks

Image-to-video benchmark: classic community stress tests, OCR legibility & effective $/s (v1)

Suite product-i2v-v1 (v1, frozen 2026-11-16) · Pass@1 · Grader: scripts/benchmark-grade.py

Standardized single-draw (n=1) 5-second image-to-video benchmark across three classic AI-video stress tests (curved metallic soda can splash orbit, rainy neon corner diner push-in, and desert highway vintage rally car lateral tracking). Graded 100% by deterministic scripts (Apple Vision OCR with language autocomplete disabled + dense Farnebäck optical flow + Laplacian sharpness retention), pricing every model uniformly against public fal.ai retail per-second rates.

1. Leaderboard

Sort by:
#Model / workflowCases passedPass rateEffective $/sNominal $/sMedian timePost-motion OCR accuracyForeground motion ratioEnd-frame sharpness retentionDelivered duration ratioCredits
1
Vidu Q4 q4-540p

Passed 2/3 I2V stress tests (100.0% mean post-motion OCR, 15.4% foreground motion, 115s median render) for $0.047/s effective usable cost at fal.ai retail rates.

2/367% (2/3)$0.047/s
1.5x retry cost
$0.032/s115s100.0%15.4%74.3%100.0%Self-funded
2
Wan 3.0 Video 3.0-standard-480p

Passed 1/3 I2V stress tests (87.3% mean post-motion OCR, 19.7% foreground motion, 93s median render) for $0.150/s effective usable cost at fal.ai retail rates.

1/333% (1/3)$0.150/s
3.0x retry cost
$0.050/s93s87.3%19.7%44.0%100.0%Self-funded
3
Grok Imagine Video 1.5 1.5-480p

Passed 1/3 I2V stress tests (79.6% mean post-motion OCR, 22.2% foreground motion, 45s median render) for $0.238/s effective usable cost at fal.ai retail rates.

1/333% (1/3)$0.238/s
3.0x retry cost
$0.080/s45s79.6%22.2%61.7%100.0%Self-funded
4
Kling 4.0 4.0-720p

Passed 1/3 I2V stress tests (98.5% mean post-motion OCR, 12.5% foreground motion, 244s median render) for $0.467/s effective usable cost at fal.ai retail rates.

1/333% (1/3)$0.467/s
3.0x retry cost
$0.157/s244s98.5%12.5%59.9%100.0%Self-funded
5
Seedance 2.5 2.5-480p

Passed 1/3 I2V stress tests (88.9% mean post-motion OCR, 15.3% foreground motion, 142s median render) for $0.656/s effective usable cost at fal.ai retail rates.

1/333% (1/3)$0.656/s
3.0x retry cost
$0.221/s142s88.9%15.3%61.8%100.0%Self-funded
6
FLUX.3 Max Video flux-3-max-720p

Passed 0/3 I2V stress tests (97.0% mean post-motion OCR, 4.6% foreground motion, 19s median render) for DNF (0/3 usable) effective usable cost at fal.ai retail rates.

0/30% (0/3)DNF$0.170/s19s97.0%4.6%81.4%100.0%Self-funded
7
LTX-2.5 Fast (Open-Weight) 2.5-fast-720p-open

Passed 0/3 I2V stress tests (50.0% mean post-motion OCR, 18.1% foreground motion, 33s median render) for DNF (0/3 usable) effective usable cost at fal.ai retail rates.

0/30% (0/3)DNF$0.090/s33s50.0%18.1%66.7%100.0%Self-funded
8
MiniMax Hailuo H3 Max Turbo h3-max-turbo-480p

Passed 0/3 I2V stress tests (85.6% mean post-motion OCR, 9.4% foreground motion, 52s median render) for DNF (0/3 usable) effective usable cost at fal.ai retail rates.

0/30% (0/3)DNF$0.025/s52s85.6%9.4%86.4%100.0%Self-funded
9
Seedance 2.0 Fast 2.0-fast-480p

Passed 0/3 I2V stress tests (88.7% mean post-motion OCR, 19.9% foreground motion, 105s median render) for DNF (0/3 usable) effective usable cost at fal.ai retail rates.

0/30% (0/3)DNF$0.115/s105s88.7%19.9%71.9%100.0%Self-funded
10
NVIDIA Cosmos 3 Super (Open-Weight) cosmos-3-super-5s-open

Passed 0/3 I2V stress tests (89.8% mean post-motion OCR, 10.4% foreground motion, 464s median render) for DNF (0/3 usable) effective usable cost at fal.ai retail rates.

0/30% (0/3)DNF$0.050/s464s89.8%10.4%83.7%100.0%Self-funded
Vidu Q4 (q4-540p) — per-case scores & raw clip evidence (tested 2026-11-16)
  • can-orbit-5s: 0/1 usable · 5.04s @ 540p · billed $0.158 · ocr_char_accuracy=100.0%, motion_disp_ratio=8.8%, sharpness_retention=100.0%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 8.8% < 12% gate
  • neon-diner-5s: 1/1 usable · 5.04s @ 540p · billed $0.158 · ocr_char_accuracy=100.0%, motion_disp_ratio=21.7%, sharpness_retention=66.9%, duration_ratio=100.0%
  • rally-track-5s: 1/1 usable · 5.04s @ 540p · billed $0.158 · ocr_char_accuracy=100.0%, motion_disp_ratio=15.6%, sharpness_retention=56.1%, duration_ratio=100.0%
Wan 3.0 Video (3.0-standard-480p) — per-case scores & raw clip evidence (tested 2026-11-16)
  • can-orbit-5s: 1/1 usable · 5s @ 480p · billed $0.250 · ocr_char_accuracy=90.9%, motion_disp_ratio=13.2%, sharpness_retention=77.7%, duration_ratio=100.0%
  • neon-diner-5s: 0/1 usable · 5s @ 480p · billed $0.250 · ocr_char_accuracy=98.7%, motion_disp_ratio=22.5%, sharpness_retention=43.3%, duration_ratio=100.0% · failure: End-frame blur: sharpness retention 43.3% < 55% gate
  • rally-track-5s: 0/1 usable · 5s @ 480p · billed $0.250 · ocr_char_accuracy=72.2%, motion_disp_ratio=23.4%, sharpness_retention=10.9%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 72.2% < 90% gate (worst post-motion frame: "6 86"); End-frame blur: sharpness retention 10.9% < 55% gate
Grok Imagine Video 1.5 (1.5-480p) — per-case scores & raw clip evidence (tested 2026-11-16)
  • can-orbit-5s: 0/1 usable · 5.04s @ 480p · billed $0.400 · ocr_char_accuracy=66.7%, motion_disp_ratio=14.6%, sharpness_retention=74.5%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 66.7% < 90% gate (worst post-motion frame: "FOUN COLD.LOU")
  • neon-diner-5s: 1/1 usable · 5.04s @ 480p · billed $0.400 · ocr_char_accuracy=100.0%, motion_disp_ratio=23.1%, sharpness_retention=83.1%, duration_ratio=100.0%
  • rally-track-5s: 0/1 usable · 5.04s @ 480p · billed $0.400 · ocr_char_accuracy=72.2%, motion_disp_ratio=28.9%, sharpness_retention=27.5%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 72.2% < 90% gate (worst post-motion frame: "D Fme 86"); End-frame blur: sharpness retention 27.5% < 55% gate
Kling 4.0 (4.0-720p) — per-case scores & raw clip evidence (tested 2026-11-16)
  • can-orbit-5s: 0/1 usable · 5.04s @ 720p · billed $0.785 · ocr_char_accuracy=95.5%, motion_disp_ratio=9.4%, sharpness_retention=72.4%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 9.4% < 12% gate
  • neon-diner-5s: 1/1 usable · 5.04s @ 720p · billed $0.785 · ocr_char_accuracy=100.0%, motion_disp_ratio=13.1%, sharpness_retention=71.7%, duration_ratio=100.0%
  • rally-track-5s: 0/1 usable · 5.04s @ 720p · billed $0.785 · ocr_char_accuracy=100.0%, motion_disp_ratio=15.2%, sharpness_retention=35.7%, duration_ratio=100.0% · failure: End-frame blur: sharpness retention 35.7% < 55% gate
Seedance 2.5 (2.5-480p) — per-case scores & raw clip evidence (tested 2026-11-16)
  • can-orbit-5s: 0/1 usable · 5.04s @ 480p · billed $1.103 · ocr_char_accuracy=100.0%, motion_disp_ratio=8.7%, sharpness_retention=94.4%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 8.7% < 12% gate
  • neon-diner-5s: 1/1 usable · 5.04s @ 480p · billed $1.103 · ocr_char_accuracy=100.0%, motion_disp_ratio=15.3%, sharpness_retention=80.5%, duration_ratio=100.0%
  • rally-track-5s: 0/1 usable · 5.04s @ 480p · billed $1.103 · ocr_char_accuracy=66.7%, motion_disp_ratio=22.1%, sharpness_retention=10.6%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 66.7% < 90% gate (worst post-motion frame: no readable text by end frame); End-frame blur: sharpness retention 10.6% < 55% gate
FLUX.3 Max Video (flux-3-max-720p) — per-case scores & raw clip evidence (tested 2026-11-16)
  • can-orbit-5s: 0/1 usable · 5.04s @ 720p · billed $0.850 · ocr_char_accuracy=90.9%, motion_disp_ratio=5.8%, sharpness_retention=93.4%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 5.8% < 12% gate
  • neon-diner-5s: 0/1 usable · 5.04s @ 720p · billed $0.850 · ocr_char_accuracy=100.0%, motion_disp_ratio=4.0%, sharpness_retention=93.3%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 4.0% < 12% gate
  • rally-track-5s: 0/1 usable · 5.04s @ 720p · billed $0.850 · ocr_char_accuracy=100.0%, motion_disp_ratio=4.1%, sharpness_retention=57.5%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 4.1% < 12% gate
LTX-2.5 Fast (Open-Weight) (2.5-fast-720p-open) — per-case scores & raw clip evidence (tested 2026-11-16)
  • can-orbit-5s: 0/1 usable · 6.04s @ 720p · billed $0.540 · ocr_char_accuracy=83.3%, motion_disp_ratio=14.3%, sharpness_retention=100.0%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 83.3% < 90% gate (worst post-motion frame: "OURTHPEE LD.LOUD.LIME.")
  • neon-diner-5s: 0/1 usable · 6.04s @ 720p · billed $0.540 · ocr_char_accuracy=33.3%, motion_disp_ratio=21.3%, sharpness_retention=96.2%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 33.3% < 90% gate (worst post-motion frame: no readable text by end frame)
  • rally-track-5s: 0/1 usable · 6.04s @ 720p · billed $0.540 · ocr_char_accuracy=33.3%, motion_disp_ratio=18.7%, sharpness_retention=3.8%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 33.3% < 90% gate (worst post-motion frame: no readable text by end frame); End-frame blur: sharpness retention 3.8% < 55% gate
MiniMax Hailuo H3 Max Turbo (h3-max-turbo-480p) — per-case scores & raw clip evidence (tested 2026-11-16)
  • can-orbit-5s: 0/1 usable · 5.17s @ 480p · billed $0.125 · ocr_char_accuracy=98.5%, motion_disp_ratio=7.9%, sharpness_retention=88.9%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 7.9% < 12% gate
  • neon-diner-5s: 0/1 usable · 5.17s @ 480p · billed $0.125 · ocr_char_accuracy=100.0%, motion_disp_ratio=10.3%, sharpness_retention=78.9%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 10.3% < 12% gate
  • rally-track-5s: 0/1 usable · 5.17s @ 480p · billed $0.125 · ocr_char_accuracy=58.3%, motion_disp_ratio=9.9%, sharpness_retention=91.3%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 58.3% < 90% gate (worst post-motion frame: "86"); Static/under-animated clip: foreground motion 9.9% < 12% gate
Seedance 2.0 Fast (2.0-fast-480p) — per-case scores & raw clip evidence (tested 2026-11-16)
  • can-orbit-5s: 0/1 usable · 5.04s @ 480p · billed $0.577 · ocr_char_accuracy=87.9%, motion_disp_ratio=11.8%, sharpness_retention=100.0%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 87.9% < 90% gate (worst post-motion frame: "URTHFEED OUD.LIME. O"); Static/under-animated clip: foreground motion 11.8% < 12% gate
  • neon-diner-5s: 0/1 usable · 5.04s @ 480p · billed $0.577 · ocr_char_accuracy=78.2%, motion_disp_ratio=23.4%, sharpness_retention=65.0%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 78.2% < 90% gate (worst post-motion frame: "OPEN24HOURS")
  • rally-track-5s: 0/1 usable · 5.04s @ 480p · billed $0.577 · ocr_char_accuracy=100.0%, motion_disp_ratio=24.6%, sharpness_retention=50.8%, duration_ratio=100.0% · failure: End-frame blur: sharpness retention 50.8% < 55% gate
NVIDIA Cosmos 3 Super (Open-Weight) (cosmos-3-super-5s-open) — per-case scores & raw clip evidence (tested 2026-11-16)
  • can-orbit-5s: 0/1 usable · 5.04s @ 480p · billed $0.250 · ocr_char_accuracy=100.0%, motion_disp_ratio=8.7%, sharpness_retention=91.9%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 8.7% < 12% gate
  • neon-diner-5s: 0/1 usable · 5.04s @ 480p · billed $0.250 · ocr_char_accuracy=100.0%, motion_disp_ratio=1.8%, sharpness_retention=100.0%, duration_ratio=100.0% · failure: Static/under-animated clip: foreground motion 1.8% < 12% gate
  • rally-track-5s: 0/1 usable · 5.04s @ 480p · billed $0.250 · ocr_char_accuracy=69.4%, motion_disp_ratio=20.7%, sharpness_retention=59.2%, duration_ratio=100.0% · failure: Post-motion OCR accuracy 69.4% < 90% gate (worst post-motion frame: "8")

2. Script pass gates & metrics

  • Post-motion OCR accuracyocr_char_accuracy

    Unit: ratio · Higher is better · Required gate: gte 0.9

  • Foreground motion ratiomotion_disp_ratio

    Unit: ratio · Higher is better · Required gate: gte 0.12

  • End-frame sharpness retentionsharpness_retention

    Unit: ratio · Higher is better · Required gate: gte 0.55

  • Delivered duration ratioduration_ratio

    Unit: ratio · Higher is better · Required gate: gte 0.9

3. Frozen test cases (3)

Curved metallic soda can splash & 25° camera orbitcan-orbit-5s · 5s @ 480p

Prompt: “Slow commercial studio camera orbit 25 degrees around the chilled emerald-green FOURTHFEED aluminum soda can while the top water splash crown, floating ice cubes, and fresh lime wedges drift gently in slow motion; keep both 'FOURTHFEED' and 'COLD. LOUD. LIME.' printed on the curved can sharp and readable.”

Pass rule: ocr_char_accuracy >= 0.90 on [FOURTHFEED, COLD, LOUD, LIME] at t>=2.5s, motion_disp_ratio >= 0.12, sharpness_retention >= 0.55, duration_ratio >= 0.90

  • Frozen image: ff-can-orbit-v2.png ↗ sha256:e6f93829af91… — Proprietary curved metallic can with condensation droplets over ink; target OCR tokens at t>=2.5s: FOURTHFEED, COLD, LOUD, LIME
Rainy night neon corner diner push-in & wet reflectionsneon-diner-5s · 5s @ 480p

Prompt: “Smooth cinematic nighttime camera push-in toward the glowing neon FOURTHFEED DINER facade in light rain as wet asphalt reflections shimmer and the pedestrian holding the clear umbrella walks slowly forward along the sidewalk; keep 'FOURTHFEED DINER' and 'OPEN 24 HOURS' razor-sharp and legible.”

Pass rule: ocr_char_accuracy >= 0.90 on [FOURTHFEED, DINER, OPEN, 24, HOURS] at t>=2.5s, motion_disp_ratio >= 0.12, sharpness_retention >= 0.55, duration_ratio >= 0.90

  • Frozen image: ff-neon-diner-v1.png ↗ sha256:ccab1c2a5298… — Proprietary rainy neon diner storefront; target OCR tokens at t>=2.5s: FOURTHFEED, DINER, OPEN, 24, HOURS
Desert highway vintage rally car lateral tracking shotrally-track-5s · 5s @ 480p

Prompt: “Fast lateral motorsport tracking shot pacing alongside the orange-and-white FOURTHFEED #86 vintage rally coupe accelerating down the desert highway past the ROUTE 66 roadside sign while desert dust trails behind the spinning tires; keep the 'FOURTHFEED 86' side-door livery crisp and legible in center frame.”

Pass rule: ocr_char_accuracy >= 0.90 on [FOURTHFEED, 86] at t>=2.5s, motion_disp_ratio >= 0.12, sharpness_retention >= 0.55, duration_ratio >= 0.90

  • Frozen image: ff-rally-track-v1.png ↗ sha256:cc65ce87cf1b… — Proprietary desert rally coupe; target OCR tokens at t>=2.5s on moving door livery: FOURTHFEED, 86

This suite as data: .json · .md · /api/v1/benchmarks/product-i2v-v1