{"path":"/benchmarks","title":"AI video model benchmarks: effective cost per usable second & OCR/motion tests | Fourth Feed","description":"Standardized 5-second 480p AI video benchmarks with frozen R2 inputs, script-verified pass gates, retry waste multipliers and effective cost per usable second.","as_of":"2026-11-16","kind":"benchmarks-index","content":{"suites":[{"suite":{"suiteId":"motion-swap-v1","track":"motion-swap","version":1,"frozenAt":"2026-11-16","title":"Character replacement & motion-transfer video benchmark: pose lock, scale & effective $/s (v1)","description":"Standardized single-draw (n=1) 5-second character-replacement benchmark comparing Kling 3.0 Motion Control, Kling 2.6 Motion Control, ByteDance DreamActor V2, Wan 2.2 Animate 14B, Higgsfield Genjutsu Lite and each::labs Lockstep on 2D limb-vector correlation, compact mascot scale preservation and background SSIM stability.","query":"best ai character swap motion transfer video tool","maxAttemptsPerCase":1,"graderScript":"scripts/benchmark-grade.py","metrics":[{"id":"pose_angle_corr","label":"2D limb-angle correlation","unit":"ratio","higherIsBetter":true,"passGate":{"op":"gte","value":0.78}},{"id":"scale_ratio_accuracy","label":"Subject scale preservation","unit":"ratio","higherIsBetter":true,"passGate":{"op":"gte","value":0.75}},{"id":"bg_ssim","label":"Background SSIM stability","unit":"ratio","higherIsBetter":true,"passGate":{"op":"gte","value":0.8}},{"id":"duration_ratio","label":"Delivered duration ratio","unit":"ratio","higherIsBetter":true,"passGate":{"op":"gte","value":0.92}}],"cases":[{"caseId":"human-dance-lock-5s","title":"Full-body fast arm-cross dance transfer (street dancer)","prompt":"Replace the dancer in the video with the character in the black FOURTHFEED crewneck sweatshirt from the reference image, keeping the exact arm crosses, footwork, camera framing and studio background.","targetDurationSec":5,"targetResolution":"480p","inputs":[{"kind":"image","r2Url":"https://assets.fourthfeed.com/benchmarks/motion-swap-v1/ff-char-dancer-v2.png","sha256":"0433a98e0ef3321a260fa2cb3f4869d505838a094d00f3adfe68ec61ac10ce23","note":"Proprietary full-body street dancer in FOURTHFEED pullover crewneck with both hands visible"},{"kind":"video","r2Url":"https://assets.fourthfeed.com/benchmarks/motion-swap-v1/ff-driving-dance-5s-v1.mp4","sha256":"a8014d1696be8a0060980dc87f2ab2c742d2bec6bb0693d822e0bd1b83304f21","note":"5s 480p 24fps proprietary full-body dance driving clip"}],"passRule":"pose_angle_corr >= 0.78, scale_ratio_accuracy >= 0.75, bg_ssim >= 0.80, duration_ratio >= 0.92"},{"caseId":"small-mascot-scale-5s","title":"Cross-scale compact 3D pug mascot swap without adult-height stretching","prompt":"Replace the performer in the video with the compact 3D fawn pug mascot in the black FOURTHFEED crewneck sweatshirt from the reference image while preserving its compact ~0.65x pug proportions and keeping the studio background intact.","targetDurationSec":5,"targetResolution":"480p","inputs":[{"kind":"image","r2Url":"https://assets.fourthfeed.com/benchmarks/motion-swap-v1/ff-char-mascot-v2.png","sha256":"6dac5089a72539df560e8259e4d7535d00a25b922cebb5a3a24065ff44d6b996","note":"Proprietary compact 3D pug mascot in FOURTHFEED crewneck; tests whether pipelines stretch small mascots to adult human height"},{"kind":"video","r2Url":"https://assets.fourthfeed.com/benchmarks/motion-swap-v1/ff-driving-gesture-5s-v1.mp4","sha256":"027b3edddb9cd5b76de11437f4c6fd3f0974bd8bc2d2b4bd095ed1d5940bde6d","note":"5s 480p 24fps proprietary full-body gesture driving clip"}],"passRule":"pose_angle_corr >= 0.78, scale_ratio_accuracy >= 0.75, bg_ssim >= 0.80, duration_ratio >= 0.92"},{"caseId":"foreground-occlusion-5s","title":"Hand-across-face occlusion & rapid head-turn recovery (techwear courier)","prompt":"Replace the subject in the video with the courier in the amber FOURTHFEED pullover anorak from the reference image, maintaining clean face and chest identity when the hand passes across the face and during the 90-degree turn.","targetDurationSec":5,"targetResolution":"480p","inputs":[{"kind":"image","r2Url":"https://assets.fourthfeed.com/benchmarks/motion-swap-v1/ff-char-courier-v2.png","sha256":"2308d4cd327fdac3e03f3a9b04da5e1114116a706809e9fa09631479abadb351","note":"Proprietary full-body courier in FOURTHFEED amber pullover anorak"},{"kind":"video","r2Url":"https://assets.fourthfeed.com/benchmarks/motion-swap-v1/ff-driving-occlusion-5s-v1.mp4","sha256":"eecf9d36bd8bff2fb29d37143e060df542ac0a74972e1215c475a94909696137","note":"5s 480p 24fps proprietary hand-across-face + head-turn driving clip"}],"passRule":"pose_angle_corr >= 0.78, scale_ratio_accuracy >= 0.75, bg_ssim >= 0.80, duration_ratio >= 0.92"}],"ours":{"name":"PugSoda","url":"https://pugsoda.com","text":"Want ready-made TikTok meme character-swap templates for your own photo or pet without tuning prompts?"}},"path":"/benchmarks/motion-swap-v1","runCount":6,"asOf":"2026-11-16","leader":{"name":"Kling Motion Control","effectiveCostPerSecUsd":0.06958250497017891,"passRate":1}},{"suite":{"suiteId":"product-i2v-v1","track":"product-i2v","version":1,"frozenAt":"2026-11-16","title":"Image-to-video benchmark: classic community stress tests, OCR legibility & effective $/s (v1)","description":"Standardized single-draw (n=1) 5-second image-to-video benchmark across three classic AI-video stress tests (curved metallic soda can splash orbit, rainy neon corner diner push-in, and desert highway vintage rally car lateral tracking). Graded 100% by deterministic scripts (Apple Vision OCR with language autocomplete disabled + dense Farnebäck optical flow + Laplacian sharpness retention), pricing every model uniformly against public fal.ai retail per-second rates.","query":"best ai image to video model benchmark","maxAttemptsPerCase":1,"graderScript":"scripts/benchmark-grade.py","metrics":[{"id":"ocr_char_accuracy","label":"Post-motion OCR accuracy","unit":"ratio","higherIsBetter":true,"passGate":{"op":"gte","value":0.9}},{"id":"motion_disp_ratio","label":"Foreground motion ratio","unit":"ratio","higherIsBetter":true,"passGate":{"op":"gte","value":0.12}},{"id":"sharpness_retention","label":"End-frame sharpness retention","unit":"ratio","higherIsBetter":true,"passGate":{"op":"gte","value":0.55}},{"id":"duration_ratio","label":"Delivered duration ratio","unit":"ratio","higherIsBetter":true,"passGate":{"op":"gte","value":0.9}}],"cases":[{"caseId":"can-orbit-5s","title":"Curved metallic soda can splash & 25° camera orbit","prompt":"Slow commercial studio camera orbit 25 degrees around the chilled emerald-green FOURTHFEED aluminum soda can while the top water splash crown, floating ice cubes, and fresh lime wedges drift gently in slow motion; keep both 'FOURTHFEED' and 'COLD. LOUD. LIME.' printed on the curved can sharp and readable.","targetDurationSec":5,"targetResolution":"480p","inputs":[{"kind":"image","r2Url":"https://assets.fourthfeed.com/benchmarks/product-i2v-v1/ff-can-orbit-v2.png","sha256":"e6f93829af912a89789f27880343d7d6807e44d1f8d06b164f5377c4fa76a7b7","note":"Proprietary curved metallic can with condensation droplets over ink; target OCR tokens at t>=2.5s: FOURTHFEED, COLD, LOUD, LIME"}],"passRule":"ocr_char_accuracy >= 0.90 on [FOURTHFEED, COLD, LOUD, LIME] at t>=2.5s, motion_disp_ratio >= 0.12, sharpness_retention >= 0.55, duration_ratio >= 0.90"},{"caseId":"neon-diner-5s","title":"Rainy night neon corner diner push-in & wet reflections","prompt":"Smooth cinematic nighttime camera push-in toward the glowing neon FOURTHFEED DINER facade in light rain as wet asphalt reflections shimmer and the pedestrian holding the clear umbrella walks slowly forward along the sidewalk; keep 'FOURTHFEED DINER' and 'OPEN 24 HOURS' razor-sharp and legible.","targetDurationSec":5,"targetResolution":"480p","inputs":[{"kind":"image","r2Url":"https://assets.fourthfeed.com/benchmarks/product-i2v-v1/ff-neon-diner-v1.png","sha256":"ccab1c2a52989a76b3986e8b200e7f65b313526e988267ba8f2fa6484a146355","note":"Proprietary rainy neon diner storefront; target OCR tokens at t>=2.5s: FOURTHFEED, DINER, OPEN, 24, HOURS"}],"passRule":"ocr_char_accuracy >= 0.90 on [FOURTHFEED, DINER, OPEN, 24, HOURS] at t>=2.5s, motion_disp_ratio >= 0.12, sharpness_retention >= 0.55, duration_ratio >= 0.90"},{"caseId":"rally-track-5s","title":"Desert highway vintage rally car lateral tracking shot","prompt":"Fast lateral motorsport tracking shot pacing alongside the orange-and-white FOURTHFEED #86 vintage rally coupe accelerating down the desert highway past the ROUTE 66 roadside sign while desert dust trails behind the spinning tires; keep the 'FOURTHFEED 86' side-door livery crisp and legible in center frame.","targetDurationSec":5,"targetResolution":"480p","inputs":[{"kind":"image","r2Url":"https://assets.fourthfeed.com/benchmarks/product-i2v-v1/ff-rally-track-v1.png","sha256":"cc65ce87cf1b4d4712f3368a98e760581187c458f019428ab33481c9a31db56a","note":"Proprietary desert rally coupe; target OCR tokens at t>=2.5s on moving door livery: FOURTHFEED, 86"}],"passRule":"ocr_char_accuracy >= 0.90 on [FOURTHFEED, 86] at t>=2.5s, motion_disp_ratio >= 0.12, sharpness_retention >= 0.55, duration_ratio >= 0.90"}],"ours":{"name":"KeelRouter","url":"https://keelrouter.com","text":"Calling Seedance 2.5, Kling 4.0, Flux 3 Max, Wan 3.0 and LTX 2.5 from your own code? Use one pay-per-call API gateway with $0.10 free trial credit:"}},"path":"/benchmarks/product-i2v-v1","runCount":10,"asOf":"2026-11-16","leader":{"name":"Vidu Q4","effectiveCostPerSecUsd":0.046875,"passRate":0.6666666666666666}}]}}