AI Video API Workflow

Generate video programmatically across the three leading providers — Kuaishou Kling 3.0, Google Veo 3.1, and Runway Gen-4.5 — from one workflow. This skill handles the parts that differ per vendor: authentication, request shape, model routing, and the asynchronous submit-then-poll lifecycle that all three use. It's for developers building video pipelines who don't want to maintain three separate SDK integrations and three polling loops.

When to use it

• Text-to-video or image-to-video generation from a script, batch, or backend job.
• Animating a still: pass a start frame (and optionally an end frame) for an interpolated clip.
• Routing by capability — e.g. Veo when you need synchronized native audio at 1080p, Kling when you need explicit camera moves or motion-brush control, Runway when you want Gen-4 References for character/style consistency.
• Long-running renders where you submit a job, store the id, and reconcile results later instead of holding a connection open.

How it works

Each provider exposes an async job model, and the skill normalizes them:

• Kling 3.0 — POST to the image2video / text2video endpoints returns a task_id; poll the task-status endpoint until task_status is succeed or failed. Supports camera_control, start/end image/image_tail keyframes, and motion brush via dynamic masks.
• Veo 3.1 — call through the Gemini API (generateVideos / long-running operation) or Vertex AI; poll the returned operation until done. Produces native audio and 1080p output.
• Runway Gen-4.5 — create an image_to_video task, then poll the task by id until SUCCEEDED; use Gen-4 References to lock identity/style across shots.

Usage example

curl -fsSL --create-dirs -o ~/.claude/skills/ai-video-generation-api-workflow/SKILL.md https://claudeskil.com/skills/ai-video-generation-api-workflow/SKILL.md

> Animate hero.png with Kling: slow dolly-in, 5s, then poll until done

# Skill submits image2video with camera_control, captures task_id,
# polls task-status on a backoff, and returns the signed video URL.

> Same prompt, render with Veo 3.1 for narration audio at 1080p

Tips & gotchas

• Result URLs expire. Kling and Runway hand back time-limited download links — fetch and persist the asset immediately; don't store the URL as permanent.
• Don't hammer the poll. Use exponential backoff (renders take 30s–several minutes). Tight polling can trip rate limits before the job finishes.
• Keyframe pairing is provider-specific. Kling's end frame is image_tail; passing both start and end constrains motion more than most prompts expect.
• Auth differs. Kling uses an access-key/secret JWT, Veo uses Gemini API keys or Vertex service-account ADC, Runway uses a bearer key — keep them in separate env vars.

FAQ

Can I get a webhook instead of polling? Runway supports task webhooks; Kling and Veo are poll-based, so the skill defaults to polling with backoff everywhere for consistency.

Which model for lip-synced talking video? Veo 3.1 — it's the only one of the three with native synchronized audio.

How do I keep a character consistent across clips? Use Runway Gen-4.5 with Gen-4 References, or reuse a fixed start keyframe in Kling.