Video Generation: Advanced

This page is the reference for Video Generation. Start there for the quick setup. Come here when you need to run the job yourself, or when you need the options of one provider.

Run the job yourself#

With stream: true, generateVideo() creates the job and polls it for you. Without it, generateVideo() only creates the job and returns { jobId, model }. You then poll with getVideoJobStatus().

import { generateVideo, getVideoJobStatus } from '@tanstack/ai'
import { openaiVideo } from '@tanstack/ai-openai'

async function createAndAwaitVideo(prompt: string, threadId: string) {
  const adapter = openaiVideo('sora-2')

  // 1. Create the job
  const { jobId } = await generateVideo({
    adapter,
    prompt,
    threadId,
    size: '1280x720',
    duration: 8,
  })

  // 2. Poll until the job is done or fails
  let status = await getVideoJobStatus({ adapter, jobId })
  while (status.status !== 'completed' && status.status !== 'failed') {
    await new Promise((resolve) => setTimeout(resolve, 5000))
    status = await getVideoJobStatus({ adapter, jobId })
    console.log(`Status: ${status.status} (${status.progress ?? 0}%)`)
  }

  // 3. Read the URL
  if (status.status === 'failed' || !status.url) {
    throw new Error(status.error || 'Video generation failed')
  }
  return status.url
}

const videoUrl = await createAndAwaitVideo(
  'A cat playing piano in a jazz bar',
  'thread-1',
)
console.log('Video ready:', videoUrl)

getVideoJobStatus() returns one object: { status, progress?, url?, error?, usage? }. status is 'pending', 'processing', 'completed', or 'failed'. progress is from 0 to 100, when the provider reports it. url is set when the job is complete.

If you use middleware, such as withGenerationPersistence, pass the same middleware and threadId to generateVideo() and to getVideoJobStatus(). The submit call only opens the run. The poll that sees the finished job closes the run and records the result.

Server function that polls (fetcher)#

A TanStack Start server function can run the full loop and return the finished result as JSON:

// lib/server-functions.ts
import { createServerFn } from '@tanstack/react-start'
import { generateVideo, getVideoJobStatus } from '@tanstack/ai'
import { openaiVideo } from '@tanstack/ai-openai'

export const generateVideoFn = createServerFn({ method: 'POST' })
  .inputValidator((data: { prompt: string; threadId: string }) => data)
  .handler(async ({ data }) => {
    const adapter = openaiVideo('sora-2')
    const { threadId } = data
    const { jobId } = await generateVideo({ adapter, prompt: data.prompt, threadId })

    let status = await getVideoJobStatus({ adapter, jobId })
    while (status.status !== 'completed' && status.status !== 'failed') {
      await new Promise((r) => setTimeout(r, 5000))
      status = await getVideoJobStatus({ adapter, jobId })
    }

    if (status.status === 'failed' || !status.url) {
      throw new Error(status.error || 'Video generation failed')
    }
    return { jobId, status: 'completed' as const, url: status.url }
  })

import { useGenerateVideo } from '@tanstack/ai-react'
import { generateVideoFn } from '../lib/server-functions'

const threadId = 'video-studio'

export function VideoGenerator() {
  const { generate, result, isLoading } = useGenerateVideo({
    fetcher: (input, { threadId }) =>
      generateVideoFn({ data: { ...input, threadId } }),
    threadId,
  })
  // Render the same way as in the quick setup.
}

Server function that streams (fetcher + Response)#

Return an SSE Response from the server function. You get typed input, and jobId and videoStatus update while the job runs:

// lib/server-functions.ts
import { createServerFn } from '@tanstack/react-start'
import { generateVideo, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiVideo } from '@tanstack/ai-openai'

export const generateVideoStreamFn = createServerFn({ method: 'POST' })
  .inputValidator((data: { prompt: string; threadId: string }) => data)
  .handler(({ data }) => {
    return toServerSentEventsResponse(
      generateVideo({
        adapter: openaiVideo('sora-2'),
        prompt: data.prompt,
        threadId: data.threadId,
        stream: true,
      }),
    )
  })

import { useGenerateVideo } from '@tanstack/ai-react'
import { generateVideoStreamFn } from '../lib/server-functions'

const threadId = 'video-studio'

export function VideoGenerator() {
  const { generate, result, jobId, videoStatus, isLoading } = useGenerateVideo({
    fetcher: (input, { threadId }) =>
      generateVideoStreamFn({ data: { ...input, threadId } }),
    threadId,
  })
  // Render the same way as in the quick setup.
}

Image-to-video support#

What each provider does with image parts in the prompt. The roles are in Role hints.

• OpenAI: Sora-2 and Sora-2-Pro send the image part to input_reference. The text parts become the prompt. Only one image is allowed. More than one throws.
• fal.ai: the field names come from a map that is generated from the endpoint types of the fal SDK. For example, role: 'start_frame' goes to image_url on Kling and Veo image-to-video, to first_frame_url on first-last-frame endpoints, and to start_image_url on other endpoints.
• Gemini Veo: the first image with no role, or with 'start_frame', becomes the input image. 'end_frame' goes to lastFrame. 'reference' and 'character' go to referenceImages (asset references, Veo 3.1). More than one start image throws.
• BytePlus Seedance: see the list below.
• OpenRouter: see the list below.

The fal defaults, when the map has no entry for the endpoint:

• One input goes to image_url, as the start frame.
• role: 'end_frame' goes to end_image_url.
• role: 'reference' and 'character' go to reference_image_urls.
• To set a field for one endpoint, use modelOptions. The media fields there are typed optional, even when the endpoint requires them, because they usually come from prompt parts.

BytePlus Seedance:

• One image with no role, or with 'start_frame', becomes first_frame.
• 'end_frame' goes to last_frame. It needs a first frame next to it. seedance-1-0-pro-fast-251015 rejects it.
• 'reference' and 'character' go to reference_image. Video parts go to reference_video, and audio parts go to reference_audio. This works on Seedance 2.5 and the 2.0 family. Seedance 2.5 also accepts a reference that is only audio.
• Frame roles and reference roles are two separate modes. A request that mixes them throws.

OpenRouter:

• role: 'start_frame' and 'end_frame' go to frame_images[] with frame_type: 'first_frame' or 'last_frame'. An image with no role is the start frame.
• role: 'reference' and 'character' go to input_references[].
• You can send at most one start frame and one end frame. The adapter checks the frame roles against the supported_frame_images metadata of the model. For example, Hailuo only takes a first frame.
• If you send frame images and references together, OpenRouter treats the request as image-to-video. The references then have a lower priority.
• url sources pass through as they are. data sources become data URIs. OpenRouter does not follow redirects or pass bot checks, so use URLs that it can get directly.

Sizes and durations#

OpenAI Sora#

Sizes, from the OpenAI API docs:

Durations are 4, 8, or 12 seconds. You can write each one as a number, a numeric string, or a seconds template:

• 4, "4", or "4s" (the default)
• 8, "8", or "8s"
• 12, "12", or "12s"

The adapter sends "4", "8", or "12" to the API.

snapDuration() rules for Sora:

• When two values are at the same distance, the lower value wins. 6 and "6s" snap to "4". 7 snaps to "8".
• "auto" returns undefined, because Sora does not list "auto".

Typed durations on other adapters#

Adapters that declare durations for each model narrow duration at compile time. They also have two helpers:

• availableDurations() returns the set, for example { kind: 'discrete', values: [4, 6, 8] } or { kind: 'range', min: 3, max: 10, unit: 'seconds' }.
• snapDuration(value) returns the closest valid value.

An adapter with no duration map keeps duration?: number. Its availableDurations() returns { kind: 'none' }, and its snapDuration() returns undefined. fal is different: its duration is typed from the EndpointTypeMap of @fal-ai/client, even when the runtime map has no entry.

Provider options#

OpenAI (Sora)#

From the OpenAI Sora API:

import { generateVideo } from '@tanstack/ai'
import { openaiVideo } from '@tanstack/ai-openai'

const { jobId } = await generateVideo({
  adapter: openaiVideo('sora-2'),
  prompt: 'A beautiful sunset over the ocean',
  threadId: 'thread-1',
  size: '1280x720', // '1280x720', '720x1280', '1792x1024', '1024x1792'
  duration: '8s', // 4, "4", or "4s" (same forms for 8 and 12)
  modelOptions: {
    size: '1280x720', // another way to set the size
    seconds: '8', // API value: "4" | "8" | "12"
  },
})

Google Veo (Gemini)#

Veo runs on the long-running operations API of Google. The adapter starts the operation, and getVideoJobStatus polls it until the video is ready:

import { generateVideo } from '@tanstack/ai'
import { geminiVideo } from '@tanstack/ai-gemini'

const { jobId } = await generateVideo({
  adapter: geminiVideo('veo-3.1-generate-preview'),
  prompt: 'A close-up of a luthier carving a guitar neck',
  threadId: 'thread-1',
  size: '16:9', // aspect ratio: '16:9' or '9:16'
  duration: 8, // typed per model, see below
  modelOptions: {
    resolution: '1080p', // '720p' (default), '1080p', '4k' (Veo 3.1 only)
    negativePrompt: 'cartoon, low quality',
    generateAudio: true, // Veo 3 and later make synchronized audio
  },
})

Each Veo model accepts a fixed set of durations. TypeScript checks them on the duration option:

import { generateVideo } from '@tanstack/ai'
import { geminiVideo } from '@tanstack/ai-gemini'

const adapter = geminiVideo('veo-3.1-lite-generate-preview')

adapter.availableDurations() // { kind: 'discrete', values: [4, 6, 8] }
adapter.snapDuration(7) // 6, the closest valid duration

await generateVideo({
  adapter,
  prompt: 'A timelapse of a city skyline at dusk',
  threadId: 'thread-1',
  duration: adapter.snapDuration(7),
})

Gemini Omni Flash (Interactions API)#

Gemini Omni Flash (gemini-omni-1.1-flash) is a multimodal video model from Google that can edit by conversation. It only runs on the Interactions API. The same geminiVideo() adapter routes it automatically:

• generateVideo creates a background interaction.
• getVideoJobStatus polls it by id.
• The finished clip comes back inline as a data:video/mp4;base64,... URL. When Google sends a reference instead, the Files API URI passes through. Like Veo, it needs your API key to download.

Durations:

• duration accepts any value from 3 to 10 seconds, fractions included. The default is 10 seconds.
• availableDurations() returns { kind: 'range', min: 3, max: 10, unit: 'seconds' }.
• A duration out of range is rejected when the job is created.
• snapDuration(n) clamps raw seconds into the range and rounds to whole seconds.

size is an aspectRatio_resolution template, the same shape as Grok and BytePlus. Bare '16:9' or '9:16' uses the 720p default. Add a suffix for the other tiers: '360p', '1080p', or '4k'.

import { generateVideo, getVideoJobStatus } from '@tanstack/ai'
import { geminiVideo } from '@tanstack/ai-gemini'

const adapter = geminiVideo('gemini-omni-1.1-flash')

const { jobId } = await generateVideo({
  adapter,
  prompt: 'A woman playing violin outdoors at golden hour',
  threadId: 'thread-1',
  size: '9:16_1080p', // '16:9' | '9:16', optional _360p/_720p/_1080p/_4k
  duration: 6, // 3-10 seconds. Leave it out for the 10s default.
})

const status = await getVideoJobStatus({ adapter, jobId })
// status.url is 'data:video/mp4;base64,...' when the job is complete

Omni sends image and video prompt parts as content blocks: images first, then videos, then the text prompt. It does not use the metadata.role routing of Veo. You can give it stills or short reference clips:

• data sources go inline as base64.
• url sources pass through as they are. The adapter never downloads them, so use Gemini Files API URIs. Upload large media through the Files API first.

To edit a video, pass the interaction id of an earlier generation (its jobId) as modelOptions.previous_interaction_id. Then describe the change. The model keeps everything you do not mention:

import { generateVideo } from '@tanstack/ai'
import { geminiVideo } from '@tanstack/ai-gemini'

const adapter = geminiVideo('gemini-omni-1.1-flash')

// Turn 1: generate
const first = await generateVideo({
  adapter,
  prompt: 'A woman playing violin outdoors at golden hour',
  threadId: 'thread-1',
})

// Poll first.jobId until it is complete, then:

// Turn 2: edit the result
const second = await generateVideo({
  adapter,
  prompt: 'Make the violin invisible',
  threadId: 'thread-1',
  modelOptions: { previous_interaction_id: first.jobId },
})

modelOptions also passes through the request fields of the Interactions API. For example, generation_config.video_config.task sets the task mode to 'text_to_video', 'image_to_video', 'reference_to_video', or 'edit'. Without it, the model picks the mode.

Grok (xAI Imagine)#

From the xAI video generation API. There are two models:

• grok-imagine-video-1.5: the default that xAI recommends. It makes native 1080p from text. It also takes reference images. It has no video input.
• grok-imagine-video (v1.0): the only model that can edit or extend a clip.

Both do text-to-video and image-to-video. size is an aspectRatio_resolution template, like the Grok Imagine image models. Clips are 1 to 15 seconds long.

Text-to-video:

import { generateVideo } from '@tanstack/ai'
import { grokVideo } from '@tanstack/ai-grok'

const { jobId } = await generateVideo({
  adapter: grokVideo('grok-imagine-video-1.5'),
  prompt: 'A beautiful sunset over the ocean',
  threadId: 'thread-1',
  size: '16:9_720p', // aspect ratio: '1:1' | '16:9' | '9:16' | '4:3' | '3:4' | '3:2' | '2:3'
  // resolution (optional suffix): '480p' | '720p' | '1080p'
  duration: 5, // whole seconds, 1-15
  modelOptions: {
    aspect_ratio: '16:9', // another way to set the aspect ratio
    resolution: '720p', // another way to set the resolution
    duration: 5, // another way to set the duration
  },
})

Image-to-video: add an image prompt part as the start frame. The servers of xAI download url sources, so the URL must be public. For a base64 start frame, use a data source:

import { generateVideo } from '@tanstack/ai'
import { grokVideo } from '@tanstack/ai-grok'

const { jobId } = await generateVideo({
  adapter: grokVideo('grok-imagine-video-1.5'),
  prompt: [
    { type: 'text', content: 'Slowly pan out as the waves roll in' },
    {
      type: 'image',
      source: { type: 'url', value: 'https://example.com/still.png' },
    },
  ],
  threadId: 'thread-1',
  size: '16:9_720p',
  duration: 5,
})

Reference-to-video works on grok-imagine-video-1.5 only. The output is at most 720p.

• Image parts with metadata.role: 'reference' or 'character' become reference_images. Point at them in the prompt as <IMAGE_0>, <IMAGE_1>, and so on.
• modelOptions.reference_audios takes up to 3 preset TTS voices. Point at them as <AUDIO_0>, and so on.

import { generateVideo } from '@tanstack/ai'
import { grokVideo } from '@tanstack/ai-grok'

const { jobId } = await generateVideo({
  adapter: grokVideo('grok-imagine-video-1.5'),
  prompt: [
    { type: 'text', content: '<IMAGE_0> waves at the camera while <AUDIO_0> says hello' },
    {
      type: 'image',
      source: { type: 'url', value: 'https://example.com/character.png' },
      metadata: { role: 'reference' },
    },
  ],
  threadId: 'thread-1',
  size: '16:9_720p',
  modelOptions: { reference_audios: [{ voice_id: 'eve' }] },
})

Edit and extend work on grok-imagine-video only. Send the source clip as a video prompt part, and set modelOptions.mode:

• 'edit' (/v1/videos/edits) changes only what the prompt asks for. The output keeps the duration, aspect ratio, and resolution of the source, up to 720p.
• 'extend' (/v1/videos/extensions) continues the clip. Here duration is the length of the added part, not the total length.
• In both modes, the output keeps the properties of the source clip. As a result, the adapter rejects size, aspect_ratio, and resolution. In edit mode it also rejects duration.
• On grok-imagine-video-1.5, the adapter rejects a source video part and mode.

import { generateVideo } from '@tanstack/ai'
import { grokVideo } from '@tanstack/ai-grok'

const { jobId } = await generateVideo({
  adapter: grokVideo('grok-imagine-video'),
  prompt: [
    { type: 'text', content: 'The camera keeps panning right across the bay' },
    {
      type: 'video',
      source: { type: 'url', value: 'https://example.com/clip.mp4' },
    },
  ],
  threadId: 'thread-1',
  duration: 5, // 'extend' mode: seconds added to the clip, not the total
  modelOptions: { mode: 'extend' },
})

Both models accept any whole second from 1 to 15. The adapter does not reject a raw duration. It clamps the value to [1, 15] and rounds it to the nearest second:

import { grokVideo } from '@tanstack/ai-grok'

const adapter = grokVideo('grok-imagine-video-1.5')

adapter.availableDurations() // { kind: 'range', min: 1, max: 15, step: 1, unit: 'seconds' }
adapter.snapDuration(2.5) // 3
adapter.snapDuration(99) // 15

Generated clips have an audio track. When the job is complete, the result has two usage fields:

• usage.billed: { quantity, unit: 'seconds' }, the billed seconds of video.
• usage.cost: the exact cost in USD, as the API returns it.

BytePlus (Seedance)#

Seedance uses aspect-ratio sizes, like Grok Imagine. size takes a ratio or a ratio_resolution template:

• Ratios: 16:9, 9:16, 4:3, 3:4, 1:1, 21:9, and adaptive.
• Resolutions: 480p, 720p, and 1080p. dreamina-seedance-2-0-260128 also takes 4k.
• Seedance 2.5 (dreamina-seedance-2-5-260628) takes 480p, 720p, and 1080p, and runs up to 30 seconds.
• No Seedance model has a 2K tier.

import { generateVideo } from '@tanstack/ai'
import { byteplusVideo } from '@tanstack/ai-byteplus'

const { jobId } = await generateVideo({
  adapter: byteplusVideo('dreamina-seedance-2-5-260628'),
  prompt: 'A beautiful sunset over the ocean',
  threadId: 'thread-1',
  size: '16:9_720p',
  duration: 5,
  modelOptions: {
    seed: 42,
    generate_audio: true,
    priority: 5, // Seedance 2.5 and the 2.0 family: queue priority, 0-9
  },
})

Options depend on the model, and Ark checks them on the server. Ark rejects a field that does not apply with a 400. It does not ignore the field.

Seedance video URLs expire 24 hours after the task is complete. BytePlus keeps the task record for seven days. Store the bytes, not the link. Completed jobs report usage.billed as { quantity, unit: 'tokens' }, because Seedance bills output tokens only. The full option table is in the BytePlus adapter.

Move a Seedance call between providers#

You can reach Seedance through this adapter (direct to BytePlus) or through the fal adapter. The metadata.role values are the same in both. The size shape is different:

A bare size: '720p' works on fal and throws on BytePlus. BytePlus follows the Grok Imagine template and always needs the ratio. Write the ratio ('16:9_720p'), and the same string works on both.

The mode also differs. fal puts the mode in the endpoint id, for example fal-ai/bytedance/seedance/v1/pro/image-to-video or .../reference-to-video. BytePlus takes one model id and reads the mode from the prompt parts. You cannot change this. It follows the API of each provider.

OpenRouter#

The video generation API of OpenRouter runs Seedance, Veo, Wan, Kling, Sora 2 Pro, and more behind one async jobs API. size, duration, and the options below are typed and checked for each model, from the model data that OpenRouter publishes. A size or duration that the model does not support throws before the request goes out.

import { generateVideo } from '@tanstack/ai'
import { openRouterVideo } from '@tanstack/ai-openrouter'

const { jobId } = await generateVideo({
  adapter: openRouterVideo('bytedance/seedance-2.5'),
  prompt: 'A beautiful sunset over the ocean',
  threadId: 'thread-1',
  size: '1280x720', // a union for each model, from the OpenRouter model data
  duration: 8, // checked against the durations of the model
  modelOptions: {
    resolution: '720p', // another way to set the size: resolution + aspectRatio
    aspectRatio: '16:9',
    generateAudio: true, // not in the type for models that cannot
    seed: 42, // not in the type for models that cannot
    callbackUrl: 'https://your-app.com/webhooks/openrouter-video',
    provider: { options: { byteplus: { watermark: false } } }, // passthrough
  },
})

Like Veo, duration is typed for each model. It is the union of whole seconds in the model data. The adapter has the same availableDurations() and snapDuration() helpers:

import { generateVideo } from '@tanstack/ai'
import { openRouterVideo } from '@tanstack/ai-openrouter'

const adapter = openRouterVideo('bytedance/seedance-2.5')

adapter.availableDurations()
// { kind: 'discrete', values: [4, 5, 6, ..., 30] }
adapter.snapDuration(7.4) // 7

const sliderSeconds = 7 // raw seconds from a UI control
await generateVideo({
  adapter,
  prompt: 'A timelapse of clouds',
  threadId: 'thread-1',
  duration: adapter.snapDuration(sliderSeconds),
})

Two OpenRouter behaviors:

• The finished video comes as a stream. OpenRouter download URLs need your API key in an Authorization header, so a browser cannot load them. The adapter returns the download stream. With withGenerationPersistence, your blob store hosts it. Without it, url is a base64 data: URL. See Videos with no public URL.
• The cost comes on completion. The gateway reports the real billed cost of the job as usage.cost on the result.

fal.ai#

duration is typed for each endpoint from @fal-ai/client. Popular models also have availableDurations() and snapDuration():

Models with no duration field (MiniMax, Hunyuan) type duration as undefined, so a duration there is a compile error. The full table is in the fal adapter.

import { generateVideo } from '@tanstack/ai'
import { falVideo } from '@tanstack/ai-fal'

const adapter = falVideo('fal-ai/veo3.1')
adapter.availableDurations() // { kind: 'discrete', values: ['4s', '6s', '8s'] }

await generateVideo({
  adapter,
  prompt: 'A timelapse of a city skyline at dusk',
  threadId: 'thread-1',
  duration: adapter.snapDuration(7), // '6s'
})

Result types#

These are the adapter-level types. getVideoJobStatus() merges them into one object, { status, progress?, url?, error?, usage? }. That object has no jobId and no expiresAt.

import type { TokenUsage } from '@tanstack/ai'
import type { PersistedArtifactRef } from '@tanstack/ai/client'

// From create
interface VideoJobResult {
  jobId: string // the job id to poll with
  model: string // the model of the job
  artifacts?: Array<PersistedArtifactRef>
}

// From status
interface VideoStatusResult {
  jobId: string
  status: 'pending' | 'processing' | 'completed' | 'failed'
  progress?: number // 0-100, if available
  error?: string // the error message, if failed
}

// From url
interface VideoUrlResult {
  jobId: string
  url: string // the URL to download or stream the video
  expiresAt?: Date // when the URL expires
  // Usage for the finished job, when the adapter reports it.
  // fal reports usage.billed = { quantity, unit: 'units' } (from its
  // x-fal-billable-units header). Grok Imagine reports { quantity, unit: 'seconds' }.
  usage?: TokenUsage
}

How a video run differs from an image run:

Errors#

A job can fail at creation or while it runs. Catch errors from the create call, and read status.error when the poll returns 'failed':

import { generateVideo, getVideoJobStatus } from '@tanstack/ai'
import { openaiVideo } from '@tanstack/ai-openai'

try {
  const adapter = openaiVideo('sora-2')
  const { jobId } = await generateVideo({ adapter, prompt: 'A scene', threadId: 'thread-1' })

  const status = await getVideoJobStatus({ adapter, jobId })
  if (status.status === 'failed') {
    console.error('Generation failed:', status.error)
  }
} catch (error) {
  if (error instanceof Error) {
    if (error.message.includes('Video generation API is not available')) {
      console.error('Sora API access can be necessary. Look at your OpenAI account.')
    } else if (error.message.includes('rate limit')) {
      console.error('Rate limited. Wait, then try again.')
    } else {
      console.error('Unexpected error:', error)
    }
  }
}

With stream: true, these cases send RUN_ERROR to the hook:

• The provider reports 'failed'. The error message is the provider error, or "Video generation failed".
• The job runs longer than maxDuration. The error message is "Video generation timed out".

Limits and quotas#

Rate limits and quotas change, and can differ by account tier:

• Video generation costs a lot of compute.
• Providers can limit how many jobs run at the same time.
• Providers can set a monthly quota.
• Longer videos and higher quality use more of the quota.

For the current limits, see the OpenAI documentation.

Other limits to plan for:

• The Sora API is not available in all OpenAI accounts.
• A video can take several minutes.
• Video URLs can expire.
• Progress updates can be limited or late.
• Audio support can be limited.
• The provider can cut a long prompt.

Good habits:

• Set a timeout for your polling loop.
• Show clear progress while the job runs.
• Store video URLs, and look at their expiry before you use them again.
• Check the length and content of a prompt before you send it.
• Track your usage so you do not reach your quota.

API keys#

Each video adapter reads the same environment variable as the other adapters of its provider:

• OPENAI_API_KEY: OpenAI (Sora)
• GOOGLE_API_KEY or GEMINI_API_KEY: Google (Veo, Omni)
• ARK_API_KEY or BYTEPLUS_API_KEY: BytePlus ModelArk (Seedance)
• OPENROUTER_API_KEY: OpenRouter (openRouterVideo)
• FAL_KEY: fal.ai (falVideo)
• XAI_API_KEY: xAI (grokVideo)

To pass a key yourself, use the create* factory:

import { createOpenaiVideo } from '@tanstack/ai-openai'

const adapter = createOpenaiVideo('sora-2', 'your-openai-api-key')

Download image inputs on the server#

Two paths cannot pass an image URL to the provider. They must upload the bytes:

• OpenAI Sora input_reference.
• Gemini Veo. Its predict API accepts only inline bytes or a gs:// reference.

To download an HTTP(S) URL, the server must keep the full file in memory. This can use all the memory of a small runtime, such as Cloudflare Workers. As a result, these adapters throw on an HTTP(S) image URL by default. You have two options:

• Send a data: URI, or a gs:// reference for Veo.
• Pass allowUrlFetch: true to createOpenaiVideo or createGeminiVideo. The adapter then downloads the URL.

Image Generation has the same option for images.