Volcengine video to video lip sync

Endpoint

POST /videos/generations

Full URL: https://modelfold.ai/v1/videos/generations

Authentication: every request must include the Authorization: Bearer <API_KEY> header.

Send model parameters directly in the JSON body. Do not wrap them in input. This request waits for generation to finish and returns data[].url; supported media models also offer async jobs, polling and callbacks. Replace example media URLs with your own publicly accessible files. See API gateway for billing and errors.

Input parameters for the video-to-video lip sync task.

Request Parameters

Body (application/json)

Parameter	Type	Required	Description
model	string	Yes	The model name used for video lip sync. This field is required. - This endpoint must use the volcengine/video-to-video-lip-sync model Default："volcengine/video-to-video-lip-sync"。 Allowed values：volcengine/video-to-video-lip-sync。 Example：volcengine/video-to-video-lip-sync。
mode	string	Yes	Service mode for lip-sync generation. - lite: For single-person frontal videos. Faster processing. - basic: For single-person complex scenes. Supports scene segmentation and speaker identification. Allowed values：lite / basic。 Example：lite。
video_url	string	Yes	Video URL. Supported resolution: 360p–1080p. Videos above 1080p will be compressed to 1080p, while videos below 360p are not supported. Supported formats: MOV, MP4, HDR, MKV. Recommended codec: H.264. Other formats/codecs may be transcoded. Max file size: 500 MB. Bitrate: 1–30 Mbps. Frame rate: 24–60 fps.
audio_url	string	Yes	Target pure vocal audio URL; used to drive video lip movements. Audio duration: over 1 s and up to 240 s in lite mode; over 1 s and up to 210 s in basic mode. Accepted file types: audio/mpeg, audio/wav, audio/x-wav, audio/aac, audio/mp4, audio/ogg. Max file size: 10MB.
separate_vocal	boolean	No	Enable vocal separation to suppress background noise. Default value: false. Default：false。 Example：false。
open_scenedet	boolean	No	Whether to enable scene segmentation and speaker identification. Supported only in Basic mode. Default value: false. Default：false。 Example：false。
align_audio	boolean	No	Supported in Lite mode. Whether to loop the video when the audio is longer than the video. Default value: true. Default：true。 Example：true。
align_audio_reverse	boolean	No	Supported in Lite mode. Whether to loop the video in reverse (backward). Requires align_audio to be set to true. Default value: false. Default：false。 Example：false。
templ_start_seconds	number	No	Supported in Lite mode. Start time of the template video, in seconds. Default value: 0. Default：0。 Example：0。
prompt	string	Yes	Prompt required by the ModelFold gateway; at most 2000 characters. maxLength: 2000.

Request Example

curl -X POST "https://modelfold.ai/v1/videos/generations" \
  -H "Authorization: Bearer $MODELFOLD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "volcengine/video-to-video-lip-sync",
  "mode": "lite",
  "video_url": "https://example.com/reference.mp4",
  "audio_url": "https://example.com/reference.mp3",
  "separate_vocal": false,
  "open_scenedet": false,
  "align_audio": true,
  "align_audio_reverse": false,
  "templ_start_seconds": 0,
  "prompt": "A paper boat drifting down a rainy street."
}'

{
  "model": "volcengine/video-to-video-lip-sync",
  "mode": "lite",
  "video_url": "https://example.com/reference.mp4",
  "audio_url": "https://example.com/reference.mp3",
  "separate_vocal": false,
  "open_scenedet": false,
  "align_audio": true,
  "align_audio_reverse": false,
  "templ_start_seconds": 0,
  "prompt": "A paper boat drifting down a rainy street."
}

Response

200 — Generation completed. usage contains actual billing details; result URLs are illustrative.

{
  "ok": true,
  "created": 1792134000,
  "data": [
    {
      "url": "https://modelfold.ai/uploads/outputs/result.mp4"
    }
  ],
  "usage": {}
}