Kling 3.0 Omni Reference To Video

Input mode 1: No Video Input

Endpoint

POST /videos/generations

Full URL: https://modelfold.ai/v1/videos/generations

Authentication: every request must include the Authorization: Bearer <API_KEY> header.

Send model parameters directly in the JSON body. Do not wrap them in input. This request waits for generation to finish and returns data[].url; supported media models also offer async jobs, polling and callbacks. Replace example media URLs with your own publicly accessible files. See API gateway for billing and errors.

Input parameters for the generation task. Rules: (1) customize_multi_shots and prefer_multi_shots cannot both be true; (2) customize_multi_shots = true requires 1–6 multi_prompt shots (each 1–12 s, total ≤ 15 s); (3) otherwise omit multi_prompt.

Request Parameters

Body (application/json)

Parameter	Type	Required	Description
model	string	Yes	The name of the model used to generate the task. This field is required. - This endpoint requires the kling-3.0-omni/reference-to-video model Default："kling-3.0-omni/reference-to-video"。 Allowed values：kling-3.0-omni/reference-to-video。 Example：kling-3.0-omni/reference-to-video。
prompt	string	Yes	Prompt required by the ModelFold gateway; at most 2000 characters. Video description. It must not be empty after leading and trailing whitespace is removed. Maximum length: 3,072 characters. minLength: 1. maxLength: 2000. Example：Generate a video of the character walking along a city street, using the character appearance in the reference images.。
image_urls	array<string>	Yes	Array of reference image URLs. The maximum number of images depends on the number and types of reference subjects. When there is no reference video and only multi-image subjects are used, the combined number of reference images and multi-image subjects must not exceed 7. When there is no reference video and both video character and multi-image subjects are used, the combined number of reference images and multi-image subjects must not exceed 4. HTTP, HTTPS, and OSS URLs are supported. Images must be in JPG, JPEG, or PNG format; each file must not exceed 50 MB; both width and height must be at least 300 px; and the aspect ratio must be between 0.4 and 2.5. maxItems: 7.
duration	integer	No	Total video duration in seconds. Optional; the default is 5 seconds. Supports integer values from 3 to 15. Default：5。 Allowed values：3 / 4 / 5 / 6 / 7 / 8 / 9 / 10 / 11 / 12 / 13 / 14 / 15。 minimum: 3. maximum: 15. Example：5。
resolution	string	No	Video resolution. 720p outputs a 720p video, 1080p outputs a 1080p video, and 4k outputs a 4K video. Default："720p"。 Allowed values：720p / 1080p / 4k。 Example：720p。
aspect_ratio	string	No	When no video is provided, 16:9, 9:16, and 1:1 are supported. Default："16:9"。 Allowed values：16:9 / 9:16 / 1:1。 Example：16:9。
audio	boolean	No	Whether to enable audio. When no video is provided, both true and false are supported. Example：true。
customize_multi_shots	boolean	No	Optional, default false. Set true for custom multi-shot (multi_prompt required). Default：false。 Example：true。
prefer_multi_shots	boolean	No	Whether to enable intelligent shot planning. This field is mutually exclusive with customize_multi_shots. Example：true。
multi_prompt	array<object>	No	List of custom shots. When customize_multi_shots=true, this field is required, must not be empty, and may contain up to 6 shots. When customize_multi_shots=false, it must be an empty array or omitted. minItems: 1. maxItems: 6.
multi_prompt[].prompt	string	Yes	Video description for the current shot. Maximum length: 512 characters. minLength: 1. maxLength: 512.
multi_prompt[].duration	integer	Yes	Duration of this shot in seconds, 1–12. The sum of all shots must not exceed 15. minimum: 1. maximum: 12.
elements	array<object>	No	List of one-time subject assets. The default is an empty array. The number of subjects is jointly limited by the reference images, reference video, and subject types. When there is no reference video and only multi-image subjects are used, the combined number of reference images and multi-image subjects must not exceed 7. When there is no reference video and only video character subjects are used, the number of video character subjects must not exceed 3. When there is no reference video and both video character and multi-image subjects are used, the number of video character subjects must not exceed 3, and the combined number of reference images and multi-image subjects must not exceed 4. Default：[]。
elements[].name	string	Yes	Subject name. It must be unique within the same request and can be referenced in the prompt using @name. Example：element_dog。
elements[].description	string	Yes	Subject description. Example：A happy golden retriever。
elements[].element_input_urls	array<string>	Yes	URLs of the subject image or video assets. Provide 1 to 4 URLs. minItems: 1. maxItems: 4.
elements[].element_input_audio_urls	array<string>	No	Optional list of subject reference audio URLs. If the duration can be determined, it must be between 5 and 30 seconds. Default：[]。
elements[].start_time	integer	No	Start time for trimming the video character subject, in milliseconds. The default is 0. Default：0。 Example：0。
elements[].end_time	integer	No	End time for trimming the video character subject, in milliseconds. The default is 8,000. Default：8000。 Example：8000。

Request Example

curl -X POST "https://modelfold.ai/v1/videos/generations" \
  -H "Authorization: Bearer $MODELFOLD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "kling-3.0-omni/reference-to-video",
  "prompt": "Generate a video of the character walking along a city street, using the character appearance in the reference images.",
  "image_urls": [
    "https://example.com/reference.png"
  ],
  "duration": 5,
  "resolution": "720p",
  "aspect_ratio": "16:9",
  "elements": [
    {
      "name": "element_dog",
      "description": "A happy golden retriever",
      "element_input_urls": [
        "https://example.com/reference.png"
      ]
    }
  ]
}'

{
  "model": "kling-3.0-omni/reference-to-video",
  "prompt": "Generate a video of the character walking along a city street, using the character appearance in the reference images.",
  "image_urls": [
    "https://example.com/reference.png"
  ],
  "duration": 5,
  "resolution": "720p",
  "aspect_ratio": "16:9",
  "elements": [
    {
      "name": "element_dog",
      "description": "A happy golden retriever",
      "element_input_urls": [
        "https://example.com/reference.png"
      ]
    }
  ]
}

Response

200 — Generation completed. usage contains actual billing details; result URLs are illustrative.

{
  "ok": true,
  "created": 1792134000,
  "data": [
    {
      "url": "https://modelfold.ai/uploads/outputs/result.mp4"
    }
  ],
  "usage": {}
}

Input mode 2: Video Input Only

Endpoint

POST /videos/generations

Full URL: https://modelfold.ai/v1/videos/generations

Authentication: every request must include the Authorization: Bearer <API_KEY> header.

Send model parameters directly in the JSON body. Do not wrap them in input. This request waits for generation to finish and returns data[].url; supported media models also offer async jobs, polling and callbacks. Replace example media URLs with your own publicly accessible files. See API gateway for billing and errors.

Input parameters for the generation task. Rules: (1) customize_multi_shots and prefer_multi_shots cannot both be true; (2) customize_multi_shots = true requires 1–6 multi_prompt shots (each 1–12 s, total ≤ 15 s); (3) otherwise omit multi_prompt.

Request Parameters

Body (application/json)

Parameter	Type	Required	Description
model	string	Yes	The name of the model used to generate the task. This field is required. - This endpoint requires the kling-3.0-omni/reference-to-video model Default："kling-3.0-omni/reference-to-video"。 Allowed values：kling-3.0-omni/reference-to-video。 Example：kling-3.0-omni/reference-to-video。
prompt	string	Yes	Prompt required by the ModelFold gateway; at most 2000 characters. Video description. It must not be empty after leading and trailing whitespace is removed. Maximum length: 3,072 characters. minLength: 1. maxLength: 2000. Example：Generate natural, smooth shot transitions based on the input video.。
video_urls	array<string>	Yes	Array of video URLs. Exactly 1 video is required. HTTP, HTTPS, and OSS URLs are supported. Videos must be in MP4 or MOV format; each file must not exceed 200 MB; duration must be between 3 and 15.5 seconds; width and height must each be between 700 and 4,553 px; total pixels must not exceed 8,294,400; the aspect ratio must be between 0.4 and 2; and the frame rate must be between 24 and 60 fps. minItems: 1. maxItems: 1.
resolution	string	No	Video resolution. 720p outputs a 720p video, 1080p outputs a 1080p video, and 4k outputs a 4K video. Default："720p"。 Allowed values：720p / 1080p / 4k。 Example：720p。
aspect_ratio	string	Yes	Only auto can be used with video-only input. Default："auto"。 Allowed values：auto。 Example：auto。
audio	boolean	Yes	No audio is generated when a video is provided; set false. Example：false。
elements	array<object>	No	List of one-time subject assets. The number of subjects is jointly limited by the reference images, reference video, and subject types. - When a reference video and only multi-image subjects are used, the combined number of reference images and multi-image subjects must not exceed 4. - When a reference video and only video character subjects are used, the number of video character subjects must not exceed 1. - When a reference video is provided, video character subjects cannot be used together with multi-image subjects or reference images. Default：[]。
elements[].name	string	Yes	Subject name. It must be unique within the same request and can be referenced in the prompt using @name. Example：element_dog。
elements[].description	string	Yes	Subject description. Example：A happy golden retriever。
elements[].element_input_urls	array<string>	Yes	URLs of the subject image or video assets. Provide 1 to 4 URLs. minItems: 1. maxItems: 4.
elements[].element_input_audio_urls	array<string>	No	Optional list of subject reference audio URLs. If the duration can be determined, it must be between 5 and 30 seconds. Default：[]。
elements[].start_time	integer	No	Start time for trimming the video character subject, in milliseconds. The default is 0. Default：0。 Example：0。
elements[].end_time	integer	No	End time for trimming the video character subject, in milliseconds. The default is 8,000. Default：8000。 Example：8000。

Request Example

curl -X POST "https://modelfold.ai/v1/videos/generations" \
  -H "Authorization: Bearer $MODELFOLD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "kling-3.0-omni/reference-to-video",
  "prompt": "Generate natural, smooth shot transitions based on the input video.",
  "video_urls": [
    "https://example.com/reference.mp4"
  ],
  "aspect_ratio": "auto",
  "audio": false,
  "resolution": "720p",
  "elements": [
    {
      "name": "element_dog",
      "description": "A happy golden retriever",
      "element_input_urls": [
        "https://example.com/reference.png"
      ]
    }
  ]
}'

{
  "model": "kling-3.0-omni/reference-to-video",
  "prompt": "Generate natural, smooth shot transitions based on the input video.",
  "video_urls": [
    "https://example.com/reference.mp4"
  ],
  "aspect_ratio": "auto",
  "audio": false,
  "resolution": "720p",
  "elements": [
    {
      "name": "element_dog",
      "description": "A happy golden retriever",
      "element_input_urls": [
        "https://example.com/reference.png"
      ]
    }
  ]
}

Response

200 — Generation completed. usage contains actual billing details; result URLs are illustrative.

{
  "ok": true,
  "created": 1792134000,
  "data": [
    {
      "url": "https://modelfold.ai/uploads/outputs/result.mp4"
    }
  ],
  "usage": {}
}

Input mode 3: Video with Images

Endpoint

POST /videos/generations

Full URL: https://modelfold.ai/v1/videos/generations

Authentication: every request must include the Authorization: Bearer <API_KEY> header.

Send model parameters directly in the JSON body. Do not wrap them in input. This request waits for generation to finish and returns data[].url; supported media models also offer async jobs, polling and callbacks. Replace example media URLs with your own publicly accessible files. See API gateway for billing and errors.

Input parameters for the generation task. Rules: (1) customize_multi_shots and prefer_multi_shots cannot both be true; (2) customize_multi_shots = true requires 1–6 multi_prompt shots (each 1–12 s, total ≤ 15 s); (3) otherwise omit multi_prompt.

Request Parameters

Body (application/json)

Parameter	Type	Required	Description
model	string	Yes	The name of the model used to generate the task. This field is required. - This endpoint requires the kling-3.0-omni/reference-to-video model Default："kling-3.0-omni/reference-to-video"。 Allowed values：kling-3.0-omni/reference-to-video。 Example：kling-3.0-omni/reference-to-video。
prompt	string	Yes	Prompt required by the ModelFold gateway; at most 2000 characters. Video description. It must not be empty after leading and trailing whitespace is removed. Maximum length: 3,072 characters. minLength: 1. maxLength: 2000. Example：Generate a video of the character walking along a city street, using the character shown in the reference video and images.。
image_urls	array<string>	Yes	Array of reference image URLs. The maximum number of images depends on the number and types of reference subjects. When only multi-image subjects are used, the combined number of reference images and multi-image subjects must not exceed 4. Video character subjects cannot be used together with multi-image subjects or reference images. HTTP, HTTPS, and OSS URLs are supported. Images must be in JPG, JPEG, or PNG format; each file must not exceed 50 MB; both width and height must be at least 300 px; and the aspect ratio must be between 0.4 and 2.5. maxItems: 4.
video_urls	array<string>	Yes	Array of video URLs. Exactly 1 video is required. HTTP, HTTPS, and OSS URLs are supported. Videos must be in MP4 or MOV format; each file must not exceed 200 MB; duration must be between 3 and 15.5 seconds; width and height must each be between 700 and 4,553 px; total pixels must not exceed 8,294,400; the aspect ratio must be between 0.4 and 2; and the frame rate must be between 24 and 60 fps. minItems: 1. maxItems: 1.
duration	integer	No	Total video duration in seconds. Optional; the default is 5 seconds. Supports integer values from 3 to 15. Default：5。 Allowed values：3 / 4 / 5 / 6 / 7 / 8 / 9 / 10 / 11 / 12 / 13 / 14 / 15。 minimum: 3. maximum: 15. Example：5。
resolution	string	No	Video resolution. 720p outputs a 720p video, 1080p outputs a 1080p video, and 4k outputs a 4K video. Default："720p"。 Allowed values：720p / 1080p / 4k。 Example：720p。
aspect_ratio	string	No	Video aspect ratio. Default："16:9"。 Allowed values：16:9 / 9:16 / 1:1。 Example：16:9。
audio	boolean	Yes	No audio is generated when a video is provided; set false. Example：false。
elements	array<object>	No	List of one-time subject assets. When a reference video and only multi-image subjects are used, the combined number of reference images and multi-image subjects must not exceed 4. When a reference video and only video character subjects are used, the number of video character subjects must not exceed 1. When a reference video is provided, video character subjects cannot be used together with multi-image subjects or reference images. Note: This field is unavailable when video input is provided and customize_multi_shots=false (it must be an empty array or omitted). Default：[]。
elements[].name	string	Yes	Subject name. It must be unique within the same request and can be referenced in the prompt using @name. Example：element_dog。
elements[].description	string	Yes	Subject description. Example：A happy golden retriever。
elements[].element_input_urls	array<string>	Yes	URLs of the subject image or video assets. Provide 1 to 4 URLs. minItems: 1. maxItems: 4.
elements[].element_input_audio_urls	array<string>	No	Optional list of subject reference audio URLs. If the duration can be determined, it must be between 5 and 30 seconds. Default：[]。
elements[].start_time	integer	No	Start time for trimming the video character subject, in milliseconds. The default is 0. Default：0。 Example：0。
elements[].end_time	integer	No	End time for trimming the video character subject, in milliseconds. The default is 8,000. Default：8000。 Example：8000。

Request Example

curl -X POST "https://modelfold.ai/v1/videos/generations" \
  -H "Authorization: Bearer $MODELFOLD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "kling-3.0-omni/reference-to-video",
  "prompt": "Generate a video of the character walking along a city street, using the character shown in the reference video and images.",
  "image_urls": [
    "https://example.com/reference.png"
  ],
  "video_urls": [
    "https://example.com/reference.mp4"
  ],
  "audio": false,
  "duration": 5,
  "resolution": "720p",
  "aspect_ratio": "16:9",
  "elements": [
    {
      "name": "element_dog",
      "description": "A happy golden retriever",
      "element_input_urls": [
        "https://example.com/reference.png"
      ]
    }
  ]
}'

{
  "model": "kling-3.0-omni/reference-to-video",
  "prompt": "Generate a video of the character walking along a city street, using the character shown in the reference video and images.",
  "image_urls": [
    "https://example.com/reference.png"
  ],
  "video_urls": [
    "https://example.com/reference.mp4"
  ],
  "audio": false,
  "duration": 5,
  "resolution": "720p",
  "aspect_ratio": "16:9",
  "elements": [
    {
      "name": "element_dog",
      "description": "A happy golden retriever",
      "element_input_urls": [
        "https://example.com/reference.png"
      ]
    }
  ]
}

Response

200 — Generation completed. usage contains actual billing details; result URLs are illustrative.

{
  "ok": true,
  "created": 1792134000,
  "data": [
    {
      "url": "https://modelfold.ai/uploads/outputs/result.mp4"
    }
  ],
  "usage": {}
}