Multimodal responses with Gemini

Analyze multimodal input with Gemini

curl --request POST \
  --url https://gengen.farm/api/gengen/v1/responses \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "model": "gemini-3.5-flash-lite",
  "input": [
    {
      "role": "user",
      "content": [
        {
          "type": "input_video",
          "video_url": "https://example.com/input.mp4",
          "fps": 2
        },
        {
          "type": "input_text",
          "text": "Describe the sequence of actions in this video."
        }
      ]
    }
  ]
}
'

import requests

url = "https://gengen.farm/api/gengen/v1/responses"

payload = {
    "model": "gemini-3.5-flash-lite",
    "input": [
        {
            "role": "user",
            "content": [
                {
                    "type": "input_video",
                    "video_url": "https://example.com/input.mp4",
                    "fps": 2
                },
                {
                    "type": "input_text",
                    "text": "Describe the sequence of actions in this video."
                }
            ]
        }
    ]
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    model: 'gemini-3.5-flash-lite',
    input: [
      {
        role: 'user',
        content: [
          {type: 'input_video', video_url: 'https://example.com/input.mp4', fps: 2},
          {type: 'input_text', text: 'Describe the sequence of actions in this video.'}
        ]
      }
    ]
  })
};

fetch('https://gengen.farm/api/gengen/v1/responses', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "id": "<string>",
  "object": "response",
  "created_at": 123,
  "status": "completed",
  "model": "<string>",
  "output": [
    {}
  ],
  "output_text": "<string>",
  "usage": {}
}

{
  "error": {
    "code": "<string>",
    "message": "<string>"
  }
}

POST

responses

Analyze multimodal input with Gemini

curl --request POST \
  --url https://gengen.farm/api/gengen/v1/responses \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "model": "gemini-3.5-flash-lite",
  "input": [
    {
      "role": "user",
      "content": [
        {
          "type": "input_video",
          "video_url": "https://example.com/input.mp4",
          "fps": 2
        },
        {
          "type": "input_text",
          "text": "Describe the sequence of actions in this video."
        }
      ]
    }
  ]
}
'

import requests

url = "https://gengen.farm/api/gengen/v1/responses"

payload = {
    "model": "gemini-3.5-flash-lite",
    "input": [
        {
            "role": "user",
            "content": [
                {
                    "type": "input_video",
                    "video_url": "https://example.com/input.mp4",
                    "fps": 2
                },
                {
                    "type": "input_text",
                    "text": "Describe the sequence of actions in this video."
                }
            ]
        }
    ]
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    model: 'gemini-3.5-flash-lite',
    input: [
      {
        role: 'user',
        content: [
          {type: 'input_video', video_url: 'https://example.com/input.mp4', fps: 2},
          {type: 'input_text', text: 'Describe the sequence of actions in this video.'}
        ]
      }
    ]
  })
};

fetch('https://gengen.farm/api/gengen/v1/responses', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "id": "<string>",
  "object": "response",
  "created_at": 123,
  "status": "completed",
  "model": "<string>",
  "output": [
    {}
  ],
  "output_text": "<string>",
  "usage": {}
}

{
  "error": {
    "code": "<string>",
    "message": "<string>"
  }
}

Use the Responses endpoint with gemini-3.8-flash, gemini-3.7-flash, or gemini-3.5-flash-lite.

Property	Value
Method	`POST`
Endpoint	`/api/gengen/v1/responses`
Models	`gemini-3.8-flash`, `gemini-3.7-flash`, `gemini-3.5-flash-lite`
Processing	Synchronous or SSE streaming

Video understanding request

The media input can be a public or signed HTTPS URL, or a Base64 data URL. Raw gs:// URIs are not accepted; use a signed HTTPS URL instead.

curl --request POST \
  --url https://gengen.farm/api/gengen/v1/responses \
  --header 'Authorization: Bearer gengen_live_xxxxxxxxxxxxxxxx' \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "gemini-3.8-flash",
    "input": [
      {
        "role": "user",
        "content": [
          {
            "type": "input_video",
            "video_url": "https://example.com/input.mp4"
          },
          {
            "type": "input_text",
            "text": "Describe the sequence of actions in this video."
          }
        ]
      }
    ],
    "reasoning_effort": "medium"
  }'

Response

{
  "id": "google-response-id",
  "object": "response",
  "status": "completed",
  "model": "gemini-3.8-flash",
  "output": [
    {
      "type": "message",
      "status": "completed",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "A person enters the greenhouse, waters the plants, and checks the temperature display."
        }
      ]
    }
  ],
  "output_text": "A person enters the greenhouse, waters the plants, and checks the temperature display."
}

Parameters

Parameter	Type	Description
`model`	string	`gemini-3.8-flash`, `gemini-3.7-flash`, or `gemini-3.5-flash-lite`.
`input`	string or array	Required text or multimodal input. Content parts may be `input_text`, `input_image`, `input_video`, or `input_audio`.
`stream`	boolean	Set to `true` for Responses SSE events. Defaults to `false`.
`instructions`	string	High-level instructions sent as the Gemini system instruction.
`reasoning_effort`	string	`low`, `medium`, or `high` for Gemini 3.8 Flash and Gemini 3.7 Flash; Gemini 3.5 Flash-Lite also supports and defaults to `minimal`.
`temperature`, `top_p`, `top_k`	number	Sampling controls. Gemini 3.5 Flash-Lite ignores custom values.
`max_tokens`	integer	Maximum generated tokens.
`stop`	string or string[]	One or more stop sequences.
`seed`	integer	Best-effort sampling seed.
`text.format`	object	`json_object` or `json_schema` structured output.
`providerOptions.google`	object	Advanced Google `generationConfig`, `safetySettings`, `tools`, and `toolConfig`.

Video parts accept fps in the range (0, 24], startOffset, endOffset, and mediaResolution (low, medium, or high). Google defaults video sampling to 1 FPS when fps is omitted. Put the text instruction after the video part for the most reliable video analysis. Gemini 3.5 Flash-Lite rejects a request whose final non-system message uses the assistant role. Use minimal reasoning for extraction and other straightforward, latency-sensitive tasks. See the Gemini 3.8 Flash, Gemini 3.7 Flash and Gemini 3.5 Flash-Lite model pages for supported modalities. Gemini 3.8 Flash defaults to medium reasoning and rejects minimal, unsupported reasoning levels, and thinkingBudget overrides. Use low, medium, or high.

Function calling and built-in tools

Function calling, Code execution, and Computer use are supported for all three Gemini models. See Function calling and Gemini tools for complete requests, multi-turn result submission, thought signature preservation, streaming behavior, and billing.

Streaming Responses

Set stream: true to receive typed Responses events as the model generates output:

curl --no-buffer https://gengen.farm/api/gengen/v1/responses \
  --header "Authorization: Bearer $GENGEN_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{"model":"gemini-3.8-flash","input":"Explain idempotency briefly.","stream":true}'

event: response.created
data: {"type":"response.created","sequence_number":0,"response":{"id":"google-response-id","object":"response","status":"in_progress","output":[]}}

event: response.output_text.delta
data: {"type":"response.output_text.delta","sequence_number":4,"item_id":"msg_google-response-id_0","output_index":0,"content_index":0,"delta":"Idempotency means"}

The event sequence includes response.created, response.in_progress, response.output_item.added, content or function-argument deltas, corresponding done events, and a terminal response.completed, response.incomplete, or response.failed. Sequence numbers order events; item IDs and output indexes remain stable. The final response contains the full output and usage. For Function Call, consume response.function_call_arguments.delta and response.function_call_arguments.done. Gemini currently emits complete function arguments in a single delta per call. Computer use actions follow the same format. Code execution and media artifacts remain available in the output items’ providerMetadata.google.parts. Retain all final output items for the next turn. Wallet settlement completes before the terminal response event is emitted. If streaming or settlement fails, the endpoint emits event: error and closes without a successful terminal event. Missing provider usage is retained for reconciliation. Responses uses typed terminal events rather than Chat Completions’ [DONE] marker. background: true remains unsupported. This is a live HTTP stream subject to the Vercel function time limit, not a persisted background task or a resumable stream.

Authorizations

Authorization

string

header

required

A workspace API key beginning with gengen_live_.

Body

application/json

model

enum<string>

required

Gemini model used for the response.

Available options:

gemini-3.8-flash,

gemini-3.7-flash,

gemini-3.5-flash-lite

input

required

Text or multimodal conversation input.

stream

boolean

default:false

Return typed Responses SSE events, including function argument deltas and terminal response usage.

background

boolean

default:false

Background execution is not supported.

instructions

string

High-level instructions sent as the Gemini system instruction.

reasoning_effort

enum<string>

Thinking level. minimal is supported by Gemini 3.5 Flash-Lite but not Gemini 3.8 Flash or Gemini 3.7 Flash. Gemini 3.8 Flash defaults to medium and rejects thinking budgets.

Available options:

minimal,

low,

medium,

high

temperature

number

Sampling temperature. Custom values are ignored by Gemini 3.5 Flash-Lite.

top_p

number

Nucleus-sampling probability threshold. Custom values are ignored by Gemini 3.5 Flash-Lite.

Required range: 0 <= x <= 1

top_k

number

Limits sampling to the most likely K tokens. Custom values are ignored by Gemini 3.5 Flash-Lite.

Required range: x >= 1

max_output_tokens

integer

Maximum output tokens. max_tokens and max_completion_tokens are also accepted.

Required range: x >= 1

max_tokens

integer

Required range: x >= 1

stop

One or more sequences that stop generation.

seed

integer

Best-effort sampling seed.

text

object

Structured-output settings for supported Gemini models.

Show child attributes

tools

object[]

Function declarations. Chat uses nested function objects; Responses uses flat function declarations. Built-in tools belong in providerOptions.google.tools.

tool_choice

auto, none, required, or a named function object. The function name must be declared in tools.

Available options:

auto,

none,

required

parallel_tool_calls

boolean

Omit or use true. Gemini can return multiple calls; false is rejected.

providerOptions

object

Show child attributes

Response

An OpenAI-compatible Responses object or typed SSE stream.

string

object

string

Allowed value: "response"

created_at

integer

status

enum<string>

Available options:

completed,

incomplete,

failed

model

string

output

object[]

output_text

string

usage

object

Reference

BytePlus

OpenAI

Kling

Higgsfield

MiniMax

Happy Horse

Google

WonderClip

Flash VSR

Wan

Qwen

Multimodal responses with Gemini

Video understanding request

Response

Parameters

Function calling and built-in tools

Streaming Responses

Authorizations

Body

Response

​Video understanding request

​Response

​Parameters

​Function calling and built-in tools

​Streaming Responses

Authorizations

Body

Response

Video understanding request

Response

Parameters

Function calling and built-in tools

Streaming Responses