Skip to main content
POST
Analyze multimodal input with Gemini
Use the Responses endpoint with gemini-4-neon, gemini-4-hydrogen, gemini-4-carbon, gemini-4-argon, gemini-3.8-flash, gemini-3.7-flash, or gemini-3.5-flash-lite.

Video understanding request

The media input can be a public or signed HTTPS URL, or a Base64 data URL. Raw gs:// URIs are not accepted; use a signed HTTPS URL instead.

Response

Parameters

Video parts accept fps in the range (0, 24], startOffset, endOffset, and mediaResolution (low, medium, or high). Video sampling defaults to 1 FPS when fps is omitted. Put the text instruction after the video part for the most reliable video analysis. Gemini 3.5 Flash-Lite rejects a request whose final non-system message uses the assistant role. Use minimal reasoning for extraction and other straightforward, latency-sensitive tasks. See the Gemini 3.8 Flash, Gemini 3.7 Flash and Gemini 3.5 Flash-Lite model pages for supported modalities. Gemini 3.8 Flash defaults to medium reasoning and rejects minimal, unsupported reasoning levels, and thinkingBudget overrides. Use low, medium, or high.

Function calling and built-in tools

Function calling, Code execution, and Computer use are supported for all listed Gemini models. See Gemini tools for complete requests, multi-turn result submission, thought signature preservation, streaming behavior, and billing.

Streaming Responses

Set stream: true to receive typed Responses events as the model generates output:
The event sequence includes response.created, response.in_progress, response.output_item.added, content or function-argument deltas, corresponding done events, and a terminal response.completed, response.incomplete, or response.failed. Sequence numbers order events; item IDs and output indexes remain stable. The final response contains the full output and usage. For Function Call, consume response.function_call_arguments.delta and response.function_call_arguments.done. Gemini currently emits complete function arguments in a single delta per call. Computer use actions follow the same format. Code execution and media artifacts remain available in the output items’ providerMetadata.google.parts. Retain all final output items for the next turn. Wallet settlement completes before the terminal response event is emitted. If streaming or settlement fails, the endpoint emits event: error and closes without a successful terminal event. Responses uses typed terminal events rather than Chat Completions’ [DONE] marker. background: true remains unsupported. Streams must complete within the request time limit and cannot be resumed after disconnection.

Gemini 4 Carbon and Argon limits

gemini-4-carbon and gemini-4-argon share a 1,048,576-token window between input, reasoning, and output. The default output budget is 32,768 tokens for non-streaming requests and 65,536 with stream: true. Set a larger budget to enable automatic continuation. Reasoning defaults to medium; use low, medium, or high. Thinking budgets and minimal return HTTP 400. Custom temperature, top_p, and top_k values are ignored. Implicit caching is automatic; explicit cache management is not exposed.

Carbon, Hydrogen, and Neon conversation constraints

Gemini 4 Carbon, Hydrogen, and Neon reject frequency_penalty, presence_penalty, and candidate counts other than one (n or advanced candidateCount). End your history with a user message or completed tool results. Assistant prefills are not supported. Return every tool result in the order of the preceding calls, using the matching tool_call_id (Chat Completions) or call_id (Responses). Submit all pending results before another conversation turn. Preserve Google metadata as described in Gemini tools.

Gemini 4 Hydrogen and Neon

Hydrogen and Neon support Google Search, URL Context and Agentic video. Search citations and URL retrieval statuses are retained under providerMetadata.google on the response and its terminal SSE event. Use Count input tokens before generation without a wallet charge. gemini-4-neon and gemini-4-hydrogen default to 65,536 output tokens for synchronous and streaming requests. To allow a longer turn, set the output budget to an integer up to 1,048,576. Input, reasoning, tool calls, and final output share this window. GENGEN follows Gemini 4 continuation segments automatically while the request is open. Usage includes input processed again on later segments, cache reads, and generated tokens. Use streaming for large responses. Foreground calls have a 13-minute upstream deadline, and generation can finish before reaching the requested budget. Use reasoning_effort: "low", "medium", "high" (default), or "xhigh". Thinking budgets and minimal return HTTP 400. Custom temperature, top_p, and top_k are ignored. Implicit caching applies automatically to shared prefixes of at least 4,096 tokens; explicit cache references are not accepted. Preserve providerMetadata.google.parts on every returned assistant item, including final text and previous tool turns. Only the first function call in a parallel turn carries a thought signature. If your application edits or compacts previous history, set the following option; otherwise the default ERROR behavior rejects signatures whose original conversation prefix has changed:
DROP discards mismatched reasoning signatures while retaining visible history; it does not replace missing tool-call metadata. See Gemini tools. Supply Office and OpenDocument inputs through file / input_file content parts using file_data Base64 data URLs with the document’s correct MIME type. The same content parts are accepted inside tool results. Cloud Storage object references are not accepted by the GENGEN media-input interface.

Authorizations

Authorization
string
header
required

A workspace API key beginning with gengen_live_.

Body

application/json
model
enum<string>
required

Gemini model used for the response.

Available options:
gemini-4-neon,
gemini-4-hydrogen,
gemini-4-carbon,
gemini-4-argon,
gemini-3.8-flash,
gemini-3.7-flash,
gemini-3.5-flash-lite
input
required

Text or multimodal conversation input. Carbon, Hydrogen, and Neon require a final user or tool-result turn, with tool results in function-call order.

stream
boolean
default:false

Return typed Responses SSE events, including function argument deltas and terminal response usage.

background
boolean
default:false

Background execution is not supported.

instructions
string

High-level instructions sent as the Gemini system instruction.

n
integer

Carbon, Hydrogen, and Neon accept only one candidate. Omit this field or set it to 1.

reasoning_effort
enum<string>

Thinking level. Hydrogen and Neon support low, medium, high (default), and xhigh; thinking budgets and minimal are rejected. minimal is supported by Gemini 3.5 Flash-Lite but not Gemini 4 Carbon, Gemini 4 Argon, Gemini 3.8 Flash or Gemini 3.7 Flash. Gemini 4 Carbon, Gemini 4 Argon, and Gemini 3.8 Flash default to medium and reject thinking budgets.

Available options:
minimal,
low,
medium,
high,
xhigh
temperature
number

Sampling temperature. Custom values are ignored by Gemini 4 Neon, Gemini 4 Hydrogen, Gemini 4 Carbon, Gemini 4 Argon, and Gemini 3.5 Flash-Lite.

top_p
number

Nucleus-sampling probability threshold. Custom values are ignored by Gemini 4 Neon, Gemini 4 Hydrogen, Gemini 4 Carbon, Gemini 4 Argon, and Gemini 3.5 Flash-Lite.

Required range: 0 <= x <= 1
top_k
number

Limits sampling to the most likely K tokens. Custom values are ignored by Gemini 4 Neon, Gemini 4 Hydrogen, Gemini 4 Carbon, Gemini 4 Argon, and Gemini 3.5 Flash-Lite.

Required range: x >= 1
max_output_tokens
integer

Gemini 4 accepts integer output budgets from 1 through 1048576 with automatic continuation. Input and output share the context window. Neon and Hydrogen default to 65536. Carbon and Argon default to 32768 synchronously and 65536 with streaming. max_tokens and max_completion_tokens are also accepted.

Required range: x >= 1
max_tokens
integer
Required range: x >= 1
stop

One or more sequences that stop generation.

seed
integer

Best-effort sampling seed.

text
object

Structured-output settings for supported Gemini models.

tools
object[]

Function declarations. Chat uses nested function objects; Responses uses flat function declarations. Built-in tools belong in providerOptions.google.tools.

tool_choice

auto, none, required, or a named function object. The function name must be declared in tools.

Available options:
auto,
none,
required
parallel_tool_calls
boolean

Omit or use true. Gemini can return multiple calls; false is rejected.

providerOptions
object

Response

An OpenAI-compatible Responses object or typed SSE stream.

id
string
object
string
Allowed value: "response"
created_at
integer
status
enum<string>
Available options:
completed,
incomplete,
failed
model
string
output
object[]
output_text
string
providerMetadata
object

google.groundingMetadata contains sources, citation segments and Search suggestions; google.urlContextMetadata contains retrieval statuses. Available on the terminal SSE response too. Chat Completions carry these under choices[0].message (or delta). Preserve per-item google.parts for conversation replay.

usage
object