Gemini Decisions API
The AI That Answers in Milliseconds
How a model that never writes a word turns calibrated probabilities into the control plane of production software
Listen to the podcast
Inference
Breaking the Autoregressive Bottleneck at the Inference Layer
When software needs to inspect a 500-page compliance manual, review a 20-minute audio recording, and examine a dozen high-resolution photographs simultaneously to answer a thousand specific yes-or-no and routing questions, standard large language models hit a wall. Traditional generative models operate sequentially, taking tens of seconds and consuming substantial compute budgets to produce text one token at a time. At ainucu.com, AI News You Can Understand, we examine how Google's Gemini Decisions API and its underlying Gemini Decision 1 model (gemini-decision-1) invert this dynamic by returning calibrated mathematical probabilities in 150 to 250 milliseconds end to end.
To see why that speed jump happens, look at the mechanics of autoregressive generation. A standard LLM calculates the probability distribution for the first output token, emits it, appends that token back into the context window, and recalculates the state of the entire prompt to predict the second token. That loop repeats until a stop token appears. Because GPU memory bandwidth must repeatedly load and update the key-value cache for every generated token, latency scales directly with output length. Gemini Decision 1 eliminates the decoding loop completely. It executes one forward pass over the input context and maps that representation directly to a fixed set of numerical probabilities and calibrated confidence scores between 0 and 1.
Token by token, an answer of N tokens needs N sequential passes, each reloading a key-value cache that keeps growing. Gemini Decision 1 runs a single forward pass and returns a fixed set of probabilities between 0 and 1.
Chapter 1 of 6
Input
Native Multimodal Ingestion and Resolution Tuning
The input layer feeding that single forward pass accepts up to 256,000 tokens (262,144 tokens) across text, images, audio, video, and PDFs in one unified token space. In conventional pipelines, developers chain separate preprocessing services before an LLM ever sees the data: speech-to-text models transcribe audio, and optical character recognition (OCR) engines extract text from scanned PDFs. Each handoff adds latency and introduces cascading errors. If an upstream transcription model mishears a critical technical term, every downstream reasoning step inherits that mistake.
Native multimodal ingestion removes those intermediate conversion steps. Developers can also tune resolution per visual input inside the same request. Consider a hypothetical municipal transit safety audit: an agency ingests a 15-minute radio dispatch recording, a low-detail scanned maintenance log, and a high-resolution photograph of a rail switch assembly. Engineers can dial down visual resolution on the text-heavy maintenance scan to conserve input tokens while keeping maximum resolution on the rail switch photo so the model can detect hairline metal fatigue.
Set the scanned maintenance log to a low resolution and keep the rail switch photograph at maximum resolution. That saves input tokens where the scan carries little visual detail and preserves the detail needed to see hairline metal fatigue.
Output
The Three Bounded Decision Primitives
Because Gemini Decision 1 never generates free-form prose, every evaluation must fit one of three mathematically constrained primitives. Constraining the output space removes parsing ambiguity and guarantees machine-readable results:
noul(Yes/No Boolean Evaluator): Computes the probability that a defined condition holds true. Developers can supply optional true and false criteria descriptions to sharpen the decision boundary for domain-specific rules.choice(Multiple-Choice Selector): Evaluates between 2 and 255 discrete candidate options, returning both the winning label and the normalized probability distribution across every candidate so software can inspect runner-up likelihoods.score(Ordered Rubric): Grades an input along an ordered scale starting at 0 (supporting 2 to 255 levels), returning a continuous weighted score alongside individual per-level probabilities.
noul
Yes/No evaluator
One probability that a defined condition holds.
choice
Multiple-choice selector, 2 to 255 options
The winning label plus the distribution across every candidate.
score
Ordered rubric from 0, 2 to 255 levels
A continuous weighted score plus a probability for each level.
Match the task to the primitive: flagging profanity is noul, routing a request to 1 of 12 departments is choice, and grading call politeness from 0 to 4 is score.
Pick the primitive that fits each engineering task.
Try it: a freight inspection
Suppose a hypothetical logistics platform inspects incoming freight shipments using a loading-dock photo, a driver voice note, and a bill-of-lading PDF. Engineers need three outputs in one pass: (A) whether hazardous-material labels are missing, (B) which of 14 regional compliance queues should handle the shipment, and (C) an overall packaging integrity rating from 0 to 4. Which primitive maps to each task?
Show answer and feedback
Task A maps to noul because it evaluates a binary condition with a calibrated probability. Task B maps to choice because it selects one winner from 14 discrete queues (within the 2 to 255 limit) while exposing runner-up probabilities. Task C maps to score because it grades packaging integrity along an ordered numeric rubric starting at 0, yielding a continuous weighted score rather than a rigid category.
Economics
Latency Envelopes, Batching, and Zero-Output Economics
Removing the autoregressive decoding loop changes both speed and cloud billing math. Gemini Decision 1 delivers 50 to 100 milliseconds of median model compute latency, translating to 150 to 250 milliseconds end to end including network transit. That places responses inside the threshold of perceptual instantaneousness for interactive software. Furthermore, a single API request can bundle up to 1,000 independent questions against one shared multimodal input. For contexts up to 16,000 tokens, latency remains nearly flat from 1 to 100 questions because parallel classification heads evaluate the same prefilled context via matrix multiplication.
Billing follows the hardware physics. Because zero output tokens are generated, output cost is $0. Developers pay strictly for ingestion at $0.10 per 1 million input tokens, with the shared multimodal payload billed once per request regardless of how many questions are attached. Across shared benchmark suites, this makes the Decisions API nearly five times cheaper than using Gemini 3.5 Flash-Lite with the Interactions API, with no separate transcription or OCR surcharges.
One request can carry up to 1,000 questions against one shared input. Latency stays nearly flat from 1 to 100 questions for contexts up to 16,000 tokens, and the shared input is billed once at $0.10 per 1 million input tokens with $0 for output.
In Production
Production Patterns: Control Planes, Confidence Thresholds, and Guardrails
In production software, sub-second calibrated probabilities turn AI into a deterministic control plane rather than a slow content generator. Three patterns stand out across Google's own deployments:
- Dynamic Model Routing: Gemini Notebook uses the Decisions API as an inline millisecond router to classify incoming query intent and assign the right model tier and thinking budget, cutting everyday query latency by up to 40% while reserving heavy reasoning models for complex multi-step problems.
- Programmatic Confidence Thresholds and Generative UI: Because every response returns a calibrated score from 0 to 1, engineers can automate high-confidence paths (for example, auto-approving at 0.98) and escalate ambiguous cases (such as 0.52) to human review or a larger model. Google's Stitch rapid canvas experiment uses this speed to build playable wireframes in seconds and apply voice or tap edits without loading spinners.
- Consumer Speed and Zero-Trust Safety Guardrails: Google Labs CC picks instant emoji reactions from over 200 options in a fraction of a second, while Google Search deploys the Decisions API as an inline safety guardrail for Model Context Protocol (MCP) tools, screening autonomous tool payloads for adversarial behavior before execution.
Move the confidence bar higher and fewer requests run automatically; move it lower and more run without review. Adversarial requests are blocked either way.
Try it: an autonomous refund agent
Suppose an engineering team builds an autonomous customer-refund agent that can issue credits up to $500. Currently, a large reasoning model evaluates every refund request, taking 8 seconds per call and occasionally hallucinating policy exceptions. How should the team redesign this workflow using a decision control plane?
Show answer and feedback
Place Gemini Decision 1 in front of the action layer as a millisecond policy gate. Pass the customer's receipt photo, chat history, and policy rules into a single request with noul checks for eligibility and adversarial prompt injection, plus a choice check for refund tier. Auto-approve only when eligibility confidence exceeds a strict threshold (for example, 0.95) and safety checks pass; route ambiguous cases (for example, 0.45 to 0.94) to the heavy reasoning model or a human agent.
Ecosystem
Developer Ecosystem and the Structural Split in AI
Developers can prototype on a no-cost tier in Google AI Studio using first-party Python (google-genai SDK), JavaScript, and REST libraries, alongside working reference apps like Doorstep Parcels, Speed Sketch, and Find the Zoo Keeper. Ecosystem integrations span LangChain and Pydantic AI for agent orchestration, Pipecat and LiveKit for real-time voice agents, plus OpenRouter, Vercel, LiteLLM, and OpenCode, with managed deployment coming soon to Gemini in Google Cloud.
Taken together, the Decisions API highlights a clear structural split in modern AI engineering. While deep reasoning models continue pushing slower, multi-second problem solving, deterministic zero-output control planes are becoming the high-speed switchboard of production software, evaluating hundreds of multimodal criteria in milliseconds for fractions of a penny.
- Multi-step problem solving stays with deep reasoning models.
- Routing, guardrails, instant reactions and confidence gates belong to the decision control plane.
Watch the podcast on YouTube