Decision API: Turning Model Output Into Something Your Code Can Actually Trust
Ask a chat model for a yes-or-no answer and you get a paragraph. Ask it for JSON and you get JSON most of the time — frequently wrapped in markdown fences, occasionally missing a field, and almost always without a confidence number you would let near a checkout flow. For a prototype that is fine. For a triage queue, a warehouse gate, or a robot arm, it is a non-starter. The bottleneck is not model intelligence; it is interface. A decision API closes that gap by treating a decision as a typed return value rather than a sentence to be scraped.
The concept is deliberately narrow. You send state — text, JSON, or up to sixteen images or video frames — together with questions defined in code, and you receive schema-valid typed answers in a single pass. No chain-of-thought to parse, no retry-until-parseable loop, no prompt version drifting quietly in a config file. What follows is how that contract works in practice, the engineering numbers that decide whether it fits your architecture, and the patterns where it quietly replaces both rules engines and chat-LLM “reply in JSON” calls.
Typed Decisions, Not Prose: What a Decision API Actually Returns
A decision API is a single HTTPS endpoint — POST /v1/decide — where the request carries two things: the state to be judged, and the questions you want answered about it. The questions are not prose prompts; they are answer types declared in your application code. That distinction is the whole point. Because the expected shape is known before the request is sent, the caller knows exactly what will come back.
The answer types cover most operational decisions:
choice returns one label from a set you supply — billing, shipping, returns, abuse. score returns a level on an ordered scale, useful for severity, priority, or risk bands. truth returns a calibrated probability between 0.00 and 1.00, which is what you actually want when a threshold governs what happens next. number returns a value on a grid, for quantities and counts. And point and box return locations inside an image — a defect on a frame, a pallet in an aisle, a face in a crowd.
Everything validates against a schema. There is no regex layer, no defensive try/catch around JSON.parse, and no scenario where the model answers “Yes, absolutely!” when your code expected a boolean. That single property changes how the feature is tested: instead of fuzzy evaluations scored by another model, you build a golden set of request/response pairs and assert on them in CI.
The confidence attached to every answer is what makes the output actionable rather than merely structured. Each response carries a calibrated confidence, so your code can branch immediately: above the threshold, act; below it, re-check, route to a human queue, or take the safe path. Confidence stops being a display string and becomes a control-flow primitive — the difference between a system that guesses and a system that knows when it is guessing.
When the decision genuinely depends on pixels, optional boosters extend the same contract: human age estimation, AI-generated image detection, face detection and recognition, OCR, and depth or segmentation. Those signals arrive as part of the typed answer rather than as a separate vision pipeline bolted onto the side.
Latency, Determinism, and Cost: The Constraints That Actually Shape Architecture
Two numbers decide whether a decision service belongs in a synchronous request path: round-trip latency and per-call cost. DecisionNode publishes both plainly. DecisionNode Flash runs at roughly 5 ms server-side and 19 ms end to end, priced at $0.035 per 1M input tokens. DecisionNode MoE trades a little speed for a little more capability at about 9 ms server-side and $0.042 per 1M input tokens. Output tokens are always free — which matters when you are producing structured answers rather than long text — and there is a free daily tier plus batch discounts for offline scoring jobs.
To make that concrete: a million decisions averaging 200 input tokens each is 200M input tokens, or roughly seven dollars at Flash rates. That is the kind of arithmetic that moves a decision out of a nightly batch job and into the request handler.
Determinism is the second constraint, and it is routinely underestimated. The same request returns the same answer. That property unlocks four things teams usually give up on with generative models: aggressive caching keyed on the request payload, exact replay of historical traffic during incident review, regression tests that do not flake, and audit trails that survive a compliance review. Sampling-based LLMs cannot promise this even at temperature zero — different infrastructure, different day, different token.
Then there is throughput. Request/response is fine for a checkout or a ticket, but production loops can be far denser. Sessions over WebSocket handle decision loops above roughly ten per second, which is the regime drones, robots, simulators, and machine monitoring live in — one persistent connection, an ordered stream of decisions, no per-request handshake overhead.
Integration surface is intentionally unremarkable, which is a compliment. SDKs exist for TypeScript, Python, Go, Ruby, and curl, with MCP tools, webhooks, and AI agent integrations for teams wiring decisions into autonomous workflows. If your platform team already has an HTTP client and an auth pattern, the onboarding is measured in hours, not sprints.
Where Decision APIs Replace Rules Engines and Chat-LLM JSON
Most teams arrive at this architecture by exhausting two alternatives. The first is a hand-written rules engine. It is fully deterministic and extremely fast, and it works beautifully — until the rule table grows past a few hundred branches and every new edge case becomes a ticket. Business rules drift, taxonomies rot, and the operator who understood the original logic leaves. The engine becomes a liability precisely because it cannot generalize.
The second alternative is prompting a general chat model to reply in JSON. It generalizes well, but it is slow, nondeterministic, uncalibrated, and expensive at volume — and the parser you write around it quietly becomes a product of its own. That is the shape of the problem a decision api is built for: keep the generalization, keep the structured contract, drop the variance.
The pattern shows up in predictable places. Ticket routing and triage maps naturally to a choice over queue labels with a truth value for escalation, so low-confidence tickets never land in the wrong team’s backlog. Fraud and promo-abuse checks lean on calibrated probabilities with two thresholds — auto-approve above, auto-block below, human review in the gap. Order and shipping decisions combine choice and number to pick a carrier or flag an address. Moderation uses choice plus score to separate a hard violation from a borderline one. Gating asks a single truth question before a feature, discount, or payout unlocks.
On the operational side, the same primitives handle quality control on production lines and monitoring of warehouses and traffic — boxes and points on frames, with OCR or segmentation boosters resolving the text and boundaries that matter. And for AI agents, the “decide before acting” pattern is the highest-leverage use of all: check a truth value before a tool call, act above the threshold, ask for confirmation below it. Determinism keeps agent traces reproducible, which is the difference between debugging an agent and guessing at one.
What to instrument once it is in production: log the confidence alongside every decision, track the distribution of answers that fell below your threshold, and version your question definitions the same way you version a database migration. The threshold becomes the tuning surface — the schema stays fixed while behavior moves — and every change is testable against the same replayed golden set.
