When Jev Returns 429: Decide Fail-Open or Fail-Closed First

At 02:14 an unattended lane proposes git push --force to a release branch, and the Jev gate in front of it gets back 429 Too Many Requests. What happens next was decided weeks ago, either by whoever wrote the gate or by nobody. If nobody, the SDK decided: it waits, retries, waits again, and the lane sits there looking busy.

A Jev rate limit is not an edge case this month. TypeSafe publishes two limits, says both are moving, and ships SDKs that retry quietly by default. So every Jev gate has three answers (yes, no and nothing yet), and this runbook is about the third. By Tuesday each gate carries a written fail mode, both limits have a fleet-wide budget, timeouts come from latency you measured, SDK retries fit inside them, and four drills prove the late, expired, queued and rules-fallback paths work.

Chatbots suggest; agents act. An agent waiting on a verdict is still holding the tool call it was about to make.

Sep 15–21: Jev’s rate limits are two numbers that can move without notice

TypeSafe launched Jev on Sep 15, 2026, “available today in early access,” per its launch post. On Sep 20 it opened access to everyone, no waitlist. More users on the same capacity is when limits move, and the models page says so.

Two limits apply: 250,000 tokens per second and 1,200 requests per minute, and a request over either returns 429 Too Many Requests. A warning box adds that the limits are “adjusting dynamically” and “can change without notice” while TypeSafe serves heavy demand, lands new GPU capacity and lets in more users. The page does not say whether the limits apply per key, per account or across the service.

Screenshot: TypeSafe AI docs, “Models - TypeSafe AI” (undated), captured Sep 21, 2026.

The API reference lists four errors: 401, 422, 429 and 529. A 429 is about your traffic: “You have exceeded your rate limit. Back off and retry after a short delay.” A 529 is about theirs: “TypeSafe is temporarily overloaded. Retry after a short delay.”

For both, the reference says the client SDKs back off automatically, so a client on its default retry policy needs no extra handling. Sound advice for a batch job; for a gate it is the problem, because the defaults are generous (step 4).

On latency, the launch post says “End-to-end response time is 70ms-500ms for TypeSafe,” and notes that its published evals are generally run from laptops on the West Coast, where the service is based. A vendor-reported range from one coast is not an SLA, and it is not your p99.

Why a Jev gate needs its fail mode before its threshold

A classifier gate sits between an actor and an action. When the verdict doesn’t arrive, something still happens to the action: it waits, it proceeds, or it stops. If you haven’t chosen, the harness or the SDK chooses: on Claude Code, a command hook that times out doesn’t block, and the call continues through the normal permission flow. A TypeSafe JavaScript client on default settings can hold one call for minutes.

Rehearsing a provider cutoff already separates timeouts from quota and drills the move to another model. This is the narrower case: the model is up and the verdict still doesn’t come in time. Where gates belong is settled too. What follows is what a gate does when it can’t decide.

Step 1: Write each Jev gate’s fail mode before go-live

One row per gate, in the same file as its thresholds, reviewed before the gate enforces anything. “Fail open” and “fail closed” are too coarse for the four seats Jev usually takes, so name the action that runs when no verdict arrives.

Gate	Fail mode when no verdict arrives	What actually happens	Why
Write and destructive tool calls	Closed	Deny with a reason; unattended lanes pause the task for a person	A wrong deny costs a retry; a wrong pass costs the branch
Read-only tool calls (Jev as an extra veto)	Open to static rules	No Jev opinion; deny and ask rules, the prompt and the sandbox still decide	The veto was an extra; the wall behind it is unchanged
Model-tier routing	Fixed default route	One named route, the same every time, logged as a fallback	A known route beats a guess
Alert and ticket triage	Deterministic rules	Owner map and keyword rules; output tagged for re-triage	Triage can be coarse for an hour; it can’t stop

OpenRouter’s gate cookbook makes the same call for the closed row: its client throws on a non-2xx response, a missing answer or a probability outside 0 to 1, “so a broken check never turns into an approval or a review.”

Three rules keep the table honest.

Open never means allow. A gate that fails open returns no opinion, which leaves the call to whatever sat behind the gate before Jev arrived. If nothing sat behind it, you don’t have a fail-open gate. You have an unguarded tool with a latency budget.

The default route is fixed, logged and cleared for every data class. The fallback doesn’t know which repo it carries, so its route must be allowed for the most sensitive one. Check it against the last-hop register: the default is a lane whose last hop you have verified for every data class you route. It also never moves work down a tier: destructive and long-horizon tasks stay where the router’s never-route list pins them, so a 429 can send work up or sideways, never down. If no single lane is cleared for everything, a class without a cleared route fails closed: the task waits.

Rules exist before they are needed. The triage fallback is a file of deterministic rules (paths to owners, severity keywords, a catch-all queue), tested on go-live day, not drafted during the outage.

# gates.yaml (illustrative shape); enforced by the gate wrapper, never by a prompt
gates:
  tool_destructive:
    classes: [write, destructive]
    on_no_verdict: deny
    unattended: pause_task        # a person picks it up; see step 5
  tool_read:
    classes: [read]
    on_no_verdict: no_opinion     # static rules and the sandbox still apply
  router:
    on_no_verdict: route
    default_route: cleared-all-classes
    never_down_tier: [destructive, long_horizon]
  triage:
    on_no_verdict: rules
    rules_file: triage-rules.yaml
    tag: rules-fallback           # re-triage when Jev is back

Thresholds, the part that turns a verdict into a decision, live in the Jev confidence-gate table. This file covers only the case where there is no verdict to threshold.

Step 2: Budget both Jev rate limits fleet-wide

1,200 requests per minute is 20 a second. 250,000 tokens per second divided by 20 requests is 12,500 tokens each. That is the crossover: below about 12.5K tokens per request the request limit binds first, and above it the token limit does. A gate that sends 30K-token requests, inside the 32K budget for state plus the longest question, runs out at 500 requests a minute, not 1,200.

Derived from TypeSafe’s two published limits as of Sep 21, 2026; both can change without notice. Your share of the pool is smaller than any bar.

Plan as if every lane shares both limits. TypeSafe doesn’t say whether they apply per key or per account, so a key per lane isolates nothing you can confirm. Allocate one pool.

1. List every Jev caller: tool gates, the router, triage, and any backfill or eval job on the same account.
2. For each, estimate peak requests per minute and tokens per request (state plus all questions).
3. Compute each caller’s binding share: requests divided by 1,200, or tokens per minute divided by 15,000,000 (250,000 × 60), whichever is larger.
4. Sum the shares and plan to at most half of the published numbers, an illustrative margin for numbers that move.
5. Give batch work the leftovers, never the gates’ share.

Caller (illustrative)	Peak req/min	Tokens per request	Request share	Token share	Binding share
Destructive tool gate	120	3,000	10%	2.4%	10%
Model router	30	8,000	2.5%	1.6%	2.5%
Alert triage	60	20,000	5%	8%	8%
Ticket backfill (batch)	240	30,000	20%	48%	48%
Total					68.5%, over a 50% plan

The batch job breaks the budget, which is normal: it gets throttled or moved to a quiet hour, and the gates keep their share. Enforce the split in your client with a token bucket per caller, because the server’s 429 doesn’t know which caller mattered.

Gateways add their own ways to run dry. Vercel’s launch changelog says evaluation calls count toward gateway budgets, and OpenRouter’s native Decisions API documents a 402 when credits run out, plus 413 and 524. Map each to the same fail mode as a 429.

Step 3: Derive each gate’s timeout from your measured p99 and the action’s deadline

TypeSafe publishes no p99. Measure your own: a week of real gate calls through the route you actually use (direct, Vercel, OpenRouter or Cloudflare, from your lanes’ region), latency recorded for every call, 429s included. Take p99 per route; re-measure when you change door or region.

Then work backward from the action.

• Action deadline. How long the gated action can wait before the wait is itself a failure. An interactive shell call can stall a second or two before a person notices; a router picks a tier before a task that will run for an hour; a triage gate feeds a paging SLA.
• Gate timeout. At most the action deadline minus the time needed to act on the verdict, and well inside the harness’s own hook timeout, so the gate always delivers its fail mode itself. Claude Code’s default hook timeout is 600 seconds, and a hook that runs past it doesn’t block; the hook contract covers how the script exits 2 long before that.
• Retries that fit. A retry fits only if the timeout covers (retries + 1) × p99 plus backoff. If it doesn’t, the answer is zero retries, not a longer timeout.

Gate (illustrative)	Action deadline	Your p99 (example)	Gate timeout	Retries that fit
Destructive tool, interactive	3 s	450 ms	1.5 s	1
Destructive tool, unattended	10 s	450 ms	2 s	2
Router, before task start	30 s	600 ms	3 s	2
Alert triage	60 s	600 ms	5 s	2, then rules

If your measured p99 doesn’t fit inside the action deadline at all, the gate can’t be synchronous on that route. Move it closer, shrink the state, or accept that the fail mode is the normal path and redesign the seat.

One clock per gate. Retries live inside the timeout, the timeout lives inside the deadline, and anything that misses the timeout runs the fail mode.

Step 4: Cap SDK retry time inside the gate timeout

The SDK defaults are documented, and none of them was written for a one-second gate.

Client	Retries	Backoff	Retried on	Per-attempt timeout	Total budget
TypeSafe Python SDK	2	0.5 s doubling to 5 s, jitter 0.25; honours Retry-After	408, 429, 500–599, connection and timeout errors	10 s per HTTP operation	30 s per SDK call
TypeSafe JS SDK	2	500 ms doubling to 5,000 ms, jitter 0.25; server delays honoured up to 60,000 ms	408, 429, 500–599, connection and timeout errors	10,000 ms	None
AI SDK experimental_evaluate	2 (maxRetries)	Set by AI SDK core; no provider-side retry loop	Transient failures such as 429 and 529	Not stated in the reference	Your abortSignal

The Python retry policy at least has a ceiling: “Total retry budget in seconds per SDK call, including the initial attempt and delays.” The JavaScript client’s config reference is blunter about its timeout: “Timeout per attempt in milliseconds, without a total retry budget.” Its retry policy accepts a server-requested delay of up to 60,000 ms. Our arithmetic on those defaults: three 10-second attempts plus up to 60 seconds of server-requested wait before each of the two retries is about 150 seconds for one gate decision. Python stops at 30 seconds. Both are far past a 1.5-second gate, and under a 600-second hook timeout the wait shows up as an agent that quietly stopped working.

Screenshot: TypeSafe AI docs, “Interface: TypeSafeClientConfig - TypeSafe AI” (undated), captured Sep 21, 2026.

Set the budget wherever each client lets you, then enforce one deadline around the whole call anyway. On Python, a RetryPolicy can be passed as retry on the client or per call:

from typesafe_sdk import RetryPolicy, TypeSafeClient

GATE_TIMEOUT_S = 1.5                  # from step 3 (illustrative)

gate_client = TypeSafeClient(
    timeout=0.6,                      # per HTTP operation; default 10.0
    retry=RetryPolicy(
        max_retries=1,                # default 2
        backoff_max=0.2,              # default 5.0
        timeout=GATE_TIMEOUT_S,       # whole call, delays included; default 30.0
    ),
)

On the AI SDK path, which is how most Vercel users reach Jev, the experimental_evaluate reference documents both knobs you need:

import { experimental_evaluate as evaluate } from 'ai';

const GATE_TIMEOUT_MS = 1500;          // from step 3 (illustrative)

const result = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions,
  maxRetries: 1,                       // default 2
  abortSignal: AbortSignal.timeout(GATE_TIMEOUT_MS),
});
// an abort throws; the gate wrapper maps it to the fail mode

On the TypeSafe JS client, set timeout per attempt, retry.maxRetries, retry.backoffMaxMs and a low retry.maxRetryAfterMs, and still wrap the call in your own deadline, because the client has no total budget.

A Retry-After longer than the time left is itself an answer: run the fail mode now rather than sleep. The Python client already stops before a retry whose delay would reach the budget and re-raises the last error. And the docs don’t say whether a retried request is billed twice (no idempotency key exists), so count attempts, not decisions, when you cost a gate.

Step 5: Treat a late verdict as evidence, never as a decision

Put the order of checks in one wrapper so no two gates implement them differently.

# gate wrapper (illustrative); every path writes exactly one record
def gate(action, state, questions, policy, bucket, breaker, log):
    rec = log.open(action.id, policy.version)
    if breaker.is_open() or not bucket.take():          # shed load before calling
        return rec.fail_mode("shed")
    deadline = min(action.deadline, now() + policy.gate_timeout)
    try:
        answer = call_jev(state, questions, deadline)   # retries capped inside (step 4)
    except (RateLimited, Overloaded, Timeout, OutOfBudget, BadAnswer) as err:
        breaker.record(err)
        return rec.fail_mode(err.kind)
    if now() > deadline:
        rec.late(answer)                                # kept as evidence, never applied
        return rec.fail_mode("late")
    if not action.still_pending() or action.state_hash() != hash_of(state):
        return rec.expired(answer)                      # it judged a call that is gone
    return rec.decide(policy.apply(answer))

Late. The verdict arrived after the gate’s timeout. The fail mode runs; the verdict is logged with its latency and never applied, in either direction. A late “safe” applied after a deny would reopen a call the agent has moved past. A late “unsafe” on a read call that went ahead under a fail-open gate is a signal for review, not a rollback trigger.

Expired. The verdict is about an action that no longer exists in that form: the tool call was cancelled, or the file changed after the state was sent. Bind every request to the action ID, its deadline and a hash of the state you sent, and drop any verdict whose binding doesn’t match the pending call.

Queued. Under sustained 429s, don’t let gate calls pile up. A queue in front of a synchronous gate turns every verdict into a late one. Use admission control (the per-caller bucket from step 2) and a circuit breaker: after a burst of 429 or 529 responses, open the breaker for a fixed period, let every gate run its fail mode without calling Jev, then send one probe call to close it.

Every path writes one record. The Jev decision log already carries which path fired (ok, retried, 429, 529, timeout or fallback) with retry count and latency; add late, expired and shed if yours lacks them. An unattended destructive call that failed closed pauses its task and waits for a person, and that wait needs an owner and an expiry. Approval-queue hygiene covers both.

The wrapper is a guardrail, not the wall. If the gate process crashes or hangs, it can’t run its own fail mode, and on Claude Code a hook that exits 1 without valid JSON or times out lets the call continue through the normal permission flow. Deny rules and the sandbox behind the gate must hold on the day the wrapper doesn’t.

Step 6: Drill the four failure paths before the gate enforces

Point the gate wrapper at a local stub that returns whatever you tell it. Run the drills on a disposable lane, on every wrapper change and after every harness or SDK upgrade.

Drill	How to cause it	Expected	Signal in the record
Late verdict	Stub answers 300 ms after the gate timeout	Fail mode fires at the timeout; the verdict is logged, never applied	late, latency above timeout, outcome is the fail mode
Expired action	Stub delays; cancel the tool call or edit the file before it answers	Verdict dropped; nothing applied	expired, state hash mismatch
Queue under 429	Stub returns 429 with a 30 s Retry-After to every call; replay five times peak traffic	No gate waits past its timeout; breaker opens; batch callers shed first	shed and 429, breaker-open events, gate latency flat
Switch to rules	Hold the breaker open for 15 minutes	Triage runs the rules file and tags output; router takes the default route; destructive gate denies	fallback, rules-fallback tags, zero Jev calls
Recovery	Stub returns 200 again	One probe closes the breaker; tagged items re-triaged	breaker closed, re-triage count

Add one-line drills for the door-specific failures: a 529, a revoked key (a 401 should page someone as well as deny), and an exhausted gateway budget or OpenRouter 402. Then run the drill no stub can fake: kill the gate process mid-call and confirm the deny rules and sandbox stop the destructive call on their own.

How a Jev rate limit turns into a stall, and the signal for each

The stall that looks like work. A default JavaScript client honours a long Retry-After and the lane sits inside the gate. Signal: gate latency clustering at multiples of ten seconds, and sessions with a pending tool call and no progress. Fix: step 4’s budget plus your own deadline.

The silent open. The gate’s deadline creeps past the harness hook timeout, the hook dies first, and the call proceeds. Signal: tool calls in transcripts with no gate record in front of them. Fix: one deadline constant, checked against the hook timeout when settings load.

The noisy neighbour. A backfill takes the shared pool and the destructive gate starts failing closed. Signal: 429s on the gate while its own request rate is flat. Fix: per-caller buckets; batch work sheds first.

The limit moved. TypeSafe lowers the numbers without notice. Signal: a rising 429 rate at a call rate that was clean last week. Fix: the 50% plan and a weekly 429-rate check per caller.

The fallback route that wasn’t cleared. The router’s default sends a restricted repo to a lane whose last hop covers only public code. Signal: fallback records whose data class exceeds the route’s cap. Fix: one default route cleared for every class, or a deny for any class without one.

Rules forever. Triage stays on the rules file after Jev recovers because nothing closes the breaker. Signal: the rules-fallback share doesn’t fall after the 429s stop. Fix: the half-open probe, drilled.

A fail mode is fleet policy the classifier can’t write

Jev gives you a fast verdict when it gives you one. What the fleet does in the other case is a decision about your tools, data classes and deadlines, and it belongs with the rest of fleet policy: a versioned file, one wrapper every lane uses, a breaker with an owner, drills on a calendar. That is the unglamorous core of running agents as a fleet: the boring parts written down before the incident.

A classifier that answers in half a second is a good neighbour for that layer, not a substitute for it. Decide the fail mode first, then tune the threshold. The 429 arrives either way.

FAQ

What are Jev’s rate limits?

TypeSafe’s models page lists 250,000 tokens per second and 1,200 requests per minute, and a request over either returns 429. The page says both are adjusting dynamically and can change without notice. It doesn’t say whether they apply per key or per account, so budget them as one shared pool.

Sources

• TypeSafe: Models — 250,000 tokens/s and 1,200 requests/min; either returns 429; limits “adjusting dynamically”
• TypeSafe: API reference — 401, 422, 429 and 529; exponential backoff guidance
• TypeSafe Python SDK: Retries — RetryPolicy defaults, 30 s total budget per SDK call
• TypeSafe JS SDK: RetryPolicy — two retries, 500 to 5,000 ms backoff, server delays honoured up to 60,000 ms
• TypeSafe JS SDK: TypeSafeClientConfig — 10,000 ms per attempt, no total retry budget
• TypeSafe: Introducing System One Models & Jev — Sep 15, 2026; early access at launch; 70–500 ms measured from the West Coast
• AI SDK: experimental_evaluate — maxRetries (default 2) and abortSignal
• Vercel changelog: Jev on AI Gateway — Sep 16, 2026; evaluation calls count toward budgets
• OpenRouter: Decisions API reference — 402 on exhausted credits, 413, 524 and 529
• OpenRouter cookbook: Gate agent tool calls with Jev — a broken check “never turns into an approval or a review”