Durable LLM inference
What Perdurance does to an LLM request: stores it, runs the call server-side, persists every chunk, and hands the result back by id whenever you return.
Perdurance sits between your code and your model provider. You send it the request body you would have sent the provider, on the route you would have sent it to, and it answers with the provider's own response — unchanged, no envelope, nothing your SDK has to be taught.
What differs is what happens underneath. Perdurance writes the request down, runs the upstream call server-side, stores every chunk of the answer as it arrives, and records what the call cost in tokens. All of that survives your connection. It survives your process.
What that buys
A dropped connection loses nothing
The upstream call keeps running when your client disappears. Send the identical body again and you attach to the execution already in flight instead of starting a second one.
Retries do not double-bill
Inside the idempotency window an identical body hashes to the same record. Your provider is called once, however many times your client asks.
Every request is a record
The body, the answer, each chunk, the usage counts and the execution history are stored and readable by id, for as long as you keep them.
Two ways to call it
Which one you want depends on whether you have something to hold a connection with.
Synchronously, on your provider's own route. POST /chat/completions in OpenAI's dialect,
POST /messages in Anthropic's. The connection is held and you get the provider's answer on
it — a completed response, or a stream of the provider's own events when the body says
"stream": true. An unmodified vendor SDK works against this, and its own retry is what makes
it durable.
Fire-and-forget, on POST /requests. You get 202 and a request id straight back, and the
execution runs without you. This is the shape a batch job wants, or anything that cannot hold a
socket open for a long generation.
Either way the record is the same, and either way you can come back for it later with
GET /requests/{id} — whole, or as a stream replayed from the beginning and resumable from
wherever you stopped reading.
The problems it is for
Three failures cost you a generation you already paid for, and none of them are the model's fault. Each has a page.
Unstable AI inference
The six ways an inference call fails without the model failing — dropped sockets, serverless limits, evicted workers — and what each one costs.
LLM inference retries
Why retrying an LLM call bills you twice by default, and what makes a retry safe. Your SDK is already retrying whether or not you wrote the loop.
Agent SDK retries
Why an agent loop multiplies the problem across three layers of retry, and how to wire the frameworks up so it stops.
Start here
Getting started takes you from an empty account to a stored request you can fetch back. If you are operating a deployment rather than calling one, Deployment is the shorter road, and the FAQ answers what people ask before either.