Models

A model is one line of config: the provider's model id. Keys come from your environment unless you pass them.

Anthropic

import { anthropic } from "@threads/anthropic";

const claude = anthropic("claude-sonnet-5");

Reads ANTHROPIC_API_KEY.

Prompt caching

Anthropic models cache the prompt for 5 minutes by default: the system prompt and tools, and the conversation as it grows. A cached token costs a fraction of a normal input token to read back. For a thread with longer pauses, cache for an hour:

const claude = anthropic("claude-sonnet-5", { promptCache: "1h" });

• Writing to the cache costs more than plain input: 1.25 x for 5 minutes, 2 x for an hour. If you give a price without cache_write, threads fills it in from input at that rate. Without cache_read it fills in 0.1 x input, the highest read rate Anthropic lists, so a cost can come out high but never low; pass cache_read for your model's exact rate (0.05 x on claude-opus-5-5).
• A short one-off prompt may pay the write and never read it back. promptCache: false (prompt_cache=False) turns caching off.
• "1h" can't be combined with hostedTools: Anthropic caches hosted tool results for 5 minutes, so writes would be billed at two rates. cache_control in params or a hosted tool is refused too; promptCache is the one way to set caching.
• The setting is part of the thread's pinned config. Threads started before prompt caching existed continue only with promptCache: false.
• citations: true turns on citations for every document in a request. It can't be combined with a native structured-output field in params (output_format, output_config.format); agent output is unaffected.

OpenAI

Uses the Responses API.

import { openai } from "@threads/openai";

const gpt = openai("gpt-5.5", {
  params: { reasoning: { effort: "medium" } },
});

Reads OPENAI_API_KEY. Both factories read the key at check() or the first run, never when you call them. OpenAI caches prompts on its own; set params: { prompt_cache_retention: "24h" } to keep them longer.

Limits

threads needs two numbers for every model: the most input tokens one request may carry (for compaction and cost budgets) and the most output tokens the model can produce. For the model ids below you pass neither: they come from a catalog each provider's own source was checked against.

* Derived, not published: OpenAI's GPT-5.5 page gives a 1,050,000-token context window and a 128,000 output limit but no input limit, so threads uses the window minus the output limit, 922,000. OpenAI's GPT-6 Sol page publishes 922,000 as the input limit for the same window and output limit.

The id must match exactly: claude-haiku-4-5 is not claude-haiku-4-5-20251001. A dated snapshot id keeps a long-lived thread on one model. For any other id, pass both limits; leaving one out is a ConfigError that names it:

const next = anthropic("claude-next", {
  maxInputTokens: 1_000_000,
  maxOutputTokens: 128_000,
});

Each request asks for at most maxTokens (max_tokens) output tokens: 8192 by default, or the model's output limit if that is lower. A longer answer continues in a new request. Raise it for long single answers, up to the model's output limit:

const writer = anthropic("claude-sonnet-5", { maxTokens: 32_000 });

maxTokens is the only way to set the cap: putting max_tokens (or OpenAI's max_output_tokens) in params is refused. Output-token budgets read this cap, for every provider.

The limits are recorded when a thread starts. If a catalog entry is ever found wrong, it is withdrawn rather than edited: the factory then asks you for the limits, and its error prints both the corrected values (for new threads) and the old ones (to keep continuing a thread started with them).

Other providers

import { aiSdk } from "@threads/ai-sdk";

aiSdk({
  model: (fetch) => createProvider({ fetch })("model-id"),
  maxInputTokens: 128_000,
  maxOutputTokens: 8192,
});

Shared options

Model settings are fixed for the life of a thread, which keeps the prompt prefix identical on every request so provider caches keep hitting.

Cache lifetimes

Cache breaks are judged by how long the provider keeps the cache. Each model declares it: anthropic() from promptCache, openai() from its retention, the scripted test model as none. An agent uses the lifetime its models agree on, so anthropic("claude-sonnet-5", { promptCache: "1h" }) needs nothing else.

check() (and the first run) asks you to set context.cache_ttl_ms when threads can't decide:

• the model and a fallback cache for different lifetimes;
• a model's lifetime is unknown: an aiSdk() or litellm() model that doesn't pass cacheTtlMs / cache_ttl_ms, or a Model you wrote yourself whose info has no cache (declare cache: { ttl_ms: 300_000 }, or "none" if it never caches).

const assistant = agent({
  model: anthropic("claude-sonnet-5", { promptCache: "1h" }),
  fallback: [gpt],
  context: { cache_ttl_ms: 3_600_000 },
});

Retries

threads owns every retry; provider SDK retries are turned off. Rate limits, overloads and server errors are retried with exponential backoff, honoring retry-after. The defaults:

When retries run out, the run ends failed with code model_unavailable. In TypeScript, pass any subset: retry: { max_retries: 4 }. Python's retry= takes a complete Retry object from threads.log.

Fallback models

After fallback_after overloaded errors in a row, the agent switches to the next model in fallback and records the switch in the log. From then on requests go to that model, and its own context window sets the compaction thresholds.

const assistant = agent({
  model: claude,
  fallback: [gpt],
  retry: { max_retries: 4, fallback_after: 2 },
});

A fallback lasts for the rest of the turn. With the default fallback_scope: "turn", the thread's next input goes back to the settings it had before the fallback, recorded as a settings_changed with reason revert before that input's first request. With fallback_scope: "thread" the fallback stays.

The before_model_switch hook gates every switch, fallbacks and reverts alike. A deny keeps the current model: a denied fallback retries on the same model, and a denied revert keeps the fallback for that input's turn. The hook is asked once per input for a revert, even across a crash.

Every fallback model counts toward a budget: a model with no per-attempt bound for one of the budget's limits is refused at setup (budget_unenforceable).

Switch models mid-thread

An operator can switch a thread to another model it already knows (its model or one of its fallback models) with the thread's setModel (set_model). The change is recorded in the log and applies from the next request. See Thread.