What a Claude model actually is
Stateless next-token prediction, and why every architectural decision flows from that.
Claude is a large language model: given a sequence of tokens, it produces a probability distribution over the next token, samples one, appends it, and repeats. Everything else — chat, tools, agents, memory — is scaffolding built on top of that single loop. An architect who forgets this ships systems that quietly depend on magic.
Two consequences dominate design. First, the model is stateless. It has no memory of your previous API call. A conversation only exists because your application resends the whole message history on every turn. Second, everything the model 'knows' about your request must be inside the request — the system prompt, the messages, the tool definitions. There is no side channel.
Your job is rarely 'make the model smarter'. It is 'get exactly the right tokens into the window, in the right order, at the right cost, and validate what comes out.'
Tokens are sub-word chunks. Roughly 1 token ≈ 3.5–4 English characters, so ~750 words ≈ 1,000 tokens. Code, JSON, and non-Latin scripts tokenize less efficiently — a JSON blob can cost 2–3× the tokens of the same information in prose. Pricing, latency, and context limits are all measured in tokens, so token accounting is a first-class architectural skill.
- The API is stateless — conversation state lives in your application, not in Claude.
- Every input token is paid for on every turn; history grows quadratically in cost if unmanaged.
- Output tokens are billed at a higher rate than input tokens on every Claude model.