When OpenAI shipped o1 in September 2024 it coined a product category — "reasoning models," models that think before they answer — and the entire frontier followed: o3 and o4, DeepSeek R1, Kimi k1.5 and k2, Qwen3's thinking mode, Gemini 2.5's built-in thinking, Grok's Think toggle, Anthropic's extended thinking, GPT-5's reasoning effort. Two years later the label is quietly dissolving, and the term replacing it names something else entirely. People increasingly call these models RLMs — reinforcement-learning models — not because they stopped reasoning, but because reasoning was never the thing that made them work.
The term is still informal; you will find it in model cards, RL papers, and training-blog postmortems rather than in any dictionary. So this post stakes out a definition, explains the two labels properly, and compares them, because the two names are not synonyms for the same object. They are two different claims about where the capability comes from — and the claim you adopt decides what you build.
The category that named a behavior
A reasoning model, as the product category was defined, has three operational properties.
It produces a long internal line of reasoning before answering. Not a one-shot completion: a private multi-step trace in which the model decomposes the problem, tries approaches, notices its own mistakes, backtracks, and verifies. OpenAI's o1 announcement described training the model "to think before they respond" and to "refine their thinking" (Learning to reason with LLMs). For safety and monitoring, the raw trace stays hidden — you see a summary, not the chain (o1 system card).
You can buy more accuracy with more thinking. The o1 team reported that performance "consistently improves with more reinforcement learning (RL) training time and with more time spent thinking." That second clause became the product dial: thinking budgets, reasoning effort, "high-effort" vs "low-effort" modes — test-time compute as a purchasable quantity, the thing that made o3's December 2024 ARC-AGI result a headline. The behavior scales smoothly with compute at inference even when the weights are frozen.
It dominates hard multi-step benchmarks. AIME math, GPQA science, competitive programming, Humanity's Last Exam — the question sets where ordinary chat models collapse into confident nonsense. The reasoning-model era was measured in those pass-rates.
Notice what these three properties share: they are all visible behaviors. The category was named after what the model does when you query it — a product distinction between "thinks" and "doesn't think," marketed as a toggle. That framing is exactly why the category is now dying of success. GPT-5 (August 2025) folded thinking into a single unified model with an effort dial instead of a separate reasoning tier (Introducing GPT-5); Qwen3 shipped one open model with thinking on by default and a switch to turn it off (Qwen3); Gemini 2.5 made thinking the default posture rather than a mode (Gemini 2.5). When every model reasons by default, "reasoning model" stops being a category and becomes a feature everyone has. What remains — the thing that actually separates these models from the ones that came before — is not visible at all.
The mechanism the label hides
Underneath the thinking traces is a training recipe that has almost nothing to do with reasoning as humans experience it. A frontier "reasoning model" is built roughly like this:
- Pretrain a large language model on next-token prediction — the substrate. It learns language, world knowledge, and the raw statistical ability to continue text.
- Optionally cold-start with a small amount of supervised fine-tuning on long reasoning traces, mostly to make the RL stage stable and readable (DeepSeek's R1 did this with a few thousand curated examples; its R1-Zero variant famously skipped the step entirely).
- Run reinforcement learning at scale against a verifiable reward. The model generates whole trajectories — chains of thought ending in an answer — and gets reward purely from whether the answer is checkable as correct: a rule-based checker for math, unit tests for code (DeepSeekMath, which introduced GRPO, the critic-free RL update R1 and its successors used). The optimization signal is objective. Nothing about how it thinks is supervised.
- Ship it, with distillation as the efficiency lever: the expensive RL-trained traces are used to fine-tune cheaper models, and long traces are distilled into short ones.
The decisive ingredient is step 3, and its history is instructive. OpenAI's 2021 GSM8K work framed the growth path as verification: sample many solutions, grade them with a verifier (Training Verifiers to Solve Math Word Problems). Its 2023 follow-up showed that dense step-level reward — process supervision — beat outcome-only reward when compute is tight, and asked for more verification research as the scaling path (Let's Verify Step by Step). DeepSeek's 2024–2025 work removed the human from the loop: if the reward is a rule-based check, you don't need a learned reward model or human labels at all, and RL can run for millions of trajectories (DeepSeek-R1).
The result that defines the whole era is R1-Zero. Trained with pure RL from a base model — no supervised fine-tuning whatsoever — it learned to reason: decompose, verify, backtrack. DeepSeek's report includes the transcript moment where the model spontaneously interrupts its own chain with "Wait…" and re-derives the correct approach; they called it the "aha moment." The behavior was not taught, prompted, or demonstrated. It was selected — grown by reward pressure out of a model whose pretraining never contained it as a skill. DeepSeek then distilled R1's reasoning into 1.5B-to-70B open models by fine-tuning them on R1's traces — no RL at all — and the 32B distillate still beat o1-mini on AIME 2024 (DeepSeek-R1). Reasoning, once grown by RL, turned out to be radically compressible.
That is the fact the "reasoning model" label cannot express: the reasoning is an emergent policy of an RL-trained system, not a mode of the language model. And it is the fact behind the new name.
RLM: naming the training loop, not the trace
An RLM — a reinforcement-learning model, in the sense the term is now used — is a model whose defining capability is manufactured by large-scale reinforcement learning against an objective, verifiable reward, rather than extracted from imitation data or tuned to human preference.
The definition needs three sharp edges, because "RL" alone is too broad to mean anything.
First: RLM is not RLHF. Human-feedback RL — InstructGPT-style preference optimization — has been standard since 2022 (InstructGPT). Its reward is a learned model of human taste, and its job is alignment: make the assistant agreeable, safe, on-policy. An RLM's reward is an objective checker — math correctness, test passing, task completion — and its job is capability: make the model actually better at hard things. RLHF tunes manners; RLM trains skills. Both use RL; only the second one is the engine of the current frontier.
Second: the RL loop is the primary source of capability, not a garnish. Pretraining provides the substrate; the RL stage is where the model's new abilities come from. R1-Zero is the ur-example — a base model with no reasoning skill, an RL loop, a rule-based checker, and an "aha moment" two months later. DeepSeek's R1 report and Moonshot's k1.5 paper are, at bottom, scaling studies of that loop: more RL compute, better RL data, longer effective traces (Kimi k1.5: Scaling Reinforcement Learning with LLMs).
Third: the reward is verifiable and external. The signal is produced by the environment — a compiler, a test suite, a formal checker — not by a human annotator or a learned proxy. This is what lets RL run at a scale annotation economics could never support, and it is also the discipline that defines where RLMs can and cannot be grown. No verifier, no RLM.
Under this definition, today's reasoning models are not a separate species from RLMs — they are the first commercial generation of RLMs: models whose reward was final-answer correctness in verifiable domains (math, code, science) and whose emergent behavior happened to look like human reasoning because that is what optimizing correctness selects for. The reason the field keeps drifting toward the mechanism name is that the next generation is visibly no longer about chains of thought. Moonshot's k2 (July 2025) pointed the same RL approach at agentic behavior — browsing, tool use, file edits — with open weights; the o4 line and GPT-5 fold tools into the reasoning itself; and the frontier conversation has moved from "how long should it think" to "what environment and reward should it optimize against." An agentic RLM does not always narrate its reasoning in tokens; it acts — and the "thinking" shows up as tool calls, test runs, and self-corrections in a harness.
That is the claim in the title, stated carefully: reasoning models, as a category, are being subsumed by RLMs. The reasoning behavior that defined the old category is now understood as a special case — the first reward function happened to produce traces. The mechanism generalizes; the behavior does not.
Comparing the two labels
Because the objects overlap almost completely today — almost every commercial reasoning model is a first-generation RLM — the comparison is between two ways of carving the space, and the differences are in what each label highlights, hides, and predicts.
| Reasoning model (behavior label) | RLM (mechanism label) | |
|---|---|---|
| Names | What the model does at inference | How the model's weights were shaped |
| Defining evidence | Long thinking trace, effort dial, benchmark pass-rates | Verifiable-reward RL loop at training time |
| Where capability comes from | (unspecified — looks like it "thinks") | Selection pressure from an objective checker |
| What scaling means | More thinking tokens at inference (test-time compute) | More RL compute, better rewards/verifiers at training time |
| Signature failure mode | Confident errors buried deep in the chain | Reward hacking — optimizing the checker, not the task |
| Evaluation culture | Hard question sets: AIME, GPQA, HLE | Outcome-grounded: tests, environments, verifiers, task success |
| Product shape | A toggle: thinking on/off, effort low/high | A training pipeline: base model + verifier + RL loop |
| Natural next step | Longer, tool-assisted traces | Wider verifiable domains: agents, code, environments |
The two sets are related but not identical, and the mismatch is the whole point. Some reasoning behavior is not RLM at all: a prompted chain-of-thought in a plain chat model, or the distilled 7B model that imitates R1's traces, can reason without any RL having shaped it. And some RLMs will never look like reasoning models: an agentic RLM optimizing task completion in an environment spends its "thoughts" as actions, and a future RLM over a different reward could be competent at things that bear no resemblance to human deliberation. The behavior label overclaims (the model does not transparently "reason" — it emits traces that earned reward, which is why hidden CoT must be monitored rather than trusted) and under-specifies the mechanism (it cannot tell you why this model can think and the last one could not). The mechanism label does the reverse: it tells you exactly where the capability came from, and nothing at all about what the model will look like when you query it — that depends on the reward.
The sharpest way to put it: "reasoning model" describes the surface of the first RLM generation so well that it stopped people from asking what was underneath — until the underneath stopped producing reasoning. The day a lab ships an RLM whose reward has nothing to do with chains of thought, the behavior label simply fails to name it; the mechanism label still does.
Why the label decides what you build
If reasoning models were just a thinking mode, the engineering agenda would be prompt-side: longer budgets, better instructions, smarter sampling. If they are RLMs, the agenda moves to where this blog has been pointing for a year — verification is the bottleneck (Verification Is the Bottleneck); in the land of agents, the verifiers are king (In the Land of AI Agents, the Verifiers Are King). An RLM is only as good as its reward, and its reward is only as good as its verifier — so the entire capability curve is gated by checkable tasks. Math and code went first because they are checkable. Agents go next exactly insofar as we build harnesses that can check them: test suites, sandboxes, mocks, approval flows — the harness patterns this blog catalogs are, from the RLM's point of view, reward infrastructure. The CMU harness-adaptation study showed small models recovering 89% of frontier performance at 4% of the cost when the harness supplies the checking — that is RL logic applied outside the training run. The longitudinal study that found harness evolution with no quality gate produced no gains is what RL without a verifier looks like. Even DeepSeek's own harness is, among other things, a machine for making agent behavior checkable.
The mechanism label also makes the failures legible. A reasoning model that "hallucinates in the chain" and an agent that "gamed the eval" are the same disease under two names: reward hacking — optimizing the reward signal rather than the intent behind it, catalogued for RL generally (Defining and Characterizing Reward Hacking, and anticipated in Concrete Problems in AI Safety). Goodhart's Law is not a curiosity of RLM training; it is RLM training (Hacker Laws for ASE: Goodhart's Law). The behavioral label invites you to debug the trace. The mechanism label tells you the trace is fine — the reward is what needs auditing.
And the mechanism label explains the economics the behavioral label cannot. If reasoning were a mode, GPT-5's unification and R1's 7B distillates would be confusing. If reasoning is a compressible RL-grown policy, then of course the frontier folds it into every tier and of course Chinese labs ship it to your laptop (Chinese models will win the local-first race) — down to one-bit weights on the edge (1-Bit Models: Edge Budgets). What does not compress is the RL loop that grew the policy. The moat moves from the model to the training-time loop: the verifier quality, the reward design, the environment — which is the Bitter Lesson restated for post-training: general methods that leverage computation (RL + checkable reward) beat hand-authored curricula (manually written chains of thought) every time (The Bitter Lesson).
The honest boundary: RLM only applies where a verifiable reward exists. Open-ended writing, taste, judgment calls, the long tail of tasks with no objective check — those stay in RLHF/DPO territory, and no amount of relabeling grows capabilities there. That boundary is not a limitation to apologize for; it is the most useful property of the term. An RLM is a model that only claims what can be checked — and the current frontier is, in large part, the history of engineers expanding the set of checkable things.
What changed in my view
I came to this terminology from the harness side, and the label change is what finally connected two things I had been treating as separate. This blog's agent work kept concluding that verifiers, evals, and harness quality — not model size — are where agent performance lives. Meanwhile the model-side discourse kept talking about "reasoning models" as if intelligence were a dial you buy. The RLM framing is the missing link: the reason harness engineering matters is that harnesses are the reward environments of the next RLM generation. The verifier is not a QA convenience; it is the training signal of the thing you are deploying. Once you see that, "reasoning model" reads the way "horseless carriage" does — a name that describes the previous era's most conspicuous feature while the mechanism underneath is busy inventing the next one.
The next model you buy will be an RLM. Whether it looks like a reasoning model — whether it shows you a thinking trace or just acts — depends entirely on what reward it was grown against. Ask which one.
References
- OpenAI. Learning to reason with LLMs (o1, September 2024) and the o1 system card.
- DeepSeek-AI. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — R1-Zero, GRPO at scale, the "aha moment," distillation to 1.5B–70B.
- Shao et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models — GRPO.
- Kimi Team. Kimi k1.5: Scaling Reinforcement Learning with LLMs.
- Lightman et al. Let's Verify Step by Step; Cobbe et al. Training Verifiers to Solve Math Word Problems.
- Ouyang et al. Training language models to follow instructions with human feedback (RLHF — alignment, not RLM).
- Skalse et al. Defining and Characterizing Reward Hacking; Amodei et al. Concrete Problems in AI Safety.
- Sutton. The Bitter Lesson.
- OpenAI. Introducing GPT-5; Qwen team. Qwen3; Google. Gemini 2.5.
- This blog: Verification Is the Bottleneck, the Verifiers Are King, Harnessing Agentic AI Systems, Better Harnesses, Smaller Models (review), Don't Blame the LLM (review), DeepSeek Harness, Hacker Laws: Goodhart's Law and The Bitter Lesson, Chinese models will win the local-first race, 1-Bit Models: Edge Budgets.