--- title: "AI primer: why calibrated decision models" type: concept tags: [rlcd, rlhf, rlvr, calibration, training] created: 2026-09-17 updated: 2026-09-17 confidence: high sources: - raw/docs/introduction__machine-learning-primer.md - raw/docs/models.md jev_version: "jev-1.13.0" summary: "Why TypeSafe post-trains with RLCD for calibrated decisions instead of RLHF/RLVR for generated text." --- # AI primer: why calibrated decision models > **TL;DR** RLHF optimizes for human preference, RLVR for verifiable rewards; TypeSafe adds a third path, **RLCD** (Reinforcement Learning for Calibrated Decisions), which returns decisions and probabilities instead of text. Calibration means that across many predictions, outcomes given probability `0.2` occur about 20% of the time — a property of groups, never a guarantee about a single answer. ## What it is Most AI products are built around a conversation between a model and a person. TypeSafe starts from a different bet: **large-scale automation will be dominated by AI-to-AI and AI-to-software interactions**, so the machine interface matters more than the chat interface. > **Machine Native Intelligence** (TypeSafe's term): AI with software-like properties such as structure, reliability, observability, testability, speed, consistency, and low cost. ### Building prod, not God TypeSafe is not trying to build a model that does everything. It is designed for production systems where code needs a narrow decision it can inspect and act on. TypeSafe's stated expectation: large-scale AI automation will be **closer to 99% machine-to-machine interactions and 1% human interaction**. That shifts the design target from responses that feel good to read toward outputs that behave predictably inside software. (See [[entities/manifesto]].) ## How it works (mechanism) ### Three post-training approaches Pretrained language models have been adapted in two major ways; TypeSafe adds a third. RLHF and RLVR are given for context — TypeSafe's training path is RLCD. | Approach | Full name | What it produced / optimizes for | |---|---|---| | **RLHF** | Reinforcement learning from human feedback | Turned pretrained models into chatbots. Trains models to produce responses people prefer. | | **RLVR** | Reinforcement learning with verifiable rewards | Created reasoning models that are strong at tasks such as mathematics, but slower and more expensive. | | **RLCD** | Reinforcement learning for calibrated decisions | Trains TypeSafe to return decisions and calibrated probabilities instead of generated text. | RLHF was used to train InstructGPT and ChatGPT and was co-invented by **Diogo Almeida**, cofounder of TypeSafe. (See [[entities/team]].) ### RLCD and calibrated decisions RLCD optimizes for a different output contract: - The model does not generate text. - It returns decisions and probabilities. - Higher probability should correspond to a greater chance that the answer is correct. Calibration makes uncertainty usable by software. Across many predictions from a well-calibrated model: - Outcomes assigned a probability of `0.2` should occur about 20% of the time. - Outcomes assigned a probability of `0.8` should occur about 80% of the time. - Outcomes assigned a probability of `1.0` should occur 100% of the time. **These rates describe groups of predictions, not a guarantee about any single answer.** This is the single most important caveat on the page for anyone writing code against Jev: you cannot audit calibration one request at a time. See [[concepts/confidence]] for deciding when software should act or escalate. ### The problems with RLHF RLHF teaches a model to say things that people prefer. That objective works well for chatbots, but it can also reward **sycophancy** and **confident-sounding hallucinations**. Preference optimization also causes **mode dropping**: the model learns to favor a particular style, such as instruction following, while reducing the probability of other possible outputs. Mode dropping is a milder version of **mode collapse** — the classic generative-adversarial-network failure mode where a generator learns to produce the same kind of output repeatedly because that output continues to fool the discriminator. > **Warning from the docs:** an output can be compelling to a person without being reliable enough for unattended automation. Human preference and machine trustworthiness are different optimization targets. RLHF remains a good fit for conversational models. TypeSafe's position is that production automation needs a different training objective — one centered on constrained decisions and calibrated uncertainty. ## Why it matters for code - **Probabilities are features, not decoration.** Because they are trained to be calibrated, you can threshold, weight, and sum them; you can also feed them into a downstream classical ML model. See [[patterns/composite-scoring]] and [[cookbooks/autoresearch-feature-discovery]]. - **Uncertainty is a control signal.** A model that can say "I don't know" lets code split into act / confirm / escalate paths. See [[patterns/confidence-routing]]. - **Validate calibration on your own data in aggregate.** Bucket answers by reported probability and measure the hit rate per bucket; a single wrong high-probability answer is not evidence of miscalibration. (inferred: the page states the group property but does not prescribe a validation procedure.) - **RLCD is the reason Jev is not customizable per account.** Jev is not fine-tuned or LoRA-adapted with customer data; the same weights serve every account, and you shape answers through `state`, `instructions`, and `criteria`. See [[reference/models-and-pricing]]. ## Gotchas - "Calibrated" is not "accurate". A model can be perfectly calibrated and still be wrong on any given item; calibration says the *reported* confidence matches the *observed* frequency. - Probability `1.0` in the docs' framing means 100% of the time across the group — do not read a single `1.0` as a proof. - Nothing on this page claims Jev is smarter than an RLVR reasoning model; the claim is that it is trained for a different output contract. Intelligence claims live in [[concepts/workflow-evals]] and [[concepts/system-one]]. ## Related - [[concepts/system-one]] — the model class this training produces - [[concepts/confidence]] — how calibration surfaces as a field you branch on - [[concepts/how-to-build]] — designing workflows around calibrated answers - [[entities/manifesto]] — "Composable AI — Build Prod, Not God" - [[entities/team]] — Diogo Almeida and the founders - [[reference/models-and-pricing]] — why Jev is not fine-tuned per customer ## Sources - raw/docs/introduction__machine-learning-primer.md (https://docs.typesafe.ai/introduction/machine-learning-primer) - raw/docs/models.md (https://docs.typesafe.ai/models) — for "Jev is not fine-tuned or LoRA-adapted with customer data"