Which response would a person prefer?
Human approval and usefulness
Instruction-following, tone, interaction quality
A preferred answer is not necessarily a true one
Overview
When ChatGPT launched in November 2022, the breakthrough was not intelligence alone but making that intelligence approachable — a model people could interact with naturally. The same human-centered design, however, introduces a tension: an answer a person prefers is not necessarily a decision a business should execute.
This newsletter article follows that tension from conversational assistance to agentic execution. Its central claim is that the defining artifact of this AI era is not the model but the arrangement around it — the loop where a machine produces and a human closes. The next frontier is a system that knows when to act, when to ask, and can prove the distinction is working.
TypeSafe's Jev makes the question concrete: a model designed to return structured decisions and calibrated probabilities that software can consume directly, rather than another conversational assistant.
The core question
What has to change before a specific decision no longer needs a person to approve it every time?
- More capability? Better context? A different training objective?
- A reliable estimate of uncertainty? A deterministic verifier?
- Or a better-designed workflow around all of them?
The first breakthrough made intelligence conversational. The next may make it reliably actionable.
The Three Objectives: Comparison Matrix
Preference, correctness, and calibration solve three different problems — and a strong AI system may need all three.
| Objective | Preference | Correctness | Calibration |
|---|---|---|---|
| Question it answers | Which response would a person prefer? | Did the system get the answer right? | Does stated confidence match how often it is actually right? |
| What it optimizes | Human approval and usefulness | Objective verifiability | Trustworthy uncertainty |
| Best suited for | Instruction-following, tone, interaction quality | Mathematics, code, tasks with clear verifiers | Decisions where uncertainty must gate action |
| Primary limitation | A preferred answer is not necessarily a true one | Overconfidence can be rewarded when a guess and an honest "I don't know" score the same | Local to the measured environment; works across groups, not single cases |
Did the system get the answer right?
Objective verifiability
Mathematics, code, tasks with clear verifiers
Overconfidence can be rewarded when a guess and an honest "I don't know" score the same
Does stated confidence match how often it is actually right?
Trustworthy uncertainty
Decisions where uncertainty must gate action
Local to the measured environment; works across groups, not single cases
The Argument, Section by Section
1
Where the Loop Came From
RLHF and InstructGPT closed the gap between predicting text and following instructions, but optimizing for approval and optimizing for truth are not the same objective. A human preference tells us which answer a person likes, not which is factually correct.
2
The Human as Error Correction, and as a Cost
When every case still requires human review, every case still carries a human cost. This is the dividing line between assistance, where the human catches mistakes, and automation, where selected decisions proceed without case-by-case check.
3
Three Objectives, Not Three Mutually Exclusive Machines
Preference, correctness, and calibration answer three different questions, and a strong AI system may need all three. Accuracy alone does not solve automation because a model can be accurate on average yet dangerous when it does not know when it is likely to be wrong.
4
What Calibration Actually Gives You
Calibration makes a probability mean something, but it is not accuracy and it works across groups of predictions, not individual cases. The goal is useful predictions with justified confidence, so uncertainty becomes something the workflow can use.
5
What Jev Is Trying to Build
TypeSafe introduced Jev in early access on September 15, 2026, founded by Diogo Almeida, a co-author of the InstructGPT paper. Jev returns structured outputs via three primitives — Choice, Noul, and Score — and is best understood as a decision component inside a larger agentic system.
6
The Product Claim and the Evidence Are Different Things
Evidence splits into three buckets: what TypeSafe claims, what outside testing observes, and architectural speculation. Early calibration results are encouraging but do not yet prove the category on enterprise workflows.
7
The Limitations That Matter in Production
Option order, wording, and rounding can move probabilities enough to change whether a case is automated or reviewed. Calibration is local to the environment where it was measured, and separate decisions can still fail together.
8
From a Probability to a Decision Rule
A probability says how likely something is; a decision rule says what to do with it. The most likely answer is not always the economically correct action, and there is no universal confidence threshold for automation.
9
Why Calibration Does Not Remove the Need for a System
A trustworthy probability is not enough to run a business process, and reliability compounds across steps — ten decisions each 99% reliable yield only about 90.4% end-to-end. A reliable component is valuable; a reliable workflow still has to be engineered.
10
Closing the Loop Through Measurement
The strongest architecture has two loops — an operational loop for each decision and a measurement loop over time. The human moves from reviewing every decision to maintaining the evidence that allows some decisions to run without them.
11
What Reprices When Review Becomes Selective
Product, governance, and work all reprice once review becomes selective: value shifts from assistance toward completed decisions, oversight toward evidence, and work toward exceptions and system design.
12
The Scaling Relay
This fits the broader AI Supercycle as a rotating bottleneck: once raw capability becomes good enough, delegation, reliability, and economics become the constraint that matters next.
13
Is This the End of RLHF?
RLHF is not over; the narrower claim is that preference optimization alone is not enough for every automation problem. The real experiment is comparative — whether decision-native systems automate more work, more reliably, at lower total cost.
14
The Human Loop That Stays
Some human checkpoints exist because the system is not reliable enough yet (forced loops); others exist because the decision requires judgment, accountability, values, or authority (chosen loops). The goal is not zero humans — it is zero unnecessary review.
15
What a Serious Deployment Would Test
Start with one bounded decision with an observable outcome, then test prediction, policy, workflow, unattended execution, and economics. A deployment earns automation when a defined slice of work runs with less human effort, acceptable outcomes, and risk inside an agreed boundary.
16
The Mental Models
The article rests on a small set of portable ideas — the loop as objective, synchronous versus statistical oversight, the three objectives, the cost of the loop, confidence versus calibration, and oversight as measurement.
17
The Next Question
The important shift may be a system that makes a narrower promise but proves it more reliably — one that knows when to act, when to ask, and can prove the distinction is working.
How to Test a Decision Before Automating It
A staged checklist for moving from human review to selective automation on a single bounded decision.
1
Choose one bounded, observable decision
Pick a decision whose outcome can be observed, whose examples occur often enough to measure, whose actions are clear, and which has a practical fallback when the system is unsure.
2
Test the prediction
Check whether the model makes the right call, whether its probabilities are reliable, whether wording or option order moves the result, and what happens when context is missing — especially near the automation threshold.
3
Test the policy
Decide which mistakes matter most, which cases may be automated, which still require approval, and what happens when confidence is too low.
4
Test the workflow
Verify that the right action is actually executed, permissions are enforced, duplicate requests are handled safely, and the process can recover when something downstream fails.
5
Test unattended execution
Run the system in shadow mode on held-out cases before letting it act freely, then continue auditing a sample of automated decisions after launch.
6
Test the economics
Include inference, remaining human review, monitoring, maintenance, remediation, and the cost of errors — then compare the total with the existing process and simpler alternatives.
7
Measure calibration and error rates
Sample decisions, collect outcomes, and measure calibration against real outcomes on the task where the system will actually operate.
8
Monitor for drift
Watch for changes in customer behavior, policy, products, and fraud patterns that make a once-reliable probability mean something different today.
9
Adjust thresholds, policies, or models
Change thresholds, policies, or models when performance deteriorates, and let automation earn autonomy through evidence — keeping it only while the evidence holds.
Frequently Asked Questions
1What is Jev?▼
Jev is TypeSafe AI's decision model, introduced in early access on September 15, 2026. Instead of free-form text, it returns structured decisions and calibrated probabilities that software can consume directly, using three primitives — Choice, Noul, and Score.
2Who built Jev and why is the founder notable?▼
Jev was built by TypeSafe AI, founded by Diogo Almeida, a co-author of the InstructGPT paper. His background in RLHF makes Jev's alternative training objective — calibration rather than preference — a deliberate product bet.
3What are Jev's three core primitives?▼
Choice selects among defined alternatives; Noul returns the probability of a yes-or-no proposition; and Score evaluates something against an ordered scale. Several questions can be sent in one call against the same shared state.
4What is RLCD?▼
Reinforcement Learning for Calibrated Decisions is TypeSafe's stated training approach for Jev. It is designed around one idea: produce probabilities that software can use directly, making the numbers attached to decisions reflect how often those decisions are right.
5Is this the end of RLHF?▼
No. The stronger claim is narrower: preference optimization is not enough for every problem automation needs to solve. RLHF remains useful for assistants, but calibration can be reached through better base models, external verifiers, or dedicated decision models.
6What is the difference between preference, correctness, and calibration?▼
Preference asks which answer a person prefers; correctness asks whether the answer satisfied an objective verifier; calibration asks whether stated confidence matches how often the model is actually right. They are complementary, and a strong system may need all three.
7What is calibration and why does it matter?▼
Calibration means that when a model says it is 90% confident, it is right about 90% of the time. It matters because accuracy alone does not solve automation — a model that does not know when it is likely to be wrong is dangerous to leave unattended.
8Is a probability the same as a decision?▼
No. A probability tells you how likely something is; a decision rule tells you what to do with it. The most likely answer is not always the economically correct action — thresholds depend on the cost of error, the cost of review, and reversibility.
9Does calibration remove the need for a harness?▼
No. A calibrated decision cannot create a missing business rule, grant permission, prevent duplicate execution, or verify that the intended outcome happened. The model supplies the judgment; the harness turns that judgment into governed execution.
10What are forced loops and chosen loops?▼
Forced loops keep humans in place because the system cannot yet be trusted to act reliably. Chosen loops keep humans in place because the decision requires judgment, accountability, values, or authority. Forced loops should shrink as evidence improves; chosen loops may remain permanently.
11What should a serious deployment test first?▼
Start with one bounded decision with an observable outcome, frequent examples, clear actions, and a practical fallback. Then test prediction, policy, workflow, unattended execution, and total economics before expanding scope.
12What are the two loops in a robust automation architecture?▼
The operational loop handles each decision — prediction, policy, action, result. The measurement loop watches over time — outcome, audit, calibration, drift detection, updated policy. The second loop keeps the connection between prediction, action, and consequence maintained.
Core Technical Glossary
Reinforcement Learning from Human Feedback (RLHF)
Training that combines supervised examples with reinforcement learning guided by human rankings of responses, turning a text-predictor into an instruction-following assistant.
Reinforcement Learning for Calibrated Decisions (RLCD)
TypeSafe's training objective for Jev, aimed at producing probabilities that correspond to how often decisions are actually right.
Calibration
The property that stated confidence matches observed correctness across a group of predictions, making risk measurable enough to act on.
Preference
One of three objectives: which response a person would prefer — useful for instruction-following and tone, but not a guarantee of truth.
Correctness
One of three objectives: whether the answer satisfied an objective verifier, as in mathematics or code.
Sycophancy
A failure mode where a model becomes excessively agreeable, favoring convincing or expectation-aligned answers over correct ones.
Harness
The operating layer around a model that supplies context, permissions, approval rules, failure handling, recovery, and a record of what happened.
Meta-harness
The layer that coordinates many agents, models, tools, permissions, context, and execution environments — the product that makes agents useful together.
Operational Loop
The loop that handles each decision: prediction, policy, action, result.
Measurement Loop
The loop that watches over time: outcome, audit, calibration, drift detection, and updated policy.
Seat Anchor
The per-seat pricing assumption that value scales with the number of humans using the software; automation shifts the unit toward decisions and outcomes.
Decision Rule
The policy that maps a probability and its context to an action, driven by the cost of error, the cost of review, and reversibility.
Knowledge Graph Explorer
Explore the entities and relationships extracted from the newsletter article. Drag nodes to pin them; double-click to unpin; click the pane to arm zoom.
Advanced Settings
Physics
Predicates
Display
SPARQL Workbench
Run these queries against the Knowledge Graph (hosted in URIBurner, Virtuoso-backed).
S1 List all entity types and counts▼
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?type (SAMPLE(?s) AS ?sampleEntity) (SAMPLE(?label) AS ?sampleLabel) (COUNT(?s) AS ?entityCount)
WHERE {
GRAPH <https://linkeddata.uriburner.com/DAV/demos/daas/jev-beyond-human-in-the-loop-ai-deepseek_v4pro-1.ttl> {
?s rdf:type ?type .
OPTIONAL { ?s rdfs:label ?label }
}
}
GROUP BY ?type ORDER BY DESC(?entityCount)
S2 Software applications and their creators▼
PREFIX schema: <http://schema.org/>
SELECT ?appIri ?app ?creatorIri ?creator
WHERE {
GRAPH <https://linkeddata.uriburner.com/DAV/demos/daas/jev-beyond-human-in-the-loop-ai-deepseek_v4pro-1.ttl> {
?appIri a schema:SoftwareApplication ; schema:name ?app .
OPTIONAL { ?appIri schema:creator ?creatorIri . ?creatorIri schema:name ?creator }
}
}
ORDER BY ?app