Migrating from /preview/evaluation to /v1/systemone

TL;DR POST /preview/evaluation is replaced by POST /v1/systemone. Auth is unchanged. document → state, the prompts array → a questions map, options/levels → a unified criteria per type, the responses array → an answers map, and the answer value fields are renamed (probability → noul, chosen → choice, expectation → score). confidence is computed differently, so re-tune any thresholds.

This is a breaking change: the endpoint, the request shape, and the response shape all changed (raw/docs/migrating-to-v1.md). Auth is unchanged — keep sending Authorization: Bearer <API_KEY> and Content-Type: application/json.

At a glance

(Table reproduced from raw/docs/migrating-to-v1.md.)

1. Endpoint rename

evaluation became systemone.

- POST https://api.typesafe.ai/preview/evaluation
+ POST https://api.typesafe.ai/v1/systemone

2. prompts array becomes a questions map

In preview, you sent a prompts array where each item carried its own key, and answers came back in the same order. In v1, you send a questions map: the map key is an id of your choice, and answers come back mapped to the same id.

// preview — POST /preview/evaluation
{
  "model": "jev-latest",
  "document": "Sample document",
  "prompts": [
    { "key": "test_noul", "type": "noul", "instructions": "Test instructions" }
  ]
}

// v1 — POST /v1/systemone
{
  "model": "jev-latest",
  "state": "Sample document",
  "questions": {
    "test_noul": { "type": "noul", "instructions": "Test instructions" }
  }
}

Migration consequence: any code that relied on positional ordering of responses must switch to key lookup. The keys are yours and are "not sent to the underlying model" (raw/docs/api.md).

3. Unified criteria per question type

Every question type now describes itself with a field named criteria. The shape differs by type:

Choice criteria

// preview
{
  "key": "test_choice",
  "type": "choice",
  "instructions": "Test instructions",
  "options": [
    {
      option: "Option A",
      description: "Description A"
    },
    {
      option: "Option B",
      description: "Description B"
    }
  ]
}

// v1 (within "questions")
"test_choice": {
  "type": "choice",
  "instructions": "Test instructions",
  "criteria": {
    "Option A": "Description A",
    "Option B": "Description B"
  }
}

Score criteria

// preview
{
  "key": "test_score",
  "type": "score",
  "instructions": "Test instructions",
  "levels": [
    { "level": 0, "description": "Poor" },
    { "level": 1, "description": "Good" },
    { "level": 2, "description": "Excellent" }
  ]
}

// v1 (within "questions")
"test_score": {
  "type": "score",
  "instructions": "Test instructions",
  "criteria": ["Poor", "Good", "Excellent"]
}

4. responses array becomes an answers map

Preview returned a responses array in the same order as your prompts, with each item containing a key property. v1 returns an answers map keyed by the question id you chose.

// preview
{
  "responses": [
    { "key": "test_noul", "type": "noul", "probability": 0.92 }
  ]
}

// v1
{
  "answers": {
    "test_noul": { "type": "noul", "noul": 0.92 }
  }
}

The key property is gone from each answer — the map key carries it.

5. Renamed answer value fields

6. Choice probabilities becomes a map

A Choice answer's probabilities changed from an array of { option, probability } objects to a map of option → probability. Score answers now also include a probabilities map over their levels, which preview did not return at all.

// preview
"probabilities": [
  { "option": "billing", "probability": 0.08 },
  { "option": "technical", "probability": 0.85 }
]

// v1
"probabilities": { "billing": 0.08, "technical": 0.85 }

7. Confidence is a new computation

Verbatim: "The computation behind confidence changed to make the value reliable, so the value will differ from preview even for an identical evaluation. Any logic your integration uses based on confidence should be carefully re-evaluated."

v1 also returns the full probability distribution for both Choice and Score, so you can compute your own statistic. To reproduce preview's confidence, use a normalized-entropy computation (verbatim from raw/docs/migrating-to-v1.md):

import math

def normalized_entropy_confidence(probabilities: dict[str, float]) -> float:
    """Reproduce preview's confidence: 1 − normalized Shannon entropy of the
    distribution. Returns 1.0 when all weight is on one outcome, 0.0 when the
    distribution is perfectly uniform."""
    ps = [p for p in probabilities.values() if p > 0]
    n = len(probabilities)
    if n <= 1:
        return 1.0
    entropy = -sum(p * math.log(p) for p in ps)
    return 1.0 - entropy / math.log(n)


# choice: probabilities maps each option -> probability
# score:  probabilities maps each level (string key) -> probability
answer = response.answers["department"]
my_confidence = normalized_entropy_confidence(answer.probabilities)

The source does not state what the new v1 computation is; see Confidence vs probability.

8. Usage reports token counts

Preview reported a placeholder usage.billing_units. v1 reports token counts instead: usage.input_tokens and usage.output_tokens.

- "usage": { "billing_units": 1 }
+ "usage": { "input_tokens": 312, "output_tokens": 48 }

Billing follows: input tokens are charged, output tokens are free. See Models, aliases, pricing, rate limits, context.

9. document becomes state

Verbatim: "The field that carries the content to evaluate is named state. Preview, and the first v1 releases, called it document. v1 now accepts only state; a request with document fails validation."

- "document": "Sample document",
+ "state": "Sample document",

The content itself is unchanged: a string, an object, or an array. See State: what you send Jev.

Note the three-stage history implied here: preview used document, the first v1 releases also used document, and current v1 accepts only state. A client pinned to an early v1 build still needs this change.

Full before/after example

Request

// preview — POST /preview/evaluation
{
  "model": "jev-latest",
  "document": "Hi, I've been trying to connect my Stripe account for 3 days and it keeps failing. I'm losing sales. Please help ASAP.",
  "prompts": [
    {
      "key": "is_urgent",
      "type": "noul",
      "instructions": "Does this message convey urgency?"
    },
    {
      "key": "department",
      "type": "choice",
      "instructions": "Which team should handle this?",
      "options": [
        { "option": "billing", "description": "Payment/invoicing" },
        { "option": "technical", "description": "Bugs/integrations" },
        { "option": "sales", "description": "Pricing/upgrades" }
      ]
    },
    {
      "key": "frustration",
      "type": "score",
      "instructions": "How frustrated is the customer?",
      "levels": [
        { "level": 0, "description": "Calm, stating facts" },
        { "level": 1, "description": "Frustrated but civil" },
        { "level": 2, "description": "Very angry" }
      ]
    }
  ]
}

// v1 — POST /v1/systemone
{
  "model": "jev-latest",
  "state": "Hi, I've been trying to connect my Stripe account for 3 days and it keeps failing. I'm losing sales. Please help ASAP.",
  "questions": {
    "is_urgent": {
      "type": "noul",
      "instructions": "Does this message convey urgency?"
    },
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this?",
      "criteria": {
        "billing": "Payment/invoicing",
        "technical": "Bugs/integrations",
        "sales": "Pricing/upgrades"
      }
    },
    "frustration": {
      "type": "score",
      "instructions": "How frustrated is the customer?",
      "criteria": ["Calm, stating facts", "Frustrated but civil", "Very angry"]
    }
  }
}

Response

// preview response
{
  "model": "jev-latest",
  "responses": [
    { "key": "is_urgent", "type": "noul", "probability": 0.92 },
    {
      "key": "department",
      "type": "choice",
      "chosen": "technical",
      "probabilities": [
        { "option": "billing", "probability": 0.08 },
        { "option": "technical", "probability": 0.85 },
        { "option": "sales", "probability": 0.07 }
      ],
      "confidence": 0.82
    },
    {
      "key": "frustration",
      "type": "score",
      "expectation": 1.6,
      "confidence": 0.78
    }
  ],
  "usage": { "billing_units": 1 }
}

// v1 response
{
  "model": "jev-latest",
  "answers": {
    "is_urgent": { "type": "noul", "noul": 0.92 },
    "department": {
      "type": "choice",
      "choice": "technical",
      "probabilities": { "billing": 0.08, "technical": 0.85, "sales": 0.07 },
      "confidence": 0.82
    },
    "frustration": {
      "type": "score",
      "score": 1.6,
      "legend": { "0": "Calm, stating facts", "1": "Frustrated but civil", "2": "Very angry" },
      "probabilities": { "0": 0.05, "1": 0.3, "2": 0.65 },
      "confidence": 0.78
    }
  },
  "usage": { "input_tokens": 312, "output_tokens": 48 }
}

Note that v1 adds legend on Score answers, which preview did not return.

Python SDK

The v1 API is served by a new package, typesafe-sdk, installed from PyPI with pip install typesafe-sdk. Verbatim: "The previous typesafe-client package (every release, 0.1.x and 1.0.x) sends document and no longer works against the API." All of the API changes above apply, plus these client renames:

(Table reproduced from raw/docs/migrating-to-v1.md.)

Migration checklist

1. Change the URL to https://api.typesafe.ai/v1/systemone. Auth headers stay the same.
2. Rename document → state.
3. Convert prompts[] to a questions map, dropping each item's key into the map key.
4. Convert Choice options[] → criteria map and Score levels[] → ordered criteria array; fill any skipped levels, since they are no longer expressible.
5. Optionally add Noul criteria.true / criteria.false, new in v1.
6. Switch response reads from responses[i] to answers[key].
7. Rename probability → noul, chosen → choice, expectation → score.
8. Convert Choice probabilities array handling to map handling; start using Score probabilities and legend.
9. Re-tune every confidence threshold, or compute your own statistic from probabilities.
10. Replace usage.billing_units with usage.input_tokens / usage.output_tokens in any cost accounting.
11. Python: uninstall typesafe-client, pip install typesafe-sdk, and apply the rename table.

Related

• HTTP API: POST /v1/systemone and GET /v1/models — the v1 contract in full
• OpenAPI component schemas — v1 schemas field by field
• Python SDK: install, clients, system_one() — the typesafe-sdk client
• Python SDK changelog — when each rename landed
• Confidence vs probability — what the new confidence means
• State: what you send Jev — allowed shapes for state
• Versions and timeline (models, SDKs, API, company) — API, model, and SDK versions over time

Sources

• raw/docs/migrating-to-v1.md (https://docs.typesafe.ai/migrating-to-v1)
• raw/docs/api.md (https://docs.typesafe.ai/api)