Jev 1.13 jaggedness: known failure modes

TL;DR jev-1.13 is fast, calibrated and good at common-sense judgment, but it reads instructions literally, is not a calculator, cannot order dates, loses accuracy on indirection and on large irrelevant states, is not hardened against adversarial text, gets confused by criteria that contradict the instruction, does not honour structural invariants across separate questions, and is not trained to generate text. Keep arithmetic, dates, counting and text generation in your code.

Applies to jev-1.13. Last reviewed 2026-09-17 (per raw/docs/model-jaggedness__jev-1.13.md).

What "jaggedness" means here

jev-1.13 does the best on System One tasks. It may struggle with tasks that require additional levels of indirection. It can be quite literal in its understanding. It struggles with tasks that require numeric precision. Upstream states that many of these edges will be fixed in later versions.

The nine failure modes

(Table reproduced from the source; the "do this instead" column is upstream's wording verbatim.)

1. Literal reading

jev-1.13 answers the question you wrote, not the one you meant. Scoping words, negations, and implied conditions are read at face value. A question will be answered based on the words written in the instruction, whereas a person might have read the intent behind the instructions.

Instead: state the exact condition in the instructions. Be specific. Put boundary cases in the criteria. When you look at a wrong answer and find yourself explaining what you really meant, that explanation is the missing half of the instruction. Where interpretation is unavoidable, split it into two literal questions and combine them in code.

See Writing instructions and criteria that Jev reads correctly for worked before/after rewrites.

2. Math and numbers

Jev is not a calculator. Implement any mathematical logic in code. Jev performs better on semantic questions than mathematical ones.

Counting

jev-1.13 does not count reliably — characters in a word, occurrences of a term in a passage, items in a long list. The model recognizes the shape of an answer rather than tallying, and the error grows with the size of the thing being counted.

Before asking a counting question, ask why the count needs a model at all. If the unit is something a regular expression or a parser can find, the count belongs in code.

Instead: count in code. Iterate over the candidates, ask one question for each, then add up the answers yourself.

from typesafe_sdk import Noul, TypeSafeClient

client = TypeSafeClient(model="jev-1.13")
YES = 0.5  # up to you on what you want the threshold to be, depends on your usecase.

items = ["typesafe", "apple", "california", "banana", "likes", "calibration", "orange", "vertex"]

result = client.system_one(
    {"items": items},
    {
        f"item_{i}": Noul(instructions=f"Is `items[{i}]` the name of a fruit?")
        for i in range(len(items))
    },
)

count = sum(result.nouls[f"item_{i}"].noul > YES for i in range(len(items)))

Note: this upstream snippet reads result.nouls[...], while every other Python sample in the docs reads response.answers[...]. Both spellings appear in the sources; see Contradictions below and Python SDK responses, answers, usage, models.

Numeric representations

jev-1.13 performs better on semantic representations than numeric. Questions about colors using hex values underperform compared to those using English names; given RGB triples or hex values it cannot reliably judge whether two values are near each other. Similarly, questions about high-level programming languages perform better than questions about low-level assembly or binary-encoded instructions.

Instead: do the conversion in code and pass in either the computed number or a named bucket. Keep the model for the part that is genuinely a judgment, such as whether a color reads as a warning.

Math using Score

Do not use score outputs (expectations and probability) to compute the exact magnitude of a number between two levels of a criterion. You can use the expectation to check if it passes a particular threshold, but jev-1.13's score levels are weak in numerical calibration: it will not be able to help you reconstruct the exact number by interpolating between the nearest two levels. See Score questions.

3. Date and time comparison

jev-1.13 reads dates as text, not as ordered quantities. Asking which of two dates comes first, how far apart they are, or whether one falls inside a window is unreliable. It gets worse with mixed formats, relative references, and domain boundaries such as quarters, settlement windows, and accrual periods.

Instead: split the work. Extraction is a judgment, so give it to the model; arithmetic is not, so keep it in code.

Every part of a date is a small closed set: twelve months, thirty-one possible days, a bounded range of years. That turns extraction into a Choice over enumerated options rather than free-form parsing, and it gives you somewhere to put an explicit "not stated" option so a missing part is reported rather than guessed. Code assembles the parts into a real date and owns everything after that: ordering, duration, offset, and weekday.

Cookbook: Date extraction has the worked version, including relative dates and confidence gating.

4. Indirection

Instructions carrying double negatives or complex indirection are answered less reliably. A question about a property of a property, or one that requires multiple hops of reasoning, costs accuracy.

Instead: write your instructions as directly as possible. When possible, identify the relevant parts of state by name — the backticked dot-and-index paths shown in Primitives: Choice, Score, Noul.

5. Large state full of irrelevant detail

Accuracy falls as the state grows with content unrelated to the decision. Unrelated detail acts as a distractor, and a large state makes it harder to tell which part of the input produced a wrong answer.

Instead: retrieve and filter in code first, and send only the fields the question needs. When it's not possible to filter in state, use a Noul to filter for relevance — Cookbook: Classifying RAG passages has a worked example.

jev-1.13 has a bounded context window; see Models, aliases, pricing, rate limits, context for the exact token limits.

6. Adversarial content

State is data, and jev-1.13 does not treat it as hostile by default. Content written to adversarially steer the model — an injected instruction, a deliberately misleading framing, or text that argues for its own classification — can move the answer. TypeSafe says it expects to improve on this in the future.

Instead: be explicit in the criteria. Test your integration thoroughly before deploying it to many users. See Testing and evaluating a Jev workflow.

7. Contradictory instructions and criteria

When the instructions and the criteria ask for different things, jev-1.13 might get confused. The best performance comes from clear phrasing. For example, a Noul where true maps to no and false maps to yes will perform worse. Aim for instructions which are easy for the average person to read and understand.

Instead: treat the criteria as an extension of the instruction. Align the two using clear and precise language.

8. Common-sense structural invariants

jev-1.13 is extremely consistent — expect quantitatively similar outputs for semantically similar inputs. However, many structural invariants one might imagine to hold are simply not guaranteed by the model.

"Is the customer asking for a refund?", asked as a Noul and as a yes/no Choice on the ticket "I'm not happy with the fit. What are my options here?":

The comparable numbers are noul and probabilities["yes"], and it is not obvious how to interpret either the Choice output and confidence for the Noul question or vice versa.

The same question and its negation, "Is the customer asking for something other than a refund?", as two Nouls on the ticket "I was charged twice for the same order. Can someone look into this?":

There are many reasons that P(noul) and 1 - P(not noul) may not be directly comparable.

Instead: don't rely on expected structural invariance, and word questions to mean directly what you want. Don't carry a threshold tuned on a Noul over to a Choice, and don't hold the model to arithmetic identities between separate questions. A Choice over options and one Noul per option answer different questions: the Choice is relative, settling which option, while each Noul is absolute and can be low for all of them. Cookbook: Skill suggestion uses both on the same shortlist — the Choice to pick a skill and the Nouls to decide whether to suggest one at all.

9. Generation

jev-1.13 is not trained to generate text. You can force it to by chaining choices, but this will not work well and will be very slow. For data extraction, it is better to extract possible options using regex or a generative model and let jev-1.13 pick the correct extraction.

Instead: when the answer space is bounded, turn extraction into a Choice over the options rather than asking for the value itself. If you really need to generate text, use a different model.

Checklist: avoid the following

Reproduced from the source's closing reminder:

• Asking the model something code can compute exactly.
• Hiding several judgments inside one question.
• System Two tasks: more layers of indirections.
• Giving it more context in state than the question needs. Jev suffers from context rot, so unrelated material in the state costs you accuracy.

TypeSafe invites reports of new failure modes on its Discord (https://discord.com/invite/WUujKYBp8s).

Contradictions noticed between sources

• result.nouls[...] vs response.answers[...]. The counting snippet on the jaggedness page reads answers through result.nouls[...]; every Python sample in raw/docs/primitives.md, raw/docs/primitives__choice.md and raw/docs/primitives__score.md reads response.answers[...]. Treat answers as the documented accessor and check Python SDK responses, answers, usage, models before relying on nouls.
• Positional system_one arguments. The same snippet calls client.system_one({...}, {...}) positionally and sets the model on the client (TypeSafeClient(model="jev-1.13")), whereas other samples use client.system_one(state=..., questions=...). Both appear in the sources.

Related

• Writing instructions and criteria that Jev reads correctly — how to write around modes 1, 4 and 7
• Choosing between Choice, Score, Noul — how to avoid modes 8 and 9 by picking the right type
• Testing and evaluating a Jev workflow — testing edge cases before deploying
• Choice questions, Score questions, Noul (yes/no) questions — the primitives referenced above
• State: what you send Jev — filtering the state before you send it
• Models, aliases, pricing, rate limits, context — context window and model aliases
• Jev (model) — the model itself
• Cookbook: Date extraction, Cookbook: Classifying RAG passages, Cookbook: Skill suggestion

Sources

• raw/docs/model-jaggedness__jev-1.13.md (https://docs.typesafe.ai/model-jaggedness/jev-1.13)
• raw/docs/primitives__choice.md (https://docs.typesafe.ai/primitives/choice)
• raw/docs/primitives__noul.md (https://docs.typesafe.ai/primitives/noul)