--- title: "Cookbooks overview" type: cookbook tags: [cookbook, index, catalog, patterns] created: 2026-09-17 updated: 2026-09-17 confidence: high sources: - raw/docs/cookbooks.md - raw/docs/cookbooks__autoformat.md - raw/docs/cookbooks__autoresearch_feature_discovery.md - raw/docs/cookbooks__citation_check.md - raw/docs/cookbooks__classification_using_confidence.md - raw/docs/cookbooks__classifying_rag_passages.md - raw/docs/cookbooks__consistency_choice_cookbook.md - raw/docs/cookbooks__consistency_noul_cookbook.md - raw/docs/cookbooks__date_extraction_cookbook.md - raw/docs/cookbooks__entity_alignment.md - raw/docs/cookbooks__function_calling.md - raw/docs/cookbooks__hierarchical_classification.md - raw/docs/cookbooks__llm_guardrails.md - raw/docs/cookbooks__parallel_questions.md - raw/docs/cookbooks__pre_parsed_value_extraction_cookbook.md - raw/docs/cookbooks__rerank_typesafe.md - raw/docs/cookbooks__sde_cascade.md - raw/docs/cookbooks__semantic_find.md - raw/docs/cookbooks__skill_suggestion.md jev_version: "jev-1.13.0" sdk_python: "0.6.0" summary: "Catalog of all 18 TypeSafe cookbooks — goal, primitives and domain each — plus how a cookbook is structured and why the published pip line does not work." --- # Cookbooks overview > **TL;DR** Eighteen worked recipes, each one state + a set of questions + combining logic in your own code. Every published cookbook installs `cooksafe` from `https://pypi.typesafe.ai/`, which **returned 404 publicly as of 2026-09-17** — install `pip install typesafe-sdk` instead and reimplement the two helpers (`JsonCache`, `make_playground_link`) yourself; nothing in the recipes themselves depends on them. ## The catalog One-line goals are the cookbook's own summary line, condensed. Primitives are the question types actually constructed in the code. | slug | goal | primitives | domain | |---|---|---|---| | [[cookbooks/autoformat]] | Reconstruct Markdown from plain text that lost its formatting, in two requests: one restitches hard-wrapped lines, one classifies every block (heading, list, code, callout) with companion questions read only when relevant | Choice, Noul | document structure | | [[cookbooks/autoresearch-feature-discovery]] | Run an autoresearch loop that proposes questions, turns free text into numeric features, and uses model errors to improve a supervised CatBoost regressor | Score, Noul | tabular ML / feature engineering | | [[cookbooks/citation-check]] | Catch wrong or hallucinated citations against the source document; one `Choice` decides whether the quote's context supports the claim, and its confidence flags the citation for review | Choice | LLM verification | | [[cookbooks/classification-using-confidence]] | Classify SEC annual reports into 75 SIC industry groups with one `Choice` each, then read the answer's confidence to decide whether to report that group or the broader division above it | Choice | finance / taxonomy | | [[cookbooks/classifying-rag-passages]] | Score each retrieved passage in one request, then decide in code which reach the answering model — keep and flag contradictions, drop hidden instructions and prompt injections | Noul | RAG safety | | [[cookbooks/consistency-choice]] | Add an `uncertain` outcome to moderation decisions and compare label agreement with the share of automatic actions, over 15 repeats per condition | Choice | trust & safety / evaluation | | [[cookbooks/consistency-noul]] | Run a 14-question claims rubric 15 times and route probabilities from 0.30–0.70 to human review while keeping the underlying noul values visible | Noul | insurance claims / evaluation | | [[cookbooks/date-extraction]] | Ask for the parts of a date named in a document, then resolve and validate them to a `date` in code with confidence-based review | Choice | extraction | | [[cookbooks/entity-alignment]] | Decide which of 450 candidate pairs from two beer catalogues describe the same product; one `Score` whose three levels *are* merge / leave unlinked / hand to a curator, with three `Noul`s riding along to say which field disagrees | Score, Noul | knowledge graphs | | [[cookbooks/function-calling]] | Turn natural-language trading requests into calls to ordinary typed functions by mapping function names and closed-set arguments to confidence-aware questions | Choice, Noul | agents / tool use | | [[cookbooks/hierarchical-classification]] | Classify documents through deep patent, retail, biomedical and source-code hierarchies using parallel beam search over `Choice` probabilities | Choice | taxonomy classification | | [[cookbooks/llm-guardrails]] | Screen every message into and out of an LLM app with one request — hazard `Noul`s plus a severity `Score` — then threshold to pass, review, block, or route to support | Noul, Score | LLM safety | | [[cookbooks/parallel-questions]] | Run a 13-question regulatory briefing over the GDPR article and show batching is 12.2x cheaper and 10.0x faster with no change in answers | Noul, Choice, Score | cost / benchmarking | | [[cookbooks/pre-parsed-value-extraction]] | Use regexes to find candidate emails, phone numbers and amounts, then have Jev select the requested span so code can normalize a verbatim value | Choice, Noul | extraction | | [[cookbooks/rerank]] | Build 30-passage BM25 shortlists for 40 CLERC legal queries, then one question per query-candidate pair raises top-1 from 5% to 18% and top-10 from 38% to 62% | Noul | retrieval / legal | | [[cookbooks/sde-cascade]] | Use a 2-stage structured-data-extraction cascade (mini → verify → reasoning) to get most of a big reasoning model's quality at a fraction of the cost | Noul | extraction / cost | | [[cookbooks/semantic-find]] | Build semantic search over GitHub's Terms of Service: score 218 line ids against a plain-language query with a `Choice`, and use a `Noul` to check whether the document contains an answer | Choice, Noul | search within a document | | [[cookbooks/skill-suggestion]] | Pick at most one skill for an agent turn out of 182: one request ranks every skill and asks whether the turn needs one, a second re-reads the top three and can reject all of them | Choice, Noul | agents / context management | ## How a cookbook is structured Every published cookbook follows the same arc, and the useful part for an agent is always the middle three: 1. **A framing paragraph** naming the problem and what you end up with (a named function: `find()`, `guard()`, `suggest()`, `extract_date()`, a `Dispatcher`). 2. **Setup** — a `pip install` line, the environment variables to set, and a `TypeSafeClient(...)` construction. Most pin a model constant, usually `TYPESAFE_MODEL = "jev-1.12"`, and record the date the numbers were sampled. 3. **The state** — a document, a dict, or a pair of fields. Jev is given the same state for every question in a request. 4. **The questions** — the literal `Noul` / `Choice` / `Score` objects with their `instructions` and `criteria`. This is the transferable part: the instruction strings are engineered artifacts, and the criteria carry almost all the domain knowledge. 5. **Combining logic in ordinary Python** — thresholds, precedence, sorts, gates. TypeSafe's consistent line is that the model supplies the assessment and your application owns the decision. 6. **Results** — printed tables or charts, with a cached JSON file shipped alongside so re-rendering reproduces the published numbers with no API spend. 7. **A playground share link** (`https://console.typesafe.ai/playground#share/...`) holding the same state and questions. Recurring design moves worth lifting wholesale: put every question for one state in **one request** ([[patterns/fan-out]]); make the options be spans, ids or names taken from the document so the answer is a verbatim copy; add an explicit escape hatch (`none`, `uncertain`, a separate existence `Noul`) because `Choice` probabilities always sum to 1; and frame verification questions so that **`true` means something is wrong**, then aggregate with `max` rather than a mean. ## Install and the `cooksafe` caveat Thirteen of the eighteen cookbooks print a line of this shape (extra packages vary): ```bash pip install ipython "typesafe-sdk>=0.5.7" cooksafe --extra-index-url https://pypi.typesafe.ai/ ``` Variants add `anthropic openai matplotlib` (the model-comparison cookbooks), `bm25s datasets` (rerank), `openai datasets jsonschema` (sde-cascade), `polars matplotlib numpy` (function-calling), `phonenumbers` (pre-parsed-value-extraction), `catboost numpy` (autoresearch), `matplotlib` (entity-alignment, classification-using-confidence). [[cookbooks/hierarchical-classification]] prints no install line at all but still imports `cooksafe`. **`cooksafe` is a helper package served from TypeSafe's own package index, not PyPI, and `pypi.typesafe.ai` returned 404 publicly as of 2026-09-17.** The recipes do not depend on it. Do this instead: ```bash pip install typesafe-sdk ``` and reimplement the two helpers the cookbooks import: - **`JsonCache(Path("json_cache.json"))`** — a decorator that memoizes a function's return value into a JSON file keyed on its arguments, so re-running replays the published numbers without calling any API. Cookbooks deliberately include arguments that exist only to shape the cache key: a `sample_index` per repeat, a `rubric_hash` digest of the state and question text so editing a question forces a fresh sample instead of serving a stale answer, and the model name so an alias resolving to a new version does not reuse old answers. Delete the file to run live. - **`make_playground_link(state, questions, models=[...])`** — encodes the state and question set into a `https://console.typesafe.ai/playground#share/...` URL. Purely a convenience for the published pages. Several cookbooks also pass `api_key=os.environ.get("TYPESAFE_API_KEY", "cache-only")` so a keyless kernel can replay the cache; `"cache-only"` is a cookbook convention, not an API feature. ## Version notes Published cookbook code pins `jev-1.12` (or `jev-latest` in the self-consistency cookbooks) and `"typesafe-sdk>=0.5.7"`; this wiki documents `jev-1.13.0` and Python SDK `0.6.0`, so re-running live may shift probabilities. Where a cookbook builds a `Score`, it already uses the 0.6.0 **ordered-sequence** `criteria` form (a list of level descriptions, level 0 first) rather than an int-keyed dict — see [[cookbooks/llm-guardrails]] and [[cookbooks/parallel-questions]]. > **Source caveat:** `raw/docs/cookbooks.md`, fetched from `https://docs.typesafe.ai/cookbooks.md`, is byte-identical to `raw/docs/cookbooks__consistency_noul_cookbook.md` — the upstream index page did not capture a catalog. This table was therefore compiled from the eighteen individual cookbook sources listed in the frontmatter. ## Related - [[patterns/overview]] — the architectural patterns the cookbooks instantiate - [[patterns/fan-out]] — one request, many questions - [[patterns/confidence-routing]] — the pass / review / escalate shape - [[concepts/primitives]] — Choice, Score and Noul - [[guides/choosing-a-primitive]] — which question type a task wants - [[guides/writing-instructions-and-criteria]] — how the instruction strings in these recipes are written - [[guides/testing-and-evaluation]] — the consistency cookbooks as an evaluation method - [[concepts/use-case-map]] — the same territory organised by industry ## Sources - raw/docs/cookbooks.md (https://docs.typesafe.ai/cookbooks.md) — duplicate of the consistency-noul cookbook, see caveat above - raw/docs/cookbooks__autoformat.md (https://docs.typesafe.ai/cookbooks/autoformat.md) - raw/docs/cookbooks__autoresearch_feature_discovery.md (https://docs.typesafe.ai/cookbooks/autoresearch_feature_discovery.md) - raw/docs/cookbooks__citation_check.md (https://docs.typesafe.ai/cookbooks/citation_check.md) - raw/docs/cookbooks__classification_using_confidence.md (https://docs.typesafe.ai/cookbooks/classification_using_confidence.md) - raw/docs/cookbooks__classifying_rag_passages.md (https://docs.typesafe.ai/cookbooks/classifying_rag_passages.md) - raw/docs/cookbooks__consistency_choice_cookbook.md (https://docs.typesafe.ai/cookbooks/consistency_choice_cookbook.md) - raw/docs/cookbooks__consistency_noul_cookbook.md (https://docs.typesafe.ai/cookbooks/consistency_noul_cookbook.md) - raw/docs/cookbooks__date_extraction_cookbook.md (https://docs.typesafe.ai/cookbooks/date_extraction_cookbook.md) - raw/docs/cookbooks__entity_alignment.md (https://docs.typesafe.ai/cookbooks/entity_alignment.md) - raw/docs/cookbooks__function_calling.md (https://docs.typesafe.ai/cookbooks/function_calling.md) - raw/docs/cookbooks__hierarchical_classification.md (https://docs.typesafe.ai/cookbooks/hierarchical_classification.md) - raw/docs/cookbooks__llm_guardrails.md (https://docs.typesafe.ai/cookbooks/llm_guardrails.md) - raw/docs/cookbooks__parallel_questions.md (https://docs.typesafe.ai/cookbooks/parallel_questions.md) - raw/docs/cookbooks__pre_parsed_value_extraction_cookbook.md (https://docs.typesafe.ai/cookbooks/pre_parsed_value_extraction_cookbook.md) - raw/docs/cookbooks__rerank_typesafe.md (https://docs.typesafe.ai/cookbooks/rerank_typesafe.md) - raw/docs/cookbooks__sde_cascade.md (https://docs.typesafe.ai/cookbooks/sde_cascade.md) - raw/docs/cookbooks__semantic_find.md (https://docs.typesafe.ai/cookbooks/semantic_find.md) - raw/docs/cookbooks__skill_suggestion.md (https://docs.typesafe.ai/cookbooks/skill_suggestion.md)