---
title: "Cookbook: Hierarchical classification"
type: cookbook
tags: [cookbook, classification, choice, beam-search, taxonomy]
created: 2026-09-17
updated: 2026-09-17
confidence: high
sources:
  - raw/docs/cookbooks__hierarchical_classification.md
jev_version: "jev-1.13.0"
sdk_python: "0.6.0"
summary: "Walk a deep taxonomy to a leaf by asking one Choice per sibling set, and keep K paths alive with a geometric-mean beam search instead of a greedy walk."
---

# Cookbook: Hierarchical classification

> **TL;DR** One `Choice` per sibling set, with `criteria` mapping opaque keys (`c0`, `c1`, …) to the child labels; the answer's `probabilities` are the edges. Score a path by `product(edge_probabilities) ** (1 / decisions)` and keep the best `K = 3` — beam search matched 4 of 4 expected leaves where greedy matched 2 of 4.

## Goal

Classify a document into a deep hierarchy by traversing it node by node to the correct leaf. Greedy search takes the highest-probability child at every node and cannot recover from one early mistake; beam search keeps `K` plausible paths, classifies every frontier in parallel, and lets deeper evidence repair an ambiguous early decision.

The cookbook argues that decomposing classification into a hierarchy buys **observability** (which nodes your misclassifications occur in; how often each node and edge is traversed) and **testability** (unit-test hierarchy updates and measure their impact).

## Inputs / state shape

The **state** is the plain document string. The hierarchy is a nested `dict[str, Tree]` of direct-child menus, loaded from four pinned sources:

| Hierarchy | Version | Source |
|---|---|---|
| CPC patents | 2026.05 | `CPCSchemeXML202605.zip` (cooperativepatentclassification.org) |
| Shopify products | 2026-02 | `Shopify/product-taxonomy` `v2026-02` `dist/en/categories.txt` |
| MeSH biomedical subjects | 2026 | `desc2026.zip` (nlmpubs.nlm.nih.gov) |
| CookSafe files | snapshot 2026-08-06 | `codebase_files.txt` (frozen repo listing) |

MeSH is a DAG — one descriptor can appear under multiple parents — so the demo expands its official tree-number paths. The CookSafe listing is a frozen snapshot rather than a live walk, because **line order is significant: sibling options are asked in the order they appear, so it is part of the question, not presentation**.

The four documents and their expected leaves, verbatim:

| Hierarchy | `document` | `expected_leaf` |
|---|---|---|
| CPC patents | `Patent abstract: a freestanding structural wooden perch for poultry or pet birds. The elevated roost has crossbars sized for bird feet and mounts inside an aviary.` | `A01K31/12 Perches for poultry or birds, e.g. roosts` |
| Shopify products | `Furniture listing: a wall-mounted window shelf bed. This padded floating shelf uses suction cups and a washable cushion as a sunny perch for one cat.` | `Cat Window Beds & Perches` |
| MeSH | `Clinical abstract: Crohn disease with transmural ileocolonic inflammation, skip lesions, abdominal pain, and chronic diarrhea. Colonoscopy showed cobblestoning and biopsy found noncaseating granulomas; treatment with infliximab produced remission.` | `C06.405.469.432.500 Crohn Disease` |
| CookSafe files | `Developer search: find the experimental Python module under x/eugene that implements BM25, dense, and fused retrievers for legal RAG.` | `retrievers.py` |

## Questions asked

Exactly one question type, asked once per sibling set. Verbatim:

```python
def child_question(labels: tuple[str, ...]) -> tuple[Choice, dict[str, str]]:
    """Build the direct-child Choice and its reversible option mapping."""
    keys = {f"c{i}": label for i, label in enumerate(labels)}
    question = Choice(
        instructions="Which direct child category best matches this document?",
        criteria=keys,
    )
    return question, keys
```

- `type`: `Choice`
- `instructions`: `Which direct child category best matches this document?`
- `criteria`: `{"c0": "<label 0>", "c1": "<label 1>", ...}` — the option *keys* are synthetic (`c0`, `c1`, …) and the child labels are the option *descriptions*, so the mapping is reversible and label text never has to be a valid key.

The same instructions string is reused at every depth; all the node-specific information lives in `criteria`.

## Combining logic in code

```python
import os
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path

from typesafe_sdk import Choice, RetryPolicy, TypeSafeClient

HIERARCHIES = load_hierarchies()
MODEL, BEAM_WIDTH, MAX_DEPTH, EPSILON = "jev-1.12", 3, 12, 1e-9
client = TypeSafeClient(
    api_key=os.environ["TYPESAFE_API_KEY"],
    retry=RetryPolicy(max_retries=5, backoff_initial=1.0, backoff_max=20.0),
)


def choose(state: str, labels: tuple[str, ...]) -> dict[str, float]:
    """Ask one atomic direct-child question and return its distribution."""
    if len(labels) == 1:
        return {labels[0]: 1.0}
    question, keys = child_question(labels)
    response = client.system_one(
        state=state, questions={"child": question}, model=MODEL
    )
    probabilities = response.answers["child"].probabilities
    return {label: probabilities[key] for key, label in keys.items()}


def extend_candidate(candidate: dict, label: str, probabilities: dict[str, float]) -> dict:
    """Append one edge and recompute its geometric-mean path score."""
    is_decision: bool = len(probabilities) > 1
    probability_product: float = candidate["probability_product"] * (
        max(probabilities[label], EPSILON) if is_decision else 1.0
    )
    decision_count: int = candidate["decision_count"] + is_decision
    return {
        "path": candidate["path"] + (label,),
        "probability_product": probability_product,
        "decision_count": decision_count,
        "score": probability_product ** (1 / decision_count) if decision_count else 1.0,
    }
```

The beam itself, trimmed of diagram bookkeeping:

```python
def beam_search(hierarchy) -> dict:
    """Parallel width-three beam search using geometric-mean probability."""
    beam = [{"path": (), "probability_product": 1.0, "decision_count": 0, "score": 1.0}]
    for _ in range(MAX_DEPTH):
        expandable = [c for c in beam if subtree(hierarchy.tree, c["path"])]
        finished = [c for c in beam if not subtree(hierarchy.tree, c["path"])]
        if not expandable:
            break
        with ThreadPoolExecutor(max_workers=BEAM_WIDTH) as executor:
            distributions = list(
                executor.map(
                    lambda c: choose(
                        hierarchy.document, tuple(subtree(hierarchy.tree, c["path"]))
                    ),
                    expandable,
                )
            )
        expanded = []
        for candidate, probabilities in zip(expandable, distributions, strict=True):
            for label in probabilities:
                expanded.append(extend_candidate(candidate, label, probabilities))
        beam = sorted(finished + expanded, key=lambda c: c["score"], reverse=True)[:BEAM_WIDTH]
    return {"beam": sorted(beam, key=lambda c: c["score"], reverse=True)}
```

Greedy is the same walk with `label = max(probabilities, key=probabilities.get)` and no beam.

**Metrics.**

- `path_score = product(edge_probabilities) ** (1 / decisions)` — used for pruning and comparing paths, length-normalized so shallow and deep leaves compare fairly.
- `separation = top_path_score / second_path_score` — reported but **not** used for pruning. Near `1×` is ambiguous; a large ratio means clear separation.
- Alternatives the cookbook names: `min(top_prob/second_top_prob)` optimizes for paths with very clear decisions at every node; `exp(mean(log(probs)))` avoids precision errors for hierarchies deeper than ~10 layers.

Note that a sibling set of size 1 is not a decision: `choose` short-circuits to `{label: 1.0}` with no API call, and `decision_count` is not incremented.

## Results / what the cookbook reports

With `BEAM_WIDTH = 3`, `MAX_DEPTH = 12`, model `jev-1.12`: **beam search matched 4 of 4 expected leaves; greedy search matched 2 of 4.** Keeping three paths recovered the expected classification for CPC patents and Shopify products.

| Hierarchy | Expected leaf | Greedy leaf | Beam K=3 leaf | Greedy correct | Beam correct |
|---|---|---|---|---|---|
| CPC patents | `A01K31/12 Perches for poultry or birds, e.g. roosts` | `E99Z99/00 Subject matter not otherwise provided for in this section` | `A01K31/12 Perches for poultry or birds, e.g. roosts` | no | yes |
| Shopify products | `Cat Window Beds & Perches` | `Pet Chairs` | `Cat Window Beds & Perches` | no | yes |
| MeSH biomedical subjects | `C06.405.469.432.500 Crohn Disease` | `C06.405.469.432.500 Crohn Disease` | `C06.405.469.432.500 Crohn Disease` | yes | yes |
| CookSafe files | `retrievers.py` | `retrievers.py` | `retrievers.py` | yes | yes |

Four examples is the whole evaluation — treat this as a demonstration, not a benchmark (inferred).

## Adapting it to a new domain

1. Produce a nested `dict` of direct-child menus for your taxonomy (filesystem, org chart, moderation policy, LLM skill catalog, ontology). Fix the sibling order and keep it fixed, since order is part of the question.
2. Leave `child_question` alone unless your labels need more than a name — if a label is cryptic, put a description in `criteria` instead of the bare label.
3. Tune `BEAM_WIDTH` (cost per level scales with it, latency barely does because frontiers run in parallel) and `MAX_DEPTH` to your deepest path.
4. Log `separation_ratio` per document and route low-separation documents to review.

## Gotchas

- **Choice option limits.** Very wide sibling sets will hit the `Choice` option ceiling (255 options, per [[cookbooks/pre-parsed-value-extraction]] and [[cookbooks/semantic-find]]); split a wide menu into chunks and run the same procedure over the winners (inferred for this cookbook, stated in the others).
- **Deep trees lose precision.** The cookbook explicitly recommends `exp(mean(log(probs)))` past ~10 layers; `EPSILON = 1e-9` floors any zero edge so the product never collapses.
- **A single-child level is free but invisible** — it contributes no decision and no probability.
- **MeSH is a DAG**, so the same descriptor appears on several paths; the demo expands tree numbers into distinct paths rather than deduplicating.
- **Retries matter at this call volume.** The client is constructed with `RetryPolicy(max_retries=5, backoff_initial=1.0, backoff_max=20.0)`.
- **Install line.** This cookbook prints no `pip install` line; it imports `from cooksafe import JsonCache` and reads `os.environ["TYPESAFE_API_KEY"]`. Sibling cookbooks use `pip install ... "typesafe-sdk>=0.5.7" cooksafe --extra-index-url https://pypi.typesafe.ai/`. `cooksafe` lives on TypeSafe's private index and `pypi.typesafe.ai` returned **404 publicly as of 2026-09-17**, so use `pip install typesafe-sdk` and reimplement `JsonCache` — a decorator that memoizes a function's return value into a JSON file keyed on its arguments, so re-running replays cached answers with no API spend.

## Related

- [[concepts/choice]] — probabilities over a fixed option set
- [[concepts/advanced-structure]] — structured instructions, options, levels, criteria
- [[patterns/fan-out]] — running every frontier's question concurrently
- [[patterns/intent-routing]] — the shallow, one-level version of this
- [[cookbooks/classification-using-confidence]] — reporting a parent class when the child is uncertain
- [[cookbooks/skill-suggestion]] — rank-then-re-read as an alternative to beam search
- [[cookbooks/overview]] — the full cookbook catalog

## Sources

- raw/docs/cookbooks__hierarchical_classification.md (https://docs.typesafe.ai/cookbooks/hierarchical_classification.md)