---
title: "Composite scoring"
type: pattern
tags: [patterns, score, ranking, weights, composition]
created: 2026-09-17
updated: 2026-09-17
confidence: high
sources:
  - raw/docs/patterns__composite-scoring.md
  - raw/docs/patterns.md
  - raw/github/skills/skills/typesafe-ai/SKILL.md
jev_version: "jev-1.13.0"
summary: "Break a ranking judgment into independent Score dimensions, normalize each to 0–1, and combine them with weights your code owns and can retune."
---

# Composite scoring

> **TL;DR** Don't ask "rank this candidate." Ask four independent Score questions in one call, divide each `score` by `len(criteria) - 1` to normalize to 0–1, and combine with weights in code. Different weight vectors over the same answers give you different rankings (senior IC vs. engineering manager) without another API call.

## Problem

From `raw/docs/patterns__composite-scoring.md`:

> Oftentimes we want to rank a set of items based on several criteria at once.

A single "how good is this candidate" question buries the trade-offs inside the model: you cannot see why one resume outranked another, and adjusting the priorities means rewriting a prompt and re-running everything. You also cannot reuse the judgment for a second role with different priorities.

## Pattern

> Composite scoring is an easy way to think about this: break the judgment into independent dimensions, score each one separately, and combine them with weights you control in code.

Three moves:

1. **Decompose** into dimensions that can be judged independently.
2. **Score each** with its own rubric — an ordered `criteria` array where every level "describes concrete situations and stands on its own" (agent skill).
3. **Normalize and weight in code**, where the policy is readable and editable.

## Implementation

The documented example is resume screening: "you are processing resumes for engineering roles. You want to rank the candidates based on several criteria, and ultimately select the top X candidates for further review."

### Step 1: score each dimension independently

Questions, verbatim from the source (the resume text is the `state`; add `"state": ...` and `"model": "jev-latest"` for a complete request, see [[reference/http-api]]):

```json title="questions"
{
  "python_depth": {
    "type": "score",
    "instructions": "How much depth of python experience does this candidate have, based on the supplied resume?",
    "criteria": [
      "No Python experience mentioned",
      "Mentioned but no detail",
      "Used in projects, some specifics",
      "Primary language, multiple projects",
      "Deep expertise: architecture, performance, libraries"
    ]
  },
  "team_leadership": {
    "type": "score",
    "instructions": "How much experience does this candidate have managing or leading engineering teams?",
    "criteria": [
      "No management experience mentioned",
      "Informal mentorship or tech lead role",
      "Led a small team or project",
      "Managed a team with direct reports",
      "Managed multiple teams or an engineering org"
    ]
  },
  "system_design": {
    "type": "score",
    "instructions": "How much experience does this candidate have designing large-scale or distributed systems?",
    "criteria": [
      "No architecture work mentioned",
      "Contributed to design discussions",
      "Designed components of a larger system",
      "Owned architecture of a significant system",
      "Designed systems at scale across multiple domains"
    ]
  },
  "generalist": {
    "type": "score",
    "instructions": "How much evidence is there that this candidate picks up unfamiliar tools, roles, or domains outside their core specialty?",
    "criteria": [
      "Only one domain or role mentioned",
      "Some variety but within a narrow field",
      "Worked across a few different areas or tech stacks",
      "Regularly moved between domains, wore many hats",
      "Track record of ramping up in unfamiliar areas and delivering"
    ]
  }
}
```

Each rubric has five levels, so raw `score` values run 0–4. Every level is a concrete, standalone description rather than a bare adjective — that is what makes the levels comparable across candidates. In SDK v0.6.0 `Score.criteria` is an ordered sequence, not an int-keyed dict; see [[concepts/score]].

### Step 2: combine with weights

> Each dimension is normalized to 0–1 and weighted. The weights give you an easy way to adjust the relative importance of each dimension, without losing any of the nuance of the individual scores.

```python title="scoring.py"
py      = response.answers["python_depth"].score / 4
lead    = response.answers["team_leadership"].score / 4
arch    = response.answers["system_design"].score / 4
general = response.answers["generalist"].score / 4

# Senior IC
ic_score = (0.40 * py) + (0.10 * lead) + (0.40 * arch) + (0.10 * general)

# Engineering Manager
em_score = (0.15 * py) + (0.40 * lead) + (0.20 * arch) + (0.25 * general)
```

The divisor `4` is `len(criteria) - 1` for these five-level rubrics — the maximum attainable `score`. Both weight vectors sum to 1.00, so each composite also lands in 0–1.

Why this beats one big question, per the source:

> This gives you the ability to rank the candidates based on the composite score. But more importantly, it gives you visibility into how exactly the final score is being calculated. If the highest ranking candidates are not matching your expectations, you can adjust the weights to find the right balance.

And, critically, you re-rank without re-calling the model. The agent skill: "Changing a weight or display filter need not rerun inference when evidence and question meanings are unchanged."

## When it fails

- **The rule is not compensating.** Weighted sums let a high score on one dimension offset a low one. The agent skill draws the line: "Weighted scores suit compensating preferences; an 'any serious violation' rule needs separate conditions." Hard disqualifiers belong in `if` statements, not coefficients.
- **Rubrics of different lengths compared without normalizing.** A raw `score` of 2 means something different on a 3-level and a 5-level rubric. Normalize by `len(criteria) - 1` before combining or comparing, as the example does.
- **Dimensions that are not independent.** Decomposing too far destroys the thing being judged. The skill: "Split independently useful dimensions, without destroying the relationship being judged."
- **`score` is an expected value, not a level.** It "can land between levels" (see [[reference/http-api]]), so `score > 1.5` is a real threshold, not a rounding error. Do not `int()` it away.
- **Ranking across items needs comparable questions.** The skill: "use comparable per-item Scores for graded ranking" — identical rubrics per item, or the composites are not comparable.
- **Ignoring `confidence` on the dimensions.** Score answers carry `confidence`; a composite silently averages away per-dimension uncertainty. Gate separately if the stakes warrant it — see [[patterns/confidence-routing]].

## Variants

- **Multiple weight vectors over one evaluation.** The example's `ic_score` / `em_score` — one call, two rankings.
- **User-tunable weights.** The agent skill's "Turn judgments into reusable data": "Score dimensions once, then let code or user controls change weights, thresholds, rankings, and views."
- **Scores as ML features.** Same source: "With labeled outcomes, those signals can become classical ML features." See [[cookbooks/autoresearch-feature-discovery]].
- **Composite scoring for reranking retrieved candidates.** See [[cookbooks/rerank]].
- **Mixing Noul gates with the weighted sum.** Use a Noul for the disqualifying condition and the weighted composite for the rest (inferred from the "any serious violation" guidance above).

## Related

- [[concepts/score]] — rubric rules, `legend`, `probabilities`, `confidence`
- [[patterns/overview]] — the four-pattern catalog
- [[guides/writing-instructions-and-criteria]] — writing levels that stand on their own
- [[patterns/confidence-routing]] — gating the dimensions you combine
- [[cookbooks/rerank]] — ranking retrieved candidates
- [[cookbooks/autoresearch-feature-discovery]] — judgments as features

## Sources

- raw/docs/patterns__composite-scoring.md (https://docs.typesafe.ai/patterns/composite-scoring)
- raw/docs/patterns.md (https://docs.typesafe.ai/patterns)
- raw/github/skills/skills/typesafe-ai/SKILL.md (https://github.com/typesafe-ai/skills/blob/main/skills/typesafe-ai/SKILL.md)