Jev AI Hierarchical Classification: Beam Search Over Choice Probabilities
TL;DR: Jev AI hierarchical classification is not one question answered over a messy list. It is a walk down a tree: at each node one
Choiceover that node’s direct children, and the probabilities that come back become the edge probabilities of the tree. Take the highest-probability child every level and you have greedy search — cheap, and unable to recover from an early mistake. Keep the bestKpaths instead and you have beam search, which still costs one round of calls per level because the questions in a round run in parallel. TypeSafe’s cookbook measures both on three public taxonomies and reports the difference in one line: beam search matched 3 of 3 expected leaves, greedy matched 2 of 3. This page walks that cookbook, including the scoring formula, the code, and the point where the pattern stops being worth it.
Where these facts come from
Everything below was checked on 2026-11-17 against TypeSafe’s own pages and the public examples repository:
- Hierarchical classification cookbook — the three hierarchies, the greedy and beam implementations, the scoring formula, and the printed results table. The runnable notebook is at
github.com/typesafe-ai/typesafe-public-examplesundercookbooks/hierarchical_classification, with the data licences in that folder’sDATA_LICENSE.md. - Choice primitive — a Choice returns a selected option, a probability for every option, and a confidence; a Choice accepts up to 255 options; and the same page points at this cookbook for deep hierarchies.
- Models page — model ID
jev-1.14.0, the aliasesjev-latestandjev-previewboth resolving to it, and pricing at $42 per billion tokens / $0.042 per million tokens.
The cookbook states plainly that its numbers came from jev-1.14.0 on 2026-11-03. That is a dated measurement, not a general benchmark, and the sample is three documents — one per hierarchy. It is enough to explain the mechanism. It is not evidence of accuracy on your taxonomy.
Why hierarchies are a different problem from classification
The cookbook opens with a list rather than a definition: “Many kinds of data are hierarchies: taxonomies, file systems, websites, codebases, org charts, biological ontologies, LLM skills and moderation policies.”
That list matters, because it locates the problem somewhere other than “I have a lot of labels.” A flat classification with 30 labels is one Choice with 30 options, and the Choice page’s advice applies directly: hand the model the full list, not a shortlist, because options cost a few tokens each and recall beats cleverness. A hierarchy is not that. The candidate set at the root is small, the correct answer is a leaf six or eight levels down, and the sibling options at that leaf are separated by distinctions that only make sense once you know where you are.
The pattern the cookbook builds is: start at the root, ask one Choice over the root’s children, follow the most probable child, repeat until you reach a leaf. That leaf is the final classification. That is greedy search, and it is the obvious thing to do — which is exactly why the page spends most of its length showing where it breaks.
A second thing falls out of asking one question per node, and the cookbook calls it out as a pair of benefits rather than a side effect:
- Observability. You can identify which nodes your misclassifications occur most in, and you can measure how many times each node and edge is traversed.
- Testability. Hierarchy updates can be unit tested and their impact on classification performance measured, rather than shipped and hoped for.
Both benefits get easier because the tree is explicit in your code. If you classified the whole document against a flattened list of leaves, you would have a single answer and no per-node evidence to look at.
Greedy search, spelled out
The method is described in one sentence: “choose the highest-probability child and discard every alternative. One early mistake cannot be recovered.”
In the code, greedy search is a loop bounded by MAX_DEPTH, and it records every distribution it receives so the traversal diagram can label each edge:
def greedy_search(hierarchy: Hierarchy) -> dict:
"""Follow only the locally highest-probability child."""
path, probability_product, decision_count, records = (), 1.0, 0, []
for _ in range(MAX_DEPTH):
labels = tuple(subtree(hierarchy.tree, path))
if not labels:
break
probabilities = choose(hierarchy.document, child_question(labels), model=TYPESAFE_MODEL)
records.append(choice_record(path, probabilities))
label = max(probabilities, key=probabilities.get)
if len(probabilities) > 1:
probability_product *= max(probabilities[label], EPSILON)
decision_count += 1
path += (label,)
score: float = (
probability_product ** (1 / decision_count) if decision_count else 1.0
)
return {"path": path, "score": score, "records": records}
Two details in that loop carry more weight than they look like they do.
A node with a single child is not a decision. In choose, the guard is if len(keys) == 1: return {label: 1.0 for label in keys.values()} — one option short-circuits to certainty with no request at all. In extend_candidate the same idea appears as is_decision: bool = len(probabilities) > 1, and decision_count only increments when it is true. Deep taxonomies are full of chains where a concept has exactly one child; if you counted those as decisions, a deep leaf would be scored as if it had answered more questions than a shallow one, when in fact it answered no more than one.
Every probability is floored at EPSILON = 1e-9. max(probabilities[label], EPSILON) keeps a product from collapsing to zero, which would make the geometric mean zero and destroy any comparison between paths. The floor is not a modelling statement about how unlikely an option is; it is there so an arithmetic operation stays usable.
Where greedy search actually fails
The cookbook’s failure case is worth quoting in full, because the shape of the mistake is the lesson. Three labelled examples are run through both strategies. The retail one is a Shopify listing:
“Furniture listing: a wall-mounted window shelf bed. This padded floating shelf uses suction cups and a washable cushion as a sunny perch for one cat.”
The expected leaf is Cat Window Beds & Perches. Greedy search lands on Pet Chairs.
Nothing in the document is wrong, and no answer in the chain is arbitrary. The listing says furniture, says wall-mounted, says padded shelf, says perch. Chair is a plausible sibling of bed at some level of the tree, and once greedy has committed, no later answer gets a vote. That is the entire content of the cookbook’s line about early mistakes: the mistake is not at the leaf, where the information would have resolved it, but at a level above the leaf, where the information was thinner.
The patent example does not fail — a freestanding structural wooden perch for poultry or pet birds resolves to A01K31/12 Perches for poultry or birds, e.g. roosts under greedy — and neither does the biomedical one, where a Crohn disease abstract with transmural ileocolitis, skip lesions and infliximab lands on C06.405.469.432.500 Crohn Disease. Two correct out of three under greedy is the finding, and it is a finding about this sample of three, not a rate.
Beam search: keep K paths, still one round per level
The fix is one sentence: “retain K plausible paths and classify every frontier in parallel. When an early choice is close, a clearer decision further down can put the right path back on top. The leaf of the path with the highest geometric-mean probability is the final classification.”
The reason this is affordable is on the Choice page and repeated in the cookbook: System One questions can run in parallel. “The beam’s K questions at each level run in parallel, so beam search takes one round of calls per level, as greedy search does.”
That sentence is the whole cost argument, and it is worth reading precisely. Beam search does not add latency rounds. It adds requests inside a round, and therefore tokens. In the cookbook’s own implementation the concurrency is explicit — a ThreadPoolExecutor(max_workers=BEAM_WIDTH) mapping choose over the current frontier, one Choice per expandable candidate:
for _ in range(MAX_DEPTH):
expandable = [
candidate
for candidate in beam
if subtree(hierarchy.tree, candidate["path"])
]
finished = [
candidate
for candidate in beam
if not subtree(hierarchy.tree, candidate["path"])
]
if not expandable:
break
with ThreadPoolExecutor(max_workers=BEAM_WIDTH) as executor:
distributions = list(
executor.map(
lambda candidate: choose(
hierarchy.document,
child_question(tuple(subtree(hierarchy.tree, candidate["path"]))),
model=TYPESAFE_MODEL,
),
expandable,
)
)
Then every child of every expandable candidate is turned into a new candidate, and the beam is re-sorted:
beam = sorted(
finished + expanded,
key=lambda candidate: candidate["score"],
reverse=True,
)[:BEAM_WIDTH]
finished is the part worth noticing. A path that has reached a leaf has no children, so it stops being queried but stays in the ranking. That is how a short path competes with a long one on equal terms — the alternatives are not “deeper is better,” they are the geometric means.
The scoring formula, and why it is a geometric mean
The cookbook gives the formula twice, once in prose and once in code, and the prose version is the one to memoize:
path_score = product(edge_probabilities) ** (1 / decisions)
with a caveat attached: “for hierarchies deeper than about 10 levels, compute it as exp(mean(log(probs))) to avoid floating-point precision loss.” The code carries the caveat as a comment rather than an implementation — “Use log space for very deep trees to avoid floating-point precision loss” — and extend_candidate computes the power directly. If your taxonomy runs past ten levels, that comment is the queue to switch.
Why geometric mean instead of the raw product: “The geometric mean, unlike the raw product, doesn’t penalize a deep leaf just for taking more decisions.” A raw product of eight probabilities is smaller than a product of four, even if all eight are high. Comparing a six-level path to a three-level path on raw products would be comparing path lengths. The exponent 1 / decisions normalizes that away.
The second number the cookbook reports is the separation ratio:
separation = top_path_score / second_path_score
“reported only, not used for pruning. Near 1×, the top two paths are almost tied; the further above 1×, the clearer the winner.” In compare_searches it is computed as top_score / max(second_score, EPSILON), and the printed column is formatted as f"{result['separation_ratio']:.2f}×".
Separation being report-only is a design choice you may want to revisit in your own code. It is the number that tells you whether the beam’s top pick was decisive — a 1.01× separation means the winner was a coin flip between two paths, and a threshold on separation is a natural trigger for escalating to a human or to a second opinion. The cookbook does not do that; it prints the number and leaves the decision to you. If you do escalate on it, remember that it is a ratio between paths that survived pruning, so a beam of width 1 has no second path and no separation at all.
What the run produced
The result is stated once, then tabulated:
Beam search matched 3 of 3 expected leaves; greedy search matched 2 of 3. Keeping three paths recovered the expected classification for Shopify products.
| Hierarchy | Expected leaf | Greedy leaf | Beam K=3 leaf | Greedy correct | Beam correct |
|---|---|---|---|---|---|
| CPC patents | A01K31/12 Perches for poultry or birds, e.g. roosts | A01K31/12 Perches for poultry or birds, e.g. roosts | A01K31/12 Perches for poultry or birds, e.g. roosts | yes | yes |
| Shopify products | Cat Window Beds & Perches | Pet Chairs | Cat Window Beds & Perches | no | yes |
| MeSH biomedical subjects | C06.405.469.432.500 Crohn Disease | C06.405.469.432.500 Crohn Disease | C06.405.469.432.500 Crohn Disease | yes | yes |
Read the table for what it is. Three documents, three hierarchies, one disagreement. The useful claim it supports is mechanical — a width-three beam can carry a path that greedy already dropped — and the useful claim it does not support is anything resembling an accuracy figure. The cookbook publishes no node counts per hierarchy and no aggregate precision or recall, and its own table has no column for either. Treat 3-of-3 as a demonstration, not a baseline.
The second half of the cookbook’s output is a diagram per hierarchy: one static SVG each, rendering the nodes both strategies queried, with edge labels on the top edges (EDGE_TOP_K = 5 in the layout constants) and the two paths marked. For CPC and MeSH, greedy and beam agree, so the diagram is a single route. For Shopify it shows the dropped branch — which is the picture you would otherwise have to reconstruct from a log.
The three hierarchies, and the MeSH wrinkle
- CPC 2026.05 — patent subject matter, “from broad technology sections to narrow inventions. Published by the European Patent Office and the US Patent and Trademark Office.” The loader walks the scheme XML’s
classification-itemelements, keeps the first parent seen for each classification symbol, and builds roots from the symbols whoselevelattribute is 2, labelling each node"{symbol} {title}". - Shopify 2026-02 — “retail product categories, from store departments to specific product types. From Shopify’s Product Taxonomy (MIT).” Loaded from
categories.txt, split on" : "for the id and then on" > "for the path. - MeSH 2026 — “biomedical subjects from broad domains to specific conditions. MeSH is a DAG, so one descriptor can appear under several parents; we give each of its official tree numbers a separate path. Source: U.S. National Library of Medicine. This is the January 2026 build of MeSH 2026 and may not be the most current MeSH data.”
The MeSH case is the one that generalizes. Most biomedical ontologies, and plenty of enterprise ones, are graphs dressed as trees. The cookbook’s answer is not to build a graph search — it is to enumerate the paths, which converts the DAG into a tree by construction:
for tree_number in sorted(names_by_tree_number):
parts: list[str] = tree_number.split(".")
prefixes: list[str] = [
".".join(parts[:index]) for index in range(1, len(parts) + 1)
]
category_code: str = tree_number[0]
category_path: tuple[str, ...] = (
f"{category_code} {MESH_CATEGORIES[category_code]}",
*(f"{prefix} {names_by_tree_number[prefix]}" for prefix in prefixes),
)
_insert(tree, category_path)
The first level is the category letter mapped through a sixteen-entry table — A Anatomy, B Organisms, C Diseases, and so on through V Publication Characteristics and Z Geographicals. Every dotted prefix above the tree number becomes an intermediate level. One descriptor with three tree numbers becomes three leaves in three places, and the classifier can reach it from any of them.
Since NLM updates its current MeSH file in place, the cookbook hosts a pinned copy in the examples repository rather than linking to NLM — with the note that the build is January 2026 and may lag.
Setup, cache semantics, and what the numbers cost to reproduce
The documented requirements are short: Python 3.10 or later, and
pip install typesafe-sdk ipython 'cooksafe>=0.3.0,<0.4.0'
The code replays saved answers, “so it runs as published without an API key.” Set TYPESAFE_API_KEY to change a request; in Colab the notebook copies keys from the Secrets panel, guarded by a try: from google.colab import userdata that degrades to userdata = None outside Colab. The client is constructed with api_key=os.environ.get("TYPESAFE_API_KEY", "cache-only") and a RetryPolicy(max_retries=5, backoff_initial=1.0, backoff_max=20.0).
The first run “downloads the three hierarchies (about 28 MB) into datasets/.” Downloads are pinned URLs written through a .tmp file and then replaced, so a partial download never becomes the cache.
The answers have their own cache, and its semantics are worth copying into your own pipelines:
CACHE = Path("hierarchical_classification_cache.json"), decorated with@json_cachefromcooksafe.DOWNLOAD_CACHE = Truefills the cache from the published answers, “so the run makes no requests.”- To run live, “set it to
Falseand deleteCACHE; aCACHEalready on disk is never overwritten.”
That last clause is the important one. It means a run that half-succeeded cannot be silently mixed with a run that replayed — you delete the file or you do not, and the file’s existence decides which mode you were in.
Constants are all in one block, which is the practical answer to tuning:
TYPESAFE_MODEL = "jev-1.14.0" # the page's model; "jev-latest" runs the newest, live
BEAM_WIDTH, MAX_DEPTH, EPSILON = 3, 12, 1e-9
MAX_DEPTH = 12 is the loop bound for both strategies, BEAM_WIDTH = 3 is both the beam size and the thread-pool size, and EPSILON is the probability floor described above.
Token cost: the honest arithmetic
The latency claim is stated in the cookbook: one round of calls per level, same as greedy. The token arithmetic is not stated, and it is worth doing because it is where beam search actually spends money.
Per the models page, a request carries a state plus questions, and Jev “ingests the state once and evaluates every question against it in parallel,” with a per-request context limit covering “the state plus all questions combined.” In this cookbook, each choose call sends the document as the state with exactly one Choice question. So at each level, a width-K beam sends K requests, each re-sending the same document as state. That is K times the state tokens per level compared with greedy, on top of the extra questions. Batching the frontier’s K questions into one request with K questions and one state would cut that — the same state, more questions, one ingestion — but it would also change what caching does per candidate, which is presumably why the cookbook keeps one question per call.
Pricing, for scale: $42 per billion tokens and $0.042 per million tokens for jev-1.14.0, unchanged from jev-1.13.0. A document of a few thousand tokens at width 3 over eight levels is a token bill in the fractions of a cent per document, not dollars. The reason to reason about it is drift: width and depth multiply, and a taxonomy that needs BEAM_WIDTH = 8 over twelve levels is 8× a run you already thought was cheap.
When not to do this
Three cases where the walk is the wrong shape:
- The children set is small enough to ask in one shot. A
Choicetakes up to 255 options, and the Choice page’s advice is to give the full list rather than a shortlist. If your entire taxonomy fits under 255 leaves, flatten it and ask once — you get one distribution over the real answer set instead of a chain of local decisions, and you delete both the depth bound and the beam width. - You already have a confidence-based routing decision. If the question is “is this urgent, and should it go to a human,” that is a
Nouland a threshold — see confidence and fallback — not a tree walk. Trees are for taxonomy; thresholds are for triage. - Sibling distinctions are not separable from the labels. The walk works because a
Choicequestion at a node has good options. If your node labels are internal codes nobody would recognize — team ids, SKU prefixes, opaque help-desk categories — the only information the model gets is the code string. The Choice page’s advice applies at every level: the option names and descriptions are what do the separating work.
A fourth, softer case: when the leaf must be audited. Each level is an independent judgement with its own distribution, and the path score is a product of them. That is a lot of surface to defend if the classification has regulatory weight. Greedy produces a shorter audit trail; a single flat Choice produces the shortest.
Where this fits next to the rest of the toolkit
The pieces this page leans on are all on this site. The three primitives cover what a Choice, a Noul and a generative answer are for; state and questions covers how a request is shaped and why the state is ingested once; request patterns covers the fan-out and routing shapes this beam is a special case of; and structured extraction covers the other direction — pulling values out rather than choosing among them.
Two adjacent scenarios: support routing is the routing problem a two-level taxonomy would solve, and RAG reranking is the sibling problem of ranking candidates rather than walking a tree. The ticket triage case and the product FAQ scoring case show the request and response shapes at the level of a single Choice.
If you want the other structure the cookbook family builds on the same primitive — turning a typed function’s arguments into questions — that is Jev AI function calling. And if a walk goes wrong in a way that looks like the model, not your tree, the failure modes page lists the documented jagged edges before you start rewriting prompts.
FAQ
Is this the same as asking one Choice with every leaf as an option?
No, and the difference is where the information comes from. A flat Choice over all leaves asks the model to hold the whole taxonomy at once; a walk asks a small question at each node and uses the answer to choose the next question. The cookbook’s justification for the walk is composite: the candidate set at each step stays small, and the traversal is observable node by node. If your taxonomy fits in 255 options and the leaves are mutually distinguishable, the flat version is simpler and gives you one distribution over the answers you actually care about.
Why keep three paths and not two or eight?
The cookbook pins BEAM_WIDTH = 3 and reports that width 3 recovered the Shopify case, which is the only evidence in the page. There is no sweep over widths, so there is no measured answer to “what should K be.” The mechanism suggests the trade: wider means more chances to keep the correct branch alive and linearly more state tokens per level, and past a point the extra paths are ones you would never choose anyway.
Why is confidence not used as the path score?
Because a path score multiplies edges and a per-question confidence is computed from how one distribution is spread — two different quantities on the same 0-1 scale. The cookbook uses probabilities, takes the geometric mean across decisions, and reports separation between the top two paths. If you want the other view, the answer’s confidence per node is available in the same response and is the natural thing to log for observability, which is exactly the use the cookbook makes of the recorded distributions.
What happens at the depth bound?
MAX_DEPTH = 12 stops both loops. Beam search breaks out when nothing is expandable — that is, when every candidate in the beam is a leaf — and otherwise stops after twelve rounds. A taxonomy deeper than twelve levels would be truncated, and the code has no error for that; the caveat about computing scores in log space above ten levels is the cookbook’s own signal that it expects shallow trees.
Do I have to use these three taxonomies?
No. The loader functions are specific to CPC XML, Shopify’s categories.txt and MeSH descriptor XML, but the walk itself only needs a Tree — nested dicts of labels — plus a document string and an expected leaf. Any hierarchy you can express as nested dicts goes through beam_search and greedy_search unchanged. The one requirement the MeSH case demonstrates is that graphs must be expanded into paths first.
Does this replace reranking or thresholding? No. It classifies a document into a taxonomy position. Reranking orders candidates by relevance; thresholding turns one probability into a yes/no with a floor. The cookbook’s separation ratio is the only place the two ideas touch, and there the cookbook deliberately stops at reporting.