Blog

Build hierarchical classifiers with local decision models

Build a caller owned label hierarchy with local decision questions. Trace parent and child choices, validate paths and interpret branch probabilities.

A small brown groundhog stands beside a wooden track with two levels of branches.

A confident child can follow the wrong parent

If I send a technical support request down the billing branch, a confident child answer still gives me the wrong path. My next question offers only billing’s children. Greedy traversal has already discarded the correct child in another branch.

I start with a taxonomy: a label tree defined before classification. In the local classifier per parent approach, each parent has a multiclass decision over its children. We follow the selected parent downward. Here, local describes the decision’s place in the hierarchy; running inference on our own machine is a separate meaning.

Our example uses the same local model to choose one path through that tree. A confidence cascade escalates a decision to another model. Multilabel classification answers separate label questions and can return several labels. Those solve different problems from choosing a child under the parent we selected.

With Ardana, I put that traversal in application code. Ardana plans the questions before inference, so I need the parent answer before constructing the child request. For our billing and technical branches, that means two sequential calls. Smaller sibling sets change the questions; they do not establish better accuracy or latency.

Send a parent question before its child

I built a small support-v1 tree for this example. Its root offers billing, technical, and other. Billing splits into refund and invoice, while technical splits into login and integration. Other is already a leaf. The instructions classify the first stated request.

We own this tree in the application. I include the taxonomy version and current category path as ordinary metadata inside the request state. My caller decides which children to offer next.

For the root, I send the original ticket with an empty category path. This request contains one choice question. I use descriptions to spell out what belongs in each branch.

{
	"model": "decider:2b",
	"state": {
		"ticket": "Please refund the duplicate charge.",
		"taxonomy_version": "support-v1",
		"category_path": []
	},
	"questions": {
		"next": {
			"type": "choice",
			"instructions": "Classify the first stated request in ticket. Given membership in category_path, choose its direct child from criteria. At the root, use other for an uncovered request.",
			"criteria": {
				"billing": "Requests for a refund or an invoice.",
				"technical": "Requests about signing in or a broken integration.",
				"other": "The main request is outside those four supported subjects."
			}
		}
	}
}

My caller waits for the parent answer before constructing the child request. Ardana plans all question rows before inference, so placing both questions in one request would not create that dependency. If billing wins, I keep the ticket and append billing to the category path.

{
	"model": "decider:2b",
	"state": {
		"ticket": "Please refund the duplicate charge.",
		"taxonomy_version": "support-v1",
		"category_path": ["billing"]
	},
	"questions": {
		"next": {
			"type": "choice",
			"instructions": "Classify the first stated request in ticket. Given membership in category_path, choose its direct child from criteria. At the root, use other for an uncovered request.",
			"criteria": {
				"refund": "Return a payment, including a duplicate charge.",
				"invoice": "Obtain or correct an invoice; no refund is requested."
			}
		}
	}
}

For technical, I offer login and integration. Billing and technical paths each require two sequential calls. When the root selects other, my caller stops after one call because that node has no children.

Keep branch probabilities distinct from confidence

I use the selected option’s probability when estimating the full path. Ardana’s confidence score rescales the winning probability based on the number of options. It is a different quantity, so I keep it out of the multiplication. From each choice answer, I read the selected entry:

"probabilities"[selected_option]

With the input held fixed, the chain rule multiplies the parent probability by the child probability conditional on that parent. Our hypothetical parent probability of 0.6 and conditional child probability of 0.9 give a path estimate of 0.54. These are example numbers, not inference results. The 0.9 describes the child choice inside that branch. Substituting an independent label’s marginal probability would change the calculation.

I still have to justify treating a branch-restricted answer as a conditional probability. That interpretation is a modeling assumption. Multiplication gives us the path estimate under that assumption; it does not establish calibration of the final path. Returned choice probabilities are also rounded to four decimal places, so their sum can differ slightly from one.

Greedy routing can also discard the leaf with the higher probability. In another hypothetical example, I select billing at 0.6 over technical at 0.4. Billing’s best child, refund, has a conditional probability of 0.55, giving 0.6 × 0.55 = 0.33 for the path. The discarded technical/login path has 0.4 × 0.95 = 0.38. Choosing each local maximum does not guarantee the highest-probability leaf.

For beam ranking, TypeSafe’s cookbook uses a geometric mean of the edge probabilities to normalize the product by the number of decisions on the path. I keep this length-normalized ranking score separate from the chain-rule probability estimate because they have different meanings.

Measure the whole path as well as each branch

I would evaluate the complete route alongside each branch decision. Take a hypothetical set of 100 cases that require both a parent and a child decision, with 80 correct parents and 72 correct complete paths. Our parent accuracy is 80/100, with 20 parent errors. Among the 80 correctly routed cases, child accuracy is 72/80, or 90%, with eight child errors.

Those eight errors explain the drop from parent accuracy to complete-path accuracy. Our exact-path result is 72/100, or 72%, leaving 28 incorrect paths across the original set. The arithmetic is 0.8 × (72/80) = 0.72, and the second factor uses only cases whose parent we got right. I cannot replace it with accuracy from an unrelated child test set: every ancestor must be correct for the complete path to count.

Exact-path accuracy also loses information that I want when diagnosing mistakes. A correct parent followed by a wrong child has some correct ancestry, even though the final path fails. HiClass’s hierarchical precision and recall compare predicted and true labels after expanding both sets with their ancestors, giving credit for shared ancestry. I would report those metrics alongside exact paths so that partial success stays visible without counting it as a correct route.

For a separate diagnostic, I would supply the true parent and evaluate the child decision under that branch. This oracle-parent test tells us how the child performs when routing is supplied correctly. It answers a different question from child performance on the subset our parent decision routed correctly. I would keep both results separate from the complete-path result; neither establishes an end-to-end improvement for this example.

Start with a small explicit tree

I start with a tree small enough to inspect every route by hand. Before adding branches, I want to see which choices each request makes available and where the route stops. That gives us a manageable place to check the caller’s behavior while we work through real inputs.

For the local setup, I use Ardana’s documented default model. I run these commands before sending the example requests. They install Ardana, pull the model, and start the server, in that order.

curl -fsSL https://ardana.ai/install.sh | sh
ardana pull decider:2b
ardana serve --default-model decider:2b

Once the server is running, we can use the complete request body from earlier. Our example still needs two sequential calls for a path that includes a child decision. Running both calls locally does not turn the dependent hierarchy into one forward pass.

We have no measured inference accuracy or latency result for this tree. Restricting the choices changes the questions we ask, but it does not establish a performance improvement. To try it with your own inputs, get Ardana from the repository.