Version classifier labels and their descriptions

A label name is part of the classifier input

When I rename a model-facing label, I change the classifier input. Splitting one label into two also changes the choices available for the same case. Even if our application still sends the result to the same handler, the model receives a different question. I treat the names and their meanings as a contract worth versioning.

In Ardana v0.3.2, I can attach a description to each option. The renderer places that description beside its name in the prompt. An inclusion or exclusion written there remains text for the model to interpret; it does not become an executable rule. The returned selection and probability entries use the supplied names, so those names also reach the code consuming the answer.

The September 22, 2026 preprint by Sun and Xu tests what happens when descriptions swap between fixed option names. Its experiments use two encoder checkpoints and hosted Jev in English. Ardana is outside that evaluation, so I cannot turn it into an accuracy claim for our model. We need to define what each option includes and excludes.

Write inclusion and exclusion boundaries

I start with the case where two routes look correct. In our hypothetical example, a customer reports a crash and asks for their money back. I assign that ticket to billing because the refund request takes precedence over the product failure. A crash without a refund request or charge issue belongs in technical support under this rubric.

We specify inclusion, exclusion, and precedence. Billing includes refunds for product failures; technical excludes refund requests. Save our hypothetical request as request.json:

{
	"model": "decider:2b",
	"state": {
		"ticket": "The app crashes. Please refund my subscription."
	},
	"questions": {
		"route": {
			"type": "choice",
			"instructions": "Choose exactly one route for this support ticket. Refund requests take precedence over charge issues, then product failures, then pre-purchase questions. Use other for unsupported or unclear requests.",
			"criteria": {
				"billing": "Charges, invoices, payment methods, or explicit requests to return money already paid; includes refunds requested because of a product failure; excludes pre-purchase price questions.",
				"technical": "Product errors or access failures without a refund request or charge issue; excludes pre-purchase questions.",
				"sales": "Pre-purchase questions about price or features, without an existing charge issue, refund request, or product failure.",
				"other": "Requests outside the listed routes, or too little information to identify a route; excludes a clear match to another route."
			}
		}
	}
}

Our application contract stores the revision, stable application IDs, ordered names, descriptions, and shared-task mappings separately. I use those IDs to keep dispatch identity explicit alongside the text we send to the model. The request contains the model, ticket state, and question with its instructions and descriptions. The API request type declares no taxonomy-version or stable-label-ID field.

We also keep expected labels in separate fixtures. I expect billing for our hypothetical case, but I still need to check that against a model answer. The response does not echo our descriptions or contract revision, so I keep that context with the request record. We still have to test whether the model follows the boundaries written in the prompt.

Record renames and label splits separately

I keep a migration ledger beside the contract so each revision says what changed and where existing cases go. A UI-only rename gets an entry saying the model-facing text is unchanged. We can change the name people see without changing the option submitted to the model.

A model-facing rename gets its own entry. In our proposed rename control, “technical” becomes “product_support” while the application ID stays “TECHNICAL” and the description stays the same. Ardana returns supplied option names, so I update the mapping from the new name to that stable dispatch ID. We test this rename separately from a label split.

Keeping a name also does not freeze its meaning. When I revise a description, I record a contract revision and review the affected expected labels in our evaluation cases. The ledger states which boundary moved and which cases need a new decision. I do not carry expected labels forward just because the option name survived.

When we add a category, I define its scope within the existing division of cases and revisit the neighboring categories and any catch-all boundary. For a removal, I record whether affected cases get replacement routing, require review, or fall outside the revised scope. We keep historical records so earlier decisions remain attached to their original contract.

For our hypothetical split, the ledger records “billing” becoming the narrower categories “payments” and “refunds”, each with a new application ID. Both map back to the broader billing outcome for a declared coarse comparison. Our rename and split comparisons stay separate, with expected labels adjudicated for each revision.

Compare fresh answers on a shared task

I require fresh inference after candidate edits. The causal engine applies softmax across the question’s candidate logits, and descriptions enter the prompt. I cannot treat a renormalized old vector as a new answer. Our confidence calculation uses candidate count n and maximum probability p_max, so changing the count can change confidence even when the maximum stays fixed:

(n * p_max - 1) / (n - 1)

For our hypothetical billing split, I sum fresh payments and refunds probabilities to get a shared billing probability. This is justified by the declared disjoint split and their common billing mapping. The sum neither recovers the old vector nor establishes equal calibration. Removing a category without a justified shared mapping can prevent comparable scoring altogether.

Native accuracy answers whether we chose the right label for that revision. Coarse accuracy answers whether we chose the right shared outcome, which cannot establish whether payments and refunds were distinguished correctly. I also report two coarse decisions separately: mapping the highest-probability fine label, and choosing the highest-probability outcome after aggregation. Those operations can select different outcomes.

In my proposed comparison, I hold case IDs, state, model artifact, runtime, preprocessing, instructions, and unchanged-option order fixed. I adjudicate expected labels separately for each revision and test renaming separately from splitting. Alongside accuracy, I report shared-task probability scores, boundary transitions, failures, and unmappable cases against the original denominator.

Commit the contract beside the request

I keep the contract and the exact request together in version control. The contract records our revision, application IDs, ordered labels, descriptions, instructions, and shared-task mappings. Expected labels stay in separate fixtures, with gold decisions reviewed for each revision. The response does not echo the descriptions or contract revision, so I keep that association outside the response.

With the earlier request body saved as request.json, I use the documented CLI workflow. I pull the model, then run the saved request.

ardana pull decider:2b
ardana run decider:2b --request request.json --json

Alternatively, I can send the same request over HTTP. I start the server in one terminal and run the request command from another.

ardana serve

curl http://127.0.0.1:8000/v1/systemone -H 'content-type: application/json' -d @request.json

I tune descriptions on development cases and reserve held-out cases for the comparison. For that comparison, I keep case IDs, state, model artifact, runtime, preprocessing, instructions, and unchanged-option order fixed. Our report keeps native correctness, shared-task probability scores, and boundary transitions visible. Failures and unmappable cases stay in the original denominator.

For a first migration test, I would change one option name and keep the comparison controls fixed. I would run both requests on the same held-out cases and inspect the changed decisions before testing a split separately. Start with the commands above and the Ardana repository, then commit that rename comparison beside your contract.