How to handle requests outside your classifier's labels

The request may not belong to any label

Suppose I build a router for billing and technical support, and someone asks it to translate a poem. Neither handler supports that request, even though the request itself is clear. Sending it to either handler is a routing failure. I need an outcome that says the request falls outside what this system does.

CLINC’s authors define out-of-scope requests as requests belonging to none of a system’s supported intents. I keep that separate from ambiguity inside our supported work. A vague request for account help could need clarification before I can pick a handler. The poem request already tells me enough to know that neither handler applies.

Ardana normalizes probabilities over the supplied options and picks the option with the highest probability. We do not automatically insert an unknown category in that readout. I cannot treat those scores as a separate measurement of whether our application supports the request.

A valid label tells me which option won; it does not establish that any option fits. That boundary needs an explicit outcome we can test.

A none option still needs testing

I start with Rafe and Das’s decision-gate preprint. Submitted September 29, 2026, its v2 is dated October 6, 2026. The CLINC-150 condition tests 600 supported and 200 unsupported utterances. For Jev, a held-out threshold targeting 5% in-scope risk accepted 31.0% of unknown requests; adding an out-of-scope option lowered false acceptance to 19.5% under the same target.

Model choice mattered. Four open decision checkpoints, including decider-2b, selected the out-of-scope option on at most 16.5% of unsupported requests. The evaluation uses public benchmarks. We cannot check exposure for the hosted model and Laya because their training data are unreleased.

For an older test design, I look at Xu et al.’s Know-No study. It was submitted June 23, 2024; v3 is dated July 3, 2024. The benchmark removes correct labels from Banking77, reading-comprehension questions, and equation inference, then tests a none option, a none instruction, and no hint. Its experiments cover five historical model versions, including gpt-4o-2024-05-13, and average three shuffled-option runs. I would use that removal step to check whether the model recognizes that no valid answer remains.

Neither study measures this Ardana routing setup. I would keep the removed-label test as a separate check, with its expected outcome written down before running it. What should our none option mean for the services we support?

Write down the boundary between outcomes

I give each outcome a job and describe what it covers. In our software support example, billing covers charges, invoices, or refunds for the product subscription. Technical support covers errors, sign-in failures, or help using the product. Sales covers questions about buying the product or its available plans. I send supported requests to the corresponding handler.

Clarification needs its own action. I use it when a request may concern our product but does not establish which supported service is needed. The application should ask for the missing information. When a request clearly concerns another product or a service our team does not provide, I treat it as unsupported. I would have the application explain the support boundary.

Before adding more labels, I would review overlapping descriptions. Microsoft’s CLU guidance recommends inspecting confusion among known intents, considering merges, or adding examples that distinguish them. It separately recommends confusing negative examples for its None intent, such as book-purchase wording near a flight-booking intent. I would include both ambiguous supported requests and unsupported requests that share label vocabulary in our fixtures, with expected actions written down before testing.

Ardana lets us attach descriptions to choices, so these boundaries can accompany the labels. Our application still has to implement each action. Microsoft also documents that its built-in CLU evaluation does not apply its None threshold. For our implementation, I would test the policy the application runs and the action it takes after each choice.

Make the scope choices explicit in the request

I would put the boundary definitions in the request itself. Ardana accepts shared state and questions, and a choice question can carry a criteria map from names to descriptions. That gives me a place to define each service and the two fallback outcomes. The example below is synthetic input I have not run, and I make no claim about the model’s response.

{
	"model": "decider:2b",
	"state": "Please book a dentist appointment for tomorrow.",
	"questions": {
		"route": {
			"type": "choice",
			"instructions": "Route a request to support for this software product. Choose out_of_scope when the request is clearly outside the supported services. Choose needs_clarification when more information is needed to distinguish supported services.",
			"criteria": {
				"billing": "Charges, invoices, or refunds for this product subscription.",
				"technical": "Errors, sign-in failures, or help using this product.",
				"sales": "Questions about buying this product or its available plans.",
				"needs_clarification": "The request may concern this product, but does not establish which supported service is needed.",
				"out_of_scope": "The request clearly concerns a different product or a service this support team does not provide."
			}
		}
	}
}

For a local check, I would save this body as request.json. The server command below starts Ardana with its default address, 127.0.0.1:8000. I would leave that process running and send the request from another terminal.

ardana serve

This curl invocation sends the saved body to the server. It uses the request exactly as written, including the dentist appointment outside the described software support services. I have not run this example or measured its result.

curl http://127.0.0.1:8000/v1/systemone -H 'content-type: application/json' --data-binary @request.json

My application needs an action for every choice the classifier supplies. I would send billing, technical, and sales choices to their corresponding handlers. For out_of_scope, I would explain that the requested service is unavailable through this support team. For needs_clarification, I would ask for the missing information before choosing a handler.

I can also ask a separate yes/no scope question with noul. I would treat that answer as another model judgment, with its own behavior to test. The choice readout normalizes probabilities over the supplied options and selects the highest one. It does not insert an unknown outcome automatically, and its response fields do not measure a separate probability that the request belongs to our supported scope.

Count unknown requests that reach a handler

Before changing the rubric or routing policy, I would set aside the final test examples and leave them untouched. My proposed test has four slices: clear supported requests, ambiguous supported requests, clearly unsupported requests, and unsupported requests sharing label vocabulary. The fixture expectations are authored examples, not observed model predictions.

Using the paper’s denominators, I would measure false acceptance as unsupported requests routed divided by unsupported requests tested. Known-case coverage is answered supported requests divided by supported requests tested. Routing error among answered supported requests is incorrect routes divided by answered supported requests.

For this test, I would count a request as answered only when it reaches a supported handler. Clarification can then be the expected action for an ambiguous request while still reducing coverage. I would record supported requests rejected as unsupported separately. That keeps a correct clarification, an incorrect rejection, and a wrong handler assignment visible as different outcomes.

I would check the application’s final action against each fixture’s expectation. Ardana’s probabilities are normalized over the supplied options; they do not establish that the request belongs in scope. Next, run the authored fixtures through your local routing path, inspect every unsupported request that reaches a handler, and reserve fresh examples for the final test.