Blog

Run THX-01 as a local decision model

Run THX-01 through Ardana's CLI and HTTP API. See how its encoder answers typed questions and where its token and option limits apply.

A compact circuit module and a larger processor connect to the same output lights.

A small encoder can answer typed questions

I want a support-ticket handler to choose a department and tell me whether the customer is asking for a refund. Those are decisions with defined answer shapes. In Ardana, I can ask for a choice among named alternatives or a probability of yes.

The model I can use for that step is HAL-X AI’s THX-01. It is a non-autoregressive model based on mmBERT-base. The upstream model card lists 322M parameters: a 307M encoder and a 15M decision head.

For ordered answers, we can ask for a score across 2 to 10 levels. A choice question can have 2 to 24 alternatives with THX-01 in Ardana. I define the question and its answer shape together, so the calling code has a concrete contract.

Pull THX-01 and ask a question

I used Ardana 0.3.2 for this run. If we need to install Ardana first, the macOS and Linux command is below. Windows has a PowerShell installer as well.

curl -fsSL https://ardana.ai/install.sh | sh
irm https://ardana.ai/install.ps1 | iex

With Ardana installed, I pull THX-01 directly from Hugging Face. The verified download was 643.8 MB and needed no conversion step. I did not install a Python package or pass custom tokenizer or layout flags.

ardana pull hf.co/doofz/THX-01

The pull installs the local alias thx-01 by lowercasing the Hugging Face repository name. I can inspect the installed model with this command. It reports the Candle runtime and THX-01 profile. Ardana loads the model through its native safetensors path, so there is no GGUF conversion between the download and the question.

ardana show thx-01

Now I can pass a support ticket and all three questions to the installed model. The command asks for a department, whether the customer requested a refund, and how many times the card was charged. I ran this exact command successfully:

ardana run thx-01 "My card was charged twice for order A-104. Please refund the duplicate." --choice "Which department should handle this?" billing technical sales --noul "Does the customer ask for a refund?" --score "How many times was the card charged for the order?" Never Once Twice "Three times or more"

The inline form assigns an identifier to each question in the order I supplied it. For this ticket, the identifiers below correspond to the department, refund, and charge-count questions. I can use that order to follow the answers without naming each question in a request file.

q1
q2
q3

The encoder scores each question separately

For each question, we build a sequence containing the question type, instructions, options, and state. Each option gets a mask position immediately before its text. The state follows the options, with separators marking the boundaries. We repeat this layout for every question in the request:

[CLS] <type> question: <instructions> [SEP] [MASK] <option> [MASK] <option> … [SEP] <state> [SEP]

Inside the model, we have the encoder, a question-type embedding, and two transformer head layers. The mask positions mark where the model scores the options. A marker scorer produces one logit per option. That gives us a set of option scores without generating answer tokens one after another.

In Ardana, I run each question sequence separately in F32. We then apply temperature-scaled softmax to turn its logits into probabilities. The upstream Python implementation batches its question sequences, so its execution differs from ours here. Sending several questions in one request still means separate sequence runs in Ardana.

We run one sequence per question, even when several questions share the same state in one request.

Token accounting follows those runs. We sum the lengths of all question sequences for the reported input token count, including the state each time it appears. Adding a question therefore adds another sequence containing that same state. The verified response reports zero output tokens because we score options instead of generating text.

The whole question must fit the window

We give each question a 1,024-token window for its complete sequence. The question, options, state, and separators all use that budget. A state that fills 1,024 tokens leaves no room for the rest. Ardana rejects an overlong state instead of trimming it to fit.

I keep instructions and option labels short because those parts can be shortened during sequence construction. They have a separate head budget of 256 tokens, which can expand for options up to half the window. Each option initially keeps at most 48 tokens. Put the distinction between alternatives into a few clear words.

Our THX integration accepts 2 to 24 choice options, even though the general API allows more. Score questions accept 2 to 10 ordered levels. In our HTTP checks, 25 choice options received a 422 response naming the 24-option limit. A sequence requiring 6032 tokens received a 413 response with this error code:

"request_too_large"

We also have a narrower feature set than the upstream package. Upstream supports extraction and a two-round tournament for more than 24 options. Ardana does not implement THX’s native numeric or excerpt extraction, action output, or tournament handling. I would keep the alternatives within the supported limit before sending the request; a larger choice set will not trigger an automatic tournament.

Send the same ticket through HTTP

I save the complete request below as request.json. It keeps the same ticket and all three questions, with explicit names for the answers. Those names also survive when I pass the file to the CLI, so I can use the same request in both places.

{
	"model": "thx-01",
	"state": "My card was charged twice for order A-104. Please refund the duplicate.",
	"questions": {
		"department": {
			"type": "choice",
			"instructions": "Which department should handle this?",
			"criteria": ["billing", "technical", "sales"]
		},
		"refund": {
			"type": "noul",
			"instructions": "Does the customer ask for a refund?"
		},
		"charges": {
			"type": "score",
			"instructions": "How many times was the card charged for the order?",
			"criteria": ["Never", "Once", "Twice", "Three times or more"]
		}
	}
}

For HTTP, I start Ardana’s server with the command below. The default address is 127.0.0.1:8000. I verified this on an isolated loopback port. The commands here use the default address so they work together as shown.

ardana serve

In another terminal, I send the saved file with this request. Its body contains the model, ticket, and questions, so I do not repeat question arguments in the command. The response excerpt below comes from this run.

curl http://127.0.0.1:8000/v1/systemone -H 'content-type: application/json' --data-binary @request.json
{
	"answers": {
		"department": {
			"choice": "billing",
			"confidence": 0.77,
			"x_p_max": 0.8467
		},
		"refund": {
			"type": "noul",
			"noul": 0.9883
		},
		"charges": {
			"score": 1.9,
			"confidence": 0.6858,
			"legend": {
				"0": "Never",
				"1": "Once",
				"2": "Twice",
				"3": "Three times or more"
			}
		}
	},
	"usage": {
		"input_tokens": 152,
		"output_tokens": 0
	}
}

I read the charge score as the expected level, rounded to two decimal places, rather than a selected level’s index. The legend gives those levels their meaning. Department confidence differs from the maximum probability shown beside it. The refund answer directly gives the probability of yes. Input usage sums all question sequences, including the repeated ticket text.

I also ran the saved request through the CLI with the command below to compare the two paths. The parsed HTTP response exactly matched the CLI response for this request in this run. That tells me the two paths agreed for this ticket. It does not guarantee identical results for every request across runs.

ardana run thx-01 --request request.json --json

Try it on tickets you have labelled

I would start with tickets you have already labelled for department, refund intent, and charge count. Keep those labels outside the request and compare them with each answer. Check disagreements by question so a correct department does not hide a missed refund. Before choosing a routing threshold, check whether the department probabilities match how often those answers are correct on your tickets.

Our example verified that the request executes and that CLI and HTTP responses match for this run. It did not measure accuracy or calibration. I would keep the upstream model card alongside that evaluation when deciding what to use. Replace the ticket text in the saved request, keep the explicit question IDs for comparisons, and try the same command:

ardana run thx-01 --request request.json --json