Enforce logical constraints on local model decisions

Valid answers can form an invalid decision

Our example policy cannot allow an urgent message to skip review and receive an automatic reply. Urgency requires review, and review excludes an automatic reply. If we select urgency and automatic reply but leave review unselected, we have broken that policy. These are rules we authored for the example.

Fastino’s GLiNER2.5 announcement puts implications, exclusions and label counts inside constrained decoding. Its current tutorial says the default relaxes constraints when no feasible assignment is found. We still have to check whether the returned assignment is feasible.

Ardana currently scores these questions separately, with no cross-question constraint field in its request. Our example adds no GLiNER2.5 decoder integration. I propose putting the validator in caller code, after label selection.

Before dispatch, we validate the complete assignment: our example selects labels with scores at or above 0.5, checks every declared rule, and sends contradictions for review. Passing those checks establishes rule satisfaction without proving that the labels are correct.

Ask for the signals the policy needs

To get the signals our policy needs, we ask separate questions about urgency, human review and automatic reply suitability. The two routing questions cover billing and technical support. I wrote this example but did not run it, so it contains no observed model results.

{
	"model": "decider:2b",
	"state": "My card was charged twice. Please fix this today.",
	"questions": {
		"urgent": {
			"type": "noul",
			"instructions": "Does the customer request urgent action?"
		},
		"review": {
			"type": "noul",
			"instructions": "Does this ticket need review by a person?"
		},
		"auto_reply": {
			"type": "noul",
			"instructions": "Is an automatic reply appropriate for this ticket?"
		},
		"billing": {
			"type": "noul",
			"instructions": "Should this ticket go to billing?"
		},
		"technical": {
			"type": "noul",
			"instructions": "Should this ticket go to technical support?"
		}
	}
}

All five question IDs map to the noul yes/no answer type. For each ID, we read the probability of yes from the returned answer, using the lookup below. Ardana normalizes that question’s two option scores and rounds the yes probability to four decimal places. We keep these as individual scores for the caller.

response["answers"][key]["noul"]

I use the shipped commands below to pull the model and start the server. These commands were verified in code for the research, but were not executed.

ardana pull decider:2b
ardana serve

With the example saved as request.json, we can submit it over HTTP. The CLI also accepts the same request file and can return JSON. Both entry points take our questions; the application still owns the rules connecting their answers.

curl http://127.0.0.1:8000/v1/systemone -H 'content-type: application/json' -d @request.json

ardana run decider:2b --request request.json --json

Ardana allocates a separate row for each yes/no question and formats those answers separately. That does not establish statistical independence between the scores. Our request has no field declaring cross-question constraints. We apply the example threshold and check the whole selected assignment in caller code next.

Validate the selected assignment before dispatch

I put this check between receiving the answers and dispatching an action. We first turn the probabilities into a selected assignment, then check that whole assignment against our policy. Our example cutoff is 0.5, inclusive, so a score exactly at the cutoff selects the label. I chose that cutoff for this example. It need not be the right threshold for another application.

import math

KEYS = ("urgent", "review", "auto_reply", "billing", "technical")
THRESHOLD = 0.5

Before thresholding, I check the consumed answer fields. The answer map must contain exactly our five question IDs, and each answer must carry the yes/no type tag. Each probability must be a finite number between 0 and 1, including both endpoints. I reject booleans explicitly, even though Python lets them behave like integers.

For our implication rule, selecting urgent requires selecting review. Our exclusion rule prevents review and automatic reply from being selected together. The route rule requires exactly one selection across billing and technical, so both an empty route selection and a double selection violate it. I collect every violation from this assignment so the caller can see which rules failed.

def validate(response):
    answers = response.get("answers")
    if not isinstance(answers, dict) or set(answers) != set(KEYS):
        raise ValueError("question IDs")
    probabilities = {}
    for key in KEYS:
        answer = answers[key]
        if not isinstance(answer, dict) or answer.get("type") != "noul":
            raise ValueError("answer type")
        value = answer.get("noul")
        if type(value) not in (int, float) or not math.isfinite(value) or not 0 <= value <= 1:
            raise ValueError("probability value")
        probabilities[key] = value
    selected = {key for key, value in probabilities.items() if value >= THRESHOLD}
    violations = []
    if "urgent" in selected and "review" not in selected:
        violations.append("urgent requires review")
    if {"review", "auto_reply"} <= selected:
        violations.append("review excludes auto_reply")
    if len(selected & {"billing", "technical"}) != 1:
        violations.append("exactly one route required")
    return {
        "status": "review" if violations else "accept",
        "selected": sorted(selected),
        "violations": violations,
        "probabilities": probabilities,
    }

When a consumed field is malformed, I raise an error for the caller to handle. A well-formed assignment that breaks a rule gets a review outcome. An assignment that satisfies all three rules gets an accept outcome. Our proposed caller permits dispatch only on that outcome. Acceptance means this candidate satisfies the rules. It does not mean the model understood the request correctly.

I return the selected labels and violations alongside the original probabilities. We keep those probabilities unchanged when an assignment fails. This validator performs no solver search, automatic relabeling, or probability repair. We can test contradictions directly with authored answers.

Test contradictions and keep joint confidence out

I test the caller rules with probabilities I wrote by hand. These fixtures contain only the answer fragments consumed by our validator, so they do not test a complete HTTP response. Save the displayed Python snippets in order as constraints.py, with this construction and loop below the validator from the previous section. Each tuple follows the five question IDs in the order already defined.

def fixture(values):
    return {"answers": {key: {"type": "noul", "noul": value}
                        for key, value in zip(KEYS, values)}}


CASES = [
    ("valid", (0.9, 0.9, 0.1, 0.8, 0.2), []),
    ("implication", (0.9, 0.2, 0.1, 0.8, 0.2), ["urgent requires review"]),
    ("exclusion", (0.1, 0.9, 0.8, 0.8, 0.2), ["review excludes auto_reply"]),
    ("no route", (0.1, 0.1, 0.8, 0.2, 0.2), ["exactly one route required"]),
    ("two routes", (0.1, 0.1, 0.8, 0.8, 0.7), ["exactly one route required"]),
    ("boundary", (0.5, 0.5, 0.4999, 0.5, 0.4999), []),
]

if __name__ == "__main__":
    for name, values, expected in CASES:
        response = fixture(values)
        result = validate(response)
        assert result["violations"] == expected
        assert result["status"] == ("review" if expected else "accept")
        assert result["probabilities"] == dict(zip(KEYS, values))
        assert validate({"answers": dict(reversed(list(response["answers"].items())))}) == result
        print(name + ": " + result["status"] + " | " + "; ".join(result["violations"]))

python3 constraints.py

Our authored sample output is below. The implication and exclusion cases each break one rule, while the route cases select either none or both. This output covers the six displayed cases.

valid: accept |
implication: review | urgent requires review
exclusion: review | review excludes auto_reply
no route: review | exactly one route required
two routes: review | exactly one route required
boundary: accept |

I include the boundary case because equality changes the selected assignment. Our example selects scores at 0.5 and leaves 0.4999 unselected, so that case passes. The broader source run passed seven named fixtures and all 32 Boolean assignments, plus checks for malformed values, structure, reversed key ordering and unchanged scores. Those assertions require normal Python execution without optimization. No model inference was performed.

We still cannot turn those accepted scores into joint confidence. In our checked counterexample, one distribution gives each outcome (0,0), (0,1), (1,0) and (1,1) probability 0.25. Another gives those same outcomes probabilities 0.5, 0, 0 and 0.5. Both give each variable a marginal probability of 0.5, but the probability that both are true is 0.25 in the first and 0.5 in the second.

I would need independence to multiply marginals into a joint probability. Passing our checks establishes only that this selected assignment satisfies our declared rules. We preserve the probabilities and send contradictions for review without manufacturing a confidence score for the whole decision.

Make review an explicit caller outcome

I would put the validator directly before dispatch and let only an accepted candidate through. For each decision, I would keep the raw answers alongside the selected labels and any rule violations. A contradictory assignment goes to review with its original probabilities intact. Our caller can then show which rules failed without silently changing labels or manufacturing replacement scores.

Rule satisfaction is the narrow guarantee here. An accepted assignment satisfies our declared checks, but the labels can still be wrong about the input. A rejected assignment also does not establish that the policy is infeasible. Our validator checks this candidate without searching for another valid assignment, so a failed check leaves that question open.

The next step is to connect this review outcome to the caller that already consumes our answers. Our fixture checks exercised the validator without running model inference, so they establish no improvement in model quality. To try this boundary with your own questions and rules, start from the Ardana repository.