Nokia Open-Sources AnyJev: A Coaching-Free Layer That Turns Any Open LLM Right into a Calibrated Resolution Mannequin

Nokia’s utilized analysis group has open-sourced AnyJev, a Python library that turns an open LLM into a call mannequin. It wants no coaching. It targets a typical manufacturing job: selecting one reply from a set set as a substitute of writing a sentence.

Is it deployable? Sure, it installs from PyPI, ships below Apache-2.0, and has transformers and vLLM backends with shared-prefix scoring.

What’s AnyJev?

AnyJev borrows its interface from Jev. Jev is the System One resolution mannequin that TypeSafe AI launched in September 2026 (our coverage). You give AnyJev a typed query and get again a call with a likelihood you may threshold. That likelihood is learn from the mannequin’s next-token distribution. Nothing is generated, parsed, or skilled.

The library helps 3 query sorts:

• A alternative query picks certainly one of Okay choices.
• A noul query is sure or no.
• A rating query locations the reply in certainly one of a number of ordered bins.

The Downside With Studying Logits Instantly

Many open initiatives already limit the following token to the choice labels and skim the scores. The Nokia analysis group flags 2 flaws in that shortcut. First, the reply can change when the choices are reordered. Second, the possibilities are usually not calibrated.

The levels doc names 2 causes:

• The primary is prior bias: the mannequin favors some labels, corresponding to “Yes” over “No”, regardless of the enter.
• The second is place bias: the mannequin favors sure slots within the possibility checklist.

How AnyJev Works: L0 and L1

Each resolution carries a degree subject.

L0 (zero labels, on by default) applies 2 fixes:

• Cyclic shifts. For a query with Okay choices, the checklist is proven in Okay rotations, so each possibility seems in each place as soon as. The outcomes are mixed in log house as a geometrical imply. If the place bias is additive in logit house, this removes it precisely.
• Prior correction. By default, AnyJev makes use of batch calibration. It retains a operating imply of the anticipated distributions on actual inputs and divides it out at energy 0.75. The correction begins after 8 objects.

L0 prices Okay prefills per resolution, batched over a shared prefix. That’s about 0.25 s per resolution at batch 32 on one H100, with Okay = 20.

L1 (100 to 500 labels per query) provides temperature scaling on prime of L0. The fitted values are saved as a small JSON artifact. L1 reshapes confidence however doesn’t change the rating of solutions.

Benchmark Outcomes

On Qwen3-8B with BANKING77 (20-way, 300 check objects), the numbers seem like this:

A number of different outcomes from the repo:

• L0 lowered order flips on all 9 mannequin and process rows examined.
• On a typed-decisions set, Qwen3-32B with L1 reached an ECE of 0.036, in contrast with 0.144 printed for Jev. On accuracy, the fine-tuned Laya nonetheless leads.
• The full ablation table covers Qwen, OLMo, Granite, Phi and Mistral fashions.
• Wu says the group tried AnyJev on an inside Nokia routing downside and noticed promising outcomes.

Use AnyJev

# pip set up "anyjev[hf]"
from anyjev import Decider, Query
from anyjev.backends.hf import HFBackend

d = Decider(HFBackend("Qwen/Qwen3-8B"))
route = Query.alternative("Which team should handle this?",
                        ["billing", "technical", "sales", "other"], title="route")
r = d.resolve({"conversation": [...]}, [route])
r["route"].distribution   # possibilities per possibility

For serving, you begin vLLM with prefix caching and level a VLLMBackend at it.