Contrastive-LM has launched CLM-8B, the primary open mannequin in a brand new class known as Contrastive Language Fashions (CLMs). CLM doesn’t generate textual content. It scores a set of candidate actions in opposition to the present state and returns chances. Their fundamental baseline is Jev, the proprietary System One mannequin from TypeSafe AI.
Is it deployable? Sure. The Apache-2.0 head weighs 75 MB. It runs on 1 NVIDIA GPU beneath Linux, with vLLM serving the Qwen3-8B encoder.
What a System One Mannequin Does
Jev entered restricted early entry on 15 September 2026. It returns typed values with chances as an alternative of textual content. CLM targets the identical interface. The CLM GitHub repo serves CLM-8B behind a TypeSafe-compatible API. It exposes 3 query sorts:
- Noul: returns the likelihood {that a} assertion is true.
- Alternative: picks one possibility from a declared set, with chances.
- Rating: returns an anticipated stage on an ordered rubric.
A request written for TypeSafe’s API may be replayed by CLM’s Python shopper.
How CLM Works
CLM trains a state encoder and an motion encoder with a bidirectional InfoNCE loss. Every encoder is a frozen Qwen3-8B spine plus a 20M-parameter trainable projection head. Coaching pulls every state towards the motion truly taken and pushes it away from the others.
At inference, CLM scores every candidate by the dot product of the state and motion embeddings. A softmax over these scores turns into the reply distribution. The identical primitive ranks best-of-N options, routes instruments and solutions typed choices.
This design disaggregates states and actions. In an agent loop, the state adjustments each step whereas the motion set stays principally mounted. clm-serve reserves a slab of GPU reminiscence, just like vLLM’s KV cache, and reuses cached vectors. On 1 RTX 4090 with 3 actions, revisited states drop from 1.7 ms to 0.6 ms. The mannequin card stories CLM working 13× sooner than Jev with about 1,000 candidates.
A 3-Stage Coaching Recipe
- Pre-training on ~60M Nemotron DQA question-answer pairs.
- Mid-training on ~30M artificial arduous negatives generated by Gemini 2.5 Flash-Lite.
- Submit-training on ~1M agent trajectories from Agent Information Protocol, Infinite-Terminals and LiteCoder-Terminal-SFT.
On ~100K held-out questions, pre-training alone reaches 52.1% top-1 accuracy. Mid-training lifts it to 69.2%. Coaching on arduous negatives from the beginning peaks at 62.4%, then overfits.
Zero-Shot Outcomes In opposition to Jev
| Process | CLM-8B latency | Jev latency | CLM-8B success | Jev success |
|---|---|---|---|---|
| T-Rex recreation | 16.5 ms | 149.8 ms | 5/5 | 5/5 |
| Software calling (BFCL v4) | 76.8 ms | 125.5 ms | 95.2% | 99.2% |
| WikiRacing | 79.8 ms | 225 ms | 26/30 | 30/30 |
| Tremendous Mario | 33.5 ms | 132.6 ms | 5/5 | 5/5 |
The 9× determine comes from the T-Rex game, the place actions repeat throughout states. CLM matches Jev on T-Rex and Tremendous Mario. It trails on instrument calling and WikiRacing whereas working sooner on each job.
CLM as a Verifier for Coding Brokers
Right here a generator samples a number of candidate options and the verifier picks one. Opus 5 produced DeepSWE candidates (best-of-4). Fable 5 produced Terminal-Bench 2.1 candidates (best-of-5). The crew evaluated 38 held-out DeepSWE duties and 30 held-out Terminal-Bench 2.1 duties. Latency was measured on an H100.
| Benchmark | Cross@1 | CLM (fine-tuned) | Jev | CLM latency | Jev latency |
|---|---|---|---|---|---|
| DeepSWE | 73.7% | 81.6% | 71.1% | 79 ms | 449 ms |
| Terminal-Bench 2.1 | 84.0% | 87.6% | 83.1% | 32 ms | 131 ms |
The analysis crew stories these as new SOTA verifier outcomes. Jev scores under go@1 on each benchmarks, so deciding on with Jev is worse than taking 1 pattern. CLM runs 4.1× to five.7× sooner. These numbers use light-weight fine-tuned heads, not the zero-shot checkpoint. They’re held-out subset outcomes, not full leaderboard submissions.
Interactive Explainer
