Kyutai, with Inria and École des Ponts, has launched REKI, a 2.4B vision-language-action mannequin (VLA) for robotic manipulation. The REKI vision-language-action mannequin is pretrained solely on public information. It additionally carries an inexpensive reminiscence of its latest previous. On three out-of-distribution benchmarks, solely fashions about twice its dimension beat it. All weights are live on Hugging Face right this moment.
TL;DR
- Measurement: 2.4B parameters (2.2B Qwen3.5-2B VLM + 0.2B flow-matching motion skilled). Reminiscence covers the final 3 frames, 1 s aside.
- Runs on: Benchmarked on 1 NVIDIA H100. 21.8 ms per motion chunk, compiled, from 2 cameras at 256×256. Weights ship in BF16. No quantized variants introduced.
- Efficiency: Finest amongst VLAs pretrained on public information on LIBERO-Plus and RoboTwin Clean2Rand.
- Finest: 90.4% on LIBERO-Plus, the highest rating in Kyutai’s comparability.
- Worst: 38.4% on RoboCasa365, nicely behind Xiaomi-Robotics-1 (57.4%).
What’s REKI?
REKI is a vision-language-action mannequin from Kyutai that maps digicam views and an instruction to robotic actions. It pairs a Qwen3.5-2B spine with a 200M-parameter motion skilled. The skilled generates motion chunks by move matching, built-in in 10 Euler steps. Robotic state and move time enter via adaptive RMSNorm. Every robotic state structure will get its personal small state MLP.
How does REKI’s reminiscence work?
Most VLAs act on the present body alone. Prior reminiscence strategies feed previous frames to the VLM, unroll coaching, or add modules. Every choice provides value.
REKI locations reminiscence contained in the motion skilled as an alternative. Earlier than every body’s motion tokens sit 32 discovered register tokens. They cross-attend the encoded immediate and views of that body. In addition they self-attend the registers of earlier frames.
Registers by no means learn motion tokens. In order that they rely solely on observations, not move time or robotic state. They’re computed as soon as per body and cached in a rolling window.
The associated fee is small, per the project page:
- Studying cached registers provides 0.1 ms to a 21.8 ms chunk.
- The cache makes use of about 2 MiB of GPU reminiscence per previous body.
- Coaching packs a 3-frame window into 1 batched go.
Reminiscence additionally fixes a failure mode. With out it, arms usually oscillate between 2 poses. On RoboTwin, oscillating episodes fall from 18.6% to 4.1% with reminiscence. Kyutai attributes this to perceptual aliasing.
How does REKI carry out on benchmarks?
REKI is pretrained as soon as, then finetuned individually per benchmark. Scores beneath come from the model card and venture web page.
- LIBERO-Plus: 90.4%, forward of Qwen-RobotManip (89.0%) and π0.5 (85.7%).
- RoboTwin Clean2Rand: 65.8%, behind solely Qwen-RobotManip (68.5%).
- RoboCasa365: 38.4%, forward of RLDX-1 (36.0%) and Qwen-RobotManip (35.9%).
- LIBERO (4 suites): 97.6%.
- RoboTwin 2.0 Simple / Laborious: 85.9% / 84.6%.
Pretraining helps most out of distribution. It provides 13.8 factors on Clean2Rand and 30.3 below litter. Reminiscence provides 1.5 factors on LIBERO-Plus and 5.4 on RoboCasa365.
window.addEventListener(“message”,operate(e){if(e.information&&e.information.mtpRekiHeight){doc.getElementById(“mtp-reki-frame”).type.top=e.information.mtpRekiHeight+”px”;}});
How does REKI examine with different VLAs?
| Function | REKI (Kyutai) | Qwen-RobotManip (Qwen) | π0.5 (Physical Intelligence) | ABot-M0 (AMAP) |
|---|---|---|---|---|
| Whole dimension | 2.4B | 4.7B (per Kyutai) | 3.3B (per Kyutai) | 4.6B (per Kyutai) |
| VLM spine | Qwen3.5-2B | Qwen3.5-4B | Gemma-based | Not disclosed |
| Pretraining information | Public solely | Open-source + human video | Includes own robot data | Open-source (UniACT) |
| Reminiscence | 32 registers, 3 frames | In-context variant | None | None |
| LIBERO-Plus | 90.4 | 89.0 | 85.7 (Kyutai eval) | 80.5 |
| RoboTwin Clean2Rand (imply) | 65.8 | 68.5 | 63.8 | 55.7 |
| RoboCasa365 | 38.4 | 35.9 | 16.9 | Not disclosed |
| H100 latency, compiled | 21.8 ms | Not disclosed | 33.8 ms (openpi, Kyutai) | 46.1 ms (Kyutai) |
| Weights | Open, CC BY-NC-SA 4.0 | Not released | Open, Apache-2.0 repo | Open, Apache-2.0 repo |
Notes: Clean2Rand means are computed from the 6 situations in Desk 5 of the Qwen-RobotManip report. The Qwen report lists π0.5 at 84.4 on LIBERO-Plus.
What information was REKI educated on?
The combination holds 1.0B frames plus 12.2M vision-language samples. Actual-robot teleoperation gives 639.8M frames, simulation 193.9M and selfish human video 170.9M. Sources embody DROID, BridgeData V2, AgiBot World, EgoDex and EgoVerse. Pretraining ran 200,000 steps on 32 H100 GPUs. Kyutai retains every dataset’s native management conference reasonably than remapping actions.
How do you deploy REKI?
Six checkpoints are listed: the pretrained base plus finetunes for LIBERO, RoboTwin (2 variants), RoboCasa365 and RoboCasa-GR1. Actual-robot deployment makes use of the LeRobot async-inference gRPC protocol. On a UR5 arm, REKI was finetuned for 3k steps (1 hour) on 50 demonstrations per process.
from reki import RekiPolicy
coverage = RekiPolicy.from_pretrained("kyutai/reki-2b-libero", gadget="cuda")
coverage.reset() # clears the register cache at episode begin
actions = coverage.predict(pictures={"camera_top": img0, "camera_wrist_left": img1},
state=proprio, process="put the bowl on the plate")
Key Takeaways
- REKI is a 2.4B VLA pretrained solely on public information.
- It scores 90.4% on LIBERO-Plus, high in Kyutai’s comparability.
- 32 cached register tokens give reminiscence for 0.1 ms additional.
- Reminiscence cuts pose oscillations from 18.6% to 4.1% on RoboTwin.
- Weights are open however non-commercial (CC BY-NC-SA 4.0).
Take a look at the Paper, Project Page, Model Weights and GitHub Repo. All credit score goes to the researcher of this venture. Additionally, be happy to comply with us on Twitter and don’t overlook to affix our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
The publish Meet Kyutai REKI: The 2.4B VLA Model Bringing Efficient Memory to Robot Manipulation appeared first on MarkTechPost.

