How can AI teams use their infrastructure to run Tinker-style reinforcement learning using large language models with a single engine? Anyscale Team and NovaSky Team (UC Berkeley). Releases SkyRL tx v0.1.0 It allows them to install a Tinker compliant training engine and an inference engine on their own hardware. They can still use the same API as Tinker’s managed service.
This is what the research team says SkyRL Tx The Tinker API is implemented as an unified inference and training engine, allowing people to implement Tinker on their own infrastructure. This version v0.1.0 of the series is first in its line to fully support reinforcement learning.
Tinker API in Brief
Tinker, from Thinking Machines, is a learning API that’s built around four key functions. forward_backward The gradient is accumulated by performing a pass forward, a pass backward and an accumulation of the two passes. optim_step Model weights are updated based on gradients. Sample Generates tokens as a result of interaction, evaluation and RL actions. save_state Checkpoints are written for the resumed training.
Tinker provides low-level primitives that allow users to implement their own reinforcement or supervised learning loops using regular Python, with the service handling GPU scheduling and distributed computation.
SkyRL Tx is based on this API, and provides a local backend. The Tinker model is retained, but the hosted environment can be removed.
SkyRL tx and SkyRL
SkyRL, a reinforcement learning tool for large models of language that include skyrl-agent Agents with a long-term horizon, skyrl-train For training and skyrl-gym For tool environments like math, coding and SQL.
This stack contains: skyrl-tx This is an experimental library for cross-platform platforms that exposes REST like APIs in Tinker to post train models. SkyRL tx is the system layer which connects RL, training codes and environment logic to GPU resources using the Tinker GUI.
Inference engine for architecture that trains
SkyRL-tx architecture can be described as a backward pass engine with an inference algorithm. The SkyRL tx architecture has four major components:
- REST API server It is a service that receives requests from incoming users.
- Database This queue also tracks meta-data about models, futures, checks, and requests. This implementation supports Postgres, as well as other SQL databases like SQLite.
- Engine This schedules requests and groups them across all users. The engine instances can be configured to serve one base model. They also allow for multiple LoRA adapters.
- Worker It executes forward- and backward-passes, holds the model definitions, and maintains optimizer states. Multi-workers would enable multi node sharding to be more advanced in future versions
What does v0.1.0 bring to the table?
The release of v0.1.0 focuses on the reinforcement learning and performance improvement. The release of v0.1.0 focuses on performance and reinforcement learning improvements. official release Highlights several concrete improvements
- It is much quicker to sample now that the samples are properly batched, sharded and jitted in the engine.
- The ability to use different sampling parameters, seeds, and tokens for each request is now available. This can be useful when many experiments are based on the same base model.
- After several fixes the RL Loop now works properly in the engine.
- Micro-batching and gradient checkpointing for sampling is implemented.
- Postgres has been added to the list of supported database backends, alongside SQLite.
Running RL End to End on 8 H100 Graphics Cards
This release has a recipe of code for implementing reinforcement learning on an 8-GPU cluster.
The SkyRL repository is cloned first and the SkyRL repository in then re-synthesized. skyrl-tx Start the engine by:
uv run --extra gpu --extra tinker -m tx.tinker.api
--base-model Qwen/Qwen3-4B
--max-lora-adapters 3
--max-lora-rank 1
--tensor-parallel-size 8
--train-micro-batch-size 8 > out.log
They then clone Tinker Cookbook, from the Thinking Machines Team and copy it into the tinker_cookbook/recipes folder run:
Export TINKER_API_KEY=dummy
Export WANDB_API_KEY=
Run uv --with wandb -with Tinker Rl_loop.py
base_url=http://localhost:8000
model_name="Qwen/Qwen3-4B"
lora_rank=1
max_length=1024
save_every=100
It will produce a curve of rewards that shows the RL Loop is correctly running through the SkyRL tx.
What you need to know
- SkyRL tx V0.1.0 is a Tinker-compatible local engine which unifies the training of LLM and its inference.
- It exposes Tinker primitives like forward_backwards and optim_step over REST. However, it handles batching, LoRA Adapters and device positioning internally.
- It is broken down into four components: API server (or gateway), SQL database, scheduler engine, and workers who execute both forwards and backwards passes.
- This version includes end to end support for reinforcement learning. It also adds faster jitted and shaled sampling, parameters per request, gradient checkpointing and Postgres Support.
SkyRL Tx V0.1.0 offers a simple solution to dev teams who wish to use Tinker’s style of reinforcement learning with their own clusters. It also provides an interface that matches the Tinker API. It is cleaner and less divergent to treat the system like an inference engine, which also runs backwards passes. This upgrade includes support for LoRA gradient checkpointing and Postgres. This release transforms Tinker compatibility to a local RL-backend that is actionable for LLM
Click here to find out more Repo The following are some examples of how to get started: Official Release. Check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter Join our Facebook group! 100k+ ML SubReddit Subscribe now our Newsletter. Wait! Are you using Telegram? now you can join us on telegram as well.

