The Hugging Face app (HF) is now available. Smol2Operator. A reproducible recipe for turning a vision-language-model (VLM), which has no previous UI background, into a GUI agent that uses tools. The release covers data transformation utilities, training scripts, transformed datasets, and the resulting 2.2B-parameter model checkpoint—positioned as a complete blueprint for building GUI agents from scratch rather than a single benchmark result.
What’s New?
- A small VLM can be used for a two-phased post-training. Start here SmolVLM2-2.2B-Instruct—a model that “initially has no grounding capabilities for GUI tasks”—Smol2Operator first instills perception/grounding, then layers agentic reasoning with supervised fine-tuning (SFT).
- Unified Action Space across heterogeneous Sources: Conversion pipelines normalize disparate GUI actions taxonomies, (mobile, desktop and web), into a single function API.
Click here to learn more about,You can also type,You can also drag the slider to your right.The normalized [0,1] An example of this is the use of coordinates to allow for coherent training on datasets. The An Action Space Converter Supports remapping of custom vocabulary.
What is Smol2Operator and why use it?
Smol2Operator’s action schema and coordinates can be used to create a pipeline that is compatible with most GUI agents. Smol2Operator’s action-space unification The following are some examples of how to get started: Normalized Coordinate The strategy will make the datasets interoperable, and ensure that training is stable even when image resizing occurs. VLM preprocessing does this a lot. The engineering overhead associated with the assembly of multi-source GUI is reduced, and small agents can be used to reproduce agent behavior.
What is it? How it works?
- Data standardization:
- Standardize the parameter names, remove duplicate actions and convert pixels to normalized coordinates.
- Phase 1 (Perception/Grounding):
- The unified action dataset is used to measure SFT and learn about element localization. ScreenSpot-v2 The localization is shown on the screenshot.
- Phase 2:
- SFT can be used in conjunction with the API to create a step-by-step action plan based on grounded perception.
As grounding skills are learned, the HF Team shows a similar scaling strategy to a 460M “nanoVLM,” Indicating the portability between capacities of the method (numbers appear in the table).
Next steps, Scope and Limits
- No, not at all “SOTA at all costs” push: The HF team frames the work as a Process blueprint—owning data conversion → grounding → reasoning—rather than chasing leaderboard peaks.
- Focus on evaluation Demonstrations will be held in the vicinity of ScreenSpot-v2 Future work will include the creation of qualitative and end-to-end task video; benchmarks for broader tasks that cross environments, OS, or have a long-horizon. The HF team highlights potential benefits from RL/DPO for policy adaptation beyond SFT.
- Ecosystem Trajectory: ScreenEnv’s roadmap includes wider OS coverage (Android/macOS/Windows), which would increase external validity of trained policies.
The following is a summary of the information that you will find on this website:
Smol2Operator, a reproducible and open-source pipeline for upgrading systems SmolVLM2-2.2B-Instruct—a VLM with zero GUI grounding—into an agentic GUI coder via a two-phase SFT process. This release transforms AGUVIS datasets to a standard API, includes training notebooks with preprocessing code and a demo space, as well as a final checkpoint. This release prioritizes portability and transparency of the process over leaderboards, and is a good fit for the smolagents Runtime ScreenEnv For evaluation, this guide offers a blueprint that can be used by teams to build small GUI agents for operators.
Click here to find out more Technical details, and Full Collection on HF. Feel free to check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter.

