How to bridge the gap between artistic intent and technical execution
Digital photography is not complete without photo retouching. It allows users to edit image components such as contrast, tone and exposure to produce visually appealing content. Whether it’s for professional or personal reasons, many users want to improve images according to their aesthetic preferences. Photo retouching is a complex art that requires both creative and technical skills.
It is the difference between automated and manual solutions that causes most of the problems. Although professional software such as Adobe Lightroom has many retouching tools, it can take a lot of time and effort to master these programs. AI-driven techniques tend to be overly simplistic, and do not offer nuanced editing or the necessary control. They also have difficulty generalizing to different visual scenarios or supporting complicated user instructions.
AI Photo Editing: Limitations and Current Models
Photo retouching has traditionally been handled by traditional tools that rely on reinforcement learning and zeroth-order optimization. Other methods use diffusion-based image synthesis. The strategies make progress, but they are largely limited by the inability of these methods to maintain finely-granulated regional control or high-resolution images, as well as the inability to preserve the original content. Even the most recent models such as GPT-4o or Gemini-2 Flash offer text-driven edits, but they compromise user control and often erase critical details.
JarvisArt is a multimodal AI retoucher that integrates Chain-of Thought APIs and Lightroom APIs
Researchers from Xiamen University, the Chinese University of Hong Kong, Bytedance, the National University of Singapore, and Tsinghua University introduced JarvisArt—an intelligent retouching agent. This system utilizes a Multimodal large language model Image editing can be flexible with instructions. JarvisArt has been trained to mimic the decision-making processes of professional artists. It interprets user intent using both visual and verbal cues and executes retouching action across more than 200 Adobe Lightroom tools via a customized integration protocol.
The method integrates three main components. First, the researchers constructed a high-quality dataset, MMArt, which includes 5,000 standard and 50,000 Chain-of-Thought–annotated samples spanning various editing styles and complexities. JarvisArt then undergoes two phases of training. In the first phase, JarvisArt is fine-tuned under supervision to develop reasoning and selection capabilities. It’s followed by Group Relative Policy Optimization for Retouching (GRPO-R), which incorporates customized tool-use rewards—such as retouching accuracy and perceptual quality—to refine the system’s ability to generate professional-quality edits. A2L, or Agent-to Lightroom protocol, ensures that tools are executed in a transparent and seamless manner within Lightroom. This allows for users to adjust their edits dynamically.
JarvisArt Benchmarking Capabilities & Real Performance
JarvisArt’s ability to apply nuanced editing and interpret complex instructions was assessed using MMArt, a benchmark created from edits made by real users. The system delivered a 60% improvement in average pixel-level metrics for content fidelity compared to GPT-4o, maintaining similar instruction-following capabilities. The system was able to handle both global and local image refinements. It can also manipulate images with arbitrary resolution. It can, for example, adjust the skin texture, brightness of eyes, and hair definition according to region-specific directions. All of these results are achieved with the same aesthetic goals as defined by users, demonstrating a combination of practical control and high quality in multiple editing tasks.
Conclusion: An agent that combines technical precision with creativity to stifle innovation
The researchteam tackled a significant challenge—enabling intelligent, high-quality photo retouching that does not require professional expertise. By combining data-synthesis, reasoning driven training and integration with commercially available software, the researchers’ method bridges between automation and control. JarvisArt is a powerful and practical solution for users looking to achieve both quality and flexibility in image editing.
Take a look at the Paper You can also find out more about the following: GitHub Page. The researchers are the sole credit holders for this work. Ready to connect with 1 Million+ AI Devs/Engineers/Researchers? MarkTechPost helps NVIDIA, LG AI Research and other top AI companies reach their audience.[Learn More]
Nikhil has been an intern at Marktechpost. The Indian Institute of Technology in Kharagpur offers him a dual degree integrated with Materials. Nikhil has a passion for AI/ML and is continually researching its applications to fields such as biomaterials, biomedical sciences, etc. He has a background in Material Science and is always exploring advancements.
