In recent months, there has been growing interest in applying diffusion models—originally designed for continuous data, such as images—to natural language processing tasks. DLMs are Discrete Diffusion Language Models which use text generation to denoise the output. DLMs are more flexible than traditional autoregressive model and offer better structure control. They also allow parallel decoding, which is a great advantage. A non-sequential model can also be generated faster. Despite these benefits, most current multimodal large language models (MLLMs)—such as LLaMA, Qwen-VL, and InternVL—still rely solely on autoregressive methods.
In the study of diffusion-based languages, both continuous and discrete spaces have been explored. DiffuSeq, SED and other continuous approaches use relaxed or embedded categorical space for smoother creation. SDDM models and RDM on the other hand, tailor diffusion to language structures. The training techniques used vary but are commonly based on masked loss language models or score matching based on entropy. AR-Diffusion, SSD-LM and other hybrid models combine the autoregressive with diffusion strategies, allowing them to take advantage of their strengths. Open-source MLLMs, such as LLaVA or InternVL, have improved through joint pretraining and visual instruction tuning, but still use an autoregressive generator scheme.
Dimple was developed at National University of Singapore by researchers. This is the world’s first Discrete DMLLM. Dimple integrates a visual encoder and a discrete diffusion based language model. To overcome the instability and performance issues of purely diffusion-based training, they introduce a two-phase training method—Autoregressive-then-Diffusion—combining initial autoregressive alignment with subsequent diffusion-based masked language modeling. On benchmarks, Dimple-7B outperforms LLaVA NEXT with a 3.9% advantage. Confident Decoding, for token generation dynamically and exploring Structure Priors to control output precisely are also implemented by the team. These innovations improve the inference process, flexibility of generation, and controllability structurally without impacting performance.
Dimple is a Discrete Diffusion Multimodal LLM which integrates a diffusion-based model with a vision encoder. In order to address diffusion training inefficiencies, like sparse supervision or limited coverage of generation, the model will be trained in two phases. First, with autoregressive learning using a mask that focuses on causal attention for alignment between vision and language, followed by diffusion training. Dynamic attention is used during inference. “Confident Decoding” The strategy adjusts updates to tokens based on the prediction confidence. Dimple outperforms similar-scale auto-regressive models on several benchmarks despite using significantly less training samples. However, it still trails larger-scale, state-of the-art systems.
The experiments evaluate Dimple, a DMLLM, against autoregressive models on instruction-following tasks. Dimple is a DMLLM that was trained using a hybrid strategy combining autoregressive tuning and diffusion tuning. It exhibits excellent performance on many benchmarks, outperforming models with similar data. Dimple, despite being behind other models with much larger training datasets but still achieving good results thanks to a strong base language model. The combination of autoregressive with diffusion tuning reduces length biases and increases consistency, according to ablation studies. The model is more efficient in understanding multimodal tasks with the addition of prefilling.
Dimple was developed to address the problems associated with purely discrete training. These include length bias and instability. Dimple uses a hybrid approach to training that begins with autoregressive tuning and then diffusion tuning. This results in the Dimple-7B, a model which is 3.9% better than LLaVA NEXT. Prefilling and a decoding strategy called confident decoding significantly decrease inference steps. Dimple is able to produce structured and controlled outputs by using structure priors. This allows for finer-grained control on format and duration capabilities, which autoregressive model struggles to achieve.
Click here to find out more Paper, Model on Hugging Face You can also find out more about the following: GitHub Page. The researchers are the sole owners of all credit. Also, feel free to follow us on Twitter Join our Facebook group! 95k+ ML SubReddit Subscribe now our Newsletter.
Sana Hassan is a dual-degree IIT Madras student and consulting intern with Marktechpost. She loves to apply technology and AI in order to solve real-world problems. Sana Hassan, an intern at Marktechpost and dual-degree student at IIT Madras is passionate about applying technology and AI to real-world challenges.


