DTCA: Reinforcement Learning credit assignment for diffusion language models
In brief
- DTCA assigns credits to tokens in diffusion language models trained with reinforcement learning.
- The method measures prediction confidence improvement of masked positions after token commitment.
- Gains reach 9.0 points on MATH500 and 5.3 points on GSM8K with LLaDA-8B-Instruct.
Researchers introduce DTCA, a framework for token-level credit assignment in reinforcement learning for diffusion language models (DLMs). According to the announcement, DTCA estimates token contributions by measuring the improvement in prediction confidence of remaining masked positions after token commitment, then groups these signals by part-of-speech categories. The resulting weights are used to select informative denoising timesteps and assign token-level learning signals during RL training.
Experiments on LLaDA-8B-Instruct show DTCA improves mathematical reasoning performance, with gains of up to 9.0 points on MATH500 and 5.3 points on GSM8K compared to the base model.
This summary gives you the essentials. The full article, its figures and quotes are on Hugging Face Daily Papers.
Read the original articleAI-generated summary of the source, checked by rules. How we write · Report an error