DTCA

Denoising Token Credit Assignment for
Reinforcement Learning in Diffusion Language Models

Learning which denoising tokens deserve credit.

Hongze Chen¹ Miao Pan¹ Lizhou Liang¹ Qinfeng Li¹ Xinyin Ma² Yongliang Shen¹ Weishi Wang³ Daniel Dahlmeier³ Xuhong Zhang¹ Wenqi Zhang¹

OMNIAI

¹ Zhejiang University² National University of Singapore³ SAP

Paper Code

Why POS for credit assignment in DLM

Different parts of speech exhibit distinct commitment patterns across denoising steps and contribute differently to subsequent refinement.

Paper Figure 1: part-of-speech groups show different commitment trajectories and mean confidence gains on GSM8K training rollouts.

How DTCA works

Measure contributions

Commit one token. Measure the change in maximum prediction confidence at the remaining masks.

F before

0.510

F after

—

{Δ P}_{i} = \max (0, F ({x_{t}}^{(i)}; ℳ) - F (x_{t}; ℳ))

\begin{array}{l} {Δ P}_{i} = \max (0, \\ F ({x_{t}}^{(i)}; ℳ) \\ - F (x_{t}; ℳ)) \end{array}

ΔP—

Average by POS

Average the gains for tokens in the same POS.

m_{c} = \frac{1}{N_{c}} \sum_{i : c_{i} = c} {Δ P}_{i}

Choose useful steps

Score each step. Keep the steps with high scores.

q_{t} = \sum_{i \in 𝒜_{t}} w_{c_{i}} (1 - p_{i})

Target score coverage70%

Assign token credit

Redistribute rewards among tokens according to POS weights while preserving the total reward.

Token advantage Before After

Balance entropy flow

Adjust token advantages. Balance the predicted entropy flow in each response.

Predicted entropy change ΔH Before After

Predicted entropy change

+0.1200.000

Better mathematical reasoning.

Table 1 · Mathematical reasoning accuracy (%)
Method	MATH500			GSM8K
Method	128	256	512	128	256	512
LLaDA-8B-Instruct	30.8	36.4	39.2	75.3	83.2	82.9
LLaDA-1.5	32.6	39.4	39.8	76.2	81.6	83.0
Uniform SFT	24.6	30.0	32.6	74.8	80.7	78.6
d1	33.6	37.6	37.2	75.8	82.4	81.8
ESPO	37.6	38.0	40.2	78.1	83.2	84.0
TraceRL	32.6	34.2	34.0	76.6	80.4	81.8
Coupled-GRPO	35.2	40.2	40.2	78.0	83.6	83.2
RLDF	36.2	41.2	42.6	77.2	83.4	82.7
DTCA w/o Entropy Calibration	37.4	41.8	39.8	78.3	83.9	81.9
DTCA w/o Contribution Credit	36.2	39.4	41.2	77.1	82.9	82.3
DTCA (ours)	39.8	43.2	44.0	80.6	85.3	85.4
Δ vs. Base	+9.0	+6.8	+4.8	+5.3	+2.1	+2.5

Low → High · per column

Cite DTCA

@misc{chen2026dtcadenoisingtokencredit,
      title={DTCA: Denoising Token Credit Assignment for Reinforcement Learning in Diffusion Language Models},
      author={Hongze Chen and Miao Pan and Lizhou Liang and Qinfeng Li and Xinyin Ma and Yongliang Shen and Weishi Wang and Daniel Dahlmeier and Xuhong Zhang and Wenqi Zhang},
      year={2026},
      eprint={2610.35201},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2610.35201},
}