LoopUno: Where Does the Time Go When a Looped LLM Drafts Its Own Tokens?

Diffusion drafting lets a language model guess several of its own next tokens in one pass and keep the ones it would have sampled anyway, so the output is unchanged. Each decode step runs the full model twice, once with a LoRA to draft and once to verify, and it still decodes faster than standard decoding. We time every part of the step to see why. Verifying 7 drafts adds 0.3 ms to a 13 ms pass. The LoRA adds 3.1 ms, three quarters of the overhead, so a drafting step costs 1.15× a standard two-pass step. The speedup is the tokens kept per forward pass divided by this cost ratio. We propose LoopUno, an adaptation of Uno that brings diffusion drafting to looped transformers, and validate it on the looped agentic model Nanbeige4.2-3B. The second loop makes each forward pass 1.9× slower and doubles the drafting overhead with it, so the cost ratio stays at 1.15×. After one hour of LoRA training a step keeps 4 tokens where standard decoding keeps 2, and function calls decode 2× faster at matching accuracy.

Diffusion drafting lets a language model guess several of its own next tokens in one forward pass. Uno[1] does this with a LoRA, in the style of a discrete diffusion model. The original model then checks the guesses and keeps the tokens it would have sampled itself, so the samples are unchanged by construction. The drafter is the full model with the LoRA switched on, so every decode step runs the full model twice: once to draft and once to verify. Why is this faster at all?

We timed every part of a decode step to find out. Verification is nearly free: checking 7 drafts adds 0.3 ms to a 13 ms forward pass. Drafting costs a LoRA: 3.1 ms per step, three quarters of the overhead. A drafting step takes 1.15× as long as two standard forward passes. After one hour of training it keeps about 4 tokens where the two standard passes keep 2. Tokens per forward (TPF) is the number of tokens a step keeps divided by its forward passes. The cost ratio is the time of a drafting step divided by the time of two standard passes. The speedup is TPF ÷ cost ratio.

We ask this question on a looped model. A looped model stores one stack of layers and runs it more than once for every token. Each run through the stack is a loop. Nanbeige4.2-3B[2] stores 22 layers and runs two loops. The second loop adds capacity without adding parameters. With 3B parameters the model outperforms 9B and 12B models on agentic benchmarks[2]. The price is time. Every token costs two loops, so this 3B model decodes as slowly per token as an 8B model.

A looped model buys capability with time per token. A diffusion drafter buys time back with tokens per pass. Uno was built for standard layer stacks. In a looped model one LoRA has to serve weights that every token uses twice, and the draft has to survive two loops. The LoRA also runs once per loop, so its cost doubles. The forward pass doubles too, and the cost ratio stays at 1.15×, the same as on the standard Qwen3-8B. A looped model can keep its small memory footprint and recover part of its latency.

Nanbeige4.2-3B gives a second reason to try. It is an agentic model, trained for function calling and code-agent work. Agents are where decoding speed is felt most: one task takes many tool calls, and every call waits on the decoder. Agents are also where a speedup must leave the output alone, because one changed argument breaks a tool call. A lossless drafter fits this setting. So we measure the speedup on function calls and agent trajectories as well as on math, with a LoRA that saw only math and reasoning data.

Our training budget is one hour on one 8×A100 node. The released Uno LoRA for Qwen3-8B used 14.7B training tokens. Ours uses 31M.

Three questions

  1. Does diffusion drafting fit a looped model, and what does the loop cost? We extend Uno's training and decoding to the looped architecture and time the second loop.
  2. Where does the time of a decode step go? We split the cost ratio into the draft pass, the verify pass and the work outside the model, and we separate the LoRA from the extra rows.
  3. How much speed does one hour of training buy? We measure TPF on math, function calling and agent trajectories, compare the measured speed with TPF ÷ cost ratio, and report the cases that came out slower or flat.

Background

Draft with a LoRA, verify with the original model

Uno keeps the base model frozen. It adds one LoRA that is active only on noise rows, the positions that hold a mask token and wait to be drafted. The draft pass takes the last token and 7 mask positions. The first row runs with the LoRA off and gives the exact next token. The 7 noise rows give the drafts. In the verify pass the original model, with the LoRA off, checks the 7 drafts and keeps the longest prefix it agrees with. At the first rejected position it emits its own token. If it accepts all 7, it emits the token after them. Each pass therefore yields one exact token, as in AR decoding, and the accepted drafts come on top. The verification rule is lossless: the output follows the original model's distribution exactly.

The speedup is TPF divided by the cost ratio

An Uno step keeps tokens from three sources.

SourceTokens
First row of the draft pass1
Accepted drafts0–7
Verify pass1

A step keeps between 2 and 9 tokens. A step that accepts 6 drafts keeps 8 tokens in 2 passes, so its TPF is 4. An AR step keeps 2 tokens in 2 passes, so its TPF is 1. At our TPF of 2.18 a step keeps 4.4 tokens on average: the 2 exact tokens and 2.4 drafts.

TPF counts tokens only. Time enters through the cost ratio, the time of an Uno step divided by the time of an AR step. The speedup over AR decoding is

speedup ≈ TPF ÷ cost ratio.

A step with TPF 2.18 and cost ratio 1.15 runs 2.18 ÷ 1.15 ≈ 1.9× faster than AR decoding. Drafting pays off once TPF exceeds the cost ratio.

Training is self-distillation

Each training sequence enters the batch twice. The clean copy goes through the frozen model and acts as the teacher. The block-noised copy goes through the model with the LoRA and acts as the student. The loss is the total-variation (TV) distance between the two next-token distributions on the response tokens. Under lossless verification a drafted token is accepted with probability 1 − TV, so the loss is the quantity decoding depends on. The dataset labels are never used. The data only supplies realistic contexts.

Q1: Does diffusion drafting fit a looped model, and what does the loop cost?

The answer is yes. Uno was designed for models that run each layer once per token. Our contribution is to extend it to the looped architecture, in training and in decoding.

Looped training. Uno's training code was written for Qwen3. We extended it to run a looped stack: the layers run twice per token, and the final norm closes each loop, as in Nanbeige4.2. With this change the model inside Uno's code gives the same outputs as the official Nanbeige4.2.

A LoRA that works in loop mode. The two loops share their weights, so the drafting LoRA sits on weights that every token uses twice. We train one LoRA through both loops. It is active in both and learns from both. It needs no loop-specific parameters and keeps the shape of the released LoRA.

Looped decoding. The draft-and-verify decoder now supports looped models. A layer sees different inputs in its first and second loop, so the decoder keeps one cache per loop while the weights stay shared.

The second loop costs 6.2 ms per token

All timings in Q1 and Q2 use 1 A100 GPU, 4 sequences, greedy decoding, CUDA graphs and up to 768 generated tokens per sequence, unless stated. Each value is a median over decode steps.

We time one forward pass of AR decoding. We also time Nanbeige4.2-3B with the second loop removed. Its output is then meaningless, and the timing isolates what the second loop costs.

ModelLoopsOne forward pass
Nanbeige4.2-3B, second loop removed17.0 ms
Nanbeige4.2-3B213.2 ms
Qwen3-8B113.3 ms

The second loop adds 6.2 ms, which is 89% on top of the one-loop pass. With it the 3B model takes the same time per token as Qwen3-8B. Each accepted draft saves one such pass.

Q2: Where does the time of a decode step go?

Two AR passesOutside the model7 extra rowsLoRA0 ms10 ms20 ms30 msNanbeige4.2-3B, second loop removedARAR step, Two AR passes: 14.0 msAR step, Outside the model: 0.7 ms14.7 msUnoUno step, Two AR passes: 14.0 msUno step, Outside the model: 0.9 msUno step, 7 extra rows: 0.3 msUno step, LoRA: 1.5 ms16.7 msNanbeige4.2-3BARAR step, Two AR passes: 26.4 msAR step, Outside the model: 0.7 ms27.1 msUnoUno step, Two AR passes: 26.4 msUno step, Outside the model: 0.9 msUno step, 7 extra rows: 0.6 msUno step, LoRA: 3.1 ms31.1 msQwen3-8BARAR step, Two AR passes: 26.5 msAR step, Outside the model: 0.8 ms27.3 msUnoUno step, Two AR passes: 26.5 msUno step, Outside the model: 0.9 msUno step, 7 extra rows: 1.4 msUno step, LoRA: 2.5 ms31.3 ms
Figure 1. Time of one decode step at short context, on 1 A100 with 4 sequences and greedy decoding. Each part is a median over decode steps, and the number at each bar is the median step time. An Uno step runs the same two passes as an AR step. The LoRA is the draft pass minus the verify pass. The 7 extra rows are what two 8-row passes add over two 1-row passes. On all three models the LoRA is the largest part of the overhead.

One Uno step costs 1.15 AR steps

The engine records the GPU time of each forward pass. The rest of a step is sampling, verification and bookkeeping outside the model.

Part of a stepAR stepUno stepExtra
First pass (the draft pass in Uno)13.2 ms16.6 ms3.4 ms
Second pass (the verify pass in Uno)13.2 ms13.5 ms0.3 ms
Outside the model0.7 ms0.9 ms0.2 ms
Whole step27.1 ms31.1 ms4.0 ms

Measured on Nanbeige4.2-3B with block size 8. An Uno step costs 4.0 ms more than an AR step, a cost ratio of 1.15×. It breaks even at TPF 1.15, when it keeps 2.3 tokens, 0.3 more than an AR step.

Verification is nearly free. The verify pass checks 7 drafts and costs 0.3 ms more than a pass that carries one token. At small batch a forward pass spends its time reading the weights, and the weights are read once for all rows.

The LoRA takes three quarters of the overhead

The draft pass and the verify pass carry the same 8 rows. Only the draft pass runs the LoRA. We repeat the Uno run with the LoRA switched off, and the two passes then cost the same: 13.48 ms and 13.45 ms. The gap between the draft pass and the verify pass is therefore the cost of the LoRA.

Source of extra timeExtra per step
LoRA in the draft pass3.1 ms
7 extra rows in both passes0.6 ms
Sampling and verification0.2 ms

The LoRA costs five times as much as the 7 extra rows it serves. Drafting is expensive because of the LoRA. The full-model passes underneath it are ones AR decoding pays for too.

The overhead doubles with the loop

We repeat the same split with the second loop removed, and on Qwen3-8B with its released LoRA at block size 8. Both LoRAs have rank 128 and cover the same seven projections in every layer.

ModelAR stepUno stepExtraCost ratioLoRA
Nanbeige4.2-3B, second loop removed14.7 ms16.7 ms2.0 ms1.14×1.5 ms
Nanbeige4.2-3B27.1 ms31.1 ms4.0 ms1.15×3.1 ms
Qwen3-8B27.3 ms31.3 ms4.1 ms1.15×2.5 ms

The second loop makes the AR step 1.85× longer and the drafting overhead 2.0× larger. The cost ratio stays at 1.15×, the same as for the standard 8B model. By this measure a looped model hosts drafting as cheaply as a standard one.

The LoRA costs about 0.07 ms per layer per loop in all three rows. The one-loop model applies 22 layers, Nanbeige4.2-3B applies 44 and Qwen3-8B applies 36. A looped model therefore pays for the LoRA once per loop. On Qwen3-8B the gap between the draft pass and the verify pass is 2.5 ms at block size 8 and 2.4 ms at block size 16. The LoRA cost follows the number of layers and loops and stays flat in the number of drafted rows.

The cost grows with the context

We let the same four sequences run for 4096 generated tokens and time the steps in order.

Generated tokensAR stepUno stepCost ratio
0–50026.9 ms30.7 ms1.14×
1500–200029.9 ms35.2 ms1.17×
3500–400030.3 ms41.4 ms1.37×

The LoRA costs 3.0 ms at both ends of the run. The growth sits in the extra rows. With the LoRA off, a pass with 8 rows costs 0.2 ms more than a pass with one row at the start of the run and 3.9 ms more at the end. Every row attends to the whole cached context, so the attention work of the extra rows grows with the context.

Sampling adds a second term. At temperature 1.0 the time outside the model is 2.2 ms for an AR step and 4.2 ms for an Uno step.

Our evaluations sample at temperature 1.0 and generate up to 4096 tokens. Under that setting the median step costs are:

ModelBlockAR stepUno stepCost ratio
Qwen3-8B1628.9 ms35.8 ms1.24×
Nanbeige4.2-3B828.8 ms37.1 ms1.29×

With a cost ratio of 1.29, the speedup formula gives TPF ÷ 1.29. We call this the per-step estimate of the speedup on Nanbeige4.2-3B.

Q3: How much speed does one hour of training buy?

The recipe

SettingReleased Uno (Qwen3-8B)LoopUno (Nanbeige4.2-3B)
Training tokens14.7B31M
Block-size scheduleprogressive, from 2fixed at 8
Learning rate1e-51e-4
Warmup–3 steps
Tokens per optimizer step0.52M0.52M
Hardware2 nodes × 8 GPUs1 node × 8 A100-40G

Our run uses one stage, one block size and a constant learning rate. It sees about 470 times fewer tokens than the released LoRA. One optimizer step takes 52 seconds, so 60 optimizer steps finish in 52 minutes.

Most of the gain arrives in the first hour

We evaluate on the first 200 problems of gsm8k at temperature 1.0.

Optimizer stepTraining timeTokensTPF
0 (AR)––1.00
87 min4M1.40
1614 min8M1.62
3228 min17M1.88
6052 min31M2.04
1201.7 h63M2.12
2403.5 h126M2.18
1.01.52.02.53.00 min30 min1 h2 h3 h4 htraining time, one 8×A100 nodeAR 1.00Qwen3-8B released adapter 2.79lr 1e-5lr 1e-4
Figure 2. Tokens per forward (TPF) on gsm8k against training time. With learning rate 1e-4 the curve passes 2 at 52 minutes. With 1e-5 it reaches the same level after 3.8 hours. Both flatten between 2.1 and 2.2.

TPF passes 2 inside the first hour. A decode step then keeps 4 tokens where an AR step keeps 2. Quadrupling the training time afterwards adds 0.14 TPF.

The released learning rate works and takes four times longer

With the released learning rate of 1e-5 the same model reaches TPF 1.75 at optimizer step 60 and 2.04 at step 260, after 3.8 hours. The run ends at 2.05 after 300 steps. Both learning rates end in the same range. A learning rate of 1e-4 gets there in about a quarter of the time.

The plateau

Both curves flatten near 2.2 tokens per pass. A decode step keeps two exact tokens, one from each pass, plus the drafts that survive verification. Verification keeps a prefix: the first rejected draft ends the step, and every draft after it is discarded.

SettingTPFDrafts per stepDrafts kept
Nanbeige4.2-3B, math2.1872.4
Nanbeige4.2-3B, function calls2.5873.2
Qwen3-8B, released LoRA2.79153.6

On math a step keeps about a third of its drafts. Suppose each draft is accepted with the same probability. Keeping 2.4 of 7 then needs a probability of about 0.73, and one step in ten accepts the whole block. The headroom lies in the acceptance rate, which measures how well the model's own next tokens can be predicted several positions ahead. Function calls are easier to predict, and they keep more drafts per step.

So one hour buys most of what this recipe gives on this model.

Three tasks

The LoRAs were trained only on math and reasoning data. We test them on that domain and on two agentic tasks.

TaskCheckpointTPFAR tok/sUno tok/sSpeedup
gsm8k52 min2.04154522741.47×
gsm8k3.5 h2.18154522341.45×
BFCL v352 min2.34165332731.98×
BFCL v33.5 h2.58165334642.10×
SWE replay52 min2.065388481.58×
SWE replay3.5 h2.345388711.62×

Measured on 8 A100 GPUs. BFCL v3 covers 1000 function-calling prompts under greedy decoding. SWE replay cuts 300 recorded SWE-agent trajectories at a random assistant turn and generates the next action, with a median prompt of 6.4k tokens.

The LoRA transfers to tool calls without seeing any. Function calls are highly structured, and they reach the highest TPF of the three tasks.

Accuracy follows the original model. On BFCL the AR decoder scores 87.2% and Uno 87.5%. On gsm8k the AR decoder scores 96.0% and the Uno checkpoints score between 93.0% and 96.0%. The gsm8k runs sample at temperature 1.0 on 200 problems, where the standard error is about 1.7 points.

Each task pays the step cost of its setting

The speedup formula also runs backwards. TPF divided by the measured speedup gives the cost ratio a task paid on average over the whole run, idle time included. We call it the implied cost ratio.

TaskAfter 52 minAfter 3.5 h
gsm8k1.391.51
BFCL v31.181.23
SWE replay1.311.45

BFCL decodes greedily, like the timing runs in Q2, and pays 1.18–1.23. Q2 measured 1.15× for greedy decoding at short context. SWE replay starts from a median prompt of 6.4k tokens and pays 1.31–1.45. Q2 measured 1.37× once the context passed 3.5k tokens.

On gsm8k the per-step estimate is TPF ÷ 1.29, and the measured speed trails it. At TPF 2.04 the estimate is 1.58× and the measurement is 1.47×. At TPF 2.18 the estimate is 1.69× and the measurement is 1.45×. The gap comes from the tail of each batch. A few long sequences keep running after the others finish, and the GPUs are underfilled during that time.

0.75×1.00×1.25×1.50×1.75×2.00×0 min30 min1 h2 h3 htraining time, one 8×A100 nodeAR 1×per-step estimatemeasured end to end
Figure 3. Speedup over AR decoding on gsm8k, learning rate 1e-4. The per-step estimate is TPF ÷ 1.29. Measured values are output tokens per second divided by the AR value on the same 200 problems.

Where it falls short

Early checkpoints are slower than AR. After 7 minutes of training the model keeps 1.40 tokens per pass and decodes at 0.91× of AR speed. The per-step estimate puts break-even at TPF 1.29. End to end, the first checkpoint that beats AR has TPF 1.62 and runs at 1.13×. In the run with learning rate 1e-5, a checkpoint with TPF 1.75 measured 1.02×.

More training raised TPF and left gsm8k speed flat. From 52 minutes to 3.5 hours TPF rises from 2.04 to 2.18. Measured speed moves from 1.47× to 1.45×. The implied cost ratio rises from 1.39 to 1.51, and we have not isolated the cause. The agentic tasks do gain: BFCL moves from 1.98× to 2.10× and SWE replay from 1.58× to 1.62×.

Single timings are noisy. In the run with learning rate 1e-5, the five checkpoints with TPF between 1.94 and 2.05 measured between 1.09× and 1.48×. TPF is the stable number in this post. Each tok/s value comes from one pass over 200 problems.

TPF stops at 2.2. A step keeps 2.4 of its 7 drafts. The released Qwen3-8B LoRA keeps 3.6 of 15 and reaches TPF 2.79.

Practical guidance

Limits of this study

Reproduce

Code and every script behind the numbers above are on GitHub. The adapters are on Hugging Face. The path from a fresh clone to the tables is six commands:

python nanbeige/build_uno_model.py models/nanbeige42_3b_uno 166107   # assemble the model folder
bash loopuno/prep_data.sh                                            # tokenize 60k OpenThoughts3 conversations
bash loopuno/train.sh                                                # 60 steps, lr 1e-4, block 8
bash loopuno/eval_gsm8k.sh checkpoints/loopuno-b8-lr1e-4             # TPF, tok/s, accuracy per checkpoint
bash loopuno/agentic/run_all.sh checkpoints/loopuno-b8-lr1e-4/checkpoint-60
python loopuno/bench_step_cost.py nanbeige 8 graph                   # per-step cost, AR vs Uno

References

  1. Subham Sekhar Sahoo, Lingjie Chen, Khiem Pham, Jonathan Geuter, Chaitanya Dwivedi, Varad Pimpalkhute, Yash Akhauri, Alexander Moreno, Mikhail Yurochkin, Zhenting Wang, Mostafa Elhoushi, Nolan Dey, Shane Bergsma, Joel Hestness, John Thickstun, Eric Xing, Zhengzhong Liu. Unlocking Lossless Speedups in LLMs via Discrete Diffusion. arXiv:2609.04010, 2026.
  2. Nanbeige LLM Lab. Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Model. arXiv:2607.22083, 2026.