LoopUno: Where Does the Time Go When a Looped LLM Drafts Its Own Tokens?
Diffusion drafting lets a language model guess several of its own next tokens in one pass and keep the ones it would have sampled anyway, so the output is unchanged. Each decode step runs the full model twice, once with a LoRA to draft and once to verify, and it still decodes faster than standard decoding. We time every part of the step to see why. Verifying 7 drafts adds 0.3 ms to a 13 ms pass. The LoRA adds 3.1 ms, three quarters of the overhead, so a drafting step costs 1.15× a standard two-pass step. The speedup is the tokens kept per forward pass divided by this cost ratio. We propose LoopUno, an adaptation of Uno that brings diffusion drafting to looped transformers, and validate it on the looped agentic model Nanbeige4.2-3B. The second loop makes each forward pass 1.9× slower and doubles the drafting overhead with it, so the cost ratio stays at 1.15×. After one hour of LoRA training a step keeps 4 tokens where standard decoding keeps 2, and function calls decode 2× faster at matching accuracy.
Diffusion drafting lets a language model guess several of its own next tokens in one forward pass. Uno[1] does this with a LoRA, in the style of a discrete diffusion model. The original model then checks the guesses and keeps the tokens it would have sampled itself, so the samples are unchanged by construction. The drafter is the full model with the LoRA switched on, so every decode step runs the full model twice: once to draft and once to verify. Why is this faster at all?
We timed every part of a decode step to find out. Verification is nearly free: checking 7 drafts adds 0.3 ms to a 13 ms forward pass. Drafting costs a LoRA: 3.1 ms per step, three quarters of the overhead. A drafting step takes 1.15× as long as two standard forward passes. After one hour of training it keeps about 4 tokens where the two standard passes keep 2. Tokens per forward (TPF) is the number of tokens a step keeps divided by its forward passes. The cost ratio is the time of a drafting step divided by the time of two standard passes. The speedup is TPF ÷ cost ratio.
We ask this question on a looped model. A looped model stores one stack of layers and runs it more than once for every token. Each run through the stack is a loop. Nanbeige4.2-3B[2] stores 22 layers and runs two loops. The second loop adds capacity without adding parameters. With 3B parameters the model outperforms 9B and 12B models on agentic benchmarks[2]. The price is time. Every token costs two loops, so this 3B model decodes as slowly per token as an 8B model.
A looped model buys capability with time per token. A diffusion drafter buys time back with tokens per pass. Uno was built for standard layer stacks. In a looped model one LoRA has to serve weights that every token uses twice, and the draft has to survive two loops. The LoRA also runs once per loop, so its cost doubles. The forward pass doubles too, and the cost ratio stays at 1.15×, the same as on the standard Qwen3-8B. A looped model can keep its small memory footprint and recover part of its latency.
Nanbeige4.2-3B gives a second reason to try. It is an agentic model, trained for function calling and code-agent work. Agents are where decoding speed is felt most: one task takes many tool calls, and every call waits on the decoder. Agents are also where a speedup must leave the output alone, because one changed argument breaks a tool call. A lossless drafter fits this setting. So we measure the speedup on function calls and agent trajectories as well as on math, with a LoRA that saw only math and reasoning data.
Our training budget is one hour on one 8×A100 node. The released Uno LoRA for Qwen3-8B used 14.7B training tokens. Ours uses 31M.
Three questions
- Does diffusion drafting fit a looped model, and what does the loop cost? We extend Uno's training and decoding to the looped architecture and time the second loop.
- Where does the time of a decode step go? We split the cost ratio into the draft pass, the verify pass and the work outside the model, and we separate the LoRA from the extra rows.
- How much speed does one hour of training buy? We measure TPF on math, function calling and agent trajectories, compare the measured speed with TPF ÷ cost ratio, and report the cases that came out slower or flat.
Background
Draft with a LoRA, verify with the original model
- AR decoding is standard decoding: one forward pass yields one token.
- A block is the group of positions a draft pass fills: one exact token and 7 drafts. Our block size is 8.
- A decode step is two forward passes of the full model. In an Uno step the first is a draft pass and the second is a verify pass. Our engine runs AR decoding in steps of two passes as well, so an AR step yields two tokens and its time compares directly with an Uno step.
- Tokens per forward (TPF) is the number of tokens kept divided by the number of forward passes. A decode step keeps 2 × TPF tokens. AR decoding has TPF 1.00.
Uno keeps the base model frozen. It adds one LoRA that is active only on noise rows, the positions that hold a mask token and wait to be drafted. The draft pass takes the last token and 7 mask positions. The first row runs with the LoRA off and gives the exact next token. The 7 noise rows give the drafts. In the verify pass the original model, with the LoRA off, checks the 7 drafts and keeps the longest prefix it agrees with. At the first rejected position it emits its own token. If it accepts all 7, it emits the token after them. Each pass therefore yields one exact token, as in AR decoding, and the accepted drafts come on top. The verification rule is lossless: the output follows the original model's distribution exactly.
The speedup is TPF divided by the cost ratio
An Uno step keeps tokens from three sources.
| Source | Tokens |
|---|---|
| First row of the draft pass | 1 |
| Accepted drafts | 0–7 |
| Verify pass | 1 |
A step keeps between 2 and 9 tokens. A step that accepts 6 drafts keeps 8 tokens in 2 passes, so its TPF is 4. An AR step keeps 2 tokens in 2 passes, so its TPF is 1. At our TPF of 2.18 a step keeps 4.4 tokens on average: the 2 exact tokens and 2.4 drafts.
TPF counts tokens only. Time enters through the cost ratio, the time of an Uno step divided by the time of an AR step. The speedup over AR decoding is
speedup ≈ TPF ÷ cost ratio.
A step with TPF 2.18 and cost ratio 1.15 runs 2.18 ÷ 1.15 ≈ 1.9× faster than AR decoding. Drafting pays off once TPF exceeds the cost ratio.
Training is self-distillation
Each training sequence enters the batch twice. The clean copy goes through the frozen model and acts as the teacher. The block-noised copy goes through the model with the LoRA and acts as the student. The loss is the total-variation (TV) distance between the two next-token distributions on the response tokens. Under lossless verification a drafted token is accepted with probability 1 − TV, so the loss is the quantity decoding depends on. The dataset labels are never used. The data only supplies realistic contexts.
Q1: Does diffusion drafting fit a looped model, and what does the loop cost?
The answer is yes. Uno was designed for models that run each layer once per token. Our contribution is to extend it to the looped architecture, in training and in decoding.
Looped training. Uno's training code was written for Qwen3. We extended it to run a looped stack: the layers run twice per token, and the final norm closes each loop, as in Nanbeige4.2. With this change the model inside Uno's code gives the same outputs as the official Nanbeige4.2.
A LoRA that works in loop mode. The two loops share their weights, so the drafting LoRA sits on weights that every token uses twice. We train one LoRA through both loops. It is active in both and learns from both. It needs no loop-specific parameters and keeps the shape of the released LoRA.
Looped decoding. The draft-and-verify decoder now supports looped models. A layer sees different inputs in its first and second loop, so the decoder keeps one cache per loop while the weights stay shared.
The second loop costs 6.2 ms per token
All timings in Q1 and Q2 use 1 A100 GPU, 4 sequences, greedy decoding, CUDA graphs and up to 768 generated tokens per sequence, unless stated. Each value is a median over decode steps.
We time one forward pass of AR decoding. We also time Nanbeige4.2-3B with the second loop removed. Its output is then meaningless, and the timing isolates what the second loop costs.
| Model | Loops | One forward pass |
|---|---|---|
| Nanbeige4.2-3B, second loop removed | 1 | 7.0 ms |
| Nanbeige4.2-3B | 2 | 13.2 ms |
| Qwen3-8B | 1 | 13.3 ms |
The second loop adds 6.2 ms, which is 89% on top of the one-loop pass. With it the 3B model takes the same time per token as Qwen3-8B. Each accepted draft saves one such pass.
Q2: Where does the time of a decode step go?
One Uno step costs 1.15 AR steps
The engine records the GPU time of each forward pass. The rest of a step is sampling, verification and bookkeeping outside the model.
| Part of a step | AR step | Uno step | Extra |
|---|---|---|---|
| First pass (the draft pass in Uno) | 13.2 ms | 16.6 ms | 3.4 ms |
| Second pass (the verify pass in Uno) | 13.2 ms | 13.5 ms | 0.3 ms |
| Outside the model | 0.7 ms | 0.9 ms | 0.2 ms |
| Whole step | 27.1 ms | 31.1 ms | 4.0 ms |
Measured on Nanbeige4.2-3B with block size 8. An Uno step costs 4.0 ms more than an AR step, a cost ratio of 1.15×. It breaks even at TPF 1.15, when it keeps 2.3 tokens, 0.3 more than an AR step.
Verification is nearly free. The verify pass checks 7 drafts and costs 0.3 ms more than a pass that carries one token. At small batch a forward pass spends its time reading the weights, and the weights are read once for all rows.
The LoRA takes three quarters of the overhead
The draft pass and the verify pass carry the same 8 rows. Only the draft pass runs the LoRA. We repeat the Uno run with the LoRA switched off, and the two passes then cost the same: 13.48 ms and 13.45 ms. The gap between the draft pass and the verify pass is therefore the cost of the LoRA.
| Source of extra time | Extra per step |
|---|---|
| LoRA in the draft pass | 3.1 ms |
| 7 extra rows in both passes | 0.6 ms |
| Sampling and verification | 0.2 ms |
The LoRA costs five times as much as the 7 extra rows it serves. Drafting is expensive because of the LoRA. The full-model passes underneath it are ones AR decoding pays for too.
The overhead doubles with the loop
We repeat the same split with the second loop removed, and on Qwen3-8B with its released LoRA at block size 8. Both LoRAs have rank 128 and cover the same seven projections in every layer.
| Model | AR step | Uno step | Extra | Cost ratio | LoRA |
|---|---|---|---|---|---|
| Nanbeige4.2-3B, second loop removed | 14.7 ms | 16.7 ms | 2.0 ms | 1.14× | 1.5 ms |
| Nanbeige4.2-3B | 27.1 ms | 31.1 ms | 4.0 ms | 1.15× | 3.1 ms |
| Qwen3-8B | 27.3 ms | 31.3 ms | 4.1 ms | 1.15× | 2.5 ms |
The second loop makes the AR step 1.85× longer and the drafting overhead 2.0× larger. The cost ratio stays at 1.15×, the same as for the standard 8B model. By this measure a looped model hosts drafting as cheaply as a standard one.
The LoRA costs about 0.07 ms per layer per loop in all three rows. The one-loop model applies 22 layers, Nanbeige4.2-3B applies 44 and Qwen3-8B applies 36. A looped model therefore pays for the LoRA once per loop. On Qwen3-8B the gap between the draft pass and the verify pass is 2.5 ms at block size 8 and 2.4 ms at block size 16. The LoRA cost follows the number of layers and loops and stays flat in the number of drafted rows.
The cost grows with the context
We let the same four sequences run for 4096 generated tokens and time the steps in order.
| Generated tokens | AR step | Uno step | Cost ratio |
|---|---|---|---|
| 0–500 | 26.9 ms | 30.7 ms | 1.14× |
| 1500–2000 | 29.9 ms | 35.2 ms | 1.17× |
| 3500–4000 | 30.3 ms | 41.4 ms | 1.37× |
The LoRA costs 3.0 ms at both ends of the run. The growth sits in the extra rows. With the LoRA off, a pass with 8 rows costs 0.2 ms more than a pass with one row at the start of the run and 3.9 ms more at the end. Every row attends to the whole cached context, so the attention work of the extra rows grows with the context.
Sampling adds a second term. At temperature 1.0 the time outside the model is 2.2 ms for an AR step and 4.2 ms for an Uno step.
Our evaluations sample at temperature 1.0 and generate up to 4096 tokens. Under that setting the median step costs are:
| Model | Block | AR step | Uno step | Cost ratio |
|---|---|---|---|---|
| Qwen3-8B | 16 | 28.9 ms | 35.8 ms | 1.24× |
| Nanbeige4.2-3B | 8 | 28.8 ms | 37.1 ms | 1.29× |
With a cost ratio of 1.29, the speedup formula gives TPF ÷ 1.29. We call this the per-step estimate of the speedup on Nanbeige4.2-3B.
Q3: How much speed does one hour of training buy?
The recipe
| Setting | Released Uno (Qwen3-8B) | LoopUno (Nanbeige4.2-3B) |
|---|---|---|
| Training tokens | 14.7B | 31M |
| Block-size schedule | progressive, from 2 | fixed at 8 |
| Learning rate | 1e-5 | 1e-4 |
| Warmup | – | 3 steps |
| Tokens per optimizer step | 0.52M | 0.52M |
| Hardware | 2 nodes × 8 GPUs | 1 node × 8 A100-40G |
Our run uses one stage, one block size and a constant learning rate. It sees about 470 times fewer tokens than the released LoRA. One optimizer step takes 52 seconds, so 60 optimizer steps finish in 52 minutes.
Most of the gain arrives in the first hour
We evaluate on the first 200 problems of gsm8k at temperature 1.0.
| Optimizer step | Training time | Tokens | TPF |
|---|---|---|---|
| 0 (AR) | – | – | 1.00 |
| 8 | 7 min | 4M | 1.40 |
| 16 | 14 min | 8M | 1.62 |
| 32 | 28 min | 17M | 1.88 |
| 60 | 52 min | 31M | 2.04 |
| 120 | 1.7 h | 63M | 2.12 |
| 240 | 3.5 h | 126M | 2.18 |
TPF passes 2 inside the first hour. A decode step then keeps 4 tokens where an AR step keeps 2. Quadrupling the training time afterwards adds 0.14 TPF.
The released learning rate works and takes four times longer
With the released learning rate of 1e-5 the same model reaches TPF 1.75 at optimizer step 60 and 2.04 at step 260, after 3.8 hours. The run ends at 2.05 after 300 steps. Both learning rates end in the same range. A learning rate of 1e-4 gets there in about a quarter of the time.
The plateau
Both curves flatten near 2.2 tokens per pass. A decode step keeps two exact tokens, one from each pass, plus the drafts that survive verification. Verification keeps a prefix: the first rejected draft ends the step, and every draft after it is discarded.
| Setting | TPF | Drafts per step | Drafts kept |
|---|---|---|---|
| Nanbeige4.2-3B, math | 2.18 | 7 | 2.4 |
| Nanbeige4.2-3B, function calls | 2.58 | 7 | 3.2 |
| Qwen3-8B, released LoRA | 2.79 | 15 | 3.6 |
On math a step keeps about a third of its drafts. Suppose each draft is accepted with the same probability. Keeping 2.4 of 7 then needs a probability of about 0.73, and one step in ten accepts the whole block. The headroom lies in the acceptance rate, which measures how well the model's own next tokens can be predicted several positions ahead. Function calls are easier to predict, and they keep more drafts per step.
So one hour buys most of what this recipe gives on this model.
Three tasks
The LoRAs were trained only on math and reasoning data. We test them on that domain and on two agentic tasks.
| Task | Checkpoint | TPF | AR tok/s | Uno tok/s | Speedup |
|---|---|---|---|---|---|
| gsm8k | 52 min | 2.04 | 1545 | 2274 | 1.47× |
| gsm8k | 3.5 h | 2.18 | 1545 | 2234 | 1.45× |
| BFCL v3 | 52 min | 2.34 | 1653 | 3273 | 1.98× |
| BFCL v3 | 3.5 h | 2.58 | 1653 | 3464 | 2.10× |
| SWE replay | 52 min | 2.06 | 538 | 848 | 1.58× |
| SWE replay | 3.5 h | 2.34 | 538 | 871 | 1.62× |
Measured on 8 A100 GPUs. BFCL v3 covers 1000 function-calling prompts under greedy decoding. SWE replay cuts 300 recorded SWE-agent trajectories at a random assistant turn and generates the next action, with a median prompt of 6.4k tokens.
The LoRA transfers to tool calls without seeing any. Function calls are highly structured, and they reach the highest TPF of the three tasks.
Accuracy follows the original model. On BFCL the AR decoder scores 87.2% and Uno 87.5%. On gsm8k the AR decoder scores 96.0% and the Uno checkpoints score between 93.0% and 96.0%. The gsm8k runs sample at temperature 1.0 on 200 problems, where the standard error is about 1.7 points.
Each task pays the step cost of its setting
The speedup formula also runs backwards. TPF divided by the measured speedup gives the cost ratio a task paid on average over the whole run, idle time included. We call it the implied cost ratio.
| Task | After 52 min | After 3.5 h |
|---|---|---|
| gsm8k | 1.39 | 1.51 |
| BFCL v3 | 1.18 | 1.23 |
| SWE replay | 1.31 | 1.45 |
BFCL decodes greedily, like the timing runs in Q2, and pays 1.18–1.23. Q2 measured 1.15× for greedy decoding at short context. SWE replay starts from a median prompt of 6.4k tokens and pays 1.31–1.45. Q2 measured 1.37× once the context passed 3.5k tokens.
On gsm8k the per-step estimate is TPF ÷ 1.29, and the measured speed trails it. At TPF 2.04 the estimate is 1.58× and the measurement is 1.47×. At TPF 2.18 the estimate is 1.69× and the measurement is 1.45×. The gap comes from the tail of each batch. A few long sequences keep running after the others finish, and the GPUs are underfilled during that time.
Where it falls short
Early checkpoints are slower than AR. After 7 minutes of training the model keeps 1.40 tokens per pass and decodes at 0.91× of AR speed. The per-step estimate puts break-even at TPF 1.29. End to end, the first checkpoint that beats AR has TPF 1.62 and runs at 1.13×. In the run with learning rate 1e-5, a checkpoint with TPF 1.75 measured 1.02×.
More training raised TPF and left gsm8k speed flat. From 52 minutes to 3.5 hours TPF rises from 2.04 to 2.18. Measured speed moves from 1.47× to 1.45×. The implied cost ratio rises from 1.39 to 1.51, and we have not isolated the cause. The agentic tasks do gain: BFCL moves from 1.98× to 2.10× and SWE replay from 1.58× to 1.62×.
Single timings are noisy. In the run with learning rate 1e-5, the five checkpoints with TPF between 1.94 and 2.05 measured between 1.09× and 1.48×. TPF is the stable number in this post. Each tok/s value comes from one pass over 200 problems.
TPF stops at 2.2. A step keeps 2.4 of its 7 drafts. The released Qwen3-8B LoRA keeps 3.6 of 15 and reaches TPF 2.79.
Practical guidance
- Check parity first. Compare logits with the official implementation under the attention backend used in training. Self-distillation hides a wrong port.
- Look for a norm inside the loop. For a looped model, find out whether the final norm closes every loop.
- Start with one stage. A fixed block size of 8 and learning rate 1e-4 gave TPF 2 in one hour here.
- Stop early. On this model, training past one hour bought 0.14 TPF in three hours.
- Time the two passes separately. The split shows whether the LoRA, the extra rows or the sampler holds the overhead. Here the LoRA held three quarters of it.
- Measure the step cost at your context length. It was 1.15× at short context and 1.37× after 4k tokens here.
- Report TPF and tok/s together. TPF is stable across runs. tok/s decides whether users see a speedup. TPF divided by the speedup gives the cost ratio a run actually paid.
- Expect more on structured output. Function calls gained the most.
Limits of this study
- Step timings use 1 GPU, 4 sequences and 4 gsm8k prompts. Larger batches were not timed.
- The one-loop timing runs the looped weights with the second loop removed. It measures time only.
- The agentic results score single function-calling turns and replay recorded trajectories. The actions are generated and never executed, so SWE replay measures speed only.
- Each gsm8k point uses 200 problems and one timing pass.
- BFCL accuracy uses a simplified AST checker, applied identically to both decoders. Absolute scores may differ from the official leaderboard.
- The study covers one looped model.
Reproduce
Code and every script behind the numbers above are on GitHub. The adapters are on Hugging Face. The path from a fresh clone to the tables is six commands:
python nanbeige/build_uno_model.py models/nanbeige42_3b_uno 166107 # assemble the model folder
bash loopuno/prep_data.sh # tokenize 60k OpenThoughts3 conversations
bash loopuno/train.sh # 60 steps, lr 1e-4, block 8
bash loopuno/eval_gsm8k.sh checkpoints/loopuno-b8-lr1e-4 # TPF, tok/s, accuracy per checkpoint
bash loopuno/agentic/run_all.sh checkpoints/loopuno-b8-lr1e-4/checkpoint-60
python loopuno/bench_step_cost.py nanbeige 8 graph # per-step cost, AR vs UnoReferences
- Subham Sekhar Sahoo, Lingjie Chen, Khiem Pham, Jonathan Geuter, Chaitanya Dwivedi, Varad Pimpalkhute, Yash Akhauri, Alexander Moreno, Mikhail Yurochkin, Zhenting Wang, Mostafa Elhoushi, Nolan Dey, Shane Bergsma, Joel Hestness, John Thickstun, Eric Xing, Zhengzhong Liu. Unlocking Lossless Speedups in LLMs via Discrete Diffusion. arXiv:2609.04010, 2026.
- Nanbeige LLM Lab. Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Model. arXiv:2607.22083, 2026.