Grounded diffusion VLA — self-correcting decode
A block-diffusion VLA commits answer tokens over several forward passes. With the self-correction objective it can un-commit and rewrite a token it already wrote, and commit many tokens at once and repair the mistakes — fewer passes, same answer.
1 · Rewriting an already-committed token mid-decode
▶ play the first block. blue=just committed grey=committed & kept red=challenged (re-masked) green=re-committed to a NEW token The chips list every token the model overwrote after first committing it.
Q:
To which vehicle should you yield? Why?
Input:
- camera: front camera
- navigation command: N/A
- ego state: N/A
Output:✓ correct · Lingo-Judge 0.631
reference: None. / I shouldn't yield to any vehicle: I'm going straight on a green traffic light on a protected intersection.
model answer: I should not yield to any specific vehicle as I am following a cyclist ahead of me, but I should be prepared to yield if the cyclist changes.
step 1 yield → step 4 notstep 2 the → step 6 any
Q:
Should you slow down and why?
Input:
- camera: front camera
- navigation command: N/A
- ego state: N/A
Output:✓ correct · Lingo-Judge 0.929
reference: No, clear road ahead. / I should not slow down as the cars in front of me are driving at the same speed (or slightly faster).
model answer: No, I should not slow down as the road is clear and there are no obstacles or hazards that would require for me to reduce my current speed.
step 15 maintain → step 18 reducestep 14 I → step 16 mestep 16 require → step 18 would
Q:
What is the current action and its justification? Answer in the form "action, justification".
Input:
- camera: front camera
- navigation command: N/A
- ego state: N/A
Output:✓ correct · Lingo-Judge 0.851
reference: Slowing down in our rightmost lane, the traffic light is green, we are entering a "right turn-only" lane and will need to wait for the traffic to make the planned right turn. / The car is slowing down
model answer: I am decelerating to let the pedestrian safely cross the road ahead as the traffic lights are green and it is the law to yield to pedestrians.
step 1 stopping → step 3 decstep 3 ahead → step 5 road
Q:
Can you proceed forward at this moment? Explain why or why not.
Input:
- camera: front camera
- navigation command: N/A
- ego state: N/A
Output:✓ correct · Lingo-Judge 0.908
reference: Once the pedestrians move out of the way it is then safe to proceed because the lights are green and road clear. / Despite the green traffic lights, I can only proceed once the pedestrians have cleare
model answer: Yes, I can safely proceed forward this moment as the traffic lights are green and there are no pedestrians crossing the road in the path of my vehicle.
step 2 proceed → step 4 safely
Q:
Are you legally allowed to make a right turn at this intersection?
Input:
- camera: front camera
- navigation command: N/A
- ego state: N/A
Output:✓ correct · Lingo-Judge 0.926
reference: No. The lane I am currently in only allows me to continue straight at this intersection. / No.
model answer: No, I am not legally allowed to turn right at this intersection as the traffic signs indicate that it is not allowed to turn right at this intersection.
step 13 right → step 15 turnstep 3 make → step 5 to
Q:
Is there a traffic light in the vicinity? If so, what color is it displaying?
Input:
- camera: front camera
- navigation command: N/A
- ego state: N/A
Output:✓ correct · Lingo-Judge 0.926
reference: There is a traffic light and it is red. / Yes, red.
model answer: Yes, there is a traffic light in the vicinity. It is displaying red, indicating that I should stop and wait for it to change to proceeding.
step 10 green → step 12 proceedingstep 10 before → step 12 to
2 · Commit many tokens at once, then self-correct → speedup
Same answer, three decode strategies. Fewer forward passes = faster. 1-shot commits every token in one pass (fast, but wrong); conservative denoises one token per step (right, but ~32 passes); self-correction commits in parallel and fixes its errors in a few extra passes — reaching the conservative answer far sooner.
Q:
What should you pay attention to and why?
Input:
- camera: front camera
- navigation command: N/A
- ego state: N/A
Output:self-correct: 2.9× fewer forwards
reference: The speed of the bus incase it slows down, or if any pedestrians appear from between the buses to avoid a collision. / The bus ahead of you to be able to match its speed. The busses to the left, incas
1-shot:I should pay attention to the bus lane on the left and andakingakingakingakingaking to a safe safe to to overtake the the it. (match 48%)
self-correct:I should pay attention to the bus lane on the left because I need to keep a safe distance from it and to to correctly centered in the lane. (match 96%)
conservative (slow):I should pay attention to the bus lane on the left because I need to keep a safe distance from it and to be correctly centered in the lane.
Q:
What is the current action and its justification? Answer in the form "action, justification".
Input:
- camera: front camera
- navigation command: N/A
- ego state: N/A
Output:self-correct: 2.7× fewer forwards
reference: The car slows down, because the traffic in front is stopped and a scooterist changes lane from the left lane to the right lane in front of it. / Decelerate, to slow and come to a stop behind a vehicle
1-shot:I am maintaining speed and and to to to to the the the the the the the the is is enough enough to the the to on overtake. (match 35%)
self-correct:I am maintaining speed and steering right to overtake the parked vehicle on the left because it is stopped and there is enough space on the overtake. (match 93%)
conservative (slow):I am maintaining speed and steering right to overtake the parked vehicle on the left because it is stopped and there is enough space on the right side.
Q:
Are you well positioned to go straight at this intersection?
Input:
- camera: front camera
- navigation command: N/A
- ego state: N/A
Output:self-correct: 2.5× fewer forwards
reference: Yes. / Yes, I am correctly positioned to go straight.
1-shot:Yes, I am positioned positioned positioned go straight at this as as lane lane lane lane lane lane lane lane lane lane the lane lane the the lane. (match 45%)
self-correct:Yes, I am well positioned to go straight at this intersection as I am in the left lane and there is no oncoming traffic in my lane. (match 100%)
conservative (slow):Yes, I am well positioned to go straight at this intersection as I am in the left lane and there is no oncoming traffic in my lane.
Q:
Is there a traffic light? If yes, what color is displayed?
Input:
- camera: front camera
- navigation command: N/A
- ego state: N/A
Output:self-correct: 2.3× fewer forwards
reference: Yes, green. / Yes, a temporary traffic light. It is showing green.
1-shot:Yes, there is a traffic light the the currently,...,,,,,,,,,, green green to to green. (match 38%)
self-correct:Yes, there is a traffic light on the right side of the road. The traffic light is currently green, indicating that it is safe to proceed. (match 100%)
conservative (slow):Yes, there is a traffic light on the right side of the road. The traffic light is currently green, indicating that it is safe to proceed.
Q:
How many pedestrians are crossing the road?
Input:
- camera: front camera
- navigation command: N/A
- ego state: N/A
Output:self-correct: 2.3× fewer forwards
reference: 2 / 2 pedestrians are crossing the road.
1-shot:There are two pedestrians crossing the road road, a a a crossing, the the,,,, the the the the the crossing crossing the road. (match 32%)
self-correct:There are two pedestrians crossing the road, a man in a white coat and a man in a black coat, both crossing on the zebra crossing. (match 100%)
conservative (slow):There are two pedestrians crossing the road, a man in a white coat and a man in a black coat, both crossing on the zebra crossing.
Q:
Do you have the priority to go straight at this intersection?
Input:
- camera: front camera
- navigation command: N/A
- ego state: N/A
Output:self-correct: 2.3× fewer forwards
reference: Yes. / Yes.
1-shot:Yes, I have the the to go straight at this intersection I the am the the the the the the the the the right right the the right. (match 48%)
self-correct:Yes, I have the priority to go straight at this intersection as I am in the left lane and there is no oncoming traffic in my lane. (match 100%)
conservative (slow):Yes, I have the priority to go straight at this intersection as I am in the left lane and there is no oncoming traffic in my lane.