VTLA Plan B: Why Evo-RL Should Not Be the Main Framework

Evo-RL is valuable, but it belongs late in the stack

After looking through Evo-RL, my conclusion is simple: it is useful for real-robot post-training and correction loops, but it should not be the main framework if the end goal is VTLA (Vision-Tactile-Language-Action).

Where Evo-RL should sit

L1  Simulation & data generation
L2  Visuo-tactile policy learning
L3  Sim2Real
L4  Real-robot deployment & continual improvement

Best place for Evo-RL: L4

Evo-RL is strongest when you already have a working policy and want rollout, human intervention, correction, value/advantage estimation, and retraining on real hardware.

Recommended Plan B

My more concrete answer is:

L1  Visuo-tactile simulation + benchmark
    -> UniVTAC first
    -> TacSL if you need Isaac-Lab-native, GPU-heavy tactile simulation

L2  Policy backbone
    -> Diffusion Policy
    -> vision + tactile + proprio fusion
    -> action chunking in continuous space

L3  Contact-aware upgrade
    -> tactile-force alignment / force-aware adapter
    -> use when insertion fails because of force misuse, not because of perception

L4  Real-robot continual learning
    -> Evo-RL after deployment
    -> rollout -> intervention -> value/advantage -> retraining

• UniVTAC first: it already unifies data generation, encoder pretraining, and benchmark tasks such as Insert Hole and Insert HDMI.
• TacSL as an alternative: choose it when your team is already standardized on Isaac Lab and you care more about tactile simulation throughput than unified benchmark design.
• Diffusion Policy first: it handles multimodal continuous control and action chunking better than simple MSE regression, which matters a lot for insertion-style correction.
• Force-aware upgrade second: once the robot can reach and touch reliably, add a tactile-force alignment module inspired by force-aware VTLA work, because tactile is most valuable when it becomes a force/control signal rather than just “another image”.
• Evo-RL last: only after a working policy exists, use rollout, intervention, value, advantage, and retraining on real hardware.

Why this stack is safer than “Evo-RL first”

The key systems insight is that tactile should not enter the stack at the same place as language and high-level semantics.

Language / task
   -> high-level goal and stage selection

Vision
   -> global localization, approach, coarse alignment

Tactile / force
   -> contact onset, slip, over-force, peg-hole micro-correction

Evo-RL
   -> post-deployment data flywheel
   -> not the first source of competence

• Vision solves “where”, tactile solves “what is happening after contact”.
• Diffusion Policy solves “how to output a smooth corrective action chunk”.
• Evo-RL solves “how to keep getting better after deployment”.

If you start from Evo-RL directly, you mix together perception errors, contact-model errors, action-generation errors, calibration errors, and online learning instability. That makes debugging too expensive for a first VTLA system.

The first task I would choose

I would still start with single-arm + two-finger tactile gripper + blind insertion, but I would define it more tightly:

• Task: peg insertion / HDMI-style insertion / USB-C dummy insertion.
• Robot: one arm, no dexterous hand yet.
• Tactile: start with contact depth / contact area / binary contact or coarse tactile, not maximum sensor complexity on day one.
• Observations: fixed camera + wrist camera + proprio + tactile stream.
• Action space: delta end-effector pose + gripper command, chunked over a short horizon.

This task is small enough to finish, but rich enough to reveal whether your tactile signal is actually useful after vision reaches its limit.

Concrete acceptance criteria

• Simulation success: ≥ 85% under random pose perturbation.
• Vision-only gap: VTLA should beat a vision-only policy by at least 10 to 15 points on insertion success.
• Contact quality: fewer penetration-like failures, fewer jam events, fewer unstable retries.
• Real-robot success: ≥ 70% over 30 consecutive trials with small pose mismatch and mild occlusion.
• Ablation requirement: if removing tactile barely changes results, your “VTLA” stack is not really using tactile yet.

Why not start from Evo-RL directly?

Because that mixes every hard problem together at the same time: tactile sensing, contact simulation, policy design, calibration, deployment, and RL loops. If something fails, you will not know whether the issue is the tactile model, the simulation, the backbone, or the post-training loop.

A safer sequence is:

Prove VTLA in simulation
-> deploy a minimal real-robot version
-> then add Evo-RL for continual improvement

A practical 3-month path

• Month 1: build the minimum stack. One task, one simulator, one policy, one tactile representation. Finish the benchmark and prove the policy can close the loop after contact.
• Month 2: harden the stack. Run robustness tests on pose error, occlusion, slip, and contact-force failure. Add a force-aware adapter only if contact misuse is now the bottleneck.
• Month 3: move to the smallest real setup. Keep the task identical, preserve logging, then connect Evo-RL only after the base policy can already complete the task sometimes without online learning.

The exact Plan B I would defend to a research lead

Phase 1: UniVTAC-style visuo-tactile simulation and benchmark.
Phase 2: Diffusion Policy baseline with vision + tactile + proprio fusion.
Phase 3: force-aware tactile upgrade if insertion failure is mostly about force misuse.
Phase 4: Evo-RL-style real-robot continual improvement loop.

That is the lowest-risk route because each layer answers a different question:

• UniVTAC / TacSL: can we generate and evaluate contact-rich data properly?
• Diffusion Policy: can we turn multimodal state into stable corrective actions?
• Tactile-force alignment: can we make contact signals physically meaningful?
• Evo-RL: can the deployed robot learn from failure and human correction?