DexVantage

Camera, depth and teleop planning for robot-learning data collection

Status: Beta
Version: No tagged release
Platform: Desktop app
Source: github.com/Sherin-SEF-AI/DexVantage
License: GPL-3.0

01 · Problem

Robot-learning policies depend on demonstrations whose cameras actually see the fingertips, object and contact at the moment of grasp, with valid depth where the policy needs it. DexVantage answers these questions before a lab builds or changes its rig, and again after the rig has recorded.

02 · Architecture

How it is put together.

DexVantage is a Linux desktop application for planning and checking the camera, depth and tactile sensing of robot manipulation and humanoid data-collection rigs. It shows where wrist, head, overhead and third-person cameras should go so the fingertips, object and contact point stay visible at the grasp, where depth is valid, and what the teleoperator can reach. After recording, it audits LeRobot, RLDS and ROS 2 MCAP datasets against the plan and exports matching cameras to MuJoCo, Isaac Lab, LeRobot and ROS 2. It is the sibling of Vantage, built to the same engineering standards.

FIG. 01Modules and data flows, from the repository layout and documentation.

1
Desktop UI
Qt main window, 3D viewport and shader renderer, and one analysis page per study
dexvantage/ui/
2
Studies
Each study is registered once and becomes a window page and a batch flag (coverage, grasp, depth validity, teleop, bandwidth and others)
dexvantage/core/studies/
3
Robots and kinematics
URDF/MJCF/USD robots posed by MuJoCo and checked against an independent product-of-exponentials chain
dexvantage/core/robot/
4
Datasets and audit
LeRobot, RLDS and ROS 2 bag readers and writers, and the dataset auditor
dexvantage/core/datasets/
5
Exporters
MJCF, URDF, OpenCV, Kalibr, Isaac/USD, LeRobot and report; the registry writes a manifest with SHA-256 and provenance
dexvantage/core/exporters/
6
Validation suite
Checks each model against an independent reference with a stated tolerance and writes the HTML report
dexvantage/validation/
7
Assistant and MCP server
Assistant dock (Claude provider) and an MCP server over stdio for MCP clients
dexvantage/mcp/

03 · Capabilities

Specification.

Robots: URDF, MJCF, USD and MuJoCo Menagerie models; forward and inverse kinematics; six bundled samples (single arm, bimanual, dexterous hand, humanoid upper body)
Visibility studies: Grasp self-occlusion with a Contact Visibility Score, task-volume coverage and resolution, multi-finger occlusion matrix, VLA token resolution
Depth and optics: Depth validity (min-Z, finger shadows, projector cone, sigma_z), lens models (pinhole, Brown-Conrady, Kannala-Brandt and others), lighting and glare, motion blur and rolling shutter
Rig engineering: Bandwidth, hosts and power; cable flex across joints; printable brackets; teleop overlap and dead zones; action-chunk latency
Calibration: Hand-eye calibration (ChArUco and AprilGrid, AX = XB and AX = ZB, bundle adjustment, pose planner) and targetless link-silhouette calibration
Datasets: LeRobot v2.1/v3.0 readers and writers, ROS 2 bag reader (MCAP, SQLite) and MCAP writer, RLDS reader, dataset auditor, campaign planner
Exports: MJCF, URDF and tf, OpenCV and Kalibr, Isaac Lab and USD, LeRobot, masks, report and a publication kit; each export carries a provenance block
Validation and automation: Batch flag per study with CI exit codes, Python API, MCP server over stdio; 239 of 239 validation checks passed at the 2026-11-08 gate (dexvantage --validate)
Runs on: PyQt6 desktop application for Linux workstations, with batch mode for CI or a Jetson