LabeloxAV

A data engine for autonomous driving, built for Indian roads

Status: Live
Version: 0.1.1
Platform: GPU workstation
Source: github.com/Sherin-SEF-AI/LabeloxAV
License: Apache-2.0

01 · Problem

Global driving datasets never saw much of what is on Indian roads: autorickshaws, cattle on the carriageway, overloaded two-wheelers, hand carts, potholes. LabeloxAV turns raw footage into training data by machine-labelling it, sending uncertain and safety-critical labels to human review, and promoting a retrained model only through a governance gate.

02 · Architecture

How it is put together.

LabeloxAV takes raw driving footage, machine-labels it through a three-path fusion pipeline, routes every label through a confidence gate to human review, mines the rare and risky moments, and retrains its own models in a closed loop. It uses one ontology of 178 governed classes, tuned for things global datasets rarely cover: autorickshaws, cattle on the carriageway, overloaded two-wheelers, hand carts and potholes. A new model is promoted only when it beats the current one against a sealed gold set without regressing a safety class.

FIG. 01Modules and data flows, from the repository layout and documentation.

1
Ingest
Decodes video and MCAP into frames, runs the quality gate, and writes a per-frame PII anonymisation audit
services/ingest/
2
Auto-label
Runs the detection stack and writes machine-sourced objects with confidence and provenance
services/autolabel/
3
Confidence gate and promotion
Auto-accepts confident agreed detections, routes the rest to review, and runs the champion/challenger promotion gate
services/govern/
4
API and review
FastAPI backend with deny-by-default token auth; review confirms, reclassifies, adjusts or rejects objects
services/api/
5
Active learning
Builds the next review batch from the gate's per-class deficits and the error detectors
services/activelearn/
6
Web app
Next.js App Router frontend that reads and writes only through the API
web/
7
Domain packs
Domain specifics (ontology, scene model, ingest) behind a registry; AV and security packs
packs/

03 · Capabilities

Specification.

Auto-labelling paths: Closed-set detector (YOLO11), open-vocabulary detection and segmentation (YOLO-World + SAM 2), VLM verifier (Qwen2.5-VL via Ollama)
Confidence gate: Isotonic-calibrated confidence routes each object to auto-accept, review or annotate; safety classes use a separate threshold
Ontology: One ontology with 178 governed classes, India-specific classes included
Governance: Append-only prediction plane, sealed gold sets, a champion/challenger promotion gate with safety-class floors
Active learning: Candidates ranked by uncertainty, diversity, rarity and error-proneness to build the next review batch
Modalities: Dashcam video and MCAP ingest with face and plate blurring, 2D annotation, lanes, LiDAR bird's-eye cuboid annotation, HD map layers
Interchange formats: COCO, YOLO, Pascal VOC, Mapillary Vistas, KITTI, BDD100K, OpenLABEL, nuScenes, CVAT XML, Label Studio JSON, Parquet
Deployment: Docker Compose install script or Debian package; annotation, review, governance, export and search work without a GPU
Runs on: FastAPI + Next.js web app on Docker Compose needing about 8 GB RAM and about 4.5 GB of images; its auto-labelling, embedding and training paths need a CUDA GPU