ISOFRAMES
GitHub

FrameForge

An annotation tool for dashcam footage of Indian roads

Source on GitHub All systems
Status
Research
Version
No tagged release
Platform
Desktop app
Source
github.com/Sherin-SEF-AI/FrameForge
License
Not specified
01 · Problem

Common driving datasets were collected outside India, and models trained on them have not seen objects like auto-rickshaws, cattle or deep potholes. Labeling local dashcam footage by hand at polygon level is slow and costly, so FrameForge auto-labels footage and puts a human in the loop to correct it.

02 · Architecture

How it is put together.

FrameForge is a PyQt6 desktop tool that runs vision models on dashcam video to generate pseudo-labels, then gives one engineer a correction and review interface to fix them. The pipeline chains YOLO11-seg, a SAM box refiner, text-prompted Grounded SAM and SegFormer semantic segmentation, followed by human review, export to five label formats, and one-click YOLO fine-tuning whose best.pt can be loaded back in. It ships a 16-class Indian road taxonomy that includes potholes, speed bumps, auto-rickshaws, cattle and construction debris.

Entry pointmain.pyMain windowgui.pyInference enginesinference_engine.pyVideo handlervideo_handler.pyExport handlerexport_handler.pyTaxonomytaxonomy.pyframesframespseudo-labels for human reviewpseudo-labels for human…corrected annotationscorrected annotationsEntry pointmain.pyMain windowgui.pyInference enginesinference_engine.pyVideo handlervideo_handler.pyExport handlerexport_handler.pyTaxonomytaxonomy.pyframesframespseudo-labels for human reviewpseudo-labels for human…corrected annotationscorrected annotations
FIG. 01Modules and data flows, from the repository layout and documentation.
  1. 1

    Entry point

    QApplication with dark Fusion theme

    main.py
  2. 2

    Main window

    MainWindow, FrameViewer and QThread workers for all heavy operations

    gui.py
  3. 3

    Inference engines

    InferenceEngine, GroundedSAMEngine, SAMBoxRefiner, SemanticSegmentationEngine

    inference_engine.py
  4. 4

    Video handler

    OpenCV seek-based frame access

    video_handler.py
  5. 5

    Export handler

    YOLO, COCO, CVAT, LabelStudio and semantic PNG export

    export_handler.py
  6. 6

    Taxonomy

    16-class Indian road taxonomy with ID, name and color maps

    taxonomy.py
03 · Capabilities

Specification.

Instance segmentation
yolo11n/s/m/l-seg and FastSAM; FP16 on CUDA with FP32 CPU fallback
Zero-shot detection
Grounded SAM (Grounding DINO + SAM) with text prompts; 16-class taxonomy as default prompt
Mask refinement
SAM box refiner with MobileSAM or SAM vit_b/l/h
Semantic segmentation
HuggingFace SegFormer (Cityscapes / ADE20K variants), per frame or whole video
Editing
Box and polygon drawing, class reassignment, delete, zoom/pan, 20-level per-frame undo
Active learning queue
Flags empty, low-confidence, crowded and unexported frames; CSV export
Export formats
YOLO seg, COCO JSON, CVAT XML 1.1, LabelStudio JSON, semantic PNG
Training loop
YOLO fine-tuning as a subprocess with streamed output; reload best.pt
Runs on
PyQt6 desktop app launched with python3 main.py; README recommends a CUDA GPU for Grounded SAM, SAM and SegFormer
Stack
  • Python
  • PyQt6
  • OpenCV
  • Ultralytics
  • PyTorch
  • torchvision
  • NumPy
  • Pillow
  • transformers (optional)
  • segment-anything (optional)
04 · Related

More systems.

Data and labeling · LiveLabeloxAV

A data engine for autonomous driving, built for Indian roads

Data and labeling · LiveEDGE-DATA

An Android app that records driving data and labels it on the device

Next systemAstra Perception

ADAS for unstructured road environments, split into a vision engine and a control ECU