Overview
LingBot-Map is a groundbreaking feed-forward 3D foundation model designed for high-efficiency streaming 3D reconstruction. This ECCV 2026 Best Paper Award candidate introduces a novel Geometric Context Transformer that unifies coordinate grounding, dense geometric cues, and long-range drift correction within a single, powerful streaming framework. Leveraging a paged KV cache attention mechanism, LingBot-Map achieves stable inference at ~20 FPS on 518×378 resolution, even for sequences exceeding 10,000 frames, and surpasses state-of-the-art performance on diverse benchmarks.
Key Features
- Geometric Context Transformer (GCT): Architecturally integrates anchor context, pose-reference windows, and trajectory memory to process geometric information efficiently within a streaming paradigm.
- High-Efficiency Streaming Inference: A feed-forward architecture with a paged KV cache attention (via FlashInfer) enables robust, real-time reconstruction for exceptionally long sequences.
- State-of-the-Art Performance: Demonstrates superior accuracy and robustness compared to existing streaming methods and iterative optimization-based approaches.
- Optional Sky Masking: Improves outdoor scene reconstruction quality by filtering out sky points using an integrated ONNX segmentation model.
- Flexible Deployment: Supports both interactive visualization via a web-based viewer (
demo.py) and offline, high-throughput rendering pipelines (demo_render/batch_demo.py).
Typical Use Cases
- Robotics and Autonomous Systems: Real-time 3D mapping and localization for drones, self-driving cars, and indoor robots.
- Augmented Reality (AR) and Virtual Reality (VR): Creating immersive 3D environments from live video feeds for enhanced user experiences.
- 3D Content Creation: Rapid generation of 3D models from video for film, gaming, and virtual production.
- Scene Understanding and Analysis: Extracting dense geometric information from continuous video streams for various perception tasks.
