Perception

Semantic 3D Reconstruction

Semantic 3D reconstruction is the joint recovery of a scene's 3D geometry and per-element semantic labels, producing maps whose surfaces, voxels, or points carry object categories or open-vocabulary embeddings. Classical pipelines fuse depth into TSDF or voxel grids while projecting 2D segmentation into the volume; recent systems distill CLIP-style features into 3D fields so maps can be queried with natural language. The output supports object-aware planning rather than purely geometric collision avoidance.

Why it matters for physical AI

Language-queryable 3D maps let robots ground instructions like "fetch the mug by the sink" in metric space, connecting foundation model semantics to executable motion.