Perception

Correspondence Problem

The correspondence problem is the task of identifying which elements in one image, point cloud, or observation match the same physical entity in another, underlying stereo depth, optical flow, feature-based SLAM, point cloud registration, and cross-instance semantic matching. Solutions range from handcrafted descriptors like SIFT to learned dense descriptors such as Dense Object Nets, which enable category-level manipulation by matching keypoints across object instances.

Why it matters for physical AI

Dense learned correspondences let robots transfer grasps and skills across object instances and viewpoints, providing spatial grounding that complements the semantic knowledge of vision-language models.