CubicLayer is a geospatial voxel editor app for the Apple Vision Pro.
It extends ARKit to allow for unbounded relocalization using the environment mesh as seen below. By fusing the device odometry and phone GNSS data it does long distance drift compensation and true North heading estimation for many miles.
Relocalization on Fire Watch Tower
Shows the mesh based relocalization system searching the current environment's mesh in blue for the prior mesh segment in red. When the mesh is found the voxel world is moved to where it was relative to the structure. UI in alpha state.
Remote Scene Update
Shows the relocalization then a remote client connected to the same server backend. Small block edits are sent via UDP and broadcast to clients.
Relocalization Debug View
Shows the mesh we are searching for in orange with the environment in blue moving once the pose is found. The cubes on each mesh are the keypoints.
Relocalization on Some Rocks
Shows another relocalization claculation, this time showing the released visualization. The orange points are a sample of the keypoints to indicate the search is ongoing. Again we see the prior red mesh on top of the current blue environment mesh once a transform is computed.
Final UI
Shows the final UI. Hand pinches are detected 50-100ms before the UI registers interactions. I've added dynamic latency for setting blocks based on the menus positions in the user's view frustum and orientation.