Remote collaboration across wall-sized displays creates a key challenge: how to support audio-video communication among users as they move in front of the display. We present CamRay, a platform that uses camera arrays embedded in wall-sized displays to capture video of users and present it on remote displays according to the users’ positions. We investigate two settings: in Follow-Remote, the position of the video window follows the position of the remote user; in Follow-Local, the video window always appears in front of the local user. We report the results of a controlled experiment showing that with Follow-Remote, participants are faster, use more deictic instructions, interpret them more accurately, and use fewer words. However, some participants preferred the virtual face-to-face created by Follow-Local when checking for their partners’ understanding. We conclude with design recommendations to support remote collaboration across wall-sized displays.