Ishan Chatterjee and I produced an autonomous hors d’oeuvres waiter for our final project in Prof. Radhika Nagpal’s Autonomous Robot Systems class. Its hardware included: Turtlebot (Roomba-like locomotion), a Microsoft Kinect, a low-power laptop, a webcam, a speaker, and a portable food tray. Its goal was to:

  • Approach people at the party
  • Offer people food
  • Recognize when supplies have run out
  • Return to base to get refilled

The software we wrote constructed a mapping of the environment, using the Kinect’s depth cameras and odometry readings from the Turtlebot’s motion. It built this obstacle navigation up as it explored the room, using its webcam to detect human faces to offer food. Once the robot had offered food, it used its webcam to detect the amount of food remaining, returning to its origin where it would be refilled as needed. The robot returned by backtracking on the built mapping and seeking visual cues. A human operator would then refill the tray and reactivate the serving routine.