The best part of waking up is a robot filling your cup!
Developed by researchers at Cornell University, the aptly-named Robobarista may appear to be just an ordinary robot, however it packs the skills of a talented Starbucks barista. Impressively, it is capable of learning how to intuitively operate machines by following the same methods a human would when introduced to a device, like a coffeemaker.
The Robobarista can autonomously make an espresso, as well as carry out other mundane tasks, using instructions provided by Internet users. To do this, team had to first collect enough crowdsourced information from online volunteers to teach the robot how to manipulate objects it had never seen before. The Robobarista then reads these instructions — such as “hold the cup of espresso below the hot water nozzle and push down the handle to add hot water” — and completes a said command by using a database of deep learning algorithms.
“In order for robots to interact within household environments, robots should be able to manipulate a large variety of objects and appliances in human environments, such as stoves, coffee dispensers, juice extractors, and so on,” the team writes. “Consider the espresso machine above — even without having seen the machine before, a person can prepare a cup of latte by visually observing the machine and by reading the instruction manual. This is possible because humans have vast prior experience of manipulating differently-shaped objects.”
Robobarista’s functionality is based on a two-step process. Generally speaking, the idea is to get the robot to recognize certain things — including buttons, handles, nozzles and levers — and produce similar results as its human counterparts. This way, when it sees a knob, for instance, the robot can scan through its database of known objects and properly identify it. Once it has confirmed that the said control is indeed a knob, it can figure out how to physically operate it based on all of the similar gizmos in its database, the device’s online instruction manual, and how it understands a person’s use of the gadget.
The team notes that their focus was on the generalization of manipulation trajectory through part-based transfer using point-clouds without knowing objects a priori and without assuming any of the sub-steps like approaching and grasping.
“We formulate the manipulation planning as a structured prediction problem and design a deep learning model that can handle large noise in the manipulation demonstrations and learns features from three different modalities: point-clouds, language and trajectory,” the team explains.
To help instruct an action, users select one of the preset steps and then navigate a series of options to control the robot’s movements. Ultimately, every user will complete the task slightly differently, therefore building up the droid’s skillset when it draws on hundreds of these instructions. As this database grows, so does its potential to carry out more chores in and around the house.
For each item, the team captures raw RGB-D images through a Microsoft Kinect camera and laser rangefinder, then stitches them with Kinect Fusion to form a denser point-cloud in order to incorporate different viewpoints of the objects. The crowdsourced instructions are translated into coordinates, which the robot uses to plan the trajectory of its arm to control a new machine.
“Instead of trying to figure out how to operate an espresso machine, we figure out how to operate each part of it,” the team adds. “In various tests on various machines so far, the robot has performed with 60% accuracy when operating a device that it has never seen before.”
Don’t drink coffee? No need to fret. Since Robobarista can master directions over the Internet via Amazon’s Mechanical Turk, the friendly bot can do a lot more than just make a mean cup ‘o joe. In fact, it can fill up a water bottle or pour a bowl of cereal as well. Talk about the perfect Rosie-like robot for the morning rush!
Up until now, robots have typically been configured to complete the same command repeatedly, like the recently-unveiled gadget capable of whipping up dinner by following a set of preprogrammed recipes. However, Cornell’s latest creation has been built to intuitively account for variables and work around them.
If you’ve come by any of our event booths in the past, you know how much we love coffee. Perhaps, we should call upon Robobarista for our next shows! Interested in learning more? Be sure to read the Cornell team’s paper. The student researchers are still working with crowdsourcing to educate their robot, and you can sign up to assist in their efforts here.
Atmel | Bits & Pieces 