At Tech’s Institute for Robotics and Intelligent Machines, Lakshmi Velayudhan Nair is teaching robots how to improvise. Nair is a fourth-year IC Ph.D. student under advisor Professor Sonia Chernova, who directs the Robot Autonomy and Interactive Learning (RAIL) Lab.
The RAIL lab’s overarching goal is to teach robots how to operate in human environments. Nair’s research focuses on enabling robots to deal with unforeseen situations by constructing tools out of items in its environment, or “MacGyvering.” The term originates with Angus MacGyver, the star of the eponymous TV show who would construct objects impromptu using whatever items he had available. Nair wants to give robots the same creative ability to innovate and improvise.
In order to MacGyver, the robot reasons about two key aspects of each object that it has around it. First, it considers the item’s shape and appearance. Neural networks enable the robot to learn which shapes are most suited for certain functions and to predict whether an available piece is suited for that particular function. For example, the robot may need a ladle but does not have one available. Through training, the robot has some notion of what a ladle looks like, and it searches its surroundings for items that are similar to the requisite components, namely a concave surface and a handle.
Second, once the robot has identified the necessary constituents, it will attempt to find a way to connect them. The group considered three attachment methods: magnetic fastening, piercing and grasping.
If the items have magnets affixed to them, the robot will figure out the locations of the magnets and determine whether their placement facilitates the appropriate orientation. For example, to make a hammer, the robot will need to fasten two items perpendicularly to form a T-shape.
In order to survey candidates for piercing attachment, the robot uses a handheld spectrometer which gives information about a material’s properties. Using neural net training, the robot determines whether an item can be pierced based on its material properties.
To connect pieces using pliers or tongs, Nair and her group use existing grasp sampling approaches to teach the robot where objects can be grasped. The robot subsequently uses that information to determine whether those grasp locations allow the correct attachment.
Once the robot has sampled the locations where these objects can be grasped, it checks whether those locations facilitate the desired orientation. The group has applied these three methods to teach the robot how to make a ladle — out of pliers and bowls — and a screwdriver — out of pliers, a coin, a spatula and a hammer — among others.
Nair commented that the most challenging part of her research is determining whether the tool that the robot has made is “good” or whether the tool that was constructed is actually the “right” one.
To answer this question, Nair and her group instruct the robot to make the tool and subsequently attempt to use it. If it works, then they consider it to be a good construction.
The second key challenge for the group is that, since MacGyvering is actually quite difficult for humans themselves, inventing examples that can test their approach is surprisingly hard. However, humans could potentially learn a great deal by pushing robots into that space and discovering what they are capable of creating.
In the future, Nair hopes that allowing robots to think creatively will assist humans under significant cognitive duress. Nair’s research has applications to disaster scenarios or search and rescue missions where the available tools cannot be predicted. An innovative robot has the potential to provide invaluable assistance to humans in dire circumstances and perhaps someday save a life.
To learn more about this and other recent projects and publications in the RAIL laboratory, check out their website rail.gatech.edu.