Teaching machines to recognize objects - John Hopkins Univ researcher Elli Angelopoulou develops new object recognition technique for robots - Brief Article

USA Today (Society for the Advancement of Education), June, 1997

A packaging robot, working in a candy factory in the near future, aims its electronic eye at a tray crammed with chocolates. The robot must fill a box with a specific mix of candy, but how can it tell a cream-filled morsel from a chocolate-covered peanut cluster? A new object recognition technique developed at Johns Hopkins University, Baltimore, Md., might help this robot hold onto its job. Using three lights, a video camera, and a computer. researcher Elli Angelopoulou has devised a new way to transform visual images into electronic "signatures." Each is a distinctive series of 11 numbers. By comparing these signatures, a computer can tell how closely two objects resemble one another. For instance, a smooth, dome-shaped candy and a bumpy nut cluster would have far different signatures.

Angelopoulou, a doctoral student in the Department of Computer Science and an admitted chocoholic, has used the technique to identify coffee mugs, disposable razors, children's toys, and, yes, chocolate candies. It even can distinguish between two objects that have identical shapes, but different colors. Her technique primarily is for recognizing objects with curved surfaces, a task that long has troubled computer vision scientists. The next step is for robotics experts to incorporate Angelopoulou's breakthrough into a mechanical packaging machine.

A mechanical device that interacts with its environment must have some sort of visual information about its surroundings, explains Angelopoulou. "A robot or a vehicle that moves by itself needs to `see' where it's going. We need a way for it to recognize objects." Scientists have invented electronic ways to identify some objects the way people often do -- by following the edges until a rectangular shape, for example, is recognized as a door. Such systems do not work well on curved objects, such as vases, that have few edges to follow, though.

With her technique, Angelopoulou aims one lamp in the same direction as her camera lens and positions the other two on opposite sides of an object such as a coffee mug. She shoots the mug three times, first switching one lamp on and off, then repeating the process with the other two. The camera captures this mug in about 10,000 pixels -- the tiny points on a video screen that blend together and form an image. For each pixel, Angelopoulou's software computes a DeCov value, short for determinant of the photometric covariance matrix. These values provide critical information about how the object curves. Within a few minutes, the computer sorts and reduces these values to a distinctive series of 11 numbers. These become the signature for the mug. If another picture produces the same or nearly the same series of numbers, the computer "recognizes" that object, too, as a coffee mug. Different colors reflect light differently, so the system can separate blue and red mugs that have the same shape.

The system is less sensitive to the electronic "noise" or interference that hampers other computer vision systems and does not require extremely precise placement and measurement of lighting. The process does have limitations, though. It will recognize a mug from many upright angles, but the computer will be fooled if the camera looks at the flat bottom. Also, the computer only will recognize identical objects if the lighting remains unchanged.

A factory could provide such controlled lighting, and Angelopoulou hopes other scientists someday will install her system in a robot that could sort and pack mechanical parts, toys, or even chocolates. "Usually, what we do in smaller labs is just try to solve a small part of the problem," she says. "Then we hope someone else will integrate it into a device that solves a bigger part of the problem."