Virtual trip: omni-directional photo database using probe cars

GPS World, Nov, 2006 by Kenji Muto, Kazuoki Matsugatani, Masumi Egawa, Hideaki Nanba

Want to see exactly where you're headed before you get there? With an omni-directional photo database developed by a Japanese automotive technology company, drivers can find the correct route by taking a virtual drive-through composed of realistic images collected by probe cars.

Probe cars are effective for collecting traffic information and other valuable data. Recent research has focused on analytical methods for using the data gathered by various types of sensors, such as GPS systems, speedometers, gyrocompasses, and--the focus of this article--cameras. Using both GPS and cameras, the system described in this article could assist drivers and passengers with navigation by showing them images of their chosen routes.

We believe these photos are valuable for drivers, because images of intersections or streets help drivers find the correct route. To take photos easily and effectively, we used omni-directional cameras mounted onto a probe car. The probe car uploads the photos to a probe center that processes them, creating a database that also contains information on the position and direction of the probe cars when each photo was taken. This database, accessible by personal computers and automobile displays, relates the data to geographical information, enabling clients accessing the system to enjoy a virtual drive-through.

In this article, we discuss the design and development of the system software and propose an application using this system. First, we introduce the concept of our system and the architecture and the specifications of our prototype. We then explain features of the omni-directional photo database. Finally, we describe our client application and the result of its evaluation.

System Overview

One of the most important features of our system is the omni-directional camera. We mounted this camera--normally used as a surveillance camera--on top of a probe car. FIGURE 1 shows the camera and example images. The camera shown in (a) is composed of a charge-coupled device unit (a unit with a light-sensitive integrated circuit) and parabolic mirror. By reflecting the mirror, the camera acquires the 360-degree ring-shaped image shown in (b). After the image is processed by a computer program, we obtain the panorama view shown in (c).

FIGURE 2 outlines the process used to collect images. As shown in (a), the camera was developed for fixed surveys at a single location, and a user can view the landscape at only a single point. However, we mounted the camera on the probe car, as shown in (b). In this way, the moving camera was able to continuously collect landscape images. Collected landscape images are stored in a database, along with the position and direction of the probe cars. By browsing this series of images, a client can search for any location, from any perspective. Clients are therefore able to confirm their desired locations using realistic information and ascertain how they should proceed.

System Architecture. We developed our prototype system to investigate feasibility and analyze problems in order to assist drivers. The system consists of probe cars, networks, probe center, and clients (FIGURE 3). The probe cars upload images to the probe center over the network. The probe center processes the images, creates a database, and delivers the information to the clients. The clients browse a series of images and experience a virtual drive-through. Components of the system include the following items.

Probe Car. The probe car comprises an omni-directional camera, a GPS receiver, a wireless module, and a probe-control unit. The probe-control unit collects GPS data and camera images and transmits them to the probe center via the wireless module. In the wireless module, we installed two types of wireless tranceivers--a wireless local area network (WLAN) unit and a cellular communication unit. During data transmission, the mobile router, connected to these wireless units, automatically selects the apropriate unit with optimum radiowave conditions. Using this wireless module, the probe car is able to reliably transmit data.

Network. The probe cars are connected to the probe center over a wireless network and the Internet. As explained in the previous section, we used two types of wireless networks, a WLAN and a cellular network. WLAN has a wider-bandwidth and allows high-speed communication. The WLAN is suitable for transmitting heavy data, including figures for omni-directional photos. The service area of the WLAN, however, is relatively narrow and the probe cars are not able to use WLAN connection everywhere, so we used the cellular system as a backup. The data, transmitted from the probe car via the wireless network, is transmitted to the probe center over the Internet.

Probe Center. The probe center converts the ring-shaped images of the omni-directional camera into panorama-view images. The center also processes the GPS data and creates a database. The center delivers a series of images to clients along with geographic data. Most of these functions are implemented by the database processing software, or "omni-directional photo database."