Winner-take-all neural network with massively optoelectronic interconnections

American Journal of Applied Sciences, Feb, 2009 by Wissam H. Ali, Ahmed N. Abdalla, Wa'l H. Ali

INTRODUCTION

In a winner-take-all function, a collective dynamic competition takes place, which receives the maximum input and suppresses activity in all the other nodes of the network. The mechanism that is responsible for this type of behavior is the competition for a limited resource such as laser resonator gain, current on a bus, or current limited by a common load resistor. The latter is the case considered in this research. This functional unit can be used in a wide variety of applications requiring arbitration. The application of most interest is in competitive neural networks for unsupervised clustering applications, where the winner-take-all is used as a powerful nonlocal non-linearity. In these networks, each node receives a weighted sum of input from a statistical clustered input space. The weight vectors lead to a neuron that represents prototypes of each of the clusters and the largest inner product is related to the prototype to which an input pattern most closely matches. The winner-take-all network selects the largest inner product, corresponding to the best pattern match and assigns class membership.

In this study an optically controlled winner-take-all circuit is described being, based on a pnpn structure that can be used in the optical implementation of a competitive network.

Such a network is implemented as a parallel-optical system that incorporates a diffractive-optical element (DOE). Its performance as a scheduler for both crossbar and self-routing switching fabrics is measured.

In this study, the implementation of a neural network that exploits an optical interconnect to perform a real task. The operation and the design of such a scheduler and operational experimental implementation of an SOF is described. The scheduler uses a neural network in a winner-take-all strategy to optimize decisions on the throughput of Koheen (self organizing).

WTA NEURAL NETWORKS

The basis of the winner-take-all circuit is an electrical network (1), that has the capability of both lateral and global inhibition. Global inhibition is essential to perform the winner-take-all function and can be realized as a special case of this network, obtained by removing all the local couplings between the cells.

One of the most important uses of this network in Self-Organizing Feature (SOF) mapping in networks is one of the most fascinating topics in the neural network field. Such networks can learn to detect regularities and correlations in their input and adapt their future responses to that input accordingly. The neurons of competitive networks learn to recognize groups of similar input vectors. Self-organizing maps learn to recognize groups of similar input vectors in such a way that neurons physically close together in the neuron layer respond to similar input vectors (2).

In competitive learning, the neurons in a competitive layer are distributed to recognize frequently presented input vectors. The architecture for a competitive network is shown in Fig. 1, where in this figure the input vector p and the input weight matrix I[W.sub.1, 1] are accepted to produces a vector having S1 elements. The elements are the negative values of the distances between the input vector and vectors I[W.sub.1, 1] formed from the rows of the input weight matrix.

[FIGURE 1 OMITTED]

The net input n1 of a competitive layer is computed by finding the negative value of distance between input vector p and the weight vectors with the biases b. If all biases are zero, the maximum net input neuron can have

This occurs when the input vector p equals the neuron's weight vector. The competitive transfer function accepts a net input vector for a layer and returns neuron output of 0 for all neurons except for the winner, that is the neuron associated with the most positive element of net input n1. The winner's output is 1. If all biases are 0, then the neuron whose weight vector is closest to the input vector has the least negative net input and, therefore, wins the competition in order to output a 1. Biases are used with competitive layers for reasons to be considered later in this study.

OPTOELECTRONIC WTA

To demonstrate the operating principle of the proposed design a commercially available photothyristor is used as active nonlinear device, in the form of silicon pnpn structure. A network of photothyristor connected in parallel to a power supply through a load resistor, R is considered as shown in Fig. 2.

[FIGURE 2 OMITTED]

In order to demonstrate the winner-take-all principle in its pure form, it has to be shown that only one node wins the competition regardless of the input. To show this every device is illuminated with sufficient intensity necessary to switch the entire device. After switching-off the light, the competition begins and only that one node with the maximum light input wins the competition and carries nearly all the total current.

In the implementation described here, both a crossbar and a multistage self-routing switching fabric with random-access input queuing is considered. The novelty in this approach is the use of an optoelectronic neural network to perform the input-output matching. The use of neural-network hardware can yield excellent performance on resource-allocation and optimization problems at low cost, is importance is in exploiting analog circuit capabilities and creates a naturally highly parallel approach to the problem. Such a neural network is, however, intractable to be built to any scalable extent in silicon because of the high degree of connectivity required (3).