Adaptive resonance theory

from Wikipedia, the free encyclopedia

Adaptive Resonance Theory ( ART ) translates as adaptive resonance theory . ART is an architectural concept for neural networks and was developed by Stephen Grossberg and Gail A. Carpenter .

Basic structure

Basic ART structure

The simple ART system belongs to the class of unsupervised learning algorithms.

It typically consists of a comparison field F1 and a detection field F2, an alertness parameter P and a reset module R. Neurons in F1 represent attributes, neurons in F2 categories. Therefore, both fields usually consist of a different number of neurons. F1 and F2 are fully bi-directional networked. Each compound is rated with a specific weight. These weights are ART's long-term memory. All weights from F1 to F2 are combined in a weight matrix W, the others belong to the weight matrix Z. The vigilance parameter has a considerable influence on the system: a high value causes fine-grained memories (many, small categories), a low value provides more Abstraction (fewer, coarser categories).

Classification

During the classification, an attribute vector I is applied to the F1 neurons. This is evaluated with the weights in W and passed on to the F2 neurons. The F2 neuron that receives the maximum signal determines the category to which I belongs. The output of all other F2 neurons is set to zero.

The result is evaluated with a Z and returned to the F1 shift. If the difference between I and the returning signal is within the limit set by P, the system is in resonance and the weights can be trained. Otherwise the active F2 neuron is deactivated by a reset signal until a new input vector is available. A new run follows that leads to another F2 neuron (another category). Thus, during the search, categories are tried little by little until either a category has been found that is similar enough or all categories have been compared.

If I does not fit into any of the stored categories, a previously unused F2 neuron is used and the weights for this are initialized with the values ​​of I. If no free F2 neuron is available (memory capacity is exhausted), the new pattern cannot be learned.

In the state of resonance, the weights of the selected category are adapted to the attributes of I. Hence the name of the system: It learns in the state of resonance by adapting stored categories to new cases, provided the new case does not deviate too much from the stored category.

training

There are two different ways to train ART networks: slow and fast.

Slow learning happens when a satisfactory category has been found (based on vibrating systems, this state is called resonance). The weights in this category are fitted to the attributes of I using differential equations. The degree of adaptation depends on how long I is presented.

In fast learning, the adjustment of the weights is determined using algebraic equations.

While fast learning is effective and efficient for many applications, slow learning is more biologically plausible.

ART types

ART 1

ART 1 is the simplest and original variant. ART-1 can only process binary attributes.

ART 2

ART 2 extends ART-1 and enables the processing of continuous attributes.

ART 2-A

ART 2-A is a streamlined form of ART-2 with dramatically improved runtime and rarely worse results.

ART 3

ART 3 is based on ART-2 and mimics chemical processes in biological neural networks.

Fuzzy ART

Fuzzy ART extends ART 1 to include the use of fuzzy logic . This enables gradual assignment to different categories at the same time. By coding the attributes in a complementary manner, the absence of attributes can be processed, which reduces unnecessary proliferation of categories.

Distributed ART

Distributed ART

ARTMAP

ARTMAP overview

ARTMAP, also called Predictive ART . Two ART modules are combined with a connection network V to form a monitored, learning system. The first module processes the attribute vector I and the second the result vector C. The control system S controls the vigilance parameter of the attribute system as a function of the result categories. Thus, different rough categories of the attributes can be generated.

If only a simple (monitored, learning) classification system is required, the second module can be simplified.

Fuzzy ARTMAP

Fuzzy ARTMAP is simply an ARTMAP made up of two Fuzzy ART modules.

Distributed ARTMAP

Distributed ARTMAP

ARTMAP-IC

ARTMAP-IC

Default ARTMAP

Default ARTMAP is a combination of fuzzy ARTMAP and partially distributed ARTMAP.

ARTMAP overview

Work out Classify Instance counting
Fuzzy WTA WTA No
default WTA distributed No
IC WTA distributed Yes
Distributed distributed distributed Yes

WTA = "the winner takes it all" Only the F2 neuron with the largest output is active, the output of all others is zero.

Distributed = all F2 neurons are active.

Instance counting = the outputs of the F2 neurons are weighted by the F3 neurons. Categories that appeared more frequently in the training cases get a higher rating.

literature

  • Philip D. Wasserman: Neural computing: theory and practice . Van Nostrand Reinhold, New York 1989, ISBN 0-442-20743-3 .

Web links

Individual evidence

  1. ^ GA Carpenter, S. Grossberg: Adaptive Resonance Theory . ( Memento of the original from May 19, 2006 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. (PDF; 59 kB), In MA Arbib (Ed.): The Handbook of Brain Theory and Neural Networks . Second edition. MIT Press, Cambridge MA 2003, pp. 87-90. @1@ 2Template: Webachiv / IABot / cns.bu.edu
  2. ^ S. Grossberg: Competitive learning: From interactive activation to adaptive resonance . ( Memento of the original from September 7, 2006 in the Internet Archive ) Info: The archive link was automatically inserted and not yet checked. Please check the original and archive link according to the instructions and then remove this notice. (PDF; 8.1 MB) In: Cognitive Science (journal) , 1987, 11, pp. 23-63 @1@ 2Template: Webachiv / IABot / www.cns.bu.edu
  3. ^ GA Carpenter, S. Grossberg: A massively parallel architecture for a self-organizing neural pattern recognition machine . ( Memento of the original from September 6, 2006 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. (PDF; 8.9 MB) In: Computer Vision, Graphics And Image Processing (magazine) , 1987, pp. 54-115. @1@ 2Template: Webachiv / IABot / www.cns.bu.edu
  4. ^ GA Carpenter, S. Grossberg: ART 2: Self-organization of stable category recognition codes for analog input patterns . ( Memento of the original from September 4, 2006 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. (PDF; 3.8 MB) In: Applied Optics , 26 (23), (1987), pp. 4919-4930 @1@ 2Template: Webachiv / IABot / cns-web.bu.edu
  5. ^ GA Carpenter, S. Grossberg, DB Rosen: ART 2-A: An adaptive resonance algorithm for rapid category learning and recognition . ( Memento of the original from May 19, 2006 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. (PDF; 1.4 MB) In: Neural Networks (journal) , 1991, 4, pp. 493–504 @1@ 2Template: Webachiv / IABot / cns.bu.edu
  6. ^ GA Carpenter, S. Grossberg: ART 3: Hierarchical search using chemical transmitters in self-organizing pattern recognition architectures . ( Memento of the original from September 6, 2006 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. (PDF; 2.0 MB) In: Neural Networks (journal) , 1990, 3, pp. 129–152 @1@ 2Template: Webachiv / IABot / cns.bu.edu
  7. ^ GA Carpenter, S. Grossberg, DB Rosen: Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system . ( Memento of the original from May 19, 2006 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. (PDF; 1.2 MB) In: Neural Networks (journal) , 1991, 4, pp. 759–771 @1@ 2Template: Webachiv / IABot / cns.bu.edu
  8. ^ GA Carpenter: Distributed Learning, Recognition and Prediction by ART and ARTMAP Neural Networks . (PDF; 1.6 MB) In: Neural Networks (magazine) , vol. 10, 1997, no. 8, pp. 1473-1494 (English); accessed on May 12, 2015.
  9. ↑ GA Carpenter, S. Grossberg, JH Reynolds: ARTMAP: Supervised real-time learning and classification of nonstationary data by a self-organizing neural network ( Memento of the original from May 19, 2006 in the Internet Archive ) Info: The archive link was inserted automatically and not yet checked. Please check the original and archive link according to the instructions and then remove this notice. (PDF; 2.2 MB) In: Neural Networks (magazine) , 1991, 4, pp. 565-588 @1@ 2Template: Webachiv / IABot / cns.bu.edu
  10. ^ GA Carpenter, S. Grossberg, N. Markuzon, JH Reynolds, DB Rosen: Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps . ( Memento of the original from May 19, 2006 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. (PDF; 1.6 MB) In: IEEE Transactions on Neural Networks , 1992, 3, pp. 698-713 @1@ 2Template: Webachiv / IABot / cns.bu.edu
  11. ↑ Gail A. Carpenter, Boriana L. Milenova, Benjamin W. Noeske: Distributed ARTMAP: a neural network for learning almost distributed supervised . ( Memento of September 1, 2006 in the Internet Archive ) (PDF; 1.4 MB) In: Neural Networks (magazine) , 1998, 11, pp. 793–813 (English); accessed on May 12, 2015.
  12. ^ Gail A. Carpenter, Natalya Markuzon: ARTMAP-IC and medical diagnosis: Instance counting and inconsistent cases . ( Memento of September 4, 2006 in the Internet Archive ) (PDF; 227 kB) In: Neural Networks (magazine) , 1998, 11, pp. 323–336 (English); accessed on May 12, 2015.
  13. ^ Gail A. Carpenter: Default ARTMAP . ( Memento of September 3, 2006 in the Internet Archive ) (PDF; 1.8 MB) CAS-CNS Technical Report TR-2003-008, 2003 (English); accessed on May 12, 2015.