Modular neural network: Difference between revisions

Revision as of 18:53, 3 October 2017

Revision as of 19:08, 3 October 2017

Line 1:

{{Underlinked|date=December 2012}}

A '''modular neural network''' is an [[artificial neural network]] characterized by a series of independent neural networks moderated by some intermediary. Each independent neural network serves as a module and operates on separate inputs to accomplish some subtask of the task the network hopes to perform.{{sfn|Azam|2000}} The intermediary takes the outputs of each module and processes them to produce the output of the network as a whole. The intermediary only accepts the modules' outputs—it does not respond to, nor otherwise signal, the modules. As well, the modules do not interact with each other.

==Biological basis==

As [[artificial neural network]] research progresses, it is appropriate that artificial neural networks continue to draw on their biological inspiration and emulate the segmentation and modularization found in the brain. The brain, for example, divides the complex task of visual perception into many subtasks.{{sfn|Happel|Murre|1994}} Within a part of the [[brain]], called the [[thalamus]], lies the [[lateral geniculate nucleus]] (LGN), which is divided into different layers that separately process color and contrast: both major components of [[Visual perception|vision]].{{sfn|Hubel|Livingstone|1990}} After the LGN processes each component in parallel, it passes the result to another region to compile the results.

Certainly someSome tasks that the brain handles, like vision, haveemploy a hierarchy of sub-networks. However, it is not clear whether there is some intermediary which ties these separate processes together on a grander scale. Rather, as the tasks grow more abstract, the isolation and compartmentalization breaks down between the modules and they begin to communicate backwith andeach forth.other, At this point,unlike the modular neural network analogy is either incomplete or inadequatemodel.

== Design ==

Unlike a single large network that can be assigned to arbitrary tasks, each module in a modular network must be assigned a specific task and connected to other modules in specific ways by a designer. In the vision example, the brain evolved (rather than learned) to create the LGN. In some cases, the designer may choose to follow biological models. In other cases, other models may be superior. The quality of the result will be a function of the quality of the design.

==Complexity==

One of the major benefits of a modularModular neural network is the ability tonetworks reduce a single large, unwieldy neural network to smaller, potentially more manageable components.{{sfn|Azam|2000}} There are someSome tasks it appears are forintractably practical purposes intractablelarge for a single neural network as its size increases. The following are benefits of using a modular neural networknetworks over a single all-encompassing neural network.include:

===Efficiency===

The possible connections[[neuron]] increases(node) atconnections a dauntingincrease rateexponentially as nodes are added to thea network. Since computationComputation time depends on the number of nodes and their connections, any increase here will havehas drastic consequences in thefor processing time. Assigning As the greater task is further compartmentalized, the possible connections each node can make are limited, and thespecific subtasks willto hopefullyindividual executemodules more efficiently than trying to tacklereduce the wholenumber taskof atnecessary onceconnections.

===Training===

A large [[neural network]] attempting to model multiple parameters can suffer from interference as new data can dramatically alter existing connections or just serve to confuse. Each Withmodule some foresight into the subtasks tocan be solved,trained eachindependently neuraland networkmore canprecisely bemaster tailoredits for itssimpler task. This means the training [[algorithm]] used, and the training data used for each sub-network can be unique and implemented much more quickly. In large part this is due to the possible combinations of interesting factors diminishing as the number of inputs decreases.

===Robustness===

Regardless of whether a large neural network is biological or artificial, it remains largely susceptible to interference at and failure in any one of its nodes. By compartmentalizing subtasks, failure and interference are much more readily diagnosed and their effects on other sub-networks are eliminated as each one is independent of the other.

== Notes ==