I just read an interesting essay on
Connectionism at the Stanford Encyclopedia, which basically regards the conceptual issues surrounding neural networks. I didn't realize all of the challenges fuzzy logic is presenting to traditional computing.
So, traditional computing is based on symbolic logic, and through the middle of the 20th century it was common to compare the mind to a computer ... in fact, the book I started reading on human memory for my summer class did just that. The idea is to represent a problem as a string of symbols and establish rules for manipulating those symbols.
However, that's not how neural nets (nor the brain) work. Instead, they create distributed and overlapping patterns that are referred to as "sub-symbolic". The operation of the individual neurons does not predict what the pattern will be, and therefore the emergent properties of the net don't "supervene" on the neurons ... i.e. the pattern can't be predicted from properties of the neurons.
However, I don't think that's strong emergence. It seems just a game of defining the system. If one includes the external influences as part of the "system", it's probably possible to predict the result.
Still, it introduces some very interesting concepts related to emergence. I think that if the training sets are fed to the net in a different order ... or if different subsets of training are used, the way the pattern is established in the net will be different ... even though the meta-result (the represented symbol) is essentially the same. Given this meta characteristic, the manner in which these sub-symbolic patterns are interacting feels as if it would have the potential to produce unpredictable results.
For example, one of the strengths of nets is their ability to deal with new situations. Traditional symbolic programming can only accomplish the tasks that are programmed. However, a net will respond with a "result" no matter the input. It will respond to situations it was not trained to handle. How the net responds, however, seems to have the potential to be unpredictable.
Maybe.