Neuroinformatics I: Fuzzy Neural Networks of More-Equal-Less Logic (Static)

. This article analyzes the possibilities of neural nets composed of neurons – the summa-tors of continuously varied impulse frequencies characterized by non-linearity N , when informational operations of fuzzy logic are performed. According to the facts of neurobiological research the neurons are divided into stellate and pyramidal ones, and their functional-static characteristics are presented. The operations performed by stellate neurons are characterized as qualitative (not quantitative) informational estimations “more”, “less”, “equal”, i.e., they function according to “more-equal-less” (M-E-L) logic. Pyramidal neurons with suppressing entries perform algebraic signal operations and as a result of them the output signals are controlled by means of universal logical function “NON disjunction” (Pierce arrow or Dagger function). It is demonstrated how stellate and pyramidal neurons can be used to synthesize the neural nets functioning in parallel and realizing all logical and elementary algebraic functions as well as to perform the conditional controlled operations of information processing. Such neural nets functioning by principles of M-E-L and suppression logic can perform signals’ classiﬁcation, ﬁltration and other informational procedures by non-quantitative assessment, and their informational possibilities (the amount of qualitative states), depending on the number n of analyzing elements-neurons, are proportional to n ! or even to (2 n ) ∗ n ! , i.e., much bigger than the possibilities of traditional informational automats functioning by binary principle. In summary it is stated that neural nets are informational subsystems of parallel functioning and analogical neurocomputers of hybrid action.


Introduction
The abundant and mounting findings of various experimental research works carried on the nervous system during the latter decades enforce us to review the theoretical neuroconcepts, namely, the neuroinformatics.Fuzzy logic, evolved out of fuzzy or Zadeh sets several decades ago (Zadeh, 1965), is of special interest, as well as its application to the neural networks (Zadeh, 1975;Bezdek, 1996;Garliauskas, 1998;2003).
The theory of formal neural networks created by McCulloch and Pitts more than half a century ago by means of binary logic (Boolean algebra) in order to elucidate the neural network functioning on the information processing has stimulated the rise of the theory of finite automats and has considerably promoted the upraise of computers and informatics.This theory based on the theoretical conception of nervous system which functions on the principle "everything or nothing" nowadays becomes unacceptable when the action of biological neural networks is revealed.Neurophysiological researches demonstrate that the single nerve impulse is not significant in the neural network, and the main function is determined by series of multiple impulses (the bursts).Furthermore, it is not reported that there is such an exact synchronization of nerve impulses in the nervous system as it is in the computers.The only obviously common thing in both nervous system and computers is that both of them receive, save, process and transmit the information, i.e., both of them are informational subsystems, though each of them perform every action mentioned above by means of absolutely different tools and principles.As a matter of fact, we have to admit that it is not yet known which principles underlie the action of animal and human nervous system, and today it seems to be the utmost secret of nature.
There were attempts to interpret the nervous system's informational action by means of Perceptron principles based on statistics (Rosenblatt, 1962) and statistical binary neural networks (Klimasauskas, 1996;Raudys, 2001).Although such interpretation models some of biological phenomena, still it provokes great doubts concerning the effective application of these conceptions in neurobiology.The neurobiologists doubt about various neuro-computer feedforward, feedback propagation, Hopfield's, Boltzman's, Hemming's, Kohonen's and other theories of neural networks (Kohonen, 1978;Raudys, 2001;Komarcova and Maksimov, 2002;Garliauskas, 2003), because all of them, although based on some features of nervous system set-up, are better fit to elucidate the functioning of computers than to interpret the research results of neurophysiologists.
When theoretical reasoning concerning neural networks is applied to the facts of neurobiology, the attention should be paid to the conception of neural logic, which was found even before the uprise of fuzzy sets and fuzzy logic, though it meant the same (Mueller et al., 1962;Kirvelis and Pozin, 1967;Kirvelis, 1998;2000a).Neural logic and the neural nets analyzed in the light of it demonstrate that neural structures are able to perform quite precise informational procedures if they execute the analysis of signals by means of non-quantitative evaluation such as "more", "less" or "equal".This conception of neural nets must be acceptable in interpretation of the neurophysiologic facts as well as it must be valuable in search for new principles of parallel information processing and analogical neurocomputers of hybrid actions (Jasinevičius and Petrauskas, 1983).
The aim of this research is to formulate the theoretical more-equal-less (M-E-L) conception of neural nets and to demonstrate its possibilities in interpretation of neurobiological facts as well as in creation of new neuroinformational technologies.

Neuron and Neural Nets in the Organized System
Neural nets have formed during the process of biological evolution as one of several bioinformational (genetic, hormonal, pheromonal, neural) subsystems of animal's control (Kirvelis, 2003).Similar subsystems of informational control can be found in every advanced technical system (for instance, in a robot) as well as in a society, i.e., in such systems, which are considered to be the organized systems (Kirvelis, 2000b).

Organized System and the Controlling Neural Subsystem
The Fig. 1 demonstrates the functional purpose of neural nets in the organized system.Various physical environmental and internal effects, changes and states are reflected with a help of receptors R in the neural subsystem's special structures of neurons (neural nets) as coded impulses and sent to the central units of information processing.Here the information is recorded and compared, the action plan is determined and the information from here is sent as coded neural impulses to the neural control structures of executive organs, called effectors.Generally, executive organs are the structures that perform matter and energy transformations (e.g., muscles, glands).Executive organs (effectors) are commonly closely linked with the receptors analyzing their status and together with the controlling neural structures compose the lowest level closed-loop coding-decoding (CLCD) system (Kirvelis, 2004).Organism's executive organs are responsible for resources input as well as actions towards the environment in order to realize certain motives and aims, thus they compose their own CLCD system through affected environment and external receptor structures.It is obvious that functional activity of all organized systems is based on the informational CLCD principles (Kirvelis and Beitas, 2003).On the level of neural subsystem these CLCD principles are realized by neural nets where neural information circulates and the main universal functional element is the neuron.-receptors or elementary coders, -effectors or elementary decoders.

Neuron and its Static Features
Neurobiological research shows that neuron is the summing transformer of signals (impulse frequencies) X i , which has up to ten thousands entries (synapses) S i and one functional outlet (axon) Y .The latter one can branch and contact (through synapses) with many other neurons and with itself (Fig. 2).The neurons' inputs X i and outputs Y are variable impulse sequences, frequencies of which have continuous value from 0 till F max =∼ 1000 [imp/s].The input signals of neural nets are received from receptors R and they can have various gradual physical character.The neuron has two different types of entries: exciting (positive synaptic coefficient +S i ) and inhibitory (negative synaptic coefficient −S j ).There are neurons of various geometric shapes but functionally the most noteworthy are stellate and pyramidal ones.The neuron's static transfer character is characterized by non-linearity N , which is the specific feature of potentiality to become suppressed if the sum of entry signals is negative S i X i < 0, or less than certain threshold Θ, i.e., if the value of output signals is equal to zero Y = 0.If the sum of entry signals exceeds S i X i > F max , it will function in the supersaturation mode and its value of output signals will not exceed F max .However such an operating mode is observed only in some interneurons.The majority of biological neurons is either suppressed or transmits the total value of excitement according to sublinear dependence (see the dotted line in Fig. 2).Hence, usually the static part of signal transfer is approximated by straight line in investigation of neural structures' functioning.
The pyramidal neurons of higher structures of nervous system, namely the cerebral cortex, have additional possibilities.Their functional organization and mechanisms of action seemingly hold the greatest still unrevealed secrets of nervous system.It is typical that the pyramidal neurons have plenty of throughout appearing and forming synaptic contacts, especially on the dendrites, where entry signals are summed.It seems that the synaptic contacts found on the pyramidal neurons' body perform different function, namely the suppression, which means that the pyramidal neuron will transmit the positive sum of signals to the output (to other neurons) only in case when there are no signals in Z entries.(Neurobiological research shows that the pyramidal neuron's body is enlaced by other neurons' axons that have a plenty of synaptic contacts and those synapses are inhibitory.It means that the latter neurons holding these axons act on the pyramidal neuron very suppressively, i.e., the pyramidal neuron experiences a huge suppression.)Thus the pyramidal neuron will response and forward the signals unless and until there isn't any signal suppressing the neuron's body.It will realize universal logic operation -Pierce arrow (Dagger function) (Fig. 2).

Neuron with Feedback Relations
Neurobiological research reveals that there are a lot of feedback relations in the nervous system and neurons have axonal processes, namely the collaterals that sometimes terminate even on the same neurons.Assuming that if neuron's feedback relations are positive, synapses will be exciting (+S 0 ), and if neuron's feedback relations are negative, synapses will be inhibitory (−S 0 ), it is possible to estimate their impact on functional characteristics of neuron (Fig. 3).When neuron's non-linearity N and linear characteristic is accepted, it is obvious that feedback relations will influence only the linear characteristic's working phase steepness.Neuron's synaptic coefficient S i or steepness of its static characteristic will be reduced 1/(1 + S 0 ) times if feedback relations are inhibitory (S 0 < 0), and will be increased 1/(1 − S 0 ) times if feedback relations are exciting (S 0 > 0).In addition, it is worthy of note that due to limitation of neuron's maximal excitement F max , exciting feedback relations acquire additional static features: 1) when S 0 approximates to => +1, the synaptic coefficients of entries increase (the characteristic's linear phase steepness increases); 2) when S 0 = +1, neuron becomes an ideal "relay" (the straight of static characteristic becomes perpendicular); 3) when 1 < S 0 < (1 + Θ/F max ), neuron becomes "relay with backlash" (hysteretic loop effect); 4) when S 0 > (1+Θ/F max ), neuron becomes an element of reverberational memory, which can be extinguished only if the value of inhibitory signal is more than χ = (S 0 − 1)F max − Θ.
Reportedly, when neuron has feedback relations, non-linearity radically determines its static features and undoubtedly more influences its dynamic characteristics.The feedback relations, in neurobiology called lateral relations, are especially important in neural layers.

Reciprocal Neurons
Neurobiological research shows that neurons located in neural net can not transmit negative signals that are results of summation.In this case negative signals (summation results) are possible in neuron pairs where reciprocal (inverse) neuron realizes the negative signals (Fig. 4).Neurons of pair have the same structure of synaptic connections, but signs of their output are opposite, i.e., the coefficients of correspondent synapses are equal but have the opposite signs.
Such a pair of neurons is a linear algebraic summator, where one neuron of the pair sends signals Y − when sum of incoming signals X is negative and the other neuron of the pair sends signals Y + when sum of incoming signals X is positive.When the sum of synaptic coefficients with the same synaptic sign does not exceed 1 (e.g., S + 1 for exciting synapses) such neurons always function in linear phase and are never maximally excited.In that case the pair of reciprocal neurons realizes the scalar product of entry vector X and entry vector of synaptic connections S or, in other words, estimates the correlation of these vectors by value Y .If both neurons have the threshold Θ, such an algebraic summator will be non-linear and have insensible zone ±Θ.
In summary, it can be said that the reciprocal pair of neurons is the filter of corresponding features of certain entry signals and especially of signals coming from receptor structures.The neuron excitement means that entry vector has this feature and the intensity of excitement expresses the quantity of this feature.

More-Equal-Less Logic as Fuzzy Logics Neural Networks
The gist of neuroinformatics is realization of logical operations by means of neural structures.In cases when neurons filter the features of objects reflected in receptors and estimate their expression by means of continuous values, their logical analysis must be performed by methods of continuous logic.There are a lot of various variants of continuous logic called differently: infinite, continuous, neuronal, analogical, syncretic, fuzzy logic, etc.It is relevant to various hybrid computers, informational technologies as well as neuroinformatics.There are plenty of algebraic algorithms for their realization, e.g., Rfunctions (Rvachev, 1982), neural logic (Kirvelis and Pozin, 1967), fuzzy logic (Zadeh, 1975).For the neural structures it is the easiest to apply the neural logic expressed in algebraic terms, which virtually expresses the main features of all mentioned logics.

Neuronal Fuzzy Logic
The features of reflected objects filtrated by two neurons' continuous entry signals of opposite signs and their magnitude expressed by intensity of neuron's excitation Y can be further processed by means of fuzzy logic.The merest neuronal structures that realize elementary operations of fuzzy logic are shown in Fig. 5.
Neuron which has one exciting and one suppressive entry realizes the conditional difference or informs about the elementary inequality saying that X 1 > X 2 , and presents the expression of this inequality on output Y .Such neuron which has one suppressive entry X and one exciting entry with constant F max realizes logical inversion Y = F max −X which corresponds to one of the most important operations of traditional logic, namely the negation Y = .NON.X.
The structure of two similar neurons can perform other principal elementary operations of continuous or fuzzy logic -conjunction (logical multiplication) or disjunction (logical addition).In neuronal fuzzy logic they are expressed as follows: Selection of minimal value is understood as fuzzy conjunction and selection of maximal value is understood as fuzzy disjunction.It is obvious in the logical analysis of two features; however the same conception is also applied in the fuzzy logical analysis of multiple featured neuronal structures (Fig. 6).As stated above, the principal operation performed synthesizing neuronal fuzzy logic's MIN and MAX structures is N {X i − X k } and it is realized by separate neuron.It is seen clearly when minimum and maximum separation procedures of multiple entries' neurons are written down: It is obvious that the same neurons are necessary to realize the negation of multiple entries, namely inversion, when neuron-invertor is set for every entry.(It is worthy to note that complete neural network has such invertors only in the primary receptor structures where only the positive signals dominate.In the further neuroinformational procedures the inverted and non-inverted signal vectors function in parallel.) It all goes to show that the basic neuronal fuzzy logic's operator is element-neuron which performs the "more" and "less" comparisons.It is understandable that various schemes performing any functions of fuzzy logic can be synthesized from such neuronal structures.There can be even synthesized such schemes as "uncertain", "equal", "indefinite" and similar ones that are disclaimed by traditional categorical logic stating that there are only two possible variants "yes" or "no", and no third variant is possible.
It is this particular feature which differentiates categorical logic from fuzzy logic and makes the latter closer to behaviour of animals and humans.It can be easily interpreted by schemes of neuronal structures' possible functioning.It is undoubtedly well demonstrated when conditional operators IF used in computer programming language are realized by means of neurons.

Neural Structure -the Operator of Arithmetical Condition
The operator of arithmetical condition used in programming languages is expressed as follows: It means that if performing computational procedures the arithmetical value calculated according to algebraic expression is negative, the further operation will be performed considering the address m 1 indicated in the program, if the value is positive, the operation will be performed considering the address m 3 , and if it is equal to zero, the operation will be performed considering the address m 2 .This conception exhibits such actions as "less", "more", "equal", which are naturally performed by neurons or unsophisticated structures of some neurons.Even the pair of reciprocal neurons (Fig. 4) performs "less" ( < 0) and "more" (0 < ) operations when the first neuron sends signals to one group of neurons and the second neuron sends signals to another group of neurons.They can not generate the signals and act on the same groups of neurons simultaneously.Fig. 7 demonstrates the complete neuronal operator of arithmetical condition designed for parallel net of neurons.The neuron located between the reciprocal neurons of the pair will be excited only if both the reciprocal neurons are still, i.e., both of them fulfill condition s i x i = 0.If one of the reciprocal neurons is excited, the middle neuron will be extinguished by intense suppression −S 0 of one of the reciprocal neurons.
The middle neuron which has zero or indefinite identification status can also suppress both of the reciprocal neurons.Such an interaction of three neurons realizes the function given below: The features of fuzzy logic are apparently demonstrated by the presented diagrams of all three neurons' reactions.The excited neuron Y − means "less", Y + -"more" and Y 0 -"uncertain" or "equal".

Neural Structure -the Operator of Logical Condition
The operator of logical condition used in programming languages is expressed as follows:

IF (Logical function) Arithmetic-algebraic expression
It means that arithmetical value according to algebraic function will be given only if logical function is "YES".Otherwise the arithmetic-algebraic function is ignored.
The neural structure which realizes the operator of logical condition is demonstrated in Fig. 8. Logical functions can be realized by pyramidal neurons which are connected with each other by suppressing connections.Two pyramidal neurons connected in series by inhibitory connections realize the double negation, subsequently as a result of it forms the proposition.Several suppressions converged in one pyramidal neuron realize the Pierce arrow (Dagger function), which is expressed as follows: If stellate neuron transmits information about the features of reflected object by signal batches W and transfers them to the suppressive entries of primary pyramidal neuron which transfers them further to the suppressing entries of following pyramidal neuron, then such a neuron will realize the described function: The excited neuron Y will act on corresponding groups of neurons by its connections and perform the selective procedure of information processing and transmission.Generally speaking (in conception of computer technique), such neuron is called the neuronal microprocessor functioning by principles of hybrid computer because pyramidal neuron allows consonantly integrate analogical and logical operations.It can form much more complex concepts than stellate neuron.Stellate neurons form initial concepts filtering according to the principle "more", "less", "equal", whereas pyramidal neurons interconnect those concepts by logical "suppression" or Pierce arrow's functions and thus form superior concepts.Since stellate neurons-summators operate by analogical signals, their switchover from suppression to excitation and vice versa is not pronounced and such feature enables to attribute them to fuzzy sets and fuzzy logic.
Examples of the simplest neural structures given here help to understand the possibilities of much more complex neural nets, i.e., the functional organization of parallel neural structures operating by multidimensional signal vectors.

Neural Net -the Analyzer of Multidimensional Signals
As mentioned above, nervous system is the net of many thousands of neurons functioning in parallel and the abundance of parallel channels of information which start at receptors and end at effectors with collateral informational interactions as well.Therefore, after getting to know the possibilities of elementary neural net, it is essential to design the possibilities of neural analyzer with more complex and numerous entry signals.The neuroscheme of three-feature fuzzy analyzer given in Fig. 9 also fairly clear demonstrates the possibilities of n-dimensional analyzer of neural signals.
When multidimensional positive-value signal coming from n + 1 entries is analyzed, first of all it is recommended to compose analyzer of [(n + 1)n]/2 elementary reciprocal pairs of neurons, which would analyze the signals' interrelationships of every entry pair by principle "more-less", i.e., the excited would become only that neuron which receives more intense signal to its exciting synapsis.It is obvious that altogether there can be (n + 1)! states of analyzing structure which correspond to the permutational combinations of the quantity of entry signals and to the respective number of reciprocal chains of primary analyzer.This neuronal analyzer is differentiator and every N (X i − X k ), N (X k − X i ) neuron of its reciprocal pair "cuts" the space of entry signals by hyperplane to two symmetric pieces which pass through the central axis "all equal" (X 1 = X 2 = . . .= X n = X n−1 ), divide the plane X i 0X k through its middle and pass through all other axes as well (Fig. 10).Such neuronal analyzer subdivides all the positive quasi-octant to (n + 1)! symmetric sectors and every of them match the direction of entry signals' vectors, which in its turn fulfills the corresponding inequable alignment according to the value of signal Hereafter the neural net can be organized by means of pyramidal neurons and their inhibitory connections in such way that the corresponding pyramidal neuron would be excited only in that case if entry signals' vector lies in that sector.It is possible to form N 1 = (n + 1)! pyramidal neurons which would identify a single concept.
It should be noted that stellate neurons' analyzer with entry of n + 1 positive continuous signals can have 2 [(n+1)n]/2 independent states when only the orientations (signs) of reciprocal pairs' excitement states are assessed.If signals' entries are only positive continuous values, n reciprocal pairs will be sufficient for fuzzy logic, whereas all other (n + 1)/2 pairs will have doubling role thus increasing the reliability of analyzer.(Fig. 9 shows inhibitory connections of doubling pairs by dotted line).When next, the second layer neuronal differentiator is formed it is already possible to assess the factorial combinations of entry signals ranged not only according to their value, but also according to the excitement states' signs of n reciprocal pairs.Therefore due to the second analyzing layer it gets possible to filter or form N 2 = 2 n n! determinative pyramidal neurons which distinguish for their much more selective identification of entry signals' vectors.(The Fig. 10.Three-dimensional positive-feature vectors situated according to more-equal-less fuzzy logic (geometric picture).In the middle X 1 = X 2 = X 3 .result of second level analysis shown by dotted line in Fig. 9 divides every sector of first level analysis to two more pieces).
In theory, if there are n + 1 positive continuous entry signals, it is possible to form the neuronal analyzer of n hierarchical levels which functions by principle of "more-less" fuzzy logic and is able to identify Such amount of states is possible only in theory because when level of differentiation is increased in practice, the noise becomes very pronounced and the analyzer becomes unreliable.However, even in case of two analyzing levels, there are so many selective states of multidimensional positive continuous signals that it is possible to identify only slightly varied signals.In general, neural analyzer distinguishes for its great possibilities of resolution and identification.
Pyramidal neurons, similarly as the operator of arithmetic condition (Fig. 7) can also filter situations marked "equal" and combinations of various equalities.This is achieved when both suppressive entries of both neurons of the reciprocal pair are brought into the same pyramidal neuron.It enables to increase the number of multidimensional entry signals' qualitative identification's possibilities to much greater extent.
It should be also noted that since pyramidal neurons filter only the direction of multidimensional entry signals' vector in sector, such neural identificator is insensible and invariant to the variations of sensibility and noise levels of all neurons.

Conclusions
1. Neurons -the summators of impulse frequencies characterized by non-linearity N , and the elementary neural structures made of them perform operations of fuzzy logic which can be described as qualitative (not quantitative) informational estimations "more", "less", "equal", i.e., they function according to "more-equal-less" (M-E-L) logic.
2. Pyramidal neurons with suppressing entries perform algebraic signal operations and as a result of them the output signals are controlled by means of universal logical function "NON disjunction" (Pierce arrow or Dagger function).
3. Stellate and pyramidal neurons can be used to synthesize the neural nets functioning in parallel and realizing all logical and elementary algebraic functions as well as to perform the conditional controlled operations of information processing.
4. Neural nets are informational subsystems of parallel functioning and analogical neurocomputers of hybrid action.

Fig. 2 .
Fig. 2. Schemes of stellate and pyramidal neurons, their functional characteristics and graphical picture of their neural non-linearity N .

Fig. 6 .
Fig. 6.Neuronal structures that realize multiple entries' MIN and MAX operations of fuzzy logic.

Fig. 7 .
Fig. 7. Neural structure -the operator of arithmetic condition IF (Arithmetic expression s i x i ) m − , m 0 , m + .
. The concept of linguistic variable and its application to approximate reasoning III.Kirvelis has graduated from Kaunas Polytechnic Institute(computer sciences, 1962)and postgraduate bionics studies in the Institute of Control Problems at Academy of Sciences USSR (Moscow) 1967.He is an associated professor at Vilnius University, Department of Biochemistry and biophysics, PhD degree in biophysics.His research interest and teaching activity include biometrics, neuro-bioinformatics, neuro-biocybernetics, and biophysics of organized systems.He is a member of the Information Society National Committee of Seimas of the Republic of Lithuania.G.Dagytė has graduated from Vilnius University, Faculty of Medicine in 2003.She is an assistant professor at Vilnius University, Department of Anatomy, Histology and Anthropology.Her research interests are cognitive neuroscience, neuroanatomy as well as biophysical processes in brain, and teaching topic is human anatomy. D.