The human brain consists of two major classes of cells in the brain: neurons and glia. The basic difference between these lies in their electrical excitability. Neurons are electrically excitable, whereas glia is nonexcitable nerve cells [1] . Here the focus is put on astrocytes, the main glial cells besides oligodendrocytes and microglia. Astrocytes are the most numerous glial cell types and account for up to one third of brain mass.

The arrangement of astrocytes in non-overlapping domains [2] [3] with other astrocyte domains via gap junctions is basically similar to that within a single astrocyte [4] [5] . Perisynaptic astrocytic processes (PSAPs) emanate from the body of the astrocyte stem processes and envelope neuronal synapses building tripartite synapses [6] . A tripartite synapse consists of the pre-synapse and the post- synapse as neuronal components, and the astrocyte with its network as the glial component [7] . Importantly gap junctions (GJs) not only connect different astrocytes but also PSAPs of the same astrocyte with the potential to form an operational signaling network within this cell [8] [9] .

In the present perspective I will elaborate on this intriguing hypothesis, arguing that such cellular astrocytic networks could carry out self-reflexive operations based on their cyclical structure. This cyclical structure is supported by arguments of proemial counting which, differently from classical counting, consists of oriented counting operations [10] [11] .

This perspective is organized as following. First I will present the essential biological picture focusing on the morphology of astrocytes and network of astrocytes. Then, based on proemial counting, I will introduce a formal description of cyclic organization of intracellular astrocytic networks known as permutograph. It can be shown that permutographs enable the generation of Hamilton loops possibly running in the astroglial networks. These cyclic pathways are interpreted as intentional action programs. Furthermore, based on a model of the reticular formation (RF) of the brainstem it is suggested that astrocytes may codetermine neuronal functions in the RF and may also be organized in astroglial networks where cyclic pathways are generated. The modular organization of the RF may generate specific action programs. In the direction of the proposed model of autonomous agents, a sample computer model of the RF is discussed. Finally, I present a link for astrocytic self-reflection in connection with the emergence of the “self” in living organisms.

The biological system of our interest is brought forth by gap junctions (GJs) between astrocytes [12] . These junctions which consist of connexin proteins of different kind [13] interconnect astrocytes forming extended networks of variegate topologies [14] . Most importantly however, GJs are also found within a single astrocyte, connecting its processes (PSAPs) in a nontrivial fashion, thereby forming intracellular networks, called self-reflexive gap junctions [4] [9] [15] .

Primarily five to eight processes per cell emanate from the cell body of protoplasmic astrocytes [16] . They subdivide progressively to form finer and finer processes of up to 100.000 per astrocyte. Most of these processes interact with synapses, termed perisynaptic astrocytic processes (PSAPs). Fewer and thicker ones contact capillaries, arterioles and venules, termed perivascular astrocytic processes [17] . The remaining processes (average of 11) contact the processes of their nearest neighbor astrocytes and form gap junctions with these to form the astrocyte network [12] . Importantly, PSAPs extend to or withdraw from synapses in an activity-dependent manner [18] .

Figure 1 illustrates a self-reflexive network within an astrocyte. Six perisynaptic astrocytic processes (PSAP₁ ・・・ PSAP₆) are completely interconnected via fifteen gap junctions (g.j.). Each PSAP is in tight morphological and functional association with a synapse (Sy₁, only one is shown). As elaborated below, the bidirectional interactions between PSAPs via GJs generate cyclic pathways in the sense of self-reflexive operations (see Section 6).

In Figure 2 a diagram of modulation of synapses by motile PSAPs is shown. A stem process (a) which emerges from the astrocyte soma arborizes in PSAPs. These PSAPs contact and retract [19] receptors (1 ・・・ 6) on the synaptic sheath. As an example, astrocytic processes interchange between contacting (directed

arrows) and retracting (inverse arrows). In rapid or longer time scales PSAPs select a new action program. This selection operation exerts a feedforward function modifying the action program as a relevant synaptic pattern (1, 3, and 5) that feeds back to the endfeet of PSAPs, hereby generating a new pattern of astrocytic-synaptic interaction. The action pattern is activated within milliseconds (t₁), the action program is generated in seconds to minutes (t₂) and the integration of action programs in the soma may occur in minutes to hours or longer (t₃) [20] . As observed in experiments, the body of the astrocyte exerts a pacemaking via motile PSAPs [21] . Since this rhythmic pulsation of the astrocyte occurs in a time scale of minutes, the soma of the astrocyte could play a significant role in the modulation of synaptic information processing by PSAPs.

Basically, astrocytes enable a balanced and easily tunable feedforward and feedback response to neuronal activity that has the potential to regulate the propagation and processing of information across neuronal circuits, because each PSAP may ensheathe several synapses [22] which may belong to different dendrites and/or different neurons, thereby interacting with multiple flows of information. I argue that in order to multiplex such a body of variegate information, PSAPs function in conjunction with synaptic elements as “information structuring” elements, filtering the synaptic flow of information.

The close morphological relations between astrocytes and synapses together with the functional expression of relevant receptors in the astroglial cells led to the development of a new concept of synaptic organization, known as the “tripartite synapse” [6] . In line with this concept, synapses are built from equally important parts: the pre-synaptic terminal, the postsynaptic neuronal membrane, and the surrounding astrocyte [1] .

Figure 3 outlines a tripartite synapse. It consists of the pre- and post-synapse

as the neuronal component and the astrocyte as the glial component. Astrocyte-synaptic interactions in a chemical tripartite synapse are outlined as follows [23] : Neurotransmitters (NT) released from vesicles (Ve) in the pre-synapse activate both postsynaptic receptors and G-protein coupled receptors (GPCRs) on the astrocyte [24] . Retrograde messengers (rm) like cannaboids can also bind astrocyte GPCRs transiently elevating Ca²⁺. In parallel, neuromodulators such as acetylcholine and noradrenaline may trigger astrocytic GPCR-dependent Ca²⁺ signaling. In addition, other pathways for activity-dependent increases of astrocytic Ca²⁺ not involving GPCRs also exist [25] [26] . The increase of cytosolic Ca²⁺ may promote release of gliotransmitters (GT) from the astrocyte affecting pre- and post-synaptic transmission. On the pre-synapse GT (e.g. glutamate, adenosine-triphosphate, adenosine) activate extrasynaptic receptors which modulate the release probability of synaptic transmission. Postsynaptically, N-me- thyl-D-aspartate receptors are activated and controlled by astrocyte-de- rived D-serine. Dependent on the brain regions other substances such as glutamate, gamma-amino-buturic acid or tumor necrosis factor α (TNFα) released from the astrocyte also bind postsynaptic and extrasynaptic receptors. Importantly, pre- and post-synaptic signaling pathways and the release of GT may only function under special conditions based on permission factors such as ambient TNFα [27] [28] .

In addition to experiments, mathematical and computer modeling is proving crucial to test predictions on the possible functions of astrocytes in neuronal networks and may generate novel ideas as to how astrocytes contribute to the complexity of the brain [29] [30] . Volman and coworkers [31] suggest a biophysical model for the coupling between synaptic transmission and the local calcium concentration on an enveloping astrocyte domain. Due to this interaction the astrocyte is able to modulate the information flow from pre-synaptic to postsynaptic cells in a specific way according to previous activity at this and other nearby synapses. Hence, the astrocyte may exert a “gate-keeping effect”. Nadkarni and coworkers [32] further showed that control of synaptic transmission by astrocytes could optimize synaptic information transfer. In all these studies, the modulatory action of astrocytes on synaptic function is crucially mediated by intracellular dynamics of astrocytic Ca²⁺. Recall in fact that astrocytes respond to synaptic activity through their intracellular Ca²⁺ dynamics, which in turn feeds back to neurons by triggering release of gliotransmitters (GT) [23] [33] .

Low frequency synaptic activity leading to local astrocyte Ca²⁺ signals is likely to exert localized gliotransmission that will be restricted to cause feedback modulation of the active synapse. However, with an increased frequency of synaptic activity the capability of the astrocyte Ca²⁺ signal to spread through the processes and even fill the entire astrocyte, bears the potential for the resulting gliotransmission to exert feedforward actions on the synapse at multiple distinct locations [34] . Experimental findings indicate that tripartite synapses and the astroglial network process information on feedforward and feedback loops [17] . Following, I introduce a formal description of these loops and show how they can be combined to generate cyclic pathways.

Taking a closer look at Figure 3 we can see that astrocyte-synaptic interactions run in loops. The activation of astrocytic receptors by NT and neuromodulatory afferents is fed back by GT and rm building loops. DePitta and coworkers [23] provide a simple formal description of loop generation in tripartite synapses. Whereas a conventional synapse consisting of the two components of the pre- and post-synapse processes information only directionally via one input and one output function, a tripartite synapse consisting of the three components of the pre- and post-synapse and the astrocyte processes information in loops. In these loops no unidirectionality is given, since two input functions and two output functions operate [23] . Most importantly, the notion of “no unidirectionality” is at the core of proemial counting [11] . The notion of proemial counting is illustrated in Figure 4 where 6 synapses, labeled from a to f, are proemially interre-

lated. In proemial counting formalism, each synapse constitutes a counting “place”. A single counting step of proemial counting could start for example from a, then move to b and finally return to a, thereby making a cyclic pathway [a, b, a]. This counting can be repeated for all the other synapses generating cyclic pathways in a hierarchical order, i.e.

a b a

a b c a

a b c d a

a b c d e a

a b c d e f a

It may be noted that the proemial relation in each counting step not only generates cyclic pathways but also leads to an increase in the extent and composition of the pathway itself, insofar as progressively more synaptic places are included in pathways of higher hierarchy. In this fashion as the pathway structure increases in complexity, so does the underpinning counting mechanism. And because we may assume that each astrocyte-synapse coupling is different in its receptor composition, then each pathway includes a specific set of synaptic receptors which are expected to convey unique information to and from the astrocyte.

A further complication is that not all synapses need to be activated with the possibility of incomplete pathways to be generated. This concept is illustrated in Figure 5 showing an incomplete activation of synapses dependent on the selection mechanism of relevant synapses in astrocyte-synaptic information processing, as is the case in normal adult brain function [20] . Here, the synapses a, b, and d are activated so that the proemial relationship rejects synapse c. The resulting cyclic pathways are “leaky” consisting only of two cycles: (a b a) and (a d a). As already described, the selection pattern of synapses is postulated to be defined by GJs in the astrocyte in combination with the spatiotemporal dynamics of activation of synapses within the astrocytic anatomical domain. The fact that however synapses or bundle of synapses can be regarded as autonomous functional units [35] suggests that the network of tripartite synapses brought forth by GJs in an astrocyte could similarly be considered as a self-consistent operational unit. This is supported by the recognition that cyclic pathways in such networks are essentially Hamilton loops, i.e. loops originating by a minimal, preserved number of elementary logical operations. Cyclic pathways as Hamilton loops may not only generate autonomous functional units but could also play a role in energy preservation.

In graph theory, Hamilton loops are circular pathways that visit each node of a graph once and only once except for the node where they start and end (which is clearly visited twice) [36] . Consider for example the lexicographically-ordered sequence 1-2-3-4 and the quadrivalent permutation providing 4-3-2-1 (Table 1). This permutation can be generated by a composition of the following three

This permutation system consists of 24 permutations (1 × 2 × 3 × 4, ・・・, 4 × 3 × 2 × 1) according to the formula n = 4! (factorial) = 1 × 2 × 3 × 4 = 24. The 24 permutations are lexicographically arranged.

exchange operators: N₁ ・・・ N₂ ・・・ N₃. Thus it is immediately clear that N₁ exchanges the integers (values) 1↔2; N₂ exchanges 2↔3, and N₃ exchanges 3↔4.

Generally, a Hamilton loop results from any combination of the above three operators that span the whole space of permutations of the sequence 1-2-3-4. In a quadrivalent system for example there are 44 possible Hamilton loops [33] . Table 2 illustrates one such loop resulting in the sequence of operators:

The first permutation (P = 1 × 2 × 3 × 4) is permutated via a sequence of negation operators (N₁ × 2 × 3 ・・・ 2 × 1 × 2) generating the permutations once until the loop is closed (1234) in the sense of a Hamilton loop.

N₁-N₂-N₃-N₂-N₃-N₂-N₁-N₂-N₁-N₂-N₃-N₂-N₃-N₂-N₁-N₂-N₁-N₂-N₃-N₂-N₃-N₂-N₁-N₂

Such sequences of permutations can be formalized in networks of permutations known as permutographs [37] . Figure 6 shows a quadrivalent permutograph where the exchange operators N₁, N₂, N₃ are represented between permutations (1 ・・・ 24). The various Hamilton loops are different according to exchange operator sequence. For example, the Hamilton loop in Table 2 is depicted in the permutograph (Figure 6) by a dash-dotted line.

Importantly Hamilton loops that are generated by permutations may re- present action programs where astrocyte-synaptic interaction and astroglial networks may play a key role. It is experimentally identified that feedforward and feedback mechanisms between astrocytes and neurons at each synapse are responsible for the coding and integration of calcium waves as they travel through the astroglial network [38] [39] [40] . This bidirectional communication between astrocytes and synapses corresponds to my perspective that permutations operating on exchange relations can generate cyclic pathways as Hamilton loops described above [41] . Importantly each PSAP is a member of the astroglial network. Astrocytic processes establish contact with approximately 145.000 synapses, each of which acts as a subcellular microdomain for information processing via Ca²⁺ signaling [42] .

In the present perspective the generation of cycles between PSAPs via gap junctions (GJs) in a single astrocyte and in the gap junction network between astrocytes may represent action cycles that function as action programs. Astrocytes may exert a double function in this process. They are both activated by the neuronal system and may also autonomously determine effects on neuronal activation [21] . An action program, formally described as a Hamilton loop, could function as an intentional program defined as follows: an intentional program generates a multirelational structure in the GJ networks based on the principle of that program [29] [41] . If a cyclic pathway is frequently activated by the neuronal system it could function as a hypothetical action program for testing in a changing environment.

Action programs that astrocytic networks could deploy are seen in several brain

nuclei and notoriously in vital brain nuclei in the brain stem. Basically, the respiratory neuronal network activity in the reticular formation (RF) requires efficient functioning of astrocytes. Hülsman and coworkers analyzed neuron-as- trocyte communication in the pre-Bötzinger Complex (preBötC) of rhythmic slice preparations from neonatal mice [43] . In a model simulation it was confirmed that preinspiratory astrocytic signals arise from coupled neuronal and astrocytic oscillations, supporting a concept that slow oscillatory changes of astrocyte functions modulate neighboring neuronal activity to add variability in respiratory rhythm [44] . Morquette and coworkers [45] demonstrated that in the rat trigeminal sensorimotor circuit for mastication this ability depends on the regulation of the extracellular Ca²⁺ concentration by astrocytes. In this circuit astrocytes respond to sensory stimuli that induce neuronal activity. Excitingly, functional calcium imaging from hundreds of preBötC cells revealed that a subset of putative astrocytes exhibited rhythmic calcium elevations ahead of inspiratory neuronal activity with a time lag of approximately 2 seconds. These preinspiratory putative astrocytes were able to maintain rhythmic activities even during blockade of neuronal activity with tetrodotoxin, whereas the rhythm frequency was decreased and the intercellular phases of these rhythms were decoupled. In addition, optogenetic stimulation of preBötC putative astrocytes caused firing of inspiratory neurons [46] . These experiments indicate that astrocytes may be involved in respiratory rhythm generation. Moreover, it cannot be excluded that astrocytes codetermine neuronal functions in the RF and that they may be organized in astroglial networks where cyclic pathways that function as action programs could be generated.

Since McCulloch’s seminal work [47] the RF has been identified as one of the brain nuclei which prominently integrate different signals. The RF is a set of interconnected nuclei located throughout the brain stem. The functions are modulatory with premotor functions localized in more caudal regions. While there are some well-localized nuclei within the RF, most of the RF is involved in integrative functions. A unique characteristic of RF neurons is their widespread system of axon collaterals making extensive synaptic contacts with their ability to travel over long distances in the brain.

On the behavioral level animals or humans must not only select action programs adapting to the environmental situation, but it is also possible to intend an action in the environment. Considering the pattern generation nuclei and their network, these units or modules enable specific motoric and autonomous functions elementary for behavior. For example, if one intends to eat, mastification is necessary so that the modular organization of the nucleus trigeminus in the RF must be activated. Although selection mechanisms of action programs may already operate in the RF, intended modes of behavior may be mainly selected in the astroglial networks of the prefrontal cortex.

With concern to our graphtheoretical formal approach to a simulation of the RF, Humphries and coworkers [48] suggested a formal model of the RF similar to our model, but it does not refer to astrocytes and the astroglial networks. Based on the modular organization of the RF, the McCulloch group developed a computer simulation of the RF, called vertebrate central command system [49] . Originally, I also worked out a computer system for simulating RF operations which focused on the neuronal system [50] . It was implemented as follows:

1) Intrasystem intention takes action, in other words: constant readiness to make decisions;

2) Data input to neurons through appropriate environmental information;

3) Action taking by the entire system in the form of a specific behavior, with the programs for actions being selected on the basis of environment imposed level of priority;

4) Intrasystem reconstruction in the sense of intention to take further action.

However, my recent model of the RF also refers to the astroglial networks [51] . If we suppose that information processing in the RF is based on interactions between astrocytes and neurons in tripartite synapses and in the astroglial networks, then the astroglial network may play a central role in the specification and integration of action programs. Experimental data obtained from calcium imaging experiments indicate that not all astrocytes in a defined brain area express the same amount or the same catalogue of membrane receptors. This implies that only subpopulations of astrocytes initially respond to a defined neuronal signal. However, the expression of a large number of GJ channels enables communication with nonresponding astrocytes, but when communication takes place, almost all astrocytes can sense the signal in time [8] . Here, one should also consider reflexive GJs in a single astrocyte.

On a modular basis a computer system of the RF is outlined in Figure 7. The simulation of the glial system (astrocytes and astroglial networks) entails a central processing module, a command computer structured on the basis of a permutograph (Figure 6). The permutograph consists of a plurality of storage modules corresponding to the edges of the graph. The connection lines establish internal circuits corresponding to the exchange operator sequences of the permutograph in the form of Hamilton loops, each of which correlates with a behavior pattern generated in the RF. The command computer is controlled by input computers in which preprogrammed intended action is connected with environmental information. The relations computer is positioned in different types of perception systems [51] . In my first version of a computer system for the simulation of the RF the command computer was situated in the neuronal network [50] , which is not necessary if one attributes the generation of intended action programs to the astroglial network. Here, the neuronal network is only implemented by the executive computer for executing the modes of behavior.

Since the RF interacts with all other brain regions, especially the limbic system and the cerebral cortex, it also enables the integration of generated action programs with the actual information of perception and motor systems [52] . As an example, we consider the action sequence “look”, “forward”, “stop”, “retreat”. This program sequence is executed by a storage module associated with Hamilton loops 1 ・・・ 4. Each Hamilton loop represents an intentional action program excited in the neuronal system dependent on the environmental information computed by the perception systems. This run is monitored by a timing control unit. During the run of the program the perception and relations computer

constantly process new information that is compared to the intended actions. For example: let us suppose that during the execution of the intended action program the perception system detects an obstacle. An object stands in the way. Program 3 gets active (“stop”) and it is necessary to “retreat” (Program 4), then “look” for a new path (Program 1) and move “forward” (Program 2). If after weighting the relations computer identifies this new program sequence as high priority, the relations computer decides to interrupt the execution in the command computer and switches to the new program sequence.

The significance of the present perspective lies in presenting a brain-based model for robot brains towards machine consciousness. The model focuses on astrocytes that generate functional loops by means of self-reflexive GJs. These loops are ascribable to operational networks, from the perspective of logic, where each node is represented by a tripartite synapse and self-contained. These networks implement action programs and can be ultimately responsible for a whole spectrum of information processing in living organisms, from action to intentions. Importantly, the central role of astrocytes in information processing [5] has been envisaged decades ago by Galambos in his “glial-neuronal theory of the brain” [53] .

Finally, my hypothesis that the astrocyte functions as a mediator for self-re- flexive agents implies that it is able to interpret the environment only in terms of “its own” previously acquired situation models and has to integrate previous and novel models in a consistent behavior-related way [54] . Considering proemial counting it represents a self-reflexive or self-referential operation. Since the concept of self characterizes living systems with first person consciousness, an agent may only be capable of reflecting its interactions with the inner and outer environment as “its own” in the sense of auto-reflection [55] . Currently, it is an open question if the modeler of an agent is able to implement his/her Self in a robot brain.

I am very grateful to Birgitta Kofler-Westergren for preparing the final version of this paper.

Mitterauer, B.J. (2017) The Astrocyte as a Mediator for Self-Reflexive Agents. Advances in Bioscience and Biotechnology, 8, 195-210. https://doi.org/10.4236/abb.2017.86015