From terry@salk.edu Fri Mar 14 18:37:52 1997 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id SAA08500 for ; Fri, 14 Mar 1997 18:37:45 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.7.6/8.7.3) with SMTP id SAA14532 for ; Fri, 14 Mar 1997 18:37:25 -0600 (CST) Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa04110; 14 Mar 97 18:05:50 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa04108; 14 Mar 97 17:40:40 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa25294; 14 Mar 97 17:40:17 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa04919; 14 Mar 97 17:00:19 EST Received: from helmholtz.salk.edu by RI.CMU.EDU id aa06407; 14 Mar 97 16:59:10 EST Received: (from terry@localhost) by helmholtz.salk.edu (8.7.5/8.7.3) id NAA19854; Fri, 14 Mar 1997 13:59:05 -0800 (PST) Date: Fri, 14 Mar 1997 13:59:05 -0800 (PST) From: Terry Sejnowski Message-Id: <199703142159.NAA19854@helmholtz.salk.edu> To: connectionists@cs.cmu.edu Subject: Telluride Deadline April 1 Cc: terry@helmholtz.salk.edu ****** Deadline for application is April 1, 1997 ***** "NEUROMORPHIC ENGINEERING WORKSHOP" JUNE 23 - JULY 13, 1997 TELLURIDE, COLORADO Christof Koch (Caltech) Terry Sejnowski (Salk Institute/UCSD) and Rodney Douglas (Zurich, Switzerland) invite applications for a three-week summer workshop that will be held in Telluride, Colorado in 1997. The 1996 summer workshop on "Neuromorphic Engineering", sponsored by the National Science Foundation, the Gatsby Foundation and by the "Center for Neuromorphic Systems Engineering" at Caltech, was an exciting event and a great success. A detailed report on the workshop is available at http://www.klab.caltech.edu/~timmer/telluride.html GOALS: Carver Mead introduced the term "Neuromorphic Engineering" for a new field based on the design and fabrication of artificial neural systems, such as vision systems, head-eye systems, and roving robots, whose architecture and design principles are based on those of biological nervous systems. The goal of this workshop is to bring together young investigators and more established researchers from academia with their counterparts in industry and national laboratories, working on both neurobiological as well as engineering aspects of sensory systems and sensory-motor integration. The focus of the workshop will be on "active" participation, with demonstration systems and hands-on-experience for all participants. Neuromorphic engineering has a wide range of applications from nonlinear adaptive control of complex systems to the design of smart sensors. Many of the fundamental principles in this field, such as the use of learning methods and the design of parallel hardware, are inspired by biological systems. However, existing applications are modest and the challenge of scaling up from small artificial neural networks and designing completely autonomous systems at the levels achieved by biological systems lies ahead. The assumption underlying this three week workshop is that the next generation of neuromorphic systems would benefit from closer attention to the principles found through experimental and theoretical studies of brain systems. FORMAT: The three week summer workshop will include background lectures, practical tutorials on aVLSI design, hands-on projects, and special interest groups. Participants are encouraged to get involved in as many of these activities as interest and time allow. There will be two lectures in the morning that cover issues that are important to the community in general. Because of the diverse range of backgrounds among the participants, the majority of these lectures will be tutorials, rather than detailed reports of current research. These lectures will be given by invited speakers. Participants will be free to explore and play with whatever they choose in the afternoon. Projects and interest groups meet in the late afternoons, and after dinner. The aVLSI practical tutorials will cover all aspects of aVLSI design, simulation, layout, and testing over the workshop of the three weeks. The first week covers basics of transistors, simple circuit design and simulation. This material is intended for participants who have no experience with aVLSI. The second week will focus on design frames for silicon retinas, from the silicon compilation and layout of on-chip video scanners, to building the peripheral boards necessary for interfacing aVLSI retinas to video output monitors. Retina chips will be provided. The third week will feature a session on floating gates, including lectures on the physics of tunneling and injection, and experimentation with test chips. Projects that are carried out during the workshop will be centered in a number of groups, including active vision, audition, olfaction, motor control, central pattern generator, robotics, multichip communication, analog VLSI and learning. The "active perception" project group will emphasize vision and human sensory-motor coordination. Issues to be covered will include spatial localization and constancy, attention, motor planning, eye movements, and the use of visual motion information for motor control. Demonstrations will include a robot head active vision system consisting of a three degree-of-freedom binocular camera system that is fully programmable. The "central pattern generator" group will focus on small walking robots. It will look at characteristics and sources of parts for building robots, play with working examples of legged robots, and discuss CPG's and theories of nonlinear oscillators for locomotion. It will also explore the use of simple aVLSI sensors for autonomous robots. The "robotics" group will use robot arms and working digital vision boards to investigate issues of sensory motor integration, passive compliance of the limb, and learning of inverse kinematics and inverse dynamics. The "multichip communication" project group will use existing interchip communication interfaces to program small networks of artificial neurons to exhibit particular behaviors such as amplification, oscillation, and associative memory. Issues in multichip communication will be discussed. PARTIAL LIST OF INVITED LECTURERS: Andreas Andreou, Johns Hopkins. Richard Andersen, Caltech. Dana Ballard, Rochester. Avis Cohen, Maryland. Tobi Delbruck, Arithmos. Steve DeWeerth, Georgia Tech Rodney Douglas, Zurich. Christof Koch, Caltech. John Kauer, Tufts. Shih-Chii Liu, Caltech and Rockwell. Stefan Schaal, Georgia Tech Terrence Sejnowski, UCSD and Salk. Shihab Shamma, Maryland. Mark Tilden, Los Alamos. Paul Viola, MIT. LOCATION AND ARRANGEMENTS: The workshop will take place at the "Telluride Summer Research Center," located in the small town of Telluride, 9000 feet high in Southwest Colorado, about 6 hours away from Denver (350 miles) and 5 hours from Aspen. Continental and United Airlines provide many daily flights directly into Telluride. Participants will be housed in shared condominiums, within walking distance of the Center. Bring hiking boots and a backpack, since Telluride is surrounded by beautiful mountains (several mountains are in the 14,000 range). The workshop is intended to be very informal and hands-on. Participants are not required to have had previous experience in analog VLSI circuit design, computational or machine vision, systems level neurophysiology or modeling the brain at the systems level. However, we strongly encourage active researchers with relevant backgrounds from academia, industry and national laboratories to apply, in particular if they are prepared to talk about their work or to bring demonstrations to Telluride (e.g. robots, chips, software). Internet access will be provided. Technical staff present throughout the workshops will assist with software and hardware issues. We will have a network of SUN workstations running UNIX, MACs and PCs running LINUX (and windows). COST: Scholarships are available to reimburse some participants for up to $500 for domestic travel and for all housing expenses. Please specify on the application whether such financial help is needed. DURATION: Unless otherwise arranged with one of the organizers, we expect participants to stay for the duration of this three week workshop. Because of the intensity of the workshop and the focus on projects, spouses and families cannot be accommodated during the workshop. HOW TO APPLY: The deadline for receipt of applications is April 1, 1997. Applicants should be at the level of graduate students or above (i.e. post-doctoral fellows, faculty, research and engineering staff and the equivalent positions in industry and national laboratories). We actively encourage qualified women and minority candidates to apply. Application should include: 1. Name, address, telephone, e-mail, FAX, and minority status (optional). 2. Curriculum Vitae. 3. One page summary of background and interests relevant to the workshop. 4. Description of special equipment needed for demonstrations that could be brought to the workshop. 5. Two letters of recommendation Complete applications should be sent to: Terrence Sejnowski The Salk Institute 10010 North Torrey Pines Road San Diego, CA 92037 email: terry@salk.edu FAX: (619) 587 0417 Applicants will be notified around May 1, 1997. From ataxr@IMAP1.ASU.EDU Mon Mar 17 00:49:38 1997 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id AAA15353 for ; Mon, 17 Mar 1997 00:49:32 -0600 Received: from cs.uwa.oz.au (bilby.cs.uwa.oz.au [130.95.1.11]) by lucy.cs.wisc.edu (8.7.6/8.7.3) with SMTP id AAA20314 for ; Mon, 17 Mar 1997 00:49:26 -0600 (CST) Received: from (parma.cs.uwa.oz.au [130.95.1.7]) by cs.uwa.oz.au (8.6.8/8.5) with SMTP id MAA19981; Mon, 17 Mar 1997 12:56:49 +0800 Message-Id: <199703170456.MAA19981@cs.uwa.oz.au> From: Asim Roy To: reinforce@cs.uwa.edu.au Subject: Does plasticity imply local learning? And other questions Date: Sat, 15 Mar 1997 10:48:28 -0500 (EST) I am posting the responses I have so far without comment. Some of the responses provide a great deal of insight on this topic. I hope this will generate more interest in the questions raised. The original posting is attached below for reference. Asim Roy Arizona State University ============================================================= >From Anthony Harris Here are some thoughts: One general point is that I'm not entirely sure about what is meant by theglobal/local distinction. Certainly action at a distance can't take place; something physical happens to the cell/connection in question in order for it to change. As I understand it, the prototypical local learning is a Hebbian rule, where all the information specifying plasticity is in the pre and post-synaptic cells (ie "local" to the connection), while a global learning rule is mediated by something distal to the cell in question (i.e. a neuromodulatory signal). But of course the signal must contact the actual cell via diffusion of a chemical substance (e.g. dopamine). So one different distinction might be how specific the signal is; i.e. in a local rule like LTP the information acts only on the single connection, while a modulatory signal could change all the connections in an area by a similar amount. However, the effects of a neuromodulator could in turn be modulated by the current state of the connection - hence a global signal might act very differently at each connection. Which would make the global signal seem local. So I'm not sure the distinction is clearcut. Maybe its better to consider a continuum of physical distance of the signal to change and specificity of the signal at individual connections. A couple of specific comments follow: > A) Does plasticity imply local learning? > > The physical changes that are observed in synapses/cells in > experimental neuroscience when some kind of external stimuli is > applied to the cells may not result at all from any specific > "learning" at the cells.The cells might simply be responding to a > "signal to change" - that is, to change by a specific amount in a > specific direction. In animal brains, it is possible that the > "actual" learning occurs in some other part(s) of the brain, say > perhaps by a global learning mechanism. This global mechanism can > then send "change signals" to the various cells it is using to > learn a specific task. So it is possible that in these > neuroscience experiments, the external stimuli generates signals > for change similar to those of a global learning agent in the > brain and that the changes are not due to "learning" at the cells > themselves. Please note that scientific facts/phenomenon like > LTP/LTDor synaptic plasticity can probably be explained equally > well by many theories of learning (e.g. local learning vs. global > learning, etc.). However, the correctness of an explanation would > have to be I think it would be difficult to explain the actual phenomenon of LTP/LTD as a response to some signal sent by a different part of the brain, since a good amount of the evidence comes from in vitro work. So clearly the "change signals" can't be coming from some distant part of the brain - unless the slices contain the necessary machinery for generating the change signal. Also, its of course possible that LTP/LTD local learning rules act in concert with global signals (as you mention below); these global signals being sent by nonspecific neuromodulators (an idea brought up plenty of times before). I'm not sure about the differences in the LTP/LTD data collected in vivo versus in vitro; I'm sure there are people out there studying it carefully, and this could provide insight. > > B) "Pure" local learning does not explain a number of other > activities that are part of the process of learning!! > > When learning is to take place by means of "local learning" in a > network of cells, the network has to be designed prior to its > training. Setting up the net before "local" learning can proceed > implies that an external mechanism is involved in this part of > the > learning process. This "design" part of learning precedes actual > training or learning by a collection of "local learners" whose > only > knowledge about anything is limited to the local learning law to > use! Of course, changing connection strengths seems to be the last phase of the "learning/development" process. Correct numbers of cells need to be generated, they have to get to their correct locations, proper connections between subpopulations need to be established and refined, and only at this point is there a substrate for "local" learning. All of these can be affected to a certain extent by environment. For example, the number of cells in the spinal cord innervating a peripheral target can be downregulated with limb bud ablation; conversely, the final number can be upregulated with supernumerary limb grafts. Another well known example is the development of ocular dominance columns. Here, physical connections can be removed (in normal development), or new connections can be established (evidence for this from the reverse suture experiments), depending on the given environment. What would be quite interesting would be if all these developmental phases are guided by similar principles, but acting over different spatial and temporal scales, and mediated by different carriers (e.g. chemical versus electrical signals). Alas, if only I had a well-articulated, cogent principle in hand with which to unify these disparate findings; my first Nobel prize would be forthcoming. In lieu of this, we're stuck with my ramblings. > > In order to learn properly and quickly, humans generally collect > and store relevant information in their brains and then "think" > about it (e.g. what problem features are relevant, problem > complexity, etc.). So prior to any "local learning," there must > be processes in the brain that examine this "body of > information/facts" about a problem in order to design the > appropriate network that would fit the problem complexity, select > the problem features that are meaningful, etc. It would be very > difficult to answer the questions "What size net?" and "What > features to use?" without looking at the problem in great detail. > A bunch of "pure" local learners, armed with their local learning > laws, would have no clue to these issues of net design, > generalization and feature selection. > > So, in the whole, there are a "number of activities" that need to > be performed before any kind of "local learning" can take place. > These aforementioned learning activities "cannot" be performed > by a collection of "local learning" cells! There is more to the > process of learning than simple local learning by individual > cells.Many learning "decisions/tasks" must precede actual training > by "local learners." A group of independent "local learners" simply > cannot start learning and be able to reproduce the learning > characteristics and processes of an "autonomous system" like the > brain. > > Local learning, however, is still a feasible idea, but only > within a general global learning context. A global learning > mechanism would be the one that "guides" and "exploits" these > local learners. However, it is also possible that the global > mechanism actually does all of the computations (learning) > and "simply sends signals" > to the network cells for appropriate synaptic adjustment. Both of > these possibilities seem logical: (a) a "pure" global mechanism > that learns by itself and then sends signals to the cells to > adjust, or (b) a global/local combination where the global > mechanism performs certain tasks and then uses the local > mechanism for training/learning. > > Note that the global learning mechanism may actually be > implemented with a collection of local learners!! > Notwithstanding the last remark, the above paragraphs perhaps run the risk of positing a little global homunculus that "does all the computations" and simply "sends signals" to the cells. I might be confused by the distinction between local and global learning. All we have to work with are cells that change their properties based on signals impinging upon them, be they chemical or electrical and originating near or far from the synapse, so it seems that a "global" learning mechanism *must* be implemented by local learners. (Again, if by local you specifically mean LTP/LTD or something similar, then I agree - other mechanisms are also at work). > The basic argument being made here is that there are many tasks > in a "learning process" and that a set of "local learners" armed > with their local learning laws is incapable of performing all of > those tasks. So local learning can only exist in the context of > global learning and thus is only "a part" of the total learning > process. > > It will be much easier to develop a consistent learning theory > using the global/local idea. The global/local idea perhaps will > also give us a better handle on the processes that we call > "developmental" and "evolutionary." One last comment. I'm not sure that the "developmental" vs. "learning" distinction is meaningful, either (I'm not hacking on your statements above, Asim; I think this distinction is more or less a tacit assumption in pretty much all neuroscience research). I read these as roughly equivalent to "nature vs. nurture" or "genetics vs. environment". I would claim that to say that any phenomenon is controlled by "genetics" is a scientifically meaningless statement. The claim that such-and-such a phenomenon is genetic is the modern equivalent of saying "The thing is there cause thats how god made it". Genes don't code for behavioral or physical attributes per se, they are simply a string of DNA which code for different proteins. Phenotypes can only arise from the genetic "code" by a complex interaction between cells and signals from their environment. Now these signals can be generated by events outside the organism or within the organism, and I would say that the distinction between development and learning is better thought of as whether the signals for change arise wholly within the organism or if the signals at least in part arise from outside the organism. Any explanation of either learning or development has to be couched in terms of what the relevant signals are and how they affect the system in question. anthony ============================================================ From: Russell Anderson, Ph.D. Smith-Kettlewell Eye Research Institute anderson@skivs.ski.org I read over the replies you received with interest. 1. In regards to Response #1 (j. Faith) I am not sure how relevant canalization is to your essay, but I wrote a paper on the topic a few years back: "Learning and Evolution: A Quantitative Genetics Approach" J. Theor. Biol. 175:89-101 (1995). Incidentally, the phenomenon known as "canalization" was described much earlier by Baldwin, Osborn, and Morgan (in 1896), and is more generally known as the "Baldwin effect" If you're interested, I could mail you a copy. 2. I take issue with the analogies used by Brendan McCane. His analogy of insect colonies is confused or irrelevant: First, the behavior of insects, for the purpose of this argument, does not indicate any individual (local) learning. Hence, the analogy is inappropriate. Second, The "global" learning occuring in the case of insect colonies operates at the level of natural selection acting on the genes, transmitted by the surviving colonies to new founding Queens. In this sense, individual ants are genetically ballistic ("pure developmental"). The genetics of insect colonies are well-studied in evolutionary biology, and he should be referred to any standard text on the topic (Dawkins, Dennett, Wilson, etc.) The analogy using computer science metaphors is likewise flawed or off-the-subject. ============================================================= From: Steven M. Kemp | Department of Psychology | email: steve_kemp@unc.edu Davie Hall, CB# 3270 | University of North Carolina | Chapel Hill, NC 27599-3270 | fax: (919) 962-2537 I do not know if it is quite on point, but Larry Stein at the University of California at Irvine has done fascinating work on a very different type of neural plasiticity called In-Vitro Reinforcement (IVR). I have been working on neural networks whose learning algorithm is based on his data and theory. I don't know whether you would call those networks "local" or "global," but they do have the interesting characteristic that all the units in the network receive the same globally distributed binary reinforcement signal. That is, feedback is not passed along the connections, but distributed simultaneously and equally across the network after the fashion of nondirected dopamine release from the ventral tegmental projections. In any event, I will forward the guts of a recent proposal we have written here to give you a taste of the issues involved. I will be happy to provide more information on this research if you are interested. (Steven Kemp did mail me parts of a recent proposal. It is long, so I did not include it in this posting. Feel free to write to him or me for a copy of it.) ============================================================ From: "K. Char" I have few quick comments: 1. The answer to some parts of the discussions seem to lie in the notion of a *SEQUENCE*. That is: global->local->(final) global; clearly the initial global is not the same as the final global. Some of the discussants seem to prefer the sequence: local->global. A number of such possibilities exists. 2. The next question is: who dictates the sequence? Is it a global mechanism or a local mechanism? 3. In the case of the bee, though it had an individual goal how was this goal arrived at? 4. In the context of neural networks (artificial or real): who dictates the node activation functions, the topology and the learning rules? Does every node find its own activation function? 5. Finally how do we form concepts? Do the concepts evolve as a result of local interactions at the neuron level or through the interaction of micro-concepts at a global level which then trigger a local mechanism? 6. Here the next question could be: how did these micro-concepts evolve in the very first place? 7. Is it possible that these neural structures provide the *very motivation* for the formation of concepts at the global level in order to adapt these structures effectively? If so, does this motivation arise from the environment itself? ============================================================ Response # 1: As you mention, neuroscience tends to equate network plasticity with learning. Connectionists tend to do the same. However this raises a problem with biological systems because this conflates the processes of development and learning. Even the smartest organism starts from an egg, and develops for its entire lifespan - how do we distinguish which changes are learnt, and which are due to development. No one would argue that we *learn* to have a cortex, for instance, even though it is due to massive emryological changes in the central nervous system of the animal. This isn't a problem with artificial nets, because they do not usually have a true developmental process and so there can be no confusion between the two; but it has been a long-standing problem in the ethology literature, where learnt changes are contrasted with "innate" developmental ones. A very interesting recent contribution to this debate is Andre Ariew's "Innateness and Canalization", in Philosophy of Science 63 (Proceedings), in which he identifies non-learnt changes as being due to canalised processes. Canalization was a concept developed by the biologist Waddington in the 40's to describe how many changes seem to have fixed end-goals that are robust against changes in the environment. The relationship between development and learning was also thoroughly explored by Vygotsky (see collected works vol 1, pages 194-210). I'd like to see what other sorts of responses you get, Joe Faith Evolutionary and Adaptive Systems Group, School of Cognitive and Computing Sciences, University of Sussex, UK. ================================================================= Response # 2: I fully agree with you, that local learning is not the one and only ultimate approach - even though it results in very good learning for some domains. I am currently writing a paper on the competitive learning paradigm. I am proposing, that this competition that occurs e.g. within neurons should be called local competition. The network as a whole gives a global common goal to these local competitors and thus their competition must be regarded as cooperation from a more global point of view. There is a nice paper by Kenton Lynne that integrates the ideas of reinforcement and competition. When external evaluations are present, they can serve as teaching values, if nor the neurons compete locally. @InProceedings{Lynne88, author = {K.J.\ Lynne}, title = {Competitive Reinforcement Learning}, booktitle = {Proceedings of the 5th International Conference on Machine Learning}, year = {1988}, publisher = {Morgan Kaufmann}, pages = {188--199} } ---------------------------------------------------------- Christoph Herrmann Visiting researcher Hokkaido University Meme Media Laboratory Kita 13 Nishi 8, Kita- Tel: +81 - 11 - 706 - 7253 Sapporo 060 Fax: +81 - 11 - 706 - 7808 Japan Email: chris@meme.hokudai.ac.jp http://aida.intellektik.informatik.th-darmstadt.de/~chris/ ============================================================= Response #3: I've just read your list of questions on local vs. global learning mechanisms. I think I'm sympathatic to the implications or presuppositions of your questions but need to read them more carefully later. Meanwhile, you might find very interesting a two-part article on such a mechanism by Peter G. Burton in the 1990 volume of _Psychobiology_ 18(2).119-161 & 162-194. Steve Chandler =============================================================== Response #4: A few years back, I wrote a review article on issues of local versus global learning w.r.t. synaptic plasticity. (Unfortunately, it has been "in press" for nearly 4 years). Below is an abstract. I can email the paper to you in TeX or postscript format, or mail you a copy, if you're interested. Russell Anderson ------------------------------------------------ "Biased Random-Walk Learning: A Neurobiological Correlate to Trial-and-Error" (In press: Progress in Neural Networks) Russell W. Anderson Smith-Kettlewell Eye Research Institute 2232 Webster Street San Francisco, CA 94115 Office: (415) 561-1715 FAX: (415) 561-1610 anderson@skivs.ski.org Abstract: Neural network models offer a theoretical testbed for the study of learning at the cellular level. The only experimentally verified learning rule, Hebb's rule, is extremely limited in its ability to train networks to perform complex tasks. An identified cellular mechanism responsible for Hebbian-type long-term potentiation, the NMDA receptor, is highly versatile. Its function and efficacy are modulated by a wide variety of compounds and conditions and are likely to be directed by non-local phenomena. Furthermore, it has been demonstrated that NMDA receptors are not essential for some types of learning. We have shown that another neural network learning rule, the chemotaxis algorithm, is theoretically much more powerful than Hebb's rule and is consistent with experimental data. A biased random-walk in synaptic weight space is a learning rule immanent in nervous activity and may account for some types of learning -- notably the acquisition of skilled movement. ========================================================== Response #5: Asim Roy typed ... > > B) "Pure" local learning does not explain a number of other > activities that are part of the process of learning!! ... > > So, in the whole, there are a "number of activities" that need to > be > performed before any kind of "local learning" can take place. > These aforementioned learning activities "cannot" be performed by > a collection of "local learning" cells! There is more to the > process of learning than simple local learning by individual cells. > Many learning "decisions/tasks" must precede actual training by > "local learners." A group of independent "local learners" simply > cannot start learning and be able to reproduce the learning > characteristics and processes of an "autonomous system" like the > brain. I cannot see how you can prove the above statement (particularly the last sentence). Do you have any proof. By analogy, consider many insect colonies (bees, ants etc). No-one could claim that one of the insects has a global view of what should happen in the colony. Each insect has its own purpose and goes about that purpose without knowing the global purpose of the colony. Yet an ants nest does get built, and the colony does survive. Similarly, it is difficult to claim that evolution has a master plan, order just seems to develop out of chaos. I am not claiming that one type of learning (local or global) is better than another, but I would like to see some evidence for your somewhat outrageous claims. > Note that the global learning mechanism may actually be > implemented with a collection of local learners!! You seem to contradict yourself here. You first say that local learning cannot cope with many problems of learning, yet global learning can. You then say that global learning can be implemented using local learners. This is like saying that you can implement things in C, that cannot be implemented in assembly!! It may be more convenient to implement it in C (or using global learning), but that doesn't make it impossible for assembly. ------------------------------------------------------------------- Brendan McCane, PhD. Email: mccane@cs.otago.ac.nz Comp.Sci. Dept., Otago University, Phone: +64 3 479 8588. Box 56, Dunedin, New Zealand. There's only one catch - Catch 22. =============================================================== Response #6: In regards to arguments against global learning:I think no one seriously questions this possibility, but think that global learning theories are currently non-verifiable/ non-falsifyable. Part of the point of my paper was that there ARE ways to investigate non-local learning, but it requires changes in current experimental protocols. Anyway, good luck. I look forward to seeing your compilation. Russell Anderson 2415 College Ave. #33 Berkeley, CA 94704 ============================================================== Response #7: I am sorry that it has taken so long for me to reply to your inquiry about plasticity and local/global learning. As I mentioned in my first note to you, I am sympathetic to the view that learning involves some sort of overarching, global mechanism even though the actual information storage may consist of distributed patterns of local information. Because I am sympathetic to such a view, it makes it very difficult for me to try to imagine and anticipate the problems for such views. That's why I am glad to see that you are explicitly trying to find people to point out possible problems; we need the reality check. The Peter Burton articles that I have sent you describes exactly the kind of mechanism implied by your first question: Does plasticity imply local learning? Burton describes a neurological mechanism by which local learning could emerge from a global signal. Essentially he posits that whenever the new perceptual input being attended to at any given moment differs sufficiently from the record of previously recorded experiences to which that new input is being compared, the difference triggers a global "proceed-to-store" signal. This signal creates a neural "snapshot" (my term, not Burton's) of the cortical activations at that moment, a global episodic memory (subject to stimulus sampling effects, etc.). Burton goes on to describe how discrete episodic memories could become associated with one another so as to give rise to schematic representations of percepts (personally I don't think that positing this abstraction step is necessary, but Burton does it). As neuroscientists sometimes note, while it is widely assumed that LTP/LTD are local learning mechanisms, the direct evidence for such a hypothesis is pretty slim at best. Of course of of the most serious problems with that view is that the changes don't last very long and thus are not really good candidates for long term (i.e., life long) memory. Now, to my mind, one of the most important possibilities overlooked in LTP studies (inherently so in all in vitro preparations and so far as I know --which is not very far because this is not my field--in the in vivo preparations that I have read about) is that LTP/D is either an artifact of the experiment or some sort of short term change which requires a global signal to become consolidated into a long term record. Burton describes one such possible mechanism. Another motivation for some sort of global mechanism comes from the so-called 'binding problem' addressed especially by the Damasio's, but others too. Somehow somewhere all the distributed pieces of information about what an orange is, for example, have to be tied together. A number of studies of different sorts have demonstarted repeatedly that such information is distributed throughout cortical areas. Burton distinguishes between "perceptual learning" requiring no external teacher (either locally or globally) and "conceptual learning", which may require the assistance of a 'teacher'. In his model though, both types of learning are activated by global "proceed-to-learn" signals triggered in turn by the global summation of local disparities between remembered episodes and current input. I'll just mention in closing that I am particularly interested in the empirical adequacy of neuropsychological accounts such as Burton's because I am very interested in "instance-based" or "exemplar-based" models of learning. In particular, Royal Skousen's _Analogical Modeling of Language_ (Kluwer, 1989) describes an explicit, mathematical model for predicting new behavior on analogy to instances stored in long term memory. Burton's model suggests a possible neurological basis for such behavior. Steve Chandler ============================================================== Response #8: ******************************************************************* Fred Wolf E-Mail: fred@chaos.uni-frankfurt.de Institut fuer Theor. Physik Robert-Mayer-Str. 8 Tel: 069/798-23674 D-60 054 Frankfurt/Main 11 Fax: (49) 69/798-28354 Germany could you please point me to a few neuroBIOLOGICAL references that justify your claim that > > A predominant belief in neuroscience is that synaptic plasticity > and LTP/LTD imply local learning (in your sens). > I think many people appreciate that real learning implies the concerted interplay of a lot of different brain systems and should not even be attempted to be explained by "isolated local learners". See e.g. the series of review-papers on memory in a recent volume of PNAS 93 (1996) (http://www.pnas.org/). Good luck with your general theory of global/local learning. best wishes Fred Wolf ============================================================== Response #9: I am into neurocomputing for several years. I read your arguments with interest. They certainly deserve further attention. Perhaps some combination of global-local learning agents would be the right choice. - Vassilis G. Kaburlasos Aristotle University of Thessaloniki, Greece ============================================================== =============================================================== Original Memo: A predominant belief in neuroscience is that synaptic plasticity and LTP/LTD imply local learning. It is a possibility, but it is not the only possibility. Here are some thoughts on some of the other possibilities (e.g. global learning mechanisms or a combination of global/local mechanisms) and some discussion on the problems associated with "pure" local learning. The local learning idea is a very core idea that drives research in a number of different fields. I welcome comments on the questions and issues raised here. This note is being sent to many listserves. I will collect all of the responses from different sources and redistribute them to all of the participating listserves. The last such discussion was very productive. It has led to the realization by some key researchers in the connectionist area that "memoryless" learning perhaps is not a very "valid" idea. That recognition by itself will lead to more robust and reliable learning algorithms in the future. Perhaps a more active debate on the local learning issue will help us resolve this issue too. A) Does plasticity imply local learning? The physical changes that are observed in synapses/cells in experimental neuroscience when some kind of external stimuli is applied to the cells may not result at all from any specific "learning" at the cells. The cells might simply be responding to a "signal to change" - that is, to change by a specific amount in a specific direction. In animal brains, it is possible that the "actual" learning occurs in some other part(s) of the brain, say perhaps by a global learning mechanism. This global mechanism can then send "change signals" to the various cells it is using to learn a specific task. So it is possible that in these neuroscience experiments, the external stimuli generates signals for change similar to those of a global learning agent in the brain and that the changes are not due to "learning" at the cells themselves. Please note that scientific facts and phenomenon like LTP/LTD or synaptic plasticity can probably be explained equally well by many theories of learning (e.g. local learning vs. global learning, etc.). However, the correctness of an explanation would have to be judged from its consistency with other behavioral and biological facts, not just "one single" biological phenomemon or fact. B) "Pure" local learning does not explain a number of other "activities" that are part of the process of learning!! When learning is to take place by means of "local learning" in a network of cells, the network has to be designed prior to its training. Setting up the net before "local" learning can proceed implies that an external mechanism is involved in this part of the learning process. This "design" part of learning precedes actual training or learning by a collection of "local learners" whose only knowledge about anything is limited to the local learning law to use! In addition, these "local learners" may have to be told what type of local learning law to use, given that a variety of different types can be used under different circumstances. Imagine who is to "instruct and set up" such local learners which type of learning law to use? In addition to these, the "passing" of appropriate information to the appropriate set of cells also has to be "coordinated" by some external or global learning mechanism. This coordination cannot just happen by itself, like magic. It has to be directed from some place by some agent or mechanism. In order to learn properly and quickly, humans generally collect and store relevant information in their brains and then "think" about it (e.g. what problem features are relevant, complexity of the problem, etc.). So prior to any "local learning," there must be processes in the brain that "examine" this "body of information/facts" about a problem in order to design the appropriate network that would fit the problem complexity, select the problem features that are meaningful, etc. It would be very difficult to answer the questions "What size net?" and "What features to use?" without looking at the problem (body of information)in great detail. A bunch of "pure" local learners, armed with their local learning laws, would have no clue to these issues of net design, generalization and feature selection. So, in the whole, there are a "number of activities" that need to be performed before any kind of "local learning" can take place. These aforementioned learning activities "cannot" be performed by a collection of "local learning" cells! There is more to the process of learning than simple local learning by individual cells. Many learning "decisions/tasks" must precede actual training by "local learners." A group of independent "local learners" simply cannot start learning and be able to reproduce the learning characteristics and processes of an "autonomous system" like the brain. Local learning or local computation, however, is still a feasible idea, but only within a general global learning context. A global learning mechanism would be the one that "guides" and "exploits" these local learners or computational elements. However, it is also possible that the global mechanism actually does all of the computations (learning) and "simply sends signals" to the network of cells for appropriate synaptic adjustment. Both of these possibilities seem logical: (a) a "pure" global mechanism that learns by itself and then sends signals to the cells to adjust, or (b) a global/local combination where the global mechanism performs certain tasks and then uses the local mechanism for training/learning. Thus note that the global learning mechanism may actually be implemented with a collection of local learners or computational elements!! However, certain "learning decisions" are made in the global sense and not by "pure" local learners. The basic argument being made here is that there are many tasks in a "learning process" and that a set of "local learners" armed with their local learning laws is incapable of performing all of those tasks. So local learning can only exist in the context of global learning and thus is only "a part" of the total learning process. It will be much easier to develop a consistent learning theory using the global/local idea. The global/local idea perhaps will also give us a better handle on the processes that we call "developmental" and "evolutionary." And it will, perhaps, allow us to better explain many of the puzzles and inconsistencies in our current body of discoveries about the brain. And, not the least, it will help us construct far better algorithms by removing the "unwarranted restrictions" imposed on us by the current ideas. Any comments on these ideas and possibilities are welcome. Asim Roy Arizona State University From maire@fit.qut.edu.au Mon Mar 17 12:35:21 1997 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id MAA27472 for ; Mon, 17 Mar 1997 12:35:14 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.7.6/8.7.3) with SMTP id MAA02582 for ; Mon, 17 Mar 1997 12:35:06 -0600 (CST) Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa07254; 17 Mar 97 12:24:07 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa07251; 17 Mar 97 11:44:50 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa27797; 17 Mar 97 11:44:24 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa03759; 17 Mar 97 0:04:48 EST Received: from [131.181.2.4] by CS.CMU.EDU id aa14852; 17 Mar 97 0:03:55 EST Received: from floyd.fit.qut.edu.au (floyd.fit.qut.edu.au [131.181.2.114]) by sky.fit.qut.edu.au (8.8.5/8.8.5/tony) with SMTP id PAA27630 for ; Mon, 17 Mar 1997 15:02:48 +1000 (EST) Message-Id: <3.0.16.19970317150311.2ca710b0@sky.fit.qut.edu.au> X-Sender: maire@sky.fit.qut.edu.au X-Mailer: Windows Eudora Pro Version 3.0 (16) Date: Mon, 17 Mar 1997 15:03:33 +1000 To: connectionists@cs.cmu.edu From: maire@fit.qut.edu.au Subject: CADE-14 workshop CFP Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" ============================================================= FIRST CALL FOR PAPERS CADE-14 WORKSHOP July 13, 1997, Townsville, Australia -------------------------------------------- CONNECTIONIST SYSTEMS FOR KNOWLEDGE REPRESENTATION AND DEDUCTION -------------------------------------------- Joachim Diederich, Frederic Maire & Ross Hayward Neurocomputing Research Centre Queensland University of Technology Brisbane 4001 Queensland, Australia Phone: +61 7 3864-2143 Fax: +61 7 3864-1801 E-mail: {joachim,maire,hayward}@fit.qut.edu.au GOALS The objective of the workshop is to provide a discussion platform for researchers interested in Artificial Intelli- gence (AI), Neural Networks (NN), Automated Reasoning and Deduction. The workshop should be of considerable interest to computer scientists, mathematicians and engineers as well as to cognitive scientists and people interested in NN applications which try to bridge the gap between symbolic AI systems and connectionist networks. INTRODUCTION Connectionist systems are attractive because they have highly desirable properties such as fine-grain parallelism, fault tolerance and automatic learning. For a long time, they lagged behind symbolic AI systems for knowledge representation and automated reasoning. But over the last ten years, several connectionist knowledge representation systems have been introduced with greater expressive and inferential power than previous systems (e.g. Pinkas 1991, Shastri & Ajjanagadde 1993, Lange & Dyer 1989, Diederich & Kurfess 1994, Derthick, 1988). SIGNIFICANCE The rapid and successful proliferation of applications incorporating Artificial Neural Network methods and systems in fields as diverse as commerce, science, industry and medicine, offers a clear testament to the capability of the NN paradigm. However, NNs are generally weak methods for knowledge representation. In contrast to symbolic systems, neural networks have no explicit, declarative knowledge representation and therefore have considerable difficulties in generating complex or embedded (e.g. recursive) structures. In neural networks, knowledge is encoded in numeric parameters (weights) and generally distributed. For NNs to gain an even wider degree of user acceptance and to enhance their overall utility as learning and generalisation tools, it is highly desirable (if not essential) to overcome their limitations as representational systems. DISCUSSION POINTS FOR WORKSHOP PARTICIPANTS 1. Oscillatory or signature passing models such as SHRUTI (Shastri & Ajjanagadde, 1993) or ROBIN (Lange & Dyer, 1989). 2. Systems based on energy minimisation such as Pinkas (1991a,b) or Derthick (1988). 3. Integrated modular systems that employ multi-layer feedforward networks and simple recurrent networks (e.g. Diederich & Kurfess, 1994). Learning and representation need to interact here and the representational expressiveness needs to be improved. 4. Logical formalism representable in connectionist networks 5. Representing reasoning processes in a connectionist architecture 6. Relevance of the connectionist approach to overcome the main obstacles to the automation of reasoning (clause retention, inadequate focus, redundant information, clause generation, demodulation, metarules etc.) 7. Learning for Connectionist Representation Systems 8. Learning direction strategy to reduce the severity of the obstacle of inadequate focus. SUBMISSION OF WORKSHOP EXTENDED ABSTRACTS/PAPERS Authors are invited to submit 3 copies of either an extended abstract or full paper relating to one of the topic areas listed above. Papers should be written in English in single column format and should be limited to no more than eight, (8) sides of A4 paper including figures and references. We encourage e-mail submissions in Postscript. Please include the following information in an accompanying cover letter: Full title of paper, presenting author's name, address, and telephone and fax numbers, authors e-mail address. Submission Deadline is April 21,1997 with notification to authors by May 5, 1997 and final postscript versions for the proceedings due by June 2, 1997. For further information, inquiries, and paper submissions please contact: Joachim Diederich, Frederic Maire & Ross Hayward Neurocomputing Research Centre Queensland University of Technology Brisbane 4001 Queensland, Australia Phone: +61 7 3864-2143 Fax: +61 7 3864-1801 E-mail: {joachim,maire,hayward}@fit.qut.edu.au More information about the CADE-14 workshop series is available from: WWW: http://www.cs.jcu.edu.au/~cade-14/ Information about Workshop participation fees are available from: WWW: http://www.cs.jcu.edu.au/~cade-14/CADE-14/RegoForm.html From bert@mbfys.kun.nl Mon Mar 17 18:49:27 1997 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id SAA28671 for ; Mon, 17 Mar 1997 18:49:19 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.7.6/8.7.3) with SMTP id SAA10173 for ; Mon, 17 Mar 1997 18:49:17 -0600 (CST) Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa07268; 17 Mar 97 12:34:19 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa07256; 17 Mar 97 11:45:43 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa27812; 17 Mar 97 11:45:20 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa10456; 17 Mar 97 11:34:47 EST Received: from [131.174.173.155] by RI.CMU.EDU id aa19726; 17 Mar 97 11:34:03 EST Received: from bertus by septimius.mbfys.kun.nl via bertus.mbfys.kun.nl [131.174.173.249] with ESMTP id RAA26810 (8.6.10/2.4) for ; Mon, 17 Mar 1997 17:32:44 +0100 From: Bert Kappen Received: by bertus via bert@localhost for Connectionists@cs.cmu.edu id RAA21938 (SMI-8.6/3.1); Mon, 17 Mar 1997 17:34:12 +0100 Date: Mon, 17 Mar 1997 17:34:12 +0100 Message-Id: <199703171634.RAA21938@bertus> To: Connectionists@cs.cmu.edu Subject: paper available Stimulus dependent correlations in stochastic networks (25 pages) Content-Length: 950 FTP-host: ftp.mbfys.kun.nl FTP-file: snn/pub/reports The file Kappen.Featurelinking.ps.Z is now available for copying from the Neuroprose repository: Stimulus dependent correlations in stochastic networks (25 pages) ABSTRACT: It has been observed that cortical neurons display synchronous firing for some stimuli and not for others. The resulting synchronous cell assemblies are thought to form the basis of object perception. In this paper this 'dynamic linking' phenomenon is demonstrated in networks of binary neurons with stochastic dynamics. Analytical treatment within the mean field theory and linear response theory is possible and is compared with simulations. We establish that correlations are a sensitive function of the spatial coherence in the stimulus. We discuss the possibility to use these correlations as a mechanism for scene segmentation. The papar has been accepted for publication in Physical Review E. Bert Kappen From ataxr@IMAP1.ASU.EDU Tue Mar 18 00:13:20 1997 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id AAA05203 for ; Tue, 18 Mar 1997 00:13:06 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.7.6/8.7.3) with SMTP id AAA16448 for ; Tue, 18 Mar 1997 00:13:04 -0600 (CST) Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa08082; 17 Mar 97 22:41:19 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa08080; 17 Mar 97 22:30:44 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa28158; 17 Mar 97 22:30:01 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa12652; 15 Mar 97 2:44:28 EST Received: from post4.inre.asu.edu by EDRC.CMU.EDU id aa28988; 15 Mar 97 2:44:21 EST Received: from smtp1.asu.edu by asu.edu (PMDF V5.0-6 #7723) id <01IGIGLWH75C987UM7@asu.edu> for connectionists@cs.cmu.edu; Sat, 15 Mar 1997 00:44:13 -0700 (MST) Received: from Asim.Roy (ss6-07.inre.asu.edu [129.219.100.104]) by smtp1.asu.edu (8.6.12/8.6.12) with SMTP id AAA08650 for ; Sat, 15 Mar 1997 00:44:05 -0700 Date: Sat, 15 Mar 1997 00:45:25 -0500 (EST) From: Asim Roy Subject: Does plasticity imply local learning? And other questions To: connectionists@cs.cmu.edu Message-id: MIME-version: 1.0 X-Mailer: Simeon for Windows Content-type: TEXT/PLAIN; CHARSET=US-ASCII Content-transfer-encoding: 7BIT Priority: NORMAL X-Authentication: none [ Moderator's note: Asim Roy led a discussion across several newsgroups on the topic of plasticity and local learning. Below is a summary of the responses he received. -- DST ] I am posting the responses I have so far without comment. Some of the responses provide a great deal of insight on this topic. I hope this will generate more interest in the questions raised. The original posting is attached below for reference. Asim Roy Arizona State University ============================================================= >From Anthony Harris Here are some thoughts: One general point is that I'm not entirely sure about what is meant by theglobal/local distinction. Certainly action at a distance can't take place; something physical happens to the cell/connection in question in order for it to change. As I understand it, the prototypical local learning is a Hebbian rule, where all the information specifying plasticity is in the pre and post-synaptic cells (ie "local" to the connection), while a global learning rule is mediated by something distal to the cell in question (i.e. a neuromodulatory signal). But of course the signal must contact the actual cell via diffusion of a chemical substance (e.g. dopamine). So one different distinction might be how specific the signal is; i.e. in a local rule like LTP the information acts only on the single connection, while a modulatory signal could change all the connections in an area by a similar amount. However, the effects of a neuromodulator could in turn be modulated by the current state of the connection - hence a global signal might act very differently at each connection. Which would make the global signal seem local. So I'm not sure the distinction is clearcut. Maybe its better to consider a continuum of physical distance of the signal to change and specificity of the signal at individual connections. A couple of specific comments follow: > A) Does plasticity imply local learning? > > The physical changes that are observed in synapses/cells in > experimental neuroscience when some kind of external stimuli is > applied to the cells may not result at all from any specific > "learning" at the cells.The cells might simply be responding to a > "signal to change" - that is, to change by a specific amount in a > specific direction. In animal brains, it is possible that the > "actual" learning occurs in some other part(s) of the brain, say > perhaps by a global learning mechanism. This global mechanism can > then send "change signals" to the various cells it is using to > learn a specific task. So it is possible that in these > neuroscience experiments, the external stimuli generates signals > for change similar to those of a global learning agent in the brain > and that > the changes are not due to "learning" at the cells > themselves. Please note that scientific facts/phenomenon like LTP/LTD > or synaptic plasticity can probably be explained equally well by > many theories of learning (e.g. local learning vs. global learning, > etc.). However, the correctness of an explanation would have to > be I think it would be difficult to explain the actual phenomenon of LTP/LTD as a response to some signal sent by a different part of the brain, since a good amount of the evidence comes from in vitro work. So clearly the "change signals" can't be coming from some distant part of the brain - unless the slices contain the necessary machinery for generating the change signal. Also, its of course possible that LTP/LTD local learning rules act in concert with global signals (as you mention below); these global signals being sent by nonspecific neuromodulators (an idea brought up plenty of times before). I'm not sure about the differences in the LTP/LTD data collected in vivo versus in vitro; I'm sure there are people out there studying it carefully, and this could provide insight. > > B) "Pure" local learning does not explain a number of other > activities that are part of the process of learning!! > > When learning is to take place by means of "local learning" in a > network of cells, the network has to be designed prior to its > training. Setting up the net before "local" learning can proceed > implies that an external mechanism is involved in this part of > the > learning process. This "design" part of learning precedes actual > training or learning by a collection of "local learners" whose > only > knowledge about anything is limited to the local learning law to > use! Of course, changing connection strengths seems to be the last phase of the "learning/development" process. Correct numbers of cells need to be generated, they have to get to their correct locations, proper connections between subpopulations need to be established and refined, and only at this point is there a substrate for "local" learning. All of these can be affected to a certain extent by environment. For example, the number of cells in the spinal cord innervating a peripheral target can be downregulated with limb bud ablation; conversely, the final number can be upregulated with supernumerary limb grafts. Another well known example is the development of ocular dominance columns. Here, physical connections can be removed (in normal development), or new connections can be established (evidence for this from the reverse suture experiments), depending on the given environment. What would be quite interesting would be if all these developmental phases are guided by similar principles, but acting over different spatial and temporal scales, and mediated by different carriers (e.g. chemical versus electrical signals). Alas, if only I had a well-articulated, cogent principle in hand with which to unify these disparate findings; my first Nobel prize would be forthcoming. In lieu of this, we're stuck with my ramblings. > > In order to learn properly and quickly, humans generally collect > and store relevant information in their brains and then "think" > about it (e.g. what problem features are relevant, problem > complexity, etc.). So prior to any "local learning," there must > be processes in the brain that examine this "body of > information/facts" about a problem in order to design the > appropriate network that would fit the problem complexity, select > the problem features that are meaningful, etc. It would be very > difficult to answer the questions "What size net?" and "What > features to use?" without looking at the problem in great detail. > A bunch of "pure" local learners, armed with their local learning > laws, would have no clue to these issues of net design, > generalization and feature selection. > > So, in the whole, there are a "number of activities" that need to > be performed before any kind of "local learning" can take place. > These aforementioned learning activities "cannot" be performed > by a collection of "local learning" cells! There is more to the > process of learning than simple local learning by individual cells. > Many learning "decisions/tasks" must precede actual training by > "local learners." A group of independent "local learners" simply > cannot start learning and be able to reproduce the learning > characteristics and processes of an "autonomous system" like the > brain. > > Local learning, however, is still a feasible idea, but only > within a general global learning context. A global learning > mechanism would be the one that "guides" and "exploits" these > local learners. However, it is also possible that the global > mechanism actually does all of the computations (learning) > and "simply sends signals" > to the network cells for appropriate synaptic adjustment. Both of > these possibilities seem logical: (a) a "pure" global mechanism > that learns by itself and then sends signals to the cells to > adjust, or (b) a global/local combination where the global > mechanism performs certain tasks and then uses the local mechanism > for training/learning. > > Note that the global learning mechanism may actually be implemented > with a collection of local learners!! > Notwithstanding the last remark, the above paragraphs perhaps run the risk of positing a little global homunculus that "does all the computations" and simply "sends signals" to the cells. I might be confused by the distinction between local and global learning. All we have to work with are cells that change their properties based on signals impinging upon them, be they chemical or electrical and originating near or far from the synapse, so it seems that a "global" learning mechanism *must* be implemented by local learners. (Again, if by local you specifically mean LTP/LTD or something similar, then I agree - other mechanisms are also at work). > The basic argument being made here is that there are many tasks > in a "learning process" and that a set of "local learners" armed > with their local learning laws is incapable of performing all of > those tasks. So local learning can only exist in the context of > global learning and thus is only "a part" of the total learning > process. > > It will be much easier to develop a consistent learning theory > using the global/local idea. The global/local idea perhaps will > also give us a better handle on the processes that we call > "developmental" and "evolutionary." One last comment. I'm not sure that the "developmental" vs. "learning" distinction is meaningful, either (I'm not hacking on your statements above, Asim; I think this distinction is more or less a tacit assumption in pretty much all neuroscience research). I read these as roughly equivalent to "nature vs. nurture" or "genetics vs. environment". I would claim that to say that any phenomenon is controlled by "genetics" is a scientifically meaningless statement. The claim that such-and-such a phenomenon is genetic is the modern equivalent of saying "The thing is there cause thats how god made it". Genes don't code for behavioral or physical attributes per se, they are simply a string of DNA which code for different proteins. Phenotypes can only arise from the genetic "code" by a complex interaction between cells and signals from their environment. Now these signals can be generated by events outside the organism or within the organism, and I would say that the distinction between development and learning is better thought of as whether the signals for change arise wholly within the organism or if the signals at least in part arise from outside the organism. Any explanation of either learning or development has to be couched in terms of what the relevant signals are and how they affect the system in question. anthony ============================================================ From: Russell Anderson, Ph.D. Smith-Kettlewell Eye Research Institute anderson@skivs.ski.org I read over the replies you received with interest. 1. In regards to Response #1 (j. Faith) I am not sure how relevant canalization is to your essay, but I wrote a paper on the topic a few years back: "Learning and Evolution: A Quantitative Genetics Approach" J. Theor. Biol. 175:89-101 (1995). Incidentally, the phenomenon known as "canalization" was described much earlier by Baldwin, Osborn, and Morgan (in 1896), and is more generally known as the "Baldwin effect" If you're interested, I could mail you a copy. 2. I take issue with the analogies used by Brendan McCane. His analogy of insect colonies is confused or irrelevant: First, the behavior of insects, for the purpose of this argument, does not indicate any individual (local) learning. Hence, the analogy is inappropriate. Second, The "global" learning occuring in the case of insect colonies operates at the level of natural selection acting on the genes, transmitted by the surviving colonies to new founding Queens. In this sense, individual ants are genetically ballistic ("pure developmental"). The genetics of insect colonies are well-studied in evolutionary biology, and he should be referred to any standard text on the topic (Dawkins, Dennett, Wilson, etc.) The analogy using computer science metaphors is likewise flawed or off-the-subject. ============================================================= From: Steven M. Kemp | Department of Psychology | email: steve_kemp@unc.edu Davie Hall, CB# 3270 | University of North Carolina | Chapel Hill, NC 27599-3270 | fax: (919) 962-2537 I do not know if it is quite on point, but Larry Stein at the University of California at Irvine has done fascinating work on a very different type of neural plasiticity called In-Vitro Reinforcement (IVR). I have been working on neural networks whose learning algorithm is based on his data and theory. I don't know whether you would call those networks "local" or "global," but they do have the interesting characteristic that all the units in the network receive the same globally distributed binary reinforcement signal. That is, feedback is not passed along the connections, but distributed simultaneously and equally across the network after the fashion of nondirected dopamine release from the ventral tegmental projections. In any event, I will forward the guts of a recent proposal we have written here to give you a taste of the issues involved. I will be happy to provide more information on this research if you are interested. (Steven Kemp did mail me parts of a recent proposal. It is long, so I did not include it in this posting. Feel free to write to him or me for a copy of it.) ============================================================ From: "K. Char" I have few quick comments: 1. The answer to some parts of the discussions seem to lie in the notion of a *SEQUENCE*. That is: global->local->(final) global; clearly the initial global is not the same as the final global. Some of the discussants seem to prefer the sequence: local->global. A number of such possibilities exists. 2. The next question is: who dictates the sequence? Is it a global mechanism or a local mechanism? 3. In the case of the bee, though it had an individual goal how was this goal arrived at? 4. In the context of neural networks (artificial or real): who dictates the node activation functions, the topology and the learning rules? Does every node find its own activation function? 5. Finally how do we form concepts? Do the concepts evolve as a result of local interactions at the neuron level or through the interaction of micro-concepts at a global level which then trigger a local mechanism? 6. Here the next question could be: how did these micro-concepts evolve in the very first place? 7. Is it possible that these neural structures provide the *very motivation* for the formation of concepts at the global level in order to adapt these structures effectively? If so, does this motivation arise from the environment itself? ============================================================ Response # 1: As you mention, neuroscience tends to equate network plasticity with learning. Connectionists tend to do the same. However this raises a problem with biological systems because this conflates the processes of development and learning. Even the smartest organism starts from an egg, and develops for its entire lifespan - how do we distinguish which changes are learnt, and which are due to development. No one would argue that we *learn* to have a cortex, for instance, even though it is due to massive emryological changes in the central nervous system of the animal. This isn't a problem with artificial nets, because they do not usually have a true developmental process and so there can be no confusion between the two; but it has been a long-standing problem in the ethology literature, where learnt changes are contrasted with "innate" developmental ones. A very interesting recent contribution to this debate is Andre Ariew's "Innateness and Canalization", in Philosophy of Science 63 (Proceedings), in which he identifies non-learnt changes as being due to canalised processes. Canalization was a concept developed by the biologist Waddington in the 40's to describe how many changes seem to have fixed end-goals that are robust against changes in the environment. The relationship between development and learning was also thoroughly explored by Vygotsky (see collected works vol 1, pages 194-210). I'd like to see what other sorts of responses you get, Joe Faith Evolutionary and Adaptive Systems Group, School of Cognitive and Computing Sciences, University of Sussex, UK. ================================================================= Response # 2: I fully agree with you, that local learning is not the one and only ultimate approach - even though it results in very good learning for some domains. I am currently writing a paper on the competitive learning paradigm. I am proposing, that this competition that occurs e.g. within neurons should be called local competition. The network as a whole gives a global common goal to these local competitors and thus their competition must be regarded as cooperation from a more global point of view. There is a nice paper by Kenton Lynne that integrates the ideas of reinforcement and competition. When external evaluations are present, they can serve as teaching values, if nor the neurons compete locally. @InProceedings{Lynne88, author = {K.J.\ Lynne}, title = {Competitive Reinforcement Learning}, booktitle = {Proceedings of the 5th International Conference on Machine Learning}, year = {1988}, publisher = {Morgan Kaufmann}, pages = {188--199} } ---------------------------------------------------------- Christoph Herrmann Visiting researcher Hokkaido University Meme Media Laboratory Kita 13 Nishi 8, Kita- Tel: +81 - 11 - 706 - 7253 Sapporo 060 Fax: +81 - 11 - 706 - 7808 Japan Email: chris@meme.hokudai.ac.jp http://aida.intellektik.informatik.th-darmstadt.de/~chris/ ============================================================= Response #3: I've just read your list of questions on local vs. global learning mechanisms. I think I'm sympathatic to the implications or presuppositions of your questions but need to read them more carefully later. Meanwhile, you might find very interesting a two-part article on such a mechanism by Peter G. Burton in the 1990 volume of _Psychobiology_ 18(2).119-161 & 162-194. Steve Chandler =============================================================== Response #4: A few years back, I wrote a review article on issues of local versus global learning w.r.t. synaptic plasticity. (Unfortunately, it has been "in press" for nearly 4 years). Below is an abstract. I can email the paper to you in TeX or postscript format, or mail you a copy, if you're interested. Russell Anderson ------------------------------------------------ "Biased Random-Walk Learning: A Neurobiological Correlate to Trial-and-Error" (In press: Progress in Neural Networks) Russell W. Anderson Smith-Kettlewell Eye Research Institute 2232 Webster Street San Francisco, CA 94115 Office: (415) 561-1715 FAX: (415) 561-1610 anderson@skivs.ski.org Abstract: Neural network models offer a theoretical testbed for the study of learning at the cellular level. The only experimentally verified learning rule, Hebb's rule, is extremely limited in its ability to train networks to perform complex tasks. An identified cellular mechanism responsible for Hebbian-type long-term potentiation, the NMDA receptor, is highly versatile. Its function and efficacy are modulated by a wide variety of compounds and conditions and are likely to be directed by non-local phenomena. Furthermore, it has been demonstrated that NMDA receptors are not essential for some types of learning. We have shown that another neural network learning rule, the chemotaxis algorithm, is theoretically much more powerful than Hebb's rule and is consistent with experimental data. A biased random-walk in synaptic weight space is a learning rule immanent in nervous activity and may account for some types of learning -- notably the acquisition of skilled movement. ========================================================== Response #5: Asim Roy typed ... > > B) "Pure" local learning does not explain a number of other > activities that are part of the process of learning!! ... > > So, in the whole, there are a "number of activities" that need to > be > performed before any kind of "local learning" can take place. > These aforementioned learning activities "cannot" be performed by > a collection of "local learning" cells! There is more to the > process of learning than simple local learning by individual cells. > Many learning "decisions/tasks" must precede actual training by > "local learners." A group of independent "local learners" simply > cannot start learning and be able to reproduce the learning > characteristics and processes of an "autonomous system" like the > brain. I cannot see how you can prove the above statement (particularly the last sentence). Do you have any proof. By analogy, consider many insect colonies (bees, ants etc). No-one could claim that one of the insects has a global view of what should happen in the colony. Each insect has its own purpose and goes about that purpose without knowing the global purpose of the colony. Yet an ants nest does get built, and the colony does survive. Similarly, it is difficult to claim that evolution has a master plan, order just seems to develop out of chaos. I am not claiming that one type of learning (local or global) is better than another, but I would like to see some evidence for your somewhat outrageous claims. > Note that the global learning mechanism may actually be implemented > with a collection of local learners!! You seem to contradict yourself here. You first say that local learning cannot cope with many problems of learning, yet global learning can. You then say that global learning can be implemented using local learners. This is like saying that you can implement things in C, that cannot be implemented in assembly!! It may be more convenient to implement it in C (or using global learning), but that doesn't make it impossible for assembly. ------------------------------------------------------------------- Brendan McCane, PhD. Email: mccane@cs.otago.ac.nz Comp.Sci. Dept., Otago University, Phone: +64 3 479 8588. Box 56, Dunedin, New Zealand. There's only one catch - Catch 22. =============================================================== Response #6: In regards to arguments against global learning:I think no one seriously questions this possibility, but think that global learning theories are currently non-verifiable/ non-falsifyable. Part of the point of my paper was that there ARE ways to investigate non-local learning, but it requires changes in current experimental protocols. Anyway, good luck. I look forward to seeing your compilation. Russell Anderson 2415 College Ave. #33 Berkeley, CA 94704 ============================================================== Response #7: I am sorry that it has taken so long for me to reply to your inquiry about plasticity and local/global learning. As I mentioned in my first note to you, I am sympathetic to the view that learning involves some sort of overarching, global mechanism even though the actual information storage may consist of distributed patterns of local information. Because I am sympathetic to such a view, it makes it very difficult for me to try to imagine and anticipate the problems for such views. That's why I am glad to see that you are explicitly trying to find people to point out possible problems; we need the reality check. The Peter Burton articles that I have sent you describes exactly the kind of mechanism implied by your first question: Does plasticity imply local learning? Burton describes a neurological mechanism by which local learning could emerge from a global signal. Essentially he posits that whenever the new perceptual input being attended to at any given moment differs sufficiently from the record of previously recorded experiences to which that new input is being compared, the difference triggers a global "proceed-to-store" signal. This signal creates a neural "snapshot" (my term, not Burton's) of the cortical activations at that moment, a global episodic memory (subject to stimulus sampling effects, etc.). Burton goes on to describe how discrete episodic memories could become associated with one another so as to give rise to schematic representations of percepts (personally I don't think that positing this abstraction step is necessary, but Burton does it). As neuroscientists sometimes note, while it is widely assumed that LTP/LTD are local learning mechanisms, the direct evidence for such a hypothesis is pretty slim at best. Of course of of the most serious problems with that view is that the changes don't last very long and thus are not really good candidates for long term (i.e., life long) memory. Now, to my mind, one of the most important possibilities overlooked in LTP studies (inherently so in all in vitro preparations and so far as I know --which is not very far because this is not my field--in the in vivo preparations that I have read about) is that LTP/D is either an artifact of the experiment or some sort of short term change which requires a global signal to become consolidated into a long term record. Burton describes one such possible mechanism. Another motivation for some sort of global mechanism comes from the so-called 'binding problem' addressed especially by the Damasio's, but others too. Somehow somewhere all the distributed pieces of information about what an orange is, for example, have to be tied together. A number of studies of different sorts have demonstarted repeatedly that such information is distributed throughout cortical areas. Burton distinguishes between "perceptual learning" requiring no external teacher (either locally or globally) and "conceptual learning", which may require the assistance of a 'teacher'. In his model though, both types of learning are activated by global "proceed-to-learn" signals triggered in turn by the global summation of local disparities between remembered episodes and current input. I'll just mention in closing that I am particularly interested in the empirical adequacy of neuropsychological accounts such as Burton's because I am very interested in "instance-based" or "exemplar-based" models of learning. In particular, Royal Skousen's _Analogical Modeling of Language_ (Kluwer, 1989) describes an explicit, mathematical model for predicting new behavior on analogy to instances stored in long term memory. Burton's model suggests a possible neurological basis for such behavior. Steve Chandler ============================================================== Response #8: ******************************************************************* Fred Wolf E-Mail: fred@chaos.uni-frankfurt.de Institut fuer Theor. Physik Robert-Mayer-Str. 8 Tel: 069/798-23674 D-60 054 Frankfurt/Main 11 Fax: (49) 69/798-28354 Germany could you please point me to a few neuroBIOLOGICAL references that justify your claim that > > A predominant belief in neuroscience is that synaptic plasticity > and LTP/LTD imply local learning (in your sens). > I think many people appreciate that real learning implies the concerted interplay of a lot of different brain systems and should not even be attempted to be explained by "isolated local learners". See e.g. the series of review-papers on memory in a recent volume of PNAS 93 (1996) (http://www.pnas.org/). Good luck with your general theory of global/local learning. best wishes Fred Wolf ============================================================== Response #9: I am into neurocomputing for several years. I read your arguments with interest. They certainly deserve further attention. Perhaps some combination of global-local learning agents would be the right choice. - Vassilis G. Kaburlasos Aristotle University of Thessaloniki, Greece ============================================================== =============================================================== Original Memo: A predominant belief in neuroscience is that synaptic plasticity and LTP/LTD imply local learning. It is a possibility, but it is not the only possibility. Here are some thoughts on some of the other possibilities (e.g. global learning mechanisms or a combination of global/local mechanisms) and some discussion on the problems associated with "pure" local learning. The local learning idea is a very core idea that drives research in a number of different fields. I welcome comments on the questions and issues raised here. This note is being sent to many listserves. I will collect all of the responses from different sources and redistribute them to all of the participating listserves. The last such discussion was very productive. It has led to the realization by some key researchers in the connectionist area that "memoryless" learning perhaps is not a very "valid" idea. That recognition by itself will lead to more robust and reliable learning algorithms in the future. Perhaps a more active debate on the local learning issue will help us resolve this issue too. A) Does plasticity imply local learning? The physical changes that are observed in synapses/cells in experimental neuroscience when some kind of external stimuli is applied to the cells may not result at all from any specific "learning" at the cells. The cells might simply be responding to a "signal to change" - that is, to change by a specific amount in a specific direction. In animal brains, it is possible that the "actual" learning occurs in some other part(s) of the brain, say perhaps by a global learning mechanism. This global mechanism can then send "change signals" to the various cells it is using to learn a specific task. So it is possible that in these neuroscience experiments, the external stimuli generates signals for change similar to those of a global learning agent in the brain and that the changes are not due to "learning" at the cells themselves. Please note that scientific facts and phenomenon like LTP/LTD or synaptic plasticity can probably be explained equally well by many theories of learning (e.g. local learning vs. global learning, etc.). However, the correctness of an explanation would have to be judged from its consistency with other behavioral and biological facts, not just "one single" biological phenomemon or fact. B) "Pure" local learning does not explain a number of other "activities" that are part of the process of learning!! When learning is to take place by means of "local learning" in a network of cells, the network has to be designed prior to its training. Setting up the net before "local" learning can proceed implies that an external mechanism is involved in this part of the learning process. This "design" part of learning precedes actual training or learning by a collection of "local learners" whose only knowledge about anything is limited to the local learning law to use! In addition, these "local learners" may have to be told what type of local learning law to use, given that a variety of different types can be used under different circumstances. Imagine who is to "instruct and set up" such local learners which type of learning law to use? In addition to these, the "passing" of appropriate information to the appropriate set of cells also has to be "coordinated" by some external or global learning mechanism. This coordination cannot just happen by itself, like magic. It has to be directed from some place by some agent or mechanism. In order to learn properly and quickly, humans generally collect and store relevant information in their brains and then "think" about it (e.g. what problem features are relevant, complexity of the problem, etc.). So prior to any "local learning," there must be processes in the brain that "examine" this "body of information/facts" about a problem in order to design the appropriate network that would fit the problem complexity, select the problem features that are meaningful, etc. It would be very difficult to answer the questions "What size net?" and "What features to use?" without looking at the problem (body of information)in great detail. A bunch of "pure" local learners, armed with their local learning laws, would have no clue to these issues of net design, generalization and feature selection. So, in the whole, there are a "number of activities" that need to be performed before any kind of "local learning" can take place. These aforementioned learning activities "cannot" be performed by a collection of "local learning" cells! There is more to the process of learning than simple local learning by individual cells. Many learning "decisions/tasks" must precede actual training by "local learners." A group of independent "local learners" simply cannot start learning and be able to reproduce the learning characteristics and processes of an "autonomous system" like the brain. Local learning or local computation, however, is still a feasible idea, but only within a general global learning context. A global learning mechanism would be the one that "guides" and "exploits" these local learners or computational elements. However, it is also possible that the global mechanism actually does all of the computations (learning) and "simply sends signals" to the network of cells for appropriate synaptic adjustment. Both of these possibilities seem logical: (a) a "pure" global mechanism that learns by itself and then sends signals to the cells to adjust, or (b) a global/local combination where the global mechanism performs certain tasks and then uses the local mechanism for training/learning. Thus note that the global learning mechanism may actually be implemented with a collection of local learners or computational elements!! However, certain "learning decisions" are made in the global sense and not by "pure" local learners. The basic argument being made here is that there are many tasks in a "learning process" and that a set of "local learners" armed with their local learning laws is incapable of performing all of those tasks. So local learning can only exist in the context of global learning and thus is only "a part" of the total learning process. It will be much easier to develop a consistent learning theory using the global/local idea. The global/local idea perhaps will also give us a better handle on the processes that we call "developmental" and "evolutionary." And it will, perhaps, allow us to better explain many of the puzzles and inconsistencies in our current body of discoveries about the brain. And, not the least, it will help us construct far better algorithms by removing the "unwarranted restrictions" imposed on us by the current ideas. Any comments on these ideas and possibilities are welcome. Asim Roy Arizona State University From rao@cs.rochester.edu Tue Mar 18 00:15:48 1997 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id AAA05205 for ; Tue, 18 Mar 1997 00:13:07 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.7.6/8.7.3) with SMTP id AAA16450 for ; Tue, 18 Mar 1997 00:13:06 -0600 (CST) Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa08121; 17 Mar 97 22:58:25 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa08099; 17 Mar 97 22:50:26 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa28180; 17 Mar 97 22:49:02 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa23855; 17 Mar 97 22:35:53 EST Received: from cayuga.cs.rochester.edu by EDRC.CMU.EDU id aa13356; 17 Mar 97 22:35:14 EST Received: from skunk.cs.rochester.edu (skunk.cs.rochester.edu [192.5.53.143]) by cayuga.cs.rochester.edu (8.6.9/H) with ESMTP id WAA26806; Mon, 17 Mar 1997 22:35:09 -0500 Received: (from rao@localhost) by skunk.cs.rochester.edu (8.6.9/K) id WAA22044; Mon, 17 Mar 1997 22:34:36 -0500 Date: Mon, 17 Mar 1997 22:34:36 -0500 Message-Id: <199703180334.WAA22044@skunk.cs.rochester.edu> From: Rajesh Rao To: connectionists@cs.cmu.edu, comp-neuro@smaug.bbb.caltech.edu, submission@vislist.com, cvnet@skivs.ski.org, eyemov-l@spcvxa.spc.edu, cogpsych@ripken.oit.unc.edu, cogneuro@ptolemy-ethernet.arc.nasa.gov, cogpsy@neuro.psy.soton.ac.uk, cogpsy@phil.ruu.nl Subject: Tech Report: Eye movements - a computational study The following report describing a computational model of eye movements in visual cognition is available for retrieval via ftp. Keywords: Saccades, spatiochromatic filters, saliency maps, spatial memory, object-centered maps, reference frames Comments and suggestions welcome (This message has been cross-posted - my apologies to those who received it more than once). -- Rajesh Rao Internet: rao@cs.rochester.edu Dept. of Computer Science VOX: (716) 275-2527 University of Rochester FAX: (716) 461-2018 Rochester NY 14627-0226 WWW: http://www.cs.rochester.edu/u/rao/ =========================================================================== Eye Movements in Visual Cognition: A Computational Study Rajesh P.N. Rao, Gregory J. Zelinsky, Mary M. Hayhoe, and Dana H. Ballard Technical Report 97.1 National Resource Laboratory for the Study of Brain and Behavior University of Rochester March 1997 Abstract Visual cognition depends critically on the moment-to-moment orientation of gaze. Gaze is changed by saccades, rapid eye movements that orient the fovea over targets of interest in a visual scene. Saccades are ballistic; a prespecified target location is computed prior to the movement and visual feedback is precluded. Once a target is fixated, gaze is typically held for about 300 milliseconds, although it can be held for both longer and shorter intervals. Despite these distinctive properties, there has been no specific computational model of the gaze targeting strategy employed by the human visual system during visual cognitive tasks. This paper proposes such a model that uses iconic scene representations derived from oriented spatiochromatic filters at multiple scales. Visual search for a target object proceeds in a coarse-to-fine fashion with the target's largest scale filter responses being compared first. Task-relevant target locations are represented as saliency maps which are used to program eye movements. Once fixated, targets are remembered by using spatial memory in the form of object-centered maps. The model was empirically tested by comparing its performance with actual eye movement data from human subjects in natural visual search tasks. Experimental results indicate excellent agreement between eye movements predicted by the model and those recorded from human subjects. Retrieval information: FTP-host: ftp.cs.rochester.edu FTP-pathname: /pub/u/rao/papers/tr97.1.ps.Z URL: ftp://ftp.cs.rochester.edu/pub/u/rao/papers/tr97.1.ps.Z 35 pages; 1385K compressed, 6667K uncompressed ------------------------------------------------------------------------- Anonymous ftp instructions: >ftp ftp.cs.rochester.edu Connected to anon.cs.rochester.edu. 220 anon.cs.rochester.edu FTP server (Version wu-2.4(3)) ready. Name: [type 'anonymous' here] 331 Guest login ok, send your complete e-mail address as password. Password: [type your e-mail address here] ftp> cd /pub/u/rao/papers/ ftp> get tr97.1.ps.Z ftp> bye >uncompress tr97.1.ps.Z >lpr tr97.1.ps From rojicek@utia.cas.cz Tue Mar 18 23:31:11 1997 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id XAA05363 for ; Tue, 18 Mar 1997 23:31:06 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.7.6/8.7.3) with SMTP id XAA14386 for ; Tue, 18 Mar 1997 23:31:04 -0600 (CST) Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa09004; 18 Mar 97 12:36:03 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa09002; 18 Mar 97 12:10:22 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa28764; 18 Mar 97 12:09:33 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa03570; 18 Mar 97 10:16:40 EST Received: from [147.231.12.1] by RI.CMU.EDU id aa27034; 18 Mar 97 10:15:38 EST Received: from sorcerer (sorcerer.utia.cas.cz [147.231.14.155]) by visla.utia.cas.cz (8.8.5/8.7.1) with ESMTP id QAA01084 for ; Tue, 18 Mar 1997 16:15:30 +0100 (MET) Message-ID: <332EB1F8.3D73@utia.cas.cz> Date: Tue, 18 Mar 1997 16:17:12 +0100 From: Jiri Rojicek Reply-To: rojicek@utia.cas.cz Organization: UTIA X-Mailer: Mozilla 4.0b2 (WinNT; I) MIME-Version: 1.0 To: connectionists@cs.cmu.edu Subject: Curse of dimensionality - new book now available X-Priority: 3 (Normal) Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by visla.utia.cas.cz id QAA01084 I'd like to inform you about a new book dealing with the 'curse of dimensionality' -------------------------------------------------------------------------= -------------------------- Computer Intensive Methods in Control and Signal Processing The Curse of Dimensionality -------------------------------------------------------------------------= -------------------------- K. Warwick, University of Reading, England M. K=E1rny, Institute of Info. Theory, Prague, Czech Republic (Eds.) 0-8176-3989-6 * 1997 * $69.95 * Hardcover * 320 pages * 47 Illustrations For further information and ordering please visit the page: http://www.birkhauser.com/cgi-win/ISBN/0-8176-3989-6/ ********************************************************************** Jiri Rojicek Ji=F8=ED Roj=ED=E8ek (in Win CP 1250) rojicek@utia.cas.cz http://www.utia.cas.cz/AS_dept/rojicek/ tel: +420 - 2 - 66052310 ********************************************************************** From parodi@cptsu2.univ-mrs.fr Wed Mar 19 17:50:28 1997 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id RAA23150 for ; Wed, 19 Mar 1997 17:50:21 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.7.6/8.7.3) with SMTP id RAA04009 for ; Wed, 19 Mar 1997 17:50:19 -0600 (CST) Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab10844; 19 Mar 97 17:27:26 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa10806; 19 Mar 97 16:44:55 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa29729; 19 Mar 97 16:44:50 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa29552; 19 Mar 97 12:57:27 EST Received: from cptsu2.univ-mrs.fr by CS.CMU.EDU id aa13623; 19 Mar 97 12:54:17 EST Received: by cptsu2.univ-mrs.fr (5.61+++/1.34) id AA10605; Wed, 19 Mar 97 17:09:48 +0100 Date: Wed, 19 Mar 97 17:09:48 +0100 From: Olivier Parodi Message-Id: <9703191609.AA10605@cptsu2.univ-mrs.fr> To: Connectionists@cs.cmu.edu Subject: Cargese Summer School, 2d Announcement SUMMER SCHOOL on NEURONAL INFORMATION PROCESSING : FROM BIOLOGICAL DATA TO MODELLING AND APPLICATIONS Cargese -- Corse du Sud (France) June 30 -- July 12, 1997 organized at the INSTITUT D'ETUDES SCIENTIFIQUES DE CARGESE CNRS (UMS 820) Universite de Nice-Sophia Antipolis - Universite de Corte F 20130 CARGESE Sponsored by Centre National de la Recherche Scientifique DGA-DRET (French Ministry of Defence) Conseil Executif de la Corse THEME In the past ten years, statistical mechanics and dynamics of neuronal automata have been extensively studied. Most of the work has been based on over-simplified models of neurons. Recent developments in Neurosciences have, however, considerably modified our knowledge of both the operating modes of neurons and information processing in the cortex. Multi-unit recordings have allowed precise temporal correlations to be detected, within temporal windows of the order of 1 ms. Simultaneously, oscillations corresponding to a quasi-periodic spike-firing, synchronized over several visual cortical areas, have been observed with anaesthesied cats and with monkeys. Last but not least, recent work on the neuronal operating modes have emphasized the role played by the dendritic arborization. These developments have led to considerable interest for coding scheme relying on precise spatio-temporal patterns both from the theoretical and experimental points of view. This prompts us to consider, for information processing, new models which would proceed, e.g., from a synchronous detection of correlated spike firing, and could be particularly robust against noise. Such models might bring about original technical applications for information processing and control. Further developments in this field may be of major importance for our understanding of the basic mechanisms of perception and cognition. They should also lead to new concepts in applications directed towards artificial perception and pattern recognition. Up to now, artificial systems for pattern recognition are far from reaching the standards of human vision. Systems based on a temporal coding by spikes may now be expected to bring about major improvements in this field. The aim of the school is to provide students and people engaged in both applied and basic research with state of the art in every relevant field (Neurosciences, Physics, Mathematics, Information and Control Theory) and to encourage further interdisciplinary and international exchanges. LECTURES Pr. Ad Aertsen (Freiburg am Brisgau) - Dynamic organization of cortical activity. Pr. P. Combe (Marseille) - Learning: a geometrical approach. Pr. J. Demongeot (Grenoble) - Dynamics of modular architectures. Pr. L. van Hemmen (Munchen) - Coding by spikes. Pr. J. Herault (Grenoble) - Information processing in retina. Pr. J. Hertz (Copenhague) - Temporal coding and learning. Pr. A. Holley (Lyon) - Information processing in the mammalian olfactory system. Dr. R. Lestienne (Paris) - Temporal coding with and without clocks. Pr. C. von der Malsburg (Bochum) - The importance of neural synchrony in the visual system. Dr. C. Masson (Paris) - From complex signals to adapted behavior: the example of a small olfactory brain. Dr. J. P. Nadal (Paris) - Information theory and neuronal architecture. Dr. O. Parodi (Marseille) - Temporal coding and correlation detection. Dr. P. Roelfsema (Frankfurt) - Oscillations in the visual cortex of mammalians. Pr. S. A. Solla (ATT) - Dynamics of on-line learning processes. Dr. T. Schanze & Prof. R. Eckhorn (Marburg): Neural mechanisms of visual feature binding and separation investigated with microelectrodes and models. OFFICIAL LANGUAGES The official languages of the School are English and French. Lectures will given in English. DIRECTOR OF THE SCHOOL Olivier PARODI, Centre de Physique Theorique CNRS-Luminy, case 907, F 13288 MARSEILLE CEDEX 09, France Tel. (33) 4 91 26 95 30 Fax (33) 4 91 26 95 53 e-mail parodi@cpt.univ-mrs.fr REGISTRATION FEES Students: free CNRS and members of CNRS institutes: free University: 1500 FF Industry: 2500 FF ACCOMMODATION GRANTS 1 - The School is sponsored by the Formation Permanente du CNRS, which can support accomodation expenses of at least 16 CNRS participants. 2 - The Organizing Committee will consider grants for students and young participants. PRACTICAL INFORMATION The school will be held at the Institut d'Etudes Scientifiques de Cargese. Lectures and Seminars will be given from 9 am to 12:30 pm and from 4 to 7:15 pm.(except on Sunday) from July 1 to 11 included. All participants are expected to arrive on June 30 and leave on July 12. TRAVEL : Cargese is located approximatively 60 kms North of Ajaccio. The best way to get to Cargese is to reach Ajaccio. You are asked to make your own travelling arrangements. However, in order to reduce the cost of your travel, two groups will be organized on regular flights between Paris and Ajaccio and Marseille and Ajaccio on June 30 and July 12 (approx. 1200-1400 FF for a Paris- Ajaccio return ticket). We also consider renting buses for the Ajaccio-Cargese and return journeys. Details will be sent later. ACCOMODATION : The Institute is located 2 km south of the village, on the sea shore. On working days, lunches will be served at the Institute (800 FF for the session, including refreshment and coffe breaks). There are various housing possibilities: - shared room at the Institute or in apartments in the village - 1920 FF (per person for the session) - single room in shared apartments in the village - 2640 FF for the session - rented apartment in the village for your family - 270 to 500 FF per day - hotels in the village - 250 to 400 FF per person per day - camping on the grounds of the Institute - 20 FF per person per day. You have to bring your own equipment, showers are at your disposal. Breakfast will be served at the Institute for people staying on the grounds. WARNING: There are no banks nor cash machines in Cargese and credit cards are not accepted everywhere. Think to bring enough French cash before leaving Paris, Marseille or Ajaccio. NOTE : We cannot provide accomodation before or after the dates of the School. POSTER SESSION : One or several poster sessions will be organized. Participants are encouraged to prepare posters on their own work. REGISTRATION : You have to fill the enclosed application and return it by e-mail to cargese@cpt.univ-mrs.fr BEFORE MARCH 31, 1997 with, if necessary, a letter justifying your grant request. The School will accept up to 50 students, including those supported by the Formation Permanente of CNRS. In case of over-demand, participants will be selected by the Scientific Committee, with a balance between junior and senior students, and a preference for students carrying an active research in the field. ------------------ http://cpt.univ-mrs.fr/Cargese ---------------------- From ollis@nucleus.hut.fi Wed Mar 19 17:50:34 1997 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id RAA23148 for ; Wed, 19 Mar 1997 17:50:20 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.7.6/8.7.3) with SMTP id RAA04006 for ; Wed, 19 Mar 1997 17:50:18 -0600 (CST) Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa10802; 19 Mar 97 17:14:47 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa10800; 19 Mar 97 16:42:22 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa29712; 19 Mar 97 16:41:55 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa25433; 19 Mar 97 8:14:35 EST Received: from santra.hut.fi by CS.CMU.EDU id aa10936; 19 Mar 97 8:13:51 EST Received: from nucleus.hut.fi (nucleus.hut.fi [130.233.173.9]) by santra.hut.fi (8.8.5/8.8.2z) with SMTP id PAA28698; Wed, 19 Mar 1997 15:13:48 +0200 (EET) Received: from itl-pc16.hut.fi by nucleus.hut.fi via SMTP (951211.SGI.8.6.12.PATCH1502/940406.SGI) id PAA21311; Wed, 19 Mar 1997 15:13:47 +0200 Message-ID: <332FE675.47D5@nucleus.hut.fi> Date: Wed, 19 Mar 1997 15:13:25 +0200 From: Olli Simula Organization: Helsinki University of Technology X-Mailer: Mozilla 3.01 (Win95; I) MIME-Version: 1.0 Newsgroups: comp.ai.neural-nets,comp.theory.self-org-sys,sci.stat.math,sci.engr,sci.med.informatics,sci.cognitive,comp.ai.nat-lang CC: olli.simula@hut.fi, connectionists@cs.cmu.edu Subject: ICONIP'97 Special Session on System Monitoring, Modeling, and Analysis Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit ICONIP'97, The Fourth International Conference on Neural Information Processing, November 24-28, 1997. Dunedin/Queenstown, New Zealand Special Session on System Monitoring, Modeling, and Analysis CALL FOR PAPERS Adaptive and intelligent systems based on neural computation and related techniques have successfully been applied in the analysis of various complex processes. This is due to the inherent learning capability of neural networks which is superior in analyzing systems that cannot be modeled analytically. In addition to various fields of engineering, like pattern recognition, industrial process monitoring, and telecommunications, practical applications include information retrieval, data analysis, and financial applications. A special session devoted to these areas of neural computation will be organized at ICONIP'97. The scope of the special session covers neural networks methods and related techniques as well as applications in the following areas: - monitoring, modeling, and analysis, of complex industrial processes - telecommunications applications, including resource management and optimization - data analysis and fusion, including financial applications - time series modeling and forecasting Prospective authors are invited to submit papers to the special session on any area of neural techniques on system monitoring, modeling, and analysis including, but not limited to the topics listed above. The submissions must be received by May 30, 1997. Please, send five copies of your manuscript to Prof. Olli Simula, Special Session Organizer Helsinki University of Technology, Laboratory of Computer and Information Science, Rakentajanaukio 2 C, FIN-02150 Espoo, Finland. More detailed instructions for manuscript submission procedure can be found at WWW, on the special session home page: http://nucleus.hut.fi/ICONIP97/ssmonitor/ For the most up-to-date information about ICONIP'97, please browse the conference home page: http://divcom.otago.ac.nz:800/com/infosci/kel/iconip97.htm Important dates: Papers due: May 30, 1997 Notification of acceptance: July 20, 1997 Final camera-ready papers due: August 20, 1997 ------------------------------------------------------------------------ From Paul.Vitanyi@cwi.nl Wed Mar 19 23:54:57 1997 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id XAA23752 for ; Wed, 19 Mar 1997 23:54:49 -0600 Received: from cs.uwa.oz.au (bilby.cs.uwa.oz.au [130.95.1.11]) by lucy.cs.wisc.edu (8.7.6/8.7.3) with SMTP id XAA09283 for ; Wed, 19 Mar 1997 23:54:45 -0600 (CST) Received: from (parma.cs.uwa.oz.au [130.95.1.7]) by cs.uwa.oz.au (8.6.8/8.5) with SMTP id MAA05100; Thu, 20 Mar 1997 12:25:45 +0800 Message-Id: <199703200425.MAA05100@cs.uwa.oz.au> From: Paul.Vitanyi@cwi.nl To: inductive@unb.ca, ml@ics.uci.edu, reinforce@cs.uwa.edu.au Subject: Book Announcement: 2nd Edition Li-Vitanyi on Kolmogorov Complexity Date: Wed, 19 Mar 1997 15:48:52 +0100 Ming Li and Paul Vitanyi, AN INTRODUCTION TO KOLMOGOROV COMPLEXITY AND ITS APPLICATIONS, REVISED AND EXPANDED SECOND EDITION, Springer-Verlag, New York, 1997, xx+637 pp, 41 illus. Hardcover \$49.95/ISBN 0-387-94868-6 (Graduate Texts in Computer Science Series) After four years and two printings the second edition has now appeared. During the preparation the book has been out of stock for a year. In interaction with many readers and teachers of courses and seminars, all reported errors and problems have been corrected. The book is revised and expanded by about 90 pages. The price has been *lowered* by over $9. See the web page "http://www.cwi.nl/~paulv/kolmogorov.html". >From the ``PREFACE TO THE SECOND EDITION'': When this book was conceived ten years ago, few scientists realized the width of scope and the power for applicability of the central ideas. Partially because of the enthusiastic reception of the first edition, open problems have been solved and new applications have been developed. We have added new material on the relation between data compression and minimum description length induction, computational learning, and universal prediction; circuit theory; distributed algorithmics; instance complexity; CD compression; computational complexity; Kolmogorov random graphs; shortest encoding of routing tables in communication networks; resource-bounded computable universal distributions; average case properties; the equality of statistical entropy and expected Kolmogorov complexity; and so on. Apart from being used by researchers and as reference work, the book is now commonly used for graduate courses and seminars. In recognition of this fact, the second edition has been produced in textbook style. We have preserved as much as possible the ordering of the material as it was in the first edition. The many exercises bunched together at the ends of some chapters have been moved to the appropriate sections. The comprehensive bibliography on Kolmogorov complexity at the end of the book has been updated, as have the ``History and References'' sections of the chapters. Many readers were kind enough to express their appreciation for the first edition and to send notification of typos, errors, and comments. Their number is too large to thank them individually, so we thank them all collectively. BLURB: Written by two experts in the field, this is the only comprehensive and unified treatment of the central ideas and their applications of Kolmogorov complexity---the theory dealing with the quantity of information in individual objects. Kolmogorov complexity is known variously as `algorithmic information', `algorithmic entropy', `Kolmogorov-Chaitin complexity', `descriptional complexity', `shortest program length', `algorithmic randomness', and others. The book is ideal for advanced undergraduate students, graduate students and researchers in computer science, mathematics, cognitive sciences, artificial intelligence, philosophy, statistics and physics. The book is self contained in the sense that it contains the basic requirements of computability theory, probability theory, information theory, and coding. Included are also numerous problem sets, comments, source references and hints to the solutions of problems, course outlines for classroom use, as well as a great deal of new material not included in the first edition. CONTENTS: Preface to the First Edition v How to Use This Book viii Acknowledgments x Preface to the Second Edition xii Outlines of One-Semester Courses xii List of Figures xix 1 Preliminaries 1 1.1 A Brief Introduction 1 1.2 Prerequisites and Notation 6 1.3 Numbers and Combinatorics 8 1.4 Binary Strings 12 1.5 Asymptotic Notation 15 1.6 Basics of Probability Theory 18 1.7 Basics of Computability Theory 24 1.8 The Roots of Kolmogorov Complexity 47 1.9 Randomness 49 1.10 Prediction and Probability 59 1.11 Information Theory and Coding 65 1.12 State Symbol Complexity 84 1.13 History and References 86 2 Algorithmic Complexity 93 2.1 The Invariance Theorem 96 2.2 Incompressibility 108 2.3 C as an Integer Function 119 2.4 Random Finite Sequences 127 2.5 *Random Infinite Sequences 136 2.6 Statistical Properties of Finite Sequences 158 2.7 Algorithmic Properties of 167 2.8 Algorithmic Information Theory 179 2.9 History and References 185 3 Algorithmic Prefix Complexity 189 3.1 The Invariance Theorem 192 3.2 *Sizes of the Constants 197 3.3 Incompressibility 202 3.4 K as an Integer Function 206 3.5 Random Finite Sequences 208 3.6 *Random Infinite Sequences 211 3.7 Algorithmic Properties of 224 3.8 *Complexity of Complexity 226 3.9 *Symmetry of Algorithmic Information 229 3.10 History and References 237 4 Algorithmic Probability 239 4.1 Enumerable Functions Revisited 240 4.2 Nonclassical Notation of Measures 242 4.3 Discrete Sample Space 245 4.4 Universal Average-Case Complexity 268 4.5 Continuous Sample Space 272 4.6 Universal Average-Case Complexity, Continued 307 4.7 History and References 307 5 Inductive Reasoning 315 5.1 Introduction 315 5.2 Solomonoff's Theory of Prediction 324 5.3 Universal Recursion Induction 335 5.4 Simple Pac-Learning 339 5.5 Hypothesis Identification by Minimum Description Length 351 5.6 History and References 372 6 The Incompressibility Method 379 6.1 Three Examples 380 6.2 High- Probability Properties 385 6.3 Combinatorics 389 6.4 Kolmogorov Random Graphs 396 6.5 Compact Routing 404 6.6 Average-Case Complexity of Heapsort 412 6.7 Longest Common Subsequence 417 6.8 Formal Language Theory 420 6.9 Online CFL Recognition 427 6.10 Turing Machine Time Complexity 432 6.11 Parallel Computation 445 6.12 Switching Lemma 449 6.13 History and References 452 7 Resource-Bounded Complexity 459 7.1 Mathematical Theory 460 7.2 Language Compression 476 7.3 Computational Complexity 488 7.4 Instance Complexity 495 7.5 Kt Complexity and Universal Optimal Search 502 7.6 Time-Limited Universal Distributions 506 7.7 Logical Depth 510 7.8 History and References 516 8 Physics, Information, and Computation 521 8.1 Algorithmic Complexity and Shannon's Entropy 522 8.2 Reversible Computation 528 8.3 Information Distance 537 8.4 Thermodynamics 554 8.5 Entropy Revisited 565 8.6 Compression in Nature 583 8.7 History and References 586 References 591 Index 618 If you are seriously interested in using the text in the course, contact Springer-Verlag's Editor for Computer Science, Martin Gilchrist, for a complimentary copy. Martin Gilchrist marting@springer-sc.com Suite 200, 3600 Pruneridge Ave. (408) 249-9314 Santa Clara, CA 95051 If you are interested in the text but won't be teaching a course, we understand that Springer-Verlag sells the book, too. To order, call toll-free 1-800-SPRINGER (1-800-777-4643); N.J. residents call 201-348-4033. For information regarding examination copies for course adoptions, write Springer-Verlag New York, Inc. , 175 Fifth Avenue, New York,NY 10010. You can order through the Web site: "http://www.springer-ny.com/" For U.S.A./Canada/Mexico- e-mail: orders@springer-ny.com or fax an order form to: 201-348-4505. For orders outside U.S.A./Canada/Mexico send this form to: orders@springer.de Or call toll free: 800-SPRINGER - 8:30 am to 5:30 pm ET (that's 777-4643 and 201-348-4033 in NJ). Write to Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY, 10010. Visit your local scientific bookstore. Mail payments may be made by check, purchase order, or credit card (see note below). Prices are payable in U.S. currency or its equivalent and are subject to change without notice. Remember, your 30-day return privilege is always guaranteed! Your complete address is necessary to fulfill your order. From jordan@psyche.mit.edu Thu Mar 20 21:21:41 1997 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id VAA19484 for ; Thu, 20 Mar 1997 21:21:31 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.7.6/8.7.3) with SMTP id VAA03327 for ; Thu, 20 Mar 1997 21:21:29 -0600 (CST) Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa12518; 20 Mar 97 16:43:45 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa12512; 20 Mar 97 16:21:28 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa00860; 20 Mar 97 16:20:54 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id ab29772; 20 Mar 97 15:58:33 EST Received: from PSYCHE.MIT.EDU by EDRC.CMU.EDU id aa18176; 20 Mar 97 15:58:15 EST Received: by psyche.mit.edu (4.1/DA1.0.4) id AA02744; Thu, 20 Mar 97 15:58:04 EST From: Michael Jordan Message-Id: <9703202058.AA02744@psyche.mit.edu> Subject: Call for Workshop Proposals---NIPS*97 To: connectionists@cs.cmu.edu Date: Thu, 20 Mar 97 15:58:03 EST Cc: Michael Jordan X-Mailer: ELM [version 2.3 PL0] CALL FOR PROPOSALS NIPS*97 Post Conference Workshops December 5 and 6, 1997 Breckenridge, Colorado Following the regular program of the Neural Information Processing Systems 1997 conference, workshops on current topics in neural information processing will be held on December 5 and 6, 1997, in Breckenridge, Colorado. Proposals by qualified individuals interested in chairing one of these workshops are solicited. Past topics have included: Active Learning, Architectural Issues, Attention, Audition, Bayesian Analysis, Bayesian Networks, Benchmarking, Computational Complexity, Computational Molecular Biology, Control, Neuroscience, Genetic Algorithms, Grammars, Hybrid HMM/ANN Systems, Implementations, Music, Neural Hardware, Network Dynamics, Neurophysiology, On-Line Learning, Optimization, Recurrent Nets, Robot Learning, Rule Extraction, Self-Organization, Sensory Biophysics, Signal Processing, Symbolic Dynamics, Speech, Time Series, Topological Maps, and Vision. The goal of the workshops is to provide an informal forum for researchers to discuss important issues of current interest. There will be two workshop sessions a day, for a total of six hours, with free time in between for ongoing individual exchange or outdoor activities. Concrete open and/or controversial issues are encouraged and preferred as workshop topics. Representation of alternative viewpoints and panel-style discussions are particularly encouraged. Workshop organizers will have responsibilities including: 1) coordinating workshop participation and content, which involves arranging short informal presentations by experts working in an area, arranging for expert commentators to sit on a discussion panel and formulating a set of discussion topics, etc. 2) moderating or leading the discussion and reporting its high points, findings, and conclusions to the group during evening plenary sessions 3) writing a brief summary and/or coordinating submitted material for post-conference electronic dissemination. Submission Instructions ----------------------- Interested parties should submit via e-mail a short proposal for a workshop of interest by May 20, 1997. Proposals should include a title, a description of what the workshop is to address and accomplish, the proposed length of the workshop (one day or two days), the planned format (mini-conference, panel discussion, or group discussion, combinations of the above, etc), and the proposed number of speakers. Where possible, please also indicate potential invitees (particularly for panel discussions). Please note that this year we are looking for fewer "mini-conference" workshops and greater variety of workshop formats. The time allotted to workshops is six hours each day, in two sessions of three hours each. We strongly encourage that the organizers reserve a significant portion of time for open discussion. The proposal should motivate why the topic is of interest or controversial, why it should be discussed and who the targeted group of participants is. In addition, please send a brief resume of the prospective workshop chair, a list of publications, and evidence of scholarship in the field of interest. Submissions should include contact name, address, e-mail address, phone number and fax number if available. Proposals should be mailed electronically to mpp@watson.ibm.com. All proposals must be RECEIVED by May 20, 1997. If e-mail is unavailable, mail so as to arrive by the deadline to: NIPS*97 Workshops c/o Steven J. Nowlan Motorola, Lexicus Division 3145 Porter Drive Palo Alto, CA 94304 Questions may be addressed to either of the Workshop Co-Chairs: Steven J. Nowlan Richard Zemel Motorola, Lexicus Division University of Arizona steven@lexicus.mot.com zemel@aruba.ccit.arizona.edu PROPOSALS MUST BE RECEIVED BY MAY 20, 1997 -Please Post- From terry@salk.edu Thu Mar 20 21:21:42 1997 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id VAA19479 for ; Thu, 20 Mar 1997 21:21:29 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.7.6/8.7.3) with SMTP id VAA03323 for ; Thu, 20 Mar 1997 21:21:26 -0600 (CST) Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa11550; 20 Mar 97 2:46:36 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa11548; 20 Mar 97 2:36:36 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa00212; 20 Mar 97 2:36:32 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa14451; 20 Mar 97 1:59:31 EST Received: from helmholtz.salk.edu by EDRC.CMU.EDU id aa13923; 20 Mar 97 1:58:58 EST Received: (from terry@localhost) by helmholtz.salk.edu (8.7.5/8.7.3) id WAA27040; Wed, 19 Mar 1997 22:58:53 -0800 (PST) Date: Wed, 19 Mar 1997 22:58:53 -0800 (PST) From: Terry Sejnowski Message-Id: <199703200658.WAA27040@helmholtz.salk.edu> To: connectionists@cs.cmu.edu Subject: NEURAL COMPUTATION 9:3 Cc: terry@helmholtz.salk.edu Neural Computation - Contents Volume 9, Number 3 - April 1, 1997 Article A Neuro-Mimetic Dynamic Scheduling Algorithm for Control: Analysis and Applications Harpreet S. Kwatra, Francis J. Doyle III, Ilya A. Rybak, and James S. Schwaber Notes Conductance-Based Integrate and Fire Models Alain Destexhe A Simple Model of Transmitter Release and Facilitation Richard Bertram Functional Periodic Intracortical Couplings Induced by Structured Lateral Inhibition in a Linear Cortical Network S. P. Sabatini, G. M. Bisio and L. Raffo Letters Possible Roles of Spontaneous Waves and Dendritic Growth for Retinal Receptive Field Development Pierre-Yves Burgi and Norberto M. Grzywacz Singularities in Primate Orientation Maps K. Obermayer and G. G. Blasdel Topographic Receptive Fields and Patterned Lateral Interaction in a Self-Organizing Model of the Primary Visual Cortex Joseph Sirosh and Risto Miikkulainen The Formation of Topographic Maps that Maximize the Average Mutual Information of the Output Responses to Noiseless Input Signals Marc M. Van Hulle Self-Organization of Firing Activities in Monkey's Motor Cortex: Trajectory Computation from Spike Signals Siming Lin, Jennie Si, and A. B. Schwartz Hyperparameter Selection for Self-Organizing Maps Akio Utsugi Supervised Networks Which Self-Organize Class Outputs Ramesh R. Sarukkai How Well Can We Estimate the Information Carried in Neuronal Responses from Limited Samples? David Golomb, John Hertz, Stefano Panzeri, Alessandro Treves and Barry Richmond Covariance Learning of Correlated Patterns in Competitive Networks Ali A. Minai A Mobile Robot that Learns its Place Sageev Oore, Geoffrey E. Hinton, and Gregory Dudek ----- ABSTRACTS - http://www-mitpress.mit.edu/jrnls-catalog/neural.html SUBSCRIPTIONS - 1997 - VOLUME 9 - 8 ISSUES ______ $50 Student and Retired ______ $78 Individual ______ $250 Institution Add $28 for postage and handling outside USA (+7% GST for Canada). (Back issues from Volumes 1-8 are regularly available for $28 each to institutions and $14 each for individuals Add $5 for postage per issue outside USA (+7% GST for Canada) mitpress-orders@mit.edu MIT Press Journals, 55 Hayward Street, Cambridge, MA 02142. Tel: (617) 253-2889 FAX: (617) 258-6779 -----