From harry@brain.Jpl.Nasa.Gov Mon Oct 2 15:18:45 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 2 Oct 95 15:18:43 -0500; AA06194 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 2 Oct 95 15:18:42 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa13706; 2 Oct 95 15:10:51 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa13703; 2 Oct 95 14:45:04 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa18151; 2 Oct 95 14:44:04 EDT Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa05034; 2 Oct 95 14:07:14 EDT Received: from brain.jpl.nasa.gov by EDRC.CMU.EDU id aa18370; 2 Oct 95 14:06:15 EDT Received: (from harry@localhost) by brain.jpl.nasa.gov (8.6.10/8.6.6) id LAA22519 for Connectionists@CS.CMU.EDU; Mon, 2 Oct 1995 11:05:52 -0700 Date: Mon, 2 Oct 1995 11:05:52 -0700 From: Harry Langenbacher Message-Id: <199510021805.LAA22519@brain.jpl.nasa.gov> To: Connectionists@cs.cmu.edu Subject: Job opening at JPL JOB OPPORTUNITIES NEURAL NETWORK / FUZZY LOGIC SYSTEM DESIGN involving VLSI HARDWARE, and SOFTWARE DEVELOPMENT at the Jet Propulsion Laboratory, Pasadena CA, USA Requires: Ph D (EE or Computer Science) and 2 or more years experience for a position as Member of Technical Staff. PhD (EE or Computer Science) for post-doctoral position. Areas of specialization, expertise, knowledge, and interests: All aspects of analog and digital VLSI and system design. Knowledgeable in new computing paradigms such as neural networks, fuzzy logic, genetic algorithms, etc. Skilled with computer hardware and software development. Interest in innovation. Skills Sought: Experience with VLSI circuit design & layout tools such as Spice and Magic. Expertise in C, Unix, X, MSDOS, MS-Windows, etc. Knowledge of computer interface techniques and hardware. Familiarity with computer graphics, and some knowledge of conventional signal/image processing techniques and algorithms. The Job: Development and integration of state of the art full-custom/ASIC VLSI concurrent processing architectures for real-time sensor signal processing, pattern-recognition, data fusion, etc. Contact: e-mail (preferred), or FAX, mail, or phone to harry@jpl.nasa.gov Harry Langenbacher JPL Mail-stop 302-231 4800 Oak Grove Dr Pasadena CA 91109 USA Jet Propulsion Laboratory Concurrent Processing Devices Group, Phone: 818-354-9513 FAX : 818-393-4540 From ken@phy.ucsf.edu Mon Oct 2 22:44:08 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 2 Oct 95 22:44:05 -0500; AA10758 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 2 Oct 95 22:44:03 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa14064; 2 Oct 95 21:44:48 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa14062; 2 Oct 95 21:36:40 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa18465; 2 Oct 95 21:36:17 EDT Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa11433; 2 Oct 95 20:46:33 EDT Received: from coltrane.ucsf.EDU by RI.CMU.EDU id aa18702; 2 Oct 95 20:45:38 EDT Received: by coltrane.ucsf.edu (5.65/GSC4.24) id AA04586 for Connectionists@CS.CMU.EDU; Mon, 2 Oct 1995 17:45:24 -0700 Date: Mon, 2 Oct 1995 17:45:24 -0700 From: Ken Miller Message-Id: <9510030045.AA04586@coltrane.ucsf.edu> To: Connectionists@cs.cmu.edu, cneuro@smaug.bbb.caltech.edu Subject: paper available: modeling joint development of ocular dominance and orientation maps Cc: Edgar Erwin FTP-host: phy.ucsf.edu FTP-filename: /pub/erwin/CNS95proc.ps.Z URL: ftp://phy.ucsf.edu/pub/erwin/CNS95proc.ps.Z The following paper is now available by anonymous ftp, from the above addresses, or from my or Ed Erwin's home pages (addresses below). Sorry, hard copies are not available. Modeling Joint Development of Ocular Dominance and Orientation Maps in Primary Visual Cortex by Ed Erwin and Kenneth D. Miller To appear in the Proceedings of the Computation and Neural Systems (CNS) 1995 conference, Monterey. (In press) ABSTRACT: We have combined earlier correlation-based models of striate ocular dominance and orientation preference map formation into a joint model. Cortical feature preferences are defined through patterns of synaptic connectivity to LGN cells which develop due to firing correlations of those LGN cells. Model parameters include spatial correlation patterns between ON- and OFF-center cells in separate eye layers of the LGN. A linear transformation yields correlation functions which predict whether orientation preferences, ocular dominance, or both, will develop. The model thus predicts the correlations between LGN cells which would be necessary to explain formation of visual maps by a linear process. Kenneth D. Miller www: http://keck.ucsf.edu/~ken internet: ken@phy.ucsf.edu Ed Erwin www: http://keck.ucsf.edu/~erwin internet: erwin@phy.ucsf.edu Both: Dept. of Physiology University of California, San Francisco 513 Parnassus San Francisco, CA 94143-0444 fax: (415) 476-4929 From jordan@psyche.mit.edu Tue Oct 3 23:03:46 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 3 Oct 95 23:03:44 -0500; AA29447 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 3 Oct 95 23:03:42 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa15737; 3 Oct 95 22:50:01 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa15725; 3 Oct 95 22:26:23 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa19856; 3 Oct 95 22:25:43 EDT Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa28409; 3 Oct 95 18:22:43 EDT Received: from PSYCHE.MIT.EDU by CS.CMU.EDU id aa20317; 3 Oct 95 18:22:08 EDT Received: by psyche.mit.edu (4.1/DA1.0.4) id AA06579; Tue, 3 Oct 95 18:22:03 EDT Date: Tue, 3 Oct 95 18:22:02 EDT From: Michael Jordan To: connectionists@cs.cmu.edu Subject: workshop announcement Message-Id: This is an announcement of a post-NIPS workshop on Learning in Bayesian Belief Networks and Other Graphical Models. Bayesian belief networks are probabilistic graphs that have interesting relationships to neural networks. Undirected belief networks are closely related to Boltzmann machines. Directed belief networks (the more popular variety) are related to feedforward neural networks, but have a stronger probabilistic semantics. Many interesting probabilistic models, including HMM's, Kalman filters, mixture models, factor analytic models, etc., can be viewed as special cases of belief networks. In the area of inference (i.e., the calculation of posterior probabilities of certain nodes given that other nodes are clamped), the research on belief networks is quite mature. The inference algorithms provide a clean probabilistic framework for e.g., inverting a network, calculating posterior probabilities of hidden nodes, calculating most probable configurations, etc. In the area of learning, there have been interesting developments in the area of structural learning (deciding which links and which nodes to include in the graph) and learning in the presence of hidden variables. The organizing committee for the workshop includes: Wray Buntine, Greg Cooper, Dan Geiger, David Heckerman, Geoffrey Hinton, Mike Jordan, Steffen Lauritzen, David Mackay, David Madigan, Radford Neal, Steve Omohundro, Judea Pearl, Stuart Russell, Peter Spirtes, and Ross Shachter. Many of these people will be giving presentations at the workshop. A short bibliography follows for those who might like to read up on Bayesian belief networks in anticipation of the workshop. Mike Jordan ------------------ Short Bibliography ------------------ The list below provides a few useful references, with an emphasis on recent review papers, tutorials, and textbooks. The list is not meant to be comprehensive along any dimension... Many additional pointers to the literature can be found on the Uncertainty in Artificial Intelligence homepage; see http://www.auai.org. If I had to pick two papers that I would most recommend for someone wanting to get up to speed quickly on belief networks, I'd recommend the Spiegelhalter, et al. paper and the Heckerman tutorial. Mike ----------------------- A good place to start to learn about the most popular algorithm for general inference in belief networks, as well as some of the basics on learning: Spiegelhalter, D. J., Dawid, A. P., Lauritzen, S. L., & Cowell, R. G. (1993). Bayesian Analysis in Expert Systems, {\em Statistical Science, 8}, 219-283. If you want more details on the inference algorithm: Lauritzen, S. L., \& Spiegelhalter, D. J. (1988). Local computations with probabilities on graphical structures and their application to expert systems (with discussion). {\em Journal of the Royal Statistical Society B, 50}, 157-224. A tutorial on the recent work on learning in belief networks: Heckerman, D. (1995). A tutorial on learning Bayesian networks. [available through http://www.auai.org]. If you want more on learning: Buntine, W. (1994). Operations for learning with graphical models. {\em Journal of Artificial Intelligence Research, 2}, 159-225. [available through http://www.auai.org]. A very readable general textbook on belief networks from a statistical perspective (focusing on ML estimation and model selection): Whittaker, J. (1990). {\em Graphical Models in Applied Multivariate Statistics}. New York: John Wiley. An introductory textbook: Neapolitan, E. (1990). {\em Probabilistic Reasoning in Expert Systems}. New York: John Wiley. The classical text on belief networks; emphasizes inference and AI issues: Pearl, J. (1988). {\em Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference}. San Mateo, CA: Morgan Kaufman. A recent paper that unifies (almost) all of the extant algorithms for inference in belief networks: Shachter, R. D., Anderson, S. K., \& Szolovits, P. (1994). Global conditioning for probabilistic inference in belief networks. {\em Proceedings of the Uncertainty in Artificial Intelligence Conference}, 514-522. From watrous@scr.siemens.com Wed Oct 4 00:00:51 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 4 Oct 95 00:00:49 -0500; AA29802 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 4 Oct 95 00:00:46 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa15723; 3 Oct 95 22:42:53 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa15721; 3 Oct 95 22:25:33 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa19850; 3 Oct 95 22:25:26 EDT Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa24922; 3 Oct 95 14:13:10 EDT Received: from scr.siemens.com by EDRC.CMU.EDU id aa23739; 3 Oct 95 14:12:49 EDT Received: from tiercel.scr.siemens.com (tiercel.scr.siemens.com [129.73.7.90]) by scr.siemens.com (8.6.10/8.6.6) with ESMTP id OAA25061 for ; Tue, 3 Oct 1995 14:12:47 -0400 Received: (from watrous@localhost) by tiercel.scr.siemens.com (8.6.10/8.6.6) id OAA04375; Tue, 3 Oct 1995 14:11:45 -0400 Date: Tue, 3 Oct 1995 14:11:45 -0400 From: Raymond L Watrous Message-Id: <199510031811.OAA04375@tiercel.scr.siemens.com> To: Connectionists@cs.cmu.edu Subject: paper available on patient-adaptive ECG classification Cc: watrous@learning.scr.siemens.com FTP-HOST: scr.siemens.com FTP-filename: /pub/learning/Papers/watrous/cic_95.ps.Z The following paper (4 pages, 3 figures) is now available via anonymous ftp: A Patient-Adaptive Neural Network ECG Patient Monitoring Algorithm Raymond Watrous, Geoffrey Towell Siemens Corporate Research 755 College Road East Princeton, NJ 08540 Abstract A new, patient-adaptive ECG Patient Monitoring algorithm is described. The algorithm combines a patient-independent neural network classifier with a three-parameter patient model. The patient model is used to modulate the patient-independent classifier via multiplicative connections. Adaptation is carried out by gradient descent in the patient model parameter space. The patient-adaptive classifier was compared with a well-established baseline algorithm on six major databases, consisting of over 3 million heartbeats. When trained on an initial 77 records and tested on an additional 382 records, the patient-adaptive algorithm was found to reduce the number of Vn errors on one channel by a factor of 5, and the number of Nv errors by a factor of 10. We conclude that patient adaptation provides a significant advance in classifying normal vs. ventricular beats for ECG Patient Monitoring. +=+=+= The paper will appear in the proceedings of Computers in Cardiology, September 10-13, 1995, Vienna, Austria. We regret that we are unable to provide hard copies. Raymond Watrous +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ Learning Systems Department Phone: (609) 734-6596 Siemens Corporate Research FAX: (609) 734-6565 755 College Road East Princeton, NJ 08540 watrous@learning.scr.siemens.com From cas-cns@PARK.BU.EDU Thu Oct 5 15:17:38 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 5 Oct 95 15:17:32 -0500; AA03067 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 5 Oct 95 15:17:28 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa18039; 5 Oct 95 14:05:02 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa18027; 5 Oct 95 13:30:38 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01156; 5 Oct 95 13:29:35 EDT Received: from MAILBOX.SRV.CS.CMU.EDU by B.GP.CS.CMU.EDU id aa27668; 5 Oct 95 13:23:53 EDT Received: from PARK.BU.EDU by MAILBOX.SRV.CS.CMU.EDU id aa24371; 5 Oct 95 13:23:26 EDT Received: from mac-duffee.bu.edu by cns.bu.edu (8.6.10/BU_Server-1.1) id NAA29047; Thu, 5 Oct 1995 13:01:26 -0400 Message-Id: <199510051701.NAA29047@cns.bu.edu> Date: Thu, 05 Oct 1995 13:01:56 -0400 From: BU CNS To: cas-cns@PARK.BU.EDU Subject: Boston University - Cognitive & Neural Systems Organization: Department of Cognitive & Neural Systems Newsgroups: comp.ai,comp.cog-eng,comp.software-eng,comp.ai.neural-nets,bu.general,bu.seminars,ne.seminars,news.announce.conferences Reply-To: cas-cns@PARK.BU.EDU X-Newsreader: Yet Another NewsWatcher 2.0.2 (A copy of this message has also been posted to the following newsgroups: comp.ai, comp.cog-eng,comp.software-eng,comp.ai.neural-nets,bu.general,bu.seminars,ne.seminars,news.announce.conferences) ************************************************************** DEPARTMENT OF COGNITIVE AND NEURAL SYSTEMS (CNS) AT BOSTON UNIVERSITY ************************************************************** Ennio Mingolla, Acting Chairman, 1995-96 Stephen Grossberg, Chairman Gail A. Carpenter, Director of Graduate Studies The Boston University Department of Cognitive and Neural Systems offers comprehensive graduate training in the neural and computational principles, mechanisms, and architectures that underlie human and animal behavior, and the application of neural network architectures to the solution of technological problems. Applications for Fall, 1996, admission and financial aid are now being accepted for both the MA and PhD degree programs. To obtain a brochure describing the CNS Program and a set of application materials, write, telephone, or fax: DEPARTMENT OF COGNITIVE & NEURAL SYSTEMS Boston University 111 Cummington Street, 2nd Floor (ON OR AFTER 10/30/95, PLEASE ADDRESS Boston, MA 02215 MAIL TO 677 BEACON STREET) 617/353-9481 (phone) 617/353-7755 (fax) or send via email your full name and mailing address to: rll@cns.bu.edu Applications for admission and financial aid should be received by the Graduate School Admissions Office no later than January 15. Late applications will be considered until May 1; after that date applications will be considered only as special cases. Applicants are required to submit undergraduate (and, if applicable, graduate) transcripts, three letters of recommendation, and Graduate Record Examination (GRE) scores. The Advanced Test should be in the candidate's area of departmental specialization. GRE scores may be waived for MA candidates and, in exceptional cases, for PhD candidates, but absence of these scores may decrease an applicant's chances for admission and financial aid. Non-degree students may also enroll in CNS courses on a part-time basis. Description of the CNS Department: The Department of Cognitive and Neural Systems (CNS) provides advanced training and research experience for graduate students interested in the neural and computational principles, mechanisms, and architectures that underlie human and animal behavior, and the application of neural network architectures to the solution of technological problems. Students are trained in a broad range of areas concerning cognitive and neural systems, including vision and image processing; speech and language understanding; adaptive pattern recognition; cognitive information processing; self-organization; associative learning and long-term memory; computational neuroscience; nerve cell biophysics; cooperative and competitive network dynamics and short-term memory; reinforcement, motivation, and attention; adaptive sensory-motor control and robotics; active vision; and biological rhythms; as well as the mathematical and computational methods needed to support advanced modeling research and applications. The CNS Department awards MA, PhD, and BA/MA degrees. The CNS Department embodies a number of unique offerings. It has developed a curriculum that features 15 interdisciplinary graduate courses each of which integrates the psychological, neurobiological, mathematical, and computational information needed to theoretically investigate fundamental issues concerning mind and brain processes and the applications of neural networks to technology. Each course is typically taught once a week in the evening to make the program available to qualified students, including working professionals, throughout the Boston area. Nine additional research course are also offered. In these courses, one or two students meet regularly with one or two professors to pursue advanced reading and collaborative research. Students develop a coherent area of expertise by designing a program that includes courses in areas such as Biology, Computer Science, Engineering, Mathematics, and Psychology, in addition to courses in the CNS Department. The CNS Department prepares students for PhD thesis research with scientists in one of several Boston University research centers or groups, and with Boston-area scientists collaborating with these centers. The unit most closely linked to the department is the Center for Adaptive Systems (CAS). Students interested in neural network hardware work with researchers in CNS, the College of Engineering, and at MIT Lincoln Laboratory. Other research resources include distinguished research groups in neurophysiology, neuroanatomy, and neuropharmacology at the Medical School and the Charles River campus; in sensory robotics, biomedical engineering, computer and systems engineering, and neuromuscular research within the Engineering School; in dynamical systems within the Mathematics Department; in theoretical computer science within the Computer Science Department; and in biophysics and computational physics within the Physics Department. In addition to its basic research and training program, the Department offers a colloquium series, seminars, conferences, and special interest groups which bring many additional scientists from both experimental and theoretical disciplines into contact with the students. The CNS Department is moving in October, 1995 into its own new four-story building, which features a full range of offices, laboratories, classrooms, library, lounge, and related facilities for exclusive CNS use. 1995-96 CAS MEMBERS and CNS FACULTY: Jelle Atema Professor of Biology Director, Boston University Marine Program (BUMP) PhD, University of Michigan Sensory physiology and behavior Aijaz Baloch Research Associate of Cognitive and Neural Systems PhD, Electrical Engineering, Boston University Neural modeling of role of visual attention of recognition, learning and motor control, computational vision, adaptive control systems, reinforcement learning Helen Barbas Associate Professor, Department of Health Sciences, Boston University PhD, Physiology/Neurophysiology, McGill University Organization of the prefrontal cortex, evolution of the neocortex Jacob Beck Research Professor of Cognitive and Neural Systems PhD, Psychology, Cornell University Visual Perception, Psychophysics, Computational Models Daniel H. Bullock Associate Professor of Cognitive and Neural Systems and Psychology PhD, Psychology, Stanford University Real-time neural systems, sensory-motor learning and control, evolution of intelligence, cognitive development Gail A. Carpenter Professor of Cognitive and Neural Systems and Mathematics Director of Graduate Studies, Department of Cognitive and Neural Systems PhD, Mathematics, University of Wisconsin, Madison Pattern recognition, categorization, machine learning, differential equations Laird Cermak Professor of Neuropsychology, School of Medicine Professor of Occupational Therapy, Sargent College Director, Memory Disorders Research Center, Boston Veterans Affairs Medical Center PhD, Ohio State University Michael A. Cohen Associate Professor of Cognitive and Neural Systems and Computer Science Director, CAS/CNS Computation Labs PhD, Psychology, Harvard University Speech and language processing, measurement theory, neural modeling, dynamical systems H. Steven Colburn Professor of Biomedical Engineering PhD, Electrical Engineering, Massachusetts Institute of Technology Audition, binaural interaction, signal processing models of hearing William D. Eldred III Associate Professor of Biology BS, University of Colorado; PhD, University of Colorado, Health Science Center Visual neural biology Paolo Gaudiano Assistant Professor of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Computational and neural models of vision and adaptive sensory-motor control Jean Berko Gleason Professor of Psychology AB, Radcliffe College; AM, PhD, Harvard University Psycholinguistics Douglas Greve Research Associate of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Stephen Grossberg Wang Professor of Cognitive and Neural Systems Professor of Mathematics, Psychology, and Biomedical Engineering Director, Center for Adaptive Systems Chairman, Department of Cognitive and Neural Systems PhD, Mathematics, Rockefeller University Theoretical biology, theoretical psychology, dynamical systems, applied mathematics Frank Guenther Assistant Professor of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Biological sensory-motor control, spatial representation, speech production Thomas G. Kincaid Chairman and Professor of Electrical, Computer and Systems Engineering, College of Engineering PhD, Electrical Engineering, Massachusetts Institute of Technology Signal and image processing, neural networks, non-destructive testing Nancy Kopell Professor of Mathematics PhD, Mathematics, University of California at Berkeley Dynamical systems, mathematical physiology, pattern formation in biological/physical systems Ennio Mingolla Associate Professor of Cognitive and Neural Systems and Psychology Acting Chairman 1995-96, Department of Cognitive and Neural Systems PhD, Psychology, University of Connecticut Visual perception, mathematical modeling of visual processes Alan Peters Chairman and Professor of Anatomy and Neurobiology, School of Medicine PhD, Zoology, Bristol University, United Kingdom Organization of neurons in the cerebral cortex, effects of aging on the primate brain, fine structure of the nervous system Andrzej Przybyszewski Senior Research Associate of Cognitive and Neural Systems MSc, Technical Warsaw University; MA, University of Warsaw; PhD, Warsaw Medical Academy Adam Reeves Adjunct Professor of Cognitive and Neural Systems Professor of Psychology, Northeastern University PhD, Psychology, City University of New York Psychophysics, cognitive psychology, vision William Ross Research Associate of Cognitive and Neural Systems BSc, Cornell University; MA, PhD, Boston University Mark Rubin Research Assistant Professor of Cognitive and Neural Systems Research Physicist, Naval Air Warfare Center, China Lake, CA (on leave) PhD, Physics, University of Chicago Neural networks for vision, pattern recognition, and motor control Robert Savoy Adjunct Associate Professor of Cognitive and Neural Systems Scientist, Rowland Institute for Science PhD, Experimental Psychology, Harvard University Computational neuroscience; visual psychophysics of color, form, and motion perception Eric Schwartz Professor of Cognitive and Neural Systems; Electrical, Computer and Systems Engineering; and Anatomy and Neurobiology PhD, High Energy Physics, Columbia University Computational neuroscience, machine vision, neuroanatomy, neural modeling Robert Sekuler Adjunct Professor of Cognitive and Neural Systems Research Professor of Biomedical Engineering, College of Engineering, BioMolecular Engineering Research Center Jesse and Louis Salvage Professor of Psychology, Brandeis University AB,MA, Brandeis University; Sc.M., PhD, Brown University Allen Waxman Adjunct Associate Professor of Cognitive and Neural Systems Senior Staff Scientist, MIT Lincoln Laboratory PhD, Astrophysics, University of Chicago Visual system modeling, mobile robotic systems, parallel computing, optoelectronic hybrid architectures James Williamson Research Associate of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Image processing and object recognition. Particular interests are: dynamic binding, self-organization, shape representation, and classification Jeremy Wolfe Adjunct Associate Professor of Cognitive and Neural Systems Associate Professor of Ophthalmology, Harvard Medical School Psychophysicist, Brigham & Women's Hospital, Surgery Dept. Director of Psychophysical Studies, Center for Clinical Cataract Research PhD, Massachusetts Institute of Technology From rsun@cs.ua.edu Fri Oct 6 00:38:26 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 6 Oct 95 00:38:22 -0500; AA09171 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 6 Oct 95 00:38:20 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa18025; 5 Oct 95 13:54:34 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa18023; 5 Oct 95 13:28:47 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01147; 5 Oct 95 13:28:40 EDT Received: from MAILBOX.SRV.CS.CMU.EDU by B.GP.CS.CMU.EDU id aa27487; 5 Oct 95 13:14:23 EDT Received: from athos.cs.ua.edu by MAILBOX.SRV.CS.CMU.EDU id aa24354; 5 Oct 95 13:13:42 EDT Received: by athos.cs.ua.edu (AIX 3.2/UCB 5.64/4.03) id AA16982; Thu, 5 Oct 1995 09:11:57 -0500 Date: Thu, 5 Oct 1995 09:11:57 -0500 From: Ron Sun Message-Id: <9510051411.AA16982@athos.cs.ua.edu> To: cneuro@smaug.bbb.caltech.edu, cogpsy@phil.ruu.nl, connectionists@MAILBOX.SRV.CS.CMU.EDU, hybrid@cns.brown.edu, neuron-request@cattell.psych.upenn.edu, nl-kr@cs.rpi.edu ANNOUNCING A NEW MAILING LIST: The mailing lists. ------------------------------------------- As we discussed at the CSI workshop at IJCAI in this August, we now establish this new mailing list for the specific purpose of exchanging information and ideas regarding hybrid models, especially models integrating symbolic and connectionist processes. Other hybrid models, such as fuzzy logic+neural networks and GA+NN, are also covered. This is an unmoderated list. Conference and workshop announcements, papers and technical reports, informed discussions of specific topics in hybrid model areas, and other pertinent messages are appropriate items for submission. Email your submission to hybrid-list@cs.ua.edu, which will be automatically forwarded to all the recipients of the list. Information regarding subscription is attached below. For questions and suggestions regarding this list, send email to rsun@cs.ua.edu (only if you have to). This mailing list has incorporated the old HYBRID list at Brown U. maintained by Michael Perrone (thanks to Michael), and included names of those who attended the 1995 CSI workshop or expressed interest in it. (To remove your name from the list, see the instruction at the end of this message.) Regards, --Ron Sun ============================================================================== The University of Alabama Department of Computer Science has set up a list service for this: To subscribe to this list service, send an e-mail message to the userid "listproc@cs.ua.edu" with NO SUBJECT, but a one-line text message, as shown below: SUBSCRIBE hybrid-list YourFirstName YourLastName You should receive a response back indicating your addition to the list. After this, you can submit items to the list by simply e-mail'ing a message to the userid: "hybrid-list@cs.ua.edu". The message will automatically be sent to all individuals on the list. To unsubscribe to this list service, send an e-mail message to the userid "listproc@cs.ua.edu" with NO SUBJECT, but a one-line text message, as shown below: UNSUBSCRIBE hybrid-list ============================================================================== From john@dcs.rhbnc.ac.uk Fri Oct 6 00:38:28 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 6 Oct 95 00:38:26 -0500; AA09177 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 6 Oct 95 00:38:23 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab18318; 5 Oct 95 20:47:25 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab18316; 5 Oct 95 20:36:49 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01581; 5 Oct 95 20:36:38 EDT From: John Shawe-Taylor Message-Id: <199510051324.OAA07218@platon.cs.rhbnc.ac.uk> X-Authentication-Warning: platon.cs.rhbnc.ac.uk: Host localhost didn't use HELO protocol To: Connectionists@cs.cmu.edu Subject: Technical Report Series in Neural and Computational Learning Date: Thu, 05 Oct 95 14:24:13 +0100 X-Mts: smtp The European Community ESPRIT Working Group in Neural and Computational Learning Theory (NeuroCOLT): three new reports available ---------------------------------------- NeuroCOLT Technical Report NC-TR-95-051: ---------------------------------------- On the Computational Power of Continuous Time Neural Networks by Pekka Orponen, University of Helsinki, Finland Abstract: We investigate the computational power of continuous-time neural networks with Hopfield-type units. We prove that polynomial-size networks with saturated-linear response functions are at least as powerful as polynomially space-bounded Turing machines. ---------------------------------------- NeuroCOLT Technical Report NC-TR-95-052: ---------------------------------------- Computational Machine Learning in Theory and Praxis by Ming Li, University of Waterloo, Canada Paul Vitanyi, CWI and Universiteit van Amsterdam, The Netherlands. Abstract: In the last few decades a computational approach to machine learning has emerged based on paradigms from recursion theory and the theory of computation. Such ideas include learning in the limit, learning by enumeration, and probably approximately correct (pac) learning. These models usually are not suitable in practical situations. In contrast, statistics based inference methods have enjoyed a long and distinguished career. Currently, Bayesian reasoning in various forms, minimum message length (MML) and minimum description length (MDL), are widely applied approaches. They are the tools to use with particular machine learning praxis such as simulated annealing, genetic algorithms, genetic programming, artificial neural networks, and the like. These statistical inference methods select the hypothesis which minimizes the sum of the length of the description of the hypothesis (also called `model') and the length of the description of the data relative to the hypothesis. It appears to us that the future of computational machine learning will include combinations of the approaches above coupled with guaranties with respect to used time and memory resources. Computational learning theory will move closer to practice and the application of the principles such as MDL require further justification. Here, we survey some of the actors in this dichotomy between theory and praxis, we justify MDL via the Bayesian approach, and give a comparison between pac learning and MDL learning of decision trees. ---------------------------------------- NeuroCOLT Technical Report NC-TR-95-053: ---------------------------------------- On the relations between distributive computability and the BSS model by Sebastiano Vigna, University of Milan, Italy Abstract: This paper presents an equivalence result between computability in the BSS model and in a suitable distributive category. It is proved that the class of functions $R^m\to R^n$ (with $n,m$ finite and $R$ a commutative, ordered ring) computable in the BSS model and the functions distributively computable over a natural distributive graph based on the operations of $R$ coincide. Using this result, a new structural characterization, based on iteration, of the same functions is given. ----------------------- The Report NC-TR-95-051 can be accessed and printed as follows % ftp cscx.cs.rhbnc.ac.uk (134.219.200.45) Name: anonymous password: your full email address ftp> cd pub/neurocolt/tech_reports ftp> binary ftp> get nc-tr-95-051.ps.Z ftp> bye % zcat nc-tr-95-051.ps.Z | lpr -l Similarly for the other technical report. Uncompressed versions of the postscript files have also been left for anyone not having an uncompress facility. A full list of the currently available Technical Reports in the Series is held in a file `abstracts' in the same directory. The files may also be accessed via WWW starting from the NeuroCOLT homepage: http://www.dcs.rhbnc.ac.uk/neural/neurocolt.html Best wishes John Shawe-Taylor From rsun@cs.ua.edu Fri Oct 6 13:06:02 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 6 Oct 95 13:05:57 -0500; AA17665 Received: from bilby.cs.uwa.oz.au by lucy.cs.wisc.edu; Fri, 6 Oct 95 13:05:54 -0500 Received: from (mafm@parma.cs.uwa.oz.au [130.95.1.7]) by cs.uwa.oz.au (8.6.8/8.5) with SMTP id SAA15951; Fri, 6 Oct 1995 18:22:34 +0800 Message-Id: <199510061022.SAA15951@cs.uwa.oz.au> From: rsun@cs.ua.edu (Ron Sun) Subject: Hybrid Models mailing list To: bayes-news@STAT.CMU.EDU, ga-list@aic.nrl.navy.mil, met-ai@comp.vuw.ac.nz, mlnet@swi.psy.uva.nl, neuron@hplabs.hpl.hp.com, reinforce@cs.uwa.edu.au Date: Thu, 5 Oct 1995 13:01:46 -0500 ANNOUNCING A NEW MAILING LIST: The mailing lists. ------------------------------------------- As we discussed at the CSI workshop at IJCAI in this August, we now establish this new mailing list for the specific purpose of exchanging information and ideas regarding hybrid models, in artificial intelligence and cognitive science, especially models integrating symbolic and connectionist processes. Other hybrid models, such as fuzzy logic+neural networks and GA+NN, are also covered. This is an unmoderated list. Conference and workshop announcements, papers and technical reports, informed discussions of specific topics in hybrid model areas, and other pertinent messages are appropriate items for submission. Email your submission to hybrid-list@cs.ua.edu, which will be automatically forwarded to all the recipients of the list. Information regarding subscription is attached below. For questions and suggestions regarding this list, send email to rsun@cs.ua.edu (only if you have to). This mailing list has incorporated the old HYBRID list at Brown U. maintained by Michael Perrone (thanks to Michael), and included names of those who attended the 1995 CSI workshop or expressed interest in it. (To remove your name from the list, see the instruction at the end of this message.) Regards, --Ron Sun ============================================================================== The University of Alabama Department of Computer Science has set up a list service for this: To subscribe to this list service, send an e-mail message to the userid "listproc@cs.ua.edu" with NO SUBJECT, but a one-line text message, as shown below: SUBSCRIBE hybrid-list YourFirstName YourLastName You should receive a response back indicating your addition to the list. After this, you can submit items to the list by simply e-mail'ing a message to the userid: "hybrid-list@cs.ua.edu". The message will automatically be sent to all individuals on the list. To unsubscribe from this list service, send an e-mail message to the userid "listproc@cs.ua.edu" with NO SUBJECT, but a one-line text message, as shown below: UNSUBSCRIBE hybrid-list ============================================================================== From berns@fzi.de Fri Oct 6 13:40:32 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 6 Oct 95 13:40:29 -0500; AA18119 Received: from bilby.cs.uwa.oz.au by lucy.cs.wisc.edu; Fri, 6 Oct 95 13:40:25 -0500 Received: from (mafm@parma.cs.uwa.oz.au [130.95.1.7]) by cs.uwa.oz.au (8.6.8/8.5) with SMTP id SAA15958; Fri, 6 Oct 1995 18:23:22 +0800 Message-Id: <199510061023.SAA15958@cs.uwa.oz.au> From: Karsten Berns Sender: berns@fzi.de To: reinforce@cs.uwa.oz.au Cc: berns@fzi.de Subject: walking machine and RL Date: Fri, 6 Oct 1995 09:08:19 +0100 Dear Malcolm, I don"t know if you have read our paper about reinforcement learning and walking machines. If you need further information about this or about other reinforcement approaches for mobile robots please contact w. Ilg. He is a group member of us. He responsible in our group for RL in robotics. If you have a walking machine in your institute please fill in the questonary of walking machines in our www pages (http;//www.fzi.de/divisions/ipt/WMC/questionary.html). I have build up this questionary to bring together the different research groups of walking machines. I hope that I could help you Regards Karsten From ajit@austin.ibm.com Fri Oct 6 20:26:51 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 6 Oct 95 20:26:48 -0500; AA24646 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 6 Oct 95 20:26:46 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa18318; 5 Oct 95 20:46:06 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa18316; 5 Oct 95 20:36:47 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01576; 5 Oct 95 20:36:01 EDT Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa01243; 5 Oct 95 16:41:41 EDT Received: from netmail.austin.ibm.com by EDRC.CMU.EDU id aa04915; 5 Oct 95 16:41:02 EDT Received: from ding.austin.ibm.com (ding.austin.ibm.com [129.35.58.226]) by netmail.austin.ibm.com (8.6.12/8.6.11) with SMTP id PAA99847 for ; Thu, 5 Oct 1995 15:40:55 -0500 Received: by ding.austin.ibm.com (AIX 3.2/UCB 5.64/4.03-client-2.6) for Connectionists@CS.CMU.EDU at austin.ibm.com; id AA14254; Thu, 5 Oct 1995 15:40:53 -0500 Date: Thu, 5 Oct 1995 15:40:53 -0500 From: Dingankar Message-Id: <9510052040.AA14254@ding.austin.ibm.com> To: Connectionists@cs.cmu.edu Subject: Dissertation available in Neuroprose Reply-To: ajit@austin.ibm.com Return-Receipt-To: ajit@austin.ibm.com Organization: IBM, Austin Work: (512) 838-6850 Fax: (512) 838-5882 Ftp-Host: archive.cis.ohio-state.edu Ftp-Filename: /pub/neuroprose/Thesis/dingankar.thesis.ps.Z **DO NOT FORWARD TO OTHER GROUPS** Sorry, no hardcopies available. URL: ftp://archive.cis.ohio-state.edu/pub/neuroprose/Thesis/dingankar.thesis.ps.Z BiBTeX entry: @PhdThesis{ajit-phd, author = "Ajit Trimbak Dingankar", title = "On Applications of Approximation Theory to Identification, Control and Classification", school = "The University of Texas at Austin", year = 1995, address = "Austin, Texas", } Abstract Applications of approximation theory to some problems in identification of dynamic systems, their control, and to problems in signal classification are studied. First, an algorithm is given for constructing approximations in a wide variety of settings, and a corresponding error bound is derived. Then weak sufficient conditions for perfect classification of signals are studied. Next the problem of approximating linear functionals with certain sums of integrals is studied, alongwith its relation to the approximation of nonlinear functionals. Then an approximation theoretic characterization of continuity of nonlinear maps is given. As another application of function approximation, the problem of universally approximating controllers for discrete-time continuous plants is studied. Finally, error bounds for approximation of functions defined on finite dimensional Hilbert spaces are given. ------------------------------------------------------------------------------ Ajit T. Dingankar | ajit@austin.ibm.com IBM Corporation, Internal Zip 4359 | Work: (512) 838-6850 11400 Burnet Road, Austin, TX 78758 | Fax : (512) 838-5882 From listerrj@helios.aston.ac.uk Fri Oct 6 20:26:57 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 6 Oct 95 20:26:53 -0500; AA24652 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 6 Oct 95 20:26:49 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa19093; 6 Oct 95 5:54:50 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa19091; 6 Oct 95 5:45:35 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02246; 6 Oct 95 5:45:14 EDT Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa09185; 6 Oct 95 5:27:22 EDT Received: from hermes.aston.ac.uk by CS.CMU.EDU id aa15645; 6 Oct 95 5:26:25 EDT Received: from sun.aston.ac.uk (actually host dendrite.aston.ac.uk) by hermes.aston.ac.uk with SMTP (PP); Fri, 6 Oct 1995 10:30:38 +0100 Message-Id: <27328.9510060924@sun.aston.ac.uk> To: Connectionists@cs.cmu.edu, cphc-jobs@ukc.ac.uk, allstat@mailbase.ac.uk Subject: Postdoctoral position in Neural Computing Reply-To: listerrj@aston.ac.uk X-Mailer: Mew beta version 0.98 on Emacs 19.29.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Fri, 06 Oct 1995 10:24:28 +0100 From: Richard Lister Content-Length: 1963 ---------------------------------------------------------------------- Neural Computing Research Group ------------------------------- Dept of Computer Science and Applied Mathematics Aston University, Birmingham, UK Nonstationary Feature Extraction and Tracking for the Classification -------------------------------------------------------------------- of Turning Points in Multivariate Time Series ---------------------------------------------- The Neural Computing Research Group at Aston is looking for a highly motivated individual for a 3 year postdoctoral research position. The investigation will involve generic problems of feature extraction in nonstationary environments, typically from univariate and multivariate time series. Potential candidates should be mathematically and computationally competent with a background either in artificial neural networks, dynamical systems theory, statistical pattern processing, or have relevant experience from a physics or electrical engineering background. *** Further details at http://neural-server.aston.ac.uk/ *** Conditions of Service --------------------- Salaries will be up to point 6 on the RA 1A scale, currently 15,986 UK pounds. These salary scales are currently under review, and are subject to annual increments. How to Apply ------------ Please send a full CV and publications list, together with the names of 4 referees, to: Professor D Lowe Neural Computing Research Group Department of Computer Science and Applied Mathematics Aston University Birmingham B4 7ET, U.K. Tel: 0121 333 4631 Fax: 0121 333 6215 email: d.lowe@aston.ac.uk (email submission of postscript files is welcome) Closing date: 15 November 1995 ---------------------------------------------------------------------- From winther@connect.nbi.dk Fri Oct 6 20:26:58 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 6 Oct 95 20:26:55 -0500; AA24656 Message-Id: <9510070126.AA22974@lucy.cs.wisc.edu> Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 6 Oct 95 20:26:53 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab19611; 6 Oct 95 13:41:43 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab19609; 6 Oct 95 13:27:07 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02534; 6 Oct 95 13:26:30 EDT Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa13589; 6 Oct 95 11:19:36 EDT Received: from connect.nbi.dk by EDRC.CMU.EDU id aa08612; 6 Oct 95 11:18:52 EDT Received: by connect.nbi.dk (1.38.193.4/16.2) id AA23724; Fri, 6 Oct 1995 16:18:50 +0100 From: Ole Winther Subject: Paper available: "A mean field approach to Bayes Learning in feed-forward neural networks" Date: October 6, 1995 Apparently-To: connectionists@cs.cmu.edu FTP-host: connect.nbi.dk FTP-file: neuroprose/opper.bayes.ps.Z WWW-host: http://connect.nbi.dk ---------------------------------------------- The following paper is now available: A mean field approach to Bayes Learning in feed-forward neural networks [12 pages] Manfred Opper Theoretical Physics, University of Wurzburg, Germany Ole Winther CONNECT, The Niels Bohr Institute, University of Copenhagen, Denmark Abstract: We propose an algorithm to realise Bayes optimal predictions for feed-forward neural networks which is based on the TAP mean field method developed for the statistical mechanics of disordered systems. We conjecture that our approach will be exact in the thermodynamic limit. The algorithm results in a simple built-in leave-one-out crossvalidation of the predictions. Simulations for the case of the simple perceptron and the committee machine are in excellent agreement with the results of replica theory. Please do not reply directly to this message. ----------------------------------------------- FTP-instructions: unix> ftp connect.nbi.dk (or 130.225.212.30) ftp> Name: anonymous ftp> Password: your e-mail address ftp> cd neuroprose ftp> binary ftp> get opper.bayes.ps.Z ftp> quit unix> uncompress opper.bayes.ps.Z ----------------------------------------------- Ole Winther, Computational Neural Network Center (CONNECT) Niels Bohr Institute Blegdamsvej 17 2100 Copenhagen Denmark Telephone: +45-3532-5200 Direct: +45-3532-5311 Fax: +45-3142-1016 e-mail: winther@connect.nbi.dk From arbib@pollux.usc.edu Fri Oct 6 20:27:00 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 6 Oct 95 20:26:57 -0500; AA24665 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 6 Oct 95 20:26:54 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa19705; 6 Oct 95 15:05:38 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa19703; 6 Oct 95 14:49:28 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02621; 6 Oct 95 14:48:39 EDT Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa15719; 6 Oct 95 13:50:29 EDT Received: from pollux.usc.edu by CS.CMU.EDU id aa19739; 6 Oct 95 13:50:16 EDT Received: (arbib@localhost) by pollux.usc.edu (8.6.12/8.6.4) id KAA11592; Fri, 6 Oct 1995 10:50:05 -0700 Date: Fri, 6 Oct 1995 10:50:05 -0700 From: "Michael A. Arbib" Message-Id: <199510061750.KAA11592@pollux.usc.edu> To: bayes-news@STAT.CMU.EDU, connectionists@cs.cmu.edu, ga-list@aic.nrl.navy.mil, met-ai@comp.vuw.ac.nz, mlnet@swi.psy.uva.nl, neuron@hplabs.hpl.hp.com, reinforce@cs.uwa.edu.au Subject: A New Series of Virtual Textbooks on Neural Networks Cc: rutter@mit.edu October 6, 1995 Yesterday, a visitor to my office, while speaking of his enthusiasm for "The Handbook of Brain Theory and Neural Networks", mentioned that some of his colleagues had criticized the fact that the [266] articles [in Part III] were arranged in alphabetical order, thus lacking the "logical order" to make the book easy to use for teaching. The purpose of this note is to answer such concerns. 1. The boring answer is that a Handbook is not a Textbook. Indeed, given that the 266 articles provide such a comprehensive overview - including detailed models of single neurons; analysis of a wide variety of neurobiological systems; connectionist studies; mathematical analyses of abstract neural networks; and technological applications of adaptive, artificial neural networks and related methodologies - it is hard to imagine a course that would cover the whole book, no matter in what order the articles were presented. 2. The exciting answer is that THE HANDBOOK IS A VIRTUAL LIBRARY OF TWENTY-THREE TEXTBOOKS!! Before the 266 articles of Part III come Part I and Part II. Part I provides an introductory textbook level introduction to Neural Networks. Part II provides 23 "road maps", each of which lists the articles on a particular theme, followed by an essay which offers a "logical order" in which to read these articles. Thus, the Handbook can be used to provide a "virtual textbook" on any one of the following 23 topics: Applications of Neural Networks Artificial Intelligence and Neural Networks Biological Motor Control Biological Networks Biological Neurons Computability and Complexity Connectionist Linguistics Connectionist Psychology Control Theory and Robotics Cooperative Phenomena Development and Regeneration of Neural Networks Dynamic Systems and Optimization Implementation of Neural Networks Learning in Artificial Neural Networks, Deterministic Learning in Artificial Neural Networks, Statistical Learning in Biological Systems Mammalian Brain Regions Mechanisms of Neural Plasticity Motor Pattern Generators and Neuroethology Primate Motor Control Self-Organization in Neural Networks Other Sensory Systems Vision In each case, the instructor can follow the road map to traverse the articles to provide full coverage of the topic, using the cross- references to choose supplementary material from within the Handbook, and the carefully selected list of readings at the end of each article to choose supplementary material from the general literature. As an appendix to this message, I include a sample road map, that on "Learning in Artificial Neural Networks, Deterministic". All the road maps are available on the Web at: http://www-mitpress.mit.edu/mitp/recent- books/comp/handbook-brain-theo.html If you have other queries about how best to use the Handbook, or suggestions for improving the Handbook, please feel free to contact me by email: arbib@pollux.usc.edu. With best wishes Michael Arbib ***** APPENDIX: The Road Map for "Learning in Artificial Neural Networks, Deterministic" from Part II of The Handbook of Brain Theory and Neural Networks, (M.A. Arbib, Ed.), A Bradford Book, copyright 1995, The MIT Press. LEARNING IN ARTIFICIAL NEURAL NETWORKS, DETERMINISTIC [Articles in the Road Map, listed in Alphabetical Order.] Adaptive Resonance Theory Associative Networks Backpropagation: Basics and New Developments Convolutional Networks for Images, Speech, and Time-Series Coulomb Potential Learning Kolmogorov's Theorem Learning by Symbolic and Neural Methods Learning as Hill-Climbing in Weight Space Learning as Adaptive Control of Synaptic Matrices Modular Neural Net Systems, Training of Neocognitron: A Model for Visual Pattern Recognition Neurosmithing: Improving Neural Network Learning Nonmonotonic Neuron Associative Memory Pattern Recognition Perceptrons, Adalines, and Backpropagation Recurrent Networks: Supervised Learning Reinforcement Learning Topology-Modifying Neural Network Algorithms [Articles in the Road Map, discussed in Logical Order.] Much of our concern is with supervised learning, getting a network to behave in a way which successfully approximates some specified pattern of behavior or input-output relationship. In particular, much emphasis has been placed on feedforward networks, that is, networks which have no loops, so that the output of the net depends on its input alone, since there is then no internal state defined by reverberating activity. The most direct form of this is a synaptic matrix, a one-layer neural network for which input lines directly drive the output neurons and a "supervised Hebbian" rule sets synapses so that the network will exhibit specified input- output pairs in its response repertoire. This is addressed in the article on ASSOCIATIVE NETWORKS, which notes the problems that arise if the input patterns (the "keys" for associations) are not orthogonal vectors. Association also extends to recurrent networks obtained from one layer networks by feedback connections from the output to the input, but in such systems of "dynamic memories" (e.g., Hopfield networks) there are no external inputs as such. Rather the "input" is the initial state of the network, and the "output" is the "attractor" or equilibrium state to which the network then settles. Unfortunately, the usual "attractor network" memory model, with neurons whose output is a sigmoid function of the linear combination of their inputs, has many spurious memories, i.e., equilibria other than the memorized patterns, and there is no way to decide a memorized pattern is recalled or not. The article on NONMONOTONIC NEURON ASSOCIATIVE MEMORY shows that, if the output of each neuron is a nonmonotonic function of its input, the capacity of the network can be increased, and the network does not exhibit spurious memories: when the network fails to recall a correct memorized pattern, the state shows a chaotic behavior instead of falling into a spurious memory. Historically, the earliest forms of supervised learning involved changing synaptic weights to oppose the error in a neuron with a binary output (the perceptron error-correction rule), or to minimize the sum of squares of errors of output neurons in a network with real- valued outputs (the Widrow-Hoff rule). This work is charted in the article on PERCEPTRONS, ADALINES AND BACKPROPAGATION, which also charts the extension of these classic ideas to multilayered feedforward networks. Multilayered networks pose the structural credit assignment problem: when an error is made at the output of a network, how is credit (or blame) to be assigned to neurons deep within the network? One of the most popular techniques is called backpropagation, whereby the error of output units is propagated back to yield estimates of how much a given "hidden unit" contributed to the output error. These estimates are used in the adjustment of synaptic weights to these units within the network. The article on BACKPROPAGATION: BASICS AND NEW DEVELOPMENTS places this idea in a broader mathematical and historical framework in which backpropagation is seen as a general method for calculating derivatives to adjust the weights of nonlinear systems, whether or not they are neural networks. The underlying theoretical grounding is that, given any function f: X . Y for which X and Y are codable as input and output patterns of a neural network, then, as shown in the article on KOLMOGOROV'S THEOREM, f can be approximated arbitrarily well by a feedforward network with one layer of hidden units. The catch, of course, is that many, many hidden units may be required for a close fit. It is often an empirical question whether there exists a sufficiently good approximation achievable in principle by a network of a given size P an approximation which a given learning rule may or may not find (it may, for example, get stuck in a local optimum rather than a global one). The article on NEUROSMITHING: IMPROVING NEURAL NETWORK LEARNING provides a number of "rules of thumb" to be used in applying backpropagation in trying to find effective settings for network size and for various coefficients in the learning rules. One useful perspective for supervised learning views LEARNING AS HILL-CLIMBING IN WEIGHT SPACE, so that each "experience" adjusts the synaptic weights of the network to climb (or descend) a metaphorical hill for which "height" at a particular point in "weight space" corresponds to some measure of the performance of the network (or the organism or robot of which it is a part). When the aim is to minimize this measure, one of the basic techniques for learning is what mathematicians call "gradient descent"; optimization theory also provides alternative methods such as, e.g., that of conjugate gradients, which are also used in the neural network literature. REINFORCEMENT LEARNING describes a form of "semi-supervised" learning where the network is not provided with an explicit form of error at each time step but rather receives only generalized reinforcement ("you're doing well"; "that was bad!") which yields little immediate indication of how any neuron should change its behavior. Moreover, the reinforcement is intermittent, thus raising the temporal credit assignment problem: how is an action at one time to be credited for positive reinforcement at a later time? One solution is to build an "adaptive critic" which learns to evaluate actions of the network on the basis of how often they occur on a path leading to positive or negative reinforcement. Another perspective on supervised learning is presented in LEARNING AS ADAPTIVE CONTROL OF SYNAPTIC MATRICES, which views learning as a control problem (controlling synaptic matrices to yield a given network behavior) and then uses the adjoint equations of control theory to derive synaptic adjustment rules. Gradient descent methods have also been extended to adapt the synaptic weights of recurrent networks, as discussed in RECURRENT NETWORKS: SUPERVISED LEARNING, where the aim is to match the time course of network activity, rather than the (input, output) pairs of some training set. The task par excellence for supervised learning is pattern recognition, the problem of classifying objects, often represented as vectors or as strings of symbols, into categories. Historically, the field of pattern recognition started with early efforts in neural networks (see PERCEPTRONS, ADALINES AND BACKPROPAGATION). While neural networks played a less central role in pattern recognition for some years, recent progress has made them the method of choice for many applications. As PATTERN RECOGNITION demonstrates, multilayer networks, when properly designed, can learn complex mappings in high-dimensional spaces without requiring complicated hand-crafted feature extractors. To rely more on learning, and less on detailed engineering of feature extractors, it is crucial to tailor the network architecture to the task, incorporating prior knowledge to be able to learn complex tasks without requiring excessively large networks and training sets. Many specific architectures have been developed to solve particular types of learning problem. ADAPTIVE RESONANCE THEORY (ART) bases learning on internal expectations. When the external world fails to match an ART network's expectations or predictions, a search process selects a new category, representing a new hypothesis about what is important in the present environment. The neocognitron (see NEOCOGNITRON: A MODEL FOR VISUAL PATTERN RECOGNITION) was developed as a neural network model for visual pattern recognition which addresses the specific question "how can a pattern be recognized despite variations in size and position?" by using a multilayer architecture in which local features are replicated in many different scales and locations. More generally, as shown in CONVOLUTIONAL NETWORKS FOR IMAGES, SPEECH, AND TIME SERIES, shift invariance in convolutional networks is obtained by forcing the replication of weight configurations across space. Moreover, the topology of the input is taken into account, enabling such networks to force the extraction of local features by restricting the receptive fields of hidden units to be local. COULOMB POTENTIAL LEARNING derives its name from its functional form's likeness to a coulomb charge potential, replacing the linear separability of a simple perceptron with a network that is capable of constructing arbitrary nonlinear boundaries for classification tasks. We have already noted that networks that are too small cannot learn the desired input to output mapping. However, networks can also be too large. Just as a polynomial of too high a degree is not useful for curve-fitting, a network that is too large will fail to generalize well, and will require longer training times. Smaller networks, with fewer free parameters, enforce a smoothness constraint on the function found. For best performance, it is, therefore, desirable to find the smallest network that will "properly" fit the training data. The article TOPOLOGY- MODIFYING NEURAL NETWORK ALGORITHMS reviews algorithms which adjust network topology (i.e., adding or removing neurons during the learning process) to arrive at a network appropriate to a given task. The last two articles in this road map take a somewhat different viewpoint from that of adjusting the synaptic weights in a single network. MODULAR NEURAL NET SYSTEMS, TRAINING OF presents the idea that, although single neural networks are theoretically capable of learning complex functions, many problems are better solved by designing systems in which several modules cooperate together to perform a global task, replacing the complexity of a large neural network by the cooperation of neural network modules whose size is kept small. The article on LEARNING BY SYMBOLIC AND NEURAL METHODS focuses on the distinction between symbolic learning based on producing discrete combinations of the features used to describe examples and neural approaches which adjust continuous, nonlinear weightings of their inputs. The article not only compares but also combines the two approaches, showing for example how symbolic knowledge may be used to set the initial state of an adaptive network. [This Road Map is then followed by one on "Learning in Artificial Neural Networks, Statistical"] From dwang@cis.ohio-state.edu Sat Oct 7 16:15:40 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 7 Oct 95 16:15:37 -0500; AA04150 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 7 Oct 95 16:15:35 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa19611; 6 Oct 95 13:40:12 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa19609; 6 Oct 95 13:26:59 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02529; 6 Oct 95 13:26:14 EDT Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa13541; 6 Oct 95 11:14:26 EDT Received: from news.cis.ohio-state.edu by EDRC.CMU.EDU id aa08594; 6 Oct 95 11:13:34 EDT Received: from shirt.cis.ohio-state.edu (shirt.cis.ohio-state.edu [164.107.47.4]) by news.cis.ohio-state.edu (8.6.8.1/8.6.4) with ESMTP id LAA08687; Fri, 6 Oct 1995 11:13:24 -0400 From: DeLiang Wang Received: (dwang@localhost) by shirt.cis.ohio-state.edu (8.6.7/8.6.4) id LAA01790; Fri, 6 Oct 1995 11:13:23 -0400 Date: Fri, 6 Oct 1995 11:13:23 -0400 Message-Id: <199510061513.LAA01790@shirt.cis.ohio-state.edu> To: Connectionists@cs.cmu.edu Subject: Preprint available on auditory scene analysis Cc: dwang@cis.ohio-state.edu The following preprint is available via FTP/WWW: ------------------------------------------------------------------ Primitive Auditory Segregation Based on Oscillatory Correlation ------------------------------------------------------------------ DeLiang Wang Cognitive Science: Accepted for publication The Ohio State University Auditory scene analysis is critical for complex auditory processing. We study auditory segregation from the neural network perspective, and develop a framework for primitive auditory scene analysis. The architecture is a laterally coupled two-dimensional network of relaxation oscillators with a global inhibitor. One dimension represents time and another one represents frequency. We show that this architecture, plus systematic delay lines, can in real time group auditory features into a stream by phase synchrony and segregate different streams by desynchronization. The network demonstrates a set of psychological phenomena regarding primitive auditory scene analysis, including dependency on frequency proximity and the rate of presentation, sequential capturing, and competition among different perceptual organizations. We offer a neurocomputational theory - shifting synchronization theory - for explaining how auditory segregation might be achieved in the brain, and the psychological phenomenon of stream segregation. Possible extensions of the model are discussed. (42 pages + one figure = 1.5MB + 600 KB) for anonymous ftp: FTP-HOST: ftp.cis.ohio-state.edu Directory: /pub/leon/Wang95 FTP-filenames: Wang.prep.ps.Z, fig5G.ps.Z or for WWW: http://www.cis.ohio-state.edu/~dwang Comments are most welcome - Please send to DeLiang Wang (dwang@cis.ohio-state.edu) ---------------------------------------------------------------------------- FTP instructions: To retrieve and print the files, use the following commands: unix> ftp ftp.cis.ohio-state.edu Name: anonymous Password: (your email address) ftp> binary ftp> cd /pub/leon/Wang95 ftp> get Wang.prep.ps.Z ftp> get fig5G.ps.Z ftp> quit unix> uncompress Wang.prep.ps.Z unix> uncompress fig5G.ps.Z unix> lpr {each of the two postscript files} (Wang.prep.ps may not ghostview well - some figures do not show up with my ghostview - but it should print ok) ---------------------------------------------------------------------------- From terry@salk.edu Sat Oct 7 16:15:46 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 7 Oct 95 16:15:43 -0500; AA04156 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 7 Oct 95 16:15:39 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa20390; 6 Oct 95 22:26:32 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa20388; 6 Oct 95 22:16:33 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02840; 6 Oct 95 22:16:18 EDT Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa22466; 6 Oct 95 22:14:33 EDT Received: from helmholtz.salk.edu by CS.CMU.EDU id aa23794; 6 Oct 95 22:14:07 EDT Received: by salk.edu (4.1/SMI-4.1) id AA28375; Fri, 6 Oct 95 19:14:01 PDT Date: Fri, 6 Oct 95 19:14:01 PDT From: Terry Sejnowski Message-Id: <9510070214.AA28375@salk.edu> To: connectionists@cs.cmu.edu Subject: Neural Computation 7:6 NEURAL COMPUTATION Vol 7, Issue 6, November 1995 Article: An information-maximization approach to blind separation and blind deconvolution Anthony J. Bell and Terrence J. Sejnowski Note: A perceptron reveals the face of sex Michael Gray, David Lawrence, Beatrice Golomb, and Terrence Sejnowski Letters: Self-organization as an iterative kernel smoothing process Vladmir Cherkassky and Filip Mulier On the distribution and the convergence of feature space in self-organizing maps Hujun Yin and Nigel Allinson Sorting with self-organizing maps Marco Budinich Introducing asymmetry into interneuron learning Colin Fyfe Learning and generalization with minimerror, a temperature dependent learning algorithm Bruno Raffin and Mirta B. Gordon Regularized neural networks: Some convergence rate results Halbert White and Valentina Corradi The target switch algorithm: A constructive learning procedure for feedforward neural networks Colin Campbell and C. Perez Vicente On the practical applicability of VC dimension bounds Sean B. Holden and Mahesan Niranjan LeRec: A NN/HMM hybrid for on-line handwriting recognition Yoshua Bengio, Yann LeCun, Craig Nohl, and Chris Burges ----- NOTE: IN 1996 NEURAL COMPUTATION WILL PUBLISH 8 ISSUES ABSTRACTS - http://www-mitpress.mit.edu/jrnls-catalog/neural.html SUBSCRIPTIONS - 1996 - VOLUME 8 - 8 ISSUES ______ $50 Student and Retired ______ $78 Individual ______ $220 Institution Add $28 for postage and handling outside USA (+7% GST for Canada). (Back issues from Volumes 1-7 are regularly available for $28 each to institutions and $14 each for individuals Add $5 for postage per issue outside USA (+7% GST for Canada) mitpress-orders@mit.edu MIT Press Journals, 55 Hayward Street, Cambridge, MA 02142. Tel: (617) 253-2889 FAX: (617) 258-6779 ----- From honavar@cs.iastate.edu Sat Oct 7 16:15:48 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 7 Oct 95 16:15:45 -0500; AA04158 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 7 Oct 95 16:15:42 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa20460; 6 Oct 95 22:44:03 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa20458; 6 Oct 95 22:36:00 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02895; 6 Oct 95 22:35:32 EDT Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa22725; 6 Oct 95 22:31:40 EDT Received: from cs.iastate.edu by CS.CMU.EDU id aa23870; 6 Oct 95 22:31:21 EDT Received: from ren.cs.iastate.edu (ren.cs.iastate.edu [129.186.3.41]) by cs.iastate.edu (8.7.1/8.7.1) with ESMTP id VAA29614; Fri, 6 Oct 1995 21:31:13 -0500 (CDT) From: Vasant Honavar Received: (honavar@localhost) by ren.cs.iastate.edu (8.6.12/8.6.9) id VAA12809; Fri, 6 Oct 1995 21:31:10 -0500 Message-Id: <199510070231.VAA12809@ren.cs.iastate.edu> Subject: paper available: constructive learning algorithms To: connectionists@cs.cmu.edu Date: Fri, 6 Oct 1995 21:31:09 -0500 (CDT) Cc: Vasant Honavar X-Mailer: ELM [version 2.4 PL24] Content-Type: text The following paper is now available in postscript form on the WWW via the URL: http://www.cs.iastate.edu/~honavar/publist.html Constructive Neural Network Learning Algorithms for Multi-Category Pattern Classification Technical Report TR95-15 Rajesh Parekh, Jihoon Yang, and Vasant Honavar Artificial Intelligence Research Group Department of Computer Science 226 Atanasoff Hall, Iowa State University, Ames, IA 50011. U.S.A. parekh|yang|honavar@cs.iastate.edu Abstract Constructive learning algorithms offer an approach for incremental construction of potentially near-minimal neural network architectures for pattern classification tasks. Such algorithms help overcome the need for ad-hoc and often inappropriate choice of network topology in the use of algorithms that search for a suitable weight setting in an otherwise a-priori fixed network architecture. Several such algorithms proposed in the literature have been shown to converge to zero classification errors (under certain assumptions) on a finite, non-contradictory training set in a 2-category classification problem. This paper explores multi-category extensions of several constructive neural network learning algorithms for pattern classification. In each case, we establish the convergence to zero classification errors on a multi-category classification task (under certain assumptions). Results of experiments with non-separable data sets demonstrate the feasibility of this approach to multi-category pattern classification and also suggest several interesting directions for future research. ------------------------------------------------------------ Vasant Honavar honavar@cs.iastate.edu http://www.cs.iastate.edu/~honavar/homepage.html