From edelman@wisdom.weizmann.ac.il Sun Mar 10 16:04:45 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sun, 10 Mar 96 16:04:42 -0600; AA19797 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sun, 10 Mar 96 16:04:36 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa11881; 10 Mar 96 15:01:46 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa11879; 10 Mar 96 14:38:27 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01625; 10 Mar 96 14:38:12 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa10450; 10 Mar 96 12:06:33 EST Received: from narkis.wisdom.weizmann.ac.il by EDRC.CMU.EDU id aa02810; 10 Mar 96 12:05:15 EST Received: from lachesis.wisdom.weizmann.ac.il (lachesis.wisdom.weizmann.ac.il [132.76.80.188]) by narkis.wisdom.weizmann.ac.il (8.6.5/mail.byaddr) with ESMTP id TAA21645 for ; Sun, 10 Mar 1996 19:02:20 +0200 From: Edelman Shimon Received: (edelman@localhost) by lachesis.wisdom.weizmann.ac.il (8.6.9/8.6.5) id RAA08020; Sun, 10 Mar 1996 17:01:12 GMT Date: Sun, 10 Mar 1996 17:01:12 GMT Message-Id: <199603101701.RAA08020@lachesis.wisdom.weizmann.ac.il> To: Connectionists@cs.cmu.edu In-Reply-To: <199603090225.SAA14592@mizar.usc.edu> (message from Irving Biederman on Fri, 8 Mar 1996 18:25:00 -0800) Subject: Re: Shift Invariance > Date: Fri, 8 Mar 1996 18:25:00 -0800 > From: Irving Biederman > > The communication by Shimon Edelman is, in my opinion, a bit > misleading. In response to a posting by Eric Postma that listed papers by > Biederman & Cooper (1991) and Nazir & O'Regan (1990) as evidence for shift > invariance, Edelman writes: > > "Putting Nazir & O'Regan on the same list with Biederman like that may > be misleading to someone who will not bother to read the paper. Nazir > & O'Regan actually found evidence AGAINST translation invariance in > human vision." > > One may distinguish a strong form of shift invariance, in which > there is no cost in performance from changing the position of a stimulus > with a weak form in which there is facilitation but not as much as when the > stimulus is presented at its originally experienced position. > ... [ rest of Biederman's message omitted ] > ... Many thanks to Irv Biederman for posting the details of his findings, along with a comparison with the results of Nazir & O'Regan. His effort should reduce the chance of the readers of this list jumping to premature conclusions. Note that the purpose of my previous posting was to advocate caution, certainly not to argue that all claims of invariance are wrong. Fortunately, my job in this matter is easy: just one example of a manifest lack of invariance suffices to invalidate the strong version of invariance-based theory of vision, which seems to be espoused by Goldfarb: > If we 1) DO NOT FORGET that the biological systems have at their disposal > quite adequate means to extract symbolic (structural) representation right > from the very beginning and 2) FORGET about our inadequate numeric models, > then the question would not have arisen in the first place. Symbolic > representations EMBODY shift invariance. So, here it goes... Whereas invariance does hold in many recognition tasks (in particular, in Biederman's experiments, as well as in the experiments reported in [1]), it does not in others (as, e.g., in [2], where interaction between size invariance and orientation is reported). A recent comprehensive survey of (the far from invariant) human performance in recognizing rotated objects can be found in [3]. Furthermore, not only recognition, but also perceptual learning, seems to be non-invariant in some cases; see [4,5]. FORGETTING about experimental findings will not make them go away, just as pointing out that symbolic representations EMBODY invariance will not make biological vision embrace a symbolic approach if it has not done so until now. -Shimon Dr. Shimon Edelman, Applied Math. & Computer Science Weizmann Institute of Science, Rehovot 76100, Israel The Web: http://eris.wisdom.weizmann.ac.il/~edelman fax: (+972) 8 344122 tel: 8 342856 sec: 8 343545 ----------------------------------------------------------------------------- References: [1] @article{BricoloBulthoff92, author="E. Bricolo and H. H. {B\"ulthoff}", title="Translation-invariant features for object recognition", journal="Perception", volume="21 (supp.2)", year = 1992, pages = "59" } [2] @article{BricoloBulthoff93a, author="E. Bricolo and H. H. {B\"ulthoff}", title="Further evidence for viewer-centered representations", journal="Perception", volume="22 (supp)", year = 1993, pages = "105" } [3] @InCollection{JolicoeurHumphrey94, author = "P. Jolicoeur and G. K. Humphrey", title = "Perception of rotated two-dimensional and three-dimensional objects and visual shapes", booktitle = "Perceptual constancies", publisher = "Cambridge University Press", year = 1994, editor = "V. Walsh and J. Kulikowski", chapter = 10, address = "Cambridge, UK", note = "in press" } [4] @article{KarniSagi91, author="A. Karni and D. Sagi", title="Where practice makes perfect in texture discrimination", journal=pnas, volume="88", pages="4966-4970", year="1991" } [5] @article{PoggioFahleEdelman92, author="T. Poggio and M. Fahle and S. Edelman", title="Fast perceptual learning in visual hyperacuity", journal="Science", year="1992", volume="256", pages="1018-1021", } From goldfarb@unb.ca Mon Mar 11 01:47:01 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 11 Mar 96 01:46:58 -0600; AB27019 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 11 Mar 96 01:46:56 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa12530; 11 Mar 96 0:16:23 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa12528; 11 Mar 96 0:08:52 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01803; 11 Mar 96 0:08:27 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa16148; 10 Mar 96 22:58:10 EST Received: from [131.202.3.20] by CS.CMU.EDU id aa06898; 10 Mar 96 22:57:26 EST Received: (from goldfarb@jupiter.sun.csd.unb.ca [131.202.3.10]) by unb.ca (8.7.4/960123-14:25) id XAA15869; Sun, 10 Mar 1996 23:57:13 -0400 (AST) Received: by jupiter.sun.csd.unb.ca (8.7.4/950215-16:05) id XAA10392; Sun, 10 Mar 1996 23:57:13 -0400 (AST) Date: Sun, 10 Mar 1996 23:57:12 -0400 (AST) From: Lev Goldfarb X-Sender: goldfarb@jupiter.sun.csd.unb.ca To: Connectionists@cs.cmu.edu Cc: Edelman Shimon Subject: Re: Shift Invariance In-Reply-To: <199603101701.RAA08020@lachesis.wisdom.weizmann.ac.il> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Sun, 10 Mar 1996, Edelman Shimon wrote: > Many thanks to Irv Biederman for posting the details of his findings, > along with a comparison with the results of Nazir & O'Regan. His > effort should reduce the chance of the readers of this list jumping to > premature conclusions. > > Note that the purpose of my previous posting was to advocate caution, > certainly not to argue that all claims of invariance are wrong. > Fortunately, my job in this matter is easy: just one example of a > manifest lack of invariance suffices to invalidate the strong version > of invariance-based theory of vision, which seems to be espoused by > Goldfarb: > > > If we 1) DO NOT FORGET that the biological systems have at their disposal > > quite adequate means to extract symbolic (structural) representation right > > from the very beginning and 2) FORGET about our inadequate numeric models, > > then the question would not have arisen in the first place. Symbolic > > representations EMBODY shift invariance. > > So, here it goes... Whereas invariance does hold in many recognition > tasks (in particular, in Biederman's experiments, as well as in the > experiments reported in [1]), it does not in others (as, e.g., in [2], > where interaction between size invariance and orientation is > reported). A recent comprehensive survey of (the far from invariant) > human performance in recognizing rotated objects can be found in > [3]. Furthermore, not only recognition, but also perceptual learning, > seems to be non-invariant in some cases; see [4,5]. > > FORGETTING about experimental findings will not make them go away, > just as pointing out that symbolic representations EMBODY invariance > will not make biological vision embrace a symbolic approach if it has > not done so until now. It appears that there is a considerable confusion as to what "shift invariance" is: shift invariance should not include size, orientation, or context invariance, since an encoding of these may involve additional structural information. (By the way, I do not read Biederman's message as Edelman does) -- Lev From obrad@sava.zfe.siemens.de Mon Mar 11 19:31:47 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 11 Mar 96 19:31:43 -0600; AA07892 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 11 Mar 96 19:31:41 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab13583; 11 Mar 96 17:57:11 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab13564; 11 Mar 96 17:25:22 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02367; 11 Mar 96 17:24:41 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id ah25613; 11 Mar 96 12:05:31 EST Received: from david.siemens.de by CS.CMU.EDU id aa10391; 11 Mar 96 10:40:17 EST Received: from ztivax.zfe.siemens.de (ztivax.siemens.de) by david.siemens.de with SMTP id AA08082 (5.67a/IDA-1.5 for ); Mon, 11 Mar 1996 16:40:04 +0100 Received: from sava.zfe.siemens.de (sava) by ztivax.zfe.siemens.de with SMTP id AA21849 (5.67a/IDA-1.5 for ); Mon, 11 Mar 1996 16:40:03 +0100 Received: (from obrad@localhost) by sava.zfe.siemens.de (8.6.12/8.6.12) id QAA20158; Mon, 11 Mar 1996 16:41:29 +0100 Date: Mon, 11 Mar 1996 16:41:29 +0100 From: Dragan Obradovic Message-Id: <199603111541.QAA20158@sava.zfe.siemens.de> To: Connectionists@cs.cmu.edu Subject: NEW BOOK ANNOUNCEMENT X-Sun-Charset: US-ASCII Cc: Gustavo.Deco@zfe.siemens.de, Dragan.Obradovic@zfe.siemens.de -------------------------------------------------------------------- NEW BOOK -- NEW BOOK -- NEW BOOK -- NEW BOOK -- NEW BOOK -- NEW BOOK -------------------------------------------------------------------- "An Information-Theoretic Approach to Neural Computing" -------------------------------------------------------- Gustavo Deco and Dragan Obradovic (Springer Verlag) Full details at: http://www.springer.de/springer-news/inf/inf_9602.new.html ISBN 0-387-94666-7 Summary: --------- Neural networks provide a powerful new technology to model and control nonlinear and complex systems. In this book, the authors present a detailed formulation of neural networks from the information-theoretic viewpoint. They show how this perspective provides new insights into the design theory of neural networks. In particular they show how these methods may be applied to the topics of supervised and unsupervised learning including feature extraction, linear and non-linear independent component analysis, and Boltzmann machines. Readers are assumed to have a basic understanding of neural networks, but all the relevant concepts from information theory are carefully introduced and explained. Consequently, readers from several different scientific disciplines, notably cognitive scientists, engineers, physicists, statisticians, and computer scientists, will find this to be a very valuable introduction to this topic. Contents: --------- Acknowledgments vi Foreword vii CHAPTER 1 Introduction 1 CHAPTER 2 Preliminaries of Information Theory and Neural Networks 7 Elements of Information Theory 8 Entropy and Information 8 Joint Entropy and Conditional Entropy 9 Kullback-Leibler Entropy 9 Mutual Information 10 Differential Entropy, Relative Entropy and Mutual Information 11 Chain Rules 13 Fundamental Information Theory Inequalities 15 Coding Theory 21 Elements of the Theory of Neural Networks 23 Neural Network Modeling 23 Neural Architectures 24 Learning Paradigms 27 Feedforward Networks: Backpropagation 28 Stochastic Recurrent Networks: Boltzmann Machine 31 Unsupervised Competitive Learning 35 Biological Learning Rules 36 PART I: Unsupervised Learning CHAPTER 3 Linear Feature Extraction: Infomax Principle 41 Principal Component Analysis: Statistical Approach 42 PCA and Diagonalization of the Covariance Matrix 42 PCA and Optimal Reconstruction 45 Neural Network Algorithms and PCA 51 Information Theoretic Approach: Infomax 57 Minimization of Information Loss Principle and Infomax Principle 58 Upper Bound of Information Loss 59 Information Capacity as a Lyapunov Function of the General Stochastic Approximation 61 CHAPTER 4 Independent Component Analysis: General Formulation and Linear Case 65 ICA-Definition 67 General Criteria for ICA 68 Cumulant Expansion Based Criterion for ICA 69 Mutual Information as Criterion for ICA 73 Linear ICA 79 Gaussian Input Distribution and Linear ICA 81 Networks With Anti-Symmetric Lateral Connections 84 Networks With Symmetric Lateral Connections 86 Examples of Learning with Symmetric and Anti-Symmetric Networks 89 Learning in Gaussian ICA with Rotation Matrices: PCA 91 Relationship Between PCA and ICA in Gaussian Input Case 93 Linear Gaussian ICA and the Output Dimension Reduction 94 Linear ICA in Arbitrary Input Distribution 95 Some Properties of Cumulants at the Output of a Linear Transformation 95 The Edgeworth Expansion Criteria and Theorem 4.6.2 99 Algorithms for Output Factorization in the Non-Gaussian Case 100 Experimental Results of Linear ICA Algorithms in the Non-Gaussian Case 102 CHAPTER 5 Nonlinear Feature Extraction: Boolean Stochastic Networks 109 Infomax Principle for Boltzmann Machines 110 Learning Model 110 Examples of Infomax Principle in Boltzmann Machine 113 Redundancy Minimization and Infomax for the Boltzmann Machine 119 Learning Model 119 Numerical Complexity of the Learning Rule 124 Factorial Learning Experiments 124 Receptive Fields Formation from a Retina 129 Appendix 132 CHAPTER 6 Nonlinear Feature Extraction: Deterministic Neural Networks 135 Redundancy Reduction by Triangular Volume Conserving Architectures 136 Networks with Linear, Sigmoidal and Higher Order Activation Functions 140 Simulations and Results 142 Unsupervised Modeling of Chaotic Time Series 146 Dynamical System Modeling 147 Redundancy Reduction by General Symplectic Architectures 156 General Entropy Preserving Nonlinear Maps 156 Optimizing a Parameterized Symplectic Map 157 Density Estimation and Novelty Detection 159 Example: Theory of Early Vision 163 Theoretical Background 164 Retina Model 165 PART II: Supervised Learning CHAPTER 7 Supervised Learning and Statistical Estimation 169 Statistical Parameter Estimation - Basic Definitions 171 Cramer-Rao Inequality for Unbiased Estimators 172 Maximum Likelihood Estimators 175 Maximum Likelihood and the Information Measure 176 Maximum A Posteriori Estimation 178 Extensions of MLE to Include Model Selection 179 Akaike's Information Theoretic Criterion (AIC) 179 Minimal Description Length and Stochastic Complexity 183 Generalization and Learning on the Same Data Set 185 CHAPTER 8 Statistical Physics Theory of Supervised Learning and Generalization 187 Statistical Mechanics Theory of Supervised Learning 188 Maximum Entropy Principle 189 Probability Inference with an Ensemble of Networks 192 Information Gain and Complexity Analysis 195 Learning with Higher Order Neural Networks 198 Partition Function Evaluation 198 Information Gain in Polynomial Networks 202 Numerical Experiments 203 Learning with General Feedforward Neural Networks 205 Partition Function Approximation 205 Numerical Experiments 207 Statistical Theory of Unsupervised and Supervised Factorial Learning 208 Statistical Theory of Unsupervised Factorial Learning 208 Duality Between Unsupervised and Maximum Likelihood Based Supervised Learning 213 CHAPTER 9 Composite Networks 219 Cooperation and Specialization in Composite Networks 220 Composite Models as Gaussian Mixtures 222 CHAPTER 10 Information Theory Based Regularizing Methods 225 Theoretical Framework 226 Network Complexity Regulation 226 Network Architecture and Learning Paradigm 227 Applications of the Mutual Information Based Penalty Term 231 Regularization in Stochastic Potts Neural Network 237 Neural Network Architecture 237 Simulations 239 References 243 Index 259 Ordering information: --------------------- ISBN 0-387-94666-7 US $49.95, DM 76 ------------------------------------------------------------ Dr. Gustavo Deco and Dr. Dragan Obradovic Siemens AG ZFE T SN 4 Corporate Research and Development Otto-Hahn-Ring 6 Phone: +49/89/636-49499 D-81739 Munich Fax: +49/89/636-49767 Germany E-Mail: Dragan.Obradovic@zfe.siemens.de Gustavo.Deco@zfe.siemens.de From tommi@psyche.mit.edu Mon Mar 11 19:31:48 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 11 Mar 96 19:31:46 -0600; AA07896 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 11 Mar 96 19:31:43 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ac13583; 11 Mar 96 17:58:50 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ac13564; 11 Mar 96 17:25:23 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02372; 11 Mar 96 17:24:55 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa28553; 11 Mar 96 14:29:08 EST Received: from PSYCHE.MIT.EDU by CS.CMU.EDU id aa12881; 11 Mar 96 14:28:13 EST Received: by psyche.mit.edu (4.1/DA1.0.4) id AA13475; Mon, 11 Mar 96 14:28:07 EST Date: Mon, 11 Mar 96 14:28:07 EST From: Tommi Jaakkola Message-Id: <9603111928.AA13475@psyche.mit.edu> To: connectionists@cs.cmu.edu Subject: Paper available: Upper and lower bounds on likelihoods The following paper is available on the web at http://web.mit.edu/~tommi/home.html ftp://psyche.mit.edu/pub/tommi/jaak-ul-bounds.ps.Z Computing upper and lower bounds on likelihoods in intractable networks T. S. Jaakkola and M. I. Jordan We present techniques for computing upper and lower bounds on the likelihoods of partial instantiations of variables in sigmoid and noisy-OR networks. The bounds determine confidence intervals for the desired likelihoods and become useful when the size of the network (or clique size) precludes exact computations. We illustrate the tightness of the obtained bounds by numerical experiments. -Tommi --------- The paper can be retrieved also via anonymous ftp: ftp-host: psyche.mit.edu ftp-file: pub/tommi/jaak-ul-bounds.ps.Z From goldfarb@unb.ca Mon Mar 11 19:31:50 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 11 Mar 96 19:31:48 -0600; AA07904 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 11 Mar 96 19:31:46 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ad13583; 11 Mar 96 18:00:19 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa13566; 11 Mar 96 17:26:13 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02379; 11 Mar 96 17:25:23 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa01375; 11 Mar 96 16:41:06 EST Received: from hermes.csd.unb.ca by EDRC.CMU.EDU id aa08418; 11 Mar 96 16:40:26 EST Received: (from goldfarb@jupiter.sun.csd.unb.ca [131.202.3.10]) by unb.ca (8.7.4/960123-14:25) id RAA24775 for ; Mon, 11 Mar 1996 17:40:23 -0400 (AST) Received: by jupiter.sun.csd.unb.ca (8.7.4/950215-16:05) id RAA16936; Mon, 11 Mar 1996 17:40:22 -0400 (AST) Date: Mon, 11 Mar 1996 17:40:21 -0400 (AST) From: Lev Goldfarb X-Sender: goldfarb@jupiter.sun.csd.unb.ca To: Connectionists@cs.cmu.edu Subject: Re: Shift Invariance In-Reply-To: <199603090225.SAA14592@mizar.usc.edu> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII I would like to make one more comment. Shift invariance should be properly thought of as invariance of a "final" object representation wrt TRANSLATIONS of the object (to use the term from linear algebra). This is not to be confused with the fact that the POSITION of the object is also encoded separately, when necessary. The latter has to do with the need to represent the entire "scene". -- Lev From maja@garnet.cs.brandeis.edu Tue Mar 12 15:53:24 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 12 Mar 96 15:53:06 -0600; AA26811 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 12 Mar 96 15:52:51 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa14052; 12 Mar 96 1:29:31 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa14046; 12 Mar 96 1:13:02 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02660; 12 Mar 96 1:11:59 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id ab07594; 11 Mar 96 22:36:11 EST Received: from garnet.cs.brandeis.edu by CS.CMU.EDU id aa17019; 11 Mar 96 22:35:33 EST Received: by garnet.cs.brandeis.edu (940816.SGI.8.6.9/940406.SGI) id WAA05729; Mon, 11 Mar 1996 22:35:36 -0500 Date: Mon, 11 Mar 1996 22:35:36 -0500 Message-Id: <199603120335.WAA05729@garnet.cs.brandeis.edu> From: Maja Mataric To: connectionists@cs.cmu.edu Subject: AAAI Fall Symposium on Embodied Cognition and Action !! PLEASE POST !! PLEASE POST !! PLEASE POST !! PLEASE POST !! PLEASE POST !! Call For Participation AAAI 1996 Fall Symposium on Embodied Cognition and Action ---------------------------------------------------------- to be held at MIT Nov 9-11, 1996 Submission Deadline: April 15, 1996. The role of physical embodiment in cognition has long been the subject of debate. It is largely accepted in AI that embodiment has strong implications on the control strategies for generating purposive and intelligent behavior in the world. Some theories have proposed that embodiment not only constrains but may also facilitate certain types of higher-level cognition. Evidence from neuroscience allows for postulating shared mechanisms for low-level control of embodied action (e.g., motor plans for limb movement) and higher-level cognition (e.g., abstract plans). Work in animal behavior has also addressed the potential links between the two systems and linguistic theories have long recognized the role of physical and spatial metaphors in language. The symposium will study the role of embodiment in both scaling up control and grounding cognition. We will explore ways of extending the existing typically low-level sub-cognitive systems such as autonomous robots and agents, as well as grounding more abstract typically disembodied cognitive systems. We will draw from AI, ethology, neuroscience, and other sources in order to focus on the implications of embodiment in cognition and action, and explore work that has been done in the areas of applying physical metaphors to more abstract higher-level cognition. Topics and questions of interest include: * What spatial metaphors that can be used for abstract/higher-level cognition? * What non-spatial metaphors can be applied in higher-level cognition? * What alternatives to symbolic representations (e.g., analogical, procedural, etc.) can be successfully employed in embodied cognition? * How can evidence from neuroscience and ethology benefit work in synthetic embodied cognition and embodied AI? Can we gain more than just inspiration from biological data in this area? Are there specific constraints and/or mechanisms we can usefully model? * (How) Do methods for modeling embodied insect and animal behavior scale up to higher-level cognition? * How do metaphors from embodiment apply to everyday activity? * What computational and representational structures are necessary and/or sufficient for enabling embodied cognition? * What are some successfully implemented embodied cognition systems? The symposium will focus on group discussions and panels with a few inspiring presentations and overviews of relevant work. Organizing committee: --------------------- Dana Ballard, University of Rochester, dana@cs.rochester.edu; Rod Brooks, MIT, brooks@ai.mit.edu; Daniel Dennett, Tufts University, ddennett@pearl.tufts.edu; Simon Giszter, Medical College of Pennsylvania, simon@SwampThing.medcolpa.edu; Maja Mataric (chair), Brandeis University, maja@cs.brandeis.edu; Erich Prem, Austrian AI Institute, erich@ai.univie.ac.at; Terence Sanger, MIT, tds@ai.mit.edu; Stefan Schaal, Georgia Tech, sschaal@cc.gatech.edu; Submission Information: ----------------------- We invite the participation of researchers who have been working on embodied cognition and action in the fields of AI, neuroscience, ethology, and robotics. Prospective participants should submit a brief paper (5 pages or less) or an extended abstract describing their research or interests. Papers should be submitted electronically, in postscript or plain text format, via ftp to ftp.cs.brandeis.edu/pub/faculty/maja/aaai96-fs/. Participants will have an opportunity to contribute to the final working notes. Detailed ftp instructions: -------------------------- compress your-paper (both Unix compress and gzip commands are ok) ftp ftp.cs.brandeis.edu (129.64.2.5, but check in case it has changed) give anonymous as your login name give your e-mail address as password set transmission to binary (just type the command BINARY) cd to /aaai96-fs put your-paper Relevant Dates: --------------- Apr 15, 1996: Submissions due May 17, 1996: Notification of acceptance given Aug 23, 1996: Material for inclusion into the working notes due Nov 9-11, 96: AAAI Fall Symposium The WWW home page for this symposium can be found at: http://www.cs.brandeis.edu/~maja/aaai96-fs/ From murase@synapse.fuis.fukui-u.ac.jp Tue Mar 12 15:53:25 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 12 Mar 96 15:53:05 -0600; AA26809 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 12 Mar 96 15:52:49 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa14044; 12 Mar 96 1:21:13 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa14041; 12 Mar 96 1:12:08 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02654; 12 Mar 96 1:11:24 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id ab05799; 11 Mar 96 20:31:51 EST Received: from icpc.icpc.fukui-u.ac.jp by CS.CMU.EDU id aa16148; 11 Mar 96 20:30:42 EST Received: from synapse.fuis.fukui-u.ac.jp (murase@synapse.fuis.fukui-u.ac.jp [133.7.61.1]) by icpc.icpc.fukui-u.ac.jp (8.6.2+2.2W/3.1W) with ESMTP id KAA16844 for ; Tue, 12 Mar 1996 10:14:15 +0900 Received: (from murase@localhost) by synapse.fuis.fukui-u.ac.jp (8.6.12+2.5Wb7/3.4W2) id KAA07787 for Connectionists@CS.CMU.EDU; Tue, 12 Mar 1996 10:31:02 +0900 Date: Tue, 12 Mar 1996 10:31:02 +0900 From: Kazuyuki Murase Message-Id: <199603120131.KAA07787@synapse.fuis.fukui-u.ac.jp> To: -v@synapse.fuis.fukui-u.ac.jp, Connectionists@cs.cmu.edu Subject: Associate Professor Position in Japan ASSOCIATE PROFESSOR IN BIOLOGICAL INFORMATION PROCESSING IN JAPAN The department of Information Science at Fukui University invites applications for an associate professor position in its Biological Information Processing Division starting October 1996. The position requires Ph.D. with postdoctoral research experience. Theaching and supervision of undergraduate and graduate research projects are essential. The ability of Japanese language is not required initially, but should be developed within a few years. The cadidates with specific expertise in at least one of the following areas will be given higher priority: Electrophysiology of single cells or cellular networks, Sensory mechanisms of the spinal cord, Optical imaging of neuronal activities, Modeling of excitable cells or cellular networks, Artificial neural networks, Simulation and synthesis of biological behavior. Applicants should send a curriculum vitae including a publication list and brief description of future research plans by mail to Dr. Kazuyuki Murase, Department of Information Science, Fukui University, 3-9-1 Bunkyo, Fukui 910, Japan, or by E-mail to murase@synapse.fuis.fukui-u.ac.jp. Review of applications will begin immediately and continue until the position is filled. Fukui University is one of the Japanese National Universities. From robtag@dia.unisa.it Tue Mar 12 15:53:26 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 12 Mar 96 15:53:02 -0600; AA26810 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 12 Mar 96 15:52:42 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa13562; 11 Mar 96 17:46:37 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa13560; 11 Mar 96 17:24:29 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02356; 11 Mar 96 17:23:48 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa21505; 11 Mar 96 7:16:32 EST Received: from udsab.dia.unisa.it by CS.CMU.EDU id aa09159; 11 Mar 96 7:16:07 EST Received: by udsab.dia.unisa.it; (5.65v3.2/1.1.8.2/15Nov95-1017AM) id AA24178; Mon, 11 Mar 1996 13:11:41 +0100 Date: Mon, 11 Mar 1996 13:11:41 +0100 From: Tagliaferri Roberto Message-Id: <9603111211.AA24178@udsab.dia.unisa.it> To: Connectionists@cs.cmu.edu Subject: International School on Neural Nets "E.R. Caianiello" Galileo Galilei Foundation World Federation of Scientists Ettore Maiorana Centre for Scientific Culture Galilelo Galilei Celebrations Four Hundreds Years Since the Birth of Modern Science International School on Neural Nets "E.R. Caianiello" 1st Course: Learning in Graphical Models A NATO Advanced Study Institute Erice-Sicily: 27 September - 7 October 1996 Sponsored by the: - European Union - International Institute for Advanced Scientific Studies (IIASS) - Italian Institute for Philosophical Studies - Italian Ministry of Education - Italian Ministry of University and Scientific Research - Italian National Research Institute (CNR) - Sicilian Regional Government - University of Salerno Programme and Lecturers - Introduction to Graphical Models J. Whittaker, University of Lancaster, UK - Introduction to Bayesian Methods D. Mackay, University of Cambridge, UK - Introduction to Neural Networks M. Jordan, MIT, Cambridge, MA, USA - Learning of Directed Graphs D. Heckerman, Microsoft Research, Redmond, WA, USA - The Helmholz Machine G. Hinton, University of Toronto, Canada - Model Selection G. Cooper, University of Pittsburg, PA, USA - Latent Variables Methods R. Neal, University of Toronto, Canada - Stochastic Grammars S. Omohundro, NEC Research, Princeton, NJ, USA - Statistical Mechanics and Clustering Models J. Buhmann, University of Bonn, Germany - Bayesian Learning of Graphical Models R. Cowell, University College, London, UK - Priors for Graphical Models D. Geiger, UCLA, Los Angeles, CA, USA - Independence and Decorrelation E. Oja, Helsinki University of Technology, Finland - Bayesian Learning and Gibbs Sampling D. Spiegelhalter, MRC, Cambridge, UK Purpose of the course Neural Networks and Bayesian belief networks are learning and interface methods that have been developed in two largely distinct reasearch communities. The purpose of this Course is to bring together researchers from these two communities and study both kinds of networks as istances of a general unified graphical formalism. The Course will focus on probabilistic methods for learning in graphical models, with attention paid to algorithm analysis and design, theory and applications. General Information Persons wishing to attend the Course should apply in writing to: - Prof. Maria Marinaro IIASS "E.R. Caianiello" Via G. Pellegrino, 19 84019 Vietri sul mare (SA), Italy Tel: + 39 89 761167 Fax: + 39 89 761189 They should specify: i) date and place of birth together with present nationality; ii) degree and other academic qualifications; iii) present position and place of work. Young persons with only little experience should include a letter of recommendation from the head of their research group or from a senior scientist active in the field. The total fee, which includes full board and lodging (arranged by the School), is $1000 USD. Thanks to the generosity of the sponsoring Institutions, partial support can be granted to some deserving students who need financial help. Requests to this effect must be specified and justified in the application letter. Closing date for application: July 15, 1996 No special application form is required. Admission to the Course will be decided in consultation with the Advisory Committee of the School consisting of Professors D. Hecherman, M.I. Jordan, M. Marinaro and A. Zichichi. It is regretted that it will not be possible to allow any person not selected by the Committee of the School to follow the Course. Participants must arrive in Erice on September 27, no later than 5 p.m. More information about this Course and the other activities of the Ettore Majorana Centre can be found on the WWW at the following address: http://www.ccsem.infn.it D. Heckerman - M.I. Jordan Directors of the Course M.I. Jordan - M. Marinaro Directors of the School A. Zichichi Director of the Centre From rjb@psy.ox.ac.uk Tue Mar 12 15:53:27 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 12 Mar 96 15:53:03 -0600; AA26808 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 12 Mar 96 15:52:44 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa13583; 11 Mar 96 17:55:29 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa13564; 11 Mar 96 17:25:19 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02362; 11 Mar 96 17:24:25 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa21837; 11 Mar 96 8:02:35 EST Received: from axp01.mrc-bbc.ox.ac.uk by EDRC.CMU.EDU id aa05742; 11 Mar 96 8:01:21 EST Received: from axp02.mrc-bbc.ox.ac.uk (axp02.mrc-bbc.ox.ac.uk [163.1.29.192]) by axp01.mrc-bbc.ox.ac.uk (8.7.3/8.7.3) with ESMTP id NAA15374 for ; Mon, 11 Mar 1996 13:00:46 GMT Received: (rjb@localhost) by axp02.mrc-bbc.ox.ac.uk (8.6.11/8.6.12) id NAA05904; Mon, 11 Mar 1996 13:00:44 GMT Date: Mon, 11 Mar 1996 13:00:44 GMT Message-Id: <199603111300.NAA05904@axp02.mrc-bbc.ox.ac.uk> From: Roland Baddeley To: Connectionists@cs.cmu.edu Subject: Position available in computational neuroscience The following Jobs may be of interest to readers of the connectionists mailing list. UNIVERSITY OF OXFORD DEPARTMENT OF EXPERIMENTAL PSYCHOLOGY Posts in Computational Neuroscience and Visual Neurophysiology The following posts are available as part of a long-term research programme combining neurophysiological and computational approaches to brain mechanisms of vision and memory (see Rolls, 1995, Behav. Brain Res. 66: 177-185; or Rolls, 1994, Behav. Processes 33: 113-138): (1) Computational neuroscientist to make formal models and/or analyse by simulation the functions of visual cortical areas in invariant recognition. (2) Neurophysiologist (preferably postdoctoral) to analyse the activity of single neurons in the temporal cortical visual areas. The salaries are on the RS1A (postdoctoral) scale 14,317- 21,519 pounds, with support provided by a Programme Grant. Applications including the names of two referees, or enquiries, to Dr. Edmund T. Rolls, University of Oxford, Department of Experimental Psychology, South Parks Road, Oxford OX1 3UD, England (email Edmund.Rolls@psy.ox.ac.uk). The University exists to promote excellence in education and research. The University is an Equal Opportunity Employer. From mlittman@cs.brown.edu Wed Mar 13 01:44:54 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 13 Mar 96 01:44:44 -0600; AA05567 Received: from bilby.cs.uwa.oz.au by lucy.cs.wisc.edu; Wed, 13 Mar 96 01:44:31 -0600 Received: from (mafm@parma.cs.uwa.oz.au [130.95.1.7]) by cs.uwa.oz.au (8.6.8/8.5) with SMTP id MAA28113; Wed, 13 Mar 1996 12:51:05 +0800 Message-Id: <199603130451.MAA28113@cs.uwa.oz.au> From: mlittman@cs.brown.edu To: reinforce@cs.uwa.edu.au Subject: Thesis available Date: Tue, 12 Mar 1996 13:50:58 -0500 (EST) Friends and Colleagues, I'm pleased to announce the completion of my graduate training and the availablity of my Ph.D. dissertation. -Michael -------------------------------------------------- Title: Algorithms for Sequential Decision Making Author: Michael Lederman Littman Online: http://www.cs.brown.edu/people/mll/docs/thesis.ps (2.9M) Compressed: ftp://ftp.cs.brown.edu/pub/techreports/96/cs96-09.ps.Z (1.1M) Length: 281 pages For people not interested in the technical details, the abstract, acknowledgments, and introduction are available as a separate, smaller document. Online: http://www.cs.brown.edu/people/mll/docs/thesis-mini.ps (0.6M) Length: 30 pages Abstract: Sequential decision making is a fundamental task faced by any intelligent agent in an extended interaction with its environment; it is the act of answering the question ``What should I do now?'' In this thesis, I show how to answer this question when ``now'' is one of a finite set of states, ``do'' is one of a finite set of actions, ``should'' is maximize a long-run measure of reward, and ``I'' is an automated planning or learning system (agent). In particular, I collect basic results concerning methods for finding optimal (or near-optimal) behavior in several different kinds of model environments: Markov decision processes, in which the agent always knows its state; partially observable Markov decision processes (POMDPs), in which the agent must piece together its state on the basis of observations it makes; and Markov games, in which the agent is in direct competition with an opponent. The thesis is written from a computer-science perspective, meaning that many mathematical details are not discussed, and descriptions of algorithms and the complexity of problems are emphasized. New results include an improved algorithm for solving POMDPs exactly over finite horizons, a method for learning minimax-optimal policies for Markov games, a pseudopolynomial bound for policy iteration, and a complete complexity theory for finding zero-reward POMDP policies. From J.Heemskerk@dcs.shef.ac.uk Wed Mar 13 01:52:56 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 13 Mar 96 01:52:54 -0600; AA05694 Received: from bilby.cs.uwa.oz.au by lucy.cs.wisc.edu; Wed, 13 Mar 96 01:52:48 -0600 Received: from (mafm@parma.cs.uwa.oz.au [130.95.1.7]) by cs.uwa.oz.au (8.6.8/8.5) with SMTP id MAA28095; Wed, 13 Mar 1996 12:49:34 +0800 Message-Id: <199603130449.MAA28095@cs.uwa.oz.au> From: Jan Heemskerk To: reinforce@cs.uwa.oz.au Subject: CALL FOR PARTICIPATION [connectionists] Date: Tue, 12 Mar 96 10:30:16 GMT CALL FOR PARTICIPATION ** LEARNING IN ROBOTS AND ANIMALS ** An AISB-96 two-day workshop University of Sussex, Brighton, UK: April, 1st & 2nd, 1996 Co-Sponsored by IEE Professional Group C4 (Artificial Intelligence) WORKSHOP ORGANISERS: Noel Sharkey (chair), University of Sheffield, UK. Gillian Hayes, University of Edinburgh, UK. Jan Heemskerk, University of Sheffield, UK. Tony Prescott, University of Sheffield, UK. PROGRAMME COMMITTEE: Dave Cliff, UK. Marco Dorigo, Italy. Frans Groen, Netherlands. John Hallam, UK. John Mayhew, UK. Martin Nillson, Sweden Claude Touzet, France Barbara Webb, UK. Uwe Zimmer, Germany. Maja Mataric, USA. In the last five years there has been an explosion of research on Neural Networks and Robotics from both a self-learning and an evolutionary perspective. Within this movement there is also a growing interest in natural adaptive systems as a source of ideas for the design of robots, while robots are beginning to be seen as an effective means of evaluating theories of animal learning and behaviour. A fascinating interchange of ideas has begun between a number of hitherto disparate areas of research and a shared science of adaptive autonomous agents is emerging. This two-day workshop proposes to bring together an international group to both present papers of their most recent research, and to discuss the direction of this emerging field. PROVISION LIST OF PAPERS: Robot Shaping - Priniciples, Methods & Architectures Simon Perkins and Gillian Hayes Towards Autonomous Control using Connectionist 'Infinite State Automata' Tom Ziemke Entropy-based Tradeoff between Exploration and Exploitation Ping Zhang and Stephane Canu Evolving a Hierarchical Control System for Co-operating Autonomous Robots Robert Ghanea-Hercock & David P Barnes Evolutionary Learning of task achieving behaviours Myra S Wilson, Clive King and John E Hunt The design of learning for an artifact Joanna Bryson Robot See, Robot Do: An Overview of Robot Imitation Paul Bakker and Yasuo Kuniyoshi Does Dynamics Solve the Symbol Grounding Problem of Robots? An Experiment in Navigation Learning Jun Tani Abstracting Fuzzy Behavioural Rules From Geometric Models in Mobile Robotics A G Pipe, Tc Fogarty and A Winfield Brave Mobots Use Representation Chris Thornton Explore/Exploit Strategies in Autonomous Learning Stewart W Wilson Environment memory for a mobile robot using place cells Ken Harris, David Lee and Michael Recce Representations on a mobile robot Noel Sharkey and Jan Heemskerk Layered control architectures in natural and artificial systems Tony J Prescott REGISTRATION INFORMATION: http://www.cogs.susx.ac.uk:80/users/christ/aisb/aisb96/index.html ftp ftp.cogs.susx.ac.uk From omlinc@cs.rpi.edu Wed Mar 13 16:29:51 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 13 Mar 96 16:29:47 -0600; AA20832 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 13 Mar 96 16:29:44 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa15255; 12 Mar 96 20:24:35 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa15244; 12 Mar 96 20:01:26 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa03373; 12 Mar 96 20:00:30 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa15618; 12 Mar 96 9:53:53 EST Received: from cs.rpi.edu by EDRC.CMU.EDU id aa11966; 12 Mar 96 9:53:40 EST Received: from colossus.cs.rpi.edu by cs.rpi.edu (5.67a/1.4-RPI-CS-Dept) id AA22549; Tue, 12 Mar 1996 09:54:18 -0500 (omlinc from colossus.cs.rpi.edu) Date: Tue, 12 Mar 96 09:54:08 EST From: omlinc@cs.rpi.edu Received: by colossus.cs.rpi.edu (4.1/2.3-RPI-CS-client) id AA17032; Tue, 12 Mar 96 09:54:08 EST Message-Id: <9603121454.AA17032@colossus.cs.rpi.edu> To: connectionists@cs.cmu.edu Subject: Re: Shift Invariance In his message <9602281000.ZM15421@ICSI.Berkeley.edu>, Jerry Feldman wrote: >3) Shift invariance in time and recurrent networks. > > I threw in some (even more cryptic) comments on this anticipating that some >readers would morph the original task into this form. The 0*1010* problem is >an easy one for FSA induction and many simple techniques might work for this. >But consider a task that is only slightly more general, and much more natural. >Suppose the task is to learn any FSL from the class b*pb* where b and p are >fixed for each case and might overlap. Any learning technique that just >tried to predict (the probability of) successors will fail because there >are three distinct regimes and the learning algorithm needs to learn this. >I don't have a way to characterize all recurrent net learning algorithms to >show that they can't do this and it will be interesting to see if one can. >There are a variety on non-connectionist FSA induction methods that can >effectively learn such languages, but they all depend on some overall measure >of simplicity of the machine and its fit to the data - and are thus non-local. > This isn't really correct. First, any DFA can be represented in recurrent neural networks with sigmoidal discriminants functions, i.e. a network can be constructed such that the languages recognized by a DFA and its network implementation are identical (this implies stability of the internal DFA state representation for strings of arbitary length) [1,2]. As far as learning DFA's with recurrent networks is concerned: In my experience, success of failure of a network to learn a particular grammar depends on the size of the DFA, its complexity (simple self loops as opposed to orbits of arbitary length), the training data, and the order in which the training data is concerned. For instance, we found that incremental learning where the network is first trained on the shortest strings of data [8] is often crucial to successful convergence since it is a means to overcome the problem of learning long-term dependencies with gradient descent [4] (for methods for overcoming that problem see [5,6,7]). The `simplest' language of the form b*pb* might be 1*01*. A network with second-order weights and a single recurrent state neuron can learn that language within 100 epochs when trained on the first 100 strings in alphabetical order. Furthermore, the ideal DFA can also be extracted from the trained network [3]. See for example [9,10,11,12] for other extraction approaches. For the language 1*011* which is of the form b*pb* (notice overlapping of b and p), a second-order network with 3 recurrent state neurons easily converged within 200 epochs and the ideal DFA can be extracted as well. So, here are at least two examples which contradict the claim that "Any learning technique that just tried to predict (the probability of) successors will fail because there are three distinct regimes and the learning algorithm needs to learn this. I don't have a way to characterize all recurrent net learning algorithms to show that they can't do this and it will be interesting to see if one can." Christian ------------------------------------------------------------------- Christian W. Omlin, Ph.D. Phone (609) 951-2691 NEC Research Institute Fax: (609) 951-2438 4 Independence Way E-mail: omlinc@research.nj.nec.com Princeton, NJ 08540 omlinc@cs.rpi.edu URL: http://www.neci.nj.nec.com/homepages/omlin/omlin.html ------------------------------------------------------------------- =================================== Bibliography ======================================= [1] P. Frasconi, M. Gori, M. Maggini, G. Soda, "Representation of Finite State Automata in Recurrent Radial Basis Function Networks", Machine Learning, to be published, 1996. [2] C.W. Omlin, C.L. Giles, "Stable Encoding of Large Finite-State Automata in Recurrent Neural Networks with Sigmoid Discriminants", Neural Computation, to be published, 1996. [3] C.W. Omlin, C.L. Giles, "Extraction of Rules from Discrete-Time Recurrent Neural Networks", Neural Networks , Vol. 9, No. 1, p. 41-52, 1996. [4] Y. Bengio, P. Simard, P. Frasconi, "Learning Long-Term Dependencies with Gradient Descent is Difficult", IEEE Transactions on Neural Networks (Special Issue on Recurrent Neural Networks), Vol. 5, p. 157-166, 1994. [5] T. Lin, B.G. Horne, P. Tino, C.L. Giles, "Learning Long-Term Dependencies with NARX Recurrent Neural Networks, IEEE Transactions on Neural Networks, accepted for publication. [6] S. El Hihi, Y. Bengio, "Hierarchical Recurrent Neural Networks for Long-Term Dependencies", Neural Information Processing Systems 8, MIT Press, 1996. [7] S. Hochreiter, J. Schmidhuber, "Long Short Term Memory", Technical Report, Institut fuer Informatik, Technische Universitaet Muenchen, FKI-207-95, 1995. [8] J.L. Elman, "Incremental Learning, or the Importance of Starting Small" Technical Report, Center for Research in Language, University of California at San Diego, CRL Tech Report 9101, 1991. [9] S. Das, M.C. Mozer, "A Unified Gradient-descent/Clustering Architecture for Finite State for Finite State Machine Induction", Advances in Neural Information Processing Systems 6, J.D. Cowan , G. Tesauro, J. Alspector (Eds.), p. 19-26, 1994. [10] M.P. Casey, "Computation in Discrete-Time Dynamical Systems", Ph.D. Thesis, Department of Mathematics, University of California, San Diego, 1995. [11] P. Tino, J. Sajda, "Learning and Extracting Initial Mealy Machines With a Modular Neural Network Model}", Neural Computation, Vol. 7, No. 4, p. 822-844, 1995. [12] R.L. Watrous, G.M. Kuhn, "Induction of Finite-State Languages Using Second-Order Recurrent Networks", Neural Computation, Vol. 4, No. 5, p. 406, 1992. From heckerma@MICROSOFT.com Wed Mar 13 16:29:54 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 13 Mar 96 16:29:49 -0600; AA20834 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 13 Mar 96 16:29:46 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab15255; 12 Mar 96 20:25:56 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab15244; 12 Mar 96 20:01:29 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa03378; 12 Mar 96 20:00:43 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa24831; 12 Mar 96 18:05:49 EST Received: from red-07-imc.itg.microsoft.com by CS.CMU.EDU id aa25482; 12 Mar 96 18:05:10 EST Received: by red-07-imc.itg.microsoft.com with Microsoft Exchange (IMC 4.0.829.1) id <01BB1025.455A97E0@red-07-imc.itg.microsoft.com>; Tue, 12 Mar 1996 15:04:54 -0800 Message-Id: From: David Heckerman To: "'Connectionists@CS.CMU.EDU'" Subject: paper available: Efficient Approximations for the Marginal Likelihood... Date: Tue, 12 Mar 1996 15:05:00 -0800 X-Mailer: Microsoft Exchange Server Internet Mail Connector Version 4.0.829.1 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable The following paper is available on the web at http://www.research.microsoft.com/research/dtg/heckerma/heckerma.html Efficient Approximations for the Marginal Likelihood of Incomplete Data Given a Bayesian Network D. Chickering and D. Heckerman MSR-TR-96-08 A Bayesian score often used in model selection is the marginal likelihood of data (or "evidence") given a model. We examine asymptotic approximations for the marginal likelihood of incomplete data given a Bayesian network. We consider the well-known Laplace and BIC/MDL approximations, as well as approximations proposed by Draper (1993) and Cheeseman and Stutz (1995). In experiments using synthetic data generated from discrete naive-Bayes models having a hidden root node, we find the Cheeseman-Stutz measure to be the best in that it is as accurate as the Laplace approximation and as efficient as the BIC/MDL approximation. The paper also can be retrieved via anonymous ftp: ftp-host: ftp.research.microsoft.com ftp-file: pub/tech-reports/winter95-96/tr-96-08.ps -David =00 From jordan@psyche.mit.edu Wed Mar 13 16:29:55 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 13 Mar 96 16:29:52 -0600; AA20848 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 13 Mar 96 16:29:48 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ac15255; 12 Mar 96 20:27:19 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ac15244; 12 Mar 96 20:01:30 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa03383; 12 Mar 96 20:01:02 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa26033; 12 Mar 96 19:37:12 EST Received: from PSYCHE.MIT.EDU by EDRC.CMU.EDU id aa14991; 12 Mar 96 19:36:56 EST Received: by psyche.mit.edu (4.1/DA1.0.4) id AA25557; Tue, 12 Mar 96 19:36:50 EST Date: Tue, 12 Mar 96 19:36:49 EST From: Michael Jordan To: connectionists@cs.cmu.edu Subject: International School on Neural Nets "E.R. Caianiello" Message-Id: The enclosed is a correction and amplification to the earlier message regarding next fall's ``Learning in Graphical Models'' Advanced Study Institute in Erice, Sicily. Mike Jordan ------------------------------------------------------------------------ Galileo Galilei Foundation World Federation of Scientists Ettore Maiorana Centre for Scientific Culture Galilelo Galilei Celebrations Four Hundreds Years Since the Birth of Modern Science International School on Neural Nets ``E.R. Caianiello'' 1st Course: Learning in Graphical Models A NATO Advanced Study Institute Erice-Sicily: 27 September - 7 October 1996 Sponsored by the: - European Union - International Institute for Advanced Scientific Studies (IIASS) - Italian Institute for Philosophical Studies - Italian Ministry of Education - Italian Ministry of University and Scientific Research - Italian National Research Institute (CNR) - Sicilian Regional Government - University of Salerno Lecturers will include: J. Whittaker, University of Lancaster, UK D. Madigan, University of Washington, USA D. Geiger, Technion, Israel U. Kjaerullf, Aalborg University, Denmark R. Cowell, University College, London, UK M. Studeny, Academy of Sciences, Czech Republic M. Jordan, MIT, USA S. Omohundro, NEC Research, USA D. Heckerman, Microsoft Research, USA G. Cooper, University of Pittsburg, USA W. Buntine, Thinkbank, USA L. Saul, MIT, USA J. Buhmann, University of Bonn, Germany N. Tishby, Hebrew University, Israel D. Mackay, University of Cambridge, UK D. Spiegelhalter, MRC, Cambridge, UK J. Pearl, UCLA, USA Topics will include: Introduction to graphical models (directed and undirected graphs) Inference (probabilistic propagation, junction trees, conditioning) Properties of conditional independence (Markov properties, separation) Chain graphs Mixture models, hidden Markov models Neural networks Data structures for efficient estimation (bump trees, ball trees) Bayesian methods Structure learning (metrics, search, approximations) Priors Statistical mechanical methods (decimation, mean field) Markov chain Monte Carlo (importance sampling, Gibbs sampling, hybrid MC) Bayesian graphical models (BUGS software) Learning and phase transitions Clustering and multidimensional scaling Model selection and averaging Surface learning and family discovery Online learning Causality Purpose of the course Neural networks and Bayesian belief networks are learning and inference methods that have been developed in two largely distinct reasearch communities. The purpose of this Course is to bring together researchers from these two communities and study both kinds of networks as instances of a general unified graphical formalism. The Course will focus on probabilistic methods for learning in graphical models, with attention paid to algorithm analysis and design, theory and applications. General Information Persons wishing to attend the Course should apply in writing to: - Prof. Maria Marinaro IIASS "E.R. Caianiello" Via G. Pellegrino, 19 84019 Vietri sul mare (SA), Italy Tel: + 39 89 761167 Fax: + 39 89 761189 They should specify: i) date and place of birth together with present nationality; ii) degree and other academic qualifications; iii) present position and place of work. Young persons with only little experience should include a letter of recommendation from the head of their research group or from a senior scientist active in the field. The total fee, which includes full board and lodging (arranged by the School), is $1000 USD. Thanks to the generosity of the sponsoring Institutions, partial support can be granted to some deserving students who need financial help. Requests to this effect must be specified and justified in the application letter. Closing date for application: July 15, 1996 No special application form is required. Admission to the Course will be decided in consultation with the Advisory Committee of the School consisting of Professors D. Heckerman, M.I. Jordan, M. Marinaro and A. Zichichi. It is regretted that it will not be possible to allow any person not selected by the Committee of the School to follow the Course. Participants must arrive in Erice on September 27, no later than 5 p.m. More information about this Course and the other activities of the Ettore Majorana Centre can be found on the WWW at the following address: http://www.ccsem.infn.it D. Heckerman - M.I. Jordan - J. Whittaker Directors of the Course M.I. Jordan - M. Marinaro Directors of the School A. Zichichi Director of the Centre From wermter@nats5.informatik.uni-hamburg.de Thu Mar 14 16:03:12 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 14 Mar 96 16:03:09 -0600; AA14428 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 14 Mar 96 16:03:04 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa16986; 13 Mar 96 20:15:42 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa16981; 13 Mar 96 20:00:01 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa04532; 13 Mar 96 19:59:35 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa07605; 13 Mar 96 10:44:39 EST Received: from nats5.informatik.uni-hamburg.de by EDRC.CMU.EDU id aa17822; 13 Mar 96 10:43:10 EST Received: from nats13.informatik.uni-hamburg.de (nats13.informatik.uni-hamburg.de [134.100.10.113]) by nats5.informatik.uni-hamburg.de (8.6.10/8.6.10) with ESMTP id QAA08881; Wed, 13 Mar 1996 16:42:23 +0100 From: Stefan Wermter Received: (wermter@localhost) by nats13.informatik.uni-hamburg.de (8.6.10/8.6.10) id QAA25501; Wed, 13 Mar 1996 16:42:21 +0100 Date: Wed, 13 Mar 1996 16:42:21 +0100 Message-Id: <199603131542.QAA25501@nats13.informatik.uni-hamburg.de> To: connectionists@cs.cmu.edu Subject: book on language learning: connectionist statistical symbolic approaches Reply-To: wermter@informatik.uni-hamburg.de [I am posting this to several relevant mailing lists -- apologies to those who, by subscribing to multiple lists, receive multiple copies of this announcement.] BOOK ANNOUNCEMENT ----------------- Title: Connectionist, statistical, and symbolic approaches to learning for natural language processing Editors: Stefan Wermter Ellen Riloff Gabriele Scheler Date: March 1996 (first week in Europe [order information and WWW reference for the book (access to first chapter) at the end of this message] Brief description ----------------- The purpose of this book is to present a collection of papers that represents a broad spectrum of current research in learning methods for natural language processing, and to advance the state of the art in language learning and artificial intelligence. The book should bridge a gap between several areas that are usually discussed separately, including connectionist, statistical, and symbolic methods. Table of contents ----------------- Introduction: Learning approaches for natural language processing S. Wermter, E. Riloff, G. Scheler Part 1: Connectionist Networks and Hybrid Approaches ---------------------------------------------------- Separating learning and representation N.E. Sharkey, A.J.C. Sharkey Natural language grammatical inference: a comparison of recurrent neural networks and machine learning methods S. Lawrence, S. Fong, C. L. Giles Extracting rules for grammar recognition from Cascade-2 networks R. Hayward, A. Tickle, J. Diederich Generating English plural determiners from semantic representations: a neural network learning approach G. Scheler Knowledge acquisition in concept and document spaces by using self-organizing neural networks W. Winiwarter, E. Schweighofer, D. Merkl Using hybrid connectionist learning for speech/language analysis V. Weber, S. Wermter SKOPE: A connectionist/symbolic architecture of spoken Korean processing G. Lee, J.-H. Lee Integrating different learning approaches into a multilingual spoken language translation system P. Geutner, B. Suhm, F.-D. Buo, T. Kemp, L. Mayfield, A. E. McNair, I. Rogina, T. Schultz, T. Sloboda, W. Ward, M. Woszczyna, A. Waibel Learning language using genetic algorithms T. C. Smith, I. H. Witten Part 2: Statistical Approaches --------------------------------------------------- A statistical syntactic disambiguation program and what it learns M. Ersan, E. Charniak Training stochastic grammars on semantical categories W.R. Hogenhout, Y. Matsumoto Learning restricted probabilistic link grammars E. W. Fong, D. Wu Learning PP attachment from corpus statistics A. Franz A minimum description length approach to grammar inference P. Gruenwald Automatic classification of dialog acts with semantic classification trees and polygrams M. Mast, H. Niemann, E. Noeth, E. G. Schukat-Talamazzini Sample selection in natural language learning S. P. Engelson, I. Dagan Part 3: Symbolic Approaches --------------------------------------------------- Learning information extraction patterns from examples S. B. Huffman Implications of an automatic lexical acquisition system P. M. Hastings Using learned extraction patterns for text classification E. Riloff Issues in inductive learning of domain-specific text extraction rules S. Soderland, D. Fisher, J. Aseltine, W. Lehnert Applying machine learning to anaphora resolution C. Aone, S. W. Bennett Embedded machine learning systems for natural language processing: a general framework C. Cardie Acquiring and updating hierarchical knowledge for machine translation based on a clustering technique T. Yamazaki, M. J. Pazzani, C. Merz Applying an existing machine learning algorithm to text categorization I. Moulinier, J.-G. Ganascia Comparative results on using inductive logic programming for corpus-based parser construction J. M. Zelle, R. J. Mooney Learning the past tense of English verbs using inductive logic programming R. J. Mooney, M. E. Califf A dynamic approach to paradigm-driven analogy S. Federici, V. Pirrelli, F. Yvon Can punctuation help learning? M. Osborne Using parsed corpora for circumventing parsing A. K. Joshi, B. Srinivas A symbolic and surgical acquisition of terms through variation C. Jacquemin A revision learner to acquire verb selection rules from human-made rules and examples S. Kaneda, H. Almuallim, Y. Akiba, M. Ishii, T. Kawaoka Learning from texts - a terminological metareasoning perspective U. Hahn, M. Klenner, K. Schnattinger ************************************************************** Bibliographic Data and Ordering Information: Editors: Stefan Wermter, Univ. of Hamburg, Germany Ellen Riloff, Univ. of Utah, Salt Lake City, USA Gabriele Scheler, Munich Univ. of Tech. Germany Title: Connectionist, Statistical, and Symbolic Approaches to Learning for Natural Language Processing Publisher: Springer-Verlag ISBN: 3-540-60925-3 Pages: 468 + 9 Available: Europe: March 6, 1996 North America: around March 25, 1996 Subseries: Lecture Notes in Artificial Intelligence LNAI 1040 Cover: Softcover under Color Jacket Cover List Price: DM 86.00, approx. USD 68.00 With this information, any academic bookseller worlwide with a resonable computer science program should be able to provide copies of the book. Otherwise, one also can order through any Springer office directly, particularly through Berlin and Secaucus, as mentioned in the following special offer to Springer Authors. If you aren't a Springer Author you aren't entitled to make use of the special discount, but the ordering addresses are the same. ********************************************************** SPECIAL OFFER: SPRINGER-AUTHOR DISCOUNT All Authors or Editors of Springer Books, in particular Authors contributing to any LNCS or LNAI Proceedings, are entitled to buy any book published by Springer-Verlag for personal use at the "Springer-Author" discount of 33 1/3 % off the list price. Such preferential orders can only be processed through Springer directly (and not through book stores); reference to a Springer publication has to be given with such orders to any Springer office, particularly to the ones in Berlin and New York: Springer-Verlag Order Processing Department Postfach 31 13 40 D-10643 Berlin Germany FAX: +49 30 8207 301 Springer-Verlag New York, Inc. P.O. Box 2485 Secaucus, NJ 07096-2485 USA FAX: +1 201 348 4033 Phone: 1-800-SPRINGER (1 800 777 4647), toll-free in USA Preferential orders also can be placed by sending an email to orders@springer.de Shipping charges are DEM 5.00 per book for orders sent to Berlin, and USD 2.50 (plus USD 1.00 for each additional book) for orders sent to the Secaucus office. Payment of the book(s) plus shipping charges can be made by giving a credit card number together with the expiration date (American Express, Eurocard/Mastercard, Diners, and Visa are accepted) or by enclosing a check (mail orders only). ****************************************************************************** *Dr Stefan Wermter University of Hamburg * * Dept. of Computer Science * * Vogt-Koelln-Strasse 30 * *email: wermter@informatik.uni-hamburg.de D-22527 Hamburg * *phone: +49 40 54715-531 Germany * *fax: +49 40 54715-515 * *http://www.informatik.uni-hamburg.de/Arbeitsbereiche/NATS/staff/wermter.html* ****************************************************************************** From ted@SPENCER.CTAN.YALE.EDU Thu Mar 14 16:03:15 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 14 Mar 96 16:03:11 -0600; AA14430 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 14 Mar 96 16:03:08 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa17011; 13 Mar 96 20:30:31 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa16988; 13 Mar 96 20:00:56 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa04542; 13 Mar 96 20:00:05 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa11932; 13 Mar 96 14:47:29 EST Received: from SPENCER.CTAN.YALE.EDU by RI.CMU.EDU id aa17254; 13 Mar 96 14:45:21 EST Received: from PLANCK.CTAN.YALE.EDU by SPENCER.CTAN.YALE.EDU via ESMTP; Wed, 13 Mar 1996 19:45:09 GMT Received: by PLANCK.CTAN.YALE.EDU (Sendmail-8.7.1/res.nnc.client.cf-3.6) id TAA19483; Wed, 13 Mar 1996 19:45:08 GMT Date: Wed, 13 Mar 1996 19:45:08 GMT From: ted@SPENCER.CTAN.YALE.EDU Message-Id: <199603131945.TAA19483@PLANCK.CTAN.YALE.EDU> To: connectionists@cs.cmu.edu Cc: ted@SPENCER.CTAN.YALE.EDU, brennan@NNC.YALE.EDU, narendra@NNC.YALE.EDU Subject: Postdoctoral positions available The Neuroengineering and Neuroscience Center at Yale University is seeking to build a pool of qualified scientists and engineers to participate in research on applications of pattern recognition in engineering and medicine. Applicants must have a Ph.D. and demonstrated expertise in one or more of the following fields: pattern recognition, signal processing, machine learning, adaptive control, artificial neural networks, image analysis. Successful candidates will participate in highly creative and interdisciplinary projects of major scientific and social importance. Please send curriculum vitae and list of professional references to Prof. K.S. Narendra, Director NNC 5 Science Park North New Haven, CT 06511 Yale University is an Affirmative Action/Equal Opportunity employer. Women and Minorities encouraged to apply. From laura@mpipf-muenchen.mpg.de Fri Mar 15 08:30:34 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id IAA11420 for ; Fri, 15 Mar 1996 08:30:26 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id IAA25089 for ; Fri, 15 Mar 1996 08:30:24 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa18209; 14 Mar 96 16:56:33 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa18205; 14 Mar 96 16:39:21 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa05483; 14 Mar 96 16:38:54 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa27102; 14 Mar 96 11:41:37 EST Received: from [192.129.1.18] by CS.CMU.EDU id aa10932; 14 Mar 96 11:41:05 EST Received: from mac11.mpipf-muenchen.mpg.de by ratio.mpipf-muenchen.mpg.de; (5.65v3.2/1.1.8.2/25Oct95-1145AM) id AA12203; Thu, 14 Mar 1996 17:41:13 +0100 X-Sender: rlaura@ratio.mpipf-muenchen.mpg.de Message-Id: Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Date: Thu, 14 Mar 1996 17:41:29 +0100 To: connectionists@cs.cmu.edu From: Laura Martignon Subject: paper available: "Bayesian Learning of loglinear models for neuron connectivity" Kathryn Laskey and I have just finished the paper: "Bayesian Learning of loglinear models for neuron connectivity" Kathryn Laskey Department of Systems Engineering George Mason University Fairfax, VA 22030 klaskey@gmu.edu Laura Martignon Max Planck Institute for Psychological Research 80802 M=FCnchen, Germany laura@mpipf-muenchen.mpg.de Abstract This paper presents a Bayesian approach to learning the connectivity structure of a group of neurons from data on configuration frequencies. A major objective of the research is to provide statistical tools for detecting changes in firing patterns with changing stimuli. Our framework is not restricted to the well-understood case of pair interactions, but generalizes the Boltzmann machine model to allow for higher order interactions. The paper applies a Markov Chain Monte Carlo Model Composition (MC3) algorithm to search over connectivity structures and uses Laplace's method to approximate posterior probabilities of structures. Performance of the methods was tested on synthetic data. The models were also applied to data obtained by Vaadia on multi-unit recordings of several neurons in the visual cortex of a rhesus monkey in two different attentional states. Results confirmed the experimenters' conjecture that different attentional states were associated with different interaction structures. Keywords: Nonhierarchical loglinear models, Markov Chain Monte Carlo Model composition, Laplace's Method, Neural Networks To obtain a copy of these papers, please send your email request to Laura Martignon e-mail: laura@mpipf-muenchen.mpg.de From itl-rec@thuban.crd.ge.com Fri Mar 15 21:29:26 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id VAA10095 for ; Fri, 15 Mar 1996 21:29:12 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id VAA07567 for ; Fri, 15 Mar 1996 21:29:08 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab18209; 14 Mar 96 16:58:03 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab18205; 14 Mar 96 16:39:24 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa05487; 14 Mar 96 16:39:03 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa00747; 14 Mar 96 15:17:20 EST Received: from [192.35.44.5] by EDRC.CMU.EDU id aa25310; 14 Mar 96 15:16:58 EST Received: from thuban.crd.ge.com by crdems.ge.com (5.65/GE 1.77) id AA24198; Thu, 14 Mar 96 15:12:55 -0500 Received: by thuban.crd.ge.com (4.1/GE-CRD Standard Sendmail Version S1.5)id AA20879; Thu, 14 Mar 96 15:12:55 EST Date: Thu, 14 Mar 96 15:12:55 EST From: itlrecruiting Message-Id: <9603142012.AA20879@thuban.crd.ge.com> To: connectionists@cs.cmu.edu Subject: Job: Data Mining / Neural Nets / Artificial Intelligence The Information Technology Laboratory (80 people strong and still growing) at the Corporate Research & Development Center of General Electric in Schenectady, New York has the following position to offer: R&D Staff opportunity in Data Mining/Analysis/Warehousing BACKGROUND REQUIRED: PhD in Computer Science, Statistics, Artificial Intelligence or related field with a broad knowledge base. Strong interpersonal skills, good initiative and analytical skills, adaptable to change, high self confidence. Excellent computer skills required: e.g. either hands-on experience in implementing data storage and access solutions for multi-million record databases or hands-on experience in sampling and Data Mining / knowledge discovery analysis algorithms on multi-million record databases. DESIRED ALSO: Experience with C++ object-oriented programming. WE OFFER A CHALLENGING PERSPECTIVE: Develop and apply modern statistical methods, machine learning techniques and neural nets to a variety of strategically important and technically significant problems throughout GE, involving finance, product development, manufacturing and process improvement, and product servicing and reliability. Lead work with analysts, engineers, and managers in the diverse GE businesses, e.g. Aircraft Engines, Capital Services, Medical Systems, NBC, Plastics, and Appliances and with scientists at the Research & Development Center. --------------------------------------------------------------------------- FOR YOUR INTEREST: GE is one of the world's largest and most successful companies, having leadership positions in business segments including aircraft engines, plastics, manufacturing, capital services, and others. GE has its Corporate Research and Development Center located in Schenectady, New York (more at http://www.ge.com/). It supports the advanced technology requirements of all GE businesses. The 1000-plus staff of scientists and engineers is composed of representatives of most major disciplines (more at http://www.crd.ge.com/). APPLICATION: If you meet the requirements and you are interested, please send your resume via electronic email in plain ASCII format to itl-rec@thuban.crd.ge.com (Steve Mirer). Please include, where you found this ad and put "DATA MINING" in the subject line. BTW, we are recruiting world-wide to get the best possible match. ---------------------------------------------------------------- GE is an equal opportunity employer. From arbib@pollux.usc.edu Fri Mar 15 21:29:26 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id VAA10100 for ; Fri, 15 Mar 1996 21:29:13 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id VAA07570 for ; Fri, 15 Mar 1996 21:29:11 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa18360; 14 Mar 96 19:24:40 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa18358; 14 Mar 96 19:14:52 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa05594; 14 Mar 96 19:14:32 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa04469; 14 Mar 96 18:26:45 EST Received: from pollux.usc.edu by CS.CMU.EDU id aa15251; 14 Mar 96 18:25:52 EST Received: (from arbib@localhost) by pollux.usc.edu (8.7.2/8.7.2/usc) id PAA15747 for connectionists@cs.cmu.edu; Thu, 14 Mar 1996 15:25:04 -0800 (PST) Date: Thu, 14 Mar 1996 15:25:04 -0800 (PST) From: "Michael A. Arbib" Message-Id: <199603142325.PAA15747@pollux.usc.edu> To: connectionists@cs.cmu.edu Subject: Workshop on Sensorimotor Coordination FINAL CALL FOR PAPERS Workshop on SENSORIMOTOR COORDINATION: AMPHIBIANS, MODELS, AND COMPARATIVE STUDIES Poco Diablo Resort, Sedona, Arizona, November 22-24, 1996 Co-Directors: Kiisa Nishikawa (Northern Arizona University, Flagstaff) and Michael Arbib (University of Southern California, Los Angeles). Local Arrangements Chair: Kiisa Nishikawa. E-mail enquiries may be addressed to Kiisa.Nishikawa@nau.edu or arbib@pollux.usc.edu. Further information may be found on our home page at http://www.nau.edu:80/~biology/vismot.html. Program Committee: Kiisa Nishikawa (Chair), Michael Arbib, Emilio Bizzi, Chris Comer, Peter Ewert, Simon Giszter, Mel Goodale, Ananda Weerasuriya, Walt Wilczynski, and Phil Zeigler. SCIENTIFIC PROGRAM The aim of this workshop is to study the neural mechanisms of sensorimotor coordination in amphibians and other model systems for their intrinsic interest, as a target for developments in computational neuroscience, and also as a basis for comparative and evolutionary studies. The list of subsidiary themes given below is meant to be representative of this comparative dimension, but is not intended to be exhaustive. The emphasis (but not the exclusive emphasis) will be on papers that encourage the dialog between modeling and experimentation. A decision as to whether or not to publish a proceedings is still pending. Central Theme: Sensorimotor Coordination in Amphibians and Other Model Systems Subsidiary Themes: Visuomotor Coordination: Comparative and Evolutionary Perspectives Reaching and Grasping in Frog, Pigeon, and Primate Cognitive Maps Auditory Communication (with emphasis on spatial behavior and sensory integration) Motor Pattern Generators This workshop is the sequel to four earlier workshops on the general theme of "Visuomotor Coordination in Frog and Toad: Models and Experiments". The first two were organized by Rolando Lara and Michael Arbib at the University of Massachusetts, Amherst (1981) and Mexico City (1982). The next two were organized by Peter Ewert and Arbib in Kassel and Los Angeles, respectively, with the Proceedings published as follows: Ewert, J.-P. and M. A. Arbib (Eds.) 1989. Visuomotor Coordination: Amphibians, Comparisons, Models and Robots. New York: Plenum Press. Arbib, M.A. and J.-P. Ewert (Eds.) 1991. Visual Structures and Integrated Functions, Research Notes in Neural Computing 3. Heidelberg, New York: Springer Verlag. INSTRUCTIONS FOR CONTRIBUTORS Persons who wish to present oral papers are asked to send three copies of an extended abstract, approximately 4 pages long, including figures and references. Persons who wish to present posters are asked to send a one page abstract. Abstracts may be sent by regular mail, e-mail or FAX. Authors should be aware that e-mailed abstracts should contain no figures. Abstracts should be sent no later than 1 May, 1996 to: Kiisa Nishikawa , Department of Biological Sciences, Northern Arizona University, Flagstaff, AZ 86011-5640, E-mail: Kiisa.Nishikawa@nau.edu; FAX: (520)523-7500. Notification of the Program Committee's decision will be sent out no later than 15 June, 1996. REGISTRATION INFORMATION Meeting Location and General Information: The Workshop will be held at the Poco Diablo Resort in Sedona, Arizona (a beautiful small town set in dramatic red hills) immediately following the Society for Neuroscience meeting in 1996. The 1996 Neuroscience meeting ends on Thursday, November 21, so workshop participants can fly from Washington, DC to Phoenix, AZ that evening, meet Friday, Saturday, and Sunday, with a Workshop Banquet on Sunday evening, and fly home on Monday, November 25th. Paper sessions will be held all day on Friday, on Saturday afternoon, and all day on Sunday. Poster sessions will be held on Saturday afternoon and evening. A group field trip is planned for Saturday morning. Graduate Student and Postdoctoral Participation: In order to encourage the participation of graduate students and postdoctorals, we have arranged for affordable housing, and in addition we are able to offer a reduced registration fee (see below) thanks to the generous contribution of the Office of the Associate Provost for Research and Graduate Studies at Northern Arizona University. Travel from Phoenix to Sedona: Sedona, AZ is located approximately 100 miles north of Phoenix, where the nearest major airport (Sky Harbor) is located. Workshop attendees may wish to arrange their own transportation (e.g., car rental from Phoenix airport) from Phoenix to Sedona, or they may use the Workshop Shuttle (estimated round trip cost $20 US) to Sedona on 21 November, with a return to Phoenix on 25 November. If you plan to use the Workshop Shuttle, we will need to know your expected arrival time in Phoenix by 1 October 1996, to ensure that space is available for you at a convenient time. Lodging: The following costs are for each night. Since many participants may want to extend their stay to further enjoy Arizona's scenic beauty, we have negotiated special rates for additional nights after the end of the workshop on November 24th. Attendees should make their own booking with the Poco Diablo Resort, by phone (800) 352-5710 or FAX (520) 282-9712. Thurs.-Fri. (and additional week nights before the workshop) per night: students $85 US + tax, faculty $105 + tax Sat.-Sun. (and additional week nights after the workshop) per night: students $69 + tax, faculty $89 + tax. The student room rates are for double occupancy. Thus, students willing to share a room may stay for half the stated rate. When you make your room reservations with the Poco Diablo Resort, please be sure to indicate the number of guests in your party. Graduate students and postdocs should be sure to indicate whether they want single or double occupancy. REGISTRATION FEES: Students and postdoctorals $100; faculty, guests and others $200. The registration fee includes lunch Fri. - Sun., wine and cheese reception during the Saturday evening poster session, and a Farewell Dinner on Sunday evening. Registration fees should be paid by check in US funds, made payable to "Sensorimotor Coordination Workshop", and should be sent to Kiisa Nishikawa at the address listed below, together with the completed registration form that follows at the end of this announcement. Completed registration forms and fees must be received by 1 July, 1996. Late registration fees will be $150 for students and postdoctorals and $250 for faculty. REGISTRATION FORM NAME: ADDRESS: PHONE: FAX: EMAIL: STATUS: [ ] Faculty ($200); [ ] Postdoctoral ($100); [ ] Student ($100); [ ] Other ($200). (Postdocs and students: Please attach certification of your status signed by your supervisor.) TYPE OF PRESENTATION (paper vs. poster): ABSTRACT SENT: (yes/no) AREAS OF INTEREST RELEVANT TO WORKSHOP: WILL YOU REQUIRE ANY SPECIAL AUDIOVISUAL EQUIPMENT FOR YOUR PRESENTATION? HAVE YOU MADE A RESERVATION WITH THE HOTEL? EXPECTED TIME OF ARRIVAL IN PHOENIX (ON NOVEMBER 21): EXPECTED TIME OF DEPARTURE FROM PHOENIX (ON NOVEMBER 25): DO YOU WISH TO USE THE WORKSHOP SHUTTLE TO TRAVEL FROM PHOENIX TO SEDONA? (If so, please be sure that we know your expected arrival time by 1 October!) DO YOU WISH TO PARTICIPATE IN A GROUP HIKE IN THE SEDONA AREA ON SATURDAY MORNING? Please make sure that your check (in US funds and payable to the "Sensorimotor Coordination Workshop") is included with this form. If you plan to bring a guest with you to the Workshop, please add their name(s) to this form and enclose their registration fee along with your own. Mail to: Kiisa Nishikawa, Department of Biological Sciences, Northern Arizona University, Flagstaff, AZ 86011-5640. E-mail: Kiisa.Nishikawa@nau.edu. FAX: (520)523-7500. Phone: (520)523-9497. From rjb@psy.ox.ac.uk Fri Mar 15 21:29:28 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id VAA10102 for ; Fri, 15 Mar 1996 21:29:16 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id VAA07572 for ; Fri, 15 Mar 1996 21:29:14 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa19465; 15 Mar 96 15:37:06 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa19463; 15 Mar 96 15:20:42 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa06331; 15 Mar 96 15:20:14 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa13245; 15 Mar 96 9:50:32 EST Received: from [163.1.29.191] by RI.CMU.EDU id aa26893; 15 Mar 96 9:50:08 EST Received: from axp02.mrc-bbc.ox.ac.uk (axp02.mrc-bbc.ox.ac.uk [163.1.29.192]) by axp01.mrc-bbc.ox.ac.uk (8.7.3/8.7.3) with ESMTP id OAA06983 for ; Fri, 15 Mar 1996 14:49:04 GMT Received: (rjb@localhost) by axp02.mrc-bbc.ox.ac.uk (8.6.11/8.6.12) id OAA06560; Fri, 15 Mar 1996 14:49:03 GMT Date: Fri, 15 Mar 1996 14:49:03 GMT Message-Id: <199603151449.OAA06560@axp02.mrc-bbc.ox.ac.uk> From: Roland Baddeley To: Connectionists@cs.cmu.edu Subject: Paper available on exploritory projection pursuit. The following paper is available on the web at http://www.mrc-bbc.ox.ac.uk/~rjb/ It has been accepted for publication in Network. TITLE: Searching for filters with ``interesting'' output distributions: an uninteresting direction to explore? Abstract It has been proposed that the receptive fields of neurons in V1 are optimised to generate ``sparse'', Kurtotic, or ``interesting'' output probability distributions \cite{Barlow92,Barlow94,Field94,Intrator91,Intrator92d}. We investigate the empirical evidence for this further and argue that filters can produce ``interesting'' output distributions simply because natural images have variable local intensity variance. If the proposed filters have zero D.C., then the probability distribution of filter outputs (and hence the output Kurtosis) is well predicted simply from these effects of variable local variance. This suggests that finding filters with high output Kurtosis does not necessarily signal interesting image structure. It is then argued that finding filters that maximise output Kurtosis generates filters that are incompatible with observed physiology. In particular the optimal difference--of--Gaussian (DOG) filter should have the smallest possible scale, an on--centre off--surround cell should have a negative D.C., and that the ratio of centre width to surround width should approach unity. This is incompatible with the physiology. Further, it is also predicted that oriented filters should always be oriented in the vertical direction, and of all the filters tested, the filter with the highest output Kurtosis has the lowest signal to noise (the filter is simply the difference of two neighbouring pixels). Whilst these observations are not incompatible with the brain using a sparse representation, it does argue that little significance should be placed on finding filters with highly Kurtotic output distributions. It is therefore argued that other constraints are required in order to understand the development of visual receptive fields. FILE: http://www.mrc-bbc.ox.ac.uk/ftp/users/rjb/rjb_kur.ps.Z -- Roland Baddeley Research Fellow, MRC Centre for Cognitive Neuroscience University of Oxford normal mail: Experimental Psychology email: rjb@psy.ox.ac.uk Oxford University phone: +44-1865-271914 South Parks Road fax: +44-1865-272488 Oxford, OX1 3UD UK From ib@rana.usc.edu Sat Mar 16 11:09:07 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id LAA16805 for ; Sat, 16 Mar 1996 11:08:55 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id LAA12338 for ; Sat, 16 Mar 1996 11:08:53 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab19777; 15 Mar 96 21:05:05 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab19758; 15 Mar 96 20:44:32 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa06510; 15 Mar 96 20:44:13 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa21629; 15 Mar 96 19:10:05 EST Received: from mizar.usc.edu by RI.CMU.EDU id aa29823; 15 Mar 96 19:03:29 EST Received: from [128.125.37.108] (bsl12.usc.edu [128.125.37.108]) by mizar.usc.edu (8.7.2/8.7.2/usc) with SMTP id QAA01395 for ; Fri, 15 Mar 1996 16:02:50 -0800 (PST) Message-Id: <199603160002.QAA01395@mizar.usc.edu> X-Sender: bieder@mizar.usc.edu Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Date: Fri, 15 Mar 1996 16:02:03 -0800 To: Connectionists@cs.cmu.edu From: Irving Biederman Subject: Shift Invariance Shimon Edelman (March 8) writes: [Omission of some of posting] >Note that the purpose of my previous posting was to advocate caution, >certainly not to argue that all claims of invariance are wrong. >Fortunately, my job in this matter is easy: just one example of a >manifest lack of invariance suffices to invalidate the strong version >of invariance-based theory of vision, which seems to be espoused by >Goldfarb: >So, here it goes... Whereas invariance does hold in many recognition >tasks (in particular, in Biederman's experiments, as well as in the >experiments reported in [1]), it does not in others (as, e.g., in [2], >where interaction between size invariance and orientation is >reported). A recent comprehensive survey of (the far from invariant) >human performance in recognizing rotated objects can be found in >[3]. Furthermore, not only recognition, but also perceptual learning, >seems to be non-invariant in some cases; see [4,5]. [Omission of rest of posting] It should be so easy. Of course, ALL of vision is not shift invariant (I don't believe that Goldfarb was asserting that it was) as there is clear evidence that people are, for example, quite sensitive to the location of objects when they reach out to grasp them. The issue of shift invariance was specifically raised, not for ALL of vision, but the domain of (what should be called) object recognition, what I termed "primal access", Biederman, '87, in which basic-level or (most) subordinate-level classification is made from a large and uncertain population of objects, as when channel surfing. I think that readers who are not familiar with some of the literature cited in Edelman's posting might be misled into thinking that shift invariance in object recognition is a special case. As I noted in my previous posting, the evidence is quite strong that object recognition tasks, at the same time that they show a visual (and not just verbal or conceptual) benefit from a single presentation in an experiment, also show shift invariance. (They also show size, scale, reflection, and rotation in depth invariance, as long as the same parts and relations are readily distinguished.) Edelman points out that there have been reports of view-dependency for depth rotation, not shift, in "recognition" tasks. (Goldfard specifically exempted rotation.) But even for depth rotation, readers should note that the findings of large rotation costs are found only for extremely difficult discrimination tasks, performed only rarely in normal visual activities in which viewpoint-invariant information is generally not available, such as distinguishing among a set of highly similar bent paper clips. Why would invariance not be found with extremely difficult tasks? When tasks are difficult, subjects will attempt various strategies (e.g., look to the left [a dorsal function?] for a small, distinguishing feature), that might produce a cost of view-change, but this does not mean that the representation of the feature (or object) itself is not invariant. All in all, the absence of an effect of a view-change, puts one in a simpler explanatory position (assuming adequate power), than when an effect of view change (say, a shift) is found. The latter kind of result means that one has to eliminate other task variables as potential bases of the effect, such as a search for a distinguishing feature, as noted above. A finding of an effect of a change in viewpoint in "object recognition" might or might not mean that the representation of the object is viewpoint dependent. The "view-based" camp will have to demonstrate that the representation of an object (for primal access) really does change when it is shifted, or shown at a different size, or orientation in depth (assuming that the same parts are in view). They haven't done this yet. Whether a TASK (NOT A REPRESENTATION) does or does not manifest shift invariance might well depend on the degree to which it reflects dorsal (motor interaction) vs. ventral (recognition) cortical representations. The manifestation of these invariances nicely dovetails with the phenomenon of "object constancy" noted by the Gestaltists, in which the perception of the real object is largely unaffected by its translation or rotation. It is of interest that patient D. F. studied by Milner and Goodale, who presumably has a damaged ventral pathway shows no awareness of objects while at the same time is able to reach competently for them. My views on these matters of view invariance (especially of rotation in depth) are more fully presented in: 1. Biederman, I., & Gerhardstein, P. C. (1993). Recognizing depth-rotated objects: Evidence and conditions for 3D viewpoint invariance. Journal of Experimental Psychology: Human Perception and Performance, 19, 1162-1182. 2. Biederman, I., & Gerhardstein, P. C. (1995). Viewpoint-dependent mechanisms in visual object recognition: Reply to Tarr and B=FClthoff (1995). Journal of Experimental Psychology: Human Perception and Performance, 21, 1506-1514. 3. Biederman, I., & Bar, M. (1995). One-Shot Viewpoint Invariance with Nonsense Objects. Paper presented at the Annual Meeting of the Psychonomic Society, 1995, Los Angeles, November. Available on our WWW site: http://rana.usc.edu:8376/~ib/iul.html From terry@salk.edu Sat Mar 16 11:09:08 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id LAA16803 for ; Sat, 16 Mar 1996 11:08:53 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id LAA12336 for ; Sat, 16 Mar 1996 11:08:51 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa19777; 15 Mar 96 21:03:21 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa19758; 15 Mar 96 20:44:24 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa06505; 15 Mar 96 20:43:42 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa21382; 15 Mar 96 18:47:54 EST Received: from helmholtz.salk.edu by CS.CMU.EDU id aa25174; 15 Mar 96 18:47:27 EST Received: by salk.edu (4.1/SMI-4.1) id AA08606; Fri, 15 Mar 96 15:47:24 PST Date: Fri, 15 Mar 96 15:47:24 PST From: Terry Sejnowski Message-Id: <9603152347.AA08606@salk.edu> To: connectionists@cs.cmu.edu Subject: Neural Computation 8:3 Titles Cc: terry@salk.edu Neural Computation - Volume 8, Number 3 - April 1, 1996 Long Article: A Smoothing Regularizer for Feedforward and Recurrent Neural Networks Lizhong Wu and John Moody Notes: Note on the Maxnet Dynamics John P. F. Sum and Peter K. S. Tam Optimizing Synaptic Conductance Calculation for Network Simulations William W. Lytton Letters: Parameter Extraction from Population Codes: A Critical Assessment Herman P. Snippe Energy Efficient Neural Codes William B. Levy and Robert A. Baxter A Nonlinear Hebbian Network that Learns to Detect Disparity in Random-Dot Stereograms Christopher W. Lee and Bruno A. Olshausen Coupling the Neural and Physical Dynamics in Rhythmic Movements Nicholas G. Hatsopoulos Predictive Minimum Description Length Criterion for Time Series Modeling with Neural Networks Mikko Lehtokangas, Jukka Saarinen, Pentti Huuhtanen and Kimmo Kaski Minimum Description Length, Regularization and Multi-Model Data Richard Rohwer and John C. van der Rest VC Dimension of an Integrate-and-Fire Neuron Model Anthony M. Zador and Barak A. Pearlmutter The VC-Dimension and Pseudodimension of Two-Layer Neural Networks with Discrete Inputs Peter L. Bartlett and Robert C. Williamson A Theoretical and Experimental Account of N-Tuple Classifier Performance Richard Rohwer and Michal Morciniec The Effects of Adding Noise During Backpropagation Training on a Generalization Performance Guozhong An ----- ABSTRACTS - http://www-mitpress.mit.edu/jrnls-catalog/neural.html SUBSCRIPTIONS - 1996 - VOLUME 8 - 8 ISSUES ______ $50 Student and Retired ______ $78 Individual ______ $220 Institution Add $28 for postage and handling outside USA (+7% GST for Canada). (Back issues from Volumes 1-7 are regularly available for $28 each to institutions and $14 each for individuals Add $5 for postage per issue outside USA (+7% GST for Canada) mitpress-orders@mit.edu MIT Press Journals, 55 Hayward Street, Cambridge, MA 02142. Tel: (617) 253-2889 FAX: (617) 258-6779 ----- From terry@salk.edu Sat Mar 16 22:30:43 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id WAA21313 for ; Sat, 16 Mar 1996 22:30:37 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id WAA17113 for ; Sat, 16 Mar 1996 22:30:34 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa19756; 15 Mar 96 20:53:49 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa19754; 15 Mar 96 20:43:34 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa06499; 15 Mar 96 20:43:28 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa18165; 15 Mar 96 15:32:39 EST Received: from helmholtz.salk.edu by CS.CMU.EDU id aa23291; 15 Mar 96 15:31:58 EST Received: by salk.edu (4.1/SMI-4.1) id AA05918; Fri, 15 Mar 96 12:31:49 PST Date: Fri, 15 Mar 96 12:31:49 PST From: Terry Sejnowski Message-Id: <9603152031.AA05918@salk.edu> To: connectionists@cs.cmu.edu Subject: Telluride Workshop - Deadline April 5 Cc: terry@salk.edu WORKSHOP ON NEUROMORPHIC ENGINEERING JUNE 24 - JULY 14, 1996 TELLURIDE, COLORADO Deadline for application is April 5, 1996. Christof Koch (Caltech) and Terry Sejnowski (Salk Institute/UCSD) invite applications for one three week workshop that will be held in Telluride, Colorado in 1996. The first two Telluride Workshops on Neuromorphic Engineering were held in the summer of 1994 and 1995, sponsored by NSF and co-funded by the "Center for Neuromorphic Systems Engineering" at Caltech, were resounding successes. A summary of these workshops, together with a list of participants is available from: http://www.klab.caltech.edu/~timmer/telluride.html or http://www.salk.edu/~bryan/telluride.html GOALS: Carver Mead introduced the term "Neuromorphic Engineering" for a new field based on the design and fabrication of artificial neural systems, such as vision systems, head-eye systems, and roving robots, whose architecture and design principles are based on those of biological nervous systems. The goal of this workshop is to bring together young investigators and more established researchers from academia with their counterparts in industry and national laboratories, working on both neurobiological as well as engineering aspects of sensory systems and sensory-motor integration. The focus of the workshop will be on "active" participation, with demonstration systems and hands-on-experience for all participants. Neuromorphic engineering has a wide range of applications from nonlinear adaptive control of complex systems to the design of smart sensors. Many of the fundamental principles in this field, such as the use of learning methods and the design of parallel hardware, are inspired by biological systems. However, existing applications are modest and the challenge of scaling up from small artificial neural networks and designing completely autonomous systems at the levels achieved by biological systems lies ahead. The assumption underlying this three week workshop is that the next generation of neuromorphic systems would benefit from closer attention to the principles found through experimental and theoretical studies of brain systems. FORMAT: The three week workshop is co-organized by Dana Ballard (Rochester, US), Rodney Douglas (Zurich, Switzerland) and Misha Mahowald (Zurich, Switzerland). It is composed of lectures, practical tutorials on aVLSI design, hands-on projects, and interest groups. Apart from the lectures, the activities run concurrently. However, participants are free to attend any of these activities at their own convenience. There will be two lectures in the morning that cover issues that are important to the community in general. Because of the diverse range of backgrounds among the participants, the majority of these lectures will be tutorials, rather than detailed reports of current research. These lectures will be given by invited speakers. Participants will be free to explore and play with whatever they choose in the afternoon. Projects and interest groups meet in the late afternoons, and after dinner. The aVLSI practical tutorials will cover all aspects of aVLSI design, simulation, layout, and testing over the course of the three weeks. The first week covers basics of transistors, simple circuit design and simulation. This material is intended for participants who have no experience with aVLSI. The second week will focus on design frames for silicon retinas, from the silicon compilation and layout of on-chip video scanners, to building the peripheral boards necessary for interfacing aVLSI retinas to video output monitors. Retina chips will be provided. The third week will feature a session on floating gates, including lectures on the physics of tunneling and injection, and experimentation with test chips. Projects that are carried out during the workshop will be centered in four groups: 1) active perception, 2) elements of autonomous robots, 3) robot manipulation, and 4) multichip neuron networks. The "active perception" project group will emphasize vision and human sensory-motor coordination and will be organized by Dana Ballard and Mary Hayhoe (Rochester). Issues to be covered will include spatial localization and constancy, attention, motor planning, eye movements, and the use of visual motion information for motor control. Demonstrations will include a robot head active vision system consisting of a three degree-of-freedom binocular camera system that is fully programmable. The vision system is based on a DataCube videopipe which in turn provides drive signals to the three motors of the head. Projects will involve programming the DataCube to implement a variety of vision/oculomotor algorithms. The "elements of autonomous robots" group will focus on small walking robots. It will look at characteristics and sources of parts for building robots, play with working examples of legged robots, and discuss CPG's and theories of nonlinear oscillators for locomotion. It will also explore the use of simple aVLSI sensors for autonomous robots. The "robot manipulation" group will use robot arms and working digital vision boards to investigate issues of sensory motor integration, passive compliance of the limb, and learning of inverse kinematics and inverse dynamics. The "multichip neuron networks" project group will use existing interchip communication interfaces to program small networks of artificial neurons to exhibit particular behaviors such as amplification, oscillation, and associative memory. Issues in multichip communication will be discussed. PARTIAL LIST OF INVITED LECTURERS: Dana Ballard, Rochester. Randy Beer, Case-Western Reserve. Kwabena Boahen, Caltech. Avis Cohen, Maryland. Tobi Delbruck, Arithmos, Palo Alto. Steve DeWeerth, Georgia Tech. Chris Dioro, Caltech. Rodney Douglas, Zurich. John Elias, Delaware University. Stefano Fusi, Italy Mary Hayhoe, Rochester. Geoffrey Hinton, Toronto. Ian Hoswill, NWU Christof Koch, Caltech. Shih-Chii Liu, Caltech and Rockwell. Misha Mahowald, Zurich. Stefan Schaal, Georgia Tech. Mark Tilden, Los Alamos. Terry Sejnowski, Salk Institute and UC San Diego. Paul Viola, MIT LOCATION AND ARRANGEMENTS: The workshop will take place at the "Telluride Summer Research Center," located in the small town of Telluride, 9000 feet high in Southwest Colorado, about 6 hours away from Denver (350 miles) and 5 hours from Aspen. Continental and United Airlines provide many daily flights directly into Telluride. Participants will be housed in shared condominiums, within walking distance of the Center. Bring hiking boots and a backpack, since Telluride is surrounded by beautiful mountains (several mountains are in the 14,000+ range). The workshop is intended to be very informal and hands-on. Participants are not required to have had previous experience in analog VLSI circuit design, computational or machine vision, systems level neurophysiology or modeling the brain at the systems level. However, we strongly encourage active researchers with relevant backgrounds from academia, industry and national laboratories to apply, in particular if they are prepared to talk about their work or to bring demonstrations to Telluride (e.g. robots, chips, software). Internet access will be provided. Technical staff present throughout the workshops will assist with software and hardware issues. We will have a network of SUN workstations running UNIX, one or two MACs and a few PCs running windows and LINUX. We have funds to reimburse some participants for up to $500.- of domestic travel and for all housing expenses. Please specify on the application whether such finanical help is needed. Unless otherwise arranged with one of the organizers, we expect participants to stay for the duration of this three week workshop. HOW TO APPLY: The deadline for receipt of applications is April 5, 1996. Applicants should be at the level of graduate students or above (i.e. post-doctoral fellows, faculty, research and engineering staff and the equivalent positions in industry and national laboratories). We actively encourage qualified women and minority candidates to apply. Application should include: 1. Name, address, telephone, e-mail, FAX, and minority status (optional). 2. Curriculum Vitae. 3. One page summary of background and interests relevant to the workshop. 4. Description of special equipment needed for demonstrations that could be brought to the workshop. 5. Two letters of recommendation Complete applications should be sent to: Prof. Terrence Sejnowski The Salk Institute 10010 North Torrey Pines Road San Diego, CA 92037 email: terry@salk.edu FAX: (619) 587 0417 Applicants will be notified around May 1, 1996.