From Ilpnet@ijs.si Tue Feb 20 09:11:09 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 20 Feb 96 09:11:03 -0600; AA18989 Received: from uhura.ijs.si by lucy.cs.wisc.edu; Tue, 20 Feb 96 09:09:02 -0600 Received: from cathy.ijs.si by uhura.ijs.si with SMTP using DNS (PP) id <29467-0@uhura.ijs.si>; Tue, 20 Feb 1996 15:50:21 +0100 Received: from CATHY.IJS.SI by CATHY.IJS.SI (PMDF V4.3-10 #8779) id <01I1FWVPWUJ40048G3@CATHY.IJS.SI>; Tue, 20 Feb 1996 15:50:09 +0001 Date: Tue, 20 Feb 1996 15:50:08 +0001 From: "ILPNET project, IJS" Subject: ILPNewsletter Vol. 3 No. 1 To: ml@lucy.cs.wisc.edu Message-Id: <01I1FWVQ2QPU0048G3@CATHY.IJS.SI> X-Vms-To: in%"ml@cs.wisc.edu" Mime-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Transfer-Encoding: 7BIT %------------------------------------------------------------------------------% ILP Newsletter Volume 3, Number 1, 15th February 1996 %------------------------------------------------------------------------------% Editors: Saso Dzeroski and Nada Lavrac, Jozef Stefan Institute, Ljubljana, SI %------------------------------------------------------------------------------% Address all communication related to the ILP Newsletter to ilpnet@ijs.si To subscribe/unsubscribe send email with subject SUBSCRIBE/UNSUBSCRIBE ILPNEWS Send contributions in messages with subject heading ILPNEWS CONTRIBUTION Send comments and suggestions under subject heading ILPNEWS COMMENTS Back issues of the Newsletter and other information about ILPNET and ILP available via the World Wide Web (WWW), URL http://www-ai.ijs.si/ilpnet.html %------------------------------------------------------------------------------% Contents: - Abstracts of PhD theses related to ILP * Henrik Bostrom: Explanation-Based Transformation of Logic Programs * Peter Idestam-Almquist: Generalization of Clauses * Kamal Ali: Learning Probabilistic Relational Concept Descriptions - ILP'96 - Sixth International Workshop on ILP (CFP LaTeX) - ALT'96 - Seventh International Workshop on Algorithmic Learning Theory (CFP) - IDAMAP'96 - Intelligent Data Analysis in Medicine and Pharmacology (CFP) - KR'96 Workshop on Relevance in Knowledge Representation and Reasoning (CFP) - ILP tutorial at ECAI'96 - New release of MOBAL %------------------------------------------------------------------------------% %------------------------------------------------------------------------------% Abstracts of PhD theses related to ILP %------------------------------------------------------------------------------% Title: "Explanation-Based Transformation of Logic Programs" Author: Henrik Bostrom University: Department of Computer and Systme Sciences, Stockholm University and Royal Institute of Technology Explanation-Based Generalization (EBG) is a technique for deriving a specialization of a concept definition (target concept) from a proof (explanation) of why a particular instance (training example) belongs to the concept. The reason for deriving the specialization is to provide a new definition of the target concept that can be used more efficiently than the original one to determine (some) instances of the concept. In a logic programming framework, the specialization corresponds to a clause that can be used to identify instances of a predicate more efficiently than by the original definition. However, the addition of clauses produced by EBG to a logic program may degrade, rather than improve, the efficiency of the program. Two potential causes of this problem are the inefficient organization of produced clauses and the increased redundancy. We present approaches to these problems, which are reformulations of EBG in a program transformation frame- work. The presented algorithms are shown to be meaning preserving and to produce clauses that are equivalent to the clauses produced by EBG. Worst-case analyses of the size of the resulting programs are presented and the limitations of the algorithms are discussed. Experimental results are presented showing that the efficiency of programs obtained by applying EBG can be significantly improved while organizing produced clauses efficiently and avoiding increased redundancy. %------------------------------------------------------------------------------% Title: "Generalization of Clauses" Author: Peter Idestam-Almquist University: Department of Computer and Systme Sciences, Stockholm University and Royal Institute of Technology Year: 1993 In the area of inductive learning, generalization is a main operation, and the usual definition of induction is based on logical implication. Recently there has been a rising interest in clausal representation of knowledge in machine learning. Almost all inductive learning systems that perform generalization of clauses use the generality relation theta-subsumption instead of implication. The main reason is that there is a well-known and simple technique to compute lest general generalizations under theta-subsumption (LGGthetas), but not under implication. However, there is a difference between theta-subsumption and implication, which sometimes causes LGGthetas to be overly general w.r.t. implication. We describe the well-known technique to compute LGGthetas, and the most important theoretical results connected with it. We study the theory of generalization under implication, and note that implication between clauses is undecidable. We therefore introduce a stronger form of implication, called T-implication, which is decidable between clauses. We show that for every finite set of clauses there exists a least general generalization under T-implication. We describe a technique to reduce generalizations under implication of a clause to generalizations under theta-subsumption, by replacing the clause with a set of clauses or-introduced from the clause by a sequence of literals. We also show how an or-introduced set of clauses equivalently can be described by a single clause, which we call an expansion of the original clause. Moreover we prove that for every non-tautological clause there exists a T-complete expansion, which means that every generalization under T-implication of the clause is reduced to a generalization under theta-subsumption of the expansion. We present a technique to compute generalizations under implication of a set of clauses by first computing common expansions of the clauses and then computing an LGGtheta of the common expansions. The computational complexity both of the well-known technique to compute LGGthetas and of our technique to compute generalizations under implication grows exponentially in the size of the input. %------------------------------------------------------------------------------% The following dissertation is available via anonymous FTP and through http://www.ics.uci.edu/~ali (either as a whole or by chapters). Title: "Learning Probabilistic Relational Concept Descriptions" Author: Kamal Ali University: Department of Information and Computer Science, University of California, Irvine Key words: Learning probabilistic concepts, multiple models, multiple classifiers, combining classifiers, evidence combination, relational learning, First-order learning, Noise-tolerant learning, Learning of small disjuncts, Inductive Logic Programming. A B S T R A C T This dissertation presents results in the area of multiple models (multiple classifiers), learning probabilistic relational (first order) rules from noisy, "real-world" data and reducing the small disjuncts problem - the problem whereby learned rules that cover few training examples have high error rates on test data. Several results are presented in the arena of multiple models. The multiple models approach in relevant to the problem of making accurate classifications in ``real-world'' domains since it facilitates evidence combination which is needed to accurately learn on such domains. It is also useful when learning from small training data samples in which many models appear to be equally "good" w.r.t. the given evaluation metric. Such models often have quite varying error rates on test data so in such situations, the single model method has problems. Increasing search only partly addresses this problem whereas the multiple models approach has the potential to be much more useful. The most important result of the multiple models research is that the *amount* of error reduction afforded by the multiple models approach is linearly correlated with the degree to which the individual models make errors in an uncorrelated manner. This work is the first to model the degree of error reduction due to the use of multiple models. It is also shown that it is possible to learn models that make less correlated errors in domains in which there are many ties in the search evaluation metric during learning. The third major result of the research on multiple models is the realization that models should be learned that make errors in a negatively-correlated manner rather than those that make errors in an uncorrelated (statistically independent) manner. The thesis also presents results on learning probabilistic first-order rules from relational data. It is shown that learning a class description for each class in the data - the one-per-class approach - and attaching probabilistic estimates to the learned rules allows accurate classifications to be made on real-world data sets. The thesis presents the system HYDRA which implements this approach. It is shown that the resulting classifications are often more accurate than those made by three existing methods for learning from noisy, relational data. Furthermore, the learned rules are relational and so are more expressive than the attribute-value rules learned by most induction systems. Finally, results are presented on the small-disjuncts problem in which rules that apply to rare subclasses have high error rates The thesis presents the first approach that is simultaneously successful at reducing the error rates of small disjucnts while also reducing the overall error rate by a statistically significant margin. The previous approach which aimed to reduce small disjunct error rates only did so at the expense of increasing the error rates of large disjuncts. It is shown that the one-per-class approach reduces error rates for such rare rules while not sacrificing the error rates of the other rules. The dissertation is approximately 180 pages long (single spaced) (~590K). ftp ftp.ics.uci.edu logname: anonymous password: your email address cd /pub/ali binary get thesis.ps.Z quit %------------------------------------------------------------------------------% %------------------------------------------------------------------------------% \documentstyle[fullpage]{article} \begin{document} \thispagestyle{empty} {\large\bf\center The Sixth International Workshop on\\ Inductive Logic Programming (ILP'96) \\} {\center 28-30 August, 1996 \\ Stockholm, Sweden \\} \vspace{.2in} This workshop is the sixth in a series of international workshops on Inductive Logic Programing. ILP'96 will be run in parallel with the Sixth International Workshop on Logic Program Synthesis and Transformation (LOPSTR'96). Papers should fit into one, or preferably more, of the following three areas. \begin{itemize} \item {\bf Theory.} Of particular interest are papers that either 1) prove new results concerning algorithms which use inductive learning to construct first or higher order logic descriptions or 2) reveal relationships to theoretical work done outside of ILP, especially work in program synthesis and transformation. \item {\bf Implementation.} Details of implemented inductive algorithms. Time complexity results should be included. \item {\bf Application.} Experimental results within one or more application areas should be tabulated with appropriate statistics. Sufficient details should be included to allow reproduction of results. Comparative studies of different algorithms running on the same examples, using the same background knowledge, are especially welcome, as are papers that explore new application areas for ILP. \end{itemize} ILP'96 and LOPSTR'96 will take place on board a ship which will sail from Stockholm to Helsinki and back during the workshop. \section*{Program Committee} \vspace{-.2in} \begin{tabbing} mmmmmmmmmmmmmmmm \= mmmmmmmmmmmmmmmm \= mmmmmmmmmmmmmmmm \= mmmmmmmmmmmmmmmm \kill \\ F. Bergadano \> P. Flach \> R. Mooney \> J.R. Quinlan \\ I. Bratko \> P. Idestam-Almquist \> S. Muggleton \> C. Rouveirol \\ L. De Raedt \> N. Lavra\v{c} \> M. Numao \> C. Sammut \\ S. D\v{z}eroski \> S. Matwin \> C.D. Page \> A. Srinivasan \\ \> \> \> S. Wrobel \\ \end{tabbing} \section*{Organization} \begin{tabbing} \= {\it Program Chair:} \= Stephen Muggleton \hspace{1.9in} \= {\it Local Chair:} \= Carl Gustaf Jansson \\ \>\>Oxford University Computing Laboratory \>\> University of Stockholm \\ \>\>Wolfson Building, Parks Road \>\> Email: calle@dsv.su.se \\ \>\>Oxford, OX1 3QD, U.K. \\ \>\>Email: steve@comlab.ox.ac.uk \\ \end{tabbing} \section*{Deadlines} Submissions (hardcopy only) must be received by the {\bf program chair} no later than {\bf 17 May, 1996}. Submissions should include the postal address and email address (if available) of each author; the first author will be used as the contact author unless otherwise specified. Authors will be informed of acceptance by {\bf 28 June, 1996}. Some or all of the papers accepted for presentation at ILP'96 will be selected for inclusion in a post-workshop publication, at the discretion of the program committee. Notification of acceptance for the post-workshop publication also will be made by {\bf 28 June, 1996}. Authors also will then be notified of the deadline for camera-ready copies, which will be no earlier than {\bf October 15, 1996}. \end{document} %------------------------------------------------------------------------------% %------------------------------------------------------------------------------% Call for Papers: ALT'96 The Seventh International Workshop on Algorithmic Learning Theory Coogee Holiday Inn, Sydney, Australia October 23-25, 1996 The 7th International Workshop on Algorithmic Learning Theory (ALT'96) will be held at the Coogee Holiday Inn, Sydney, Australia during October 23-25, 1996, and will be collocated with the Pacific Rim Knowledge Acquisition Workshop. The workshop is being sponsored by the Japanese Society for Artificial Intelligence (JSAI) and the University of New South Wales (UNSW). We invite submissions to ALT'96 in all areas related to algorithmic learning theory including (but not limited to): the design and analysis of learning algorithms, the theory of machine learning, computational logic of/for machine discovery, inductive inference, learning via queries, artificial and biological neural networks, pattern recognition, learning by analogy, Bayesian/MDL estimation, statistical learning, inductive logic programming, robotics, application of learning to databases, gene analysis, etc. INVITED TALKS: Invited talks will be given by Prof. J.R. Quinlan, (University of Sydney), Prof. T. Shinohara (Kyushu Institute of Technology), Prof. Les Valiant (Harvard Univ.), and Prof. Paul Vitanyi (CWI and Univ. of Amsterdam). SUBMISSIONS: Authors must submit nine copies of their extended abstracts to: Arun Sharma - ALT'96 School of Computer Science and Engineering University of New South Wales Sydney, 2052, Australia ABSTRACTS must be received by April 15, 1996. NOTIFICATION of acceptance or rejection will be mailed to the first (or designated) author by June 3, 1996. CAMERA-READY copy of accepted papers will be due July 1, 1996. FORMAT: The submitted abstract should consist of a cover page with title, authors' names, postal and e-mail addresses, an approximately 200 word summary, and a body not longer than ten (10) pages of size A4 or 7x10.5 inches in twelve-point font. Note that only the first ten (10) pages of the body will be sent out for review. Double-sided printing is strongly encouraged. POLICY: Each submitted abstract will be reviewed by the members of the program committee, and be judged on clarity, significance, and originality. Simultaneous submission of papers to any other conference with published proceedings is not allowed. Papers that have appeared in journals or other conferences are not appropriate for ALT'96. PROCEEDINGS will be published as a volume in the Lecture Notes Series in Artificial Intelligence from Springer-Verlag, and will be available at the conference. Selected papers of ALT'96 will be invited to be published in a special issue of a distinguished journal. CONFERENCE CHAIR: Prof. Setsuo Arikawa RIFIS, Kyushu University 33 Fukuoka, 812 Japan arikawa@rifis.kyushu-u.ac.jp PROGRAM COMMITTEE CHAIR: Arun Sharma, Univ. of New South Wales arun@cse.unsw.edu.au PROGRAM COMMITTEE: H. Arimura (KyuTech), Jose Balcazar (UPC, Barcelona), P. Bartlett (ANU), W. Cohen (AT&T), S. Ben David (Technion), H. Imai (U. Tokyo), K.P. Jantke (TH Leipzig), S. Kobayashi (U. Electro-Comm.), M. Numao (TiTech), S. Jain (National U. Singapore), S. Lange (TH Leipzig), L. De Raedt (Leuven), Y. Sakakibara (Fujitsu Labs) M. Sato (Osaka Pref. U.), O. Watanabe (TiTech), K. Yamanishi (NEC), T. Zeugmann (Kyushu) LOCAL ARRANGEMENTS CHAIR: Achim Hoffmann School of Computer Science and Engineering University of New South Wales Sydney 2052 Australia alt96@cse.unsw.edu.au For more information, contact: Email: alt96@cse.unsw.edu.au Homepage: http://www.cse.unsw.edu.au/~alt96/ %------------------------------------------------------------------------------% %------------------------------------------------------------------------------% IDAMAP-96 INTELLIGENT DATA ANALYSIS IN MEDICINE AND PHARMACOLOGY First Call for Papers for the Workshop at ECAI-96 12th European Conference on Artificial Intelligence August 12-16, 1996 Budapest, Hungary Organized by: Nada Lavrac, J. Stefan Institute, Slovenia (chair) Pedro Barahona, Universidade Nova de Lisboa, Portugal Riccardo Bellazzi, University of Pavia, Italy Werner Horn, Austrian Research Institute for Artificial Intelligence Elpida Keravnou, University of Cyprus (co-chair) Cristiana Larizza, University of Pavia, Italy Blaz Zupan, J. Stefan Institute, Slovenia (co-chair) GENERAL INFORMATION IDAMAP-96, an ECAI-96 workshop, will be held in Budapest, Hungary, on 13 August 1996, immediately before the main ECAI-96 conference, August 14-16, 1996. The workshop will last one full day. Gathering in an informal setting, workshop participants will have the opportunity to meet and discuss selected technical topics in an atmosphere which fosters the active exchange of ideas among researchers and practitioners. To encourage interaction and a broad exchange of ideas, the workshop will be kept small, preferably around 30 participants. TOPIC The gap between data generation and data comprehension is widening in all fields of human activity. In medicine and pharmacology overcoming this gap is particularly crucial since medical decision making needs to be supported by arguments based on basic medical and pharmacologi- cal knowledge as well as knowledge, regularities and trends extracted from data by intelligent data analysis techniques. The topic of the workshop are computational methods for intelligent data analysis aimed at narrowing the gap between data gathering and data comprehension, as well as their applications in medicine and pharmacology. Topics include, but are not limited to, effective machine learning tools, clustering, data visualization, interpretation of time-ordered data (derivation and revision of temporal trends and other forms of temporal data abstraction), learning with case bases, discovery of new diseases, new drug compounds, pharmacodynamical modelling, predicting drug activity, etc. Emphasis will also be given to solving of problems which result from automated data collection in modern hospitals, such as analysis of computer-based patient records (CPR), analysis of data from patient-data management system (PDMS), intelligent alarming, effective and efficient monitoring, etc. SCIENTIFIC PROGRAM The scientific program of the workshop will consist of presentations of accepted papers and panel discussions. Papers are invited both on methodological issues of data mining as well as on specific applications in medicine and pharmacology. The preferred length of papers is 10 pages. Panel discussions will consist of commentators' views on the presented papers as well as on discussions initialized by participants. In order to be able to organize these discussions, entries for discussions are encouraged on any topic related to the workshop. We especially encourage entries on the topic "Data mining and knowledge discovery - its practical potential in medicine and pharmacology". The preferred length of entries for panel discussions is 1 page. SUBMISSION OF PAPERS Submit papers (preferably 5 hard copies, 8-12 pages, possibly postscript) and panel discussion entries (hardcopy or electronic, 1 page) to: Nada Lavrac, Blaz Zupan J. Stefan Institute Jamova 39 61000 Ljubljana Slovenia tel. +386 61 177 3272, 177 3380 fax. +386 61 125 1038, 219 385 email: ecai96wk@ijs.si Submissions must include first author's complete contact information, including address, email, phone and fax. WORKSHOP PARTICIPATION Workshop participation is not limited to authors of submissions. A limited number of other attendees will be selected based on submitted statements of interest for participation at the workshop. A statement of interest (send an email to ecai96wk@ijs.si) should include the name, address, email, phone, fax and description of research interest. IMPORTANT DATES - Paper submission deadline April 2, 1996 - Notification to Authors April 26, 1996 - Camera-ready papers May 15, 1996 PUBLICATION OF PAPERS Accepted papers will be published in ECAI-96 working notes. It is planned to published a post-conference publication based on selected workshop papers. WORKSHOP FEE - Workshop fee is 50 ECU per participant. - Attendees at workshops must register also for the main ECAI conference. WORLD WIDE WEB (WWW) For up-to-date workshop information please check: http://www-ai.ijs.si/ailab/activities/idamap96.html %------------------------------------------------------------------------------% %------------------------------------------------------------------------------% KR'96 Pre-Conference Workshop on Relevance in Knowledge Representation and Reasoning 3-4 November, 1996 Boston, Massachusetts ------------------------------------------------------ C A L L F O R P A P E R S ------------------------------------------------------ http://www.research.att.com/orgs/ssr/people/levy/rrr-cfp.html ------------------------------------------------------ Essentially all reasoning systems use a corpus of information to reach appropriate conclusions. For example, deductive systems use initial theories (possibly encoded as predicate calculus statements) from which they draw conclusions, probabilistic systems use prior distributions (possibly encoded as a Bayesian network) to compute event probabilities, and abductive processes produce explanations based on both background theories and observations. With too little information, these systems clearly cannot work correctly. Surprisingly, too *much* information is also problematic, as it too can cause significant degradation in system performance. It is therefore critical to determine what information is irrelevant, to know what can be ignored or downplayed when considering a specific task (e.g., a specific query, or distribution of queries, to the system, or a specific observation to be explained). In some cases, ignoring irrelevant information is needed in order to draw the correct conclusions. There are many forms of irrelevance. In some contexts, the initial theory may include more information than the task requires, or information at a level of granularity that is more detailed than necessary. Here, the system may perform more effectively if it ignores or deletes certain irrelevant facts or if it ignores certain distinctions made in the representation. Another flavor of irrelevance arises during the course of reasoning: A reasoning process can ignore certain intermediate results, once it has established that they will not contribute to the eventual answer. This workshop follows the very eclectic 1994 Relevance Symposium, which investigated the notion of relevance across various fields of Artificial Intelligence and Computer Science. The current workshop, however, will focus on the use of relevance in knowledge representation and reasoning, specifically, on understanding different forms of irrelevance, and exploiting this "relevance information" to improve the performance of reasoning systems. Submissions are requested in areas relating to relevance in KR&R, including, but not limited to, the following: o Speeding up inference using relevance reasoning. o Relevance in probabilistic reasoning. o Relevance in explanation. o Relationships between relevance and belief revision and updates. o Relevance reasoning as a basis for abstraction and reformulation. o Using relevance of information to enable drawing appropriate conclusions. o Applications of relevance reasoning. o Reasoning about relevance of information, and foundations of relevance reasoning. Submission Information ====================== Authors wishing to present a paper should submit an extended abstract of at most 5000 words. Accepted participants will be invited to submit full papers for the workshop proceedings, which will be distributed to the workshop participants. Persons wishing to attend the workshop and not to present papers should submit a 1--2 page research summary that includes a list of relevant publications. Authors are encouraged to submit PostScript versions of their paper by email to either Russ Greiner (greiner@scr.siemens.com) or Alon Levy (levy@research.att.com). Authors unable to submit by email should send 4 copies of their paper to the address below. All submissions should be received by July 8, 1996. Please be sure to include e-mail address, telephone number and mailing address of the principal author. In case of multiple authors, please indicate which authors wish to participate. Notification of acceptance or rejection will be mailed to the principal author by August 16, 1996. Camera-ready copies of papers accepted for inclusion in the proceedings will be due September 17, 1996. Address for hardcopy submissions: Russell Greiner Siemens Corporate Research, Inc 755 College Road East Princeton, NJ 08540-6632 Important Dates =============== - Submissions due: July 8, 1996. - Notification of acceptance August 16, 1996. - Final version due September 17, 1996. - Workshop dates November 3-4, 1996. Program Chairs: =============== Russ Greiner (Siemens Corporate Research, greiner@scr.siemens.com) Alon Levy (AT&T Bell Laboratories, levy@research.att.com) Program Committee: ================== Adnan Darwiche (Rockwell) Jim Delgrande (Simon Frasier University) Daphne Koller (Stanford University) Gerhard Lakemeyer (University of Bonn) Alberto Mendelzon (University of Toronto) Devika Subramanian (Rice University) %------------------------------------------------------------------------------% %------------------------------------------------------------------------------% ILP tutorial @ ECAI Inductive Logic Programming Instructor: Dr. Stan Matwin Professor of Computer Science and Electrical Engineering University of Ottawa Canada Inductive Logic Programming (ILP) is a new, burgeoining field of AI, combining machine learning and logic programming. ILP learns relational (first order logic) concept descriptions from facts. One of the most active and innovative fields in AI, particulary in Europe, ILP can also be viewed as a technique of developing logic programs from known instances of their input-output behavior. ILP reaches beyond the limitations of inductive learning systems based on attribute-value representation of examples and concepts. The tutorial will clarify the goals and the motivations of ILP. Classical bottom-up and top-down methods for learning Horn clauses from examples will be described in a simplified form. We will discuss the priciples behind the successful ILP systems: FOIL and PROGOL. We will discuss how some ILP systems are capable of "creative" learning, going beyond the language in which the examples and the background knowledge are expressed. We will then survey the recent, successful applications of ILP in areas such as pharmaceutical design, music, protein structure prediction, CAD, natural language processing, etc. The tutorial does not assume any advanced background beyond the basic concepts in logic. S. Matwin is Professor of Computer Science and Electrical Engineering at the University of Ottawa, Canada, where he teaches machine learning, ILP, AI, and compiler construction. His research interests are in machine learning and its applications, with the special emphasis on ILP and environmental applications of AI. Stan has published more than 70 papers in journals and refereed international conferences. Member of the program committee of a number of conferences in machine learning, he is also the president of the Canadian Society for Computational Studies of Intelligence, and chair of IFIP Working Group 12.2 (Machine Learning). He has taught an ILP tutorial at AAAI-94 and IJCAI95. Check http://www.dfki.uni-sb.de/ecai96/tutorials for details of regfistration etc. %------------------------------------------------------------------------------% %------------------------------------------------------------------------------% Announcing: new version of Mobal available Mobal The knowledge acquisition and machine learning system MOBAL (release 4.1b9) is available free for non-commercial academic use from the anonymous ftp-server 'ftp.gmd.de' in the directory 'gmd/mlt/Mobal'. The system requires a Sun Sparc Station, SunOS 4.1 and X11R5 or later. The user interface is implemented with Tcl/Tk --- all you need to run Mobal is a Sun and X11. The information included below, plus a little more, is more colorfully available at: http://nathan.gmd.de/projects/ml/mobal.html About Mobal 4.1b9 Mobal is a sophisticated system for developing operational models of application domains in a relational knowledge representation. It integrates a manual knowledge acquisition and inspection environment, a powerful inference engine, machine learning methods for automated knowledge acquisition, a knowledge revision tool, and -- this is the new bit -- a host of services related to the topic of Theory Restructuring: * various forms of redundancy analysis & elimination * methods & strategies for changing inferential structure (folding & unfolding) * evaluation criteria for comparing empirically equivalent but syntactically different forms of a theory (such as those produced by applying one or more restructuring operators) * miscellaneous analysis, restructuring and cleanup services: o detailed overview over form & content of the theory (pos, neg, covered, uncovered instances; statistics on the number of rules, predicates, facts, sorts, integrity constraints, metarules & -facts, etc.; focus-able text and graphical views of rules, facts, predicates, etc.) o hypothetical reasoning, esp. explanation of failure to cover o detection of non-generative rules & suggested fix o detection of unused predicates o determination of minimum required inputs relative to a given goal concept See also http://nathan.gmd.de/persons/edgar.sommer/scientific.html By using Mobal's knowledge acquisition environment, you can incrementally develop a model of your domain in terms of logical facts and rules. You can inspect the knowledge you have entered in text or graphics windows, augment the knowledge, or change it at any time. The built-in inference engine can immediately execute the rules you have entered to show you the consequences of your inputs, or answer queries about the current knowledge. Mobal also builds a dynamic sort taxonomy from your inputs. If you wish, you can use machine learning methods to automatically discover additional rules based on the facts that you have entered, or to form new concepts. (Mobal can be used as a front-end for any induction algo that runs under SunOS & can do i/o via files -- if you have one we don't, consult the example interfaces that come with the distribution (in the tools dir), modify to your needs, and send us a message.) If there are contradictions in the knowledge base due to incorrect rules or facts, there is a knowledge revision tool to help you locate the problem and fix it. User Guide MOBAL's User Guide has completely reworked and extended for the new release. A draft of this version is browse-able: http://nathan.gmd.de/projects/ml/mobal.html ... and is part of the distribution package, so if you're getting that, you do not need to get the user guide separately. Acknowledgments Mobal is a result of research funded in part by the European Community within the type B ESPRIT Project 2154 "Machine Learning Toolbox" and the ESPRIT Project "Inductive Logic Programming" (ILP, PE 6020) and is based on the System BLIP developed in the project "Lerner" at the Technical University Berlin funded by the German government (BMFT) under contract ITW8501B1, and is further made possible by the existence of coffee. Thanks! Restrictions MOBAL is available in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of FITNESS FOR A PARTICULAR PURPOSE. MOBAL can be used free of charge for academic, educational, or non-commercial uses. We do emphatically request, however, that you send us mail (mobal@gmd.de) so we know where MOBAL is going. Warm words in closing Sadly, work on Mobal is not currently at the apex of our official duties, so we may be slow in responding to Stupid Questions(sm), but promise never to get angry. Please, however, consider RTFM'ing (see pointers to animals called "user guide" & "web pages" above) before attempting to grab our attention with -- examples picked at random -- requests for porting several MB's worth of code to DOS/286 or ZX Spectrum. We are currently fiddling around with an unmoderated mailing list for Mobalites; if you use Mobal, consider sending mail to majordomo@gmd.de, with content subscribe mobal-list Let us know if you are doing anything interesting with Mobal! Cheers, the MLGroup@GMD %------------------------------------------------------------------------------% From scheier@ifi.unizh.ch Tue Feb 20 20:27:04 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 20 Feb 96 20:27:02 -0600; AA28214 Message-Id: <9602210227.AA18028@lucy.cs.wisc.edu> Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 20 Feb 96 20:27:00 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa09176; 20 Feb 96 20:10:27 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa09174; 20 Feb 96 19:39:10 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa04316; 20 Feb 96 19:38:57 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa17069; 19 Feb 96 4:28:57 EST Received: from josef.ifi.unizh.ch by EDRC.CMU.EDU id aa05426; 19 Feb 96 4:28:36 EST Received: from ifi.unizh.ch by josef.ifi.unizh.ch id <02431-0@josef.ifi.unizh.ch>; Mon, 19 Feb 1996 10:28:35 +0100 To: Connectionists@cs.cmu.edu, neuron@CATTELL20.psych.upenn.edu Subject: Papers on Categorization in Autonomous Agents using Neural Networks Date: Mon, 19 Feb 1996 10:28:35 +0100 From: Christian Scheier Sender: scheier@ifi.unizh.ch The following papers deal with the problem of categorization/object recognition in autonomous agents (mobile robots). The papers can be retrieved from: ftp://claude.ifi.unizh.ch/pub/institute/ailab/techreports/ 96_01.ps.gz: Categorization in a real-world agent using haptic exploration and active perception Scheier, C. and Lambrinos, D. ABSTRACT An agent in the real world has to be able to make distinctions between different types of objects, i.e. it must have the competence of categorization. In mobile agents categorization is hard to achieve because there is a large variation in proximal sensory stimulation originating from the same object. In this paper we extend previous work on adaptive categorization in autonomous agents. The main idea of our approach is to include the agent's own actions into the classification process. In the experiments presented in this paper an agent equipped with an active vision and an arm-gripper system has to collect certain types of objects. The agent learns about the objects by actively exploring them. This exploration results in visual and haptic information that is used for learning. In essence, the categorization comes about via evolving reentrant connections between the haptic and the visual system. Results on the behavioral performance as well as the underlying internal dynamics are presented. 95_12.ps.gz: Adaptive Classification in Autonomous Agents Scheier, C. and Lambrinos, D. ABSTRACT One of the fundamental tasks facing autonomous robots is to reduce the many degrees of freedom of the input space by some sorts of classification mechanism. The sensory stimulation caused by one and the same object, for instance, varies enormously depending on lighting conditions, distance from object, orientation and so on. Efforts to solve this problem, say in classical computer vision, have only had limited success. In this paper a new approach towards classification in autonomous robots is proposed. It's cornerstone is the integration of the robots own actions into the classification process. More specifically, correlations through time-linked independent samples of sensory stimuli and of kinesthetic signals produced by self-motion of the system form the basis of the category learning. Thus, it is suggested that classification should not be seen as an isolated perceptual (sub-)system but rather as a {\it sensory-motor coordination} which comes about through a self-organizing process. These ideas are illustrated with a case study of an autonomous system that has to learn to distinguish between different types of objects. 95_05.ps.gz: Classification as Sensory-Motor Coordination: A Case Study on Autonomous Agents. Scheier, C. and Pfeifer, R. ABSTRACT In psychology classification is studied as a separate cognitive capacity. In the field of autonomous agents the robots are equipped with perceptual mechanisms for classifying objects in the environment, either by preprogramming or by some sorts of learning mechanisms. One of the well-known hard and fundamental problems is the one of perceptual aliasing, i.e. that the sensory stimulation caused by one and the same object varies enormously depending on distance from object, orientation, lighting conditions, etc. Efforts to solve this problem, say in classical computer vision, have only had limited success. In this paper we argue that classification cannot be viewed as a separate perceptual capacity of an agent but should be seen as a sensory-motor coordination which comes about through a self-organizing process. This implies that the whole organism is involved, not only sensors and neural circuitry. In this perspective, ``action selection'' becomes an integral part of classification. These ideas are illustrated with a case study of a robot that learns to distinguish between graspable and non-graspable pegs. For further informations and papers contact: -- __________________________________________________________________________ Christian Scheier Computer Science Department AI Lab University of Zurich tel: +41-1-257-4575 Winterthurerstrasse 190 fax: +41-1-363-0035 CH-8057 Switzerland http://josef.ifi.unizh.ch/groups/ailab/people/scheier.html ______________________________________ ____________________________________ From istvan@psych.ualberta.ca Wed Feb 21 14:10:36 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 21 Feb 96 14:10:34 -0600; AA19359 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 21 Feb 96 14:10:30 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa09205; 20 Feb 96 20:20:31 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa09178; 20 Feb 96 19:40:05 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa04321; 20 Feb 96 19:39:12 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id ao28409; 19 Feb 96 19:51:40 EST Received: from [129.128.52.36] by RI.CMU.EDU id aa01205; 19 Feb 96 19:26:44 EST Received: from bcp by count.psych.ualberta.ca id <51279-1>; Mon, 19 Feb 1996 17:26:35 -0700 Date: Mon, 19 Feb 1996 17:23:42 -0700 From: Istvan Berkeley Subject: Intl. NN Workshop Announcement To: connectionists@cs.cmu.edu Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII APPLICATIONS OF CONNECTIONISM IN COGNITIVE SCIENCE: AN INTERNATIONAL WORKSHOP On May 25-27, 1996, there will be a major international workshop at Carleton University, Ottawa, Canada, on the latest connectionist modelling techniques in cognitive science. The list of presenters, given below, includes some of the founders of contemporary PDP techniques, as well as younger researchers whose work stands at the forefront of new approaches and applications. Along with formal presentations, mornings will be devoted to demonstrations of the newest PDP software of potential interest to cognitive scientists, for which purpose each participant will have access to a workstation. Principal speakers: David E. Rummelhart, Stanford University Jerome A. Feldman, University of California at Berkeley Paul Skokowski, Stanford University Christopher Thornton, University of Sussex John Bullinaria, Edinburgh University Malcolm Forster, University of Wisconsin at Madison Istvan Berkeley, University of Alberta Each talk will be followed by an arranged commentary, and general discussion. For further information on registration procedures, fees and accommodations, please contact either Andrew Brook Department of Interdisciplinary Studies Carleton University Ottawa, Ontario CANADA K1S 5B6 or Don Ross Department of Philosophy Morisset Hall University of Ottawa Ottawa, Ontario CANADA K1N 6N5 Istvan S. N. Berkeley, email: istvan@psych.ualberta.ca Biological Computation Project & Department of Philosophy, c/o 4-108 Humanities Center University of Alberta Edmonton, Alberta Tel: +1 403 436 4182 T6G 2E5, Canada Fax: +1 403 492 9160 From imlm@tuck.cs.fit.edu Wed Feb 21 14:10:49 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 21 Feb 96 14:10:38 -0600; AA19365 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 21 Feb 96 14:10:35 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab09205; 20 Feb 96 20:21:44 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab09178; 20 Feb 96 19:40:18 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa04325; 20 Feb 96 19:39:52 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa29817; 19 Feb 96 22:17:50 EST Received: from tuck.cs.fit.edu by RI.CMU.EDU id aa01806; 19 Feb 96 22:16:54 EST Received: (from imlm@localhost) by tuck.cs.fit.edu (8.6.11/8.6.11) id WAA13801; Mon, 19 Feb 1996 22:12:15 -0500 Date: Mon, 19 Feb 1996 22:12:15 -0500 Message-Id: <199602200312.WAA13801@tuck.cs.fit.edu> From: IMLM Workshop (pkc) To: ml@ics.uci.edu, kdd@gte.com, INDUCTIVE@hermes.csd.unb.ca, DAI-List@ece.sc.edu, GA-List@AIC.NRL.NAVY.MIL, Connectionists@cs.cmu.edu, ai-stats@watstat.uwaterloo.ca, hybrid-list@cs.ua.edu, colt@cs.uiuc.edu, genetic-programming@cs.stanford.edu, dbworld@lucy.cs.wisc.edu Cc: imlm@tuck.cs.fit.edu, sal@cs.columbia.edu, dhw@santafe.edu Subject: 2nd CFP: AAAI-96 Workshop on Integrating Multiple Learned Models ********************************************************************* Paper submission deadline: March 18, 1996 ********************************************************************* CALL FOR PAPERS/PARTICIPATION INTEGRATING MULTIPLE LEARNED MODELS FOR IMPROVING AND SCALING MACHINE LEARNING ALGORITHMS to be held in conjunction with AAAI 1996 (collocated with KDD-96, UAI-96, and IAAI-96) Portland, Oregon August 1996 Most modern machine learning research uses a single model or learning algorithm at a time, or at most selects one model from a set of candidate models. Recently however, there has been considerable interest in techniques that integrate the collective predictions of a set of models in some principled fashion. With such techniques often the predictive accuracy and/or the training efficiency of the overall system can be improved, since one can "mix and match" among the relative strengths of the models being combined. The goal of this workshop is to gather researchers actively working in the area of integrating multiple learned models, to exchange ideas and foster collaborations and new research directions. In particular, we seek to bring together researchers interested in this topic from the fields of Machine Learning, Knowledge Discovery in Databases, and Statistics. Any aspect of integrating multiple models is appropriate for the workshop. However we intend the focus of the workshop to be improving prediction accuracies, and improving training performance in the context of large training databases. More precisely, submissions are sought in, but not limited to, the following topics: 1) Techniques that generate and/or integrate multiple learned models. In particular, techniques that do so by: * using different training data distributions (in particular by training over different partitions of the data) * using different output classification schemes (for example using output codes) * using different hyperparameters or training heuristics (primarily as a tool for generating multiple models) 2) Systems and architectures to implement such strategies. In particular: * parallel and distributed multiple learning systems * multi-agent learning over inherently distributed data A paper need not be submitted to participate in the workshop, but space may be limited so contact the organizers as early as possible if you wish to participate. The workshop format is planned to encompass a full day of half hour presentations with discussion periods, ending with a brief period for summary and discussion of future activities. Notes or proceedings for the workshop may be provided, depending on the submissions received. Submission requirements: i) A short paper of not more than 2000 words detailing recent research results must be received by March 18, 1996. ii) The paper should include an abstract of not more than 150 words, and a list of keywords. Please include the name(s), email address(es), address(es), and phone number(s) of the author(s) on the first page. The first author will be the primary contact unless otherwise stated. iii) Electronic submissions in postscript or ASCII via email are preferred. Three printed copies (preferrably double-sided) of your submission are also accepted. iv) Please also send the title, name(s) and email address(es) of the author(s), abstract, and keywords in ASCII via email. Submission address: imlm@cs.fit.edu Philip Chan IMLM Workshop Computer Science Florida Institute of Technology 150 W. University Blvd. Melbourne, FL 32901-6988 407-768-8000 x7280 (x8062) 407-984-8461 (fax) Important Dates: Paper submission deadline: March 18, 1996 Notification of acceptance: April 15, 1996 Final copy: May 13, 1996 Chairs: Salvatore Stolfo, Columbia University sal@cs.columbia.edu David Wolpert, Santa Fe Institute dhw@santafe.edu Philip Chan, Florida Institute of Technology pkc@cs.fit.edu General Inquiries: Please address general inquiries to one of the chairs or send them to: imlm@cs.fit.edu Up-to-date workshop information is maintained on WWW at: http://www.cs.fit.edu/~imlm/ or http://cs.fit.edu/~imlm/ From zhuh@helios.aston.ac.uk Wed Feb 21 18:59:05 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 21 Feb 96 18:59:03 -0600; AA24944 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 21 Feb 96 18:59:00 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ac09205; 20 Feb 96 20:23:38 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa09180; 20 Feb 96 19:43:38 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa04334; 20 Feb 96 19:43:06 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa07201; 20 Feb 96 11:55:37 EST Received: from email.aston.ac.uk by CS.CMU.EDU id aa01031; 20 Feb 96 11:54:09 EST Received: from sun.aston.ac.uk (actually host darwin.aston.ac.uk) by email.aston.ac.uk with SMTP (PP); Tue, 20 Feb 1996 16:51:34 +0000 From: zhuh Date: Tue, 20 Feb 1996 16:48:58 +0000 Message-Id: <8961.9602201648@sun.aston.ac.uk> To: connectionists@cs.cmu.edu Subject: Paper available: cross validation X-Sun-Charset: US-ASCII Content-Length: 1152 FTP-host: cs.aston.ac.uk FTP-file: neural/zhuh/nflcv.ps.Z URL: ftp://cs.aston.ac.uk/neural/zhuh/nflcv.ps.Z To appear in Neural Computation. ================================================================= No Free Lunch For Cross Validation Huaiyu Zhu and Richard Rohwer Neural Computing Research Group Aston University, Birmingham B4 7ET, UK Abstract -------- It is known theoretically that an algorithm cannot be good for an arbitrary prior. We show that in practical terms this also applies to the technique of ``cross validation'', which has been widely regarded as defying this general rule. Numerical examples are analysed in detail. Their implications to researches on learning algorithms are discussed. ================================================================= -- Huaiyu Zhu, PhD email: H.Zhu@aston.ac.uk Neural Computing Research Group http://neural-server.aston.ac.uk/People/zhuh Dept of Computer Science ftp://cs.aston.ac.uk/neural/zhuh and Applied Mathematics tel: +44 121 359 3611 x 5427 Aston University, fax: +44 121 333 6215 Birmingham B4 7ET, UK From guy@taco.mpik-tueb.mpg.de Thu Feb 22 00:33:46 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 22 Feb 96 00:33:42 -0600; AA00658 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 22 Feb 96 00:33:39 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa11159; 21 Feb 96 18:41:13 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa11157; 21 Feb 96 18:29:12 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa05496; 21 Feb 96 18:28:35 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa24469; 21 Feb 96 7:31:34 EST Received: from listserv.gmd.de by EDRC.CMU.EDU id aa03772; 21 Feb 96 7:29:52 EST Received: from mailer.mpib-tuebingen.mpg.de by listserv.gmd.de (LSMTP for OpenVMS v1.0a) with SMTP id 1EB5CB49 ; Wed, 21 Feb 1996 13:29:41 +0100 Received: from bogart.mpib-tuebingen.mpg.de by mailer.mpib-tuebingen.mpg.de; (5.65v3.2/1.1.8.2/03Nov95-0912AM) id AA28348; Wed, 21 Feb 1996 13:34:31 GMT Received: from taco.mpik-tueb.mpg.de by bogart.mpib-tuebingen.mpg.de; (5.65/1.1.8.2/03Nov95-0442PM) id AA01999; Wed, 21 Feb 1996 13:31:39 GMT Received: by taco (940816.SGI.8.6.9/930416.SGI) for Connectionists@cs.cmu.edu id NAA00184; Wed, 21 Feb 1996 13:29:38 +0100 From: "Guy M. Wallis" Message-Id: <9602211329.ZM182@taco.mpik-tueb.mpg.de> Date: Wed, 21 Feb 1996 13:29:37 +0000 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: Connectionists@cs.cmu.edu Subject: Papers available: "Object recognition and unsupervised learning" Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/wallisgm.ittrain.ps.Z FTP-filename: /pub/neuroprose/wallisgm.temporalobjrec1.ps.Z FTP-filename: /pub/neuroprose/wallisgm.temporalobjrec2.ps.Z ** Three papers available on the unsupervised ** ** learning of invariant object recognition ** The three papers listed above are now available for retrieval from the Neuroprose repository. All three papers discuss learning to associate different views of objects on the basis of their appearance in time as well as their spatial appearance. The papers are also available directly from my home page, along with a copy of my PhD thesis which, I should warn you, is rather long: http://www.mpik-tueb.mpg.de/people/personal/guy/guy.html -------------------------------------------------------------------------------- PaperI: A Model of Invariant Object Recognition in the Visual System ABSTRACT Neurons in the ventral stream of the primate visual system exhibit responses to the images of objects which are invariant with respect to natural transformations such as translation, size, and view. Anatomical and neurophysiological evidence suggests that this is achieved through a series of hierarchical processing areas. In an attempt to elucidate the manner in which such representations are established, we have constructed a model of cortical visual processing which seeks to parallel many features of this system, specifically the multi-stage hierarchy with its topologically constrained convergent connectivity. Each stage is constructed as a competitive network utilising a modified Hebb-like learning rule, called the trace rule, which incorporates previous as well as current neuronal activity. The trace rule enables neurons to learn about whatever is invariant over short time periods (e.g. 0.5 s) in the representation of objects as the objects transform in the real world. The trace rule enables neurons to learn the statistical invariances about objects during their transformations, by associating together representations which occur close together in time. We show that by using the trace rule training algorithm the model can indeed learn to produce transformation invariant responses to natural stimuli such as faces. Submitted to Journal of Computational Neuroscience 32 pages 1.6 Mb compressed ftp://archive.cis.ohio-state.edu/pub/neuroprose/wallisgm.ittrain.ps.Z or ftp://ftp.mpik-tueb.mpg.de/pub/guy/jcns7.ps.Z -------------------------------------------------------------------------------- PaperII: Optimal, Unsupervised Learning in Invariant Object Recognition ABSTRACT A means for establishing transformation invariant representations of objects at the single cell level is proposed and analysed. The association of views of objects is achieved by using both the temporal order of the presentation of these views, as well as their spatial similarity. Assuming knowledge of the distribution of presentation times, an optimal linear learning rule is derived. If we assume that objects are viewed with presentation times that are approximately Jeffrey's distributed, then the optimal learning rule is very well approximated using a simple exponential temporal trace. Simulations of a competitive network trained on a character recognition task are then used to highlight the success of this learning rule in relation to simple Hebbian learning, and to show that the theory can give quantitative predictions for the optimal parameters for such networks. Submitted to Neural Computation 15 pages 180 Kb compressed ftp://archive.cis.ohio-state.edu/pub/neuroprose/wallisgm.temporalobjrec1.ps.Z or ftp://ftp.mpik-tueb.mpg.de/pub/guy/nc.ps.Z -------------------------------------------------------------------------------- PaperIII: Using Spatio-Temporal Correlations to Learn Invariant Object Recognition ABSTRACT A competitive network is described which learns to classify objects on the basis of temporal as well as spatial correlations. This is achieved by using a Hebb-like learning rule which is dependent upon prior as well as current neural activity. The rule is shown to be capable of outperforming a supervised rule on the cross-validation test of an invariant character recognition task, given a relatively small training set. It is also shown to outperform the supervised version of Fukushima's Neocognitron, on a larger training set. Submitted to Neural Networks 13 pages 110 Kb compressed ftp://archive.cis.ohio-state.edu/pub/neuroprose/wallisgm.temporalobjrec2.ps.Z or ftp://ftp.mpik-tueb.mpg.de/pub/guy/nn.ps.Z -------------------------------------------------------------------------------- -- ----------------------------------------------------------- _/ _/ _/_/_/ _/_/_/ Guy Wallis _/_/ _/_/ _/ _/ _/ Max-Planck Institut f"ur _/ _/ _/ _/_/_/ _/ Biologische Kybernetik _/ _/ _/ _/ Spemannstr. 38 _/ _/ _/ _/_/_/ 72076 T"ubingen, Germany http://www.mpik-tueb.mpg.de/ TEL: +49-7071/601-630 Email: guy@mpik-tueb.mpg.de FAX: +49-7071/601-575 ----------------------------------------------------------- From jfeldman@ICSI.Berkeley.EDU Thu Feb 22 00:33:51 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 22 Feb 96 00:33:45 -0600; AA00666 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 22 Feb 96 00:33:41 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa11242; 21 Feb 96 18:49:07 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa11169; 21 Feb 96 18:30:02 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa05510; 21 Feb 96 18:29:25 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa27916; 21 Feb 96 11:13:30 EST Received: from icsia.ICSI.Berkeley.EDU by RI.CMU.EDU id aa09539; 21 Feb 96 11:12:56 EST Received: from waffle.ICSI.Berkeley.EDU (jfeldman@waffle.ICSI.Berkeley.EDU [128.32.201.183]) by icsia.ICSI.Berkeley.EDU (8.6.12/HUB+V8$Revision: 1.23 $) with ESMTP id IAA12668 for ; Wed, 21 Feb 1996 08:12:53 -0800 Received: (jfeldman@localhost) by waffle.ICSI.Berkeley.EDU (8.6.12/1.8) id IAA26440 for Connectionists@cs.cmu.edu; Wed, 21 Feb 1996 08:12:46 -0800 From: Jerry Feldman Message-Id: <9602210812.ZM26438@ICSI.Berkeley.edu> Date: Wed, 21 Feb 1996 08:12:42 -0800 X-Mailer: Z-Mail (3.2.0 06sep94) To: Connectionists@cs.cmu.edu Subject: shift invariance Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Shift invariance is the ability of a neural system to recognize a pattern independent of where appears on the retina. It is generally understood that this property can not be learned by neural network methods, but I have not seen a published proof. A "local" learning rule is one that updates the input weights of a unit as a function of the unit's own activity and some performance measure for the network on the training example. All biologically plausible learning rules, as well as all backprop variants, are local in this sense. It is easy to show that no local rule can learn shift invariance. Consider learning binary strings with one occurrence of the sequence 101 and otherwise all zeros. First consider length 5; there are only 3 positive examples: 10100 01010 00101 Suppose that the training data does not include the middle example. The two positive training examples have no 1's in common with the withheld example. There will be no positive examples with a 1 in position 2 or 4 and many negative examples. Thus there is no correlation between a 1 in position 2 or 4 and a good example so no local training rule will learn the correct classification. A similar argument extends to binary strings of arbitrary length so an arbitrarily small fraction of the training data can be omitted and still no local updating rules will suffice to learn shift invariance. The one dimensional case of shift invariance can be handled by treating each string as a sequence and learning a finite-state acceptor. But the methods that work for this are not local or biologically plausible and don't extend to two dimensions. The unlearnability of shift invarince is not a problem in practice because people use preprocessing, weight sharing or other techniques to get shift invariance where it is known to be needed. However, it does pose a problem for the brain and for theories that are overly dependent on learning. From wimw@mbfys.kun.nl Thu Feb 22 21:54:06 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 22 Feb 96 21:54:03 -0600; AA21361 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 22 Feb 96 21:54:00 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab12421; 22 Feb 96 10:51:19 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab12419; 22 Feb 96 10:32:03 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa06337; 22 Feb 96 10:31:29 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa15355; 22 Feb 96 9:18:17 EST Received: from septimius.mbfys.kun.nl by RI.CMU.EDU id aa14265; 22 Feb 96 9:17:42 EST Received: from dontcare by septimius.mbfys.kun.nl via majoria.mbfys.kun.nl [131.174.173.212] with ESMTP id PAA04038 (8.6.10/2.4) for ; Thu, 22 Feb 1996 15:18:23 +0100 From: Wim Wiegerinck Message-Id: <199602221418.PAA04038@septimius.mbfys.kun.nl> Subject: Paper Available: How Dependencies between Successive Examples Affect On-Line Learning. To: Connectionists@cs.cmu.edu Date: Thu, 22 Feb 1996 15:18:27 +0100 (MET) X-Mailer: ELM [version 2.4 PL23] Content-Type: text Content-Length: 2015 Dear Connectionists, The following paper, which is accepted for publication in Neural Computation, is available at URL ftp://ftp.mbfys.kun.nl/snn/pub/reports/Wiegerinck.4.ps.Z How Dependencies between Successive Examples Affect On-Line Learning. ABSTRACT: We study the dynamics of on-line learning for a large class of neural networks and learning rules, including backpropagation for multilayer perceptrons. In this paper, we focus on the case where successive examples are dependent, and we analyze how these dependencies affect the learning process. We define the representation error and the prediction error. The representation error measures how well the environment is represented by the network after learning. The prediction error is the average error which a continually learning network makes on the next example. In the neighborhood of a local minimum of the error surface, we calculate these errors. We find that the more predictable the example presentation,the higher the representation error, i.e. the less accurate the asymptotic representation of the whole environment. Furthermore we study the learning process in the presence of a plateau. Plateaus are flat spots on the error surface, which can severely slow down the learning process. In particular, they are notorious in applications with multilayer perceptrons. Our results, which are confirmed by simulations of a multilayer perceptron learning a chaotic time series using backpropagation, explain how dependencies between examples can help the learning process to escape from a plateau. FTP INSTRUCTIONS unix% ftp ftp.mbfys.kun.nl (or 131.174.83.52) Name: anonymous Password: (use your e-mail address) ftp> cd snn/pub/reports/ ftp> binary ftp> get Wiegerinck.4.ps.Z ftp> bye unix% uncompress Wiegerinck.4.ps.Z unix% lpr Wiegerinck.4.ps Wim Wiegerinck Foundation for Neural Networks Department of Medical Physics and Biophysics, University of Nijmegen, The Netherlands mailto:wimw@mbfys.kun.nl URL http://www.mbfys.kun.nl/~wimw/ From robbie@bcs.rochester.edu Thu Feb 22 21:54:09 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 22 Feb 96 21:54:06 -0600; AA21367 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 22 Feb 96 21:54:02 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ac12421; 22 Feb 96 10:52:19 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ac12419; 22 Feb 96 10:32:04 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa06342; 22 Feb 96 10:31:46 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa15960; 22 Feb 96 10:06:03 EST Received: from broca.bcs.rochester.edu by EDRC.CMU.EDU id aa09895; 22 Feb 96 10:05:29 EST Received: (robbie@localhost) by broca.bcs.rochester.edu (8.6.10/8.6.4) id KAA04738 for Connectionists@CS.CMU.EDU; Thu, 22 Feb 1996 10:04:38 -0500 Date: Thu, 22 Feb 1996 10:04:38 -0500 From: Robbie Jacobs Message-Id: <199602221504.KAA04738@broca.bcs.rochester.edu> To: Connectionists@cs.cmu.edu Subject: undergraduate summer workshop Below is an announcement for a three-day summer workshop for undergraduate students interested in the brain and cognitive sciences. It would be appreciated if you could bring this to the attention of your students. Robert Jacobs =================================================================== UNDERGRADUATE WORKSHOP ON PERCEPTION, PLASTICITY, AND ACTION (FELLOWSHIPS AVAILABLE) August 8-10, 1996 University of Rochester Department of Brain and Cognitive Sciences, Center for Visual Science, and the Program in Neuroscience The University of Rochester will host a Summer Workshop for Undergraduates on August 8-10, 1996. The Workshop will consist of lectures and laboratory demonstrations by the Rochester faculty on the coordination of perceptual mechanisms with those that control movement. It will also focus on how this coordination is established during development and modified by experience. These issues will be approached from neural, behavioral, and computational perspectives. Fellowships covering travel and living expenses will be provided for 20 students. Preference will be given to Juniors who plan to pursue advanced study in the behavioral and neural sciences. Request an application: * by email: judy@cvs.rochester.edu * by phone: (716) 275-2459 or fax: (716) 271-3043 * in writing: Dr. David R. Williams, Brain and Cognitive Sciences, Meliora Hall, University of Rochester, Rochester, NY 14627 * or apply electronically by visiting the web site at: http://www.bcs.rochester.edu/ug_workshop/ From hinton@cs.toronto.edu Thu Feb 22 21:54:11 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 22 Feb 96 21:54:08 -0600; AA21372 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 22 Feb 96 21:54:05 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa12438; 22 Feb 96 11:03:30 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa12423; 22 Feb 96 10:32:55 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa06349; 22 Feb 96 10:32:28 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa16240; 22 Feb 96 10:28:02 EST Received: from yonge.cs.toronto.edu by EDRC.CMU.EDU id aa09990; 22 Feb 96 10:26:53 EST Received: from neuron.ai.toronto.edu ([128.100.3.14]) by yonge.cs.toronto.edu with SMTP id <86537>; Thu, 22 Feb 1996 10:26:44 -0500 Received: from localhost by neuron.ai.toronto.edu with SMTP id <882>; Thu, 22 Feb 1996 10:25:48 -0500 To: Jerry Feldman Cc: Connectionists@cs.cmu.edu Subject: Re: shift invariance In-Reply-To: Your message of Wed, 21 Feb 1996 11:12:42 -0500. Date: Thu, 22 Feb 1996 10:25:32 -0500 From: Geoffrey Hinton Message-Id: <96Feb22.102548edt.882@neuron.ai.toronto.edu> Dear Jerry, Its a long time since we had a really good disagreement. Contrary to your assertions, shift invariance can be learned by backpropagation. It was one of the problems that I tried when fiddling about with backprop in the mid 80's. I published a paper demonstrating this in an obscure conference proceedings: Hinton, G.~E. (1987) Learning translation invariant recognition in a massively parallel network. In Goos, G. and Hartmanis, J., editors, PARLE: Parallel Architectures and Languages Europe, pages~1--13, Lecture Notes in Computer Science, Springer-Verlag, Berlin. So far as I know, this is also the first paper to demonstrate that weight decay can make a really big difference in generalization performance. It reduced the error rate from about 45% to about 6%, though I must confess that the amount of weight decay was determined by using the test set (as was usual in our sloppy past). I used a one dimensional "retina" with 12 pixels. On this retina there was one instance of a shape at a time. The "shape" consisted of two bright "boundary" pixels with 4 pixels between them. The 4 pixels in the sandwich could have any of the 16 binary patterns, so there were 16 very confusable shapes. For example, here are two instances of the shape corresponding to the binary number 0011 (strip off the boundary bits before reading the number): 000100111000 010011100000 The retina had wraparound so that each shape could occur in 12 different positions. This seems to me to be exactly the kind of data that you think cannot be learned. In other words, if I train a neural network to identitfy some instances of the shapes you think that it couldnt possibly generalize to instances in other positions. Of course, once you understand why it can generalize, you will decide on a way to exclude this kind of example, but so far it seems to me to fit your assertion about what cannot be done. The network had two hidden layers. In the first hidden layer there were 60 units divided into 12 groups of 5 with local receptive fields. So we are telling it about locality, but not about translation. Within each group, all 5 units receive input from the same 6 adjacent input units. In the next hidden layer there is a bottleneck of only 6 units (I didnt dare use 4), so all the information used to make the final decision has to be represented in a distributed pattern of activity in the bottleneck. There are 16 output units for the 16 shapes. The idea behind the network is as follows: Shapes are composed of features that they share with other shapes. Although we may not have seen a particular shape in a novel position, we will presumably have seen its features in those positions before. So if we have already developed translation invariant feature detectors, and if we represent our knowledge of the shape in terms of these detectors, we can generalize across translation. The "features" in this example are the values of the four pixels inside the sandwich. A hidden unit in the first hidden layer can see the whole sandwich, so it could learn to respond to the conjunction of the two boundary pixels and ONE of the four internal "shape feature" pixels. Its weights might look like this: ....+.+..+.. It would then be a position-dependent feature detector. In each location we have five such units to enable the net to develop all 4 position-dependent feature detectors (or funny combinations of them that span the space). In the next layer, we simply perform an OR for the 12 different copies of the same feature detector in the 12 different positions. So in the next layer we have position-independent feature detectors. Finally the outgoing connections from this layer represent the identity of a shape in terms of its position-independent feature detectors. Notice that the use of an OR should encourage the net to choose equivalent position-dependent feature detectors in the different locations, even though there is no explicit weight sharing. The amazing thing is that simply using backprop on the shape identities is sufficent to create this whole structure (or rather one of the zillions of mangled versions of it that uses hard-to-decipher distributed representations). Thanks to kevin lang for writing the Convex code that made this simulation possible in 1987. Please note that the local connectivity was NOT NECESSARY to get generalization. Without it the net still got 20/32 correct (guessing would be 2/32). Now, I dont for a moment believe that human shape perception is learned entirely by backpropagating from object identity labels. The simulation was simply intended to answer the philosophical point about whether this was impossible. Its taken nearly a decade for someone to come out and publicly voice the widely held belief that there is no way in hell a network could learn this. Thanks Geoff PS: I do think that there may be some biological mileage in the idea that local, position-dependent feature detectors are encouraged to break symmetry in the same way in order for later stages of processing to be able to achieve position independence by just performing an OR. An effect like this ought to occur in slightly less unrealistic models like Helmholtz machines. From pf2@st-andrews.ac.uk Thu Feb 22 21:54:16 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 22 Feb 96 21:54:10 -0600; AA21378 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 22 Feb 96 21:54:08 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa12762; 22 Feb 96 17:05:39 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa12760; 22 Feb 96 16:53:57 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa06448; 22 Feb 96 16:52:49 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa16446; 22 Feb 96 10:46:54 EST Received: from psych.st-and.ac.uk by RI.CMU.EDU id aa14634; 22 Feb 96 10:45:19 EST Received: (from pf2@localhost) by psych.st-andrews.ac.uk (8.6.12/8.6.12) id PAA24901; Thu, 22 Feb 1996 15:39:25 GMT Date: Thu, 22 Feb 1996 15:39:25 GMT From: Peter Foldiak Message-Id: <199602221539.PAA24901@psych.st-andrews.ac.uk> To: jfeldman@ICSI.Berkeley.EDU Subject: Re: shift invariance Cc: connectionists@cs.cmu.edu > Shift invariance is the ability of a neural system to recognize a pattern > independent of where appears on the retina. It is generally understood that > this property can not be learned by neural network methods, but I have > not seen a published proof. Minsky & Papert: Perceptons, 1969, MIT Press, p 54 : "... order-1 predicates invariant under the usual geometric groups can do nothing more than define simple ">=m"-type inequalities on the size or "area" of the figures. In particular, taking the translation group G we see that no first-order perceptron can distinguish the A's in the figure on p. 46 from some other translation-invarian set of figures of the same area." This doesn't say anything about multi-layer nets, i.e. you can combine feaures in a way that will be invariant. Peter Foldiak From goldfarb@unb.ca Thu Feb 22 21:54:17 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 22 Feb 96 21:54:13 -0600; AA21381 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 22 Feb 96 21:54:10 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa12812; 22 Feb 96 17:14:33 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa12766; 22 Feb 96 16:58:41 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa06479; 22 Feb 96 16:58:22 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa23475; 22 Feb 96 16:57:20 EST Received: from hermes.csd.unb.ca by CS.CMU.EDU id aa25263; 22 Feb 96 16:56:20 EST Received: from jupiter.sun.csd.unb.ca (goldfarb@jupiter.sun.csd.unb.ca [131.202.3.10]) by unb.ca (8.6.12/960123-14:25) id RAA02091; Thu, 22 Feb 1996 17:56:12 -0400 Received: by jupiter.sun.csd.unb.ca (8.6.10/950215-16:05) id RAA21393; Thu, 22 Feb 1996 17:56:11 -0400 Date: Thu, 22 Feb 1996 17:56:11 -0400 (AST) From: Lev Goldfarb X-Sender: goldfarb@jupiter.sun.csd.unb.ca To: connectionists@cs.cmu.edu Cc: inductive@unb.ca Subject: On the structure of connectionist models Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Dear connectionists: Since my posting of the workshop announcement (What is inductive learning?) several days ago, I was asked to clarify what I meant when I said that "one can show that inductive class representations (in other words, representations of concepts and categories) cannot be adequately specified within the classical (numeric) mathematical models" including, of course, connectionist models. Here are some general ideas from the paper which will be presented at the workshop. The following observations about the STRUCTURE of inductive learning models strongly suggest why the classical (numeric) mathematical models will be able to support only "weak" inductive learning models, i.e. the models that can perform reasonably only in VERY rigidly delineated environments. The questions I'm going to address in this posting on the one hand lay at the very foundations of connectionism and on the other hand are relatively simple, provided one keeps in mind that we are discussing the overall FORMAL STRUCTURE of the learning models (which requires a relatively high level of abstraction). Let's look at the structure of connectionist models through the very basic problem of inductive learning. In order to arrive at a useful formulation of the inductive learning problem and, at the same time, at a useful framework for solving the problem, I propose to proceed as follows. First and foremost, the inductive learning involves a finite set of data (objects from the class C) labeled either (C+, C-), positive and negative examples, or, more generally, simply C', examples. Since we want to compare quite different classes of models (e.g. symbolic and numeric), let us focus only on very general assumptions about the nature of the object representation (input) space: Postulate 1. Input space S satisfies a finite set A of axioms. (S, in fact, provide a formal specifications of all the necessary data properties; compare with the concept of abstract data type in computer science). Thus, for example, the vector (linear) space is defined by means of the well known set of axioms for vector addition and scalar multiplication. Next, let us attach the name "inductive class representation" (ICR) to the formal description (specification) of the class C obtained in a chosen model as a result of an inductive learning process: Postulate 2. In a learning model, ICR is specified in some (which?) formal manner. --------------------------------------------------------------------- | My first main point connects Postulate 2 to Postulate 1: ICR | | should be expressed in the "language" of the axioms from set A. | --------------------------------------------------------------------- For example, in a vector space ICR should be specified only in terms of the given data set plus the operations in the vector space, i.e. we are restricted to the spanned affine subspace or its approximation. The reason is quite simple: the only relationships that can be (mathematically) legitimately extracted from the input data are those that are expressible in the language of the input space S. Otherwise, we are, in fact, IMPLICITLY postulating some other relationships not specified in the input space by Postulate 1, and, therefore, the "discovery" of such implicit relationships in the data during the learning process is an illusion: such relationships are not "visible" in S. Thus, for example, "non-linear" relationships cannot be discovered from a finite data in a vector space, simply because a non-linear relationship is not part of the linear structure and, therefore, cannot be (mathematically) legitimately extracted from the finite input set of vectors in the vector space. What is happening (of necessity) in a typical connectionist model is that in addition to the set A of vector space axioms, some additional non-linear structure (determined by the class of non-linear functions chosen for the internal nodes of the NN) is being postulated IMPLICITLY from the beginning. Question: What does this additional non-linear structure has to do with the finite input set of vectors? (In fact, there are uncountably many such non-linear structures and, typically, none of them is directly related to the structure of the vector space or the input set of vectors.) ----------------------------------------------------------------------- | My second main point is this: if S is a vector space, in both cases, | | whether we do or don't postulate in addition to the vector space | | axioms some non-linear structure (for the internal nodes), we are | | faced with the following important question. What are we learning | | during the learning process? Certainly, we are not learning any | | interesting ICR: the entire STRUCTURE is fixed before the learning | | process. | ----------------------------------------------------------------------- It appears, that this situation is inevitable if we choose one of the classical (numeric) mathematical structures to model the input space S. However, in an appropriately defined symbolic setting (i.e. with an appropriate dynamic metric structure, see my home page) the situation changes fundamentally. To summarize (but not everything is before your eyes), the "strong" (symbolic) inductive learning models offer the ICRs that are much more flexible than those offered by the classical (numeric) models. In other words, the appropriate symbolic models offer true INDUCTIVE class representations. [The latter is given by a subset of objects + the constructed finite set of (weighted) operations that can transform objects into objects.] Lev Goldfarb http://wwwos2.cs.unb.ca/profs/goldfarb/goldfarb.htm From rolf@cs.rug.nl Thu Feb 22 21:54:33 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 22 Feb 96 21:54:31 -0600; AA21391 Message-Id: <9602230353.AA10981@lucy.cs.wisc.edu> Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 22 Feb 96 21:53:58 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa12421; 22 Feb 96 10:50:19 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa12419; 22 Feb 96 10:31:57 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa06332; 22 Feb 96 10:31:12 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa13846; 22 Feb 96 6:59:13 EST Received: from rug100.cs.rug.nl by EDRC.CMU.EDU id aa09305; 22 Feb 96 6:58:47 EST Received: by rug100.cs.rug.nl (1.37.109.11/16.2) id AA291770166; Thu, 22 Feb 1996 12:56:06 +0100 Date: Thu, 22 Feb 1996 12:56:06 +0100 From: rolf@cs.rug.nl To: jfeldman@ICSI.Berkeley.EDU Subject: Re: shift invariance Cc: Connectionists@cs.cmu.edu Dear Jerry, a short comment regarding your posting. It is no wonder that you have not seen a proof because it is simply not true that neural networks cannot do shift invariant recognition (SIR). If SIR is formalized it is most probably a computable function and ANNs can at least approximate all computable functions. No problem on a fundamental level here. K. Fukushima has shown a long time ago that a network can be wired up to do it. Nevertheless, invariant recognition seems to be a fundamental property of the visual system. It is not a _natural_ property of ANNs in the sense that you just give them a lot of natural stimuli and they develop the capability on their own. That, of course, makes one think if ANNs are a good model of the visual system, or if there is still a major point missing. I do not quite know what to make of your remark about ``theories overly dependent on learning''. If you have a better concept to offer than learning from experience I will be glad to hear about it. OK, you can say that there is ingenious machinery that does it, and the wiring is done by the genetic code, and we can go on and think about different things, but I do not consider that satisfactory explanation. I am looking forward to a discussion on the topic. Rolf +---------------------------------------------------------------------------+ | Rolf P. W"urtz | mailto:rolf@cs.rug.nl | URL: http://www.cs.rug.nl/~rolf/ | | Department of Computing Science, University of Groningen, The Netherlands | +---------------------------------------------------------------------------+ From dorffner@cns.bu.edu Sat Feb 24 01:49:07 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 24 Feb 96 01:48:58 -0600; AA22755 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 24 Feb 96 01:48:55 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa13339; 22 Feb 96 22:01:20 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa13333; 22 Feb 96 21:50:11 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa06624; 22 Feb 96 21:49:51 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa28576; 22 Feb 96 21:06:00 EST Received: from CNS.BU.EDU by CS.CMU.EDU id aa27371; 22 Feb 96 21:05:39 EST Received: from bucnsd by cns.bu.edu (8.6.10/BU_Server-1.1) id VAA04223; Thu, 22 Feb 1996 21:05:36 -0500 Received: by bucnsd (8.6.10/BU_Client-1.1) id VAA13842; Thu, 22 Feb 1996 21:05:34 -0500 From: dorffner@cns.bu.edu Mmdf-Warning: Parse error in original version of preceding line at DST.BOLTZ.CS.CMU.EDU Message-Id: <199602230205.VAA13842@bucnsd> Subject: Re: shift invariance To: connectionists@cs.cmu.edu Date: Thu, 22 Feb 1996 21:05:34 -0500 (EST) X-Mailer: ELM [version 2.4 PL25] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 5928 Hi fellow connectionists, I must say I'm a little puzzled by this discussion about shift invariance. It was started by Jerry Feldman by saying > Shift invariance is the ability of a neural system to recognize a pattern > independent of where appears on the retina. It is generally understood that > this property can not be learned by neural network methods, but I have > not seen a published proof. A "local" learning rule is one that updates the > input weights of a unit as a function of the unit's own activity and some > performance measure for the network on the training example. All biologically > plausible learning rules, as well as all backprop variants, are local in this > sense. Now I always thought that this is so obvious that it didn't need any proof. Geoff Hinton responded by disagreeing: > Contrary to your assertions, shift invariance can be learned by > backpropagation. It was one of the problems that I tried when fiddling about > with backprop in the mid 80's. I published a paper demonstrating this in an > obscure conference proceedings: He describes a model based on feature detectors and subsequent backpropagation that can actually generalize over different positions. He finishes by saying > The simulation was > simply intended to answer the philosophical point about whether this was > impossible. Its taken nearly a decade for someone to come out and publicly > voice the widely held belief that there is no way in hell a network could > learn this. > IMHO, there seems to be a misunderstanding of what the topic of discussion is here. I don't think that Jerry meant that no model consisting of neural network components could ever learn shift invariance. After all, there are many famous examples in visual recognition with neural networks (such as the Neocognitron, as Rolf W"urtz pointed out), and if this impossibility were the case, we would have to give up neural network research in perceptual modeling altogether. What I think Jerry meant is that any cascade of fully-connected feed-forward connection schemes between layers (including the perceptron and the MLP) cannot learn shift invariance. Now besides being obvious, this does raise some important questions, possibly weakening the fundamentals of connectionism. Let me explain why: - state spaces in connectionist layers (based on the assumption that activation patterns are viewed as vectors) span a Euclidean space, with each connection scheme that transfers patterns into another layer applying a certain kind of metric defining similarity. This metric is non-trivial, especially in MLPs, but it restricts the ways of what such a basic neural network component (i.e. fully connected feedforward) can view as similar. Patterns that are close in this space according to a distance measure, or patterns that have large orthogonal projections onto each other (in my analysis the basic similarity measure in MLPs) are similar according to this metric. Different patterns with a sub-pattern in different positions are obviously NOT. Neither are patterns which share common differences between components (e.g. the patterns (0.8 0.3) and (0.6 0.1)), and a whole bunch of other examples. That's why we have to be so careful about the right kind of preprocessing when we apply neural networks in engineering, and why we have to be equally careful in choosing the appropriate representations in connectionist cognitive modeling. - introducing feature detectors and other complex connectivities and learning schemes (weight sharing, or the OR Geoff mentioned) is a way of translating the original pattern space into a space where the similarity structures which we expect obey the said metric in state space again. It's the same thing we do in preprocessing (e.g. we apply an FFT to signals, since we cannot expect that the network can extract invariances in the frequency domain). - Geoff's model, necognitron, and many others do exactly that. Each single component (e.g. one feature detector) is restricted by the similarity metric mentioned above. But by applying non-linear functions, and by combining their output in a clever way they translate the original patterns into a new pattern space, where similarity corresponds to this metric again (e.g. for the final backprop network Geoff introduced). Now obviously, when we look at the human visual system, the brain does seem to do some kind of preprocessing, such as applying feature detectors, as well. So we're kind of safe here. But the above observation does make one think, whether the similarity metric a neural network basically applies is actually the right kind of basis for cognitive modeling. Think about it: By introducing complex wiring and learning schemes, and by carefully choosing representations, we go a long way to finally satisfy the actual neural network that has to do the job of extracting information from the patterns. Visual recognition is but one, although prominent, example. Now what makes us sure that deeper processes ARE of the kind a fully connected feedforward network can handle (i.e. that processes DO work on the said restricted kind of similarity metric)? Now I do not really think that we have a problem here. But some people recently have raised serious doubts. Some have suggested that perhaps replacing connectionist state spaces by the "space" that is spanned by attractors in a dynamical systems gives one a more appropriate metric. I am not suggesting this, I am just proposing that connectionists have to be alert in this respect, and keep questioning themselves whether the fundamental assumptions we're making are the appropriate ones. In this way I think Jerry's point is worth discussing. Just my 2 c. worth, Georg (email: georg@ai.univie.ac.at) P.S: I would like to acknowledge F.G. Winkler from Vienna for some of the ideas expressed in this message. From hicks@cs.titech.ac.jp Sat Feb 24 01:49:08 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 24 Feb 96 01:49:00 -0600; AA22757 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 24 Feb 96 01:48:57 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab13339; 22 Feb 96 22:02:12 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab13333; 22 Feb 96 21:50:17 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa06629; 22 Feb 96 21:50:05 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa28893; 22 Feb 96 21:26:27 EST Received: from euclid.cs.titech.ac.jp by CS.CMU.EDU id aa27433; 22 Feb 96 21:23:52 EST Received: by euclid.cs.titech.ac.jp (8.6.11+2.4W/3.2W-titmx-2.0g); Fri, 23 Feb 1996 11:21:25 +0900 Date: Fri, 23 Feb 1996 11:21:25 +0900 From: hicks@cs.titech.ac.jp Message-Id: <199602230221.LAA04831@euclid.cs.titech.ac.jp> To: jfeldman@ICSI.Berkeley.EDU Cc: connectionists@cs.cmu.edu In-Reply-To: Jerry Feldman's message of Wed, 21 Feb 1996 08:12:42 -0800 <9602210812.ZM26438@ICSI.Berkeley.edu> Subject: shift invariance jfeldman@ICSI.Berkeley.EDU Wed, 21 Feb 1996 08:12:42 wrote:: >Shift invariance is the ability of a neural system to recognize a pattern >independent of where appears on the retina. It is generally understood that >this property can not be learned by neural network methods I disagree. As you state later, the network of neurons need only "share" weights. Sharing can be forced, or can occur by independent learnign of the same (but translated) data. > The unlearnability of shift invarince is not a problem in practice because >people use preprocessing, weight sharing or other techniques to get shift >invariance where it is known to be needed. However, it does pose a problem for >the brain and for theories that are overly dependent on learning. There are two obvious ways in which shift invariance could occur in "biological" or other learning systems. 1) Nature Some of part of the patterns of connectivity in the low level vision system are decided genetically and replicated automatically (like many cellular structures throughout the boody); in effect, a kind of weight sharing. Natural selection (learning through genetics) favors patterns of connectivity which will detect frequently appearing patterns; in effect weight learning. 2) Nurture The strengths of neuronal connections in the low level vision system self-adjust in some Hebbian scheme to frequently occuring patterns. The distribution of these patterns, in actual fact, is shift invariant. (If they aren't shift invariant then there's not much point in learning them as though they were shift invariant.) Respectfully Yours, Craig Hicks From juergen@idsia.ch Sat Feb 24 01:49:09 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 24 Feb 96 01:49:02 -0600; AA22762 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 24 Feb 96 01:48:59 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa13973; 23 Feb 96 4:59:43 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa13971; 23 Feb 96 4:46:25 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa07002; 23 Feb 96 4:46:08 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa03379; 23 Feb 96 3:16:50 EST Received: from chsun.eunet.ch by EDRC.CMU.EDU id aa14157; 23 Feb 96 3:15:55 EST Received: from fava.idsia.ch by chsun.eunet.ch (8.6.10/1.34) id JAA04154; Fri, 23 Feb 1996 09:15:29 +0100 Received: from kraut.idsia.ch by fava.idsia.ch (4.1/SMI-4.1) id AA04599; Fri, 23 Feb 96 09:15:27 +0100 Date: Fri, 23 Feb 96 09:15:27 +0100 From: Juergen Schmidhuber Message-Id: <9602230815.AA04599@fava.idsia.ch> To: Connectionists@cs.cmu.edu Subject: shifts Jerry Feldman writes: >>> Shift invariance is the ability of a neural system to recognize a pattern independent of where appears on the retina. It is generally understood that this property can not be learned by neural network methods, but I have not seen a published proof. [...] It is easy to show that no local rule can learn shift invariance. [...] The one dimensional case of shift invariance can be handled by treating each string as a sequence and learning a finite-state acceptor. But the methods that work for this are not local or biologically plausible and don't extend to two dimensions. <<< It might be of interest to note that the situation changes if the neural system includes a controller that is able to generate retina- movements (to change the position of the image on the retina). There are gradient-based controllers that (in certain cases) can *learn* appropriate, 2-dimensional retina shifts. They are `local' to the extent backprop through time is `local'. See, e.g., Schmidhuber & Huber (1991): Learning to generate fovea trajectories for target detection. Int. Journal of Neural Systems, 2(1 & 2):135-141. Juergen Schmidhuber, IDSIA From listerrj@helios.aston.ac.uk Sat Feb 24 01:49:12 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 24 Feb 96 01:49:04 -0600; AA22767 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 24 Feb 96 01:49:01 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa14873; 23 Feb 96 16:31:43 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa14854; 23 Feb 96 16:02:42 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa07313; 23 Feb 96 16:01:43 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa09590; 23 Feb 96 10:02:46 EST Received: from [134.151.79.13] by CS.CMU.EDU id aa00965; 23 Feb 96 10:01:34 EST Received: from sun.aston.ac.uk (actually host laplace.aston.ac.uk) by email.aston.ac.uk with SMTP (PP); Fri, 23 Feb 1996 15:02:13 +0000 Message-Id: <8199.199602231459@sun.aston.ac.uk> To: Connectionists@cs.cmu.edu Subject: Neural Computing Research Programmer post X-Mailer: Mew version 1.02 on Emacs 19.29.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Fri, 23 Feb 1996 14:59:39 +0000 From: Richard Lister ---------------------------------------------------------------------- Neural Computing Research Group ------------------------------- Dept of Computer Science and Applied Mathematics Aston University, Birmingham, UK Research Programmer ------------------- * Full details at http://www.ncrg.aston.ac.uk/ * Applications are invited for the post of Research Programmer within the Neural Computing Research Group (NCRG) at Aston University. The NCRG is now the largest academic research group in this area in the UK, and has an extensive and lively programme of research ranging from the theoretical foundations of neural computing and pattern recognition through to industrial and commercial applications. The Group is based in spacious accommodation in the University's Main Building, and is well equipped with its own network of Silicon Graphics and Sun workstations, supported by a full-time system administrator. The successful candidate will have the opportunity to contribute in the following areas: * development of real-world applications of neural networks in connection with a wide variety of industrial and commercial research contracts * providing software contributions in support of basic research projects * production of demonstration software for use in teaching a variety of neural network courses Most of the software will be developed in C++ and Matlab, on a high- power Silicon Graphics workstation with access to the Group's SGI Challenge supercomputer. The ideal candidate will have: * a good first degree in a numerate discipline * expertise in software development (preferably in C and C++) * a good understanding of neural networks * working knowledge of basic mathematics such as calculus and linear algebra * experience of working in a UNIX environment * willingness to undertake complex and challenging problems This post provides an excellent opportunity to learn new skills within an exciting team environment Conditions of Service --------------------- The appointment will be for an initial period of one year, with the possibility of subsequent renewal. Initial salary will be on the academic 1A or 1B scales up to 15,986. How to Apply ------------ If you wish to be considered for this position, please send a full CV, together with the names and addresses of at least 3 referees, to: Hanni Sondermann Neural Computing Research Group Department of Computer Science and Applied Mathematics Aston University Birmingham B4 7ET, U.K. Tel: (+44 or 01) 21 333 4631 Fax: (+44 or 01) 21 333 6215 e-mail: h.e.sondermann@aston.ac.uk ---------------------------------------------------------------------- ~~~~~~~~~~~~~~ Richard J. Lister r.j.lister@aston.ac.uk ~~~~~~~~~~~~~~~~ Research Assistant, Neural Computing Research Group Aston University, Birmingham B4 7ET, UK ~~~~~~~~~~~~~~~~~ http://www.ncrg.aston.ac.uk/~listerrj/ ~~~~~~~~~~~~~~~~~~ From kak@ee.lsu.edu Sat Feb 24 01:49:13 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 24 Feb 96 01:49:05 -0600; AA22774 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 24 Feb 96 01:49:03 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab14873; 23 Feb 96 16:33:05 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab14854; 23 Feb 96 16:02:43 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa07319; 23 Feb 96 16:02:13 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa09674; 23 Feb 96 10:07:47 EST Received: from gate.ee.lsu.edu by CS.CMU.EDU id aa01008; 23 Feb 96 10:07:01 EST Received: by ee.lsu.edu (4.1/SMI-4.1) id AA12042; Fri, 23 Feb 96 09:06:38 CST Date: Fri, 23 Feb 96 09:06:38 CST From: Subhash Kak Message-Id: <9602231506.AA12042@ee.lsu.edu> To: hinton@cs.toronto.edu, jfeldman@ICSI.Berkeley.EDU Subject: Re: shift invariance Cc: Connectionists@cs.cmu.edu For feedback neural networks here are some more references on shift invariant learning: T. Maxwell et al, Transformation invariance using high order correlations in neural net architectures. Plasma Preprint UMLPF #88-125. Univ of Maryland, 1988. W. Widrow and R. Winter, Neural nets for adaptive filtering and adaptive pattern recognition. Computer, 21 (3):25-39, 1988. D.L. Prados and S.C. Kak, Shift invariant associative memory. IN VLSI for Artificial Intelligence, J.G. Delgado-Frias and W.R. Moore (eds.), pp. 185-197, Kluwer Academic Publishers, 1989. -Subhash Kak From franco@cim.mcgill.ca Sat Feb 24 01:49:13 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 24 Feb 96 01:49:07 -0600; AA22777 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 24 Feb 96 01:49:05 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ac14873; 23 Feb 96 16:34:26 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa14858; 23 Feb 96 16:03:34 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa07325; 23 Feb 96 16:02:34 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa10380; 23 Feb 96 10:44:50 EST Received: from Athena.McRCIM.McGill.EDU by CS.CMU.EDU id aa01390; 23 Feb 96 10:44:08 EST Received: from Poptart.McRCIM.McGill.EDU by Athena.McRCIM.McGill.EDU (8.6.10) with ESMTP id <199602231543.KAA25611@Athena.McRCIM.McGill.EDU>; Fri, 23 Feb 1996 10:43:47 -0500 From: Francesco Callari Received: (franco@localhost) by Poptart.McRCIM.McGill.EDU (8.6.10/8.6.9) id PAA15318; Fri, 23 Feb 1996 15:43:43 GMT Date: Fri, 23 Feb 1996 15:43:43 GMT Message-Id: <199602231543.PAA15318@Poptart.McRCIM.McGill.EDU> To: Connectionists@cs.cmu.edu, neuron@CATTELL20.psych.upenn.edu Cc: franco@cim.mcgill.ca Subject: Paper on active 3D object recognition and sensor planning via ANN The following paper deals with the problems of combining 3D shape information and class priors for the purposes of active, model-based object recognition and sensor planning. The proposed system correlates estimates of shape and class uncertainty to determine those sensor locations that best disambiguate the objects. The class and class sensitivity estimates are computed by an MLP network, trained using MacKay's "evidence" framework and put in the planning feedback loop of a mobile robot. FTP-host: ftp.cim.mcgill.ca FTP-file: /pub/people/franco/ambiguity96.ps.gz Active Recognition: Using Uncertainty to Reduce Ambiguity Francesco G. Callari and Frank P. Ferrie Centre for Intelligent Machines, McGill University 3480 University St., Montre\'al, Que., Canada, H3A 2A7 email: franco@cim.mcgill.ca, ferrie@cim.mcgill.ca Keywords: Active Vision, Control of Perception, Learning in Computer Vision ABSTRACT Ambiguity in scene information, due to noisy measurements and uncertain object models, can be quantified and actively used by an autonomous agent to efficiently gather new data and improve its information about the environment. In this work an information-based utility measure is used to derive from a learned classification of shape models an efficient data collection strategy, specifically aimed at increasing classification confidence when recognizing uncertain shapes. Promising experimental results with real data are reported. Submitted to: ICPR96. From oby@cs.tu-berlin.de Sat Feb 24 01:49:14 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 24 Feb 96 01:49:09 -0600; AA22786 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 24 Feb 96 01:49:06 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ad14873; 23 Feb 96 16:35:40 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab14858; 23 Feb 96 16:03:40 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa07332; 23 Feb 96 16:03:28 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa11882; 23 Feb 96 12:10:24 EST Received: from mail.cs.tu-berlin.de by RI.CMU.EDU id aa19821; 23 Feb 96 12:09:27 EST Received: from pollux.cs.tu-berlin.de (oby@pollux.cs.tu-berlin.de [130.149.31.31]) by mail.cs.tu-berlin.de (8.6.12/8.6.12) with ESMTP id SAA29192 for ; Fri, 23 Feb 1996 18:07:08 +0100 From: Klaus Obermayer Received: (oby@localhost) by pollux.cs.tu-berlin.de (8.6.12/8.6.12) id SAA29613 for Connectionists@CS.CMU.EDU; Fri, 23 Feb 1996 18:07:06 +0100 Date: Fri, 23 Feb 1996 18:07:06 +0100 Message-Id: <199602231707.SAA29613@pollux.cs.tu-berlin.de> To: Connectionists@cs.cmu.edu Content-Length: 952 GRADUATE STUDENT POSITION A graduate student position is available at the CS-department of the Technical University of Berlin to study models of neural development. A major objective is to construct and investigate models for the formation of ocular dominance and orientation selectivity in striate cortex. The candidate is expected to join a close collaboration between theorists and experimentalists. Candidates should have experience in computational modelling. The position is available initially for two years. Salary is commensurate with BAT II a/2. Applicants should send their CV, list of publications, a letter describing their interest, and name, address and phone number of two references to: Prof. Klaus Obermayer phone: 49-30-314-73442 FR2-1, KON, Informatik 49-30-314-73120 Technische Universitaet Berlin fax: 49-30-314-73121 Franklinstrasse 28/29 e-mail: oby@cs.tu-berlin.de 10587 Berlin, Germany http://www.cs.tu-berlin.de/fachgebiete/kon/