From karim@ax1303.physik.uni-marburg.de Sun Feb 25 06:51:06 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sun, 25 Feb 96 06:51:03 -0600; AA11468 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sun, 25 Feb 96 06:51:01 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa14851; 23 Feb 96 16:21:11 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa14849; 23 Feb 96 16:01:34 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa07305; 23 Feb 96 16:01:17 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa09517; 23 Feb 96 9:56:20 EST Received: from [137.248.1.8] by RI.CMU.EDU id aa19223; 23 Feb 96 9:55:34 EST Received: from ax1303.Physik.Uni-Marburg.DE by Mailer.Uni-Marburg.DE (AIX 3.2/UCB 5.64/20.07.94) id AA61886; Fri, 23 Feb 1996 15:55:00 +0100 Received: by ax1303.physik.uni-marburg.de; (5.65/1.1.8.2/14Sep95-0134PM) id AA28657; Fri, 23 Feb 1996 15:59:31 +0100 Date: Fri, 23 Feb 1996 15:59:31 +0100 From: Karim Mohraz Message-Id: <9602231459.AA28657@ax1303.physik.uni-marburg.de> To: Connectionists@cs.cmu.edu Subject: FlexNet - a flexible neural network construction algorithm The following paper is available via WWW FlexNet - a flexible neural network construction algorithm Abstract Dynamic neural network algorithms are used for automatic network design in order to avoid time consuming search for finding an appropriate network topology with trial & error methods. The new FlexNet algorithm, unlike other network construction algorithms, does not underlie any constraints regarding the number of hidden layers and hidden units. In addition different connection strategies are available, together with candidate pool training and the option of freezing weights. Test results on 3 different benchmarks showed higher generalization rates for FlexNet compared to Cascade-Correlation and optimized MLP networks. Keywords: network construction, generalization, Cascade-Correlation. This paper has been accepted for publication in the European Symposium on Artificial Neural Networks (ESANN), Bruges, Belgium , April, 96. http://www.physik.uni-marburg.de/bio/mitarbei/karim/flexnet.ps (6 pages) Sorry, no hardcopies available ``` (o o) +-------------------oOO--(_)--OOo---------------------------------------------------+ Karim Mohraz Bereich Neuronale Netze & Fuzzy Logik Bayerisches Forschungszentrum fuer Wissensbasierte Systeme F O R W I S S Erlangen New address: AG Neurophysik, Universitaet Marburg, Germany Email: karim@bio.physik.uni-marburg.de WWW: http://www.physik.uni-marburg.de/bio/mitarbei/karim.html _ +-------------------o00--( )--00o---------------------------------------------------+ (o o) ''' From carmesin@schoner.physik.uni-bremen.de Sun Feb 25 06:51:11 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sun, 25 Feb 96 06:51:08 -0600; AA11474 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sun, 25 Feb 96 06:51:06 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa17576; 24 Feb 96 21:13:58 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa17574; 24 Feb 96 20:57:41 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa08333; 24 Feb 96 20:57:00 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa29898; 24 Feb 96 6:43:13 EST Received: from schoner.physik.uni-bremen.de by EDRC.CMU.EDU id aa19383; 24 Feb 96 6:42:36 EST Received: (from carmesin@localhost) by schoner.physik.uni-bremen.de (8.7.1/8.7.1) id MAA09674; Sat, 24 Feb 1996 12:47:16 +0100 Date: Sat, 24 Feb 1996 12:47:16 +0100 Message-Id: <199602241147.MAA09674@schoner.physik.uni-bremen.de> From: Hans-Otto Carmesin To: Connectionists@cs.cmu.edu Subject: SIR: shift invariant recognition Dear Rolf and Jerry. The question raised by Rolf Wurtz is, how SIR ( shift invariant recognition) might be processed in the visual system. There is a biologically reasonable candidate network: I proposed it for experiments on so-called stroboscopic alternative motion (SAM). The most simple instance is established by TWO light dots, one of which is elicited at a time in an alternating manner. At adequate frequency an observer perceives ONE dot moving back and forth. The experiment becomes quite non-trivial with four dots at the corners of a square, two elicited at a time at diagonal positions and in an alternating manner. An observer perceives either two dots moving horizontally or two dots moving vertically (roughly speaking). The network represents each dot by a formal neuron; these neurons project to inner neurons that tend to fire in accordance with the stimulation and that are coupled with rapid formal couplings (similar to dynamic links) with a local coupling dynamics reminescent of the Hebb-rule [1-4]. A motion percept is established by the emerging nonzero couplings. It turns out that each active neuron at a time t is coupled to exactly one active neuron at a later time, t+t' say. Moreover there are prestabilized coupling weights (modeling synaptic densities) that prefer short distances in space and time. As a result: If a pattern is presented at a time t and a shifted pattern is presented at a time t+t', then the dots of the first pattern are coupled to the corresponding dots of the second pattern. This network is understood very well [3,4]: It can be solved analytically and exhibits an effective potential dynamics in coupling space. I predicted [3] a continuous phase transition and measured it together with experimental psychologists later. Another indication of biological relevance: Formally the network is very similar to networks with retinotopy emergence [5]. References: [1] H.-O. Carmesin: Statistical neurodynamics: A model for universal properties of EEG-data and perception. Acta Physica Slovaca, 44:311--330, 1994. [2] H.-O. Carmesin and S. Arndt: Neuronal self-organization of motion percepts. Technical Report 6/95, ZKW Universitt Bremen, Bremen, 1995. [3] H.-O. Carmesin: Theorie neuronaler Adaption. (Kster, Berlin, 1994. ISBN 3-89574-020-9). [4] H.-O. Carmesin: Neuronal Adaptation Theory. (Peter Lang, Frankfurt am Main, 1996. ISBN 3-631-30039-5). [5] H.-O. Carmesin: Topology preservation emergence by Hebb rule with infinitesimal short range signals. Phys. Rev. E, 53(1):993--1003, 1996. For details see: WWW: http://schoner.physik.uni-bremen.de/~carmesin/ From lpratt@fennel.mines.edu Sun Feb 25 06:51:16 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sun, 25 Feb 96 06:51:12 -0600; AA11482 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sun, 25 Feb 96 06:51:09 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa17588; 24 Feb 96 21:24:15 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ac17574; 24 Feb 96 20:57:49 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa08344; 24 Feb 96 20:57:36 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa03150; 24 Feb 96 13:17:02 EST Received: from fennel.Mines.EDU by RI.CMU.EDU id aa23804; 24 Feb 96 13:16:07 EST Received: (from lpratt@localhost) by fennel.mines.edu (8.6.12/8.6.9) id LAA01740; Sat, 24 Feb 1996 11:15:45 -0700 From: "Lorien Y. Pratt" Message-Id: <199602241815.LAA01740@fennel.mines.edu> Subject: Call for papers -- please post To: Connectionists@cs.cmu.edu, Ken Laws , transfer@mines.edu, ml@ics.uci.edu Date: Sat, 24 Feb 1996 11:15:45 -0700 (MST) Reply-To: lpratt@mines.edu X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 3440 ------------------------------------------------------------------------------- Call for papers (please post) Special Issue of the Machine Learning Journal on Inductive Transfer ------------------------------------------------------------------------------- Lorien Pratt and Sebastian Thrun, Guest Editors ------------------------------------------------------------------------------- Many recent machine learning efforts are focusing on the question of how to learn in an environment in which more than one task is performed by a system. As in human learning, related tasks can build on one another, tasks that are learned simultaneously can cross-fertilize, and learning can occur at multiple levels, where the learning process itself is a learned skill. Learning in such an environment can have a number of benefits, including speedier learning of new tasks, a reduced number of training examples for new tasks, and improved accuracy. These benefits are especially apparent in complex applied tasks, where the combinatorics of learning are often otherwise prohibitive. Current efforts in this quickly growing research area include investigation of methods that facilitate learning multiple tasks simultaneously, those that determine the degree to which two related tasks can benefit from each other, and methods that extract and apply abstract representations from a source task to a new, related, target task. The situation where the target task is a specialization of the source task is an important special case. The study of such methods has broad application, including a natural fit to data mining systems, which extract regularities from heterogeneous data sources under the guidance of a human user, and can benefit from the additional bias afforded by inductive transfer. We solicit papers on inductive transfer and learning to learn for an upcoming Special Issue of the Machine Learning Journal. Please send six (6) copies of your manuscript postmarked by June 1, 1996 to: Dr. Lorien Pratt MCS Dept. CSM Golden, CO 80401 USA One (1) additional copy should be mailed to: Karen Cullen Attn: Special Issue on Inductive Transfer MACHINE LEARNING Editorial Office Kluwer Academic Publishers 101 Philip Drive Assinippi Park Norwell, MA 02061 USA Manuscripts should be limited to at most 12000 words. Please also note that Machine Learning is now accepting submission of final copy in electronic form. Authors may want to adhere to the journal formatting standards for paper submissions as well. There is a latex style file and related files available via anonymous ftp from ftp.std.com. Look in Kluwer/styles/journals for the files README, kbsfonts.sty, kbsjrnl.ins, kbsjrnl.sty, kbssamp.tex, and kbstmpl.tex, or the file kbsstyles.tar.Z, which contains them all. Please see http://vita.mines.edu:3857/1s/lpratt/transfer.html for more information on inductive transfer. Papers will be quickly reviewed for a target publication date in the first quarter of 1997. -- Dr. Lorien Y. Pratt Department of Mathematical and Computer Sciences lpratt@mines.edu Colorado School of Mines (303) 273-3878 (work) 402 Stratton (303) 278-4552 (home) Golden, CO 80401, USA Vita, photographs, all publications, all course materials available from my web page: http://vita.mines.edu:3857/1s/lpratt From clee@it.wustl.edu Sun Feb 25 06:51:17 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sun, 25 Feb 96 06:51:14 -0600; AA11484 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sun, 25 Feb 96 06:51:12 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa17884; 25 Feb 96 1:11:35 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa17879; 25 Feb 96 1:00:08 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa08472; 25 Feb 96 0:59:27 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa11730; 25 Feb 96 0:47:59 EST Received: from wugate.wustl.edu by CS.CMU.EDU id aa14033; 25 Feb 96 0:47:51 EST Received: from it by wugate.wustl.edu (8.6.12/8.6.11) with ESMTP id XAA29456; Sat, 24 Feb 1996 23:47:16 -0600 Received: by it (940816.SGI.8.6.9/911001.SGI) id VAA23373; Sat, 24 Feb 1996 21:57:36 -0600 Date: Sat, 24 Feb 1996 21:57:36 -0600 Message-Id: <199602250357.VAA23373@it> From: Christopher Lee To: Jerry Feldman Cc: Connectionists@cs.cmu.edu Subject: shift invariance In-Reply-To: <9602210812.ZM26438@ICSI.Berkeley.edu> References: <9602210812.ZM26438@ICSI.Berkeley.edu> >>>>> "Jerry" == Jerry Feldman writes: Jerry> Shift invariance is the ability of a neural system to Jerry> recognize a pattern independent of where appears on the Jerry> retina. It is generally understood that this property can Jerry> not be learned by neural network methods, but I have not Jerry> seen a published proof. A "local" learning rule is one that Jerry> updates the input weights of a unit as a function of the Jerry> unit's own activity and some performance measure for the Jerry> network on the training example. All biologically plausible Jerry> learning rules, as well as all backprop variants, are local Jerry> in this sense. Jerry's communique has certainly certainly sparked discussion, but I feel as if his reference to "neural network methods" needs more precise definition. Perhaps Jerry could state more specifically the class of network architectures and neurons he wishes to consider? (E.g., Minsky and Papert restricted their proof to order one perceptrons.) What sort of resource limitations would you put on this network relative to the complexity of the task? (To give an absurd example of why this is important: for a small "test problem"-like space, if given an appropriate number of nodes a network could simply "memorize" all the configurations of an object at all locations. Clearly, this isn't what one would normally considering "learning" shift invariance.) On another vein that might be of interest, it's clear that shift invariance is a fundamental to the primate visual system in some way, and a fair amount of interest exists in the neurophysiology community concerning how this problem is solved; one hypothesis involves the role of attentional mechanisms in scale and translational invariance (Olshausen, Anderson, Van Essen, J. of Neuroscience. 13(11):4700-19, 1993). It is not obvious to me that anything along the lines of Jerry's proof could be applied to their (the Olshausen et al.) network model. Christopher Lee -- Washington University Department of Anatomy and Neurobiology email: clee@v1.wustl.edu From scheler@informatik.tu-muenchen.de Sun Feb 25 06:51:20 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sun, 25 Feb 96 06:51:10 -0600; AA11476 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sun, 25 Feb 96 06:51:08 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab17576; 24 Feb 96 21:15:27 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab17574; 24 Feb 96 20:57:43 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa08339; 24 Feb 96 20:57:21 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa00569; 24 Feb 96 7:52:06 EST Received: from papa.informatik.tu-muenchen.de by CS.CMU.EDU id aa10450; 24 Feb 96 7:51:12 EST Received: from star.informatik.tu-muenchen.de by papa.informatik.tu-muenchen.de id <116444>; Sat, 24 Feb 1996 13:50:55 +0100 From: Gabriele Scheler To: connectionists@cs.cmu.edu Subject: Re: Shift Invariance Cc: scheler@informatik.tu-muenchen.de Message-Id: <96Feb24.135055+0100_met.116444+24@papa.informatik.tu-muenchen.de> Date: Sat, 24 Feb 1996 13:50:42 +0100 There should be a difference made between shift-invariance, i.e. distinguishing between T1: {[a,b,c,d,e], [b,c,d,e,a], [c,d,e,a,b]} T2: {[a,d,b,c,d], [a,d,c,b,e] etc.} which is more of a purely mathematical problem, and translational invariance, i.e. detecting a pattern on a plane, no matter where it occurs. For the latter goal it is sufficient to develop a set of features in the first layer to detect that pattern in a local field, and to develop an invariant detector in the next layer, which is ON for any of the lower-level features. (develop means train for ANN). In the domain of neural networks the obvious solution to the mathematical problem would be to train a level of units as sequence encoders: A1 B1 C1 D1 ----- ---- a b ------- c ------- d and classify patterns then on how many of the sequence encoders a-d are ON. Of course this may be rather wasteful. In another learning approach called adaptive distance measures, we can reduce training effort considerably when we use a distance measure which is specifically tuned to problems of shift invariance. Of course this is nothing else than to have a class of networks with pre-trained sequence encoders available. The question here as often is not, which NN can learn this task (backprop can, Fukushima's Neocognitron can), but which is most economical in its resources - without requiring too much knowledge on the type of function to be learned. From pazzani@super-pan.ICS.UCI.EDU Mon Feb 26 03:55:09 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 26 Feb 96 03:55:02 -0600; AA23734 Received: from paris.ics.uci.edu by lucy.cs.wisc.edu; Mon, 26 Feb 96 03:54:49 -0600 Received: from super-pan.ics.uci.edu by paris.ics.uci.edu id aa06175; 25 Feb 96 20:48 PST To: ML-LIST:; Subject: Machine Learning List: Vol. 8, No. 3 Reply-To: ml@ics.uci.edu Date: Sun, 25 Feb 1996 20:32:48 -0800 From: Michael Pazzani Message-Id: <9602252048.aa06175@paris.ics.uci.edu> Machine Learning List: Vol. 8, No. 3 Sunday, February 25, 1996 Contents: Subject: Last CFP: Computational Cognitive Modeling, AAAI-96 Workshop Subject: Job Posting Subject: ILP'96 CFP Subject: Workshop: What is inductive learning? Subject: NIPS*96 CALL FOR PAPERS Subject: Job openings Subject: MS Research Decision Theory Group Announcement Subject: Knowledge Industries announces no-cost academic licenses for Subject: KDD Book Announcement Subject: 2nd CFP: AAAI-96 Workshop on Integrating Multiple Learned Models Subject: Final CFP for ISIS: Information, Statistics and Induction in Science Subject: Call for Papers of 2nd WEC Subject: Final CFP for KDD-96 Subject: CFP: AAAI-96 WS on INTELLIGENT ADAPTIVE AGENTS Subject: ICML'96 Workshop on Evolutionary Computing and Machine Learning Subject: Special Issue of the Machine Learning Journal: Inductive Transfer The Machine Learning List is moderated. Contributions should be relevant to the scientific study of machine learning. Mail contributions to ml@ics.uci.edu. Mail requests to be added or deleted to ml-request@ics.uci.edu. Back issues may be FTP'd from ics.uci.edu in pub/ml-list/V/ or N.Z where X and N are the volume and number of the issue; ID: anonymous PASSWORD: URL- http://www.ics.uci.edu/AI/ML/Machine-Learning.html ---------------------------------------------------------------------- Date: Wed, 7 Feb 1996 11:38:10 +0800 From: "Charles X. Ling" Subject: Last CFP: Computational Cognitive Modeling, AAAI-96 Workshop We are looking forward to a productive meeting. We seek for a balance between different models (such as connectionists and symbolic models). Submissions from cognitive scientists, AI researchers, and psychologists are warmly welcome. Computational Cognitive Modeling: Source of the Power AAAI-96 Workshop (During AAAI'96, IAAI 96 and KDD 96. August 4-8, 1996. Portland, Oregon) URL: http://www.cs.hku.hk/~ling for updated information. ------------------------------ Date: Wed, 7 Feb 1996 18:35:53 -0500 (EST) From: Drastic Subject: Job Posting CALL FOR RESUMES (5 Feb. 1996) Siemens Corporate Research An Equal Opportunity Employer Adaptive Information and Signal Processing Department We have an immediate opening for a research scientist. The position will be fulltime and permanent at the level of associate member of technical staff (MTS) up to senior MTS, depending on the applicant's background. Our department is involved with problem-solving that supports our mission to transfer novel technology into the divisions of Siemens, in the U.S. and worldwide. Siemens Corporate Research is the U.S. based corporate R&D lab of Siemens AG, a global company involved in communications, medical devices, power, transportation, automation, electronic systems, and other businesses. SCR is located in a suburb of Princeton, NJ, close to Princeton, Rutgers, and many other corporate R&D centers. Applicants should have a MS/MA or Ph.D. in computer science and a record of significant publications in machine learning, plus extraordinary ability to communicate with scientists and engineers who are not familiar with your own area of research. Because we are a corporate R&D lab, our projects are often customer oriented, joint ventures with one (or more) of our manufacturing divisions. This means that we value breadth in an applicant's background along with depth in one area. You may work off-site with a customer to understand their requirements, and participate in system integration. We have a project under way that calls for some experience with knowledge based decision support systems, specifically as applied to diagnosis of electronic/mechanical equipment. For this and future projects, we are particularly interested in deepening our department's level of competence in AI disciplines related to "adaptive information and signal processing". Other skills we require include: Programming ability using C, C++, MS Win95, and OLE protocols. The ability to present briefings and demonstrations, and to initiate new projects, will be essential for moving into the role of project manager. The ability to break cryptographic codes, speak with dolphins, and to snap cinderblock with your bare hands is always helpful though not required. Please send me your vita and one of your best papers, if you plan to be available soon. Please use postal mail or email, NOT the phone, so that I can retain my sanity. Thanks, George Drastal Siemens Corporate Research 755 College Road East Princeton, N.J. 08540 USA drastal@scr.siemens.com ------------------------------ Date: Tue, 13 Feb 96 21:22:38 GMT From: David Page Subject: ILP'96 CFP SIXTH INTERNATIONAL WORKSHOP ON INDUCTIVE LOGIC PROGRAMMING (ILP'96) CALL FOR PAPERS Below is the call for papers (Latex format) for ILP'96. Please note the submission deadline of 17 May, 1996. ILP'96 will take place on board a ship from Stockholm to Helsinki and back, 28-30 August. \documentstyle[fullpage]{article} \begin{document} \thispagestyle{empty} {\large\bf\center The Sixth International Workshop on\\ Inductive Logic Programming (ILP'96) \\} {\center 28-30 August, 1996 \\ Stockholm, Sweden \\} \vspace{.2in} This workshop is the sixth in a series of international workshops on Inductive Logic Programing. ILP'96 will be run in parallel with the Sixth International Workshop on Logic Program Synthesis and Transformation (LOPSTR'96). Papers should fit into one, or preferably more, of the following three areas. \begin{itemize} \item {\bf Theory.} Of particular interest are papers that either 1) prove new results concerning algorithms which use inductive learning to construct first or higher order logic descriptions or 2) reveal relationships to theoretical work done outside of ILP, especially work in program synthesis and transformation. \item {\bf Implementation.} Details of implemented inductive algorithms. Time complexity results should be included. \item {\bf Application.} Experimental results within one or more application areas should be tabulated with appropriate statistics. Sufficient details should be included to allow reproduction of results. Comparative studies of different algorithms running on the same examples, using the same background knowledge, are especially welcome, as are papers that explore new application areas for ILP. \end{itemize} ILP'96 and LOPSTR'96 will take place on board a ship which will sail from Stockholm to Helsinki and back during the workshop. \section*{Program Committee} \vspace{-.2in} \begin{tabbing} mmmmmmmmmmmmmmmm \= mmmmmmmmmmmmmmmm \= mmmmmmmmmmmmmmmm \= mmmmmmmmmmmmmmmm \kill \\ F. Bergadano \> P. Flach \> R. Mooney \> J.R. Quinlan \\ I. Bratko \> P. Idestam-Almquist \> S. Muggleton \> C. Rouveirol \\ L. De Raedt \> N. Lavrac \> M. Numao \> C. Sammut \\ S. D\u{z}eroski \> S. Matwin \> C.D. Page \> A. Srinivasan \\ \> \> \> S. Wrobel \\ \end{tabbing} \section*{Organization} \begin{tabbing} \= {\it Program Chair:} \= Stephen Muggleton \hspace{2in} \= {\it Local Chair:} \= Carl Gustaf Jansson \\ \>\>Oxford University Computing Laboratory \>\> Stockholm University \\ \>\>Wolfson Building, Parks Road \>\> Email: calle@dsv.su.se \\ \>\>Oxford, OX1 3QD, U.K. \\ \>\>Email: steve@comlab.ox.ac.uk \\ \end{tabbing} \section*{Deadlines} Submissions (hardcopy only) must be received by the {\bf program chair} no later than {\bf 17 May, 1996}. Submissions should include the postal address and email address (if available) of each author; the first author will be used as the contact author unless otherwise specified. Authors will be informed of acceptance by {\bf 28 June, 1996}. Some or all of the papers accepted for presentation at ILP'96 will be selected for inclusion in a post-workshop publication, at the discretion of the program committee. Notification of acceptance for the post-workshop publication also will be made by {\bf 28 June, 1996}. Authors also will then be notified of the deadline for camera-ready copies, which will be no earlier than {\bf October 15, 1996}. \end{document} ------------------------------ Date: Wed, 14 Feb 1996 03:05:51 -0400 (AST) From: Lev Goldfarb Subject: Workshop: What is inductive learning? WHAT IS INDUCTIVE LEARNING? On the foundations of AI and Cognitive Science Toronto - Canada May 20 - 21, 1996 A workshop in conjunction with the 11th Biennial Canadian AI Conference to be held at the Holiday Inn on King, Toronto during 21 - 24 May 1996 This workshop is a long overdue attempt to look at the inductive learning process as the central process generating various representations of objects (events). To this end one needs, first of all, to have a working definition of the inductive learning process, which has been lacking. Here is a starting point: Inductive learning process is the process that constructs class representation on the basis of a (small) finite set of examples, i.e. it constructs the INDUCTIVE class representation. This class representation must, in essence, provide INDUCTIVE definition (or construction) of the class. The constructed class representation, in turn, modifies the earlier representation of the objects (within the context specified by the ILP). Thus, any subsequent processes, e.g. pattern recognition, recall, problem solving, are performed on the basis of the newly constructed object (event) representations. To put it somewhat strongly, there are only inductive representations. Two main and strongly related reasons why inductive learning processs have not been perceived as the very central processes are a lack of their adequate understanding and a lack of their satisfactory formal model. Most of the currently popular formal models of inductive learning processs (including connectionist models) do not provide satisfactory inductive class representations. One can show that inductive class representations (in other words, representations of concepts and categories) cannot be adequately specified within the classical (numeric) mathematical models. We encourage all researchers (including graduate students) seriously interested in the foundations of the above areas to participate in the workshop. Both theoretical and applied contributions are welcomed (including, of course, those related to vision, speech, and language). While extended abstracts will be available at the workshop, we are planning to publish the expanded and reviewed versions of the presentations as a special issue of journal Pattern Recognition. EXTENDED ABSTRACT SUBMISSION Submit a copy (or e-mail version) of a 3-4 page extended abstract to Lev Goldfarb Faculty of Computer Science University of New Brunswick P.O. Box 4400 E-mail: goldfarb@unb.ca Fredericton, N.B. E3B 5A3 Tel: 506-453-4566 Canada Fax: 506-453-3566 E-mail submissions are encouraged. Important dates: Extended abstract due: Monday, March 25, 1996. Notification & review back to the author: Friday April 5, 1996. Final extended abstract due: Monday April 22, 1996. For more information about the Canadian AI Conference which is held in conjunction with two other conferences (Vision Interface and Graphics Interface) see: http://ai.iit.nrc.ca/cscsi/conferences/ai96.html ------------------------------ Date: Wed, 14 Feb 1996 12:08:29 -0500 (EST) From: Sue Becker Subject: NIPS*96 CALL FOR PAPERS CALL FOR PAPERS Neural Information Processing Systems -- Natural and Synthetic Monday December 2 - Saturday December 7, 1996 Denver, Colorado This is the tenth meeting of an interdisciplinary conference which brings together cognitive scientists, computer scientists, engineers, neuro- scientists, physicists, and mathematicians interested in all aspects of neural processing and computation. The conference will include invited talks and oral and poster presentations of refereed papers. The conference is single track and is highly selective. Preceding the main session, there will be one day of tutorial presentations (Dec. 2), and following will be two days of focused workshops on topical issues at a nearby ski area (Dec. 6-7). Major categories for paper submission, with example subcategories, are as follows: Algorithms and Architectures: supervised and unsupervised learning algorithms, constructive/pruning algorithms, decision trees, localized basis functions, layered networks, recurrent networks, Monte Carlo algorithms, combinatorial optimization, performance comparisons Applications: database mining, DNA/protein sequence analysis, expert systems, fault diagnosis, financial analysis, medical diagnosis, music processing, time-series prediction Artificial Intelligence and Cognitive Science: perception, natural language, human learning and memory, problem solving, decision making, inductive reasoning, hybrid symbolic-subsymbolic systems Control, Navigation, and Planning: robotic motor control, process control, navigation, path planning, exploration, dynamic programming, reinforcement learning Implementation: analog and digital VLSI, optical neurocomputing systems, novel neuro-devices, simulation tools, parallelism Neuroscience: systems physiology, signal and noise analysis, oscillations, synchronization, mechanisms of inhibition and neuromodulation, synaptic plasticity, computational models Speech, Handwriting, and Signal Processing: speech recognition, coding, and synthesis, handwriting recognition, adaptive equalization, nonlinear noise removal, auditory scene analysis Theory: computational learning theory, complexity theory, dynamical systems, statistical mechanics, probability and statistics, approximation and estimation theory Visual Processing: image processing, image coding and classification, object recognition, stereopsis, motion detection and tracking, visual psychophysics Review Criteria: All submitted papers will be thoroughly refereed on the basis of technical quality, significance, and clarity. Novelty of the work is also a strong consideration in paper selection, but, to encourage interdisciplinary contributions, we will consider work which has been submitted or presented in part elsewhere, if it is unlikely to have been seen by the NIPS audience. Authors should not be dissuaded from submitting recent work, as there will be an opportunity after the meeting to revise accepted manuscripts before submitting final camera-ready copy. Paper Format: Submitted papers may be up to seven pages in length, including figures and references, using a font no smaller than 10 point. Submissions failing to follow these guidelines will not be considered. Authors are encouraged to use the NIPS LaTeX style files obtainable by anonymous FTP at the site given below. Papers must indicate (1) physical and e-mail addresses of all authors; (2) one of the nine major categories listed above, and, if desired, a subcategory; (3) if the work, or any substantial part thereof, has been submitted to or has appeared in other scientific conferences; (4) the authors' preference, if any, for oral or poster presentation; this preference will play no role in paper acceptance; and (5) author to whom correspondence should be addressed. Submission Instructions: Send six copies of submitted papers to the address below; electronic or FAX submission is not acceptable. Include one additional copy of the abstract only, to be used for preparation of the abstracts booklet distributed at the meeting. SUBMISSIONS MUST BE RECEIVED BY MAY 24, 1996. From within the U.S., submissions will be accepted if mailed first class and postmarked by May 21, 1996. Mail submissions to: Michael Jordan NIPS*96 Program Chair Department of Brain and Cognitive Sciences, E10-034D Massachusetts Institute of Technology 79 Amherst Street Cambridge, MA 02139 USA Mail general inquiries and requests for registration material to: NIPS*96 Registration Conference Consulting Associates 451 N. Sycamore Monticello, IA 52310 fax: (319) 465-6709 (attn: Denise Prull) e-mail: nipsinfo@salk.edu Copies of the LaTeX style files for NIPS are available via anonymous ftp at ftp.cs.cmu.edu (128.2.206.173) in /afs/cs/Web/Groups/NIPS/formatting The style files and other conference information may also be retrieved via World Wide Web at http://www.cs.cmu.edu/Web/Groups/NIPS NIPS*96 Organizing Committee: General Chair, Michael Mozer, U. Colorado; Program Chair, Michael Jordan, MIT; Publications Chair, Thomas Petsche, Siemens; Tutorial Chair, John Lazzaro, Berkeley; Workshops Co-Chairs, Michael Perrone, IBM, and Steven Nowlan, Lexicus; Publicity Chair, Suzanna Becker, McMaster; Local Arrangements, Marijke Augusteijn, U. Colorado; Treasurer, Eric Mjolsness, UCSD; Government/Corporate Liaison, John Moody, OGI; Contracts, Steve Hanson, Siemens, Scott Kirkpatrick, IBM, Gerry Tesauro, IBM. Conference arrangements by Conference Consulting Associates, Monticello, IA. DEADLINE FOR RECEIPT OF SUBMISSIONS IS MAY 24, 1996 - please post - ------------------------------ Date: Wed, 14 Feb 1996 14:04:58 -0800 (PST) From: David Madigan Subject: Job openings Department of Statistics, University of Washington Tenure Track Assistant Professor position, beginning September 1996 (pending approval). Requires Ph.D in Statistics or in a related field. Duties include teaching undergraduate and graduate courses, and research. The Department is seeking to strengthen its cross-disciplinary ties and hopes to attract an applicant with research interests in this direction. Temporary Assistant Professor position, beginning September 1996 (pending availability of funds). Requires Ph.D in Statistics or in a related field. Appointment will be for one year. Duties include teaching and research. The position might also have a substantial consulting component. Send application, resume, copies of publications, and four recommendation letters by February 28 to: Statistics Faculty Search Committee, Department of Statistics, Box 354322, University of Washington, Seattle, WA 98195-4322, USA. The University of Washington is building a multicultural faculty and strongly encourages applications from female and minority candidates. AA/EOE. ------------------------------ Date: Thu, 15 Feb 1996 09:12:46 -0800 From: David Hovel Subject: MS Research Decision Theory Group Announcement ANNOUNCING MSBN, THE MICROSOFT BAYESIAN NETWORKS MODELING TOOL The members of the Decision Theory group at Microsoft Research have developed a Bayesian network construction and evaluation tool called Microsoft Bayesian Networks (MSBN). This program and its component parts are being made available free of charge for non-commercial uses by academic organizations and research institutions. WHERE IS IT ON THE WORLD WIDE WEB? Complete information about MSBN and how to obtain it can be downloaded from http://www.research.microsoft.com/dtg/msbn. WHAT IS MSBN MSBN is a 32-bit Bayesian network modeling tool which runs on the Windows95 and Windows NT platforms. It supports the creation and manipulation of Bayesian networks as well as their evaluation. The product and its components are provided on an as-is basis. MSBN is primarily composed of two executable files. The user interface program is MSBN32.EXE, which is written in Visual Basic 4.0. Inference and data management support are provided by MSBN32.DLL which is written in C++. The interface between the two binaries is fully documented and accessible from either Visual Basic or C++. A complete set of function declarations for Visual Basic is provided. In other words, the DLL can be used to construct alternate interfaces using Visual Basic, C++ or other Windows development languages. HOW DO I GET MSBN? Connect to the URL given above and print the MSBN Usage Agreement. Sign and mail the agreement along with your email information to Microsoft Corporation. After the agreement is filed at corporate headquarters, you will receive, via email, FTP logon information which will allow you to download the compressed library. ------------------------------ Date: Thu, 15 Feb 1996 22:55:35 GMT From: mshwe@ibm.net Subject: Knowledge Industries announces no-cost academic licenses for Knowledge Industries (KI) of Palo Alto, CA is pleased to announce to the academic and research communities that KI is providing no-cost licenses for its DX Solution Series software. This software allows you to build expert systems based on Bayesian networks. The licenses allow users to freely use KI modeling and inference software for academic research and teaching. Academic sites already taking advantage of the availability of KI software for research and teaching include departments at the University of Washington and Stanford University. The DX Solution Series consists of three principal components: a graphical knowledge acquisition tool called DXpress, a sample runtime interface called WIN-DX, and a set of embeddable inference libraries called API-DX. Currently, the software supports Windows 3.1, Windows NT 3.5x, and Windows 95. DXpress is a robust knowledge acquistition tool for building Bayesian networks. DXpress uses several knowledge acquisition acceleration techniques to reduce the amount of time needed to develop an expert system, including causal independence and probability partitions. DXpress also provides forms for entering auxiliary information used in most runtime appliations, such as questions and definitions for observations. Written entirely in C++, DXpress rapidly updates its graphical windows even when large networks are loaded. To provide a rapid develop-and-test environment, DXpress automatically calls the WIN-DX runtime system, in which you can instantly see the effects of changes to your knowledge base. Alternatively, you can build your own runtime system using API-DX, accessing API-DX from Visual C++ or Visual Basic. The KI software will be kept up-to-date on standards adopted by the UAI community for interchange of Bayesian-network models. For more information on KI and the DX Solution Series, please refer to the KI web site: http://www.kic.com. You will also find on the KI web site a copy of the KI academic license agreement, which you may download, print, sign, and return to KI for a no-cost license to use the KI software. In addition, you may send email to ki@kic.com, call us at 415-321-0400, or fax us at 415-322-3554. ------------------------------ Date: Sun, 18 Feb 1996 11:15:50 +1100 (EST) From: xindong@insect.sd.monash.edu.au Subject: KDD Book Announcement Author: Xindong Wu Book title: Knowledge Acquisition from Databases Publisher: Ablex, USA Year: 1995 ISBN: 1-56750-206-7 (cloth cover); 1-56750-205-9 (paper cover) Knowledge acquisition from databases is a research frontier for both database technology and machine learning techniques, and has seen sustained research in recent years. It also acts as a link between the two fields, thus offering a dual benefit. First, because database technology has already found wide application in many fields, machine learning research obviously stands to gain from this greater exposure and established technological foundation. Second, machine learning techniques can augment the ability of existing database systems to represent, acquire, and process a collection of expertise such as those that form part of the semantics of many advanced applications, for example, computer-aided design (CAD) and computer-aided manufacturing (CAM). This book contains three parts. Part I surveys the area of knowledge acquisition from databases and figures out some of the major problems. Part II provides an overview of symbolic methods in machine learning and describes two types of rule induction algorithms to facilitate the acquisition of knowledge from databases: the decision tree-based ID3-like algorithms and the extension matrix-based induction algorithms. The author's own HCV induction algorithm based on the newly developed extension matrix approach is described as a counterpart to ID3-like algorithms. Two practical issues, noise handling and processing real-valued attributes in the context of knowledge acquisition from databases, are addressed in detail, and a performance comparison of different learning algorithms (ID3, C4.5, NewID, and HCV) is also provided in terms of rule compactness and accuracy on a battery of experimental data sets including three famous classification problems, the MONK's problems. Finally, in Part III, an intelligent learning database system, KEshell2, which makes use of the HCV algorithm and couples machine learning techniques with database and knowledge base technology, is described with examples. The parts of the book have different but interrelated objectives and suit different levels of readership. Part II can be adopted as an inductive learning module in an artificial intelligence (AI) related undergraduate and/or postgraduate course. Part III can be integrated into a machine learning or advanced database course. Together with the brief overview in Part I, this book as a whole should be of interest to the whole intelligent databases and machine learning community and to students in machine learning, expert systems, and advanced database courses. Knowledge acquisition from databases could well form an independent honors or postgraduate course in a computer science or information systems program, and therefore this book could be adopted as a textbook. The book is based on the author's papers and reports produced over the past few years. Contact details for the publisher and a short PostScript file with a table of contents can be found at the following web address: http://www.sd.monash.edu.au/~xindong/Publication/KDD.html. ------------------------------ Date: Mon, 19 Feb 1996 22:12:15 -0500 From: IMLM Workshop (pkc) Subject: 2nd CFP: AAAI-96 Workshop on Integrating Multiple Learned Models Paper submission deadline: March 18, 1996 CALL FOR PAPERS/PARTICIPATION INTEGRATING MULTIPLE LEARNED MODELS FOR IMPROVING AND SCALING MACHINE LEARNING ALGORITHMS to be held in conjunction with AAAI 1996 (collocated with KDD-96, UAI-96, and IAAI-96) Portland, Oregon August 1996 http://www.cs.fit.edu/~imlm/ or http://cs.fit.edu/~imlm/ ------------------------------ Date: Tue, 20 Feb 1996 17:17:58 +1100 From: Jonathan Oliver Subject: Final CFP for ISIS: Information, Statistics and Induction in Science ISIS: Information, Statistics and Induction in Science Melbourne, Australia, 20-23 August 1996 Conference Chair: David Dowe Co-chairs: Kevin Korb and Jonathan Oliver Invited Speakers: Henry Kyburg, Jr. (University of Rochester, NY) Marvin Minsky (MIT) J. Ross Quinlan (Sydney University) Jorma J. Rissanen (IBM Almaden Research, San Jose, California) Ray Solomonoff (Oxbridge Research, Mass) This conference will explore the use of computational modelling to understand and emulate inductive processes in science. The problems involved in building and using such computer models reflect methodological and foundational concerns common to a variety of academic disciplines, especially statistics, artificial intelligence (AI) and the philosophy of science. This conference aims to bring together researchers from these and related fields to present new computational techniques for supporting or analysing scientific inference and to engage in collegial debate over the merits and difficulties underlying the various approaches to automating inductive and statistical inference. PROGRAM COMMITTEE Hirotugu Akaike, Lloyd Allison, Shun-ichi Amari, Mark Bedau, Jim Bezdek, Hamparsum Bozdogan, Wray Buntine, Peter Cheeseman, Honghua Dai, David Dowe, Usama Fayyad, Doug Fisher, Alex Gammerman, Clark Glymour, Randy Goebel, Josef Gruska, David Hand, Bill Harper, David Heckerman, Colin Howson, Lawrence Hunter, Frank Jackson, Max King, Kevin Korb, Henry Kyburg, Rick Lathrop, Ming Li, Nozomu Matsubara, Aleksandar Milosavljevic, Richard Neapolitan, Jon Oliver, Michael Pazzani, J. Ross Quinlan, Glenn Shafer, Peter Slezak, Ray Solomonoff, Paul Thagard, Neil Thomason, Raul Valdes-Perez, Tim van Gelder, Paul Vitanyi, Chris Wallace, Geoff Webb, Xindong Wu, Jan Zytkow. Inquiries to: isis96@cs.monash.edu.au David Dowe: dld@cs.monash.edu.au Kevin Korb: korb@cs.monash.edu.au or Jonathan Oliver: jono@cs.monash.edu.au Information is available on the WWW at: http://www.cs.monash.edu.au/~jono/ISIS/ISIS.shtml ------------------------------ Date: Tue, 20 Feb 1996 14:28:27 GMT From: R Roy Subject: 2nd WEC On-Line workshop.. 2nd On-line Workshop on Evolutionary Computation March 4-22 , 1996 Special Session on APPLICATIONS OF EVOLUTIONARY COMPUTATION IN ENGINEERING Call for Papers and Invitation to take part in the Discussion On behalf of Plymouth Engineering Design Centre, Plymouth, UK, we call for your papers for the special session on 'Applications of Evolutionary Computation in Engineering'. The session is planned to integrate experience from different engineering applications of Evolutionary Computing. The objective of the special session is to put forwarda common forum where people from different engineering disciplines can talk to each other and share their experience. The application areas of interest include but are not limited to : a. Biotechnology b. Chemical Engineering c. Computer Aided Design, Control and Manufacturing d. Engineering Economics and Management e. Systems Engineering f. Artificial Intelligence g. Pattern Recognition h. Electronics and Telecommunications i. Medicine and Medical Engineering j. Military Applications The contributions can be in the form of : a) 4 page (max) Original paper, to be published in a proceedings b) 4 page (max) paper on your already published work c) your participation in the discussion during the workshop Please refer to the ORIGINAL CALL FOR PAPERS below for the submission and further details. PLEASE MENTION ABOUT THE SPECIAL SESSION IN YOUR LETTER WHEN YOU SUBMIT A PAPER TO JAPAN. For further details about the special session please contact any one of the Joint Session Organisers from Plymouth Engineering Design Centre, Plymouth, PL4 8AA, UK: IAN PARMEE iparmee@plymouth.ac.uk RAJKUMAR ROY rroy@plymouth.ac.uk GEORGE BILCHEV gbilchev@plymouth.ac.uk ------------------------------ Date: Tue, 20 Feb 96 08:45:20 PST From: KDD-96 Account Subject: Final CFP for KDD-96 NOTE: MARCH 18TH DEADLINE for: (1) hardcopies of papers due to AAAI (2) electronic ascii of cover page due to kdd96@almaden.ibm.com FINAL CALL FOR PAPERS The Second International Conference on Knowledge Discovery and Data Mining (KDD-96) Portland, Oregon, USA, August 2-4, 1996 ======================================= Sponsored by AAAI and Collocated with AAAI-96 and UAI-96. visit the KDD-96 WWW page at http://www-aig.jpl.nasa.gov/kdd96 Knowledge Discovery in Databases (KDD), also referred to as Data Mining, is an area of common interest to researchers in machine discovery, statistics, databases, knowledge acquisition, machine learning, data visualization, high performance computing, and knowledge-based systems. The rapid growth of data and information has created a need and an opportunity for extracting knowledge from databases, and both researchers and application developers have been responding to that need. KDD applications have been developed for astronomy, biology, finance, insurance, marketing, medicine, and many other fields. The first international conference on Knowledge Discovery and Data Mining (KDD-95), held in Montreal in August 1995, was an outstanding success, attracting over 340 participants. The second international conference will follow up the success of KDD-95 by bringing together researchers and application developers from different areas focusing on unifying themes. The topics of interest include, but are not limited to: Theory and Foundational Issues in KDD: Data and knowledge representation for KDD Probabilistic modeling and uncertainty management in KDD Modeling of structured, unstructured and multimedia data Metrics for evaluation of KDD results Fundamental advances in search, retrieval, and discovery methods Definitions, formalisms, and theoretical issues in KDD Data Mining Methods and Algorithms: Algorithmic complexity, efficiency and scalability issues in data mining Probabilistic and statistical models and methods Using prior domain knowledge and re-use of discovered knowledge Parallel and distributed data mining techniques High dimensional datasets and data preprocessing Unsupervised discovery and predictive modeling KDD Process and Human Interaction: Models of the KDD process Methods for evaluating subjective relevance and utility Data and knowledge visualization Interactive data exploration and discovery Privacy and security Applications: Data mining systems and data mining tools Application of KDD in business, science, medicine and engineering Application of KDD methods for mining knowledge in text, image, audio, sensor, numeric, categorical or mixed format data Resource and knowledge discovery using the Internet This list of topics is not intended to be exhaustive but an indication of typical topics of interest. Prospective authors are encouraged to submit papers on any topics of relevance to knowledge discovery and data mining. DEMONSTRATION SESSIONS: KDD-96 also invites working demonstrations of discovery systems. Exact details on how to arrange a demo at KDD-96 will be forthcoming. BEST PAPER AWARDS: Two papers, one on research and one on applications, will be selected by the Program Committee for the best paper awards for KDD'96. In addition, a selected set of papers, after being extended to journal length and quality, will be considered for publication in the new journal, Journal of Data Mining and Knowledge Discovery. SUBMISSION AND REVIEW CRITERIA: Both research and applications papers are solicited. All submitted papers will be reviewed on the basis of technical quality, relevance to KDD, novelty, significance, and clarity. Authors are encouraged to make their work accessible to readers from other disciplines by including a carefully written introduction. Papers should clearly state their relevance to KDD. Please submit 5 *hardcopies* of a short paper (a maximum of 9 single-spaced pages not including cover page but including bibliography, 1 inch margins, and 12pt font)to be received by March 18, 1996. A cover page must include author(s) full address, E-MAIL, paper title and a 200 word abstract, and up to 5 keywords. This cover page must accompany the paper. In addition, an ASCII version of the cover page should be sent electronically via email to kdd96@almaden.ibm.com by March 18th 1995 (preferably earlier for e-mail). For the electronic title page, authors are required to use the template made available by ftp. Please visit http://www-aig.jpl.nasa.gov/kdd96/ to retrieve the electronic template. Please mail the 5 hardcopies of the full papers to : AAAI (KDD-96) 445 Burgess Drive Menlo Park, CA 94025-3496 U.S.A. Phone: (+1 415) 328-3123; Fax: (+1 415) 321-4457 Email: kdd@aaai.org *********** I m p o r t a n t D a t e s **************** * 5 copies of full papers received by: March 18, 1996 * * (in addition to an electronic ASCII title page) * * acceptance notices: April 19, 1996 * * final camera-readies due to AAAI by: May 20, 1996 * ********************************************************** Program Co-chairs: ================== Evangelos Simoudis (IBM Almaden Research Center) Jia Wei Han (Simon Fraser University) KDD-96 Organization: ==================== General Conference Chair: Usama M. Fayyad, Jet Propulsion Laboratory KDD-96 Publicity Chair: Padhraic Smyth, Jet Propulsion Laboratory KDD Sponsorship Chair: Gregory Piatetsky-Shapiro, GTE Laboratories Program Committee ================= Rakesh Agrawal (IBM Almaden Research Center, USA) Tej Anand (AT&T Global Information Solutions, USA) Ron Brachman (AT&T Bell Laboratories, USA) Wray Buntine (Thinkbank, Inc., USA) Nick Cercone (University of Regina, Canada) Peter Cheeseman (NASA AMES Research Center, USA) Bruce Croft (University of Massachusetts at Amherst, USA) Stephen G. Eick (AT&T Bell Laboratories, USA) Usama Fayyad (Jet Propulsion Laboratory, USA) Clark Glymour (Carnegie-Mellon University, USA) George Grinstein (University of Massachusetts at Lowell, USA) David Hand (Open University, UK) David Heckerman (Microsoft Corporation, USA) Se June Hong (IBM T.J. Watson Research Center, USA) Tomasz Imielinski (Rutgers University, USA) Larry Jackel (AT&T Bell Laboratories, USA) Larry Kerschberg (George Mason University, USA) Willi Kloesgen (GMD, Germany) David Madigan (University of Washington, USA) Heikki Mannila (University of Helsinki, Finland) Chris Matheus (GTE Laboratories, USA) Sham Navathe (Georgia Institute of Technology, USA) Raymond Ng (University of British Columbia, Canada) Gregory Piatetsky-Shapiro (GTE Laboratories, USA) Daryl Pregibon (AT&T Bell Laboratories, USA) Pat Riddle (Boeing Computer Services, USA) Ted Senator (US Department of the Treasury, USA) Wei-Min Shen (University of Southern California, USA) Arno Siebes (CWI, Netherlands) Avi Silberschatz (AT&T Bell Laboratories) Andrzej Skowron (University of Warsaw, Poland) Steve Smith (Dun and Bradstreet, USA) Padhraic Smyth (Jet Propulsion Laboratory, USA) Ramakrishnan Srikant (IBM Almaden Research Center, USA) Sal Stolfo (Columbia University, USA) Alex Tuzhilin (NYU Stern School, USA) Ramasamy Uthurusamy (General Motors R&D Center, USA) Xindong Wu (Monash University, Australia) Wojciech Ziarko (University of Regina, Canada) Jan Zytkow (Wichita State University, USA) For further information, send inquiries regarding * submission logistics to AAAI at kdd@aaai.org Phone: (+1 415) 328-3123; Fax: (+1 415) 321-4457 * KDD-96 sponsorship and industry participation to: Gregory Piatetsky-Shapiro at gps@gte.com Phone: 617-466-4236, Fax: 617-466-2960 * technical program and content to kdd96@almaden.ibm.com * general and publicity issues to kdd96@aig.jpl.nasa.gov ------------------------------ Date: Wed, 21 Feb 96 02:55:49 EST From: Ibrahim Fahmi Imam Subject: CFP: AAAI-96 WS on INTELLIGENT ADAPTIVE AGENTS C A L L F O R P A P E R S AAAI-96 International Workshop on Intelligent Adaptive Agents (IAA-96) August 4-8, 1996, Portland, Oregon In recent years, researchers from different fields have pushed toward greater flexibility and intelligent adaptation in their systems. The development of intelligent adaptive agents has been rapidly evolving in many fields of science. Such systems should have the capability of dynamically adapting their parameters, improve their knowledge-base or method of operation in order to accomplish a set of tasks. This workshop will focus on intelligent adaptation and its relationship to other fields of interest. Research issues that are of interest to the workshop include but are not limited to: 1) Analyzing the role of adaptation in planning, execution monitoring, and problem-solving; 2) Adaptive control in real-world engineering systems; 3) Analyzing the computational cost of adaptation vs. system robustness; 4) Controlling the adaptive process (what is the strategy? what is needed?, what is expected?, etc.); 5) Adaptive mechanisms in an open agent society; 6) Adaptation in distributed systems; The workshop seeks high quality submission in these areas. Researchers interested in submitting papers should explain the adaptive process in light of one or more of the issues presented above. Papers with real-world applications are strongly encouraged. Please send any questions to: Ibrahim F. Imam at iimam@aic.gmu.edu Program Committee Members Jaime Carbonell, Carnegie Mellon University, USA Gerald DeJong, University of Illinois at Urbana-Champaign, USA Tim Finin, University of Maryland Baltimore County, USA Brian Gaines, University of Calgary, Canada Diana Gordon, Naval Research Laboratory, USA Yves Kodratoff, Universite de Paris Sud, France Ryszard Michalski, George Mason University, USA Ashwin Ram, Georgia Institute of Technology, USA Nigel Shadbolt, University of Nottingham, England Reid Simmons, Carnegie Mellon University, USA Walter Van de Velde, Vrije Universiteit Brussel, Belgium Brad Whitehall, United Technologies Research Center, USA Stefan Wrobel, GMD, Germany Submission Information Paper submissions should not exceed eight single-spaced pages, with 1 inch margins, 12pt font. The first page must show the title, authors' names, full surface mail addresses, fax number (if possible), email addresses, short abstract (does not exceed 200 words), and a list of key words (up to 5). Electronic submissions are strongly encouraged and should be sent to iimam@aic.gmu.edu Otherwise, contact the workshop chair at (iimam@aic.gmu.edu) for mailing arrangements. An extended version of the CFP can be found in: http://www.mli.gmu.edu/~iimam/aaai96.html Important Dates Submission Deadline: March 18, 1996 Notification Date: April 15, 1996 Camera-Ready Due: May 13, 1996 Workshop: August 4, 1996 ------------------------------ Date: Fri, 23 Feb 96 13:22:57 GMT From: tcf@btc.uwe.ac.uk Subject: ICML'96 Workshop on Evolutionary Computing and Machine Learning ICML'96 Workshop on EVOLUTIONARY COMPUTING AND MACHINE LEARNING to be held in Bari, Italy, July 2-3, 1996, at the International Conference on Machine Learning. http://zen.btc.uwe.ac.uk/evol/cfp.html In the last decade, research concentrating on the interaction between evolutionary computing and machine learning has developed from the study and use of genetic algorithms and reinforcement learning in rule based systems (i.e. classifier systems) to a variety of learning systems such as neural networks, fuzzy systems and hybrid symbolic/evolutionary systems. Many kinds of learning process are now being integrated with evolutionary algorithms, e.g. supervised, unsupervised, reinforcement, on/off-line and incremental. The aim of this workshop is to bring together people involved and interested in this field to share common theory and practice, and to represent the state of the art. Submissions are invited on topics related to: machine learning using evolutionary algorithms, the artificial evolution of machine learning systems, systems exploring the interaction between evolution and learning, systems integrating evolutionary and machine learning algorithms and on applications that make use of both machine learning and evolutionary algorithms. Contributions that argue a position, give an overview, give a review, or report recent work are all encouraged. Copies of extended abstracts or full papers no longer than 15 pages should be sent (by April 23 1996) to: Terry Fogarty Faculty of Computer Studies and Mathematics University of the West of England Frenchay Phone: (+44) 117 965 6261 Bristol BS16 1QY Fax: (+44) 117 975 0416 UK Email: tcf@btc.uwe.ac.uk or: Gilles Venturini Ecole d'Ingenieurs en Informatique pour l'Industrie Universite de Tours, 64, Avenue Jean Portalis, Phone: (+33)-47-36-14-33 Technopole Boite No 4, Fax: (+33)-47-36-14-22 37913 Tours Cedex 9 Email: venturi@lri.fr FRANCE venturini@univ-tours.fr Accepted papers will constitute the workshop notes and will be refereed by the program committee for inclusion in the post-workshop proceedings in the light of scientific progress made at the workshop. Program committee: F. Baiardi, University of Pisa, Italy. H. Bersini, Universite Libre de Bruxelles, Belgium. L.B. Booker, MITRE Corporation, USA. D. Cliff, University of Sussex, UK. M. Colombetti, Politecnico di Milano, Italy. K. De Jong, George Mason University, USA. M. Dorigo, Universite Libre de Bruxelles, Belgium. T.C. Fogarty, University of the West of England, UK. A. Giordana, University of Torino, Italy. J.G. Grefenstette, Naval Research Laboratory, USA. J.A. Meyer, Ecole Normale Superieure, France. M. Patel, University of Newcastle, UK. M. Schoenauer, Ecole Polytechnique, France. R.E. Smith, University of Alabama, USA. G. Venturini, University of Tours, France. S.W. Wilson, Rowland Institute for Science, USA. Important Dates: April 23: Extended abstracts and papers due May 14: Notification of acceptance June 4: Camera-ready copy for workshop notes due July 2-3: Workshop ------------------------------ Date: Sat, 24 Feb 1996 11:15:45 -0700 (MST) From: "Lorien Y. Pratt" Subject: Special Issue of the Machine Learning Journal: Inductive Transfer Call for papers Special Issue of the Machine Learning Journal on Inductive Transfer Lorien Pratt and Sebastian Thrun, Guest Editors Many recent machine learning efforts are focusing on the question of how to learn in an environment in which more than one task is performed by a system. As in human learning, related tasks can build on one another, tasks that are learned simultaneously can cross-fertilize, and learning can occur at multiple levels, where the learning process itself is a learned skill. Learning in such an environment can have a number of benefits, including speedier learning of new tasks, a reduced number of training examples for new tasks, and improved accuracy. These benefits are especially apparent in complex applied tasks, where the combinatorics of learning are often otherwise prohibitive. Current efforts in this quickly growing research area include investigation of methods that facilitate learning multiple tasks simultaneously, those that determine the degree to which two related tasks can benefit from each other, and methods that extract and apply abstract representations from a source task to a new, related, target task. The situation where the target task is a specialization of the source task is an important special case. The study of such methods has broad application, including a natural fit to data mining systems, which extract regularities from heterogeneous data sources under the guidance of a human user, and can benefit from the additional bias afforded by inductive transfer. We solicit papers on inductive transfer and learning to learn for an upcoming Special Issue of the Machine Learning Journal. Please send six (6) copies of your manuscript postmarked by June 1, 1996 to: Dr. Lorien Pratt MCS Dept. CSM Golden, CO 80401 USA One (1) additional copy should be mailed to: Karen Cullen Attn: Special Issue on Inductive Transfer MACHINE LEARNING Editorial Office Kluwer Academic Publishers 101 Philip Drive Assinippi Park Norwell, MA 02061 USA Manuscripts should be limited to at most 12000 words. Please also note that Machine Learning is now accepting submission of final copy in electronic form. Authors may want to adhere to the journal formatting standards for paper submissions as well. There is a latex style file and related files available via anonymous ftp from ftp.std.com. Look in Kluwer/styles/journals for the files README, kbsfonts.sty, kbsjrnl.ins, kbsjrnl.sty, kbssamp.tex, and kbstmpl.tex, or the file kbsstyles.tar.Z, which contains them all. Please see http://vita.mines.edu:3857/1s/lpratt/transfer.html for more information on inductive transfer. Papers will be quickly reviewed for a target publication date in the first quarter of 1997. Dr. Lorien Y. Pratt Department of Mathematical and Computer Sciences lpratt@mines.edu Colorado School of Mines (303) 273-3878 (work) 402 Stratton (303) 278-4552 (home) Golden, CO 80401, USA Vita, photographs, all publications, all course materials available from my web page: http://vita.mines.edu:3857/1s/lpratt ------------------------------ End of ML-LIST (Digest format) **************************************** From kak@ee.lsu.edu Mon Feb 26 14:21:44 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 26 Feb 96 14:21:33 -0600; AA03870 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 26 Feb 96 14:21:26 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa15270; 23 Feb 96 21:29:39 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa15268; 23 Feb 96 21:19:31 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa07471; 23 Feb 96 21:18:41 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa20183; 23 Feb 96 18:56:34 EST Received: from gate.ee.lsu.edu by CS.CMU.EDU id aa08078; 23 Feb 96 18:56:24 EST Received: by ee.lsu.edu (4.1/SMI-4.1) id AA19148; Fri, 23 Feb 96 17:55:23 CST Date: Fri, 23 Feb 96 17:55:23 CST From: Subhash Kak Message-Id: <9602232355.AA19148@ee.lsu.edu> To: ai-stats@watstat.uwaterloo.ca, alife@cognet.ucla.edu, colt@cs.uiuc.edu, connectionists@cs.cmu.edu, neuronet@tutkie.tut.ac.jp Subject: Paper The following paper =========================================== ON BIOLOGICAL CYCLES AND INTELLIGENCE By Subhash C. Kak may be obtained by ftp from the following location: ftp://gate.ee.lsu.edu/pub/kak/bio.ps.Z -------------------------------------------- Abstract: If intelligence is taken as a measure of the organism's ability to adapt to its environment, the question of the organism's sensitivity to the rhythms of the environment becomes important. In this paper we provide a summary, including a brief historical resume, of this question of timing. We arge that artificial connectionist systems designed to range in natural environments will have to incorporate a system of inner clocks. --------------------------------------------- -Subhash Kak From hinton@cs.toronto.edu Mon Feb 26 14:21:49 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 26 Feb 96 14:21:46 -0600; AA03879 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 26 Feb 96 14:21:36 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa19803; 25 Feb 96 20:39:28 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa19801; 25 Feb 96 20:26:06 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa09151; 25 Feb 96 20:25:30 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa19247; 25 Feb 96 13:01:40 EST Received: from yonge.cs.toronto.edu by RI.CMU.EDU id aa26592; 25 Feb 96 13:01:00 EST Received: from neuron.ai.toronto.edu ([128.100.3.14]) by yonge.cs.toronto.edu with SMTP id <86544>; Sun, 25 Feb 1996 13:00:48 -0500 Received: from localhost by neuron.ai.toronto.edu with SMTP id <525>; Sun, 25 Feb 1996 13:00:37 -0500 To: Mailing List Subject: obvious, but false Date: Sun, 25 Feb 1996 13:00:36 -0500 From: Geoffrey Hinton Message-Id: <96Feb25.130037edt.525@neuron.ai.toronto.edu> In response to may email about a network that learns shift invariance, Dorffner says: > there seems to be a misunderstanding of what the topic of discussion > is here. I don't think that Jerry meant that no model consisting of neural > network components could ever learn shift invariance. After all, there are > many famous examples in visual recognition with neural networks (such as > the Neocognitron, as Rolf W"urtz pointed out), and if this impossibility > were the case, we would have to give up neural network research in > perceptual modeling altogether. > > What I think Jerry meant is that any cascade of fully-connected feed-forward > connection schemes between layers (including the perceptron and the MLP) > cannot learn shift invariance. > Now besides being obvious, this does raise some > important questions, possibly weakening the fundamentals of connectionism. I agree that this is what Jerry meant. What Jerry said was actually very reasonable. He did NOT say it was obviously impossible. He just said that it was generally understood to be impossible and he would like to see a proof. I think Jerry was right in the sense that most people I have talked to believed it to be impossible. I'd like to apologize to Jerry for the antagonistic tone of my previous message. Dorffner takes the impossibility for granted. My simulation conclusively demonstrates that translation invariance can be learned with no built in bias towards translation invariance. The only requirement is that the shapes should share features, and this is a requirement on the data, not on the network. At the risk of looking very silly, I bet that it really cannot be done if shapes do not share features. My simulation did not have built in preprocessing or weight-sharing as Dorffner seems to imply. So, unlike the neocognitron, it had no innate bias towards translation invariance. It got the "raw" retinal inputs and its desired outputs were shape identities. The version with local connectivity worked best, but as I pointed out, it also worked without local connectivity. So that version exactly fitted Dorffner's definition of what cannot be done. Geoff PS: As I noted in the paper and others have pointed out in their responses, Minsky and Papert's group invariance theorem really does prove that this task cannot be done without hidden layers (using conventional units). From hinton@cs.toronto.edu Mon Feb 26 14:21:53 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 26 Feb 96 14:21:48 -0600; AA03887 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 26 Feb 96 14:21:45 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab19803; 25 Feb 96 20:40:16 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab19801; 25 Feb 96 20:26:07 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa09155; 25 Feb 96 20:25:41 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id ac20789; 25 Feb 96 15:27:44 EST Received: from yonge.cs.toronto.edu by EDRC.CMU.EDU id aa23006; 25 Feb 96 13:34:12 EST Received: from neuron.ai.toronto.edu ([128.100.3.14]) by yonge.cs.toronto.edu with SMTP id <86531>; Sun, 25 Feb 1996 13:34:06 -0500 Received: from localhost by neuron.ai.toronto.edu with SMTP id <525>; Sun, 25 Feb 1996 13:33:54 -0500 To: Mailing List Subject: yet more on shift invariance Date: Sun, 25 Feb 1996 13:33:46 -0500 From: Geoffrey Hinton Message-Id: <96Feb25.133354edt.525@neuron.ai.toronto.edu> The argument that Jerry Feldman gave for the difficulty of learning shift invariance illustrates a nice point about learning. He talked about learning to recognize a SINGLE shape in different locations. I think he is probably right that this is impossible without built in prejudices. But the implication was that if it was true for ONE shape then it would be true for a bunch of shapes. This is where the argument breaks down. Its like the point being made by the life-time learning people, except that here the separate tasks are not presented one after another but jumbled together. Geoff From geva@fit.qut.edu.au Mon Feb 26 14:21:54 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 26 Feb 96 14:21:50 -0600; AA03894 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 26 Feb 96 14:21:47 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ac19803; 25 Feb 96 20:41:31 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ac19801; 25 Feb 96 20:26:09 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa09160; 25 Feb 96 20:26:02 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa23941; 25 Feb 96 19:26:10 EST Received: from ocean.fit.qut.edu.au by RI.CMU.EDU id aa27405; 25 Feb 96 19:25:22 EST Received: (from geva@localhost) by ocean.fit.qut.edu.au (8.7.4/8.7.2) id KAA13487; Mon, 26 Feb 1996 10:05:33 +1000 (EST) Date: Mon, 26 Feb 1996 10:05:33 +1000 (EST) From: Shlomo Geva To: connectionists@cs.cmu.edu Cc: Shlomo Geva Subject: Re: shift invariance In-Reply-To: <199602230205.VAA13842@bucnsd.qut.edu.au> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Regarding shift invariance: One might learn something about the problem by looking at the Fourier Transform in the context of shift invariance. One may perform a Discrete Fourier Transform (DFT), and take advantage of the shift invariance properties of the magnitudes components, discarding the phase and representing objects by a feature vector consisting of the the magnitudes in frequency domain alone. (This is not new and also extends to higher dimensionalities) This approach will solve many practical problems, but has an in-principle difficulty in that this procedure does not produce a unique mapping from objects to invariant features. For example, start from any object and obtain its invariant representation as above. By choosing arbitrary phase components and performing an inverse DFT we can get arbitrarily many object representations. Note that these objects are extremely unlikely to look like an original shifted object! If by chance - and it may be very remote - two of the objects you wish to recognize with shift invariance, have identical magnitudes in the frequency domain then this method will obviously fail. Now I'd like to make a conjecture. It appears to make sense to assume that this difficulty is inherent to the shift invariance requirement itself. If this is so then unless you have an additional constraint imposed on objects - they cannot be allowed to be identical under the invariant feature extraction transformation you wish to employ - then you cannot solve the problem. In other words, one needs a guarantee that all permissible objects are uniquely transformed by the procedure. It seems to follow that no general procedure, that does not take into account the nature of the objects for which the procedure is intended, can exist. I am wondering if anyone could clarify whether this is a valid argument. Shlomo Geva s.geva@qut.edu.au From juergen@idsia.ch Tue Feb 27 23:44:40 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 27 Feb 96 23:44:36 -0600; AA02041 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 27 Feb 96 23:44:34 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa20378; 26 Feb 96 3:26:36 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa20376; 26 Feb 96 3:10:51 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa09513; 26 Feb 96 3:10:16 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa28243; 26 Feb 96 3:01:09 EST Received: from chsun.eunet.ch by RI.CMU.EDU id aa28430; 26 Feb 96 3:00:23 EST Received: from fava.idsia.ch by chsun.eunet.ch (8.6.10/1.34) id JAA10093; Mon, 26 Feb 1996 09:00:02 +0100 Received: from kraut.idsia.ch by fava.idsia.ch (4.1/SMI-4.1) id AA24691; Mon, 26 Feb 96 09:00:00 +0100 Date: Mon, 26 Feb 96 09:00:00 +0100 From: Juergen Schmidhuber Message-Id: <9602260800.AA24691@fava.idsia.ch> To: connectionists@cs.cmu.edu Subject: neural text compression Now available online: SEQUENTIAL NEURAL TEXT COMPRESSION (9 pages, 68 K) IEEE Transactions on Neural Networks, 7(1):142-146, 1996 Juergen Schmidhuber, IDSIA Stefan Heil, TUM http://www.idsia.ch/~juergen Abstract: Neural nets may be promising tools for loss-free data compression. We combine predictive neural nets and statistical coding techniques to compress text files. We apply our methods to short newspaper articles and obtain compression ratios exceeding those of widely used Lempel-Ziv algorithms (the basis of UNIX functions `compress' and `gzip'). The main disadvantage of our methods is: on conventional machines they are about three orders of magnitude slower than standard methods. To obtain a copy, cut and paste this: netscape ftp://ftp.idsia.ch/pub/juergen/textcompression.ps.gz ------------ P.S.: Have you got a question on recent work on "learning to learn" and "incremental self-improvement"? Stewart Wilson asked me to place the corresponding paper "Environment-independent reinforcement acceleration" in his NetQ web site. Now it is sitting there and waiting for a friendly question or two (questions may be anonymous): netscape http://netq.rowland.org Juergen Schmidhuber, IDSIA From perso@DI.Unipi.IT Tue Feb 27 23:44:41 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 27 Feb 96 23:44:39 -0600; AA02047 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 27 Feb 96 23:44:36 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa21128; 26 Feb 96 13:34:09 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa21126; 26 Feb 96 13:16:51 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa10305; 26 Feb 96 13:15:38 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa00397; 26 Feb 96 6:42:06 EST Received: from apollo.di.unipi.it by EDRC.CMU.EDU id aa25544; 26 Feb 96 6:41:30 EST Organization: Dipartimento di Informatica di Pisa - Italy Received: from neuron.di.unipi.it (neuron.di.unipi.it [131.114.4.65]) by mailserver.di.unipi.it (8.6.12/8.6.12) with ESMTP id MAA20500 for ; Mon, 26 Feb 1996 12:36:05 +0100 From: Alessandro Sperduti Received: (perso@localhost) by neuron.di.unipi.it (8.6.12/8.6.12) id MAA11154 for connectionists@cs.cmu.edu; Mon, 26 Feb 1996 12:36:04 +0100 Message-Id: <199602261136.MAA11154@neuron.di.unipi.it> Subject: ICML'96 W/S on EC&ML To: connectionists@cs.cmu.edu Date: Mon, 26 Feb 1996 12:36:04 +0100 (MET) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 3562 ICML'96 Workshop on EVOLUTIONARY COMPUTING AND MACHINE LEARNING to be held in Bari, Italy, July 2-3, 1996, at the International Conference on Machine Learning. http://zen.btc.uwe.ac.uk/evol/cfp.html In the last decade, research concentrating on the interaction between evolutionary computing and machine learning has developed from the study and use of genetic algorithms and reinforcement learning in rule based systems (i.e. classifier systems) to a variety of learning systems such as neural networks, fuzzy systems and hybrid symbolic/evolutionary systems. Many kinds of learning process are now being integrated with evolutionary algorithms, e.g. supervised, unsupervised, reinforcement, on/off-line and incremental. The aim of this workshop is to bring together people involved and interested in this field to share common theory and practice, and to represent the state of the art. Submissions are invited on topics related to: machine learning using evolutionary algorithms, the artificial evolution of machine learning systems, systems exploring the interaction between evolution and learning, systems integrating evolutionary and machine learning algorithms and on applications that make use of both machine learning and evolutionary algorithms. Contributions that argue a position, give an overview, give a review, or report recent work are all encouraged. Copies of extended abstracts or full papers no longer than 15 pages should be sent (by April 23 1996) to: Terry Fogarty Faculty of Computer Studies and Mathematics University of the West of England Frenchay Phone: (+44) 117 965 6261 Bristol BS16 1QY Fax: (+44) 117 975 0416 UK Email: tcf@btc.uwe.ac.uk or: Gilles Venturini Ecole d'Ingenieurs en Informatique pour l'Industrie Universite de Tours, 64, Avenue Jean Portalis, Phone: (+33)-47-36-14-33 Technopole Boite No 4, Fax: (+33)-47-36-14-22 37913 Tours Cedex 9 Email: venturi@lri.fr FRANCE venturini@univ-tours.fr Accepted papers will constitute the workshop notes and will be refereed by the program committee for inclusion in the post-workshop proceedings in the light of scientific progress made at the workshop. Program committee: F. Baiardi, University of Pisa, Italy. H. Bersini, Universite Libre de Bruxelles, Belgium. L.B. Booker, MITRE Corporation, USA. D. Cliff, University of Sussex, UK. M. Colombetti, Politecnico di Milano, Italy. K. De Jong, George Mason University, USA. M. Dorigo, Universite Libre de Bruxelles, Belgium. T.C. Fogarty, University of the West of England, UK. A. Giordana, University of Torino, Italy. J.G. Grefenstette, Naval Research Laboratory, USA. J.A. Meyer, Ecole Normale Superieure, France. M. Patel, University of Newcastle, UK. M. Schoenauer, Ecole Polytechnique, France. R.E. Smith, University of Alabama, USA. G. Venturini, University of Tours, France. S.W. Wilson, Rowland Institute for Science, USA. Important Dates: April 23: Extended abstracts and papers due May 14: Notification of acceptance June 4: Camera-ready copy for workshop notes due July 2-3: Workshop Prof. Fabrizio Baiardi Dip. di Informatica, Universita di Pisa C. Italia 40, 56123 Pisa, Italy ph: +39/50/887262 email: baiardi@di.unipi.it From kim.plunkett@psy.ox.ac.uk Tue Feb 27 23:44:46 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 27 Feb 96 23:44:42 -0600; AA02053 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 27 Feb 96 23:44:39 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa21151; 26 Feb 96 13:46:28 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa21130; 26 Feb 96 13:17:49 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa10311; 26 Feb 96 13:16:11 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa06117; 26 Feb 96 11:57:11 EST Received: from oxmail.ox.ac.uk by CS.CMU.EDU id aa23093; 26 Feb 96 11:53:38 EST Received: from ep2.psych.ox.ac.uk by oxmail.ox.ac.uk with SMTP (PP) id <10849-0@oxmail.ox.ac.uk>; Mon, 26 Feb 1996 14:48:38 +0000 Received: from EP2_PSYCH/SpoolDir by ep2.psych.ox.ac.uk (Mercury 1.21); 26 Feb 96 14:42:49 +0000 Received: from SpoolDir by EP2_PSYCH (Mercury 1.21); 26 Feb 96 14:42:43 +0000 Received: from mac17.psych.ox.ac.uk by ep2.psych.ox.ac.uk (Mercury 1.21); 26 Feb 96 14:42:37 +0000 X-Mailer: InterCon TCP/Connect II 1.2.1 Message-Id: <9602261447.AA50643@mac17.psych.ox.ac.uk> Date: Mon, 26 Feb 1996 14:47:50 +0000 From: Kim Plunkett To: DEV-EUROPE@DURHAM.AC.UK, psyling@psy.gla.ac.uk, info-childes+@andrew.cmu.edu, connectionists@cs.cmu.edu Subject: LCP.txt X-Text-File-Info: "LCP.txt", part 1 of 1 REMINDER: Manuscript submissions are invited for inclusion in a Special Issue of Language and Cognitive Processes on Connectionist Approaches to Language Development. It is anticipated that most of the papers in the special issue will describe previously unpublished work on some aspect of language development (first or second language learning in either normal or disordered populations) that incorporates a neural network modelling component. However, theoretical papers discussing the general enterprise of connectionist modelling within the domain of language development are also welcome. The deadline for submissions is 1st April 1996. Manuscripts should be sent to the guest editor for this special issue: Kim Plunkett Department of Experimental Psychology Oxford University South Parks Road Oxford, OX1 3UD UK email: plunkett@psy.ox.ac.uk FAX: 1865-310447 All manuscripts will be submitted to the usual Language and Cognitive Processes peer review process. From giles@research.nj.nec.com Tue Feb 27 23:44:48 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 27 Feb 96 23:44:43 -0600; AA02055 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 27 Feb 96 23:44:41 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab21151; 26 Feb 96 13:48:10 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab21130; 26 Feb 96 13:17:58 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa10319; 26 Feb 96 13:17:36 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa07442; 26 Feb 96 13:05:59 EST Received: from zingo.nj.nec.com by EDRC.CMU.EDU id aa27076; 26 Feb 96 13:05:14 EST Received: by zingo (940816.SGI.8.6.9/YDL1.4-910307.16) id NAA17709(zingo); Mon, 26 Feb 1996 13:01:32 -0500 Received: by alta (5.52/cliff's joyful mailer #2) id AA26373(alta); Mon, 26 Feb 96 13:01:25 EST Date: Mon, 26 Feb 96 13:01:25 EST From: Lee Giles Message-Id: <9602261801.AA26373@alta> To: connectionists@cs.cmu.edu Subject: Re: Shift Invariance Cc: giles@research.nj.nec.com We and others [1, 2, 3, 4] showed that invariances, actually affine transformations, could directly be encoded into feedforward higher-order (sometimes called polynomial, sigma-pi, gated, ...) neural nets such that these networks are invariant to shift, scale, and rotation of individual patterns. As mentioned previously, similar invariant encodings can be had for associative memories in autonomous recurrent networks. Interestingly, this idea of encoding geometric invariances into neural networks is an old one [5]. [1] C.L. Giles, T. Maxwell,``Learning, Invariance, and Generalization in High-Order Neural Networks'', Applied Optics, 26(23), p 4972, 1987. Reprinted in ``Artificial Neural Networks: Concepts and Theory,'' eds. P. Mehra and B. W. Wah, IEEE Computer Society Press, Los Alamitos, CA. 1992. [2] C.L. Giles, R.D. Griffin, T. Maxwell,``Encoding Geometric Invariances in Higher-Order Neural Networks'', Neural Information Processing Systems, ed. D.Z. Anderson, Am. Inst. of Physics, N.Y., N.Y., p 301-309, 1988. [3] S.J. Perantonis, P.J.G. Lisboa, ``Translation, Rotation, and Scale Invariant Pattern Recognition by Higher-Order Neural Networks and Moment Classifiers'', IEEE Transactions on Neural Networks'', 3(2), p 241, 1992. [4] L. Spirkovska, M.B. Reid,``Higher-Order Neural Networks Applied to 2D and 3D Object Recognition'', Machine Learning, 15(2), p. 169-200, 1994. [5] W. Pitts, W.S. McCulloch, ``How We Know Universals: The Perception of Auditory and Visual Forms'', Bulletin of Mathematical Biophysics, vol 9, p. 127, 1947. Bibtex entries for the above can be found in: ftp://external.nj.nec.com/pub/giles/papers/high-order.bib -- C. Lee Giles / Computer Sciences / NEC Research Institute / 4 Independence Way / Princeton, NJ 08540, USA / 609-951-2642 / Fax 2482 www.neci.nj.nec.com/homepages/giles.html == From stefan.kremer@crc.doc.ca Tue Feb 27 23:44:52 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 27 Feb 96 23:44:47 -0600; AA02065 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 27 Feb 96 23:44:44 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa21321; 26 Feb 96 15:53:41 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa21319; 26 Feb 96 15:41:04 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa10366; 26 Feb 96 15:40:18 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa10615; 26 Feb 96 15:25:16 EST Received: from uda.dgcd.doc.ca by CS.CMU.EDU id aa25327; 26 Feb 96 15:22:47 EST Received: from digame (digame.dgcd.doc.ca) by uda.dgcd.doc.ca (4.1/SMI-4.1) id AA02547; Mon, 26 Feb 96 14:37:57 EST Received: from quepasa (quepasa.dgcd.doc.ca) by digame (4.1/SMI-4.1) id AA00433; Mon, 26 Feb 96 14:33:00 EST Message-Id: <2.2.32.19960226193500.00694f68@digame.dgcd.doc.ca> X-Sender: kremer@digame.dgcd.doc.ca X-Mailer: Windows Eudora Pro Version 2.2 (32) Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Mon, 26 Feb 1996 14:35:00 -0500 To: Connectionists@cs.cmu.edu From: "Dr. Stefan C. Kremer" Subject: Re: shift invariance and recurrent networks At 08:12 96-02-21 -0800, Jerry Feldman wrote: > The one dimensional case of shift invariance can be handled by treating >each string as a sequence and learning a finite-state acceptor. But the >methods that work for this are not local or biologically plausible and >don't extend to two dimensions. Recently, many recurrent connectionist networks have been applied to the problem of grammatical induction (i.e. inducing a grammar, or equivalently a finite state acceptor for a given set of example strings) [see, for example: Giles (1990)]. These types of networks are capable of learning many types of regular grammars (e.g. (0)*(101)(0)*). Learning of context-free grammars by connectionist networks has also been studied elsewhere [Das (1993)]. The resulting trained networks work only on the basis of local (both spatially and temporally) interactions among neighbouring processing elements. There are a variety of learning algorithms for these networks. Some like backpropagation through time [Rumelhart, 1986] are spatially local, but temporally global, some like real-time recurrent learning [Williams, 1989] are temporally local and spatially global, and some are both temporally and spatially local like Elman's truncated gradient descent [Elman, 1990] and various locally recurrent networks [Tsoi, 1994]. Don't these types of networks can handle shift invariance problems using local processing? (I'd agree that they're not biologically plausible... ;) ). > The unlearnability of shift invarince is not a problem in practice because >people use preprocessing, weight sharing or other techniques to get shift >invariance where it is known to be needed. However, it does pose a problem for >the brain and for theories that are overly dependent on learning. I'm not sure I understand this last part. Are you saying that "preprocessing" and "weight sharing" can handle shift invariance problems because they are a type of non-local processing? -Stefan P.S. Here's the refs: @INPROCEEDINGS{giles90p, AUTHOR = "C.L. Giles and G.Z. Sun and H.H. Chen and Y.C. Lee and D. Chen", TITLE = "Higher Order Recurrent Networks & Grammatical Inference", BOOKTITLE = "Advances in Neural Information Processing Systems~2", YEAR = "1990", EDITOR = "D.S. Touretzky", PUBLISHER = "Morgan Kaufmann Publishers", ADDRESS = "San Mateo, CA", PAGES = "380-387"} @INPROCEEDINGS{das93p, AUTHOR = "S. Das and C.L. Giles and G.Z. Sun ", TITLE = "Using Prior Knowledge in a NNPDA to Learn Context-Free Languages", BOOKTITLE = "Advances in Neural Information Processing Systems 5", PUBLISHER = "Morgan Kaufmann Publishers", EDITOR = "S.J. Hanson and J.D. Cowan and C.L. Giles", PAGES = "65--72", ADDRESS = "San Mateo, CA" YEAR = "1993"} @BOOK{rumelhart86b1, EDITOR = "J. L. McClelland, D.E. Rumelhart and the P.D.P. Group (Eds.)", AUTHOR = "D. Rumberlhart, G. Hinton, R. Williams", TITLE = "Learning Internal Representation by Error Propagation", VOLUME = "1: Foundations", BOOKTITLE = "Parallel Distributed Processing: Explorations in the Microstructure of Cognition", YEAR = "1986", PUBLISHER = "MIT Press", ADDRESS = "Cambridge, MA"} @ARTICLE{williams89j1, AUTHOR = "R.J. Williams and D. Zipser", TITLE = "A Learning Algorithm for Continually Running Fully Recurrent Neural Networks", JOURNAL = "Neural Computation", YEAR = "1989", VOLUME = "1", NUMBER = "2", PAGES = "270-280"} @ARTICLE{elman90j, AUTHOR = "J.L. Elman", TITLE = "Finding Structure in Time", JOURNAL = "Cognitive Science", YEAR = "1990", VOLUME = "14", PAGES = "179-211"} @ARTICLE{tsoi94j, AUTHOR = "A.C. Tsoi and A. Back", TITLE = "Locally Recurrent Globally Feedforward Networks, A Critical Review of Architectures", JOURNAL = "IEEE Transactions on Neural Networks", VOLUME = "5", NUMBER = "2", PAGES = "229-239", YEAR = "1994"} -- Dr. Stefan C. Kremer, Neural Network Research Scientist, Communications Research Centre, 3701 Carling Avenue, P.O. Box 11490, Station H Ottawa, Ontario K2H 8S2 # Tel: (613)990-8175 Fax: (613)990-8369 E-mail: Stefan.Kremer@crc.doc.ca # WWW: http://digame.dgcd.doc.ca/~kremer/ From jamie@atlas.ex.ac.uk Tue Feb 27 23:44:53 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 27 Feb 96 23:44:48 -0600; AA02067 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 27 Feb 96 23:44:45 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa21336; 26 Feb 96 16:03:46 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa21324; 26 Feb 96 15:42:06 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa10372; 26 Feb 96 15:41:26 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa10872; 26 Feb 96 15:36:56 EST Received: from hermes.ex.ac.uk by EDRC.CMU.EDU id aa27895; 26 Feb 96 15:36:03 EST Received: from atlas.dcs.exeter.ac.uk by hermes via SMTP (UAA13016); Mon, 26 Feb 1996 20:35:01 GMT From: jamie@atlas.ex.ac.uk Received: from sirius.dcs.exeter.ac.uk by atlas.dcs.exeter.ac.uk; Mon, 26 Feb 96 20:35:03 GMT Date: Mon, 26 Feb 96 20:35:05 GMT Message-Id: <1708.9602262035@sirius.dcs.exeter.ac.uk> To: Connectionists@edu.cmu.cs.cmu.edu Subject: Re: shift invariance Jerry Feldman writes: > Shift invariance is the ability of a neural system to recognize a pattern >independent of where appears on the retina. It is generally understood that >this property can not be learned by neural network methods, I agree with Jerry that connectionist networks cannot actually learn shift invariance. A connectionist network can exhibit shift invariant behavior by either being exhaustively trained on a set of patterns that happens to have this property, or by having shift invariance somehow wired into the network before the learning occurs. However, neither of these situation constitutes "learning shift invariance". On the other hand, we are still left with the problem of explaining shift invariant behavior. Some of the responses so far imply training on all patterns in all positions (exhaustive training). I don't find this approach interesting, since it doesn't address the basic issue of generalization ability. Thus the question seems to be how shift invariance can be wired in while still using a learning rule that is local, biologically plausible, etc. Geoff Hinton writes: >shift invariance can be learned by backpropagation. It was one of the >problems that I tried when fiddling about with backprop in the mid 80's. Geoff Hinton's experiment clearly does not train the network exhaustively on all patterns in all positions, so (by the above argument) I have to claim that shift invariance is wired in. The network does not use weight sharing, which would be the most direct way of wiring in shift invariance. However, it does appear to use "error sharing". As I understood Geoff's description, the weights between the position-dependent and position-independent hidden layers are not modified by learning. Each position-independent feature detector is connected to all its associated position-dependent feature detectors with links of the same weight (in particular, they are ORed). Using backprop, this has the effect of distributing the same error signal to each of these position-dependent feature detectors. Thus they all tend to converge to the same feature. In this way, fixing the weights between the two hidden layers to equal values makes the feature detectors learned in one position tend to generalize to other positions. I suspect "tend to" may be an important caveat here, but in essence the equivalence of all positions has been wired in. This equivalence is simply shift invariance. On the other hand, the learning rule is still local, so in that sense it does seem to meet Jerry's challenge. Jerry Feldman also writes: > The one dimensional case of shift invariance can be handled by treating >each string as a sequence and learning a finite-state acceptor. But the >methods that work for this are not local or biologically plausible and >don't extend to two dimensions. I have to disagree with this dismissal of recurrent networks for handling shift invariance. I'm not in a position to judge biological plausibility, but I would take issue with the claim that methods of training recurrent networks are nonlocal and can't be generalized to higher dimensions. These learning methods are to some extent temporally nonlocal, but some degree of temporal nonlocality is necessary for any computation that extends over time. The important thing is that they are just as spatially local as the feedforward methods they are based on. Jerry's own definition of locality is spatial locality: >A "local" learning rule is one that updates the input weights of a unit as a >function of the unit's own activity and some performance measure for the >network on the training example. Now finally I get to my primary gripe. Contrary to Jerry's claim, learning methods for recurrent networks can be generalized to more than one dimension. The issues for two dimensions are entirely the same as those for one. All that is necessary to extend recurrence to two dimensions is units that pulse periodically. In engineering terms, a single network is time-multiplexed across one dimension while being sequenced across the other. Conceptually, learning can be done by unfolding the network over one time dimension, then unfolding the result over the other time dimension, then using a feedforward method. The idea of using time to represent two different dimensions has in fact already been proposed. At an abstract level, this dual use of the time dimension is the core idea behind temporal synchrony variable binding (TSVB) (Shastri and Ajjanagadde, 1993). Recurrent networks use the time dimension to represent position in the input sequence (or computation sequence). TSVB also uses the time dimension to represent variables. Because the same network is being used at every point in the input sequence, recurrent networks inherently generalize things learned in one input sequence position to other input sequence positions. In this way shift invariance is "wired in". Exactly analogously, because the same network is being used for every variable, TSVB networks inherently generalize things learned using one variable to other variables. I argue in (Henderson, submitted) that this results in a network that exhibits systematicity. Replacing the labels "sequence position" and "variable" with the labels "horizontal position" and "vertical position" does not change this basic ability to generalize across both dimensions. Work on applying learning to TSVB networks is being done by both Shastri and myself. (Note that this description of TSVB is at a very abstract level. Issues of biological plausibility are addressed in (Shastri and Ajjanagadde, 1993) and the papers cited there.) Shastri, L. and Ajjanagadde, V. (1993). From simple associations to systematic reasoning: A connectionist representation of rules, variables, and dynamic bindings using temporal synchrony. Behavioral and Brain Sciences, 16:417--451. Henderson, J. (submitted). A connectionist architecture with inherent systematicity. Submitted to the Eighteenth Annual Conference of the Cognitive Science Society. - Jamie ------------------------------ Dr James Henderson Department of Computer Science University of Exeter Exeter EX4 4PT, U.K. ------------------------------ From johnd@saturn.sdsu.edu Tue Feb 27 23:44:54 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 27 Feb 96 23:44:50 -0600; AA02076 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 27 Feb 96 23:44:48 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa21379; 26 Feb 96 17:07:52 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa21376; 26 Feb 96 16:55:43 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa10411; 26 Feb 96 16:55:14 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id ab12193; 26 Feb 96 16:36:05 EST Received: from saturn.sdsu.edu by CS.CMU.EDU id aa26240; 26 Feb 96 16:35:31 EST Received: (from johnd@localhost) by saturn.sdsu.edu (4.1/8.6.10) id NAA15297; Mon, 26 Feb 1996 13:35:10 -0800 (PST) Date: Mon, 26 Feb 1996 13:35:10 -0800 (PST) From: John Donald Message-Id: <199602262135.NAA15297@saturn.sdsu.edu> To: Connectionists@cs.cmu.edu, jfeldman@ICSI.Berkeley.EDU Subject: Re: shift invariance Abu-Mostafa's "learning from hints" is a very simple approach to learning neural net representations of functions that satisfy global constraints such as shift invariance. Cf Scientific American April 1995 (!) and the references therein, eg. "Learning from hints", Yaser Abu-Mostafa, J. of Complexity 10 (165-178), 1994. His idea is to add to the training examples additional invented examples that represent the global properties. He claims significant speed-up, eg in training nets to learn an even (global property) shift invariant (global) function. From wimw@mbfys.kun.nl Thu Feb 29 19:02:09 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 29 Feb 96 19:02:06 -0600; AA14498 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 29 Feb 96 19:02:04 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa23066; 27 Feb 96 14:18:31 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa23064; 27 Feb 96 13:52:32 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa11179; 27 Feb 96 13:52:19 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa24166; 27 Feb 96 6:27:43 EST Received: from septimius.mbfys.kun.nl by EDRC.CMU.EDU id aa01341; 27 Feb 96 6:26:24 EST Received: from dontcare by septimius.mbfys.kun.nl via majoria.mbfys.kun.nl [131.174.173.212] with ESMTP id MAA16094 (8.6.10/2.4) for ; Tue, 27 Feb 1996 12:27:23 +0100 From: Wim Wiegerinck Message-Id: <199602271127.MAA16094@septimius.mbfys.kun.nl> Subject: Paper Available To: connectionists@cs.cmu.edu Date: Tue, 27 Feb 1996 12:27:31 +0100 (MET) X-Mailer: ELM [version 2.4 PL23] Content-Type: text Content-Length: 2250 Dear connectionists, The following paper is available at URL ftp://ftp.mbfys.kun.nl/snn/pub/reports/Wiegerinck.4.ps.Z How Dependencies between Successive Examples Affect On-Line Learning. W. Wiegerinck and T. Heskes RWCP Novel Functions SNN Laboratory Dept. Medical Physics and Biophysics, University of Nijmegen Geert Grooteplein 21, 6525 EZ Nijmegen, The Netherlands Neural Computation, in press. ABSTRACT We study the dynamics of on-line learning for a large class of neural networks and learning rules, including backpropagation for multilayer perceptrons. In this paper, we focus on the case where successive examples are dependent, and we analyze how these dependencies affect the learning process. We define the representation error and the prediction error. The representation error measures how well the environment is represented by the network after learning. The prediction error is the average error which a continually learning network makes on the next example. In the neighborhood of a local minimum of the error surface, we calculate these errors. We find that the more predictable the example presentation,the higher the representation error, i.e. the less accurate the asymptotic representation of the whole environment. Furthermore we study the learning process in the presence of a plateau. Plateaus are flat spots on the error surface, which can severely slow down the learning process. In particular, they are notorious in applications with multilayer perceptrons. Our results, which are confirmed by simulations of a multilayer perceptron learning a chaotic time series using backpropagation, explain how dependencies between examples can help the learning process to escape from a plateau. FTP INSTRUCTIONS unix% ftp ftp.mbfys.kun.nl (or 131.174.83.52) Name: anonymous Password: (use your e-mail address) ftp> cd snn/pub/reports/ ftp> binary ftp> get Wiegerinck.4.ps.Z ftp> bye unix% uncompress Wiegerinck.4.ps.Z unix% lpr Wiegerinck.4.ps Wim Wiegerinck Foundation for Neural Networks Department of Medical Physics and Biophysics, University of Nijmegen, The Netherlands mailto:wimw@mbfys.kun.nl URL http://www.mbfys.kun.nl/~wimw/ From edelman@wisdom.weizmann.ac.il Thu Feb 29 19:02:11 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 29 Feb 96 19:02:08 -0600; AA14500 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 29 Feb 96 19:02:06 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa23089; 27 Feb 96 14:28:43 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa23068; 27 Feb 96 13:53:24 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa11185; 27 Feb 96 13:52:59 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa25086; 27 Feb 96 8:21:34 EST Received: from narkis.wisdom.weizmann.ac.il by CS.CMU.EDU id aa01443; 27 Feb 96 8:20:47 EST Received: from lachesis.wisdom.weizmann.ac.il (lachesis.wisdom.weizmann.ac.il [132.76.80.188]) by narkis.wisdom.weizmann.ac.il (8.6.5/mail.byaddr) with ESMTP id PAA23498 for ; Tue, 27 Feb 1996 15:19:17 +0200 From: Edelman Shimon Received: (edelman@localhost) by lachesis.wisdom.weizmann.ac.il (8.6.9/8.6.5) id NAA05288; Tue, 27 Feb 1996 13:19:18 GMT Date: Tue, 27 Feb 1996 13:19:18 GMT Message-Id: <199602271319.NAA05288@lachesis.wisdom.weizmann.ac.il> To: connectionists@cs.cmu.edu In-Reply-To: (message from Shlomo Geva on Mon, 26 Feb 1996 10:05:33 +1000 (EST)) Subject: Re: shift invariance > Date: Mon, 26 Feb 1996 10:05:33 +1000 (EST) > From: Shlomo Geva > > Regarding shift invariance: > > [some stuff omitted] > > Now I'd like to make a conjecture. > It appears to make sense to assume that this difficulty is inherent > to the shift invariance requirement itself. If this is so > then unless you have an additional constraint imposed on objects - > they cannot be allowed to be identical under the invariant feature > extraction transformation you wish to employ - > then you cannot solve the problem. In other words, one needs a guarantee that > all permissible objects are uniquely transformed by the procedure. > It seems to follow that > no general procedure, that does not take into account the nature of the objects > for which the procedure is intended, can exist. > > I am wondering if anyone could clarify whether this is a valid argument. > > Shlomo Geva A number of people (see the refs below) have proved in the past that no universal invariants with respect to viewpoint exist for objects represented as point sets in 3D. The proofs hinged on the possibility of two different objects having the same 2D projection. Offhand, it seems that a similar argument could be used to prove Shlomo's conjecture. -Shimon Dr. Shimon Edelman, Applied Math. & Computer Science Weizmann Institute of Science, Rehovot 76100, Israel The Web: http://eris.wisdom.weizmann.ac.il/~edelman fax: (+972) 8 344122 tel: 8 342856 sec: 8 343545 @inproceedings{MosesUllman92, author="Y. Moses and S. Ullman", title="Limitations of non model-based recognition schemes", booktitle="Proc. 2nd European Conf. on Computer Vision, Lecture Notes in Computer Science", volume="588", pages="820-828", editor="G. Sandini", publisher="Springer Verlag", addredd="Berlin", year="1992" } @article{BurWeiRis93, author="J.B. Burns and R. Weiss and E. Riseman", title="View variation of point-set and line segment features", journal=pami, volume="15", pages = "51-68", year = 1993 } From wiskott@salk.edu Thu Feb 29 19:02:17 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 29 Feb 96 19:02:12 -0600; AA14510 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 29 Feb 96 19:02:08 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa23611; 27 Feb 96 23:06:17 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa23607; 27 Feb 96 22:55:11 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa11407; 27 Feb 96 22:54:15 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa07148; 27 Feb 96 20:24:31 EST Received: by RI.CMU.EDU id aa07781; 27 Feb 96 20:23:56 EST Received: from helmholtz.salk.edu by RI.CMU.EDU id aa07735; 27 Feb 96 20:14:37 EST Received: from bayes.salk.edu by salk.edu (4.1/SMI-4.1) id AA15794; Tue, 27 Feb 96 17:14:18 PST From: Laurenz Wiskott Received: (wiskott@localhost) by bayes.salk.edu (8.6.12/8.6.9) id RAA21840; Tue, 27 Feb 1996 17:14:15 -0800 Date: Tue, 27 Feb 1996 17:14:15 -0800 Message-Id: <199602280114.RAA21840@bayes.salk.edu> To: Connectionists@cs.cmu.edu Subject: shift invariance Dear connectionists, here is an attempt to put the arguments so far into order (and to to add a bit). I have put it into the form of a list of statements, which are, of course, subjective. You can skip the indented comments in the first reading. --------------------------------------------------------------------------- 1) With respect to shift invariance, there are two types of artificial neural nets (ANNs): a) ANNs with a build in concept of spatial order (e.g. neocognitron and other weight sharing ANNs, neural shifter circuits, dynamic link matching), let me call these ANNs structured. b) ANNs without any build in concept of spatial order (e.g. fully connected back-propagation), let me call these ANNs isotropic. (This distinction is important. For instance, Jerry Feldman's statement referred to isotropic ANNs while Rolf W"urtz' disagreement was based on structured ANNs.) 2) Structured ANNs can DO shift invariant pattern discrimination, but they do not LEARN it. (It is important to note that structured ANNs for shift invariant pattern recognition usually do NOT LEARN the shift invariance. It is already build in (I'd be glad to see a counterexample). What they learn is pattern discrimination, under the constraint that, whatever they do, it is shift invariant.) 3) The isotropic ANNs can learn shift invariant pattern recognition, given that during training the patterns are presented at ALL locations. This is not surprising and not what we are asking for. (This is what Christopher Lee pointed out:>>... for a small "test problem"-like space, if given an appropriate number of nodes a network could simply "memorize" all the configurations of an object at all locations. Clearly, this isn't what one would normally considering "learning" shift invariance.<<) 4) What we are asking for is generalization. I see two types of generalization: a) generalization of pattern recognition from one part of the input plain to another. b) generalization of shift invariance from one pattern to another. (4a example: training on patterns A {101} and B {011} in the left half-plane, i.e. {101000, 010100, 011000, 001100}, and testing on patterns A and B in the right half-plane, e.g. {000101, 000011}. 4b example: training on some patterns {111, 011, 010} in all possible locations, i.e. {111000, 011100, 001110, 000111, 011000, ..., 000010}, and on pattern A {101} in the left half-plane, i.e {101000, 010100}, and testing on pattern A in the right half-plane, e.g. {000101}. This is again an important distinction. For instance, Jerry Feldman's statement referred to generalization 4a and Geoffrey Hinton's disagreement referred to generalization 4b.) 5) Generalization 4a seems to be impossible for an isotropic ANN. (This was illustrated by Jerry Feldman and, more elaborately, by Georg Dorffner.) 6) Generalization 4b is possible. (This has been demonstrated by the model of Geoffrey Hinton.) 7) Models which generalize according to 4b usually loose discriminative power, because patterns with the same set of features but in different spatial order get confused. (This has been pointed out by Shlomo Geva. This also holds for some structured ANNs, such as the neocognitron and other weight sharing ANNs, but does not hold for the neural shifter circuits and dynamic link matching. The loss of discriminative power can be avoided by using sufficiently complex features, which has its own drawbacks.) --------------------------------------------------------------------------- Best regards, Laurenz Wiskott. =========================================================================== .---. .---. Dr. L a u r e n z W i s k o t t | |. S A L K .| | Computational Neurobiology Laboratory | ||. C N L .|| | The Salk Institute for Biological Studies | ||| L W ||| | mail: PO Box 85800, San Diego, CA 92186-5800 | |||=======||| | street, parcels: 10010 North Torrey Pines Road `---''' " ```---' La Jolla, CA 92037 phone: +1 (619) 453-4100 ext 1463; fax: +1 (619) 587-0417; email: wiskott@salk.edu; WWW: http://www.cnl.salk.edu/~wiskott/ =========================================================================== From sontag@control.rutgers.edu Thu Feb 29 19:02:19 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 29 Feb 96 19:02:13 -0600; AA14512 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 29 Feb 96 19:02:10 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa25410; 28 Feb 96 19:14:14 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa25407; 28 Feb 96 18:50:57 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa12341; 28 Feb 96 18:50:30 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa19456; 28 Feb 96 11:02:21 EST Received: from control.rutgers.edu by RI.CMU.EDU id aa10763; 28 Feb 96 11:01:25 EST Received: (from sontag@localhost) by control.rutgers.edu (8.6.12+bestmx+oldruq+newsunq/8.6.12) id LAA19838; Wed, 28 Feb 1996 11:01:03 -0500 Date: Wed, 28 Feb 1996 11:01:03 -0500 From: Eduardo Sontag Message-Id: <199602281601.LAA19838@control.rutgers.edu> To: connectionists@cs.cmu.edu Subject: TR available - classification of points in general position Reply-To: sontag@hilbert.rutgers.edu SHATTERING ALL SETS OF k POINTS IN GENERAL POSITION REQUIRES (k-1)/2 PARAMETERS Rutgers Center for Systems and Control (SYCON) Report 96-01 Eduardo D. Sontag Department of Mathematics, Rutgers University The generalization capabilities of neural networks are often quantified in terms of the maximal number of possible binary classifications that could be obtained, by means of weight assignments, on any given set of input patterns. The Vapnik-Chervonenkis (VC) dimension is the size of the largest set of inputs that can be shattered (i.e, arbitrary binary labeling is possible). Recent results show that the VC dimension grows at least as fast as the square n**2 of the number of adjustable weights n in the net, and this number might grow as fast as n**4. These results are quite pessimistic, since they imply that the number of samples required for reliable generalization, in the sense of PAC learning, is very high. On the other hand, it is conceivable that those sets of input patterns which can be shattered are all in some sense ``special'' and that if we ask instead, as done in the classical literature in pattern recognition, for the shattering of all sets in ``general position,'' then an upper bound of O(n) might hold. This paper shows a linear upper bound for arbitrary sigmoidal (as well as threshold) neural nets, proving that in that sense the classical results can be recovered in a strong sense (up to a factor of two). Specifically, for classes of concepts defined by certain classes of analytic functions depending on n parameters, it is shown that there are nonempty open sets of samples of length 2n+2 which cannot be shattered. ============================================================================ The paper is available starting from Eduardo Sontag's WWW HomePage at URL: http://www.math.rutgers.edu/~sontag/ (follow links to FTP archive, file generic-vc.ps.gz) or directly via anonymous FTP: ftp math.rutgers.edu login: anonymous cd pub/sontag bin get generic-vc.ps.gz Once file is retrieved, use gunzip to uncompress and then print as postscript. ============================================================================ Comments welcome. From jfeldman@ICSI.Berkeley.EDU Thu Feb 29 19:02:20 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 29 Feb 96 19:02:17 -0600; AA14524 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 29 Feb 96 19:02:14 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab25446; 28 Feb 96 19:23:40 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab25412; 28 Feb 96 18:52:39 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa12352; 28 Feb 96 18:51:41 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa21774; 28 Feb 96 13:01:51 EST Received: from icsia.ICSI.Berkeley.EDU by RI.CMU.EDU id aa11369; 28 Feb 96 13:01:05 EST Received: from waffle.ICSI.Berkeley.EDU (jfeldman@waffle.ICSI.Berkeley.EDU [128.32.201.183]) by icsia.ICSI.Berkeley.EDU (8.6.13/HUB+V8$Revision: 1.23 $) with ESMTP id KAA08856 for ; Wed, 28 Feb 1996 10:00:46 -0800 Received: (jfeldman@localhost) by waffle.ICSI.Berkeley.EDU (8.6.13/1.8) id KAA15423 for Connectionists@cs.cmu.edu; Wed, 28 Feb 1996 10:00:41 -0800 From: Jerry Feldman Message-Id: <9602281000.ZM15421@ICSI.Berkeley.edu> Date: Wed, 28 Feb 1996 10:00:39 -0800 X-Mailer: Z-Mail (3.2.0 06sep94) To: Connectionists@cs.cmu.edu Subject: Re: shift invariance Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii There seem to be three separate threads arising from my cryptic post and it might be useful to separate them. 1) The capabilities of spatial feedforward nets and backprop(ffbp) Everyone seems to now agree that conventional feedforward nets and backprop (ffbp) will not learn the simple 0*1010* languages of my posting. Of course any formal technique has limitations; the interesting point is that shift invariance is a basic property of apparent biological significance. Geoff Hinton's series of messages asserts that the world and experience are (could be?) structured such that ffbp will learn shift invariance in practice because patterns overlap and are dense enough in space. My inference is that Geoff would like to extend this claim (the world makes ffbp work) to everything of biological importance. Results along these lines would be remarkable indeed. 2) Understanding how the visual system achieves shift invariance. This thread has been non-argumentative. The problem of invariances and constancies in the visual system remains central in visual science. I can't think of any useful message-sized summary, but this is an area where connectionist models should play a crucial role in expressing and testing theories. But, as several people have pointed out, we can't expect much from tabula rasa learning. 3) Shift invariance in time and recurrent networks. I threw in some (even more cryptic) comments on this anticipating that some readers would morph the original task into this form. The 0*1010* problem is an easy one for FSA induction and many simple techniques might work for this. But consider a task that is only slightly more general, and much more natural. Suppose the task is to learn any FSL from the class b*pb* where b and p are fixed for each case and might overlap. Any learning technique that just tried to predict (the probability of) successors will fail because there are three distinct regimes and the learning algorithm needs to learn this. I don't have a way to characterize all recurrent net learning algorithms to show that they can't do this and it will be interesting to see if one can. There are a variety on non-connectionist FSA induction methods that can effectively learn such languages, but they all depend on some overall measure of simplicity of the machine and its fit to the data - and are thus non-local. The remark about not extending to two dimensions referred to the fact that we have no formal grammar for two dimensional patterns (although several proposals for one) and, a fortiori, no algorithm for learning same. One can, as Jamie Henderson suggests, try to linearize two-dimensional problems. But no one has done this successfully for shift (rotation, scale, etc.) invariance and it doesn't seem to me a promising approach to these issues. Jerry F. From andre@icmsc.sc.usp.br Thu Feb 29 19:02:21 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 29 Feb 96 19:02:15 -0600; AB14518 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 29 Feb 96 19:02:12 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa25446; 28 Feb 96 19:22:17 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa25412; 28 Feb 96 18:52:33 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa12348; 28 Feb 96 18:51:17 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa21624; 28 Feb 96 12:52:27 EST Received: from xavante.icmsc.sc.usp.br by RI.CMU.EDU id aa11312; 28 Feb 96 12:50:55 EST Received: from xapacura (xapacura.icmsc.sc.usp.br) by xavante (4.1/SMI-4.1) id AA01880; Wed, 28 Feb 96 14:52:21 EST Date: Wed, 28 Feb 96 14:52:21 EST From: " Andre Carlos P. de Leon F. de Carvalho " Message-Id: <9602281752.AA01880@xavante> To: Connectionists@cs.cmu.edu Subject: II Workshop on Cybernetic Vision ====================================================== First Call for Contributions II Workshop on Cybernetic Vision Instituto de Fisica e Quimica de Sao Carlos Universidade de Sao Paulo Sao Carlos, SP, Brazil 9-11 December 1996 ====================================================== As stated in his classical book *Cybernetics*, Norbert Wiener believed that the most interesting and exciting possibilities for original research were to be found in the interface between the major scientific areas. In a tribute to Wiener's visionary approach, the term CYBERNETIC VISION has been proposed to express those research activities lying in the natural/artificial vision interface. It is believed that not only more powerful and versatile artificial visual systems can be obtained through the incorporation of biological insights, but also that our understanding of the natural visual systems can benefit from advances in artificial vision research, thus sustaining a positive feedback. The I WORKSHOP ON CYBERNETIC VISION took place at the Brazilian town of Sao Carlos, SP, in 1994 and attracted the attention of many researchers from the most diverse areas. The second issue of this meeting is aimed at providing another opportunity for scientific interchange as well as the beginning of new collaborations between the related communities. Quality papers related to any of the areas below are welcomed. Prospective authors should send (through FAX, conventional mail, or e-mail) an abstract of approximately 500 words no later than 15th April 1996 to: Prof Dr Luciano da Fontoura Costa Cybernetic Vision Research Group IFSC-USP Av Dr Carlos Botelho, 1465 Caixa Postal 369 Sao Carlos, SP 13560-970 Brazil FAX: +55 162 71 3616 Electronic-mail submission of the abstracts, to be sent to the address below, is strongly encouraged. Luciano@olive.ifqsc.sc.usp.br Upon acceptance of the proposed abstracts, the authors will be asked to prepare the full manuscript (full paper or communication), for further assessment. Accepted contributions will be included in the Workshop Proceedings. The abstracts of the accepted papers will be eventually incorporated into a WWW page. Although contributions in the Portuguese language are welcomed, preference will be given to manuscript in the English language. Subjected to the author's consent, accepted papers in the English language may also be considered for publication in international journals in some of the areas covered. The organizing committee will do its best towards providing a limited number of grants. Areas covered include, but are by no means limited to: - Active Vision - Anatomy and Histology - Eletrophysiology - Ethology - Fuzzy Models - Image Analysis and Computer Vision - Medical Imaging - Modeling and Simulation of Biological Visual Systems - Neural Networks (natural and artificial) - Neurogenesys - Neuromorphometry - Neuroscience - Optical Computing - Psychophysics - Robotics - Scientific Visualization - Vertebrate and Invertebrate Vision - Vision and Consciousness ====================================================== Important Dates: * A 500-word abstract by April 15 * Feedback to author on abstract by May 17 * Three copies of the complete version of the paper by July 26 * Notification of accepted papers by September 6 * WORKSHOP December 9-11 ====================================================== Organizing Committee: Luciano da F. Costa, CVRG, IFSC-USP (Coordinator) Sergio Fukusima, USP-Rib. Preto Roland Koberle, CVRG, IFSC-USP Rafael Linden, UFRJ Valentin O. Roda, CVRG, IFSC-USP Jan F. W. Slaets, IFSC-USP ====================================================== Program Committee (preliminary): Arnaldo Albuquerque, UFMG Junior Barrera, IME-USP Paulo E. Cruvinel, EMBRAPA-CNPDIA Antonio Francisco, INPE Annie F. Frere, EESC-USP Sergio Fukusima, USP-Rib. Preto Andre P. de Leon, ICMSC-USP Rafael Linden, UFRJ Roberto de A. Lotufo, DCA-UNICAMP Ricardo Machado, IBM Joaquim P. Brasil Neto, UnB Nelson D. D. Mascarenhas, UFSCar Valdir F. Pessoa, UnB Anna H. R. C. Rillo, PCS-EPUSP ======================================================= From clee@it.wustl.edu Thu Feb 29 19:02:24 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 29 Feb 96 19:02:22 -0600; AA14536 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 29 Feb 96 19:02:19 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ad25446; 28 Feb 96 19:26:57 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab25416; 28 Feb 96 18:53:44 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa12361; 28 Feb 96 18:52:06 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa28734; 28 Feb 96 18:31:55 EST Received: from wugate.wustl.edu by EDRC.CMU.EDU id aa10658; 28 Feb 96 18:31:38 EST Received: from it by wugate.wustl.edu (8.6.12/8.6.11) with ESMTP id RAA09790 for <@wugate.wustl.edu:Connectionists@cs.cmu.edu>; Wed, 28 Feb 1996 17:31:04 -0600 Received: by it (950911.SGI.8.6.12.PATCH825/911001.SGI) id PAA01001; Wed, 28 Feb 1996 15:33:05 -0600 Date: Wed, 28 Feb 1996 15:33:05 -0600 Message-Id: <199602282133.PAA01001@it> From: Christopher Lee To: Connectionists@cs.cmu.edu Subject: paper available-Nonlinear Hebbian learning Announcing the availability of a paper that may be of relevance to those who have been following the recent discussion of shift-invariance. Key words: Hebbian learning, disparity, nonlinear systems, random-dot stereograms. --------------------------------------------------------------------- A nonlinear Hebbian network that learns to detect disparity in random-dot stereograms. C.W. Lee and B.A. Olshausen An intrinsic limitation of linear, Hebbian networks is that they are capable of learning only from the linear pairwise correlations within an input stream. In order to explore what higher forms of structure could be learned with a nonlinear Hebbian network, we have constructed a model network containing a simple form of nonlinearity and we have applied the network to the problem of learning to detect the disparities present in random-dot stereograms. The network consists of three layers, with nonlinear, sigmoidal activation functions in the second layer units. The nonlinearities allow the second layer to transform the pixel-based representation in the input into a new representation based on coupled pairs of left-right inputs. The third layer of the network then clusters patterns occurring on the second layer outputs according to their disparity via a standard competitive learning rule. Analysis of the network dynamics shows that the second-layer units' nonlinearities interact with the Hebbian learning rule to expand the region over which pairs of left-right inputs are stable. The learning rule is neurobiologically inspired and plausible, and the model may shed light on how the nervous system learns to use coincidence detection in general. (To appear in Neural Computation 8:3) This paper is available via World Wide Web at: http://v1.wustl.edu/chris/chris.html Hard copies are available upon request from clee@v1.wustl.edu, or write to: Chris Lee Campus Box 8108 Washington University 660 S. Euclid Ave St. Louis, MO 63110. From pjs@aig.jpl.nasa.gov Thu Feb 29 19:02:25 1996 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 29 Feb 96 19:02:19 -0600; AA14534 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 29 Feb 96 19:02:17 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ac25446; 28 Feb 96 19:25:20 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa25416; 28 Feb 96 18:53:42 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa12357; 28 Feb 96 18:51:57 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id ab23135; 28 Feb 96 14:20:24 EST Received: from aig.jpl.nasa.gov by CS.CMU.EDU id aa01901; 28 Feb 96 14:18:51 EST Received: from amorgos.jpl.nasa.gov by aig.jpl.nasa.gov (4.1/JPL-AIG-1.0) id AA28396; Wed, 28 Feb 96 11:18:44 PST Received: (from pjs@localhost) by amorgos.jpl.nasa.gov (8.7.1/8.7.1) id LAA03228 for connectionists@cs.cmu.edu; Wed, 28 Feb 1996 11:18:41 -0800 (PST) Date: Wed, 28 Feb 1996 11:18:41 -0800 (PST) From: "Padhraic J. Smyth" Message-Id: <199602281918.LAA03228@amorgos.jpl.nasa.gov> To: connectionists@cs.cmu.edu Subject: TR on HMMs and graphical models The following technical report is available online at: ftp://aig.jpl.nasa.gov/pub/smyth/papers/TR-96-03.ps.Z PROBABILISTIC INDEPENDENCE NETWORKS FOR HIDDEN MARKOV PROBABILITY MODELS Padhraic Smyth [a], David Heckerman [b], and Michael Jordan [c] [a] Jet Propulsion Laboratory and Department of Information and Computer Science, UCI [b] Microsoft Research [c] Department of Brain and Cognitive Sciences, MIT Abstract Graphical techniques for modeling the dependencies of random variables have been explored in a variety of different areas including statistics, statistical physics, artificial intelligence, speech recognition, image processing, and genetics. Formalisms for manipulating these models have been developed relatively independently in these research communities. In this paper we explore hidden Markov models (HMMs) and related structures within the general framework of probabilistic independence networks (PINs). The paper contains a self-contained review of the basic principles of PINs. It is shown that the well-known forward-backward (F-B) and Viterbi algorithms for HMMs are special cases of more general inference algorithms for arbitrary PINs. Furthermore, the existence of inference and estimation algorithms for more general graphical models provides a set of analysis tools for HMM practitioners who wish to explore a richer class of HMM structures. Examples of relatively complex models to handle sensor fusion and coarticulation in speech recognition are introduced and treated within the graphical model framework to illustrate the advantages of the general approach. This TR is available as Microsoft Research Technical Report TR-96-03, Microsoft Research, Redmond, WA. and as AI Lab Memo AIM-1565, Massachusetts Institute of Technology, Cambridge, MA.