From koza@CS.Stanford.EDU Sun Nov 19 09:21:46 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sun, 19 Nov 95 09:21:43 -0600; AA04752 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sun, 19 Nov 95 09:21:40 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa10454; 19 Nov 95 1:38:17 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa10452; 19 Nov 95 1:28:12 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa09324; 19 Nov 95 1:27:22 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa19673; 16 Nov 95 9:46:19 EST Received: from Sunburn.Stanford.EDU by RI.CMU.EDU id aa24953; 16 Nov 95 9:44:58 EST Received: (from koza@localhost) by Sunburn.Stanford.EDU (8.7.1/8.7.1) id GAA25909 for connectionists@cs.cmu.edu; Thu, 16 Nov 1995 06:44:52 -0800 (PST) Posted-Date: Thu, 16 Nov 1995 06:44:52 -0800 (PST) Date: Thu, 16 Nov 95 6:44:51 PST From: John Koza To: connectionists@cs.cmu.edu Subject: CFP: Genetic Programming 1996 Conference (GP-96) Message-Id: ------------------------------------------------ Paper Submission Deadline: January 10, 1996 (Wednesday) ------------------------------------------------ CALL FOR PAPERS AND PARTICIPATION Genetic Programming 1996 Conference (GP-96) July 28 - 31 (Sunday - Wednesday), 1996 Fairchild Auditorium - Stanford University - Stanford, California Proceedings will be published by The MIT Press In cooperation with the Association for Computing Machinery (ACM), SIGART, the IEEE Neural Network Council, and the American Association for Artificial Intelligence. Genetic programming is a domain-independent method for evolving computer programs that solve, or approximately solve, problems. Starting with a primordial ooze of thousands of randomly created programs composed of functions and terminals appropriate to a problem, a genetic population is progressively evolved over many generations by applying the Darwinian principle of survival of the fittest, a sexual recombination operation, and occasional mutation. This first genetic programming conference will feature contributed papers, tutorials, invited speakers, and informal meetings. Topics include, but are not limited to, applications of genetic programming, theoretical foundations of genetic programming, implementation issues, parallelization techniques, technique extensions, implementations of memory and state, representation issues, new operators, architectural evolution, evolution of mental models, cellular encoding, evolution of machine language programs, evolvable hardware, combinations with other machine learning techniques, and relations to biology and cognitive systems. ------------------------------------------------- HONORARY CHAIR: John Holland, University of Michigan INVITED SPEAKERS: John Holland, University of Michigan and David E. Goldberg, University of Illinois GENERAL CHAIR: John Koza, Stanford University PUBLICITY CHAIR: Patrick Tufts, Brandeis University ------------------------------------------------- SPECIAL PROGRAM CHAIRS: The main focus of the conference (and about two-thirds of the papers) will be on genetic programming. In addition, papers describing recent developments in the following closely related areas of evolutionary computation (particularly those addressing issues common to various areas of evolutionary computation) will be reviewed by special program committees appointed and supervised by the following special program chairs. - GENETIC ALGORITHMS: David E. Goldberg, University of Illinois, Urbana, Illinois - CLASSIFIER SYSTEMS: Rick Riolo, University of Michigan - EVOLUTIONARY PROGRAMMING AND EVOLUTION STRATEGIES: David Fogel, University of California, San Diego, California ------------------------------------------------- TUTORIALS -Sunday July 28 9:15 AM - 11:30 AM - Genetic Algorithms - David E. Goldberg, University of Illinois - Machine Language Genetic Programming - Peter Nordin, University of Dortmund, Germany - Genetic Programming using Mathematica P Robert Nachbar P Merck Research Laboratories - Introduction to Genetic Programming - John Koza, Stanford University ------------------------------------------------- Sunday July 28 1:00 PM - 3: 15 PM - Classifier Systems- Robert Elliott Smith, University of Alabama - Evolutionary Computation for Constraint Optimization - Zbigniew Michalewicz, University of North Carolina - Advanced Genetic Programming - John Koza, Stanford University ------------------------------------------------- Sunday July 28 3:45 PM - 6 PM - Evolutionary Programming and Evolution Strategies - David Fogel, University of California, San Diego - Cellular Encoding P Frederic Gruau, Stanford University (via videotape) and David Andre, Stanford University (in person) - Genetic Programming with Linear Genomes (one hour) - Wolfgang Banzhaf, University of Dortmund, Germany -JECHO - Terry Jones, Santa Fe Institute ------------------------------------------------- Tuesday July 30 - 3 PM - 5:15PM - Neural Networks - David E. Rumelhart, Stanford University - Machine Learning - Pat Langley, Stanford University -JMolecular Biology for Computer Scientists - Russ B. Altman, Stanford University ------------------------------------------------- INFORMATION FOR SUBMITTING PAPERS The deadline for receipt at the physical mail address below of seven (7) copies of each submitted paper is Wednesday, January 10, 1996. Papers are to be in single-spaced, 12- point type on 8 1/2" x 11" or A4 paper (no e-mail or fax) with full 1" margins at top, bottom, left, and right. Papers are to contain ALL of the following 9 items, within a maximum of 10 pages, IN THIS ORDER: (1) title of paper, (2) author name(s), (3) author physical address(es), (4) author e-mail address(es), (5) author phone number(s), (6) a 100-200 word abstract of the paper, (7) the paper's category (chosen from one of the following five alternatives: genetic programming, genetic algorithms, classifier systems, evolutionary programming, or evolution strategies), (8) the text of the paper (including all figures and tables), and (9) bibliography. All other elements of the paper (e.g., acknowledgments, appendices, if any) must come within the maximum of 10 pages. Review criteria will include significance of the work, novelty, sufficiency of information to permit replication (if applicable), clarity, and writing quality. The first-named (or other designated) author will be notified of acceptance or rejection by approximately Monday February 26, 1996. The style of the camera-ready paper will be identical to that of the 1994 Simulation of Adaptive Behavior conference published by the MIT Press. Depending on the number, subject, and content of the submitted papers, the program committee may decide to allocate different number of pages to various accepted papers. The deadline for the camera-ready, revised version of accepted papers will be announced, but will be approximately Wednesday March 20, 1996. Proceedings will be published by The MIT Press and will be available at the conference (and, if requested, by priority mail to registered conference attendees with U.S. addresses just prior to the conference). One author will be expected to present each accepted paper at the conference. ------------------------------------------------- FOR MORE INFORMATION ABOUT THE GP-96 CONFERENCE: On the World Wide Web: http://www.cs.brandeis.edu/~zippy/gp-96.html or via e-mail at gp@aaai.org. Conference operated by Genetic Programming Conferences, Inc. (a California not-for-profit corporation). ------------------------------------------------- FOR MORE INFORMATION ABOUT GENETIC PROGRAMMING IN GENERAL: http://www-cs- faculty.stanford.edu/~koza/. ------------------------------------------------- FOR MORE INFORMATION ABOUT DISCOUNTED TRAVEL : For further information regarding special GP-96 airline and car rental rates, please contact Conventions in America at e-mail flycia@balboa.com; or phone 1-800-929-4242; or phone 619-678-3600; or FAX 619-678-3699. ------------------------------------------------- FOR MORE INFORMATION ABOUT THE SAN FRANCISCO BAY AREA AND SILICON VALLEY AREA SIGHTS: Try the Stanford University home page at http://www.stanford.edu/, the Hyperion Guide at http://www.hyperion.com/ba/sfbay.html; the Palo Alto weekly at http://www.service.com/PAW/home.html; the California Virtual Tourist at http://www.research.digital.com/SRC/virtual- tourist/California.html; and the Yahoo Guide of San Francisco at http://www.yahoo.com/Regional_Information/States/Califo rnia/San_Francisco. ------------------------------------------------- FOR MORE INFORMATION ABOUT CONTEMPORANEOUS WEST COAST CONFERENCES: Information about the AAAI-96 conference on August 4 P 8 (Sunday P Thursday), 1996, in Portland, Oregon can be found at http://www.aaai.org/. For information on the International Conference on Knowledge Discovery and Data Mining (KDD-96) in Portland, Oregon, on August 3- 5, 1996: http://www-aig.jpl.nasa.gov/kdd96. Information about the Foundations of Genetic Algorithms (FOGA) workshop on August 3 P 5 (Saturday P Monday), 1996, in San Diego, California can be found at http://www.aic.nrl.navy.mil/galist/foga/ or by contacting belew@cs.wisc.edu. ------------------------------------------------- FOR MORE INFORMATION ABOUT MEMBERSHIP IN THE ACM, AAAI, or IEEE: For information about ACM membership, try http://www.acm.org/; for information about SIGART, try http://sigart.acm.org/; for AAAI membership, go to http://www.aaai.org/; and for membership in the IEEE Computer Society, go to http://www.computer.org. ------------------------------------------------- PHYSICAL MAIL ADDRESS FOR GP-96: GP-96 Conference, c/o American Association for Artificial Intelligence, 445 Burgess Drive, Menlo Park, CA 94025. PHONE: 415-328-3123. FAX: 415-321-4457. WWW: http://www.aaai.org/. E-MAIL: gp@aaai.org. ------------------------------------------------ REGISTRATION FORM FOR GENETIC PROGRAMMING 1996 CONFERENCE TO BE HELD ON JULY 28 P 31, 1996 AT STANFORD UNIVERSITY First Name _________________________ Last Name_______________ Affiliation________________________________ Address__________________________________ ________________________________________ City__________________________ State/Province _________________ Zip/Postal Code____________________ Country__________________ Daytime telephone__________________________ E-Mail address_____________________________ Conference registration fee includes copy of proceedings, attendance at 4 tutorials of your choice, syllabus books for 4 tutorials, conference reception, and admission to conference. Students must send legible proof of full-time student status. Conference proceedings will be mailed to registered attendees with U.S. mailing addresses via 2-day U.S. priority mail 1 P 2 weeks prior to the conference at no extra charge (at addressee's risk). If you are uncertain as to whether you will be at that address at that time or DO NOT WANT YOUR PROCEEDINGS MAILED to you at the above address for any other reason, your copy of the proceedings will be held for you at the conference registration desk if you CHECK HERE ____. Postmarked by May 15, 1996: Student P ACM, IEEE, or AAAI Member $195 Regular P ACM, IEEE, or AAAI Member $395 Student P Non-member $215 Regular P Non-member $415 Postmarked by June 26, 1996: Student P ACM, IEEE, or AAAI Member $245 Regular P ACM, IEEE, or AAAI Member $445 Student P Non-member $265 Regular P Non-member $465 Postmarked later or on-site: Student P ACM, IEEE, or AAAI Member $295 Regular P ACM, IEEE, or AAAI Member $495 Student P Non-member $315 Regular P Non-member $515 Member number: ACM # ___________ IEEE # _________ AAAI # _________ Total fee (enter appropriate amount) $ _________ __ Check or money order made payable to "AAAI" (in U.S. funds) __ Mastercard __ Visa __ American Express Credit card number __________________________________________ Expiration Date ___________ Signature _________________________ TUTORIALS: Check off a box for one tutorial from each of the 4 columns: Sunday July 28, 1996 P 9:15 AM - 11:30 AM __ Genetic Algorithms __ Machine Language GP __ GP using Mathematica __ Introductory GP Sunday July 28, 1996 P 1:00 PM - 3: 15 PM __ Classifier Systems __ EC for Constraint Optimization __ Advanced GP Sunday July 28, 1996 P 3:45 PM - 6 PM __ Evolutionary Programming and Evolution Strategies __ Cellular Encoding __ GP with Linear Genomes __ ECHO Tuesday July 30, 1996 P3:00 PM - 5:15PM __ Neural Networks __ Machine Learning __ Molecular Biology for Computer Scientists __ Check here for information about housing and meal package at Stanford University. __ Check here for information on student travel grants. No refunds will be made; however, we will transfer your registration to a person you designate upon notification. SEND TO: GP-96 Conference, c/o American Association for Artificial Intelligence, 445 Burgess Drive, Menlo Park, CA 94025. PHONE: 415- 328-3123. FAX: 415-321-4457. E-MAIL: gp@aaai.org. WWW: http://www.aaai.org/. ------------------------------------------------- PROGRAM COMMITTEE Russell J. Abbott California State University, Los Angeles and The Aerospace Corporation Hojjat Adeli Ohio State University Dennis Allison Stanford University Lee Altenberg Hawaii Institute of Geophysics and Planetology David Andre Stanford University Peter J. Angeline Loral Federal Systems Wolfgang Banzhaf University of Dortmund, Germany Rik Belew University of California at San Diego Samy Bengio Centre National d'Etudes des Telecommunications, France Forrest H. Bennett III Genetic Algorithms Technology Corporation Scott Brave Stanford University Bill P. Buckles Tulane University Walter Cedeno Primavera Systems Inc. Nichael Lynn Cramer BBN System and Technologies Jason Daida University of Michigan Patrik D'haeseleer University of New Mexico Marco Dorigo Universite' Libre de Bruxelles Bertrand Daniel Dunay System Dynamics International Andrew N. Edmonds Science in Finance Ltd., UK H.H. Ehrenburg CWI, The Netherlands Frank D. Francone FRISEC P Francone & Raymond Institute for the Study of Evolutionary Computation, Germany Adam P. Fraser University of Salford Alex Fukunaga University of California, Los Angeles Frederic Gruau Stanford University Richard J. Hampo Ford Motor Company Simon Handley Stanford University Thomas D. Haynes The University of Tulsa Hitoshi Hemmi ATR, Kyoto, Japan Vasant Honavar Iowa State University Thomas Huang University of Illinois Hitoshi Iba Electrotechnical Laboratory, Japan Christian Andrew Johnson Department of Economics, University of Santiago Martin A. Keane Econometrics Inc. Mike Keith Allen Bradley Controls Maarten Keijzer Kenneth E. Kinnear, Jr. Adaptive Computing Technology W. B. Langdon University College, London David Levine Argonne National Laboratory Kenneth Marko Ford Motor Company Martin C. Martin Carnegie Mellon University Sidney R Maxwell III Nicholas Freitag McPhee University of Minnesota, Morris David Montana BBN System and Technologies Heinz Muehlenbein GMD Research Center, Germany Robert B. Nachbar Merck Research Laboratories Peter Nordin University of Dortmund, Germany Howard Oakley Institute of Naval Medicine, UK Franz Oppacher Carleton University, Ottawa Una-May O`Reilly Carleton University, Ottawa Michael Papka Argonne National Laboratory Timothy Perkis Frederick E. Petry Tulane University Bill Punch Michigan State University Justinian P. Rosca University of Rochester Conor Ryan University College Cork, Ireland Malcolm Shute University of Brighton, UK Eric V. Siegel Columbia University Karl Sims Andrew Singleton Creation Mechanics Lee Spector Hampshire College Walter Alden Tackett Neuromedia Astro Teller Carnegie Mellon University Marco Tomassini Ecole Polytechnique Federale de Lausanne Patrick Tufts Brandeis University V. Rao Vemuri University of Califonia at Davis Peter A. Whigham Australia Darrell Whitley Colorado State University Man Leung Wong Chinese University of Hong Kong Alden H. Wright University of Montana Byoung-Tak Zhang GMD, Germany From maass@igi.tu-graz.ac.at Sun Nov 19 19:27:22 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sun, 19 Nov 95 19:27:18 -0600; AA09528 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sun, 19 Nov 95 19:27:15 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa11227; 19 Nov 95 13:19:44 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa11225; 19 Nov 95 13:07:47 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa10122; 19 Nov 95 13:07:30 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa22000; 19 Nov 95 11:29:02 EST Received: from figids02.tu-graz.ac.at by CS.CMU.EDU id aa27884; 19 Nov 95 11:28:20 EST Received: from localhost by figids02.tu-graz.ac.at with SMTP id AA28958 (5.67b/IDA-1.5t for Connectionists@cs.cmu.edu); Sun, 19 Nov 1995 17:31:49 +0100 Message-Id: <199511191631.AA28958@figids02.tu-graz.ac.at> To: Connectionists@cs.cmu.edu Cc: maass@igi.tu-graz.ac.at Subject: computing with noisy spiking neurons: paper in neuroprose Date: Sun, 19 Nov 95 17:31:42 +0100 From: Wolfgang Maass X-Mts: smtp The file maass.noisy-spiking.ps.Z is now available for copying from the Neuroprose repository. This is a 9-page long paper. Hardcopies are not available. FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/maass.noisy-spiking.ps.Z On the Computational Power of Noisy Spiking Neurons Wolfgang Maass Institute for Theoretical Computer Science Technische Universitaet Graz Klosterwiesgasse 32/2 A-8010 Graz, Austria e-mail: maass@igi.tu-graz.ac.at Abstract This article provides positive results about the computational power of neural networks that are based on a neuron model ("noisy spiking neuron") which is acceptable to most neurobiologists as being reasonably realistic for a biological neuron. In fact: this model tends to underestimate the computational capabilities of a biological neuron, since it simplifies dendritic integration. Biological neurons communicate via spike-trains, i.e. via sequences of stereotyped pulses (spikes) that encode information in their time- differences ("temporal coding"). In addition it is wellknown that biological neurons are quite "noisy". There is some "jitter" in their firing times, and neurons (as well as synapses) my fail to fire with a certain probability. It has remained unknown whether one can in principle carry out reliable computation in networks of noisy spiking neurons. This article presents rigorous constructions for simulating in real-time arbitrary given boolean circuits and finite automata on such networks. In addition we show that with the help of "shunting inhibition" such networks can simulate in real-time any McCulloch-Pitts neuron (or "threshold gate"), and therefore any multilayer perceptron (or "threshold circuit") in a reliable manner. In view of the tremendous computational power of threshold circuits (even with few layers), this construction provides a possible explanation for the fact that biological neural systems can carry out quite complex computations within 100 msec. It turns out that the assumptions that these constructions require about the shape of the EPSP's and the behaviour of the noise are surprisingly weak. This article continues the related work from NIPS '94, where we had considered computations on networks of spiking neurons without noise. The current paper will appear in Advances in Neural Information Processing Systems, vol. 8 (= Proc. of NIPS '95) . ************ How to obtain a copy ***************** Via Anonymous FTP: unix> ftp archive.cis.ohio-state.edu Name: anonymous Password: (type your email address) ftp> cd pub/neuroprose ftp> binary ftp> get maass.noisy-spiking.ps.Z ftp> quit unix> uncompress maass.noisy-spiking.ps.Z unix> lpr maass.noisy-spiking.ps (or what you normally do to print PostScript) From piuri@elet.polimi.it Sun Nov 19 19:27:24 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sun, 19 Nov 95 19:27:20 -0600; AA09533 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sun, 19 Nov 95 19:27:18 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa11234; 19 Nov 95 13:29:23 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa11230; 19 Nov 95 13:10:15 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa10136; 19 Nov 95 13:09:53 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa22125; 19 Nov 95 11:36:30 EST Received: from ipmel2.elet.polimi.it by EDRC.CMU.EDU id aa27216; 19 Nov 95 11:36:16 EST Received: by ipmel2.elet.polimi.it (5.65/DEC-Ultrix/4.3A) id AA06093; Sun, 19 Nov 1995 17:35:55 +0100 Date: Sun, 19 Nov 1995 17:35:55 +0100 From: Vincenzo Piuri Message-Id: <9511191635.AA06093@ipmel2.elet.polimi.it> To: connectionists@cs.cmu.edu Subject: NICROSP'96 - call for papers ====================================================================== NICROSP'96 1996 International Workshop on Neural Networks for Identification, Control, Robotics, and Signal/Image Processing Venice, Italy - 21-23 August 1996 ====================================================================== Sponsored by the IEEE Computer Society and the IEEE CS Technical Committee on Pattern Analysis and Machine Intelligence. In cooperation with: ACM SIGART (pending), IEEE Circuits and Systems Society, IEEE Control Systems Society, IEEE Instrumentation and Measurement Society, IEEE Neural Network Council, IEEE North-Italy Section, IEEE Region 8, IEEE Robotics and Automation Society (pending), IEEE Signal Processing Society (pending), IEEE System, Man, and Cybernetics Society, IMACS, INNS (pending), ISCA, AEI, AICA, ANIPLA, FAST. CALL FOR PAPERS This workshop is directed to create a unique synergetic discussion forum and a strong link between theoretical researchers and practitioners in the application fields of identification, control, robotics, and signal/image processing by using neural techniques. The three-days single-session schedule will provide the ideal environment for in-depth analysis and discussions concerning the theoretical aspects of the applications and the use of neural networks in the practice. Invited talks in each area will provide a starting point for the discussion and give the state of the art in the corresponding field. Panels will provide an interactive discussion. Researchers and practitioners are invited to submit papers concerning theoretical foundations of neural computation, experimental results or practical applications related to the specific workshop's areas. Interested authors should submit a half-page abstract to the program chair by e-mail or fax by February 1, 1996, for review planning. Then, an extended summary or the full paper (limited to 20 double-spaced pages including figures and tables) must be sent to the program chair by February 16, 1996 (PostScript email submission is strongly encouraged). Submissions should contain: the corresponding author, affiliation, complete address, fax, email, and the preferred workshop track (identification, control, robotics, signal processing, image processing). Submission implies the willingness of at least one of the authors to register, attend the workshop and present the paper. Papers' selection is based on the full paper: the corresponding author will be notified by March 30, 1996. The camera-ready version, limited to 10 one-column IEEE-book-standard pages, is due by May 1, 1996. Proceedings will be published by the IEEE Computer Society Press. The extended version of selected papers will be considered for publication in special issues of international journals. General Chair Prof. Edgar Sanchez-Sinencio Department of Electrical Engineering Texas A&M University College Station, TX 77843-3128 USA phone (409) 845-7498 fax (409) 845-7161 email sanchez@eesun1.tamu.edu Program Chair Prof. Vincenzo Piuri Department of Electronics and Information Politecnico di Milano piazza L. da Vinci 32, I-20133 Milano, Italy phone +39-2-2399-3606 fax +39-2-2399-3411 email piuri@elet.polimi.it Publication Chair Dr. Jose' Pineda de Gyvez Department of Electrical Engineering Texas A&M University Publicity, Registr. & Local Arrangment Chair Dr. Cesare Alippi Department of Electronics and Information Politecnico di Milano Workshop Secretariat Ms. Laura Caldirola Department of Electronics and Information Politecnico di Milano phone +39-2-2399-3623 fax +39-2-2399-3411 email caldirol@elet.polimi.it Program Committee (preliminary list) Shun-Ichi Amari, University of Tokyo, Japan Magdy Bayoumi, University of Southwestern Louisiana, USA James C. Bezdek, University of West Florida, USA Pierre Borne, Ecole Politechnique de Lille, France Luiz Caloba, Universidad Federal de Rio de Janeiro, Brazil Chris De Silva, University of Western Australia, Australia Laurene Fausett, Florida Institute of Technology, USA C. Lee Giles, NEC, USA Karl Goser, University of Dortmund, Germany Simon Jones, University of Loughborough, UK Michael Jordan, Massachussets Institute of Technology, USA Robert J. Marks II, University of Washington, USA Jean D. Nicoud, EPFL, Switzerland Eros Pasero, Politecnico di Torino, Italy Emil M. Petriu, University of Ottawa, Canada Alberto Prieto, Universidad de Granada, Spain Gianguido Rizzotto, SGS-Thomson, Italy Edgar Sanchez-Sinencio, A&M University, USA Bernd Schuermann, Siemens, Germany Earl E. Swartzlander, University of Texas at Austin, USA Philip Treleaven, University College London, UK Kenzo Watanabe, Shizuoka University, Japan Michel Weinfeld, Ecole Politechnique de Paris, France ====================================================================== From bap@valaga.salk.edu Mon Nov 20 17:47:17 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 20 Nov 95 17:47:14 -0600; AA00782 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 20 Nov 95 16:28:22 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa11535; 19 Nov 95 18:31:31 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa11533; 19 Nov 95 18:18:34 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa10330; 19 Nov 95 18:18:25 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa28734; 19 Nov 95 18:06:02 EST Received: from valaga.salk.edu by CS.CMU.EDU id aa29851; 19 Nov 95 18:04:00 EST Received: (from bap@localhost) by valaga.salk.edu (8.6.12/8.6.9) id PAA07609; Sun, 19 Nov 1995 15:03:37 -0800 Date: Sun, 19 Nov 1995 15:03:37 -0800 From: Barak Pearlmutter Message-Id: <199511192303.PAA07609@valaga.salk.edu> To: Connectionists@cs.cmu.edu Cc: dhw@santafe.edu In-Reply-To: <9511141824.AA04471@sfi.santafe.edu> (message from David Wolpert on Tue, 14 Nov 95 11:24:58 MST) Subject: Re: Response to no-free-lunch discussion Reply-To: Barak.Pearlmutter@scr.siemens.com Reviewing the theory of Kolmogorov complexity, we see that not having low Kolmogorov complexity is equivalent to being random. In other words, the minimal description of anything that does not have low Kolmogorov complexity is "nothing but noise." When you write this We have *no* a priori reason to believe that targets with "low Kolmogorov complexity" (or anything else) are/not likely to occur in the real world. it is precisely equivalent to saying We have *no* a priori reason to believe that targets with any non-random structure whatsoever are likely to occur in the real world. The NFL theory shows conclusively that there are no search algorithms which are particularly good at finding minima of such random functions. However, as scientists, we have had some success by positing the existence of non-random structure in the real world. So it seems to me that there is at least some reason to believe that the functions we optimize in practice are not completely random, in this Kolmogorov complexity sense. From rich@cs.umass.edu Mon Nov 20 17:48:02 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 20 Nov 95 17:47:59 -0600; AA00896 Received: from bilby.cs.uwa.oz.au by lucy.cs.wisc.edu; Mon, 20 Nov 95 15:04:35 -0600 Received: from (mafm@parma.cs.uwa.oz.au [130.95.1.7]) by cs.uwa.oz.au (8.6.8/8.5) with SMTP id VAA21772; Mon, 20 Nov 1995 21:59:41 +0800 Message-Id: <199511201359.VAA21772@cs.uwa.oz.au> From: rich@cs.umass.edu (Rich Sutton) To: reinforce@cs.uwa.edu.au Subject: New pointers for Rich Sutton Date: Fri, 17 Nov 1995 17:50:16 -0500 I've moved! This is to let you know that I have left GTE Labs and am now at the University of Massachusetts. My old gte email address no longer works. I can be reached at: Rich Sutton Department of Computer Science University of Massachusetts Amherst, MA 01003 USA rich@cs.umass.edu http://envy.cs.umass.edu/People/sutton/sutton.html I am also looking for a few good students. UMass has an excellent program in artificial intelligence with large efforts in vision, robotics, cognitive modeling, and reinforcement learning, among many other areas. Also brain/behavior links. Check us out on the web. p.s. My ftp site at GTE is replaced by one at UMass -- please update any pointers to ftp://ftp.cs.umass.edu/pub/anw/pub/sutton/. From vnissen@gwdg.de Mon Nov 20 17:52:24 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 20 Nov 95 17:52:19 -0600; AA01369 Received: from bilby.cs.uwa.oz.au by lucy.cs.wisc.edu; Mon, 20 Nov 95 13:46:13 -0600 Received: from (mafm@parma.cs.uwa.oz.au [130.95.1.7]) by cs.uwa.oz.au (8.6.8/8.5) with SMTP id WAA21817; Mon, 20 Nov 1995 22:00:11 +0800 Message-Id: <199511201400.WAA21817@cs.uwa.oz.au> From: vnissen To: reinforce@cs.uwa.edu.au Subject: New Book on Evol. Algorithms Available Date: Mon, 20 Nov 1995 13:14:31 +0100 (MET) The following book is now available from Springer-Verlag Publishers: Biethahn, J; Nissen, V. (eds.): Evolutionary Algorithms in Management Applications 1995. XVI, 378 pp. 116 fig., 56 tabs., Hardcover, DM 148,- ISBN 3-540-60382-4 Contents ======== Part I - Foundations V. Nissen, J. Biethahn: An Introduction to Evolutionary Algorithms V. Nissen: An Overview of Evolutionary Algorithms in Management Applications Part II - Applications in Industry B. Filipic: A GA Applied to Resource Management in Production Systems I. Rixen; C. Bierwirth, H. Kopfer: A Case Study of Operational Just-in-time Scheduling Using GAs R. Bowden; S. Bullington: An EA for Discovering Manufacturing Control Strategies M. Ettl; M. Schwehm: Determining the Optimal Network Partition and Kanban Allocation in JIT Production Lines M. Krause; V. Nissen: On Using Penalty Functions and Multicriteria Optimisation Techniques in Facility Layout E. Falkenauer:Tapping the Full Power of GA through Suitable Representation and Local Optimization: Application to Bin Packing V. Parada Daza; R. Munoz, A. Gomes de Alvarenga: A Hybrid GA for the Two-Dimensional Guillotine Cutting Problem Part III - Applications in Trade R. Broekmeulen: Facility Management of Distribution Centres for Vegetables and Fruits T. Terano; Y. Ishino; K. Yoshinaga: Integrating Machine Learning and Simulated Breeding Techniques to Analyze the Characteristics of Consumer Goods R. Marks; D. Midgley; L. Cooper: Adaptive Behaviour in an Oligopoly V. Nissen; J. Biethahn: Determining a Good Inventory Policy with a GA Part IV - Applications in Financial Services R. Bauer: GAs and the Management of Exchange Rate Risk N. Ireson; T. Fogarty: Evolving Decision Support Models for Credit Control S. Vere: Genetic Classification Trees M. de la Maza; D. Yuret: A Model of Stock Market Participants Part V - Applications in Traffic Management J. McDonnell, D. Fogel; C. Rindt; W. Recker; L. Fogel: Using Evolutionary Programming to Control Metering Rates on Freeway Ramps I. Gerdes: Application of GAs for Solving Problems Related to Free Routing for Aircraft F. Baita; F. Mason; C. Poloni; W. Ukovich: GA with Redundancies for the Vehicle Scheduling Problem Part VI - Planning in Education W. Junginger: Course Scheduling by GAs Appendix Please order through your bookseller or from: Springer-Verlag Postfach 31 13 40 D-10643 Berlin; Germany ---------------------------------------------------------------------------- From bernabe@cnm.us.es Mon Nov 20 22:22:37 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 20 Nov 95 22:22:35 -0600; AA06840 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 20 Nov 95 22:22:33 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa13185; 20 Nov 95 17:01:02 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa13183; 20 Nov 95 16:40:58 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa11084; 20 Nov 95 16:40:46 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa17517; 20 Nov 95 6:38:27 EST Received: from cnm1.cnm.us.es by CS.CMU.EDU id aa03169; 20 Nov 95 6:37:31 EST Received: from cnm12.cnm.us.es by cnm1.cnm.us.es (4.1/SMI-4.1) id AA12650; Mon, 20 Nov 95 12:37:13 +0100 Date: Mon, 20 Nov 95 12:37:13 +0100 From: "Bernabe Linares B." Message-Id: <9511201137.AA12650@cnm1.cnm.us.es> To: Connectionists@cs.cmu.edu NIPS'95 WORKSHOP ANNOUNCEMENT Vail, Colorado. Friday, December 1st, 1995 Title: NEURAL HARDWARE ENGINEERING: From Sensory Data Adquisition to High-Level Intelligent Processing Organizers: Bernabe Linares-Barranco and Angel Rodriguez-Vazquez Dept. of Analog and Mixed-Signal Circuit Design Microelectronics National Center, Sevilla Ed. CICA, Av. Reina Mercedes s/n, 41012 Sevilla, SPAIN FAX: 34-5-4624506; Phone: 34-5-4239923; email: bernabe@cnm.us.es DESCRIPTION OF THE WORKSHOP: Developing hardware for neural applications is a task that hardware engineers have faced during the past years using two distinct main approaches: (a) producing "general purpose" digital neuro-computing systems or "neural- -accelerators" with a certain degree of flexibility to emulate different neural architectures and learning rules. (b) developing "special purpose" neuro-chips, mostly using analog or mixed analog/digital circuit techniques, intended to solve a specific problem with very high speed and efficiency. Usually hardware of task (b) is used for the front end of a neural processing system, such as sensory data (image/sound) adquisition and sometimes with some extra (pre)processing functionality (noise removal, automatic gain, dimensionality reduction). On the other hand, hardware of task (a) is employed for more "intelligent" or higher level processing such as learning, clustering, recognition, abstraction, and conceptualization. However, the limits between hardware of type (a) and (b) are not very clear, and as more hardware is developed the overlap between the two approaches increases. Digital technology provides larger accuracy in the realization of mathematical operations and offers great flexibility to change learning paradigms, learning rules, or to tune critical parameters. Analog technology, on the other hand, provides very high area and power efficiency, but is less accurate and flexible. It is clear that people that are developing neural algorithms need to have some type of digital neurocomputer system where they can change rapidly the neural architecture, the topology, the learning rules, try different mathematical functions, and all that with sufficient flexibility and computing power. On the other hand, when neural systems require image or sound adquisition capabilities (retinas or cochleas) analog technology offers very high power and chip area efficiency, so that this approach seems to be the preferred one. However, what happens when it comes to develop a hardware system that needs to handle the sensory data, perform some basic processing, and continue processing up to higher level stages where data segmentation has to be performed, recognition on the segments has to be achieved, and learning and abstraction should be fulfilled? Is there any clear border among analog and digital techniques as we proceed upwards in the processing cycle from signal acquisition to conceptualization? Is it possible to take advantage of the synergy between analog and digital? How? Are these conclusions the same for vision, hearing, olfactory, or intelligent control applications? We believe it is a good point in time to make a debate between representatives of the two approaches, since both have evolved independently into a large enough degree of development and maturity as to enable pros and counters be discussed on the basis of objective, rather than subjective considerations. LIST OF SPEAKERS: 1. Nelson Morgan, University of California, Berkeley, U.S.A. "Using A Fixed-point Vector Microprocessor for Connectionist Speech Recognition Training" 2. Yuzo Hirai, Institute of Information Sciences and Electronics, University of Tsukuba, Japan. "PDM Digital Neural Networks" 3. Taher Daud, Jet Propulsion Laboratory, Pasadena, California, U.S.A. "Focal Plane Imaging Array-Integrated 3-D Neuroprocessor" 4. Ulrich Ramacher, University of Technology, Dresden, Germany. "The Siemens Electronic Eye Proyect" 5. Xavier Arreguit, CSEM, Neuchatel, Switzerland "Analog VLSI for Perceptive Systems" 6. Andreas Andreou, John Hopkins University, Baltimore, Maryland, U.S.A. "Silicon Retinas for Contrast Sensitivity and Polarization Sensing" 7. Marwan Jabri, University of Sydney, Australia. "On-Chip Learning in Analog VLSI and its Application in Biomedical Implant Devices" 8. Tadashi Shibata, Tohoku University, Sendai, Japan. "Neuron-MOS Binary-Analog Merged Hardware Computation for Intelligent Information Processing" From marshall@cs.unc.edu Tue Nov 21 07:23:39 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 21 Nov 95 07:23:37 -0600; AA13555 Received: from bilby.cs.uwa.oz.au by lucy.cs.wisc.edu; Tue, 21 Nov 95 07:23:30 -0600 Received: from (mafm@parma.cs.uwa.oz.au [130.95.1.7]) by cs.uwa.oz.au (8.6.8/8.5) with SMTP id NAA28127; Tue, 21 Nov 1995 13:59:57 +0800 Message-Id: <199511210559.NAA28127@cs.uwa.oz.au> From: Jonathan Marshall To: Vision-List@TELEOS.COM, connectionists@cs.cmu.edu, intcon@phoenix.ee.unsw.edu.au, neuron@cattell.psych.upenn.edu, reinforce@cs.uwa.edu.au, tanns@cs.unc.edu Subject: CFP: Biologically Inspired Autonomous Systems [connectionists] Date: Mon, 20 Nov 1995 13:58:45 -0400 ---------------------------------------------------------------------------- Biologically Inspired Autonomous Systems: Computation, Cognition, and Action March 4-5, 1996 Washington Duke Hotel (Duke University) Durham, North Carolina Co-Sponsored by the Duke University Departments of Electrical and Computer Engineering, Neurobiology, Biomedical Engineering, and Experimental Psychology The dramatic evolution of computer technology has caused a return to the biological paradigms which inspired many of the early pioneers of information science such as John von Neumann, Stephen Kleene and Marvin Minsky. Similarly, many fields of the life and human sciences have been influenced by paradigms initiated in systems theory, computation and control Engineering. The purpose of this workshop is to pursue this fruitful interaction of engineering and the exact sciences, with the life and human sciences, by investigating the processes which can provide systems, both artificial and natural, with autonomous and adaptive behavior. Topics of interest include Autonomous behavior of biophysically and cognitively inspired models Autonomous agents and mobile systems Collective behaviour by semi-autonomous agents Self repair and regeneration in computational and artificial structures Autonomous image understanding Brain imaging and Functional MRI Keynote Speakers: Stephen Grossberg (Boston University), Daniel Mange (EPFL), Jean-Arcady Meyer (ENS, Paris), Heinz Muehlenbein (GMD, Bonn), John Taylor (University College, London). Speakers will include: Paul Bourgine (Ecole Polytechnique), Bernadette Dorizzi (INT, Evry), Warren Hall (Duke), Ivan Havel (Center for Theoretical Studies and Prague University), Petr Lansky (Center for Theoretical Studies and Prague University), Miguel Nicollelis (Duke), Richard Palmer (Duke), David Rubin (Duke) , Nestor Schmajuk (Duke), John Staddon (Duke), John Taylor (University College, London), Ed Ueberbacher (Oak Ridge National Laboratory), Paul Werbos (NSF). Paper submissions, in the form of four page extended abstracts, are solicited in areas of relevance to this workshop. They should be sent before January 15, 1996 to one of the workshop Co-Chairs. The Workshop Proceedings will be published in book form with full papers. Workshop Co-Chairs: Erol Gelenbe Nestor Schmajuk Department of Electrical and Department of Experimental Psychology Computer Engineering Duke University Duke University Durham, NC 27708, USA Durham, NC 27708-0291, USA nestor@acpub.duke.edu erol@ee.duke.edu ---------------------------------------------------------------------------- From stefano@kant.irmkant.rm.cnr.it Tue Nov 21 15:53:53 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 21 Nov 95 15:53:49 -0600; AA19856 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 21 Nov 95 15:53:47 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa13705; 20 Nov 95 22:09:01 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa13703; 20 Nov 95 21:58:17 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa11197; 20 Nov 95 21:57:47 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa27939; 20 Nov 95 12:21:15 EST Received: from kant.irmkant.rm.cnr.it by EDRC.CMU.EDU id aa01120; 20 Nov 95 12:18:39 EST Received: by kant.irmkant.rm.cnr.it (AIX 3.2/UCB 5.64/4.03) id AA16840; Mon, 20 Nov 1995 18:15:32 GMT Date: Mon, 20 Nov 1995 18:15:32 GMT From: stefano@kant.irmkant.rm.cnr.it Message-Id: <9511201815.AA16840@kant.irmkant.rm.cnr.it> To: connectionists@cs.cmu.edu Subject: Paper available: Learning to adapt to changing environments Papers available via WWW / FTP: Keywords: Learning, Adaptation to changing environments, Evolutionary Robotics Neural Networks, Genetic Algorithms, ------------------------------------------------------------------------------ LEARNING TO ADAPT TO CHANGING ENVIRONMENTS IN EVOLVING NEURAL NETWORKS Stefano Nolfi & Domenico Parisi Institute of Psychology, C.N.R., Rome. In order to study learning as an adaptive process it is necessary to take into consideration the role of evolution which is the primary adaptive process. In addition, learning should be studied in (artificial) organisms that live in an independent physical environment in such a way that the input from the environment can be at least partially controlled by the organisms' behavior. To explore these issues we used a genetic algorithm to simulate the evolution of a population of neural networks each controlling the behavior of a small mobile robot that must explore efficiently an environment surrounded by walls. Since the environment changes from one generation to the next each network must learn during its life to adapt to the particular environment it happens to be born in. We found that evolved networks incorporate a genetically inherited predisposition to learn that can be described as: (a) the presence of initial conditions that tend to canalize learning in the right directions; (b) the tendency to behave in a way that enhances the perceived differences between different environments and determines input stimuli that facilitate the learning of adaptive changes, and (c) the ability to reach desirable stable states. http://kant.irmkant.rm.cnr.it/public.html or ftp-server: kant.irmkant.rm.cnr.it (150.146.7.5) ftp-file : /pub/econets/nolfi.changing.ps.Z for the homepage of our research group with most of our publications available online and pointers to ALIFE resources see: http://kant.irmkant.rm.cnr.it/gral.html ---------------------------------------------------------------------------- Stefano Nolfi Institute of Psychology National Research Council e-mail: stefano@kant.irmkant.rm.cnr.it From dhw@santafe.edu Tue Nov 21 15:53:57 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 21 Nov 95 15:53:54 -0600; AA19867 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 21 Nov 95 15:53:50 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab13719; 20 Nov 95 22:19:16 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab13707; 20 Nov 95 21:59:55 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa11207; 20 Nov 95 21:58:41 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa00591; 20 Nov 95 13:57:58 EST Received: from [192.12.12.1] by CS.CMU.EDU id aa06915; 20 Nov 95 13:57:21 EST Received: from santafe (santafe.santafe.edu) by sfi.santafe.edu (4.1/SMI-4.1) id AA11261; Mon, 20 Nov 95 11:54:43 MST Date: Mon, 20 Nov 95 11:54:43 MST From: David Wolpert Message-Id: <9511201854.AA11261@sfi.santafe.edu> To: Connectionists@cs.cmu.edu Subject: Some more on NFL Barak Pearlmutter writes: >>> not having low Kolmogorov complexity is equivalent to being random... (the) description of anything that does not have low Kolmogorov complexity is "nothing but noise." >>> I don't necessarily disagree with such sentiments, but one should definitely be careful about them; there are *many* definitions of "random". (Seth Lloyd has counted about 30 versions of its flip side, "amount of information".) High Kolmogorov complexity is only one of them. To illustrate just one of the possible objections to measuring randomness with Kolmogorov complexity: Would you say that a macroscopic gas with a specified temperature is "random"? To describe it exactly takes a huge Kolmogorov complexity. And certainly in many regards its position in phase space is "nothing but noise". (Indeed, in a formal sense, its position is a random sample of the Boltzmann distribution.) Yet Physicists can (and do) make extraordinarilly accurate predictions about such creatures with ease. Another important (and related) point is that encoding constant that gets buried in the definition of Kolmogorov complexity - in practive it can be very important. To put it another way, one person's "random" is another person's "highly regular"; this is precisely why the basis you use in supervised learning matters so much. >>> as scientists, we have had some success by positing the existence of non-random structure in the real world. So it seems to me that there is at least some reason to believe that the functions we optimize in practice are not completely random, in this Kolmogorov complexity sense. >>> Oh, most definitely. To give one simple example: Cross-validation works quite well in practice. However either 1) for *any* fixed target (i.e., for any prior over targets), averaged over all sets of generalizers you're choosing among, cross-validation works no better than anti-cross-validation (choose the generalizer in the set at hand having *largest* cross-validation error); and 2) for a fixed set of generalizers, averaged over all targets, it works no better than anti-cross-validation. So for cross-validation to work requires a very subtle inter-relationship between the prior over targets and the set of generalizers you're choosing among. In particular, cross-validation cannot be given a Bayesian justification without regard to the set of generalizers. Nonetheless, I (and every other evenly marginally rational statistician) have used cross-validation in the past, and will do so again in the future. >From a theoretical point of view, the fascinating question is how to characterize the needed relationship between the set of generalizers and the prior that allows cross-validation to work. David Wolpert From niranjan@eng.cam.ac.uk Tue Nov 21 15:53:58 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 21 Nov 95 15:53:54 -0600; AA19870 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 21 Nov 95 15:53:52 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa13909; 21 Nov 95 0:48:31 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa13907; 21 Nov 95 0:39:42 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa11507; 21 Nov 95 0:39:18 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa10121; 20 Nov 95 23:57:32 EST Received: from spanner.eng.cam.ac.uk by CS.CMU.EDU id aa12217; 20 Nov 95 23:56:49 EST Received: from dsl.eng.cam.ac.uk (via root@dsl.eng.cam.ac.uk [129.169.80.1]) by spanner.eng.cam.ac.uk with ESMTP id EAA18112 for ; Tue, 21 Nov 1995 04:56:46 GMT Received: from baby.eng.cam.ac.uk (via niranjan@baby.eng.cam.ac.uk [129.169.80.37]) by dsl.eng.cam.ac.uk with SMTP id EAA16669 for ; Tue, 21 Nov 1995 04:56:45 GMT From: niranjan@eng.cam.ac.uk Date: Tue, 21 Nov 95 04:56:44 GMT Message-Id: <9511210456.9837@baby.eng.cam.ac.uk> Received: by baby.eng.cam.ac.uk id AA09837 for connectionists@cs.cmu.edu; Tue, 21 Nov 95 04:56:44 GMT To: connectionists@cs.cmu.edu Subject: JOB JOB JOB -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Research Assistant Position for One Year A Research Assistant position is available in Cambridge to investigate the use of: Neural Networks in the Prediction of Risk in Pregnancy Euro-PUNCH is a collaborative Project funded by the Human Capital and Mobility Programme of the Commission of the European Communities. Thus, the post is available only to a citizen of a European Union Member State (but not British), who wishes to come to work in the United Kingdom. >From the obstetrical point of view, the principal focus of the Euro-PUNCH Project lies in the use of patient-specific measurements of an epidemiological nature (such as maternal age, past obstetrical history, etc.) as well as fetal heart rate recordings, in the forecasting of a number of specific Adverse Pregnancy Outcomes. >From the neural network point of view, the Project involves the design of pattern-processing and classification systems which can be trained to forecast problems in pregnancy. This will involve continuation of work on pattern-classification and regression analysis, using neural networks operating on a very large database of about 1.2 million pregnancies from various European countries. Challenging components of the project include dealing with missing and uncertain variables, sensitivity analysis, variable selection procedures and cluster analysis. Many leading European obstetrical centres are involved in the Euro-PUNCH project, and close collaboration with a number of these will be an essential component of the post offered. Candidates for this post are expected to have a good first degree and preferably a post-graduate degree in a relevant discipline. Come familiarity with medical statistics and neural networks is desirable but not essential. Salary (on the RA scale) will depend on age and experience, and is likely to be in the range of #14,317 to #15,986 per annum. Appointment would be subject to satisfactory health screening. Applications will close on Friday 8th December 1995. Applications (naming two referees) should be submitted to: Dr Kevin J Dalton PhD FRCOG Division of Materno-Fetal Medicine, Dept. Obstetrics & Gynaecology University of Cambridge, Addenbrooke's Hospital Cambridge CB2 2QQ Tel: +44-1223-410250 Fax: +44-1223-336873 or 215327 e-mail: kjd5@cus.cam.ac.uk Informal enquiries about the project should be directed to: (Obstetric side) Dr Kevin Dalton kjd5@cus.cam.ac.uk (Engineering Side) Dr Niranjan niranjan@eng.cam.ac.uk (Engineering Side) Dr Richard Prager rwp@eng.cam.ac.uk -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- From Olivier.Michel@alto.unice.fr Wed Nov 22 03:32:15 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 22 Nov 95 03:32:09 -0600; AA22973 Received: from bilby.cs.uwa.oz.au by lucy.cs.wisc.edu; Wed, 22 Nov 95 03:32:03 -0600 Received: from (mafm@parma.cs.uwa.oz.au [130.95.1.7]) by cs.uwa.oz.au (8.6.8/8.5) with SMTP id LAA04466; Wed, 22 Nov 1995 11:44:04 +0800 Message-Id: <199511220344.LAA04466@cs.uwa.oz.au> From: Olivier MICHEL To: reinforce@cs.uwa.edu.au, gann-list@cs.iastate.edu, GA-List@aic.nrl.navy.mil, connectionists@cs.cmu.edu, neuron@cattell.psych.upenn.edu Subject: Announcement: Khepera Simulator 1.0 [connectionists] Date: Tue, 21 Nov 1995 16:02:52 +0100 * ANNOUNCEMENT OF NEW PUBLIC DOMAIN SOFTWARE PACKAGE * ------------------------------------------------------------- - Khepera Simulator version 1.0 - ------------------------------------------------------------- Khepera Simulator is a public domain software package written by Olivier MICHEL. It allows to write your own controller for the mobile robot Khepera using C or C++ languages, to test them in a simulated environment and features a nice colorful X11 graphical interface. Moreover, if you own a Khepera robot, it can drive the real robot using the same control algorithm. It is mainly destinated to researchers studying autonomous agents. o Requirements: UNIX system, X11 library. o User Manual and examples of controllers included. o This software is free of charge for research and teaching. o Commercial use is forbidden. o Khepera is a mini mobile robot developped at EPFL by Francesco Mondada, Edo Franzi and Andre Guignard (K-Team). o You can download it from the following web site: http://wwwi3s.unice.fr/~om/khep-sim.html Olivier MICHEL om@alto.unice.fr http://wwwi3s.unice.fr/~om/ From tds@ai.mit.edu Wed Nov 22 18:10:33 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 22 Nov 95 18:10:29 -0600; AA04011 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 22 Nov 95 18:10:27 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa15076; 21 Nov 95 15:31:42 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa15065; 21 Nov 95 15:07:01 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa12149; 21 Nov 95 15:06:14 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa20038; 21 Nov 95 12:50:28 EST Received: from ai.mit.edu by CS.CMU.EDU id aa16751; 21 Nov 95 12:49:31 EST Received: from dentate.ai.mit.edu (DENTATE.MIT.EDU) by life.ai.mit.edu (4.1/AI-4.10) for connectionists@cs.cmu.edu id AA13714; Tue, 21 Nov 95 12:10:49 EST From: "Terence D. Sanger" Received: by dentate.ai.mit.edu (4.1/AI-4.10) id AA08251; Tue, 21 Nov 95 12:10:47 EST Date: Tue, 21 Nov 95 12:10:47 EST Message-Id: <9511211710.AA08251@dentate.ai.mit.edu> To: connectionists@cs.cmu.edu Subject: NIPS workshop announcement NIPS*95 Post-Conference Workshop "Vertebrate Neurophysiology and Neural Networks: Can the teacher learn from the student?" Saturday December 2, 7:30-9:30AM, 4:30-6:30PM Organizer: Terence Sanger, MIT. SUMMARY Results from neurophysiological investigations continue to guide the development of artificial neural network models that have been shown to have wide applicability in solving difficult computational problems. This workshop addresses the question of whether artificial neural network models can be applied to understanding neurophysiological results and guiding further experimental investigations. Recent work on close modelling of vertebrate neurophysiology will be presented, so as to give a survey of some of the results in this field. We will concentrate on examples for which artificial neural network models have been constructed to mimic the structure as well as the function of their biological counterparts. Clearly, this can be done at many different levels of abstraction. The goal is to discuss models that have explanatory and predictive power for neurophysiology. The following questions will serve as general discussion topics: 1. Do artificial neural network models have any relationship to ``real'' Neurophysiology? 2. Have any such models been used to guide new biological research? 3. Is Neurophysiology really useful for designing artificial networks, or does it just provide a vague ``inspiration''? 4. How faithfully do models need to address ultrastructural or membrane properties of neurons and neural circuits in order to generate realistic predictions of function? 5. Are there any artificial network models that have applicability across different regions of the central nervous system devoted to varied sensory and motor modalities? 6. To what extent do theoretical models address more than one of David Marr's levels of algorithmic abstraction (general approach, specific algorithm, and hardware implementation)? Selected examples of Neural Network models for Neurophysiological results will be presented, and active audience participation and discussion will be encouraged. SCHEDULE Saturday, December 2 7:30 - T. Sanger: Introduction and Overview 8:00 - T. Sejnowski: "Bee Foraging in Uncertain Environments using Predictive Hebbian Learning" 8:30 - A. Pouget: "Spatial Representations in the Parietal Cortex may use Basis Functions" 9:00 - Discussion --- Break --- 4:30 - S. Giszter: "Spinal Primitives and their Dynamics in Vertebrate Limb Control: A Biological Perspective" 5:00 - G. Goodhill: "Modelling the Development of Primary Visual Cortex: Determinants of Ocular Dominance Column Periodicity" 5:30 - Discussion From prechelt@ira.uka.de Wed Nov 22 18:10:36 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 22 Nov 95 18:10:32 -0600; AA04021 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 22 Nov 95 18:10:29 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab15076; 21 Nov 95 15:33:03 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab15065; 21 Nov 95 15:07:05 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa12156; 21 Nov 95 15:06:34 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa20970; 21 Nov 95 13:38:01 EST Received: from iraun1.ira.uka.de by CS.CMU.EDU id aa17208; 21 Nov 95 13:36:24 EST Received: from ira.uka.de (actually i41s25.ira.uka.de) by iraun1.ira.uka.de with SMTP (PP); Tue, 21 Nov 1995 19:34:50 +0100 To: connectionists@cs.cmu.edu Subject: NIPS Workshop: Benchmarking of NN learning algorithms Date: Tue, 21 Nov 1995 19:34:24 +0100 From: Lutz Prechelt Message-Id: <"iraun1.ira.532:21.11.95.18.34.54"@ira.uka.de> X-URL: http://wwwipd.ira.uka.de/~prechelt/NIPS_bench.html NIPS Workshop: Benchmarking of NN learning algorithms ******************************************************** Abstract: Proper benchmarking of neural network learning architectures is a prerequisite for orderly progress in this field. In many published papers deficiencies can be observed in the benchmarking that is performed. The workshop addresses the status quo of benchmarking, common errors and how to avoid them, currently existing benchmark collections, and, most prominently, a new benchmarking facility including a results database. The workshop goal is to improve benchmarking practices and to improve the comparability of benchmark tests. Workshop Chairs: o Thomas G. Dietterich , o Geoffrey Hinton , o Wolfgang Maass , o Lutz Prechelt [communicating chair] o Terry Sejnowski Length: 1 day Date: Saturday, December 2, 1995, 7:30AM-9:30AM and 4:30PM-6:30PM. Topic and purpose of the workshop ================================= Proper benchmarking of neural networks on non-toy examples is needed from an application perspective in order to evaluate the relative strenghts and weaknesses of proposed algorithms and from a theoretical perspective in order to validate theoretical predictions and see how they relate to realistic learning tasks. Despite this important role, NN benchmarking is rarely done well enough today: o Learning tasks: Most researchers use only toy problems and, perhaps, one at least somewhat realistic problem. While this shows that an algorithm works at all, it cannot explore its strenghts and weaknesses. o Design: Often the setup is designed wrongly and cannot produce valid results from a statistical point of view. o Reproducibility: In many cases, the setup is not described exactly enough to reproduce the experiments. This violates scientific principles. o Comparability: Hardly ever are two setups of different researchers so similar that one could directly compare the experiment results. This has the effect that even after a large number of experiments with certain algorithms, their differences in learning results may remain unclear. There are various reasons why we still find this situation: o unawareness of the importance of proper benchmarking; o insufficient pressure from reviewers towards good benchmarking; o unavailability of a sufficient number of standard benchmarking datasets; o lack of standard benchmarking procedures. The purpose of the workshop is to address these issues in order to improve research practices, in particular more benchmarking with more and better datasets, better reproducibility, and better comparability. Specific questions to be addressed on the workshop are [Concerning the data:] o What benchmarking facilities (in particular: datasets) are publicly available? For which kinds of domains? How suitable are they? o What facilities would we like to have? Who is willing to prepare and maintain them? o Where and how can we get new datasets from real applications? [Concerning the methodology:] o When and why would we prefer artificial datasets over real ones and vice versa? o What data representation is acceptable for general benchmarks? o What are the most common errors in performing benchmarks? How can we avoid them? o Real-life benchmarking warstories and lessons learned o What must be reported for proper reproducibility? o What are useful general benchmark approaches (broad vs. deep etc.)? o Can we agree on a small number of standard benchmark setup styles in order to improve comparability? Which styles? The workshop will focus on two things: Launching a new benchmark database that is currently being prepared by some of the workshop chairs and discussing the above questions in general and in the context of this database. The benchmark database facility is planned to comprise o datasets, o data format conversion tools, o terminological and methodological suggestions, and o a results database. Workshop format =============== We invite anyone who is interested in the above issues to participate in the discussions at the workshop. The workshop will consist of a few talks by invited speakers and extensive discussion periods. The purpose of the discussion is to refine the design and setup of the benchmark collection, to explore questions about its scope, format, and purpose, to motivate potential users and contributors of the facility, and to discuss benchmarking in general. Workshop program ================ The following talks will be given at the workshop [The list is still preliminary]. After each talk there will be time for discussion. In the morning session we will focus on assessing the state of the practice of benchmarking and discussing an abstract ideal of it. In the afternoon session we will try to become concrete how that ideal might be realized. o Lutz Prechelt. A quantitative study of current benchmarking practices. A quantitative survey of 400 journal articles on NN algorithms. (15 minutes) o Tom Dietterich. Experimental Methodology. Benchmarking goals, measures of behavior, correct statistical testing, synthetic versus real-world data. (15 minutes) o Brian Ripley. What can we learn from the study of the design of experiments? (15 minutes) o Lutz Prechelt. Available NN benchmarking data collections. CMU nnbench, UCI machine learning databases archive, Proben1, Statlog data, ELENA data (10 minutes). o Tom Dietterich. Available benchmarking data generators. (10 minutes) o Break. o Carl Rasmussen and Geoffrey Hinton. A thoroughly designed benchmark collection. A proposal of data, terminology, and procedures and a facility for the collection of benchmarking results. (45 minutes) o Panel discussion. The future of benchmarking: purpose and procedures The WWW adress for this announcement is http://wwwipd.ira.uka.de/~prechelt/NIPS_bench.html Lutz Dr. Lutz Prechelt (http://wwwipd.ira.uka.de/~prechelt/) | Whenever you Institut fuer Programmstrukturen und Datenorganisation | complicate things, Universitaet Karlsruhe; D-76128 Karlsruhe; Germany | they get (Phone: +49/721/608-4068, FAX: +49/721/694092) | less simple. From avrama@wo.erim.org Wed Nov 22 18:10:38 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 22 Nov 95 18:10:34 -0600; AA04023 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 22 Nov 95 18:10:31 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ac15076; 21 Nov 95 15:33:45 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa15067; 21 Nov 95 15:07:58 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa12166; 21 Nov 95 15:07:27 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa22816; 21 Nov 95 15:05:57 EST Received: from wopia.wo.erim.org by EDRC.CMU.EDU id aa06947; 21 Nov 95 15:04:59 EST Received: by wopia.wo.erim.org (920330.SGI/SMI-4.1) id AA05528; Tue, 21 Nov 95 15:03:20 -0500 Date: Tue, 21 Nov 1995 15:03:20 +30000 From: Avrama Blackwell To: connectionists@cs.cmu.edu Subject: position open Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII PRE- OR POST-DOCTORAL FELLOWSHIP IN NEUROCOMPUTING Applications are invited for the position of Pre- or Post-doctoral Fellow. The Fellow will be an integral member of a team collaborating with NIH/NINDS in the development of advanced models of associative learning and visual information processing. Position requires interest and experience in computational neurobiology or development of neural network models as well as good 'C' or 'C++' programming skills. For a review of recent activities of the group see: Alkon et al. In Neural Networks 7: 1005 (1994). The appointment, for one year with possibility of renewal, will be a joint appointment at ERIM (Environmental Research Institute of Michigan, Washington Office) and George Mason University. If interested, either contact Tom Vogl at the NIPS conference in Denver (Dr. Vogl will NOT be at the Workshops) or send statement of interest and CV to tvogl@erim.org. From caruana+@cs.cmu.edu Wed Nov 22 18:10:38 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 22 Nov 95 18:10:35 -0600; AA04029 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 22 Nov 95 18:10:33 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ad15076; 21 Nov 95 15:35:08 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab15067; 21 Nov 95 15:08:10 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa12171; 21 Nov 95 15:07:37 EST Received: from GS79.SP.CS.CMU.EDU by B.GP.CS.CMU.EDU id aa21908; 21 Nov 95 14:20:48 EST Received: from GS79.SP.CS.CMU.EDU by GS79.SP.CS.CMU.EDU id aa09074; 21 Nov 95 14:20:26 EST To: connectionists@cs.cmu.edu Cc: dsilver@csd.uwo.CA Subject: NIPS*95 Workshop on Transfer in Inductive Systems From: Rich Caruana Date: Tue, 21 Nov 95 14:20:09 EST Message-Id: <9072.816981609@GS79.SP.CS.CMU.EDU> Sender: Richard_Caruana@GS79.SP.CS.CMU.EDU Post-NIPS*95 Workshop, December 1-2, 1995, Vail, Colorado TITLE: "Learning to Learn: Knowledge Consolidation and Transfer in Inductive Systems" ORGANIZERS: Rich Caruana (co-chair), Danny Silver (co-chair), Jon Baxter, Tom Mitchell, Lori Pratt, Sebastian Thrun INVITED TALKS BY: Leo Breiman (Berkeley) Tom Mitchell (CMU) Tomaso Poggio (MIT) Noel Sharkey (Sheffield) Jude Shavlik (Wisconsin) WEB PAGE: http://www.cs.cmu.edu/afs/cs/usr/caruana/pub/transfer.html DESCRIPTION: Because the power of tabula rasa learning is limited, interest is increasing in methods that capitalize on previously acquired domain knowledge. Examples of these methods include: o using symbolic domain theories to bias connectionist networks o using extra outputs on a connectionist network to bias the hidden layer representation towards more predictive features o using unsupervised learning on a large corpus of unlabelled data to learn features useful for subsequent supervised learning on a smaller labelled corpus o using models previously learned for other problems as a bias when learning new, but related, problems There are many approaches: hints, knowledge-based artificial neural nets (KBANN), explanation-based neural nets (EBNN), multitask learning (MTL), knowledge consolidation, ... What they all have in common is the attempt to transfer knowledge from other sources to benefit the current inductive task. The goal of this workshop is to provide an opportunity for researchers and practitioners to discuss problems and progress in knowledge transfer in learning. We hope to identify research directions, debate theories and approaches, discover unifying principles, and begin to start answering questions like: o when will transfer help -- or hinder? o what should be transferred and how? o what are the benefits of transfer? o in what domains is transfer most useful? o is there evidence for transfer in nature? >From the 21 submissions we received, 11 were selected for short presentations. The current workshop schedule is as follows: Friday, Dec 1 ============= AM 7:30-7:35 Welcome 7:35-8:05 Tom Mitchell (invited talk) - "Situated Learning" 8:05-8:25 Lorien Pratt - "Neural Transfer For Hazardous Waste" 8:25-8:45 Nathan Intrator - "Learning Internal Reps From Multiple Tasks" 8:45-9:05 Rich Caruana - "Where is Multitask Learning Useful?" 9:05-9:30 Panel Debate and Discussion - Topics include: serial vs. parallel transfer, what should be transfered? what domains are ripe for transfer? what are the goals of transfer? (Baxter, Caruana, Intrator, Mitchell, Silver, Pratt, ...) 9:30-4:30 Extracurricular Recreation PM 4:30-5:00 Jude Shavlik (invited talk) - "Talking to Your Neural Net" 5:00-5:20 Leo Breiman (invited talk) - "Curds & Whey" 5:20-5:40 Jonathan Baxter - "Bayesian Model of Learning to Learn" 5:40-6:00 Sebastian Thrun - "Identifying Relevant Tasks" 6:00-6:30 Panel Debate and Discussion - Topics include: transfer human to machine vs. machine to machine, is practice meeting theory? is theory meeting practice? (Baxter, Caruana, Breiman, Thrun, Mitchell, Shavlik, ...) Saturday, Dec 2 =============== AM 7:30-8:00 Noel Sharkey (invited talk) - "Adaptive Generalisation" 8:00-8:20 Anthony Robbins - "Rehearsal and Catastrophic Interference" 8:20-8:40 J. Schmidhuber - "A Theoretical Model of Learning to Learn" 8:40-9:00 Bairaktaris/Levy - "Dual-weight ANNs: Short/Long Term Learning" 9:00-9:30 Panel Debate and Discussion - Topics include: catastrophic interference, is there evidence for transfer in cognition? what can nature/cogsci tell us about transfer? (Bairaktaris, de Sa, Levy, Robbins, Sharkey, Silver, ...) 9:30-4:30 More Extracurricular Recreation PM 4:30-5:00 Tomaso Poggio (invited talk) - "Virtual Examples" 5:00-5:20 Virginia de Sa - "On Segregating Input Dimensions" 5:20-5:40 Chris Thornton - "Learning to be Brave: A Constructive Approach" 5:40-6:00 Mark Ring - "Continual Learning" 6:00-6:25 Panel Debate and Discussion - Topics include: combining supervised and unsupervised learning, where do we go from here? *this space intentionally left flexible* (de Sa, Mitchell, Poggio, Ring, Thornton, ...) 6:25-6:30 Farewell Full titles and abstracts are available on the workshop web page. 20 minute talks are 12 minutes presentation and 8 minutes questions and discussion. 30 minute invited talks are 20 minutes presentation and 10 minutes questions and discussion. There are four 30-minute panels, one for each session. Although topics are listed for each panel, these are intended merely as points of departure. Everyone attending the workshop should feel free to raise any issues during the panels that seem appropriate. We encourage speakers and members of the audience to prepare a terse list (preferably using inflammatory language) of your favorite transfer issues and questions. There are 16 talks, but this is not a conference! If speakers don't abuse their question/discussion time too much, more than 50% of the workshop will be spent on questions and discussion. To promote this, talks will use few slides and will focus on a few key issues. It's a workshop. Come preapred to speak up, be controversial, and have fun. Look forward to seeing you at Vail. -Danny, Jon, Lori, Rich, Sebastian, and Tom. From omlinc@research.nj.nec.com Wed Nov 22 18:10:42 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 22 Nov 95 18:10:39 -0600; AA04039 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 22 Nov 95 18:10:37 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa15209; 21 Nov 95 17:18:47 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa15207; 21 Nov 95 17:01:32 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa12245; 21 Nov 95 17:01:02 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa24007; 21 Nov 95 15:58:03 EST Received: from zingo.nj.nec.com by EDRC.CMU.EDU id aa07171; 21 Nov 95 15:57:20 EST Received: by zingo (940816.SGI.8.6.9/YDL1.4-910307.16) id PAA08759(zingo); Tue, 21 Nov 1995 15:54:00 -0500 Received: by arosa (940816.SGI.8.6.9/cliff's joyful mailer #2) id PAA12137(arosa); Tue, 21 Nov 1995 15:53:59 -0500 Date: Tue, 21 Nov 1995 15:53:59 -0500 From: Christian Omlin Message-Id: <199511212053.PAA12137@arosa> To: connectionists@cs.cmu.edu Subject: paper available The following paper is available on the website http://www.neci.nj.nec.com/homepages/omlin/omlin.html The paper gives an overview of our work and contains an extensive bibliography on the representation of discrete dynamical systems in recurrent neural networks. -Christian =================================================================== Learning, Representation, and Synthesis of Discrete Dynamical Systems in Continuous Recurrent Neural Networks (*) C. Lee Giles (a,b) and Christian W. Omlin (a) (a) NEC Research Institute 4 Independence Way Princeton, NJ 08540 (b) Institute for Advanced Computer Studies University of Maryland College Park, MD 20742 ABSTRACT This paper gives an overview on learning and representation of discrete-time, discrete-space dynamical systems in discrete-time, continuous-space recurrent neural networks. We limit our discussion to dynamical systems (recurrent neural networks) which can be represented as finite-state machines (e.g. discrete event systems ). In particular, we discuss how a symbolic representation of the learned states and dynamics can be extracted from trained neural networks, and how (partially) known deterministic finite-state automata (DFAs) can be encoded in recurrent networks. While the DFAs that can be learned exactly with recurrent neural networks are generally small (on the order of 20 states), there exist subclasses of DFAs with on the order of 1000 states that can be learned by small recurrent networks. However, recent work in natural language processing implies that recurrent networks can possibly learn larger state systems. (*) Appeared in Proceedings of the IEEE Workshop on Architectures for Semiotic Modeling and Situation Analysis in Large Complex Systems, Monterey, CA, August 27-29, 1995. Copyright IEEE Press. From heckerma@microsoft.com Wed Nov 22 18:10:46 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 22 Nov 95 18:10:41 -0600; AA04048 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 22 Nov 95 18:10:39 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa15648; 21 Nov 95 21:46:42 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa15646; 21 Nov 95 21:34:14 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa12333; 21 Nov 95 21:34:09 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa26653; 21 Nov 95 17:53:50 EST Received: from tide03.microsoft.com by EDRC.CMU.EDU id aa07821; 21 Nov 95 17:52:53 EST Received: by tide03.microsoft.com; id OAA28601; Tue, 21 Nov 1995 14:55:44 -0800 Received: from unknown(157.54.17.73) by tide03.microsoft.com via smap (g3.0.1) id sma028504; Tue, 21 Nov 95 14:55:18 -0800 Received: from xnet2 (xnet2.microsoft.com [157.54.17.205]) by imail1.microsoft.com (8.7.1/8.7.1) with SMTP id OAA29657 for ; Tue, 21 Nov 1995 14:54:31 -0800 (PST) Message-Id: <199511212254.OAA29657@imail1.microsoft.com> X-Received: from red-57-msg by xnet2 with receive; Tue, 21 Nov 1995 14:52:17 -0800 X-Msmail-Message-Id: 130E2641 X-Msmail-Conversation-Id: 130E2641 From: David Heckerman To: Connectionists@cs.cmu.edu Date: Tue, 21 Nov 95 14:55:14 TZ Subject: NIPS95 workshop X-Msxmtid: red-57-msg951121225149MTP[01.51.01]000000a5-101828 **** TENTATIVE SCHEDULE **** NIPS*95 Workshop Learning in Bayesian Networks and Other Graphical Models Friday and Saturday, December 1-2, 1995 Marriott Vail Mountain Resort, Colorado http://www.cs.cmu.edu/afs/cs/project/cnbc/nips/NIPS.html (conference) http://www.ai.mit.edu/people/jordan/workshop.html (workshop) Topic and Purpose of the Workshop: A Bayesian network is a directed graphical representation of probabilistic relationships that people find easy to understand and use, often because the relationships have a causal interpretation. A network for a given domain defines a joint probability distribution for that domain, and algorithms exist for efficiently manipulating the joint distribution to determine probability distributions of interest. Over the last decade, the Bayesian network has become a popular representation for encoding uncertain expert knowledge in expert systems. More recently, researchers have developed methods for learning Bayesian networks from data. These approaches will be the focus of this workshop. Issues to be discussed include (1) the opposing roles of prediction and explanation; (2) search, model selection, and capacity control; (3) representation issues, including extensions of the Bayesian network (e.g., chain graphs), the role of ``hidden'' or ``latent'' variables in learning, the modeling of temporal systems, and the assessment of priors; (4) optimization and approximation methods, including gradient-based methods, EM algorithms, stochastic sampling, and the mean field algorithms. In addition, we plan to discuss known relationships among Bayesian networks, Markov random fields, Boltzmann machines, loglinear models for contingency tables, Hidden Markov models, decision trees, and feedforward neural networks, as well as to uncover previously unknown ties. Tentative Schedule: Friday, Dec 1 ------------- 7:30am - 8:50am Tutorial on Graphical Models Ross Shachter, Stanford University Bruce D'Ambrosio, Oregon State University Michael Jordan, MIT 8:50am - 9:30am Decomposable graphical models and their use in learning algorithms Steffen Lauritzen, Aalborg University Decomposable, or triangulated, graphical models occur for example as basic computational structures in probabilistic expert systems. I will first give a brief description of properties of decomposable graphical models and then present some unfinished ideas about their possible use in automatic learning procedures. 9:30am - 9:40am break 9:40am - 10:20am Likelihoods and Priors for Learning Bayesian Networks David Heckerman, Microsoft Dan Geiger, Technion I will discuss simple methods for constructing likelihoods and parameter priors for learning about the parameters and structure of a Bayesian network. In particular, I will introduce several assumptions that permit the construction of likelihoods and parameter priors for a large number of Bayesian-network structures from a small set of assessments. Two notable assumptions are parameter independence, which says that the parameters associated with each variable in a structure are independent, and likelihood equivalence, which (roughly speaking) says that data should not help to discriminate structures that represent the same assertions of conditional independence. In addition to explicating methods for likelihood and prior construction, I will show how the assumptions lead to characterizations of well-known prior distributions for the parameters of multivariate distributions. For example, when the joint likelihood is an unrestricted discrete distribution, parameter independence and likelihood equivalence imply that the parameter prior must be a Dirichlet distribution. 10:20am - 11:00am Bayesian model averaging for Markov equivalence classes of acyclic digraphs David Madigan, Michael D. Perlman, and Chris T. Volinsky, University of Washington, Seattle Acyclic digraphs (ADGs) are widely used to describe dependencies among variables in multivariate distributions. There may, however, be many ADGs that determine the same dependence (= Markov) model. Thus, the family of all ADGs with a given set of vertices is naturally partitioned into Markov-equivalence classes, each class being associated with a unique statistical model. Statistical procedures, such as model selection or model averaging, that fail to take into account these equivalence classes, may incur substantial computational or other inefficiencies. Recent results have shown that each Markov-equivalence class is uniquely determined by a single chain graph, the essential graph, that is itself Markov-equivalent simultaneously to all ADGs in the equivalence class. Here we propose two stochastic Bayesian model averaging and selection algorithms for essential graphs and apply them to the analysis of a number of discrete- variable data sets. 11:00am - 11:40am discussion ----mid-day break---- 4:30pm - 5:10pm Automated Causal Inference Peter Spirtes, Carnegie Mellon University Directed acyclic graphs can be used to represent both families of probability distributions and causal relationships. We introduce two axioms (Markov and Faithfulness) that are widely but implicitly assumed by statisticians and that relate causal structures with families of probability distributions. We then use these axioms to develop algorithms that infer some features of causal graphs given a probability distribution and optional background knowledge as input. The algorithms are correct in the large sample limit, even when latent variables and selection bias may be present. In the worst case, the algorithms are exponential, but in many cases they has been able to handle up to 100 variables. We will also present Monte Carlo simulation results on various sample sizes. 5:10pm - 5:50pm HELMHOLTZ MACHINES Geoffrey E. Hinton, Peter Dayan, Brendan Frey, Radford Neal University of Toronto and MIT For hierarchical generative models that use distributed representations in their hidden variables, there are exponentially many ways in which the model can produce each data point. It is therefore intractable to compute the posterior distribution over the hidden distributed representations given a datapoint and so there is no obvious way to use EM or gradient methods for fitting the model to data. A Helmholtz machine consists of a generative model that uses distributed representations and a recognition model that computes an approximation to the posterior distribution over representations. The machine is trained to minimize a Helmholtz free energy which is equal to the negative log probability of the data if the recognition model computes the correct posterior distribution. If the recognition model computes a more tractable, but incorrect distribution, the Helmholtz free energy is an upper bound on the negative log probability of the data, so it acts as a tractable and useful Lyapunov function for learning a good generative model. It also encourages generative models that give rise to nice simple posterior distributions, which makes perception a lot easier. Several different methods have been developed for minimizing the Helmholtz free energy. I will focus on the "wake-sleep" algorithm, which is easy to implement with neurons, and give some examples of it learning probability density functions in high dimensional spaces. 5:50pm - 6:30pm Bounding Log Likelihoods in Sigmoid Belief Networks Lawrence K. Saul, Tommi Jaakkola, and Michael I. Jordan, MIT Sigmoid belief nets define graphical models with useful probabilistic semantics. We show how to calculate a lower bound on the log likelihood of any partial instantiation of a sigmoid belief net. The bound can be used as a basis for inference and learning provided it is sufficiently tight; in practice we have found this often to be the case. The bound is computed by approximating the true posterior distribution over uninstantiated nodes, $P$, by a more tractable distribution, $Q$. Parameterized forms for $Q$ include factorial distributions, mixture models, and hierarchical distributions that exploit the presence of tractable substructures in the original belief net. Saturday, Dec 2 --------------- 7:30am - 8:10am A Method for Learning the Structure of a Neural Network from Data Gregory F. Cooper and Sankaran Rajagopalan, University of Pittsburgh A neural network can be viewed as consisting of a set of arcs and a parameterization of those arcs. Most neural network learning has focused on parameterizing a user-specified set of arcs. Relatively less work has addressed automatically learning from data which arcs to include in the network (i.e., the neural network structure). We will present a method for learning neural network structures from data (call it LNNS). The method takes as input a database and a set of priors over possible neural network structures, and it outputs the neural network structure that is most probable as found by a heuristic search procedure. We will describe the close relationship between the LNNS method and current Bayesian methods for learning Bayesian belief networks. We also will show how we can apply the LNNS method to learn hidden nodes in Bayesian belief networks. 8:10am - 8:50am Local learning in probabilistic networks with hidden variables Stuart Russell, John Binder, Daphne Koller, Keiji Kanazawa University of California, Berkeley We show that general probabilistic (Bayesian) networks with fixed structure containing hidden variables can be learned automatically from data using a gradient-descent mechanism similar to that used in neural networks. The gradient can be computed locally from information available as a direct by-product of the normal inference process in the network. Because probabilistic networks provide explicit representations of causal structure, human experts can easily contribute prior knowledge to the training process, thereby significantly improving the sample complexity. The method can be extended to networks with intensionally represented distributions, including networks with continuous variables and dynamic probabilistic networks (DPNs). Because DPNs provide a decomposed representation of state, they may have some advantages over HMMs as a way of learning certain types of stochastic temporal processes with hidden state. 8:50am - 9:30am Asymptotic Bayes Factors for Directed Networks with Hidden Variables Dan Geiger, Technion David Heckerman, Microsoft Chris Meek, Carnegie Mellon University 9:30am - 9:40am break 9:40am - 10:20am Bayesian Estimation of Gaussian Bayes Networks Richard Scheines, Carnegie Mellon University The Gibbs sampler can be used to draw a sample from the posterior distribution over the parameters in a linear causal model ( Gaussian network, structural equation model, or LISREL model). I show how this can be done, and provide several examples that demonstrate its utility. These include: estimating under-identified models and making correct small sample inferences about the parameters when the likelihood surface is non-normal, contrary to the assumptions of the asymptotic theory that all current techniques rely on. In fact the likelihood surface for structural equation models (SEMs) with latent variables is often multi-modal. I give an example of such a case, and show how the Gibbs sampler is still informative and useful in this case in contrast to standard SEM software like LISREL. 10:20am - 11:00am Learning Stochastic Grammars Stephen M. Omohundro Bayesian networks represent the distribution of a fixed set of random variables using conditional independence information. Many important application domains (eg. speech, vision, planning, etc.) have state spaces that don't naturally decompose into a fixed set of random variables. In this talk, I'll present stochastic grammars as a tractable class of probabilistic models over this kind of domain. I'll describe an algorithm by Stolcke and myself for learning Hidden Markov Models and stochastic regular grammars from training strings which is closely related to algorithms for learning Bayesian networks. I'll discuss connections between stochastic grammars and graphical probability models and argue for the need for richer structures which encompass certain properties of both classes of model. 11:00am - 11:20am Variable selection using the theory of Bayesian networks Christopher Meek, Carnegie Mellon University This talk will describe two approaches to variable selection based upon the theory of Bayesian networks. In a study about the prediction of mortality in hospital patients with pneumonia these two methods were used to select a set of variables upon which predictive models were developed. In addition several neural network methods were used to develop predictive models from the same large database of cases. The mortality models developed in this study are distinguished more by the number of variables and parameters that they contain than by their error rates. I will offer an explanation of why the two methods described lead to small sets of variables and models with fewer parameters. In addition the variable sets selected by the Bayesian network approaches are amenable to future implementation in a paper-based form and, for several of the models, are strikingly similar to variable sets hand selected by physicians. 11:20am - 11:40am Methods for Learning Hidden Variables Joel Martin Hidden variables can be learned in many ways. Which method should be used? Some have better theoretical justifications, some seem to have better pragmatic justifications. I will describe a simple class of probabilistic models and will compare several methods for learn hidden variables from data. The methods compared are EM, gradient descent, simulated annealing, genetic search, and a variety of incremental techniques. I will discuss the results by considering particular applications. ----mid-day break---- 4:30pm - 5:10pm Brains, Nets, Feedback and Time Series Clark Glymour, Thomas Richardson and Peter Spirtes, Carnegie Mellon Neural networks have been used extensively to model the differences between normal cognitive behavior and the behavior of brain damaged subjects. A network is trained to simulate normal behavior, lesioned, and then simulates, or under re-training simulates, brain-damaged behavior. A common objection to this explanatory strategy (see for example comments on Martha Farah's recent contribution in Behavioral and Brain Sciences) is that it can "explain anything." The complaint alleges that for any mathematically possible pairing of normal and brain-damaged behavior, there exists a neural net that simulates the normal and when lesioned simulates the abnormal. Is that so? We can model the normal and abnormal behaviors as probability distributions on a set of nodes, and the network as a set of simultaneous (generally non-recursive) equations with independent noises. The network and joint probability distribution then describe a cyclic graph and associated probability distribution. What is the connection between the probability distribution and the graph topology? It is easy to show that the (local) Markov condition fails for linear networks of this sort. Spirtes (and independently J. Koster, in a forthcoming paper in Annals of Statistics) has shown that the conditional independencies implied by a linear cyclic network are characterized by d-separation (in either Pearl's or Lauritzen's versions). Spirtes has also shown that d-separation fails for non-linear cyclic networks with independent errors, but has given another characterization for the non-linear case. These results can be directly applied to the methodological disputes about brains and neural net models. Assuming faithfulness, it follows that a lesioned neural net represented as a cyclic Bayes net, whether linear or non-linear, must preserve the conditional independence relations in the original network. Hence not every pairing of normal and abnormal behavior is possible according to the neural net hypothesis as formulated. This connection between Bayes nets and work on neural models suggests that we might look for other applications of Bayesian networks in cognitive neuropsychology. We might hope to see applications of discovery methods for Bayes nets to multiple single cell recordings, or to functional MRI data. Such appplications would be aided by discovery methods for cyclic Bayes nets as models of recurrent neural nets. Several important theoretical steps have been taken towards that goal by Richardson, who has found a polynomial time decision procedure for the equivlaence of linear cyclic graphs, and a polynomial (in sparse graphs) time, asymptotically correct and complete procedure for discovering equivalence classes of linear cyclic graphs (without latent variables). Research is under way on search procedures that parallel the FCI procedure of Spirtes Glymour and Scheines (1993) in allowing latent variables. The questions of equivalence and discovery for non-linear systems are relatively untouched. It may be that the proper way to model a recurrent neural net is not by a cyclic graphical model, but by a time series. That suggestion raises the question of the connections between time-series and cyclic graphical models, a matter under investigation by Richardson. Results in this area would have implications for econometrics as well as for neuropsychology, since the econometric tradition has treated feedback systems in both ways, by simultaneous linear equations and by time series, without fully characterizing the relations between the representations. While there are situations in which equilibrium models, such as cyclic graphical models appear applicable, these models, since they are not dynamic, make no predictions about the dynamic behavior of a system (time series) if it is pushed out of equilibrium. 5:10pm - 5:50pm Compiling Probabilistic Networks and Some Questions this Poses Wray Buntine Probabilistic networks (or similar) provide a high-level language that can be used as the input to a compiler for generating a learning or inference algorithm. Example compilers are BUGS (inputs a Bayes net with plates) by Gilks, Spiegelhalter, et al., and MultiClass (inputs a dataflow graph) by Roy. This talk will cover three parts: (1) an outline of the arguments for such compilers for probabilistic networks, (2) an introduction to some compilation techniques, and (3) the presentation of some theoretical challenges that compilation poses. High-level language compilers are usually justified as a rapid prototyping tool. In learning, rapid prototyping arises for the following reasons: good priors for complex networks are not obvious and experimentation can be required to understand them; several algorithms may suggest themselves and experimentation is required for comparative evaluation. These and other justifications will be described in the context of some current research on learning probabilistic networks, and past research on learning classification trees and feed-forward neural networks. Techniques for compilation include the data flow graph, automatic differentiation, Monte Carlo Markov Chain samplers of various kinds, and the generation of C code for certain exact inference tasks. With this background, I will then pose a number of important research questions to the audience. 5:50pm - 6:30pm discussion Organizers: Wray Buntine Greg Cooper Dan Geiger Clark Glymour David Heckerman Geoffry Hinton Mike Jordan Steffen Lauritzen David Madigan Radford Neal Steve Omohundro Judea Pearl Stuart Russell Richard Scheines Peter Spirtes From juergen@idsia.ch Wed Nov 22 18:10:48 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 22 Nov 95 18:10:45 -0600; AA04052 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 22 Nov 95 18:10:41 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa16460; 22 Nov 95 4:48:16 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa16458; 22 Nov 95 4:35:53 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa12934; 22 Nov 95 4:35:36 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa05555; 22 Nov 95 4:33:18 EST Received: from chsun.eunet.ch by CS.CMU.EDU id aa23640; 22 Nov 95 4:32:42 EST Received: from fava.idsia.ch by chsun.eunet.ch (8.6.10/1.34) id KAA07540; Wed, 22 Nov 1995 10:32:18 +0100 Received: from kraut.idsia.ch by fava.idsia.ch (4.1/SMI-4.1) id AA22891; Wed, 22 Nov 95 10:32:17 +0100 Date: Wed, 22 Nov 95 10:32:17 +0100 From: Juergen Schmidhuber Message-Id: <9511220932.AA22891@fava.idsia.ch> To: Connectionists@cs.cmu.edu Subject: compressibility, Kolmogorov, learning In response to Barak's and David's recent messages: David writes: >>> To illustrate just one of the possible objections to measuring randomness with Kolmogorov complexity: Would you say that a macroscopic gas with a specified temperature is "random"? To describe it exactly takes a huge Kolmogorov complexity. And certainly in many regards its position in phase space is "nothing but noise". (Indeed, in a formal sense, its position is a random sample of the Boltzmann distribution.) Yet Physicists can (and do) make extraordinarilly accurate predictions about such creatures with ease. <<< 1. Is real gas random in the Kolmogorov sense? At least the idealized microscopic gas models we are studying are not random. For simplicity, let us consider a typical discrete time gas model. Each system state can be computed by a short algorithm, given the previous state. This allows for enormous compressibility of the system history, even if the initial state had high complexity (and even more so if the initial state was simple). Even if we assume that the deterministic model is corrupted by random processes, we won't end up with a system history with maximal Kolmogorov complexity: for instance, using standard compression techiques, we can provide short codes for likely next states and long codes for unlikely next states. The only requirement is that the random processes are not *completely* random. But in the real world they are not, as can be deduced from macroscopic gas properties (considering only abstract properties of the state, such as temperature and pressure) --- as David indicated, there are simple algorithms for predicting next macroscopic states from previous macroscopic states. In case of true randomness, this would not be the case. 2. Only where there is compressibility, there is room for non-trivial learning and generalization. Unfortunately, almost all possible histories of possible universes are random and incompressible. There is no miraculous universal learning algorithm for arbitrary universes (that's more or less the realm of NFL). As has been observed repeatedly, however, our own universe appears to be one of the relatively few (but still infinitely many) compressible ones (every electron behaves the same way, etc.). In fact, much of the previous work on machine learning can be thought of exploiting compressibility: ``chunking'', for instance, exploits the possibility of re-using subprograms. Methods for finding factorial (statistically non-redundant) codes of image data exploit the enormous redundancy and compressibility of visual inputs. Similarly for ``learning by analogy'' etc. In the context of PAC learning, Ming Li and Paul Vitanyi address related issues in an interesting paper from 1989: A theory of Learning Simple Concepts Under Simple Distributions and Average Case Complexity for the Universal Distribution, Proc. 30th American IEEE Symposium on Foundations of Computer Science, pages 34-39. 3. An intriguing possibility is: there may be something like a universal learning algorithm for compressible, low-complexity universes. I am the first to admit, however, that neither the concept of Kolmogorov complexity by itself nor the universal Solomonoff-Levin distribution provide all the necessary ingredients. Clearly, a hypothetical universal learning algorithm would have to take into account the fact that computational resources are limited. This is driving much of the current work at our lab. Juergen Schmidhuber IDSIA From tds@ai.mit.edu Fri Nov 24 15:17:26 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 24 Nov 95 15:17:24 -0600; AA03506 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 24 Nov 95 15:17:20 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa17418; 22 Nov 95 17:10:12 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa17416; 22 Nov 95 16:54:29 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa13299; 22 Nov 95 16:54:04 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa08292; 22 Nov 95 9:44:16 EST Received: from ai.mit.edu by RI.CMU.EDU id aa19468; 22 Nov 95 9:43:49 EST Received: from dentate.ai.mit.edu (DENTATE.MIT.EDU) by life.ai.mit.edu (4.1/AI-4.10) for connectionists@cs.cmu.edu id AA10580; Wed, 22 Nov 95 09:43:45 EST From: "Terence D. Sanger" Received: by dentate.ai.mit.edu (4.1/AI-4.10) id AA08557; Wed, 22 Nov 95 09:43:42 EST Date: Wed, 22 Nov 95 09:43:42 EST Message-Id: <9511221443.AA08557@dentate.ai.mit.edu> To: connectionists@cs.cmu.edu Subject: off on a tangent... David Wolpert writes: > To illustrate just one of the possible objections to measuring > randomness with Kolmogorov complexity: Would you say that a > macroscopic gas with a specified temperature is "random"? To describe > it exactly takes a huge Kolmogorov complexity. And certainly in many > regards its position in phase space is "nothing but noise". (Indeed, > in a formal sense, its position is a random sample of the Boltzmann > distribution.) Yet Physicists can (and do) make extraordinarilly > accurate predictions about such creatures with ease. In thinking about this, it seems that David is right: the gas is, in some important sense, highly structured. In particular, its *statistics* are stationary no matter how they are sampled. This means that a temperature sample from any part of the gas will predict temperatures in other parts of the gas, according to the law of large numbers. Consider a different statistical model: Choose a random number to be the mean of a distribution on a finite set of random variables. Divide the set in half, choose a new random number and add it to the left half's mean and subtract it from the right half's mean. Divide the half-sets in half, choose two new random numbers, and continue to split each half by adding and subtracting random numbers until no more splits are possible. Now we have: 1) All random variables are independent and identically distributed (this is basically a random walk). 2) The mean of the distribution is equal to the first random number chosen (at each step, the mean does not change). 3) The mean of any "binary" region (half, quarter, eighth, etc.) is a poor predictor of the means of neighboring regions. 4) Sample means do not converge uniformly to the ensemble mean, unless samples are chosen randomly across region boundaries. In some sense, this is a very highly structured field of random variables. Yet prediction is much harder than for the random gas. (In case anyone is interested, this distribution arises as a model for genetic diseases of mitochondria. If a cell has N mitochondria, M of which are defective, then at cell division it will pass a random number of normal and defective mitochondria to each daughter cell, where the total number of defective ones passed on is conserved and is equal to 2M. The daughter cells, in turn, will do the same thing. The problem is that a local biopsy to count the number of defective mitochondria will not predict biopsy results from other sites.) Terry Sanger tds@ai.mit.edu From kak@gate.ee.lsu.edu Fri Nov 24 15:17:29 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 24 Nov 95 15:17:26 -0600; AA03512 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 24 Nov 95 15:17:24 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa17470; 22 Nov 95 17:23:39 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa17421; 22 Nov 95 16:56:06 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa13326; 22 Nov 95 16:55:31 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa10890; 22 Nov 95 11:56:47 EST Received: from gate.ee.lsu.edu by CS.CMU.EDU id aa26110; 22 Nov 95 11:55:56 EST Received: by gate.ee.lsu.edu (4.1/SMI-4.1) id AA22783; Wed, 22 Nov 95 10:55:45 CST Date: Wed, 22 Nov 95 10:55:45 CST From: Subhash Kak Message-Id: <9511221655.AA22783@gate.ee.lsu.edu> To: connectionists@cs.cmu.edu Subject: Paper The following paper ON GENERALIZATION BY NEURAL NETWORKS by Subhash C. Kak Abstract: We report new results on the corner classification approach to training feedforward neural networks. It is shown that a prescriptive learning procedure where the weights are simply read off based on the training data can provide adequate generalization. The paper also deals with the relations between the number of separable regions and the size of the training set for a binary data network. was recently presented at the Joint Conference on Information Science. You may ftp the paper at the following address: ftp://gate.ee.lsu.edu/pub/kak/gen.ps From geoff@salk.edu Fri Nov 24 15:17:32 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 24 Nov 95 15:17:28 -0600; AA03517 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 24 Nov 95 15:17:25 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa17730; 22 Nov 95 19:32:44 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa17728; 22 Nov 95 19:26:15 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa13417; 22 Nov 95 19:25:39 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa12750; 22 Nov 95 13:18:55 EST Received: from helmholtz.salk.edu by CS.CMU.EDU id aa26852; 22 Nov 95 13:18:22 EST Received: from gauss.sdsc.edu (gauss.salk.edu) by salk.edu (4.1/SMI-4.1) id AA23843; Wed, 22 Nov 95 10:18:18 PST Date: Wed, 22 Nov 95 10:18:18 PST From: Geoff Goodhill Message-Id: <9511221818.AA23843@salk.edu> To: connectionists@cs.cmu.edu Subject: Topographic Mappings - Tech Report available The following paper is available via ftp://salk.edu/pub/geoff/goodhill_finch_sejnowski_tech95.ps.Z or http://cnl.salk.edu/~geoff QUANTIFYING NEIGHBOURHOOD PRESERVATION IN TOPOGRAPHIC MAPPINGS Geoffrey J. Goodhill(1), Steven Finch(2) & Terrence J. Sejnowski(3) (1) The Salk Institute for Biological Studies 10010 North Torrey Pines Road, La Jolla, CA 92037, USA (2) Human Communication Research Centre University of Edinburgh, 2 Buccleuch Place Edinburgh EH8 9LW, GREAT BRITAIN (3) The Howard Hughes Medical Institute The Salk Institute for Biological Studies 10010 North Torrey Pines Road, La Jolla, CA 92037, USA & Department of Biology University of California San Diego, La Jolla, CA 92037, USA, Institute for Neural Computation Technical Report Series INC-9505, November 1995 ABSTRACT Mappings that preserve neighbourhood relationships are relevant in both practical and biological contexts. It is important to be clear about precisely what preserving neighbourhoods could mean. We give a definition of a ``perfectly neighbourhood preserving'' map, which we call a topographic homeomorphism, and prove that this has certain desirable properties. When a topographic homeomorphism does not exist (the usual case), many choices are available for quantifying the quality of a map. We introduce a particular measure, C, which has the form of a quadratic assignment problem. We also discuss other measures that have been proposed, some of which are related to C. A comparison of seven measures applied to the same simple mapping problem reveals interesting similarities and differences between the measures, and challenges common intuitions as to what constitutes a ``good'' map. 17 pages, uncompressed postscript = 154K From scott@cpl_mmag.nhrc.navy.mil Fri Nov 24 15:17:34 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 24 Nov 95 15:17:30 -0600; AA03520 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 24 Nov 95 15:17:27 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa17738; 22 Nov 95 19:38:57 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa17733; 22 Nov 95 19:27:06 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa13425; 22 Nov 95 19:26:16 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa15106; 22 Nov 95 15:07:50 EST Received: from cpl_mma.nhrc.navy.mil by RI.CMU.EDU id aa20968; 22 Nov 95 15:07:14 EST Received: by cpl_mmag.nhrc.navy.mil (5.61/) id AA24212 for delivery to Connectionists@CS.CMU.EDU; Wed, 22 Nov 95 11:37:41 -0800 Date: Wed, 22 Nov 95 11:37:41 -0800 From: Scott Makeig Message-Id: <9511221937.AA24212@cpl_mmag.nhrc.navy.mil> To: Connectionists@cs.cmu.edu Subject: Online-NIPS post-NIPS workshop: programme ****** PROGRAMME ****** NIPS*95 Workshop on ONLINE NEURAL INFORMATION PROCESSING SYSTEMS: Prospects for Neural Human-Machine Interfaces Date: Saturday, Dec. 2, 1995 Place: NIPS*95 Workshops, Vail, Colorado www: http://128.49.52.9/~www/nips.html Organizer: Scott Makeig (NHRC/UCSD) scott@salk.edu There is rapidly growing interest in the development of intelligent interfaces in which operator state information derived from psycho- physiological and/or video-based measures of the human operator is used directly to inform, interact with, or control computer-based systems. Adequate signal processing power is now available at reasonable cost to implement in near-real time a wide range of spectral, neural network, and dynamic systems algorithms for extracting information about psychological state or intent from multidimensional EEG signals, video images of the eyes and face, and other psychophysiological and/or behavioral data. This NIPS*95 conference workshop will give an opportunity for interested researchers from signal processing, neuroscience, neural networks, cognitive science, and computer design to discuss near- and medium-term prospects for, and obstacles to, practical neural human-systems interfaces (NHSI) technology for monitoring cognitive state and for using operator state information to give operator feedback, control adaptive automation or perform brain-actuated control. Aspects of cognitive state that might be monitored using NHSI technology include alertness, perception, attention, workload, intention and emotion: Programme (Saturday, Dec. 2, Vail Marriott): 7:30 Scott Makeig (NHRC/UCSD) NHSI Overview Sandy Pentland (MIT Media Lab) Video-based human-computer interaction Alan Gevins (EEG Systems Labs) EEG-based cognitive monitoring Discussion 8:30 Babak A. Taheri (SRI International) Active EEG electrode technology Tzyy-Ping Jung (Salk Institute) EEG-based alertness monitoring Magnus Stensmo (Salk Institute) Monitoring alertness via eye closures General Discussion 9:30 [free time] 12:30 Lunch (optional) 1:30 [free time] 4:30 Georg Dorffner (ANNDEE project group, Austria) Brain-actuated control Grant McMillan (Wright-Patterson Air Force Base ) Brain-actuated control Andrew Junker (CyberLink) EMG/EEG-actuated control Discussion 5:30 Curtis Padgett (UCSD) Video-based emotion monitoring Jose Principe (University of Florida) EEG-based communication Charles W. Anderson (Colorado State University) Mental task monitoring General Discussion The workshop will review ongoing progress and challenges in computational and technology areas, and discuss prospects for short- and medium-term implementations. Abstracts are available at: http://128.49.52.9/~www/nips.html NIPS*95 conference and post-conference workshop information: http://www.cs.cmu.edu/Web/Groups/NIPS/nips95.html or via ftp/email from: psyche.mit.edu in /pub/NIPS95 nips95@mines.colorado.edu From stavrosz@med.auth.gr Fri Nov 24 15:17:35 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 24 Nov 95 15:17:32 -0600; AA03526 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 24 Nov 95 15:17:29 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab17738; 22 Nov 95 19:40:09 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab17733; 22 Nov 95 19:27:08 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa13430; 22 Nov 95 19:26:39 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa20424; 22 Nov 95 18:23:31 EST Received: from antigoni.med.auth.gr by CS.CMU.EDU id aa29602; 22 Nov 95 18:22:49 EST Received: by antigoni.med.auth.gr (5.x/MED-AUTH-1.0) id AA09955; Thu, 23 Nov 1995 01:06:26 +0200 Organization: Medical School, Aristotle University of Thessaloniki Thessaloniki, Macedonia, Greece Date: Thu, 23 Nov 1995 01:06:25 +0200 (EET) From: Stavros Zanos To: connectionists@cs.cmu.edu Subject: Paper on LTP and Learning Algorithms In-Reply-To: Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII (Neural Nets: Foundations to Applications) The following paper is now available to anyone who sends a request at the following adress (use the word "reqLTP3" at the subject field): stavrosz@antigoni.med.auth.gr ********* AU: Zanos Stavros, 3rd year medical student AT: University of Thessaloniki School of Medicine Thessaloniki, Greece TI: Quantal Analysis of Hippocampal Long-Term Synaptic Potentiation , and Application to the Design of Biologically Plausible Learning Algorithms for Artificial Neural Networks AB: Quantal analysis (QA) of synaptic function has been used to examine whether the expression of long-term potentiation (LTP) in central synapses is mediated by a pre- or postsynaptic mechanism. However, it can also be used as a physiological model of synaptic transmission and plasticity; use of physiological models in network simulations provides reasonably accurate approximates of various biological parameters in a computationally efficient manner. We describe a stochastic algorithm of synaptic transmission and plasticity based on QA data from CA1 hippocampus LTP experiments. We also describe the application of such an algorithm in a typical CA1-region simulation (a simple self-organizing competitive matrix), and discuss the possible benefits of using noisy network elements (in this case, "synapses"). We show that the fluctuations in postsynaptic responses under constant static synaptic weights introduced by such an algorithm increase the storing capacity and the ability of the network to orthogonalize input vectors. A decrease in the number of required iterations for every learned input vector is also reported. Finally we examine the issue of a hypothetical "computational equivalence" of different optimization techniques when applied to similar problems, often met in the literature, since our simulation studies suggest that even small differences in the learning algorithms used could provide the network with a kind of "preference" to specific patterns of performance. ********* The above paper will appear at the 2nd European Conference of Medical Students (May 96), and it has been edited using MS Word-7 (for Win95). Those who adressed a request will receive the paper through email as an attachment compressed file. Detailed mathematical formalizations used in the simulations are available upon request. We welcome questions and/or remarks. Zanos Stavros Aristotle University of Thessaloniki School of Life Sciences, Faculty of Medicine From ken@phy.ucsf.edu Fri Nov 24 15:17:38 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 24 Nov 95 15:17:35 -0600; AA03536 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 24 Nov 95 15:17:32 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa17865; 22 Nov 95 22:14:32 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa17863; 22 Nov 95 22:07:02 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa13572; 22 Nov 95 22:06:39 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa22902; 22 Nov 95 20:44:56 EST Received: from coltrane.ucsf.EDU by CS.CMU.EDU id aa00393; 22 Nov 95 20:44:39 EST Received: by coltrane.ucsf.edu (5.65/GSC4.24) id AA04384 for Connectionists@CS.CMU.EDU; Wed, 22 Nov 1995 17:44:34 -0800 Date: Wed, 22 Nov 1995 17:44:34 -0800 From: Ken Miller Message-Id: <9511230144.AA04384@coltrane.ucsf.edu> To: Connectionists@cs.cmu.edu, cneuro@smaug.bbb.caltech.edu Subject: Postdoctoral and Predoctoral Positions in Theoretical Neurobiology POSTDOCTORAL AND PREDOCTORAL POSITIONS SLOAN CENTER FOR THEORETICAL NEUROBIOLOGY UNIVERSITY OF CALIFORNIA, SAN FRANCISCO INFORMATION ON THE UCSF SLOAN CENTER AND FACULTY AND THE POSTDOCTORAL AND PREDOCTORAL POSITIONS IS AVAILABLE THROUGH OUR WWW SITE: http://keck.ucsf.edu/sloan. E-mail inquiries should be sent to sloan-info@phy.ucsf.edu. Below is basic information on the program: The Sloan Center for Theoretical Neurobiology at UCSF solicits applications for pre- and post-doctoral fellowships, with the goal of bringing theoretical approaches to bear on neuroscience. Applicants should have a strong background and education in a theoretical discipline, such as physics, mathematics, or computer science, and commitment to a future research career in neuroscience. Prior biological or neuroscience training is not required. The Sloan Center will offer opportunities to combine theoretical and experimental approaches to understanding the operation of the intact brain. The research undertaken by the trainees may be theoretical, experimental, or a combination. The RESIDENT FACULTY of the Sloan Center and their research interests are: Allison Doupe: Development of song recognition and production in songbirds. Stephen Lisberger: Learning and memory in a simple motor reflex, the vestibulo-ocular reflex, and visual guidance of smooth pursuit eye movements by the cerebral cortex. Michael Merzenich: Experience-dependent plasticity underlying learning in the adult cerebral cortex and the neurological bases of learning disabilities in children. Kenneth Miller: Mechanisms of self-organization of the cerebral cortex; circuitry and computational mechanisms underlying cortical function; computational neuroscience. Roger Nicoll: Synaptic and cellular mechanisms of learning and memory in the hippocampus. Christoph Schreiner: Cortical mechanisms of perception of complex sounds such as speech in adults, and plasticity of speech recognition in children and adults. Michael Stryker: Mechanisms that guide development of the visual cortex. All of these resident faculty are members of UCSF's W.M. Keck Foundation Center for Integrative Neuroscience, a new center (opened January, 1994) for systems neuroscience that includes extensive shared research resources within a newly renovated space designed to promote interaction and collaboration. The unusually collaborative and interactive nature of the Keck Center will facilitate the training of theorists in a variety of approaches to systems neuroscience. In addition to the resident faculty, there are a series of VISITING FACULTY who are in residence at UCSF for times ranging from 1-8 weeks each year. These faculty, and their research interests, include: Laurence Abbott, Brandeis University: Neural coding, relations between firing rate models and biophysical models, self-organization at the cellular level William Bialek, NEC Research Institute: Physical limits to sensory signal processing, reliability and information capacity in neural coding; Sebastian Seung, ATT Bell Labs: models of collective computation in neural systems; David Sparks, University of Pennsylvania: understanding the superior colliculus as a "model cortex" that guides eye movements; Steven Zucker, McGill University: Neurally based models of vision, visual psychophysics, mathematical characterization of neuroanatomical complexity. PREDOCTORAL applicants seeking to BEGIN a Ph.D. program should apply directly to the UCSF Neuroscience Ph.D. program. Contact Patricia Arrandale, patricia@phy.ucsf.edu, to obtain application materials. THE APPLICATION DEADLINE IS Jan. 5, 1996. Also send a letter to Steve Lisberger (address below) indicating that you are applying to the UCSF Neuroscience program with a desire to join the Sloan Center. POSTDOCTORAL applicants, or PREDOCTORAL applicants seeking to do research at the Sloan Center as part of a Ph.D. program in progress in a theoretical discipline elsewhere, should apply as follows: Send a curriculum vitae, a statement of previous research and research goals, up to three relevant publications, and have two letters of recommendation sent to us. THE APPLICATION DEADLINE IS February 1, 1996. UC San Francisco is an Equal Opportunity Employer. Send applications to: Steve Lisberger Sloan Center for Theoretical Neurobiology at UCSF Department of Physiology University of California 513 Parnassus Ave. San Francisco, CA 94143-0444 From radford@cs.toronto.edu Fri Nov 24 15:17:40 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 24 Nov 95 15:17:37 -0600; AA03542 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 24 Nov 95 15:17:35 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa17871; 22 Nov 95 22:22:01 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa17867; 22 Nov 95 22:07:52 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa13578; 22 Nov 95 22:07:00 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa23376; 22 Nov 95 21:18:58 EST Received: from yonge.cs.toronto.edu by EDRC.CMU.EDU id aa13212; 22 Nov 95 21:18:25 EST Received: from neuron.ai.toronto.edu ([128.100.3.14]) by yonge.cs.toronto.edu with SMTP id <86589>; Wed, 22 Nov 1995 21:18:18 -0500 Received: by neuron.ai.toronto.edu id <1585>; Wed, 22 Nov 1995 21:18:07 -0500 From: Radford Neal To: connectionists@cs.cmu.edu Subject: Performance evaluations: request for comments Message-Id: <95Nov22.211807edt.1585@neuron.ai.toronto.edu> Date: Wed, 22 Nov 1995 21:18:03 -0500 Announcing a draft document on ASSESSING LEARNING PROCEDURES USING DELVE The DELVE development group, University of Toronto http://www.cs.utoronto.ca/neuron/delve/delve.html The DELVE development group requests comments on the draft manual for the DELVE environment from researchers who are interested in how to assess the performance of learning procedures. This manual is available via the DELVE homepage, at the URL above. Carl Rasmussen and Geoffrey Hinton will be talking about the DELVE environment at the NIPS workshop on Benchmarking of Neural Net Learning Algorithms. We would be pleased to hear any comments that attendees of this workshop, or other interested researchers, might have on the current design of the DELVE environment, as described in this draft manual. Here is the introduction to the DELVE manual: DELVE --- Data for Evaluating Learning in Valid Experiments --- is a collection of datasets from many sources, and an environment within which this data can be used to assess the performance of procedures that learn relationships using such data. Many procedures for learning from empirical data have been developed by researchers in statistics, pattern recognition, artificial intelligence, neural networks, and other fields. Learning procedures in common use include simple linear models, nearest neighbor methods, decision trees, multilayer perceptron networks, and many others of varying degrees of complexity. Comparing the performance of these learning procedures in realistic contexts is a surprisingly difficult task, requiring both an extensive collection of real-world data, and a carefully-designed scheme for performing experiments. The aim of DELVE is to help researchers and potential users to assess learning procedures in a way which is relevant to real-world problems and which allows for statistically-valid comparisons of different procedures. Improved assessments will make it easier to determine which learning procedures work best for various applications, and will promote the development of better learning procedures by allowing researchers to easily determine how the performance of a new procedure compares to that of existing procedures. This manual describes the DELVE environment in detail. First, however, we provide an overview of DELVE's capabilities, describe briefly how DELVE organizes datasets and learning tasks, and give an example of how DELVE can be used to assess the performance of a learning procedure. --------------------------------------------------------------------------- Members of the DELVE Development Group: G. E. Hinton R. M. Neal R. Tibshirani M. Revow C. E. Rasmussen D. van Camp R. Kustra Z. Ghahramani ---------------------------------------------------------------------------- Radford M. Neal radford@cs.toronto.edu Dept. of Statistics and Dept. of Computer Science radford@utstat.toronto.edu University of Toronto http://www.cs.toronto.edu/~radford ---------------------------------------------------------------------------- From lemm@LORENTZ.UNI-MUENSTER.DE Fri Nov 24 15:17:47 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 24 Nov 95 15:17:43 -0600; AA03548 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 24 Nov 95 15:17:38 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa19217; 23 Nov 95 16:42:43 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa19215; 23 Nov 95 16:29:46 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa14540; 23 Nov 95 16:29:20 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa03083; 23 Nov 95 7:31:15 EST Received: from LORENTZ.UNI-MUENSTER.DE by EDRC.CMU.EDU id aa14639; 23 Nov 95 7:27:16 EST Received: by xtp141.uni-muenster.de (5.65/DEC-Ultrix/4.3) id AA11633; Thu, 23 Nov 1995 13:26:30 +0100 Date: Thu, 23 Nov 1995 13:26:30 +0100 From: Joerg_Lemm Message-Id: <9511231226.AA11633@xtp141.uni-muenster.de> To: Connectionists@cs.cmu.edu Subject: NFL for NFL I would like to make a comment to the NFL discussion from my point of view. (Thanks to David Wolpert for remaining active in this discussion he initiated.) 1.) If there is no relation between the function values on the test and training set (i.e. P(f(x_j)=y|Data) equal to the unconditional P(f(x_j)=y) ), then, having only training examples y_i = f(x_i) (=data) from a given function, it is clear that I cannot learn anything about values of the function at different arguments, (i.e. for f(x_j), with x_j not equal to any x_i = nonoverlapping test set). 2.) We are considering two of those (influence) relations P(f(x_j)=y|Data): one, named A, for the true nature (=target) and one, named B, for our model under study (=generalizer). Let P(A and B) be the joint probability distribution for the influence relations for target and generalizer. 3.) Of course, we do not know P(A and B), but in good old Bayesian tradition, we can construct a (hyper-)prior P(C) over the family of probability distributions of the joint distributions C = P(A and B). 4.) NFL now uses the very special prior assumption P(A and B) = P(A)P(B), or equivalently P(B|A)=P(B), which means NFL postulates that there is (on average) no relation between nature and model. No wonder that (averaging over targets P(A) or over generalizers P(B) ) cross-validation works as well (or as bad) as anti-cross-validation or anything else in such cases. 5.) But target and generalizer live on the same planet (sharing the same laws, environment, history and maybe even building blocks) so we have very good reasons to assume a bias for (hyper-)priors towards correlated P(A and B) not equal to the uncorrelated product P(A)P(B)! But that's not all: We do have information which is not of the form y=f(x). We know that the probability for many relations in nature to be continuous on certain scales seems to be high (->regularization). We can have additional information about other properties of the function, e.g.symmetries, compare Abu-Mostafa's concept of hints, which produces correlations between A (=target) and B (=model). In this sense I aggree with David Wolpert on looking >>> how to characterize the needed relationship between the set of generalizers and the prior that allows cross-validation to work. >>> To summarize it in a provocative way: There is no free lunch for NFL: Only if you assume that no relation between target and model exists, then you don't find a relation between target and model! And to be precise, I say that it is rational to believe (and David does so too, I think) that in real life cross-validation works better in more cases than anti-cross-validation. Joerg Lemm From jagota@ponder.csci.unt.edu Fri Nov 24 15:17:49 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 24 Nov 95 15:17:45 -0600; AA03550 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 24 Nov 95 15:17:42 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab19217; 23 Nov 95 16:44:04 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab19215; 23 Nov 95 16:29:48 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa14545; 23 Nov 95 16:29:33 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa06693; 23 Nov 95 15:23:43 EST Received: from ponder.csci.unt.edu by RI.CMU.EDU id aa24457; 23 Nov 95 15:21:53 EST Received: by ponder (5.61/1.36) id AA04987; Thu, 23 Nov 95 14:22:51 -0600 Date: Thu, 23 Nov 95 14:22:51 -0600 From: Jagota Arun Kumar Message-Id: <9511232022.AA04987@ponder> To: Connectionists@cs.cmu.edu, juergen@idsia.ch Subject: Re: compressibility, Kolmogorov, learning A short addition to the ongoing discussion thread of Juergen, Barak, and David: The following short paper is what intrigued us (K. Regan and I) to experimentally investigate the performance of algorithms on random versus compressible (i.e., structured) instances of the maximum clique optimization problem. @article{LiV92, author = "Li, M. and P.M.B. Vitanyi", title = "Average case complexity under the universal distribution equals worst-case complexity", pages = "145--149", journal = "Information Processing Letters", volume = 42, month = "May", year = 1992 } In other words, roughly speaking, if one samples uniformly from the universal distribution, one exhibits worst-case behavior of an algorithm. I announced our (experimental, max-clique) paper on Connectionists a few months back, so won't do it again. However, this will be the subject of my talk at the NIPS workshop on optimization (7:30--8:00, Dec 1, Vail). Stop by and jump all over me. Arun Jagota From nin@cns.brown.edu Sat Nov 25 18:04:14 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 25 Nov 95 18:04:12 -0600; AA13261 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 25 Nov 95 18:04:10 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa23070; 25 Nov 95 15:27:14 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa23068; 25 Nov 95 15:14:24 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa16387; 25 Nov 95 15:13:54 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa01528; 25 Nov 95 10:43:42 EST Received: from cns.brown.edu by EDRC.CMU.EDU id aa20733; 25 Nov 95 10:43:02 EST Received: by cns.brown.edu (4.1/1.00) id AA28777; Sat, 25 Nov 95 10:41:21 EST Date: Sat, 25 Nov 95 10:41:21 EST From: Nathan Intrator Message-Id: <9511251541.AA28777@cns.brown.edu> To: Connectionists@cs.cmu.edu Subject: New preprint: multi-task training Cc: Shimon Edelman , nin@cns.brown.edu Making a low-dimensional representation suitable for diverse tasks Nathan Intrator Shimon Edelman Tel-Aviv U. Weizmann Ins. We introduce a new approach to the training of classifiers for performance on multiple tasks. The proposed hybrid training method leads to improved generalization via a better low-dimensional representation of the problem space. The quality of the representation is assessed by embedding it in a 2D space using multidimensional scaling, allowing a direct visualization of the results. The performance of the approach is demonstrated on a highly nonlinear image classification task. The paper will be described at the comming Transfer workshop at NIPS url=ftp://cns.brown.edu/nin/papers/mds.ps.Z or it can be accessed through our hope pages: http://www.wisdom.weizmann.ac.il/~edelman/shimon.html http://www.physics.brown.edu/~nin Comments are most welcome. - Nathan Intrator