From black@signal.dra.hmg.gb Mon Nov 13 00:06:19 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 13 Nov 95 00:06:17 -0600; AA25738 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 13 Nov 95 00:06:15 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa28810; 12 Nov 95 23:05:02 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa28808; 12 Nov 95 22:43:37 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa08421; 12 Nov 95 22:43:23 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa00818; 10 Nov 95 12:01:33 EST Received: from signal.dra.hmg.gb by RI.CMU.EDU id aa01639; 10 Nov 95 11:59:30 EST Received: from byron.dra.hmg.gb by signal.dra.hmg.gb (8.6.12) id QAA07781; Fri, 10 Nov 1995 16:59:23 GMT Received: from ovid.dra.hmg.gb by byron.dra.hmg.gb (8.7.1) id QAA05015; Fri, 10 Nov 1995 16:59:22 GMT Received: from localhost by ovid.dra.hmg.gb (8.7.1) id QAA03335; Fri, 10 Nov 1995 16:59:22 GMT Message-Id: To: Connectionists@cs.cmu.edu Subject: WWW page for the Pattern and Information Processing Group at DRA (UK) Date: Fri, 10 Nov 95 16:59:22 +0000 From: "John V. Black" X-Mts: smtp Dear Connectionits, Announcing a new WWW home page that covers the work of the Pattern and Information Processing Group of the Defence Research Agency (DRA) in the United Kingdom. The group consists of some 18 people, who together have a wide experience of pattern and information processing techniques and problems, which include: Analogue Systems for Information Processing Bayesian Methods Classification, Identification and Recognition Data and Information Fusion Data Analysis and Exploration Decision and Game Theory Information Processing Architectures Neural Network Techniques and Architectures Radar (Array) Signal Processing Self-Organising Systems Sensor Signal Processing Applications Statistical Pattern Processing Time-Series Analysis Tracking Uncertainty Handling (Bayesian Networks, Fuzzy Logic) The URL of the home page is http://www.dra.hmg.gb/cis5pip/Welcome.html John Black (black@signal.dra.hmg.gb) From pazzani@super-pan.ICS.UCI.EDU Mon Nov 13 23:38:54 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 13 Nov 95 23:38:50 -0600; AA15732 Received: from paris.ics.uci.edu by lucy.cs.wisc.edu; Mon, 13 Nov 95 23:38:39 -0600 Received: from super-pan.ics.uci.edu by paris.ics.uci.edu id aa18029; 13 Nov 95 18:28 PST To: ML-LIST:; Subject: Machine Learning List: Vol. 7, No. 19 Reply-To: ml@ics.uci.edu Date: Mon, 13 Nov 1995 18:20:14 -0800 From: Michael Pazzani Message-Id: <9511131828.aa18029@paris.ics.uci.edu> Machine Learning List: Vol. 7, No. 19 Monday, November 13, 1995 Contents: AISTATS-97: Preliminary Announcement AIJ on Sci Disc: new deadline CFP for ISIS: Information, Statistics and Induction in Science Machine learning program for PC/Windows Abduction and Induction Workshop Postdoctoral Fellowships at the Santa Fe Institute Baldwin Effect Bibliography textbook on machine learning Call For Papers: International Colloquium on Grammatical Inference ICNN'96 - Important Extension PhD thesis: On Growing Better Decision Trees from Data GP and Hill Climbing Comparison CfP: Workshop on Neural Network Applications in Agriculture Faculty Position at Indiana University The Machine Learning List is moderated. Contributions should be relevant to the scientific study of machine learning. Mail contributions to ml@ics.uci.edu. Mail requests to be added or deleted to ml-request@ics.uci.edu. Back issues may be FTP'd from ics.uci.edu in pub/ml-list/V/ or N.Z where X and N are the volume and number of the issue; ID: anonymous PASSWORD: URL- http://www.ics.uci.edu/AI/ML/Machine-Learning.html ---------------------------------------------------------------------- Date: Tue, 7 Nov 95 09:46:39 PST From: "Padhraic J. Smyth" Subject: AISTATS-97: Preliminary Announcement Preliminary Announcement Sixth International Workshop on Artificial Intelligence and Statistics (AISTATS-97) January 4-7, 1997 Ft. Lauderdale, Florida This is the sixth in a series of workshops which has brought together researchers in Artificial Intelligence (AI) and in Statistics to discuss problems of mutual interest. The exchange has broadened research in both fields and has strongly encouraged interdisciplinary work. Papers on all aspects of the interface between AI & Statistics are encouraged. For more details consult the AISTATS-97 Web page at: http://www.stat.washington.edu/aistats97/ A full Call for Papers will be released in early 1996. The paper submission deadline will be July 1st 1996. The workshop is organized under the auspices of the Society for Artificial Intelligence and Statistics. Program Chair: David Madigan, University of Washington General Chair: Padhraic Smyth, JPL and UCI ------------------------------ From: Raul Valdes-Perez Date: Mon, 23 Oct 95 11:28:26 EDT Subject: AIJ on Sci Disc: new deadline Earlier I posted the CFP for an AI Journal special issue on Scientific Discovery, for which the stated deadline was Nov 1. The official CFP appears in the August issue of AIJ. However, the publishers have informed me that subscribers will receive the "August" issue sometime in the next week or so. Since receipt of the official announcement coincides with the submission deadline, we are therefore extending the deadline to *** DEC 15 ***. Contact me (valdes@cs.cmu.edu) for the full CFP or for detailed submission guidelines. Raul Valdes-Perez ********************************************************* ARTIFICIAL INTELLIGENCE Special Journal Issue on Scientific Discovery Editors: Herbert Simon (Carnegie Mellon) Derek Sleeman (Aberdeen) Raul Valdes-Perez (Carnegie Mellon) Advisory Editors: Bruce Buchanan (Pittsburgh), Lindley Darden (Maryland), Gerd Grasshoff (Hamburg), Pat Langley (ISLE & Stanford), Jan Zytkow (Wichita State) Submissions: Dec 15, 1995 Appearance: Scheduled for early 1997 ------------------------------ From: Jonathan Oliver Date: Tue, 24 Oct 1995 09:34:06 +1000 Subject: CFP for ISIS: Information, Statistics and Induction in Science ISIS: Information, Statistics and Induction in Science Melbourne, Australia, 20-23 August 1996 Conference Chair: David Dowe Co-chairs: Kevin Korb and Jonathan Oliver Invited Speakers: Henry Kyburg, Jr. (University of Rochester, NY) J. Ross Quinlan (Sydney University) Jorma J. Rissanen (IBM Almaden Research, San Jose, California) Ray Solomonoff (U.S.A.) Program Committee: Lloyd Allison, Mark Bedau, Hamparsum Bozdogan, Wray Buntine, Peter Cheeseman, Honghua Dai, David Dowe, Doug Fisher, Alex Gammerman, Clark Glymour, Randy Goebel, David Hand, Bill Harper, David Heckerman, Colin Howson, Lawrence Hunter, Frank Jackson, Max King, Kevin Korb, Henry Kyburg, Ming Li, Nozomu Matsubara, Aleksandar Milosavljevic, Richard Neapolitan, Jonathan Oliver, Michael Pazzani, J. Ross Quinlan, Glenn Shafer, Peter Slezak, Ray Solomonoff, Paul Thagard, Neil Thomason, Raul Valdes-Perez, Tim van Gelder, Paul Vitanyi, Chris Wallace, Geoff Webb, Xindong Wu, Jan Zytkow. Inquiries to: isis96@cs.monash.edu.au David Dowe: dld@cs.monash.edu.au Kevin Korb: korb@cs.monash.edu.au or Jonathan Oliver: jono@cs.monash.edu.au Information is available on the WWW at: http://www.cs.monash.edu.au/~jono/ISIS/ISIS.shtml This conference will explore the use of computational modelling to understand and emulate inductive processes in science. The problems involved in building and using such computer models reflect methodological and foundational concerns common to a variety of academic disciplines, especially statistics, artificial intelligence (AI) and the philosophy of science. This conference aims to bring together researchers from these and related fields to present new computational technologies for supporting or analysing scientific inference and to engage in collegial debate over the merits and difficulties underlying the various approaches to automating inductive and statistical inference. Areas of Interest. The following streams/subject areas are of particular interest to the organisers: Concept Formation and Classification. Minimum Encoding Length Inference Methods. Scientific Discovery. Theory Revision. Bayesian Methodology. Foundations of Statistics. Foundations of Social Science. Foundations of AI. Call for Papers. Prospective authors should mail five copies of their papers to Dr. David Dowe, ISIS chair. Alternatively, authors may submit by email to isis96@cs.monash.edu.au. Email submissions must be in LaTex (using the ISIS style guide [will be available at the ISIS WWW page]). Submitted papers should be in double-column format in 10 point font and not exceeding 10 pages. An additional page should display the title, author(s) and affiliation(s), abstract, keywords and identification of which of the eight areas of interest (see http://www.cs.monash.edu.au/~jono/ISIS/ISIS.Area.Interest.html) are most relevant to the paper. Refereeing will be blind; that is, this additional page will not be passed along to referees. The proceedings will be published; details have not yet been settled with the prospective publisher. Accepted papers will have to be represented by at least one author in attendance to be published. Papers should be sent to: Dr David Dowe ISIS chair Department of Computer Science Monash University Clayton Victoria 3168 Australia Phone: +61-3-9 905 5226 FAX: +61-3-9 905 5146 Email: isis96@cs.monash.edu.au Submission (receipt) deadline: 11 March, 1996 Notification of acceptance: 10 June, 1996 Camera-ready copy (receipt) deadline: 15 July, 1996 Conference Venue ISIS will be held at the Old Melbourne Hotel, 5-17 Flemington Rd. North Melbourne. The Old Melbourne Hotel is within easy walking distance of downtown Melbourne, Melbourne University, many restaurants (on Lygon Street) and the Melbourne Zoo. It is about fifteen to twenty minutes drive from the airport. Registration A registration form will be available at the WWW site http://www.cs.monash.edu.au/~jono/ISIS/ISIS.shtml, or by mail from the conference chair. Dates for registration will be considered to be met assuming that legible postmarks are on or before the dates and airmail is used. Student registrations will be available at a discount (but prices have not yet been fixed). Relevant dates are: Early registration (at a discount): 3 June, 1996 Final registration: 1 July, 1996 ------------------------------ From: Zighed Date: Tue, 24 Oct 1995 20:26:26 +01 Subject: Machine learning program for PC/Windows We have the pleasure to offer you the first platform of "knowledge engineering" (SIPINA-W ). YOU will thus be able to : 1- generate knowledge in the form of production rules and this, by learning from a basis of examples characterized by numeric and/or symbolic data. 2-test on non-learned examples knowledge produced before. 3-add knowledge generated automatically by SIPINA-W to other knowledge which can be given by a human expert. 4-merge and optimize bases of knowledge produced artificially or by a human expert... SIPINA-W offers you numerous options which allow you to use methods such as C4.5. You will find many applications made on known examples in medical, industrial or financial fields... You can have access to SIPINA-W by: ftp.univ-lyon2.fr /pub/pc/Eric/SIPINA The version we present here is a beta test version. You can duplicate and diffuse this software freely. We only ask you to be so kind as to send us an e-mail to give us your address. We will thus be able to send you the new updatings. You can also indicate us mistakes we could have missed. Thank you and see you soon, Prof. D.A. Zighed Equipe de Recherche en Ingnierie des Connaissances Universit Lumire Lyon 2 5 av. Pierre Mends-France 69676 Bron France tel. (33) 78 77 23 76 Fax. (33) 78 77 23 75 e-mail : zighed@diogene.univ-lyon2.fr ------------------------------ From: "Peter A. Flach" Date: Thu, 26 Oct 1995 14:06:11 +0000 Subject: Abduction and Induction Workshop Dear colleagues, We are submitting a proposal to ECAI96 (the European Conference on Artificial Intelligence to be held in Budapest 12 - 16 August 1996) to organize a workshop on "Abductive and Inductive Reasoning" Below, is a draft of the proposal. To support the proposal, we would like to submit a list of colleagues who are interested in attending the workshop. Obviously, inclusion on the list does not commit you to attending, it is just to give the organizers an idea of the degree of support that we would have. Therefore, if you would like to support us in this, could you please email us as soon as possible (preferably before Tuesday, October 31) at the following address: Peter.Flach@kub.nl Description and focus Abduction and induction are important forms of hypothetical reasoning, i.e. inferences from essentially incomplete information. Abduction is generally understood as reasoning from effects to causes or explanations, and induction as inferring general rules from specific data. In Artificial Intelligence, a typical application of abduction is diagnosis, and a typical application of induction is learning from examples. As logical inference operations, both abduction and induction embody a form of "reversed" deduction: together with the background knowledge, the hypothesis should entail a given set of observations. However, the two inference operations differ significantly in the way they satisfy and use this basic specification. Abduction is used to generate a reason, an explanation, for the truth of the observation in terms of hypotheses which are typically specific to the situation and individual objects at hand. Each separate observation will get its own explanation, not necessarily related to the explanations of other observations, and not intended to account for other (later) observations. On the other hand, induction is used when we want to synthesize the information conveyed by the observations into a hypothesis that can account for all the observations together in a common way. Moreover, this hypothesis will also be able to account for other new observations on different objects than those involved in the original observations. Hence typically an abductive hypothesis consists of specific facts further describing the situation at hand, while an inductive hypothesis consists of general rules pertaining to a whole class of situations. This distinction can be traced back to Peirce's syllogistic classification of deduction, abduction and induction. However, this intuitive distinction between abductive and inductive hypotheses does not seem to be fully captured by contemporary models of abductive and inductive reasoning in Artificial Intelligence. For example, by comparing the models employed in Abductive and Inductive Logic Programming one can show superficially (using naming and syntactical transformations) that they are equivalent. This perceived "equivalence" between abduction and induction runs counter to practical experience, since the algorithms involved are rather different. This workshop is intended to improve our understanding of the inference operations underlying abductive and inductive reasoning, and the relation between the two. A specific goal will be to develop ideas upon which a sound and intuitively appealing logical semantics of abduction and induction can be built, suitable for problems in Artificial Intelligence. General topics of interest are the following: - philosophical and logical backgrounds - semantics for abduction and induction - frameworks of integration of abduction and induction - abduction in inductive Machine Learning - induction in Artificial Intelligence approaches to abduction Organization of the Workshop The workshop will consist of presentations and moderated discussions. The presentations are intended to put the issues under discussion into context; the speakers are selected by the Organizing Committee. One of the presentations will be a summary of the submitted position papers. The discussions will each address one of the main topics of the workshop; they are explicitly intended to reach some form of agreement or conclusion. The topics for discussion are preselected by the Organizing Committee, but participants will be encouraged to propose other topics at the workshop. Organizing Committee Peter Flach INFOLAB, Tilburg University POBox 90153, 5000 LE Tilburg, the Netherlands tel. +31 13 4663119 fax +31 13 4663069 email Peter.Flach@kub.nl Antonis Kakas Dept. of Computer Science, University of Cyprus POBox 537, CY-1678 Nicosia, Cyprus tel. +357 2 338705/4 fax +357 2 339062 email antonis@turing.cs.ucy.ac.cy Marc Denecker Dept. of Computer Science, Katholieke Universiteit Leuven Celestijnenlaan200A, B-3001 Heverlee, Belgium tel. +32 16 327544 fax +32 16 327996 email Marc.Denecker@cs.kuleuven.ac.be Luc De Raedt Dept. of Computer Science, Katholieke Universiteit Leuven Celestijnenlaan200A, B-3001 Heverlee, Belgium tel. +32 16 327550 fax +32 16 327996 email Luc.DeRaedt@cs.kuleuven.ac.be ------------------------------ From: Melanie Mitchell Date: Sat, 28 Oct 95 15:33:21 MDT Subject: Postdoctoral Fellowships at the Santa Fe Institute The Santa Fe Institute has an opening for one or more Postdoctoral Fellows beginning in September, 1996. The Institute's research program is devoted to the study of complex systems, especially complex adaptive systems. Systems and techniques currently under study include: the economy; the immune system; the brain; biomolecular sequence and structure; the origin of life; artificial life; models of evolution; adaptive computation and intelligent systems; complexity, entropy, and the physics of information; nonlinear modeling and prediction; the evolution of culture; the development of general-purpose simulation environments; and others. Postdoctoral Fellows work either on existing research projects or on projects of their own choosing. Candidates should have a Ph.D. (or expect to receive one before September 1996) and should have backgrounds in computer science, mathematics, economics, theoretical physics or chemistry, game theory, cognitive science, theoretical biology, dynamical systems theory, or related fields. A strong background in computational approaches is essential, as is an interest in interdisciplinary work. Evidence of this interest, in the form of previous research experience and publications, is important. Applicants should submit a curriculum vitae, list of publications, and statement of research interests, and arrange for three letters of recommendation. Incomplete applications will not be considered. All application materials must be received by February 15, 1996. Decisions will be made by April, 1996. Send applications to: Postdoctoral Committee, Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, New Mexico 87501. Send complete application packages only, preferably hard copy, to the above address. Include your e-mail address and/or fax number. SFI is an equal opportunity employer. Women and minorities are encouraged to apply. More information about SFI and its research program can be found at SFI's web site: http://www.santafe.edu. ------------------------------ From: Peter Turney Date: Mon, 30 Oct 1995 08:14:16 +0500 Subject: Baldwin Effect Bibliography Baldwin Effect Bibliography We are pleased to announce a bibliography of papers on the Baldwin effect. The Baldwin effect lies at the intersection of Machine Learning and Genetic Algorithms. The bibliography is in hypertext, with links to many of the authors and papers. The URL is: http://ai.iit.nrc.ca/baldwin/bibliography.html Any coments, corrections, or additions are welcome. Sincerely, Peter Turney, peter@ai.iit.nrc.ca Darrell Whitley, whitley@CS.ColoState.EDU Russell Anderson, rwa@milo.berkeley.edu ------------------------------ Date: Wed, 1 Nov 1995 07:56:59 -0800 From: Pat Langley Subject: textbook on machine learning My machine learning text has appeared, at long last. Below I include some general information about the book's contents and a pointer to Morgan Kaufmann's web page, for those who want to order electronically. ELEMENTS OF MACHINE LEARNING Pat Langley, ISLE & Stanford University Machine learning is the computational study of algorithms that improve performance based on experience. Recent years have seen an explosion of work on this topic, which has produced a wide variety of automated learning algorithms and results on their behavior. This textbook, designed for graduate courses in machine learning, covers the basic issues in this central area of artificial intelligence. The chapters cover the main induction algorithms that have been explored in the literature, presenting them within a coherent theoretical framework that moves beyond traditional paradigm boundaries. Four major sections introduce the basic concepts and problems in machine learning, describe algorithms for inducing simple concepts, present alternatives for organizing learned concepts into large-scale structures, and discuss adaptations of the learning methods to more complex problem-solving tasks. The text describes these computational techniques in detail and gives examples of their operation, along with exercises and pointers to the literature. Pat Langley is Director of ISLE, a nonprofit institute that focuses on machine learning, and holds an appointment at Stanford University. He has taught numerous courses in artificial intelligence, including graduate seminars and tutorials on machine learning. Langley has authored over 100 papers on the topic, has co-edited four books in the area, and was the founding editor of the journal Machine Learning. He is also co-author of the book Scientific Discovery: Computational Explorations of the Creative Processes. Table of Contents - An Overview of Machine Learning - The Induction of Logical Conjunctions - The Induction of Threshold Concepts - The Induction of Competitive Concepts - The Construction of Decision Lists - Revision and Extension of Inference Networks - The Formation of Concept Hierarchies - Other Issues in Concept Induction - The Formation of Transition Networks - The Acquisition of Search-Control Knowledge - The Formation of Macro-Operators - Prospects for Machine Learning October 1995; approx. 400 pages; cloth; ISBN 1-55860-301-8; US & Canada $49.95 The Morgan Kaufmann Series in Machine Learning, Pat Langley, Series Editor For online ordering information, see http://mkp.com/pages/3018/index.html ------------------------------ Date: Wed, 1 Nov 95 15:53:03 EST From: Lee Giles Subject: Call For Papers: International Colloquium on Grammatical Inference Third International Colloquium on Grammatical Inference (ICGI-96) Montpellier (France), September 25-27, 1996 With the help of the Special Interest Group in Natural Language Learning (SIGNLL) of the ACL. Chairperson: Laurent Miclet (IRISA-ENSSAT, Lannion, France) miclet@enssat.fr Organization : Colin de la Higuera (LIRMM, Montpellier, France) cdlh@lirmm.fr Scientific Committee J. Berstel (University Paris 6, France) M. Brent (J. Hopkins University, USA) H. Bunke (University of Bern, Switzerland) C. Cardie (Cornell University, USA) W. Daelemans (KUB, Nederlands) P. Dupont (FT-CNET, France) O. Gascuel (LIRMM, France) C. L. Giles (NEC Princeton, USA) J. Gregor (University of Tennessee, USA) J. P. Haton (CRIN-INRIA, France) F. Jelinek (J. Hopkins University, USA) T. Knuutila (University of Turku, Finland) S. Lucas (University of Essex, England) D. Luzeaux (=05ETCA, France) D. Magerman (Renaissance Technology, USA) E. Makinen (University of Tampere, Finland) R. Mooney (University of Texas, USA) G. Nagaraja (IIT Bombay, India) J. Nicolas (IRISA, France) J. Oncina, University of Alicante, Spain) L. Pitt (University of Illinois, USA) D. Powers (Flinders University, Australia) Y. Sakakibara (Fujitsu Laboratories Ltd, Japan) A. Stolcke (SRI International, Menlo Park, USA) E. Vidal (Universidad Politechnica de Valencia, Spain) Grammatical Inference (GI) is broadly understood as Machine Learning of Grammars and Languages from data. Traditionally, GI has been studied within several contexts: Information Theory, Formal Languages Theory, Computational Linguistics, Machine Learning, Pattern Recognition, Computational Learning Neural Networks, etc. This multidisciplinary perspective, however, has lead so far to a lack of a focused research community. A first attempt to correct this started with the "First Colloquium on Grammatical Inference : Theory, Applications and Alternatives" held in the University of Essex (U.K.), in April 1993. Then followed the "International Colloquium on Grammatical Inference 1994", held in Alicante (Spain), which proceedings have been published by Springer-Verlag as Volume 862 of the Lectures Notes in Artificial Intelligence. Following these successful meetings, ICGI 96 keeps aiming to provide a forum for discussion of principles, theory and applications of all those aspects of Machine Learning that explicitly focus on Grammars and Languages. Within this framework, topics of interest include, but are not limited to, the following : * Learning Paradigms for Grammars and languages : Cognitive models, Algebraic aspects, Identification in the limit and PAC-Learning ; Stochastic and Corpus-based approaches, Neural Networks, Genetic Algorithms, Fuzzy systems, etc. * Algorithms. * Heuristics. * Benchmarks. * Applications : Natural Language Processing, Language Translation ; Biological Sequences and Time Series Modelization and Prediction ; Image and Speech Recognition, Discrete Events Systems, etc. SCHEDULE : April 1, 1996 Deadline for submitted papers. June 15, 1996 Notification of acceptance and referrees comments. July 15, 1996 Camera ready copy. September 25-27, 1996 Colloquium. Please submit (not via electronic mail) three copies of your full length article (maximum 12 pages, 12 pt. font, including figures, tables, references, etc.) to : L. Miclet IRISA-ENSSAT BP 447 - 6, Rue de K=E9rampont 22305 LANNION Cedex FRANCE The Proceedings of the Colloquium will be considered for publication as a volume in the Springer-Verlag Lecture Notes Series in Artificial Intelligence. Information on ICGI'96 is on the www page : http://itkwww.kub.nl:2080/itk/Docs/Projects/Walter/icgi.html Laurent Miclet ENSSAT ________________________________________________________________ Ecole Nationale Sup=E9rieure de Sciences Appliqu=E9es et Technologie BP 447 6, rue de K=E9rampont Tel : +33 96 46 66 28 22305 LANNION Cedex Fax : +33 96 37 01 99 =46RANCE ________________________________________________________________ C. Lee Giles / Computer Sciences / NEC Research Institute / 4 Independence Way / Princeton, NJ 08540, USA / 609-951-2642 / Fax 2482 http://www.neci.nj.nec.com/homepages/giles.html ------------------------------ Date: Fri, 03 Nov 95 13:30:08 -0600 From: Amanda Nevin Subject: ICNN'96 - Important Extension INTERNATIONAL CONFERENCE ON NEURAL NETWORKS (ICNN'96) ------------------- IMPORTANT EXTENSION ------------------- Deadline for submission to ICNN'96 has been extended to December 29, 1995 Authors interested to submit papers must have their papers received by the Program Chair (Professor Bing Sheu, Powell Hall, Room 604, Department of Electrical Engineering, University of Southern California, Los Angeles, CA 90089-0271, USA) by the deadline. Papers received after that date will be returned unopened. Papers submitted must be in final publishable form and will be reviewed by senior researchers in the field using the same standard as papers submitted before the October 16, 1995, deadline. However, papers submitted after the October 16, 1995, deadline will be either accepted or rejected, and authors of accepted papers will not have a chance to revise their papers. (Authors of accepted papers submitted before the October 16 deadline will be allowed to revise their papers.) Authors submitting papers late will be notified of the final decision by February 15, 1996. Six copies (one original and five copies) of the paper must be submitted. Papers must be camera-ready on 8 1/2-by-11 white paper, one-column format in Times or similar font style, 10 points or larger with one inch margins on all four sides. Do not fold or staple the original camera-ready copy. Four pages are encouraged; however, the paper must not exceed six pages, including figures, tables, and references, and should be written in English. Submissions that do not adhere to the guidelines above will be returned unreviewed. Centered at the top of the first page should be the complete title, author name(s) and postal and electronic mailing addresses. In the accompanying letter, the following information must be included: (a) full title of paper; (b) presentation preferred (oral or poster); (c) audio visual requirements (e.g. 35 mm slide, OHP, VCR); (d) corresponding author (name, postal and e-mail addresses, telephone & fax numbers); and (e) presenter (name, postal and e-mail addresses, telephone & fax numbers). For further information on ICNN'96, please consult our World Wide Web home page at http://www-ece.rice.edu/96icnn, or send electronic mail to b-wah@uiuc.edu. ------------------------------ Date: Fri, 3 Nov 1995 18:00:37 -0500 (EST) From: Sreerama Murthy Subject: PhD thesis: On Growing Better Decision Trees from Data The following PhD thesis is now available on WWW and FTP. You may retrieve the postscript files for whole thesis as well as just the chapter(s) of interest. There is also a HTML version that allows retrieval of even individual subsections. On Growing Better Decision Trees from Data Sreerama K. Murthy Department of Computer Science, Johns Hopkins University Thesis Advisor: Steven L. Salzberg This thesis investigates the problem of growing decision trees from data, for the purposes of classification and prediction. After a comprehensive, multi-disciplinary survey of work on decision trees, some algorithmic extensions to existing tree growing methods are considered. The implications of using (1) less greedy search and (2) less restricted splits at tree nodes are systematically studied. Extending the traditional axis-parallel splits to {\it oblique} splits is shown to be practical and beneficial for a variety of problems. However, the use of more extensive search heuristics than the traditional greedy heuristic is argued to be unnecessary, and often harmful. Any effort to build good decision trees from real-world data involves ``massaging'' the data into a suitable form. Two forms of data massaging, domain-independent and domain-specific, are distinguished in this work. A new framework is outlined for the former, and the importance of the latter is illustrated in the context of two new, complex classification problems in astronomy. Highly accurate and small decision tree classifiers are built for both these problems through a collaborative effort with astronomers. To retrieve: World-Wide-Web: http://www.cs.jhu.edu/grad/murthy. FTP: Anonymous ftp to blaze.cs.jhu.edu. Directory pub/murthy. The file thesis.ps.gz is the whole thesis. For individual chapters, first get contents.ps.gz, which has the table of contents. The directory contains a postscript file for each chapter. The filenames should be self-explanatory. Dont forget to use binary transfer mode! ------------------------------ Date: Sun, 5 Nov 1995 14:59:20 -0700 (GMT-0700) From: Una-May O'Reilly Subject: GP and Hill Climbing Comparison I wholeheartedly concur with a previous posting of John Koza: " ... if we are interested in getting computers to solve problems without being explicitly programmed, the structures that we really need are COMPUTER PROGRAMS" The GP paradigm delimits of a rich class of problems and defines a very interesting algorithm. A lot of work towards understanding GP remains! In light of the interest shown in Genetic Programming comparisons and this class of program discovery problems, I am providing several references that describe my research extending back over a year on this topic. SYNOPSIS: I used a small suite of problems expressed exactly as they are for GP to experiment with Simulated Annealing and Stochastic Iterated Hill Climbing algorithms. The suite consists of 6-Multiplexer, 11-Multiplexer, two versions of a sorting task and Block Stacking. I first used a mutate operator for program parse trees then subsequently examined the effectiveness of the search when the standard GP crossover operator was substituted for mutation. To improve GP, I also tried combining the localized search of hill climbing with Genetic Programming. All these algorithms were quite effective on my suite. Though my problem suite did not use Automatically Defined Functions, it is simple to extend my mutate operator and thus include such problems. I am presently pursuing this. The papers provide various detailed comparisons (eg. likelihood of success, number of programs examined). The central issue in them is the understanding of GP through a comparative analysis. Deciding what algorithm to use for a class of program discovery problems is a issue of great scope for program discovery. I think all comparative results indicate that the new challenge is to understand the nature of the search space (also called fitness landscape) of program discovery problems. It's not clear how to rigourously define a class of program discovery problems nor to to define characteristics of them that would indicate what search algorithm is best suited to search them. The fact that other algorithms also can solve program discovery does not make me downcast about algorithms based upon evolution, either. New extensions to evolution based algorithms may yield improvements that could not result from extending other approaches (because they do not have a population of solutions, the possibility of co-adaptation, etc.). @InProceedings{OReillyOppacher94, author = "U.-M.~O'Reilly and F.~Oppacher", title = "Program Search with a Hierarchical Variable Length Representation: {G}enetic Programming, Simulated Annealing and Hill Climbing", editor = {Y.~Davidor and H.-P.~Schwefel and R.~M\"{a}nner}, pages = "397--406", booktitle = "Parallel Problem Solving From Nature -- {PPSN III}, volume 866 of Lecture Notes in Computer Science", publisher = "Springer-Verlag", address = "Berlin", year = 1994 } An extended version of the above paper is available as Santa Fe Institute Technical Report 95-04-21. @TechReport{OReillyOppacher95, author = "U.-M.~O'Reilly and F.~Oppacher", title = "Hybridized Crossover-Based Search Techniques for Program Discovery", institution = "Santa Fe Institute", year = 1995, number = "95--02--007", address = "Santa Fe, NM", month = "February" note = A shortened version of this report will appear in the proceedings of the IEEE World Congress on Evolutionary Computation, Dec 1995, Editor: David Fogel. } @bookchapter{OReillyOppacher94, author = "U.-M.~O'Reilly and F.~Oppacher", title = "A Comparative Analysis of Genetic Programming", editor = "P.J. Angeline and K. Kinnear, booktitle = "Advances in Genetic Programming II", publisher = "MIT Press", notes = "To appear as Chapter 2" } @PhdThesis{OReilly:thesis, author = "U.-M. O'Reilly", title = "An Analysis of Genetic Programming", school = "Carleton University", address = "Ottawa, Ontario, Canada", year = 1995, notes = "http://www.santafe.edu/~unamay or ftp.santafe.edu, dir pub/unamay ftp://cs.ucl.ac.uk/genetic/papers/oreilly" } Una-May O'Reilly most recently: Graduate Fellow, Graduate Student Santa Fe Institute, School of Computer Science 1399 Hyde Park Rd, Carleton University Santa Fe, NM, 87505 Ottawa, Ontario, Canada, K1S 5b6 temporarily at: unamay@bootstrap.econ.wisc.edu as of Jan 1 at: unamay@ai.mit.edu ------------------------------ Date: Mon, 06 Nov 1995 08:55:12 +0000 (GMT) From: Floor Verdenius Subject: CfP: Workshop on Neural Network Applications in Agriculture First Call for Papers Neural Network Applications in Agriculture (NNAA) ICCTA'96 Workshop Wageningen, The Netherlands June 19, 1996 INTRODUCTION Neural Networks are ready for real-world application. It is well known that Neural Networks offer solutions in domains where traditional, non- linear techniques, are not powerful enough. This type of problems often occurs in agricultural domains, where processes are non-linear of nature. In several domains the application of Neural Networks has gained interest. Recent events, such as the Neural Network sessions of the IFAC workshop "AI in Agriculture" and the international symposium "Neural Networks: Artificial Intelligence and Industrial Applications" of the Dutch Foundation for Neural Networks, illustrate the application momentum of Neural Networks. Several projects have aimed at applying NN in agriculture. But which NN-applications have been actually realised in agriculture? In which domains? Using which technologies? Solving which problems? Using which approaches? These questions are addressed by a workshop within the International Congress for Computer Technology in Agriculture (ICCTA'96), which takes place from June 16 to June 19, 1996 in Wageningen the Netherlands. THE WORKSHOP This half-day workshop offers an international platform to present real world applications and discuss aspects of Neural Network applications in agricultural domains. The foreseen public includes decision makers in agribusiness, leading practitioners, scientists from research institutes, extension people in agriculture and food and manufacturers or hard- and software. Topics of interest include (but are not restricted to): * Applications of neural networks, amongst others in the following domains: - Image Recognition - Process Control - Product Classification - Process Modelling - Signal Processing - Predictive Modelling * The process of developing the application * Application from the users/client viewpoint * Learning applications where NN were rejected CALL FOR PRESENTATIONS For this workshop we seek original papers for oral presentation, reporting real-world applications of neural networks in all above-mentioned areas. Candidate papers for the workshop concern application oriented projects aiming at implemented systems, preferably realised in a client-contractor relationship. Purely research oriented papers will be rejected for this workshop. They may of course be of interest for the ICCTA'96 session on Neural Networks and Fuzzy Systems. Candidate papers should clearly describe the application requirements, the realised functionality and the development process of the application. Further items of interest are: Integration with other system components, possible alternative techniques for the neural networks and the structuring of the development process. Additionally to submitted papers, a limited number of papers will be invited from recognised experts in the field of neural network applications in general, and/or neural network applications in agriculture. Submissions in the form of a draft paper (max. 5000 words) or an extended abstract (max. 1500 words) can be sent to: Prof. A.J. Udink ten Cate DLO-Agricultural Research Department Bornsesteeg 53 6708 PD Wageningen The Netherlands +31-(0)317-474132 e-mail: udink@co.dlo.nl Please indicate that your submission concerns the workshop NNAA of the ICCTA'96 congress. Electronic submissions (postscript or ASCII files) are preferred. Submissions are reviewed according to the ICCTA'96 review procedure. Accepted papers will appear in the congress proceedings. Additionally publication of the workshop papers in an international journal is aimed for. Deadlines Submission 15-12-95 Notification of Acceptance for the workshop 31- 1-96 Camera Ready Copies 1- 3-96 Workshop 19- 6-96 COSTS Visiters to ICCTA'96 have free access to the workshop. Exclusive registration for the workshop is also possible. Registration fees in this case will be based on the fee for the ICCTA'6 congress. WORKSHOP COMMITTEE Prof. J. DeBaerdemaeker Catholic University of Leuven (B) Prof. C.C.A.M. Gielen Foundation of Neural Networks, Nijmegen (NL) Dr. H.J. Kappen Foundation of Neural Networks, Nijmegen (NL) Dr. Th. Rath University of Hannover (D) Dr. V. Steinmetz Cemagref Montpellier, Montpellier (F) LOCAL ORGANISERS Ir. A.J.M. Timmermans & Drs. F. Verdenius e-mail {A.J.M.Timmermans,F.Verdenius}@ato.dlo.nl ATO-DLO, PO box 17, 6700 AA Wageningen, The Netherlands phone +31-(0)317-475000 in close co-operation with the International Program Committee of ICCTA'96. FURTHER INFORMATION For any additional information, please contact the local organisers. ------------------------------ Date: Wed, 8 Nov 1995 19:04:12 -0500 (EST) From: rgoldsto@bronze.ucs.indiana.edu Subject: Faculty Position at Indiana University INDIANA UNIVERSITY-BLOOMINGTON COGNITIVE SCIENCE POSITION The Cognitive Science Program and the Computer Science Department at Indiana University-Bloomington seek applicants for a joint faculty position, with rank open. Start date may be as early as Fall 1996, pending funding approval. We are looking for outstanding researchers at the forefront of the field, with ability to contribute to both Cognitive Science and Computer Science. Research area is open, including, for instance, neural net modeling, logic, reasoning, representation and information, language and discourse, robotics, computational vision and speech, visual inference, machine learning, and human-computer interaction. Applications from women and minority members are specifically encouraged. Indiana University is an Affirmative Action/Equal Opportunity Employer. The prospective faculty's office and laboratory will be based in the Computer Science department, which occupies a recently renovated spacious limestone building, and has extensive state-of-the-art computing facilities. Responsibilities will be shared with the Cognitive Science Program, one of the largest and most esteemed programs in the world today. The attractive wooded campus of Indiana University is located in Bloomington, voted one of the most cultural and livable small cities in the US, and a mere 45 minute drive from the Indianapolis airport. To be given full consideration applications must be received by February 15, 1996. The application should contain a detailed CV, copies of recent publications, brief statement of interests and future directions, and either three letters of recommendations, or a list of three references. Two copies of the application file must be sent (letters of reference may be sent to either address). Cognitive Science Search Cognitive Science Search Computer Science Department Cognitive Science Program Indiana University Psychology Department Bloomington, IN 47405 Indiana University Bloomington, IN 47405 Internet: cogsci-search@cs.indiana.edu or iucogsci@indiana.edu ____________________________ Rob Goldstone Department of Psychology/Program in Cognitive Science Indiana University Bloomington, IN. 47405 rgoldsto@indiana.edu Web site: http://cognitrn.psych.indiana.edu/ ------------------------------ End of ML-LIST (Digest format) **************************************** From isca@interpath.com Tue Nov 14 00:03:13 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 14 Nov 95 00:03:10 -0600; AA15843 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 14 Nov 95 00:03:08 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa02434; 13 Nov 95 23:37:55 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa02431; 13 Nov 95 23:20:21 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa09235; 13 Nov 95 23:19:51 EST Received: from MAILBOX.SRV.CS.CMU.EDU by B.GP.CS.CMU.EDU id aa24072; 13 Nov 95 14:34:46 EST Received: from mail-hub.interpath.net by MAILBOX.SRV.CS.CMU.EDU id aa04458; 13 Nov 95 14:32:00 EST Received: from ISCA.Interpath.net (isca.pdial.interpath.net [199.72.102.171]) by mail-hub.interpath.net (8.6.12/8.6.14) with SMTP id LAA13980; Mon, 13 Nov 1995 11:43:24 -0500 Date: Mon, 13 Nov 1995 11:43:24 -0500 Message-Id: <199511131643.LAA13980@mail-hub.interpath.net> X-Sender: isca@mail.interpath.com X-Mailer: Windows Eudora Version 1.4.3 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: isca@interpath.com From: Mary Ann Sullivan Subject: CALL FOR PAPERS: 5th Intelligent Systems Conf. - Reno, Nevada June 19-21, 1996 (Formerly GWICS) ******************************************************************************* This message is being sent to multiple addressees. If you wish to have your address removed from our mailing list, please reply to the sender and we will promptly honor your request. ******************************************************************************* CALL FOR PAPERS Fifth International Conference on Intelligent Systems (formerly GWICS) June 19 - 21, 1996 Flamingo Hilton, Reno, Nevada, U.S.A. Sponsored by the International Society for Computers and Their Applications (ISCA) ============================================================================== CONFERENCE CHAIR PROGRAM CHAIR Carl Looney Frederick C. Harris, Jr. (Univ. of Nevada, Reno) (Univ. of Nevada, Reno) ============================================================================== The International Conference on Intelligent Systems seeks quality international submissions in all areas of intelligent systems including but not limited to: Logic and Inference Cognitive Science Artificial Neural Networks Reasoning Distributed Intelligent Systems Artificial Life Case-Based Reasoning Knowledge-Based Systems Vision, Image Processing Interpretation Machine Learning and Adaptive Sys. Cellular Automata Fuzzy Systems Robotics, Control and Planning Multimedia and Human Computer Evolutionary Computation Interaction (GA,GP,ES,EP) Autonomous Agents Recognition and Classification Search Instructions to Authors: Authors must submit 5 copies of an extended abstract (at least 4 pages) or complete paper (no more than 10 double spaced pages). Please include one separate cover page containing title, author's name(s), address, affiliation, e-mail address, telephone number, and topic area. To help us assign reviewers to papers, use the topics in the list above as a guide. In cases of multiple authors, all correspondence will be sent to the first author unless otherwise requested. Abstracts may be submitted via E-mail. Submit your paper by February 15, 1996 to the program chair: Dr. Frederick C. Harris, Jr. Telephone: (702) 784-6571 University of Nevada Fax: (702) 784-1766 Dept. of Computer Science E-mail: fredh@cs.unr.edu Reno, Nevada 89557 IMPORTANT DATES: Deadline for extended summary/paper submission: February 15, 1996 Notification of acceptance: April 10, 1996 Camera ready papers due: May 10, 1996 PROGRAM COMMITTEE G. Antoniou (U. of Newcastle) A. Barducci (CNR-IROE,Italy) M. Boden (U. of Skovde) A. Canas (U. of West Florida) M. Cohen (Cal State, Fresno) D. Egbert (U. of Nevada, Reno) S. Fadali (U. of Nevada, Reno) J. Fisher (Cal State Pomona) K. Ford (U. of West Florida) P. Geril (U. of Ghent) J. Gero (U. of Sydney) D. Hudson (U. of Cal, San Fran.) P. Jog (DePaul U.) S. Kawata (Tokyo Metropolitan U.) V. R. Kumar (Fujitsu, Australia) D. Leake (Indiana U.) F. Lin (Santa Clara U.) S. Louis (U. of Nevada, Reno) J. McDonnell (NRaD, San Diego) A. McRae (Appalachian State U.) S. Narayan (UNC-Wilmington) T. Oren (U. of Ottawa) V. Patel (McGill U.) D. Pheanis (Arizona State U.) V. Piuri (Politecnico Di Milano) R. Reynolds (Wayne State U.) M. Rosenman (U. of Sydney) A. Sangster (Aberdeen U.) R. Smith (U. of Alabama) R. Sun (U. of Alabama) A. Yfantis (UNLV) S. Yoon (Widener U.) ISCA Headquarters 8820 Six Forks Road, Raleigh, NC 27615 (USA) Ph: (919) 847-3747 Fax: (919) 676-0666 E-mail: isca@interpath.com URL= http://www.isca-hq.org/isca From dhw@santafe.edu Tue Nov 14 19:22:43 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 14 Nov 95 19:22:41 -0600; AA09031 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 14 Nov 95 19:22:38 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa03513; 14 Nov 95 18:29:16 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa03483; 14 Nov 95 18:10:27 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa09857; 14 Nov 95 18:09:55 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa13163; 14 Nov 95 13:29:22 EST Received: from sfi.santafe.edu by CS.CMU.EDU id aa14419; 14 Nov 95 13:27:24 EST Received: from santafe (santafe.santafe.edu) by sfi.santafe.edu (4.1/SMI-4.1) id AA04471; Tue, 14 Nov 95 11:24:58 MST Date: Tue, 14 Nov 95 11:24:58 MST From: David Wolpert Message-Id: <9511141824.AA04471@sfi.santafe.edu> To: Connectionists@cs.cmu.edu Subject: Response to no-free-lunch discussion Some quick comments on the recent discussion of no-free-lunch (NFL) issues. As an aside, it's interesting to note how "all over the map" the discussion is, from people who whole-heartedly agree with NFL, to people who make claims diametrically opposed to it. **** Simon Perkins writes: >>> some people claim that there _is_ a general sub-class of problems for which it's at all feasible to find a solution - perhaps problems whose solutions have low komologrov complexity or something - and this might well mean that there _is_ a general good learning algorithm for this class of `interesting problems'. Any comments? >>> There's no disputing this. Restricting attention to a sub-class of problems is formally equivalent to placing a restriction on the target and/or prior over targets (in the latter case, placing a restriction on the prior's support). Certainly once things are restricted this way, we are in the domain of Bayesian analysis (and/or some versions of PAC), and not all algorithms are the same. However we have *no* a priori reason to believe that targets with "low Kolmogorov complexity" (or anything else) are/not likely to occur in the real world. Well, nothing can be proven from first principles to work well, you might say. This actually isn't always true (the loss function is important, there are minimax issues, etc.) But even in the simple scenarios in which this sentiment is essentially correct (i.e., the scenarios in which NFL holds), there is a huge body of literature which purports to prove from first principles that some algorithms *do* work better than others, without any assumption about the targets: *** E.g., claims that so long as the VC dimension of your algorithm is low, the training set large, and the misclassification rate on the training set small, then *independent of assumptions concerning the target*, you can bound how large the generalization error is. Or claims that boosting can only help generalization error, regardless of the prior over targets. Or the PAC "proof" of Occam's razor (which - absurdly - "holds" for any and all complexity measures). NFL results show up all such claims as problematic, at best. The difficulty is not with the math behind these claims, but rather with the interpretations of what that math means. *** Dimitris Tsioutsias writes: >>> As any thoughtful person in the greater mathematical programming community might point out, there's no general optimization method that could outperform any other on any kind of problem. >>> Obviously. But to give a simple example, before now, no such "thoughtful person" would have had any idea of whether there might be a "general optimization method" that only rarely performs worse than random search. Addressing such issues is one of the (more trivial) things NFL can do for you. *** Finally, Juergen Schmidhuber writes: >>> We already know that for a wide variety of non-incremental search problems, there *is* a theoretically optimal algorithm: Levin's universal search algorithm >>> I have lots of respect for Juergen's work, but on this issue, I have to disagree with him. Simply put, supervised learning (and in many regards search) is an exercise in statistics, not algorithmic information complexity theory. NFL *proves* this. In practice, it may (or may not) be a good idea to use an algorithm that searches for low Levin complexity rather than one that works by other means. But there is simply no first principles reason for believing so. Will such an algorithm beat backprop? Maybe, maybe not. It depends on things that can not be proven from first principles. (As well as a precise definition of "beat", etc.) The distribution over the set of problems we encounter in the real world is governed by an extraordinarily complicated interplay between physics, chemistry, biology, psychology, sociology and economics. There is no a priori reason for believing that this interplay respects notion of algorithmic information complexity, time-bounded or otherwise. David Wolpert From copelli@onsager.if.usp.br Tue Nov 14 19:22:45 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 14 Nov 95 19:22:42 -0600; AA09033 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 14 Nov 95 19:22:40 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab03513; 14 Nov 95 18:29:47 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa03485; 14 Nov 95 18:11:20 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa09865; 14 Nov 95 18:10:55 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa17131; 14 Nov 95 17:31:25 EST Received: from curie.if.usp.br by RI.CMU.EDU id aa17268; 14 Nov 95 17:31:00 EST Received: (from copelli@localhost) by curie.if.usp.br (8.7.1/8.7.1) id UAA01640 for Connectionists@CS.CMU.EDU; Tue, 14 Nov 1995 20:30:34 -0200 (EDT) Date: Tue, 14 Nov 1995 20:30:34 -0200 (EDT) From: Mauro Copelli da Silva Message-Id: <199511142230.UAA01640@curie.if.usp.br> To: Connectionists@cs.cmu.edu Subject: On-line learning paper FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/copelli.equivalence.ps.Z *** PAPER ANNOUNCEMENT *** The following paper is available by anonymous ftp from the pub/neuroprose directory of the archive.cis.ohio-state.edu host (see instructions below). It is 27 pages long and has been submitted to Physical Review E. Comments are welcomed. EQUIVALENCE BETWEEN LEARNING IN PERCEPTRONS WITH NOISY EXAMPLES AND TREE COMMITTEE MACHINES Mauro Copelli, Osame Kinouchi and Nestor Caticha Instituto de Fisica, Universidade de Sao Paulo CP 66318, 05389-970 Sao Paulo, SP, Brazil e-mail: copelli,osame,nestor@if.usp.br Abstract We study learning from single presentation of examples ({\em incremental} or {\em on-line} learning) in single-layer perceptrons and tree committee machines (TCMs). Lower bounds for the perceptron generalization error as a function of the noise level $\epsilon$ in the teacher output are calculated. We find that optimal local learning in a TCM with $K$ hidden units is simply related to optimal learning in a simple perceptron with a corresponding noise level $\epsilon(K)$. For large number of examples and finite $K$ the generalization error decays as $\alpha_{cm}^{-1}$, where $\alpha_{cm}$ is the number of examples per adjustable weight in the TCM. We also show that on-line learning is possible even in the $K\rightarrow\infty$ limit, but with the generalization error decaying as $\alpha_{cm}^{-1/2}$. The simple Hebb rule can also be applied to the TCM, but now the error decays as $\alpha_{cm}^{-1/2}$ for finite $K$ and $\alpha_{cm}^{-1/4}$ for $K\rightarrow\infty$. Exponential decay of the generalization error in both the perceptron learning from noisy examples and in the TCM is obtained by using the learning by queries strategy. ****************** How to obtain a copy ************************* unix> ftp archive.cis.ohio-state.edu User: anonymous Password: (type your e-mail address) ftp> cd pub/neuroprose ftp> binary ftp> get copelli.equivalence.ps.Z ftp> quit unix> uncompress copelli.equivalence.ps.Z unix> lpr copelli.equivalence.ps (or however you print PostScript files) **PLEASE DO NOT REPLY DIRECTLY TO THIS MESSAGE** From edelman@wisdom.weizmann.ac.il Wed Nov 15 03:57:18 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 15 Nov 95 03:57:15 -0600; AA14065 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 15 Nov 95 03:57:13 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa03481; 14 Nov 95 18:21:47 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa03479; 14 Nov 95 18:09:36 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa09851; 14 Nov 95 18:08:56 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa04005; 14 Nov 95 1:53:22 EST Received: from narkis.wisdom.weizmann.ac.il by RI.CMU.EDU id aa13673; 14 Nov 95 1:52:56 EST Received: from lachesis.wisdom.weizmann.ac.il (lachesis.wisdom.weizmann.ac.il [132.76.80.188]) by narkis.wisdom.weizmann.ac.il (8.6.5/mail.byaddr) with ESMTP id IAA10470 for ; Tue, 14 Nov 1995 08:53:26 +0200 From: Edelman Shimon Received: (edelman@localhost) by lachesis.wisdom.weizmann.ac.il (8.6.9/8.6.5) id GAA19020; Tue, 14 Nov 1995 06:54:55 GMT Date: Tue, 14 Nov 1995 06:54:55 GMT Message-Id: <199511140654.GAA19020@lachesis.wisdom.weizmann.ac.il> To: connectionists@cs.cmu.edu Subject: TR available: RFs From Hyperacuity to Recognition Retrieval information: FTP-host: eris.wisdom.weizmann.ac.il (132.76.80.53) FTP-pathname: /pub/watt-rfs.ps.Z URL: ftp://eris.wisdom.weizmann.ac.il/pub/watt-rfs.ps.Z 28 pages; 519 KB compressed, 2.6 MB uncompressed. Comments welcome at URL mailto:edelman@wisdom.weizmann.ac.il ---------------------------------------------------------------------- Receptive Fields for Vision: from Hyperacuity to Object Recognition Weizmann Institute CS-TR 95-29, 1995; to appear in VISION, R. J. Watt, ed., MIT Press, 1996. Shimon Edelman Dept. of Applied Mathematics and Computer Science The Weizmann Institute of Science Rehovot 76100, ISRAEL http://eris.wisdom.weizmann.ac.il/~edelman Many of the lower-level areas in the mammalian visual system are organized retinotopically, that is, as maps which preserve to a certain degree the topography of the retina. A unit that is a part of such a retinotopic map normally responds selectively to stimulation in a well-delimited part of the visual field, referred to as its {\em receptive field} (RF). Receptive fields are probably the most prominent and ubiquitous computational mechanism employed by biological information processing systems. This paper surveys some of the possible computational reasons behind the ubiquity of RFs, by discussing examples of RF-based solutions to problems in vision, from spatial acuity, through sensory coding, to object recognition. ---------------------------------------------------------------------- -Shimon From pfbaldi@cco.caltech.edu Thu Nov 16 20:38:58 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 16 Nov 95 20:38:50 -0600; AA22615 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 16 Nov 95 20:38:48 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab06274; 16 Nov 95 19:03:13 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa06257; 16 Nov 95 18:32:16 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa11434; 16 Nov 95 18:27:46 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa07112; 15 Nov 95 13:47:10 EST Received: from accord.cco.caltech.edu by CS.CMU.EDU id aa25481; 15 Nov 95 13:45:31 EST Received: by accord.cco.caltech.edu (8.6.12/DEI:4.45) id KAA27407; Wed, 15 Nov 1995 10:45:11 -0800 Date: Wed, 15 Nov 1995 10:45:09 -0800 (PST) From: Pierre Baldi To: Connectionists@cs.cmu.edu Cc: brunak@cbs.dtu.dk, krogh@sanger.ac.uk, stolorz@telerobotics.jpl.nasa.gov, stormo@boulder.colorado.edu, asl@t13.lanl.gov, jain@arris.com, pfbaldi@accord.cco.caltech.edu, stormo@exon.biotech.washington.edu Subject: Tal Grossman Memorial Workshop in Vail (NIPS95) Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII NIPS95 TAL GROSSMAN MEMORIAL WORKSHOP MACHINE LEARNING APPROACHES IN COMPUTATIONAL MOLECULAR BIOLOGY December 1, 1995 Vail, CO CURRENT LIST OF SCHEDULED PRESENTATIONS: Alan Lapedes Neural Network Representations of Empirical Protein Potentials. Gary Stormo The Use of Neural Networks for Identification of Common Domains by Maximizing Specificity. Ajay N. Jain Machine Learning Techniques for Drug Design: Lead Discovery, Lead Optimization, and Screening Strategies. Anders Krogh Maximum Entropy Weighting of Aligned Sequences of Proteins or DNA. Paul Stolorz Applying Dynamic Programming Ideas to Monte Carlo Sampling. Soren Brunak Bendability of Exons and Introns in Human DNA. Pierre Baldi Mining Data Bases of Fragments with HMMs. CURRENT LIST OF ABSTRACTS: Alan Lapedes (Los Alamos National Laboratory) asl@t13.lanl.gov Neural Network Representations of Empirical Protein Potentials. Recently, there has been considerable interest in deriving and applying knowledge-based, empirical potential functions for proteins. These empirical potentials have been derived from the statistics of interacting, spatially neighboring residues, as may be obtained from databases of known protein crystal structures. We employ neural networks to redefine empirical potential functions from the point of view of discrimination functions. This approach generalizes previous work, in which simple frequency counting statistics are used on a database of known protein structures. This generalization allows us to avoid restriction to strictly pairwise interactions. Instead of frequency counting to fix adjustable parameters, one now optimizes an objective function involving a parameterized probability distribution. We show how our method reduces to previous work in special situations, illustrating in this context the relationship of neural networks to statistical methodology. A key feature in the approach we advocate is the development of a representation to describe the location of interacting residues that exist in a sphere of small fixed radius around each residue. This is a natural ``shape representation'' for the interaction neighborhoods of protein residues. We demonstrate that this shape representation and the network's improved abilities enhances discrimination over that obtained by previous methodologies. This work is with Robert Farber and the late Tal Grossman (Los Alamos National Laboratory). Gary Stormo (University of Colorado, Boulder) stormo@exon.biotech.washington.edu The Use of Neural Networks for Identification of Common Domains by Maximizing Specificity. We describe an unsupervised learning procedure in which the objective to be maximized is ``specificity'', defined as the probability of obtaining a particular set of strings within a much larger collection of background strings. We demonstrate its use for identifying protein binding sites on unaligned DNA sequences, common sequence/structure motifs in RNA and common motifs in protein sequences. The idea behind the ``specificity'' criterion it to discover a probability distribution for strings such that the difference between the probabilities of the particular strings and the background strings is maximized. Both the probability distribution and the set of particular strings need to be discovered; the probability distribution can be any allowable distribution over the string alphabet, and the particular strings are contained within a set of longer strings, but their locations are not known in advance. Previous methods have viewed this problem as one of multiple alignment, whereas our method is more flexible in the types of patterns that can be allowed and in the treatment of the background strings. When the patterns are linearly separable from the background, a simple Perceptron works well to identify the patterns. We are currently testing more complicated networks for more complicated patterns. This work is in collaboration with Alan Lapedes of Los Alamos National Laboratory and the Santa Fe Institute, and John Heumann of Hewlett-Packard. Ajay N. Jain (Arris Pharmaceutical Corporation) jain@arris.com Machine Learning Techniques for Drug Design: Lead Discovery, Lead Optimization, and Screening Strategies. At its core, the drug discovery process involves designing small organic molecules that satisfy the physical constraints of binding to a specific site on a particular protein (usually an enzyme or receptor). Machine learning techniques can play a significant role in all phases of the process. When the structure of the protein is known, it is possible to "dock" candidate molecules into the structure and compute the likelihood that a molecule will bind well. Fundamentally, this is a thermodynamic event that is too complicated to simulate accurately. Machine learning techniques can be used to empirically construct functions that are predictive of binding affinities. Similarly, when no protein structure is known, but there exists some data on molecules exhibiting a range of binding affinities, it is possible to use machine learning techniques to capture the 3D pattern that is responsible for binding. Lastly, in cases where one has capacity to make large numbers of small molecules (libraries) to screen against multiple diverse protein targets, one can use clustering techniques to design maximally diverse libraries. This talk will briefly discuss each of these techniques in the context of drug discovery at Arris Pharmaceutical Corporation. Anders Krogh (The Sanger Centre) krogh@sanger.ac.uk Maximum Entropy Weighting of Aligned Sequences of Proteins or DNA. In a family of proteins or other biological sequences like DNA the various subfamilies are often very unevenly represented. For this reason a scheme for assigning weights to each sequence can greatly improve performance at tasks such as database searching with profiles or other consensus models based on multiple alignments. A new weighting scheme for this type of database search is proposed. In a statistical description of the searching problem it is derived from the maximum entropy principle. It can be proved that, in a certain sense, it corrects for uneven representation. It is shown that finding the maximum entropy weights is an easy optimization problem for which standard techniques are applicable. Paul Stolorz (Jet Propulsion Laboratory, Caltech) stolorz@telerobotics.jpl.nasa.gov Applying Dynamic Programming Ideas to Monte Carlo Sampling. Monte Carlo sampling methods developed originally for physics and chemistry calculations have turned out to be very useful heuristics for problems in fields such as computational biology, traditional computer science and statistics. Macromolecular structure prediction and alignment, combinatorial optimization, and more recently probabilistic inference, are classic examples of their use. This talk will swim against the tide a bit by showing that computer science, in the guise of dynamic programming, can in turn supply substantial insight into the Monte Carlo process. This insight allows the construction of powerful novel Monte Carlo methods for a range of calculations in areas such as computational biology, computational vision and statistical inference. The methods are especially useful for problems plagued by multiple modes in the integrand, and for problems containing important, though not overwhelming, long-range information. Applications to protein folding, and to generalized Hidden Markov Models, will be described to illustrate how to systematically implement and test these algorithms. Soren Brunak (The Technical University of Denmark) brunak@cbs.dtu.dk Bendability of Exons and Introns in Human DNA. We analyze the sequential structure of human exons and introns by hidden Markov models. We find that exons -- besides the reading frame -- hold a specific periodic pattern. The pattern has the triplet consensus: non-T(A/T)G and a minimal periodicity of roughly 10 nucleotides. It is not a consequence of the nucleotide statistics in the three codon positions, nor of the previously well known periodicity caused by the encoding of alpha-helices in proteins. Using DNA triplet bendability parameters from DNase I experiments, we show that the pattern corresponds to a periodic `in-phase' bending potential towards the major groove of the DNA. Similarly, nucleosome positioning data show that the consensus triplets have a preference for locations on a bent double helix where the major groove faces inward and is compressed. We discuss the relation between the bending potential of coding regions and its importance for the recognition of genes by the transcriptional machinery. This work is in collaboration with P. Baldi (Caltech), Y. Chauvin (Net-ID, Inc.), Anders Krogh (The Sanger Centre). Pierre Baldi (Caltech) pfbaldi@ccosun.caltech.edu Mining Data Bases of Fragments with HMMs. Hidden Markov Model (HMM) techniques are applied to the problem of mining large data bases of protein fragments. The study is focused on one particular protein family, the G-Protein-Coupled Receptors (GPCR). A large data base is first constructed, by randomly extracting fragments from the entire SWISS-PROT data base, at different lengths, positions, and simulated noise levels, in a way that roughly matches other existing, but not always publicly accessible, data bases. A HMM trained on the GPCR family is then used to score all the fragments, in terms of their negative log-likelihood. The discrimination power of the HMM is assessed, and quantitative results are derived on how performance degrades, as a function of fragment length, truncation position, and noise level, and on how to set discrimination thresholds. The raw score performance is further improved by deriving additional filters, based on the structure of the alignments of the fragments to the HMM. This work is in collaboration with Y. Chauvin (Net-ID, Inc.), F. Tobin and A. Williams (SmithKline Beecham). From tibs@utstat.toronto.edu Thu Nov 16 20:38:59 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 16 Nov 95 20:38:52 -0600; AA22617 Message-Id: <9511170238.AA01843@lucy.cs.wisc.edu> Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 16 Nov 95 20:38:50 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ac06274; 16 Nov 95 19:03:57 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab06257; 16 Nov 95 18:32:19 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa11440; 16 Nov 95 18:28:27 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa08888; 15 Nov 95 15:41:13 EST Received: from utstat.toronto.edu by EDRC.CMU.EDU id aa11323; 15 Nov 95 15:40:38 EST From: tibs@utstat.toronto.edu Date: Wed, 15 Nov 95 15:40 EST To: Connectionists@cs.cmu.edu Subject: new tech report available Model search and inference by bootstrap ``bumping'' Robert Tibshirani and Keith Knight University of Toronto We propose a bootstrap-based method for searching through a space of models. The technique is well suited to complex, adaptively fitted models: it provides a convenient method for finding better local minima, for resistant fitting, and for optimization under constraints. Applications to regression, classification and density estimation are described. The collection of models can also be used to form a confidence set for the true underlying model, using a generalization of Efron's percentile interval. We also provide results on the asymptotic behaviour of bumping estimates. Available at http://utstat.toronto.edu/reports/tibs or ftp: utstat.toronto.edu in pub/tibs/bumping.ps ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Rob Tibshirani, Dept of Preventive Med & Biostats, and Dept of Statistics Univ of Toronto, Toronto, Canada M5S 1A8. Phone: 416-978-4642 (PMB), 416-978-0673 (stats). FAX: 416 978-8299 tibs@utstat.toronto.edu. ftp: //utstat.toronto.edu/pub/tibs http://www.utstat.toronto.edu/~tibs/home.html +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ From juergen@idsia.ch Thu Nov 16 20:39:00 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 16 Nov 95 20:38:48 -0600; AB22611 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 16 Nov 95 20:38:46 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa06253; 16 Nov 95 18:54:02 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa06251; 16 Nov 95 18:26:50 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa11414; 16 Nov 95 18:26:26 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa22713; 15 Nov 95 4:36:00 EST Received: from chsun.eunet.ch by EDRC.CMU.EDU id aa08844; 15 Nov 95 4:35:51 EST Received: from fava.idsia.ch by chsun.eunet.ch (8.6.10/1.34) id KAA01846; Wed, 15 Nov 1995 10:35:44 +0100 Received: from kraut.idsia.ch by fava.idsia.ch (4.1/SMI-4.1) id AA03192; Wed, 15 Nov 95 10:35:42 +0100 Date: Wed, 15 Nov 95 10:35:42 +0100 From: Juergen Schmidhuber Message-Id: <9511150935.AA03192@fava.idsia.ch> To: Connectionists@cs.cmu.edu Subject: response: Levin search My response to David Wolpert's response to my response: Although it is true that ``we have *no* a priori reason to believe that targets with low Kolmogorov complexity (or anything else) are/not likely to occur in the real world'', it's also true that Levin search (LS) has the optimal order of search complexity for a broad class of *non-incremental* search problems (David did not agree). This is a well-known fact of theoretical computer science. Why is LS as efficient as any other search algorithm? Because LS effectively *runs* all the other search algorithms, but in a smart way that prevents it from loosing too much time with "wrong" search algorithms. David is right, however, by saying that ``In practice, it may (or may not) be a good idea to use an algorithm that searches for low Levin complexity rather than one that works by other means.'' One reason for this is: in practice, your problem size is always limited, and you *do* have to worry about the (possibly huge) constant buried in the notion of "optimal order of search complexity". Note that all my recent comments refer to *non-incremental* search --- for the moment, I am not addressing generalization issues. Can LS help us to improve certain kinds of *incremental* search and learning? We are currently trying to figure this one out. Juergen Schmidhuber From scheler@ICSI.Berkeley.EDU Thu Nov 16 20:39:02 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 16 Nov 95 20:38:57 -0600; AA22633 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 16 Nov 95 20:38:54 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ae06274; 16 Nov 95 19:05:51 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa06259; 16 Nov 95 18:35:55 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa11489; 16 Nov 95 18:34:57 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa22333; 16 Nov 95 12:45:46 EST Received: from icsia.ICSI.Berkeley.EDU by CS.CMU.EDU id aa05591; 16 Nov 95 12:44:45 EST Received: from tiramisu.ICSI.Berkeley.EDU (scheler@tiramisu.ICSI.Berkeley.EDU [128.32.201.72]) by icsia.ICSI.Berkeley.EDU (8.6.12/HUB+V8$Revision: 1.23 $) with ESMTP id JAA06638 for ; Thu, 16 Nov 1995 09:44:41 -0800 From: Gabriele Scheler Received: (scheler@localhost) by tiramisu.ICSI.Berkeley.EDU (8.6.12/1.8) id JAA15103 for connectionists@cs.cmu.edu; Thu, 16 Nov 1995 09:44:40 -0800 Date: Thu, 16 Nov 1995 09:44:40 -0800 Message-Id: <199511161744.JAA15103@tiramisu.ICSI.Berkeley.EDU> To: connectionists@cs.cmu.edu Subject: Job Notice ***********JOB NOTICE ******** 1 - 2 full-time positions for research associates are available in the area of connectionist natural language modeling in the Department of Computer Science, Technical University of Munich, Germany. One project concerns text-based learning, the other (pending final approval) combines modeling of language acquisition with situated learning. Several short-term or half-time paid positions for graduate students are also available. Knowledge of German is helpful, but not essential. Positions may start as early as February 1, 1996. A full job description with reference to relevant www-sites will be mailed to interested persons in Mid-December. Anyone interested in these positions should direct an informal request for further details to Dr Gabriele Scheler ICSI 1947 Center Street, Berkeley 94704-1198 scheler@icsi.berkeley.edu From thrun+@HEAVEN.LEARNING.CS.CMU.EDU Thu Nov 16 20:39:07 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 16 Nov 95 20:38:55 -0600; AA22625 Message-Id: <9511170238.AA01848@lucy.cs.wisc.edu> Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 16 Nov 95 20:38:52 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ad06274; 16 Nov 95 19:05:02 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ac06257; 16 Nov 95 18:32:56 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa11445; 16 Nov 95 18:28:48 EST Received: from HEAVEN.LEARNING.CS.CMU.EDU by B.GP.CS.CMU.EDU id aa12749; 15 Nov 95 20:40:13 EST To: connectionists@cs.cmu.edu, intcon@phoenix.ee.unsw.edu.au Subject: 2 papers available Date: Wed, 15 Nov 95 20:39:04 EST From: thrun+@HEAVEN.LEARNING.CS.CMU.EDU Sender: thrun+@HEAVEN.LEARNING.CS.CMU.EDU Dear Colleagues: I am happy to announce two new papers: Lifelong Learning: A Case Study Sebastian Thrun Machine learning has not yet succeeded in the design of robust learning algorithms that generalize well from very small datasets. In contrast, humans often generalize correctly from only a single training example, even if the number of potentially relevant features is large. To do so, they successfully exploit knowledge acquired in previous learning tasks, to bias subsequent learning. This paper investigates learning in a lifelong context. Lifelong learning addresses situations where a learner faces a stream of learning tasks. Such scenarios provide the opportunity for synergetic effects that arise if knowledge is transferred across multiple learning tasks. To study the utility of transfer, several approaches to lifelong learning are proposed and evaluated in an object recognition domain. It is shown that all these algorithms generalize consistently more accurately from scarce training data than comparable "single-task" approaches. World Wide Web URL: http://www.cs.cmu.edu/~thrun/papers/thrun.lll_case_study.ps.Z ------------------------------------------------------------------- Clustering Learning Tasks and the Selective Cross-Task Transfer of Knowledge Sebastian Thrun and Joseph O'Sullivan Recently, there has been an increased interest in machine learning methods that learn from more than one learning task. Such methods have repeatedly found to outperform conventional, single-task learning algorithms when learning tasks are appropriately related. To increase robustness of these approaches, methods are desirable that can reason about the relatedness of individual learning tasks, in order to avoid the danger arising from tasks that are unrelated and thus potentially misleading. This paper describes the task-clustering (TC) algorithm. TC clusters learning tasks into classes of mutually related tasks. When facing a new thing to learn, TC first determines the most related task cluster, then exploits information selectively from this task cluster only. An empirical study carried out in a mobile robot domain shows that TC outperforms its unselective counterpart in situations where only a small number of tasks is relevant. World Wide Web URL: http://www.cs.cmu.edu/~thrun/papers/thrun.TC.ps.Z ------------------------------------------------------------------- INSTRUCTIONS FOR RETRIEVAL: (a) If you have access to the World Wide Web, you can retrieve the documents from my homepage (URL: http://www.cs.cmu.edu/~thrun, follow the paper link) or access them directly: netscape http://www.cs.cmu.edu/~thrun/papers/thrun.lll_case_study.ps.Z netscape http://www.cs.cmu.edu/~thrun/papers/thrun.TC.ps.Z (b) If you instead wish to retrieve the documents via anonymous ftp, follow these instructions: unix> ftp uran.informatik.uni-bonn.de user: anonymous passwd: aaa@bbb.ccc ftp> cd pub/user/thrun ftp> bin ftp> get thrun.lll_case_study.ps.Z ftp> get thrun.TC.ps.Z ftp> bye unix> uncompress thrun.lll_case_study.ps.Z unix> uncompress thrun.TC.ps.Z unix> lpr thrun.lll_case_study.ps.Z unix> lpr thrun.TC.ps.Z (c) Hard-copies can be obtained directly from Technical Reports Computer Science Department Carnegie Mellon University 5000 Forbes Ave Pittsburgh, PA 15213 Email: reports@cs.cmu.edu Please refer to the first paper as TR CMU-CS-95-208 and the second paper as TR CMU-CS-95-209 Comments are welcome! Sebastian Thrun From barberd@helios.aston.ac.uk Thu Nov 16 20:39:08 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 16 Nov 95 20:39:00 -0600; AA22639 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 16 Nov 95 20:38:57 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id af06274; 16 Nov 95 19:06:43 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa06262; 16 Nov 95 18:40:00 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa11427; 16 Nov 95 18:27:27 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa07001; 15 Nov 95 13:39:52 EST Received: from hermes.aston.ac.uk by EDRC.CMU.EDU id aa10733; 15 Nov 95 13:39:19 EST Received: from sun.aston.ac.uk (actually host dendrite.aston.ac.uk) by hermes.aston.ac.uk with SMTP (PP); Wed, 15 Nov 1995 18:44:15 +0000 From: barberd Message-Id: <6923.9511151840@sun.aston.ac.uk> Subject: Paper available To: Connectionists@cs.cmu.edu Date: Wed, 15 Nov 1995 18:40:48 +0000 (GMT) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1377 The following paper, (a version of which was submitted to Europhysics Letters) is available by anonymous ftp (instructions below). FINITE SIZE EFFECTS IN ON-LINE LEARNING OF MULTI-LAYER NEURAL NETWORKS David Barber{2}, Peter Sollich{1} and David Saad{2} {1} Department of Physics, University of Edinburgh, EH9 3JZ, UK {2} Neural Computing Research Group, Aston University, Birmingham B4 7ET, United Kingdom email: D.Barber@aston.ac.uk Abstract We complement the recent progress in thermodynamic limit analyses of mean on-line gradient descent learning dynamics in multi-layer networks by calculating the fluctuations possessed by finite dimensional systems. Fluctuations from the mean dynamics are largest at the onset of specialisation as student hidden unit weight vectors begin to imitate specific teacher vectors, and increase with the degree of symmetry of the initial conditions. Including a term to stimulate asymmetry in the learning process typically significantly decreases finite size effects and training time. Ftp instructions ftp cs.aston.ac.uk User: anonymous Password: (type your e-mail address) ftp> cd neural/barberd ftp> binary ftp> get online.ps.Z ftp> quit unix> uncompress online.ps.Z **PLEASE DO NOT REPLY DIRECTLY TO THIS MESSAGE** From saadd@helios.aston.ac.uk Thu Nov 16 20:39:09 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 16 Nov 95 20:39:03 -0600; AA22644 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 16 Nov 95 20:38:59 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ag06274; 16 Nov 95 19:07:28 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa06266; 16 Nov 95 18:45:35 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa11453; 16 Nov 95 18:30:14 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa18042; 16 Nov 95 6:44:56 EST Received: from hermes.aston.ac.uk by RI.CMU.EDU id aa24442; 16 Nov 95 6:44:13 EST Received: from email.aston.ac.uk by hermes.aston.ac.uk with SMTP (PP); Thu, 16 Nov 1995 11:47:49 +0000 Received: from ganglion by email.aston.ac.uk with SMTP (PP) id <08209-0@email.aston.ac.uk>; Thu, 16 Nov 1995 11:47:36 +0000 From: saadd Message-Id: <644.9511161144@sun.aston.ac.uk> Subject: NIPS workshop: The Dynamics Of On-Line Learning To: Connectionists@cs.cmu.edu Date: Thu, 16 Nov 1995 11:44:25 +0000 (GMT) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 3985 THE DYNAMICS OF ON-LINE LEARNING NIPS workshop, Friday and Saturday, December 2-3, 1995 7:30AM to 9:30AM -- 4:30PM to 6:30PM Organizers: Sara A. Solla (CONNECT, The Niels Bohr Institute) and David Saad (Aston University) On-line learning refers to a scenario in which the couplings of the learning machine are updated after the presentation of each example. The current hypothesis is used to predict an output for the current input; the corresponding error signal is used for weight modification, and the modified hypothesis is used for output prediction at the subsequent time step. This type of algorithm addresses general questions of learning dynamics, and has attracted the attention of both the computational learning theory and the statistical physics communities. Recent progress has provided tools that allow for the investigation of learning scenarios that incorporate many of the aspects of the learning of complex tasks: multilayer architectures, noisy data, regularization through weight decay, the use of momentum, tracking changing environments, presentation order when cycling repeatedly through a finite training set... An open and somewhat controversial question to be discussed in the workshop is the role of the learning rate in controlling the evolution and convergence of the learning process. The purpose of the workshop is to review the theoretical tools available for the analysis of on-line learning, to evaluate the current state of research in the field, and to predict possible contributions to the understanding and description of real world learning scenarios. We also seek to identify future research directions using these methods, their limitations and expected difficulties. The topics to be addressed in this workshop can be grouped as follows: 1) The investigation of on-line learning from the point of view of stochastic approximation theory. This approach is based on formulating a master equation to describe the dynamical evolution of a probability density which describes the ensemble of trained networks in the space of weights of the student network. (Todd Leen, Bert Kappen, Jenny Orr) 2) The investigation of on-line learning from the point of view of statistical mechanics. This approach is based on the derivation of dynamical equations for the overlaps among the weight vectors associated with the various hidden units in both student and teacher networks. The dynamical evolution of the overlaps provides a detailed characterization of the learning process and determines the generalization error. (Sara Solla, David Saad, Peter Riegler, David Barber, Ansgar West, Naama Barkai, Jason Freeman, Adam Prugel-Bennett) 3) The identification of optimal strategies for on-line learning. Most of the work has concentrated on the learning of classification tasks. A recent Bayesian formulation of the problem provides a unified derivation of optimal on-line equations. (Shun-ichi Amari, Nestor Caticha, Manfred Opper) The names in parenthesis identify the speakers. A list of additional participants includes Michael Kearns, Yann Le Cun, Yoshiyuki Kabashima, Mauro Copelli, Noboru Murata, Klaus Mueller. PROGRAM FOR THE NIPS WORKSHOP ON "THE DYNAMICS OF ON-LINE LEARNING" Friday ------ Morning: 7:30AM to 9:30AM Introduction: Todd Leen Speakers: Jenny Orr Bert Kappen Afternoon: 4:30PM to 6:30PM Speakers: Shun-ichi Amari Nestor Caticha Manfred Opper Saturday -------- Morning: 7:30AM to 9:30AM Introduction: Sara Solla/David Saad Speakers: David Barber Ansgar West Peter Riegler Afternoon: 4:30PM to 6:30PM Speakers: Adam Prugel-Bennett Jason Freeman Peter Riegler Naama Barkai More details, abstracts and references can be found on: http://neural-server.aston.ac.uk/nips95/workshop.html From robert@fit.qut.edu.au Fri Nov 17 13:31:54 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 17 Nov 95 13:31:48 -0600; AA06815 Received: from bilby.cs.uwa.oz.au by lucy.cs.wisc.edu; Fri, 17 Nov 95 13:31:43 -0600 Received: from (mafm@parma.cs.uwa.oz.au [130.95.1.7]) by cs.uwa.oz.au (8.6.8/8.5) with SMTP id SAA17847; Fri, 17 Nov 1995 18:20:48 +0800 Message-Id: <199511171020.SAA17847@cs.uwa.oz.au> From: robert@fit.qut.edu.au (Robert Andrews) To: reinforce@cs.uwa.edu.au Subject: CFP: AISB96 Rule Extraction From Neural Networks Workshop Date: Fri, 17 Nov 1995 13:33:14 +1000 (EST) ============================================================= FIRST CALL FOR PAPERS AISB-96 WORKSHOP Society for the Study of Artificial Intelligence and Simulation of Behaviour (SSAISB) University of Sussex, Brighton, England April 2, 1996 -------------------------------------------- RULE-EXTRACTION FROM TRAINED NEURAL NETWORKS -------------------------------------------- Robert Andrews Neurocomputing Research Centre Queensland University of Technology Brisbane 4001 Queensland, Australia Phone: +61 7 864-1656 Fax: +61 7 864-1969 E-mail: robert@fit.qut.edu.au Joachim Diederich Neurocomputing Research Centre Queensland University of Technology Brisbane 4001 Queensland, Australia Phone: +61 7 864-2143 Fax: +61 7 864-1801 E-mail: joachim@fit.qut.edu.au Lee Giles NEC Research Institute 4 Independence Way Princeton, NJ 08540 The objective of the workshop is to provide a discussion platform for researchers interested in Artificial Neural Networks (ANNs), Artificial Intelligence (AI) and Cognitive Science. The workshop should be of considerable interest to computer scientists and engineers as well as to cognitive scientists and people interested in ANN applications which require a justification of a classification or inference. INTRODUCTION It is becoming increasingly apparent that without some form of explanation capability, the full potential of trained Artificial Neural Networks may not be realised. The problem is an inherent inability to explain in a comprehensible form, the process by which a given decision or output generated by an ANN has been reached. For Artificial Neural Networks to gain a even wider degree of user acceptance and to enhance their overall utility as learning and generalisation tools, it is highly desirable if not essential that an `explanation' capability becomes an integral part of the functionality of a trained ANN. Such a requirement is mandatory if, for example, the ANN is to be used in what are termed as `safety critical' applications such as airlines and power stations. In these cases it is imperative that a system user be able to validate the output of the Artificial Neural Network under all possible input conditions. Further the system user should be provided with the capability to determine the set of conditions under which an output unit within an ANN is active and when it is not, thereby providing some degree of transparency of the ANN solution. Craven & Shavlik (1994) define the rule-extraction from neural networks task as follows: "Given a trained neural network and the examples used to train it, produce a concise and accurate symbolic description of the network." The following discussion of the importance of rule-extraction algorithms is based on this definition. THE IMPORTANCE OF RULE-EXTRACTION ALGORITHMS Since rule extraction from trained Artificial Neural Networks comes at a cost in terms of resources and additional effort, an early imperative in any discussion is to delineate the reasons why rule extraction is an important, if not mandatory, extension of conventional ANN techniques. The merits of including rule extraction techniques as an adjunct to conventional Artificial Neural Network techniques include: Data exploration and the induction of scientific theories Over time neural networks have proven to be extremely powerful tools for data exploration with the capability to discover previously unknown dependencies and relationships in data sets. As Craven and Shavlik (1994) observe, `a (learning) system may discover salient features in the input data whose importance was not previously recognised.' However, even if a trained Artificial Neural Network has learned interesting and possibly non-linear relationships, these relationships are encoded incomprehensibly as weight vectors within the trained ANN and hence cannot easily serve the generation of scientific theories. Rule-extraction algorithms significantly enhance the capabilities of ANNs to explore data to the benefit of the user. Provision of a `user explanation' capability Experience has shown that an explanation capability is considered to be one of the most important functions provided by symbolic AI systems. In particular, the salutary lesson from the introduction and operation of Knowledge Based systems is that the ability to generate even limited explanations (in terms of being meaningful and coherent) is absolutely crucial for the user-acceptance of such systems. In contrast to symbolic AI systems, Artificial Neural Networks have no explicit declarative knowledge representation. Therefore they have considerable difficulty in generating the required explanation structures. It is becoming increasingly apparent that the absence of an `explanation' capability in ANN systems limits the realisation of the full potential of such systems and it is this precise deficiency that the rule extraction process seeks to redress. Improving the generalisation of ANN solutions Where a limited or unrepresentative data set from the problem domain has been used in the ANN training process, it is difficult to determine when generalisation can fail even with evaluation methods such as cross-validation. By being able to express the knowledge embedded within the trained Artificial Neural Network as a set of symbolic rules, the rule-extraction process may provide an experienced system user with the capability to anticipate or predict a set of circumstances under which generalisation failure can occur. Alternatively the system user may be able to use the extracted rules to identify regions in input space which are not represented sufficiently in the existing ANN training set data and to supplement the data set accordingly. A CLASSIFICATION SCHEME FOR RULE EXTRACTION ALGORITHMS The method of classification proposed here is in terms of: (a) the expressive power of the extracted rules; (b) the `translucency' of the view taken within the rule extraction technique of the underlying Artificial Neural Network units; (c) the extent to which the underlying ANN incorporates specialised training regimes; (d) the `quality' of the extracted rules; and (e) the algorithmic `complexity' of the rule extraction/rule refinement technique. The `translucency' dimension of classification is of particular interest. It is designed to reveal the relationship between the extracted rules and the internal architecture of the trained ANN. It comprises two basic categories of rule extraction techniques viz `decompositional' and `pedagogical' and a third - labelled as `eclectic' - which combines elements of the two basic categories. The distinguishing characteristic of the `decompositional' approach is that the focus is on extracting rules at the level of individual (hidden and output) units within the trained Artificial Neural Network. Hence the `view' of the underlying trained Artificial Neural Network is one of `transparency'. The translucency dimension - `pedagogical' is given to those rule extraction techniques which treat the trained ANN as a `black box' ie the view of the underlying trained Artificial Neural Network is `opaque'. The core idea in the `pedagogical' approach is to `view rule extraction as a learning task where the target concept is the function computed by the network and the input features are simply the network's input features'. Hence the `pedagogical' techniques aim to extract rules that map inputs directly into outputs. Where such techniques are used in conjunction with a symbolic learning algorithm, the basic motif is to use the trained Artificial Neural Network to generate examples for the learning algorithm. As indicated above the proposed third category in this classification scheme are composites which incorporate elements of both the `decompositional' and `pedagogical' (or `black-box') rule extraction techniques. This is the `eclectic' group. Membership in this category is assigned to techniques which utilise knowledge about the internal architecture and/or weight vectors in the trained Artificial Neural Network to complement a symbolic learning algorithm. An ancillary problem to that of rule extraction from trained ANNs is that of using the ANN for the `refinement' of existing rules within symbolic knowledge bases. The goal in rule refinement is to use a combination of ANN learning and rule extraction techniques to produce a `better' (ie a `refined') set of symbolic rules which can then be applied back in the original problem domain. In the rule refinement process, the initial rule base (ie what may be termed `prior knowledge') is inserted into an ANN by programming some of the weights. The rule refinement process then proceeds in the same way as normal rule extraction viz (1) train the network on the available data set(s); and (2) extract (in this case the `refined') rules - with the proviso that the rule refinement process may involve a number of iterations of the training phase rather than a single pass. DISCUSSION POINTS FOR WORKSHOP PARTICIPANTS 1. Decompositional vs. learning approaches to rule- extraction from ANNs - What are the advantages and disadvantages w.r.t. performance, solution time, computational complexity, problem domain etc. Are decompositional approaches always dependent on a certain ANN architecture? 2. Rule-extraction from trained neural networks vs. symbolic induction. What are the relative strength and weaknesses? 3. What are the most important criteria for rule quality? 4. What are the most suitable representation languages for extracted rules? How does the extraction problem vary across different languages? 5. What is the relationship between rule-initialisation (insertion) and rule-extraction? For instance, are these equivalent or complementary processes? How important is rule-refinement by neural networks? 6. Rule-extraction from trained neural networks and computational learning theory. Is generating a minimal rule-set which mimics an ANN a hard problem? 7. Does rule-initialisation result in faster learning and improved generalisation? 8. To what extent are existing extraction algorithms limited in their applicability? How can these limitations be addressed? 9. Are there any interesting rule-extraction success stories? That is, problem domains in which the application of rule-extraction methods has resulted in an interesting or significant advance. ACKNOWLEDGEMENT Many thanks to Mark Craven, and Alan Tickle for comments on earlier versions of this proposal. RELEVANT PUBLICATIONS Andrews, R Diederich, J and Tickle, A.B.: A survey and critique of techniques for extracting rules from trained artificial neural networks. To appear: Knowledge-Based Systems, 1995 (ftp:fit.qut.edu.au//pub/NRC/ps/QUTNRC-95-01- 02.ps.Z) Andrews, R and Geva, S: `Rule extraction from a constrained error back propagation MLP' Proc. 5th Australian Conference on Neural Networks Brisbane Queensland (1994) pp 9-12 Andrews, R and Geva, S `Inserting and extracting knowledge from constrained error back propagation networks' Proc. 6th Australian Conference on Neural Networks Sydney NSW (1995) Craven, M W and Shavlik , J W `Using sampling and queries to extract rules from trained neural networks' Machine Learning: Proceedings of the Eleventh International Conference (San Francisco CA) (1994) (in print) Diederich, J `Explanation and artificial neural networks' International Journal of Man-Machine Studies Vol 37 (1992) pp 335-357 Fu, L M `Neural networks in computer intelligence' McGraw Hill (New York) (1994) Fu, L M `Rule generation from neural networks' IEEE Transactions on Systems, Man, and Cybernetics Vol 28 No 8 (1994) pp 1114-1124 Gallant, S `Connectionist expert systems' Communications of the ACM Vol 31 No 2 (February 1988) pp 152-169 Giles, C L and Omlin C W `Rule refinement with recurrent neural networks' Proc. of the IEEE International Conference on Neural Networks (San Francisco CA) (March 1993) pp 801-806 Giles, C L and Omlin C W `Extraction, insertion, and refinement of symbolic rules in dynamically driven recurrent networks' Connection Science Vol 5 Nos 3 and 4 (1993) pp 307-328 Giles, C L, Miller, C B, Chen, D, Chen, H, Sun, G Z and Lee, Y C `Learning and extracting finite state automata with second-order recurrent neural networks' Neural Computation Vol 4 (1992) pp 393-405 Hayward, R.; Pop, E.; Diederich, J.: Extracting Rules for Grammar Recognition from Cascade-2 Networks. Proceeding, IJCAI-95 Workshop on Machine Learning and Natural Language Processing. McMillan, C, Mozer, M C and Smolensky, P `The connectionist scientist game: rule extraction and refinement in a neural network' Proc. of the Thirteenth Annual Conference of the Cognitive Science Society (Hillsdale NJ) 1991 Omlin, C W, Giles, C L and Miller, C B `Heuristics for the extraction of rules from discrete time recurrent neural networks' Proc. of the International Joint Conference on Neural Networks (IJCNN'92) (Baltimore MD) Vol 1 (1992) pp 33 Pop, E, Hayward, R, and Diederich, J `RULENEG: extracting rules from a trained ANN by stepwise negation' QUT NRC (December 1994) Sestito, S and Dillon, T `Automated knowledge acquisition of rules with continuously valued attributes' Proc. 12th International Conference on Expert Systems and their Applications (AVIGNON'92) (Avignon France) (May 1992) pp 645-656. Sestito, S and Dillon, T `Automated knowledge acquisition' Prentice Hall (Australia) (1994) Thrun, S B `Extracting Provably Correct Rules From Artificial Neural Networks' Technical Report IAI-TR-93-5 Institut fur Informatik III Universitat Bonn (1994) Tickle, A B, Orlowski, M, and Diederich, J `DEDEC: decision detection by rule extraction from neural networks' QUT NRC (September 1994) Towell, G and Shavlik, J `The Extraction of Refined Rules Tresp, V, Hollatz, J and Ahmad, S `Network Structuring and Training Using Rule-based Knowledge' Advances In Neural Information Processing Vol 5 (1993) pp871-878 SUBMISSION OF WORKSHOP EXTENDED ABSTRACTS/PAPERS Authors are invited to submit 3 copies of either an extended abstract or full paper relating to one of the topic areas listed above. Papers should be written in English in single column format and should be limited to no more than eight, (8) sides of A4 paper including figures and references. Centered at the top of the first page should be complete title, author name(s), affiliation(s), and mailing and email address(es), followed by blank space, abstract(15-20 lines), and text. Please include the following information in an accompanying cover letter: Full title of paper, presenting author's name, address, and telephone and fax numbers, authors e-mail address. Submission Deadline is January 15th,1996 with notification to authors by 31st January,1996. For further information, inquiries, and paper submissions please contact: Robert Andrews Queensland University of Technology GPO Box 2434 Brisbane Q. 4001. Australia. phone +61 7 864-1656 fax +61 7 864-1969 email robert@fit.qut.edu.au More information about the AISB-96 workshop series is available from: ftp: ftp.cogs.susx.ac.uk pub/aisb/aisb96 WWW: http://www.cogs.susx.ac.uk/aisb/aisb96/CFP/rule_extraction.html WORKSHOP PARTICIPATION CHARGES The workshop fees are listed below. Note that these fees include lunch. Student charges are shown in brackets. AISB NON-ASIB MEMBERS MEMBERS 1 Day Workshop 65 (45) 80 LATE REGISTRATION: 85 (60) 100 PROGRAM COMMITTEE MEMBERS R. Andrews, Queensland University of Technology A. Tickle, Queensland University of Technology S. Sestito, DSTO, Australia J. Shavlik, University of Wisconsin From koza@cs.stanford.edu Fri Nov 17 14:43:22 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 17 Nov 95 14:43:19 -0600; AA08327 Received: from bilby.cs.uwa.oz.au by lucy.cs.wisc.edu; Fri, 17 Nov 95 14:43:14 -0600 Received: from (mafm@parma.cs.uwa.oz.au [130.95.1.7]) by cs.uwa.oz.au (8.6.8/8.5) with SMTP id SAA17818; Fri, 17 Nov 1995 18:18:27 +0800 Message-Id: <199511171018.SAA17818@cs.uwa.oz.au> From: John Koza To: Reinforce@cs.uwa.edu.au Subject: Final GP-96 CFP Date: Thu, 16 Nov 95 6:38:18 PST ------------------------------------------------ Paper Submission Deadline: January 10, 1996 (Wednesday) ------------------------------------------------ CALL FOR PAPERS AND PARTICIPATION Genetic Programming 1996 Conference (GP-96) July 28 - 31 (Sunday - Wednesday), 1996 Fairchild Auditorium - Stanford University - Stanford, California Proceedings will be published by The MIT Press In cooperation with the Association for Computing Machinery (ACM), SIGART, the IEEE Neural Network Council, and the American Association for Artificial Intelligence. Genetic programming is a domain-independent method for evolving computer programs that solve, or approximately solve, problems. Starting with a primordial ooze of thousands of randomly created programs composed of functions and terminals appropriate to a problem, a genetic population is progressively evolved over many generations by applying the Darwinian principle of survival of the fittest, a sexual recombination operation, and occasional mutation. This first genetic programming conference will feature contributed papers, tutorials, invited speakers, and informal meetings. Topics include, but are not limited to, applications of genetic programming, theoretical foundations of genetic programming, implementation issues, parallelization techniques, technique extensions, implementations of memory and state, representation issues, new operators, architectural evolution, evolution of mental models, cellular encoding, evolution of machine language programs, evolvable hardware, combinations with other machine learning techniques, and relations to biology and cognitive systems. ------------------------------------------------- HONORARY CHAIR: John Holland, University of Michigan INVITED SPEAKERS: John Holland, University of Michigan and David E. Goldberg, University of Illinois GENERAL CHAIR: John Koza, Stanford University PUBLICITY CHAIR: Patrick Tufts, Brandeis University ------------------------------------------------- SPECIAL PROGRAM CHAIRS: The main focus of the conference (and about two-thirds of the papers) will be on genetic programming. In addition, papers describing recent developments in the following closely related areas of evolutionary computation (particularly those addressing issues common to various areas of evolutionary computation) will be reviewed by special program committees appointed and supervised by the following special program chairs. - GENETIC ALGORITHMS: David E. Goldberg, University of Illinois, Urbana, Illinois - CLASSIFIER SYSTEMS: Rick Riolo, University of Michigan - EVOLUTIONARY PROGRAMMING AND EVOLUTION STRATEGIES: David Fogel, University of California, San Diego, California ------------------------------------------------- TUTORIALS -Sunday July 28 9:15 AM - 11:30 AM - Genetic Algorithms - David E. Goldberg, University of Illinois - Machine Language Genetic Programming - Peter Nordin, University of Dortmund, Germany - Genetic Programming using Mathematica P Robert Nachbar P Merck Research Laboratories - Introduction to Genetic Programming - John Koza, Stanford University ------------------------------------------------- Sunday July 28 1:00 PM - 3: 15 PM - Classifier Systems- Robert Elliott Smith, University of Alabama - Evolutionary Computation for Constraint Optimization - Zbigniew Michalewicz, University of North Carolina - Advanced Genetic Programming - John Koza, Stanford University ------------------------------------------------- Sunday July 28 3:45 PM - 6 PM - Evolutionary Programming and Evolution Strategies - David Fogel, University of California, San Diego - Cellular Encoding P Frederic Gruau, Stanford University (via videotape) and David Andre, Stanford University (in person) - Genetic Programming with Linear Genomes (one hour) - Wolfgang Banzhaf, University of Dortmund, Germany -JECHO - Terry Jones, Santa Fe Institute ------------------------------------------------- Tuesday July 30 - 3 PM - 5:15PM - Neural Networks - David E. Rumelhart, Stanford University - Machine Learning - Pat Langley, Stanford University -JMolecular Biology for Computer Scientists - Russ B. Altman, Stanford University ------------------------------------------------- INFORMATION FOR SUBMITTING PAPERS The deadline for receipt at the physical mail address below of seven (7) copies of each submitted paper is Wednesday, January 10, 1996. Papers are to be in single-spaced, 12- point type on 8 1/2" x 11" or A4 paper (no e-mail or fax) with full 1" margins at top, bottom, left, and right. Papers are to contain ALL of the following 9 items, within a maximum of 10 pages, IN THIS ORDER: (1) title of paper, (2) author name(s), (3) author physical address(es), (4) author e-mail address(es), (5) author phone number(s), (6) a 100-200 word abstract of the paper, (7) the paper's category (chosen from one of the following five alternatives: genetic programming, genetic algorithms, classifier systems, evolutionary programming, or evolution strategies), (8) the text of the paper (including all figures and tables), and (9) bibliography. All other elements of the paper (e.g., acknowledgments, appendices, if any) must come within the maximum of 10 pages. Review criteria will include significance of the work, novelty, sufficiency of information to permit replication (if applicable), clarity, and writing quality. The first-named (or other designated) author will be notified of acceptance or rejection by approximately Monday February 26, 1996. The style of the camera-ready paper will be identical to that of the 1994 Simulation of Adaptive Behavior conference published by the MIT Press. Depending on the number, subject, and content of the submitted papers, the program committee may decide to allocate different number of pages to various accepted papers. The deadline for the camera-ready, revised version of accepted papers will be announced, but will be approximately Wednesday March 20, 1996. Proceedings will be published by The MIT Press and will be available at the conference (and, if requested, by priority mail to registered conference attendees with U.S. addresses just prior to the conference). One author will be expected to present each accepted paper at the conference. ------------------------------------------------- FOR MORE INFORMATION ABOUT THE GP-96 CONFERENCE: On the World Wide Web: http://www.cs.brandeis.edu/~zippy/gp-96.html or via e-mail at gp@aaai.org. Conference operated by Genetic Programming Conferences, Inc. (a California not-for-profit corporation). ------------------------------------------------- FOR MORE INFORMATION ABOUT GENETIC PROGRAMMING IN GENERAL: http://www-cs- faculty.stanford.edu/~koza/. ------------------------------------------------- FOR MORE INFORMATION ABOUT DISCOUNTED TRAVEL : For further information regarding special GP-96 airline and car rental rates, please contact Conventions in America at e-mail flycia@balboa.com; or phone 1-800-929-4242; or phone 619-678-3600; or FAX 619-678-3699. ------------------------------------------------- FOR MORE INFORMATION ABOUT THE SAN FRANCISCO BAY AREA AND SILICON VALLEY AREA SIGHTS: Try the Stanford University home page at http://www.stanford.edu/, the Hyperion Guide at http://www.hyperion.com/ba/sfbay.html; the Palo Alto weekly at http://www.service.com/PAW/home.html; the California Virtual Tourist at http://www.research.digital.com/SRC/virtual- tourist/California.html; and the Yahoo Guide of San Francisco at http://www.yahoo.com/Regional_Information/States/Califo rnia/San_Francisco. ------------------------------------------------- FOR MORE INFORMATION ABOUT CONTEMPORANEOUS WEST COAST CONFERENCES: Information about the AAAI-96 conference on August 4 P 8 (Sunday P Thursday), 1996, in Portland, Oregon can be found at http://www.aaai.org/. For information on the International Conference on Knowledge Discovery and Data Mining (KDD-96) in Portland, Oregon, on August 3- 5, 1996: http://www-aig.jpl.nasa.gov/kdd96. Information about the Foundations of Genetic Algorithms (FOGA) workshop on August 3 P 5 (Saturday P Monday), 1996, in San Diego, California can be found at http://www.aic.nrl.navy.mil/galist/foga/ or by contacting belew@cs.wisc.edu. ------------------------------------------------- FOR MORE INFORMATION ABOUT MEMBERSHIP IN THE ACM, AAAI, or IEEE: For information about ACM membership, try http://www.acm.org/; for information about SIGART, try http://sigart.acm.org/; for AAAI membership, go to http://www.aaai.org/; and for membership in the IEEE Computer Society, go to http://www.computer.org. ------------------------------------------------- PHYSICAL MAIL ADDRESS FOR GP-96: GP-96 Conference, c/o American Association for Artificial Intelligence, 445 Burgess Drive, Menlo Park, CA 94025. PHONE: 415-328-3123. FAX: 415-321-4457. WWW: http://www.aaai.org/. E-MAIL: gp@aaai.org. ------------------------------------------------ REGISTRATION FORM FOR GENETIC PROGRAMMING 1996 CONFERENCE TO BE HELD ON JULY 28 P 31, 1996 AT STANFORD UNIVERSITY First Name _________________________ Last Name_______________ Affiliation________________________________ Address__________________________________ ________________________________________ City__________________________ State/Province _________________ Zip/Postal Code____________________ Country__________________ Daytime telephone__________________________ E-Mail address_____________________________ Conference registration fee includes copy of proceedings, attendance at 4 tutorials of your choice, syllabus books for 4 tutorials, conference reception, and admission to conference. Students must send legible proof of full-time student status. Conference proceedings will be mailed to registered attendees with U.S. mailing addresses via 2-day U.S. priority mail 1 P 2 weeks prior to the conference at no extra charge (at addressee's risk). If you are uncertain as to whether you will be at that address at that time or DO NOT WANT YOUR PROCEEDINGS MAILED to you at the above address for any other reason, your copy of the proceedings will be held for you at the conference registration desk if you CHECK HERE ____. Postmarked by May 15, 1996: Student P ACM, IEEE, or AAAI Member $195 Regular P ACM, IEEE, or AAAI Member $395 Student P Non-member $215 Regular P Non-member $415 Postmarked by June 26, 1996: Student P ACM, IEEE, or AAAI Member $245 Regular P ACM, IEEE, or AAAI Member $445 Student P Non-member $265 Regular P Non-member $465 Postmarked later or on-site: Student P ACM, IEEE, or AAAI Member $295 Regular P ACM, IEEE, or AAAI Member $495 Student P Non-member $315 Regular P Non-member $515 Member number: ACM # ___________ IEEE # _________ AAAI # _________ Total fee (enter appropriate amount) $ _________ __ Check or money order made payable to "AAAI" (in U.S. funds) __ Mastercard __ Visa __ American Express Credit card number __________________________________________ Expiration Date ___________ Signature _________________________ TUTORIALS: Check off a box for one tutorial from each of the 4 columns: Sunday July 28, 1996 P 9:15 AM - 11:30 AM __ Genetic Algorithms __ Machine Language GP __ GP using Mathematica __ Introductory GP Sunday July 28, 1996 P 1:00 PM - 3: 15 PM __ Classifier Systems __ EC for Constraint Optimization __ Advanced GP Sunday July 28, 1996 P 3:45 PM - 6 PM __ Evolutionary Programming and Evolution Strategies __ Cellular Encoding __ GP with Linear Genomes __ ECHO Tuesday July 30, 1996 P3:00 PM - 5:15PM __ Neural Networks __ Machine Learning __ Molecular Biology for Computer Scientists __ Check here for information about housing and meal package at Stanford University. __ Check here for information on student travel grants. No refunds will be made; however, we will transfer your registration to a person you designate upon notification. SEND TO: GP-96 Conference, c/o American Association for Artificial Intelligence, 445 Burgess Drive, Menlo Park, CA 94025. PHONE: 415- 328-3123. FAX: 415-321-4457. E-MAIL: gp@aaai.org. WWW: http://www.aaai.org/. ------------------------------------------------- PROGRAM COMMITTEE Russell J. Abbott California State University, Los Angeles and The Aerospace Corporation Hojjat Adeli Ohio State University Dennis Allison Stanford University Lee Altenberg Hawaii Institute of Geophysics and Planetology David Andre Stanford University Peter J. Angeline Loral Federal Systems Wolfgang Banzhaf University of Dortmund, Germany Rik Belew University of California at San Diego Samy Bengio Centre National d'Etudes des Telecommunications, France Forrest H. Bennett III Genetic Algorithms Technology Corporation Scott Brave Stanford University Bill P. Buckles Tulane University Walter Cedeno Primavera Systems Inc. Nichael Lynn Cramer BBN System and Technologies Jason Daida University of Michigan Patrik D'haeseleer University of New Mexico Marco Dorigo Universite' Libre de Bruxelles Bertrand Daniel Dunay System Dynamics International Andrew N. Edmonds Science in Finance Ltd., UK H.H. Ehrenburg CWI, The Netherlands Frank D. Francone FRISEC P Francone & Raymond Institute for the Study of Evolutionary Computation, Germany Adam P. Fraser University of Salford Alex Fukunaga University of California, Los Angeles Frederic Gruau Stanford University Richard J. Hampo Ford Motor Company Simon Handley Stanford University Thomas D. Haynes The University of Tulsa Hitoshi Hemmi ATR, Kyoto, Japan Vasant Honavar Iowa State University Thomas Huang University of Illinois Hitoshi Iba Electrotechnical Laboratory, Japan Christian Andrew Johnson Department of Economics, University of Santiago Martin A. Keane Econometrics Inc. Mike Keith Allen Bradley Controls Maarten Keijzer Kenneth E. Kinnear, Jr. Adaptive Computing Technology W. B. Langdon University College, London David Levine Argonne National Laboratory Kenneth Marko Ford Motor Company Martin C. Martin Carnegie Mellon University Sidney R Maxwell III Nicholas Freitag McPhee University of Minnesota, Morris David Montana BBN System and Technologies Heinz Muehlenbein GMD Research Center, Germany Robert B. Nachbar Merck Research Laboratories Peter Nordin University of Dortmund, Germany Howard Oakley Institute of Naval Medicine, UK Franz Oppacher Carleton University, Ottawa Una-May O`Reilly Carleton University, Ottawa Michael Papka Argonne National Laboratory Timothy Perkis Frederick E. Petry Tulane University Bill Punch Michigan State University Justinian P. Rosca University of Rochester Conor Ryan University College Cork, Ireland Malcolm Shute University of Brighton, UK Eric V. Siegel Columbia University Karl Sims Andrew Singleton Creation Mechanics Lee Spector Hampshire College Walter Alden Tackett Neuromedia Astro Teller Carnegie Mellon University Marco Tomassini Ecole Polytechnique Federale de Lausanne Patrick Tufts Brandeis University V. Rao Vemuri University of Califonia at Davis Peter A. Whigham Australia Darrell Whitley Colorado State University Man Leung Wong Chinese University of Hong Kong Alden H. Wright University of Montana Byoung-Tak Zhang GMD, Germany From cas-cns@PARK.BU.EDU Sat Nov 18 03:14:06 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 18 Nov 95 03:13:59 -0600; AA16987 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 18 Nov 95 03:13:55 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa06274; 16 Nov 95 19:02:22 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa06255; 16 Nov 95 18:27:42 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa11421; 16 Nov 95 18:26:51 EST Received: from MAILBOX.SRV.CS.CMU.EDU by B.GP.CS.CMU.EDU id aa05479; 15 Nov 95 11:45:26 EST Received: from MAILBOX.SRV.CS.CMU.EDU by MAILBOX.SRV.CS.CMU.EDU id aa09877; 15 Nov 95 11:43:58 EST Received: from PARK.BU.EDU by MAILBOX.SRV.CS.CMU.EDU id aa09874; 15 Nov 95 11:43:33 EST Received: from mac-duffee.bu.edu by cns.bu.edu (8.6.10/BU_Server-1.1) id LAA28690; Wed, 15 Nov 1995 11:18:03 -0500 Message-Id: <199511151618.LAA28690@cns.bu.edu> Date: Wed, 15 Nov 1995 11:20:00 -0500 From: CAS/CNS Subject: BU - Cognitive & Neural Systems Organization: Boston University - Dept. of Cognitive & Neural Systems Reply-To: cas-cns@PARK.BU.EDU X-Newsreader: Yet Another NewsWatcher 2.0.6b4 Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: Apparently-To: ************************************************************** DEPARTMENT OF COGNITIVE AND NEURAL SYSTEMS (CNS) AT BOSTON UNIVERSITY ************************************************************** Ennio Mingolla, Acting Chairman, 1995-96 Stephen Grossberg, Chairman Gail A. Carpenter, Director of Graduate Studies The Boston University Department of Cognitive and Neural Systems offers comprehensive graduate training in the neural and computational principles, mechanisms, and architectures that underlie human and animal behavior, and the application of neural network architectures to the solution of technological problems. Applications for Fall, 1996, admission and financial aid are now being accepted for both the MA and PhD degree programs. To obtain a brochure describing the CNS Program and a set of application materials, write, telephone, or fax: DEPARTMENT OF COGNITIVE & NEURAL SYSTEMS 677 Beacon Street Boston, MA 02215 617/353-9481 (phone) 617/353-7755 (fax) or send via email your full name and mailing address to: rll@cns.bu.edu Applications for admission and financial aid should be received by the Graduate School Admissions Office no later than January 15. Late applications will be considered until May 1; after that date applications will be considered only as special cases. Applicants are required to submit undergraduate (and, if applicable, graduate) transcripts, three letters of recommendation, and Graduate Record Examination (GRE) scores. The Advanced Test should be in the candidate's area of departmental specialization. GRE scores may be waived for MA candidates and, in exceptional cases, for PhD candidates, but absence of these scores may decrease an applicant's chances for admission and financial aid. Non-degree students may also enroll in CNS courses on a part-time basis. Description of the CNS Department: The Department of Cognitive and Neural Systems (CNS) provides advanced training and research experience for graduate students interested in the neural and computational principles, mechanisms, and architectures that underlie human and animal behavior, and the application of neural network architectures to the solution of technological problems. Students are trained in a broad range of areas concerning cognitive and neural systems, including vision and image processing; speech and language understanding; adaptive pattern recognition; cognitive information processing; self-organization; associative learning and long-term memory; computational neuroscience; nerve cell biophysics; cooperative and competitive network dynamics and short-term memory; reinforcement, motivation, and attention; adaptive sensory-motor control and robotics; active vision; and biological rhythms; as well as the mathematical and computational methods needed to support advanced modeling research and applications. The CNS Department awards MA, PhD, and BA/MA degrees. The CNS Department embodies a number of unique offerings. It has developed a curriculum that features 15 interdisciplinary graduate courses each of which integrates the psychological, neurobiological, mathematical, and computational information needed to theoretically investigate fundamental issues concerning mind and brain processes and the applications of neural networks to technology. Each course is typically taught once a week in the evening to make the program available to qualified students, including working professionals, throughout the Boston area. Nine additional research course are also offered. In these courses, one or two students meet regularly with one or two professors to pursue advanced reading and collaborative research. Students develop a coherent area of expertise by designing a program that includes courses in areas such as Biology, Computer Science, Engineering, Mathematics, and Psychology, in addition to courses in the CNS Department. The CNS Department prepares students for PhD thesis research with scientists in one of several Boston University research centers or groups, and with Boston-area scientists collaborating with these centers. The unit most closely linked to the department is the Center for Adaptive Systems (CAS). Students interested in neural network hardware work with researchers in CNS, the College of Engineering, and at MIT Lincoln Laboratory. Other research resources include distinguished research groups in neurophysiology, neuroanatomy, and neuropharmacology at the Medical School and the Charles River campus; in sensory robotics, biomedical engineering, computer and systems engineering, and neuromuscular research within the Engineering School; in dynamical systems within the Mathematics Department; in theoretical computer science within the Computer Science Department; and in biophysics and computational physics within the Physics Department. In addition to its basic research and training program, the Department offers a colloquium series, seminars, conferences, and special interest groups which bring many additional scientists from both experimental and theoretical disciplines into contact with the students. The CNS Department is moving in October, 1995 into its own new four-story building, which features a full range of offices, laboratories, classrooms, library, lounge, and related facilities for exclusive CNS use. 1995-96 CAS MEMBERS and CNS FACULTY: Jelle Atema Professor of Biology Director, Boston University Marine Program (BUMP) PhD, University of Michigan Sensory physiology and behavior Aijaz Baloch Research Associate of Cognitive and Neural Systems PhD, Electrical Engineering, Boston University Neural modeling of role of visual attention of recognition, learning and motor control, computational vision, adaptive control systems, reinforcement learning Helen Barbas Associate Professor, Department of Health Sciences, Boston University PhD, Physiology/Neurophysiology, McGill University Organization of the prefrontal cortex, evolution of the neocortex Jacob Beck Research Professor of Cognitive and Neural Systems PhD, Psychology, Cornell University Visual Perception, Psychophysics, Computational Models Daniel H. Bullock Associate Professor of Cognitive and Neural Systems and Psychology PhD, Psychology, Stanford University Real-time neural systems, sensory-motor learning and control, evolution of intelligence, cognitive development Gail A. Carpenter Professor of Cognitive and Neural Systems and Mathematics Director of Graduate Studies, Department of Cognitive and Neural Systems PhD, Mathematics, University of Wisconsin, Madison Pattern recognition, categorization, machine learning, differential equations Laird Cermak Professor of Neuropsychology, School of Medicine Professor of Occupational Therapy, Sargent College Director, Memory Disorders Research Center, Boston Veterans Affairs Medical Center PhD, Ohio State University Michael A. Cohen Associate Professor of Cognitive and Neural Systems and Computer Science Director, CAS/CNS Computation Labs PhD, Psychology, Harvard University Speech and language processing, measurement theory, neural modeling, dynamical systems H. Steven Colburn Professor of Biomedical Engineering PhD, Electrical Engineering, Massachusetts Institute of Technology Audition, binaural interaction, signal processing models of hearing William D. Eldred III Associate Professor of Biology BS, University of Colorado; PhD, University of Colorado, Health Science Center Visual neural biology Paolo Gaudiano Assistant Professor of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Computational and neural models of vision and adaptive sensory-motor control Jean Berko Gleason Professor of Psychology AB, Radcliffe College; AM, PhD, Harvard University Psycholinguistics Douglas Greve Research Associate of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Stephen Grossberg Wang Professor of Cognitive and Neural Systems Professor of Mathematics, Psychology, and Biomedical Engineering Director, Center for Adaptive Systems Chairman, Department of Cognitive and Neural Systems PhD, Mathematics, Rockefeller University Theoretical biology, theoretical psychology, dynamical systems, applied mathematics Frank Guenther Assistant Professor of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Biological sensory-motor control, spatial representation, speech production Thomas G. Kincaid Chairman and Professor of Electrical, Computer and Systems Engineering, College of Engineering PhD, Electrical Engineering, Massachusetts Institute of Technology Signal and image processing, neural networks, non-destructive testing Nancy Kopell Professor of Mathematics PhD, Mathematics, University of California at Berkeley Dynamical systems, mathematical physiology, pattern formation in biological/physical systems Ennio Mingolla Associate Professor of Cognitive and Neural Systems and Psychology Acting Chairman 1995-96, Department of Cognitive and Neural Systems PhD, Psychology, University of Connecticut Visual perception, mathematical modeling of visual processes Alan Peters Chairman and Professor of Anatomy and Neurobiology, School of Medicine PhD, Zoology, Bristol University, United Kingdom Organization of neurons in the cerebral cortex, effects of aging on the primate brain, fine structure of the nervous system Andrzej Przybyszewski Senior Research Associate of Cognitive and Neural Systems MSc, Technical Warsaw University; MA, University of Warsaw; PhD, Warsaw Medical Academy Adam Reeves Adjunct Professor of Cognitive and Neural Systems Professor of Psychology, Northeastern University PhD, Psychology, City University of New York Psychophysics, cognitive psychology, vision William Ross Research Associate of Cognitive and Neural Systems BSc, Cornell University; MA, PhD, Boston University Mark Rubin Research Assistant Professor of Cognitive and Neural Systems Research Physicist, Naval Air Warfare Center, China Lake, CA (on leave) PhD, Physics, University of Chicago Neural networks for vision, pattern recognition, and motor control Robert Savoy Adjunct Associate Professor of Cognitive and Neural Systems Scientist, Rowland Institute for Science PhD, Experimental Psychology, Harvard University Computational neuroscience; visual psychophysics of color, form, and motion perception Eric Schwartz Professor of Cognitive and Neural Systems; Electrical, Computer and Systems Engineering; and Anatomy and Neurobiology PhD, High Energy Physics, Columbia University Computational neuroscience, machine vision, neuroanatomy, neural modeling Robert Sekuler Adjunct Professor of Cognitive and Neural Systems Research Professor of Biomedical Engineering, College of Engineering, BioMolecular Engineering Research Center Jesse and Louis Salvage Professor of Psychology, Brandeis University AB,MA, Brandeis University; Sc.M., PhD, Brown University Allen Waxman Adjunct Associate Professor of Cognitive and Neural Systems Senior Staff Scientist, MIT Lincoln Laboratory PhD, Astrophysics, University of Chicago Visual system modeling, mobile robotic systems, parallel computing, optoelectronic hybrid architectures James Williamson Research Associate of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Image processing and object recognition. Particular interests are: dynamic binding, self-organization, shape representation, and classification Jeremy Wolfe Adjunct Associate Professor of Cognitive and Neural Systems Associate Professor of Ophthalmology, Harvard Medical School Psychophysicist, Brigham & Women's Hospital, Surgery Dept. Director of Psychophysical Studies, Center for Clinical Cataract Research PhD, Massachusetts Institute of Technology From schwenk@robo.jussieu.fr Sat Nov 18 03:14:09 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 18 Nov 95 03:14:04 -0600; AA16993 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 18 Nov 95 03:14:01 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa08113; 17 Nov 95 16:13:46 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa08111; 17 Nov 95 15:58:00 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa12202; 17 Nov 95 15:57:12 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa16104; 17 Nov 95 14:32:32 EST Received: from shiva.jussieu.fr by CS.CMU.EDU id aa16959; 17 Nov 95 14:31:28 EST Received: from tgv.robo.jussieu.fr (tgv.robo.jussieu.fr [134.157.3.17]) by shiva.jussieu.fr (8.6.10/jtpda-5.1) with ESMTP id UAA03941 for ; Fri, 17 Nov 1995 20:30:11 +0100 Received: from lea.robo.jussieu.fr (lea.robo.jussieu.fr [134.157.3.8]) by tgv.robo.jussieu.fr (8.6.10/jtpda-5.1) with ESMTP id UAA14981 ; Fri, 17 Nov 1995 20:23:20 +0100 Received: from (schwenk@localhost) by lea.robo.jussieu.fr (8.6.11/jtpda-5.1) id PAA16089 ; Fri, 17 Nov 1995 15:28:08 -0500 From: Holger Schwenk To: Connectionists Mailing List Subject: paper available (OCR, discriminant tangent distance) Cc: Holger Schwenk Message-Id: <951117202807.22380000.adc15034@lea.robo.jussieu.fr> Date: Fri, 17 Nov 1995 20:28:07 +0000 (WET) X-Mailer: Aide de Camp Mail System [Version 0.9] **DO NOT FORWARD TO OTHER GROUPS** FTP-host: ftp.robo.jussieu.fr FTP-filename: /pub/papers/schwenk.icann95.ps.gz (6 pages, 31k) The following paper, published in International Conference on Artificial Neural Networks (ICANN*95), Springer Verlag, is available via anonymous FTP at the above location. The paper is 6 pages long. Sorry, no hardcopies available. =========================================================================== H. Schwenk and M. Milgram PARC - boite 164 Universite Pierre et Marie Curie 4, place Jussieu 75252 Paris cedex 05, FRANCE ABSTRACT Transformation invariance is known to be fundamental for excellent performances in pattern recognition. One of the most successful approach is tangent distance, originally proposed for a nearest-neighbor algorithm (Simard et al.,1995). The resulting classifier, however, has a very high computational complexity and, perhaps more important, lacks discrimination capabilities. We present a discriminant learning algorithm for a modular classifier based on several autoassociative neural networks. Tangent distance as objective function guarantees efficient incorporation of transformation invariance. The system achieved a raw error rate of 2.6% and a rejection rate of 3.6% on the NIST uppercase letters. ============================================================================ FTP instructions: unix> ftp ftp.robo.jussieu.fr Name: anonymous Password: your full email address ftp> cd pub/papers ftp> bin ftp> get schwenk.icann95.ps.gz ftp> quit unix> gunzip schwenk.icann95.ps.gz unix> lp schwenk.icann95.ps (or however you print postscript) I welcome your comments. --------------------------------------------------------------------- Holger Schwenk PARC - boite 164 tel: (+33 1) 44.27.63.08 Universite Pierre et Marie Curie fax: (+33 1) 44.27.62.14 4, place Jussieu 75252 Paris cedex 05 email: schwenk@robo.jussieu.fr FRANCE --------------------------------------------------------------------- From horne@research.nj.nec.com Sat Nov 18 03:14:13 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 18 Nov 95 03:14:07 -0600; AA17000 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 18 Nov 95 03:14:04 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab08127; 17 Nov 95 16:24:37 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa08120; 17 Nov 95 16:02:08 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa12246; 17 Nov 95 16:01:21 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa16928; 17 Nov 95 15:19:47 EST Received: from zingo.nj.nec.com by EDRC.CMU.EDU id aa21633; 17 Nov 95 15:19:20 EST Received: by zingo (940816.SGI.8.6.9/YDL1.4-910307.16) id PAA19746(zingo); Fri, 17 Nov 1995 15:16:49 -0500 Received: by telluride (940816.SGI.8.6.9/cliff's joyful mailer #2) id PAA02542(telluride); Fri, 17 Nov 1995 15:16:49 -0500 Date: Fri, 17 Nov 1995 15:16:49 -0500 From: Bill Horne Message-Id: <9511171516.ZM2540@telluride> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: Connectionists@cs.cmu.edu Subject: NIPS*95 Workshop on Neural Networks for Signal Processing Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii **** FINAL SCHEDULE **** NIPS*95 Workshop Neural Networks for Signal Processing Friday Dec 1, 1995 Marriott Vail Mountain Resort, Colorado ORGANIZERS Andrew D. Back C. Lee Giles and Bill G. Horne University of Queensland NEC Research Institute back@elec.uq.edu.au {giles,horne}@research.nj.nec.com WORKSHOP AIMS Nonlinear signal processing methods using neural network models form a topic of some recent interest. A common goal is for neural network models to outperform traditional linear and nonlinear models. Many researchers are interested in understanding, analysing and improving the performance of these nonlinear models by drawing from the well established base of linear systems theory and existing knowledge in other areas. How can this be best achieved? In the context of neural network models, a variety of methods have been proposed for capturing the time-dependence of signals. A common approach is to use recurrent connections or time-delays within the network structure. On the other hand, many signal processing techniques have been well developed over the last few decades. Recently, a strong interest has developed in understanding how better signal processing techniques can be developed by considering these different approaches. A major aim of this workshop is to obtain a better understanding of how well this development is proceeding. For example, the different model structures raise the question, "how suitable are the various neural networks for signal processing problems?". The success of some neural network models in signal processing problems indicate that they form a class of potentially powerful modeling methods, yet relatively little is understood about these architectures in the context of signal processing. As an outcome of the workshop it is intended that there should be a summary of current progress and goals for future work in this research area. SCHEDULE OF TALKS ** Session 1 - Speech Focus ** 7:30-7:40: Introduction 7:40-8:10: Herve Bourlard, ICSI and Faculte Polytechnique de Mons, "Hybrid use of hidden Markov models and neural networks for improving state-of-the-art speech recognition systems" 8:10-8:40: John Hogden, Los Alamos National Laboratory, "A maximum likelihood approach to estimating speech articulator positions from speech acoustics" 8:40-9:10: Shun-ichi Amari, A.Cichocki and H. Yang, RIKEN, "Blind separation of signals - Information geometric point of view" 9:10-9:30: Discussion ** Session 2 - Recurrent Network Focus ** 4:30-4:40: Introduction 4:40-5:10: Andrew Back, University of Queensland, "Issues in signal processing relevant to dynamic neural networks" 5:10-5:40: John Steele and Aaron Gordon, Colorado School of Mines, "Hierarchies of recurrent neural networks for signal interpretation with applications" 5:40-6:10: Stephen Piche, Pavilion Technologies, "Discrete Event Recurrent Neural Networks" 6:10-6:30: Open Forum, discussion time. For more information about the workshop see the workshop homepage: http://www.elec.uq.edu.au/~back/nips95ws/nips95ws.html or contact: Andrew D. Back Department of Electrical and Computer Engineering, University of Queensland, Brisbane, Qld 4072. Australia Ph: +61 7 365 3965 Fax: +61 7 365 4999 back@.elec.uq.edu.au C. Lee Giles, Bill G. Horne NEC Research Institute 4 Independence Way Princeton, NJ 08540. USA Ph: 609 951 2642, 2676 Fax: 609 951 2482 {giles,horne}@research.nj.nec.com -- Bill Horne horne@research.nj.nec.com http://www.neci.nj.nec.com/homepages/horne.html PHN: (609) 951-2676 FAX: (609) 951-2482 NEC Research Institute, 4 Independence Way, Princeton, NJ 08540 From at@cogsci.soton.ac.uk Sat Nov 18 21:26:48 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 18 Nov 95 21:26:45 -0600; AA24876 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 18 Nov 95 21:26:43 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab09826; 18 Nov 95 16:05:11 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab09823; 18 Nov 95 15:50:12 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa13010; 18 Nov 95 15:49:59 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa28631; 18 Nov 95 5:40:54 EST Received: from beech.soton.ac.uk by CS.CMU.EDU id aa22092; 18 Nov 95 5:40:46 EST Received: from cogsci.soton.ac.uk (neuro.psy.soton.ac.uk [152.78.226.32]) by beech.soton.ac.uk (8.6.12/hub-8.5a) with SMTP id KAA18500 for ; Sat, 18 Nov 1995 10:40:34 GMT Received: by cogsci.soton.ac.uk (5.x/SMI-SVR4) id AA05833; Sat, 18 Nov 1995 10:34:40 GMT Date: Sat, 18 Nov 1995 10:34:40 GMT From: Adriaan Tijsseling Message-Id: <9511181034.AA05833@cogsci.soton.ac.uk> To: connectionists@cs.cmu.edu Subject: Changes to CogPsy Mailinglist Dear Colleagues, There are a few changes to the CogPsy Mailinglist, but before I outline the changes, first a small description of the mailing list: The cogpsy mailing list is intended for discussion of issues and the dissemination of information important to researchers in all fields of cognitive science, especially connectionist cognitive psychology. Contributions could include: announcements of new techreports, dissertations, theses, conferences, seminars or courses discussions of research issues (including those arising from articles in Noetica) requests for information about bibliographic issues reviews of software or hardware packages The changes are: a new address: contributions should be send to cogpsy@neuro.psy.soton.ac.uk; subscriptions to cogpsy-request@neuro.psy.soton.ac.uk (make sure the Subject: field contains the word "subscribe"). a fusion with Noetica, a electronic journal on cognitive science, the url of which is: http://psych.psy.uq.oz.au/CogPsych/Noetica/ two accompanying webpages: one containing the latest contributions to the list, which is on http://www.soton.ac.uk/~coglab/coglab/CogPsy/ the other is a database of current projects in the field of cognitive science, listed by discipline and containing all information about the projects, including email- and URL-addresses. Feel free to add your own project!. It's on: http://neuro.psy.soton.ac.uk/~at/ With kind regards, Adriaan Tijsseling, CogPsy Moderator From rosen@unr.edu Sat Nov 18 21:36:59 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 18 Nov 95 21:36:56 -0600; AA24939 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 18 Nov 95 21:36:54 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa09826; 18 Nov 95 16:03:33 EST Received: from GS93.SP.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa09823; 18 Nov 95 15:50:10 EST Received: from GS93.SP.CS.CMU.EDU by GS93.SP.CS.CMU.EDU id aa13005; 18 Nov 95 15:49:22 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa21435; 17 Nov 95 18:59:08 EST Received: from archive.ccs.unr.edu by CS.CMU.EDU id aa19801; 17 Nov 95 18:58:53 EST Received: from monitor.ee.unr.edu by archive.ccs.unr.edu (8.6.12/1.34) id QAA24231; Fri, 17 Nov 1995 16:03:13 -0800 Message-Id: <199511180003.QAA24231@archive.ccs.unr.edu> Received: by monitor.ee.unr.edu ($Revision: 1.37.109.26 $/16.2) id AA146082763; Fri, 17 Nov 1995 15:59:23 -0800 Date: Fri, 17 Nov 1995 15:59:23 -0800 To: connectionists@cs.cmu.edu Subject: Missing Data Workshop -- Final Announcement Reply-To: "David B. Rosen" From: "David B. Rosen" This is the final email announcement (with updated list of presentations) for: MISSING DATA: METHODS AND MODELS A NIPS*95 Workshop Friday, December 1, 1995 INTRODUCTION Incomplete or missing data, typically unobserved or unavailable features in supervised learning, is an important problem often encountered in real-world data sets and applications. Assumptions about the missing-data mechanism are often not stated explicitly, for example independence between this mechanism and the values of the (missing or other) features themselves. In the important case of incomplete ~training~ data, one often discards incomplete rows or columns of the data matrix, throwing out some useful information along with the missing data. Ad hoc or univariate methods such as imputing the mean or mode are dangerous as they can sometimes give much worse results than simple discarding. Overcoming the problem of missing data often requires that we model not just the dependence of the output on the inputs, but the inputs among themselves as well. THE WORKSHOP This one-day workshop should provide a valuable opportunity to share and discuss methods and models used for missing data. The following short talks will be presented, with questions and discussion following each. o Leo Breiman, U.C. Berkeley Formal and ad hoc ways of handling missing data o Zoubin Ghahramani, U. Toronto Mixture models and missing data o Steffen Lauritzen and Bo Thiesson, Aalborg U. Learning Bayesian networks from incomplete data o Brian Ripley, Oxford U. Multiple imputation and simulation methods o Brian Ripley, Oxford U. Multiple imputation for neural nets and classification trees o Robert Tibshirani and Geoffrey Hinton, U. Toronto ``Coaching'' variables for regression and classification o Volker Tresp, Siemens AG Missing data: A fundamental problem in learning FURTHER INFORMATION The above is a snapshot of the workshop's web page: http://www.scs.unr.edu/~cbmr/nips/workshop-missing.html A schedule of presentation times is not yet available as of today. Sincerely, (the organizers:) Harry Burke David Rosen New York Medical College, Department of Medicine, Valhalla NY 10595 USA