From rwp@eng.cam.ac.uk Tue Aug 15 13:02:01 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 15 Aug 95 13:01:57 -0500; AA22547 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 15 Aug 95 13:01:50 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa09584; 15 Aug 95 12:48:11 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa09582; 15 Aug 95 12:23:41 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa19009; 15 Aug 95 12:23:04 EDT Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa03894; 15 Aug 95 5:38:56 EDT Received: from spanner.eng.cam.ac.uk by RI.CMU.EDU id aa27996; 15 Aug 95 5:38:27 EDT Received: from dsl.eng.cam.ac.uk (via rwp@dsl.eng.cam.ac.uk [129.169.80.1]) by spanner.eng.cam.ac.uk with ESMTP id KAA05876; Tue, 15 Aug 1995 10:38:21 +0100 Message-Id: <199508150938.6421@dsl.eng.cam.ac.uk> Received: (via rwp@localhost) by dsl.eng.cam.ac.uk id KAA06421; Tue, 15 Aug 1995 10:38:19 +0100 From: Richard Prager Date: Tue, 15 Aug 1995 10:38:19 BST X-Mailer: Mail User's Shell (7.1.1 5/02/90) To: ai-medicine@medmail.Stanford.EDU, connectionists@cs.cmu.edu Subject: Research Position for 1 Year Cambridge UK Euro-PUNCH Project Research Assistant Position for One Year Under the terms of a grant recently awarded to the Euro-PUNCH Project we expect to offer a one year's Research Assistant position in Cambridge to investigate the use of: Neural Networks in the Prediction of Risk in Pregnancy Euro-PUNCH is a collaborative Project funded by the Human Capital and Mobility Programme of the Commission of the European Communities. Thus, the post is available only to a citizen of a European Union Member State (but not British), who wishes to come to work in the United Kingdom. >From the obstetrical point of view, the principal focus of the Euro-PUNCH Project lies in the use of patient-specific measurements of an epidemiological nature (such as maternal age, past obstetrical history, etc.) as well as fetal heart rate recordings, in the forecasting of a number of specific Adverse Pregnancy Outcomes. >From the neural network point of view, the Project involves the design of pattern-processing and classification systems which can be trained to forecast problems in pregnancy. This will involve continuation of work on pattern-classification and regression analysis, using neural networks operating on a very large database of about 1.2 million pregnancies from various European countries. Challenging components of the project include dealing with missing and uncertain variables, sensitivity analysis, variable selection procedures and cluster analysis. Many leading European obstetrical centres are involved in the Euro-PUNCH project, and close collaboration with a number of these will be an essential component of the post offered. Candidates for this post are expected to have a good first degree and preferably a post-graduate degree in a relevant discipline. Come familiarity with medical statistics and neural networks is desirable but not essential. Salary (on the RA scale) will depend on age and experience, and is likely to be in the range of #14,317 to #15,986 per annum. Appointment would be subject to satisfactory health screening. Applications will close on 23rd August 1995. Interviews will be held in Cambridge and are likely to be on Wednesday 30th August 1995. Applications (naming two referees) should be submitted to: Dr Kevin J Dalton PhD FRCOG Division of Materno-Fetal Medicine, Dept. Obstetrics & Gynaecology University of Cambridge, Addenbrooke's Hospital Cambridge CB2 2QQ Tel: +44-1223-410250 Fax: +44-1223-336873 or 215327 e-mail: kjd5@cus.cam.ac.uk Informal enquiries about the project should be directed to: (Obstetric side) Dr Kevin Dalton kjd5@cus.cam.ac.uk (Engineering Side) Dr Niranjan niranjan@eng.cam.ac.uk (Engineering Side) Dr Richard Prager rwp@eng.cam.ac.uk From v.dimitrov@uws.edu.au Tue Aug 15 13:02:10 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 15 Aug 95 13:02:08 -0500; AA22553 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 15 Aug 95 13:01:57 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab09584; 15 Aug 95 12:49:15 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab09582; 15 Aug 95 12:23:42 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa19014; 15 Aug 95 12:23:27 EDT Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa03954; 15 Aug 95 5:50:12 EDT Received: from hotel.uws.EDU.AU by RI.CMU.EDU id aa28035; 15 Aug 95 5:49:23 EDT Received: from [137.154.40.191] by hotel.uws.EDU.AU with SMTP id AA02462 (5.67a/IDA-1.5 for ); Tue, 15 Aug 1995 19:48:20 +1000 Message-Id: <199508150948.AA02462@hotel.uws.EDU.AU> X-Sender: dimitrov@hotel.uws.edu.au Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Tue, 15 Aug 1995 19:49:17 +1000 To: Connectionists@cs.cmu.edu From: Vladimir Dimitrov Subject: FLAMOC'96 International Discourse on FUZZY LOGIC AND THE MANAGEMENT OF COMPLEXITY (FLAMOC'96) Sydney, 15-18 January, 1996 SECOND ANNOUNCEMENT AND FINAL CALL FOR PAPERS FLAMOC'96 is an International Discourse targeted on the growing use of Fuzzy Logic when dealing with Complexity in various fields of applications (Industry, Business, Finance, Management, Ecology, Medicine, Social Science, etc.). FLAMOC'96 intends to contribute insight and foresight regarding creation of innovative and practically efficient ways of implementing Fuzzy Logic in problem situations impregnated with Uncertainty, Intricacy, and Hazard. In the age of an increasing Technological, Environmental and Social Complexity, FLAMOC'96 emphasises the synergy between Fuzzy Logic and Neural Networks, Genetic Algorithms, Non-linear Simulation Techniques, Fractal and Chaos Theory not only in fuzzy engineering practice but also in the search for better understanding the changes around and in us, learning how to handle paradoxes and risk, how to avoid conflicts and look for collaboration and consensus, how to improve our personal and organizational achievements by integrating many diverse and contradictory requirements into coherent, complimentary, and useful outputs. FLAMOC'96 provides a rich tutorial program and an excellent opportunity to meet and talk with world-known experts in Fuzzy Logic and Soft Computing. FLAMOC'96 is the only one of its kind discourse that intents to explore the diversity of practical and theoretical applications of Fuzzy Logic in managing real life Complexity, using Fuzzy Thinking as a bridge between science and the humanities. THE PROGRAM Keynote speaker: Prof. Lotfi Zadeh Invited speakers: Prof. George Klir Prof. Michio Sugeno Three basic streams build the conceptual framework of this discourse: - Soft Computing Technology (Fuzzy Logic, Neural Networks and Genetic Algorithms): with applications in Process Control, Intelligent Manufacturing Systems, Artificial Intelligence (Expert Systems, Decision Support Systems, Knowledge Based Leaning Systems, Robotics), Fuzzy Mathematics, Data Analysis, Linguistics, Biomedical Engineering, Medical Informatics. - Fuzzy Logic in Organising Systems: with applications in Social Science (Consensus Seeking, Conflict Analysis, Human Decision Making, Public Participation, Qualitative Reasoning, Education), Philosophy (PostAristotelean Logic, Postmodernism), Psychology, Economics (Stock Market and Financial Analysis). - Approximate Reasoning in Environmental Applications: Cleaner Production, Environmental Management, Mass Load Analysis, Risk Management, Sustainable Development Practice, Ecology, Ecocybernetics. Participants of FLAMOC'96 are free to suggest other topics that might be of interest to attendees. FLAMOC'96 will include open discussions on selected by participants issues of Soft Computing and Fuzzy Set Theory, System Science and Sustainable Development, Chaos and Complexity, as well as introductory and advanced tutorials hands-on demonstrations international exhibition of fuzzy products lectures visual summaries (poster sessions) multi-paper sessions. FLAMOC'96 provides a special day for business people and managers ("FLAMOC Business Day") with presentations delivered by leading experts and tutorials on how effectively to apply Fuzzy Logic in business, marketing, finance, and management. Participants are invited to submit proposals for presentation in a form that is most appropriate to the subject matter they would cover (e.g. paper, poster presentation, tutorial, lecture, computer demonstration, discussion, exhibit of a fuzzy product, performance on fuzzy music system, etc.) THE ATTENDEES FLAMOC'96 aimed at system and complexity scientists and researchers, control engineers, social scientists, managers in business, industry and government, academics, conflict resolution practitioners and facilitators, bio-medical engineers, environmental managers and environmentalists, fuzzy soft- and hardware specialists, information science professionals and students. INTERNATIONAL PROGRAM COMMITTEE L. Zadeh (USA) - Honorary Chairman V. Dimitrov (Univ. Western Sydney, Hawkesbury, Australia) - Coordinator J. Bezdek (Univ. West Florida, USA) H. Berenji (NASA, USA) Z. Bien (Advanced Inst. Science and Technology, Korea) C. Carlsson (Abo Academy, Finland) E. Cox (Metus Systems Group, USA) J. Dimitrov (Univ. Western Sydney, Nepean, Australia) D. Driankov (Univ. Linkoeping, Sweden) S. Dyer (OrgMetrics, USA) D. Filev (Ford, USA) T. Gedeon (Univ. NSW, Australia) H. Guesgen (Univ. Auckland, New Zealand) M. Gupta (Univ. Saskatchewan, Canada) J. Kacprzyk (Academy of Sciences, Poland) N. Kasabov (Univ. Otago, New Zealand) D. Keweley (Defence Science and Technology Organisation, Australia) P. Kloeden (Deakin Univ., Australia) G. Klir (State Univ. New York, USA) L. Koczy (Techn. Univ. Budapest, Hungary) R. Kowalczyk (CRA, Advanced Technical Development, Australia) V. Kreinovich (Univ. Texas, USA) K. Leung (The Chinese Univ. Hong Kong) D. Lakov (Academy of Sciences, Bulgaria) M. Mizumoto (Osaka EC Univ., Japan) S. Murugesan (Univ.Western Sydney, Macarthur, Australia) A. Patki (Department of Electronics, India) L. Reznik (Victoria Univ. Technology, Australia) A. Ramer (Univ. NSW, Australia) B. Rieger (Univ. Trier, Germany) A. Salski (Univ. Kiel, Germany) M. Smithson (James Cook Univ., Australia) M. Sugeno (Tokyo Inst. Technology, Japan) T. Terano (Tokyo Inst. Technology, Japan) T. Vladimirova (Univ. Surrey, UK) X. Yao (Defence Force Academy, Australia) J. Yen (Texas A&M Univ., USA) ORGANIZING/MANAGEMENT COMMITTEE J. Dimitrov (Univ. Western Sydney, Nepean) - Chair V. Dimitrov (Univ. Western Sydney, Hawkesbury) X. Hu (Univ. Sydney) G. Jaros (Univ. Sydney) K. Kopra (Univ. Western Sydney, Hawkesbury) J. Peperides (Omron Electronics) G. Sheather (Univ. Technology, Sydney) D. Tayler (Hawkesbury Technology) IMPORTANT DATES 15 September 1995 Deadline for Extended Abstract (1-2 pages) Submission. Address for submission: Dr Vladimir Dimitrov School of Social Ecology, UWS-Hawkesbury, Richmond 2753, Australia Fax: +61(45) 701901 Phone: +61(47) 701903 E-mail: v.dimitrov@uws.edu.au 15 October 1995 Preliminary Acceptance 30 November 1995 Deadline for Camera Ready Copy of Full Paper (5 pages) All accepted papers will be published in the FLAMOC'96 Proceedings: "Fuzzy Logic and the Management of Complexity". After the discourse, selected papers will be published in a separate volume. REGISTRATION FEE AUS$ 450 (before 15.11.1995) and AUS$500 (after 15.11.1995). Students' Fee: AU$100. TUTORIALS Tutorials include the following topics: (1) Introduction to Fuzzy Logic (FL) and its Applications. (2) Industrial Applications of FL. (3) Applications of FL in Management Practice. (4) Applications of FL in Busines and Financial Forecasting. (5) Clinical Applications of FL. (6) Application of FL in Environmental Management. (7) Advanced Design Methodology of Fuzzy Systems: Neuro-Fuzzy and Fuzzy-Genetic Systems. (8) Fuzzy Semantics (9) Approximate Reasoning (10) Research Topics in Soft Computing. FEE FOR TUTORIALS: AUS$100 for one selected tutorial topic. AUS$180 for two selected lectures. AUS$250 for three selected lectures. AUS$300 for four or more lectures. Students' fee: AUS$50 (full day tutorials) Potential lectures are invited to submit a one page proposal for tutorial that includes: the background of the lecturer (both in research and lecturing experience), abstract and contents of the proposed lecture (not limited by the above list) to: Dr Xiheng Hu DEE, University of Sydney NSW 2006, Australia Fax: +61(2) 351 3847 Phone: +61(2) 351 6475 E-mail: hxh@ee.su.oz.au not later than 15 September 1995. Lecturers are responsible for preparation and delivery of their lectures as well as preparation of quality handout (such as copies of lecture overheads). Lecturers have FREE REGISTRATION for FLAMOC'96. EXHIBITION For information about the exhibition and space reservation, contact: Mr Kalevi Kopra 6 Boree Rd, Forestville 2087 Australia Fax: +61(2) 975 1943 Phone: +61(2) 451 5728 Proposals for the exhibition to be sent not later than 15 September 1995. FEE FOR EXHIBITION SPACE (for companies): AU$2000 SOCIAL EVENTS You may want to participate in our golf tournament on 17 January 1996 (Fee: AU$50) with fuzzy logic experts from around the world or enjoy a Conference Dinner when cruising in the beautiful Sydney Harbour (Fee: AUS$ 100 ). Or postconference tour to the Great Barrier Reef, Blue Mountains, etc. For overall information about attending at (and participating in the program of) FLAMOC'96, contact: Mrs Judith Dimitrov POBox 91, Richmond 2753 Australia Fax: +61(47) 761616 Phone: +61(47) 761514 ACCOMMODATION AUS$ 105 per room (single or shared by two or three persons) in (1) The Golden Gate Hotel (Reservations: POBox K401, Haymarket, NSW 2000, Australia; Fax +61(2) 281 2213; Phone +61(2)281 6888). (2) Country Comfort Hotel, Sydney Central (Reservations: POBox K963, Haymarket; Fax +61(2) 281 3794; Phone +61(2) 212 2544). The participants must book their staying directly with one of the above hotel as soon as possible. Both hotels are close (at walking distance) to the Graduate School of Business, University of Technology, Sydney: 1-59, Quay St., Haymarket), where FLAMOC'96 will take place, and to all major Sydney tourist attractions. ____________________________________________________________________________ _______ REGISTRATION FORM FLAMOC'96 (to be sent to: FLAMOC'96, POBox 91, Richmond 2753, AUSTRALIA; Fax: +61(47)761 616) Name........................................................................ .......................................... Address..................................................................... ......................................... Company..................................................................... ....................................... Mailing Address..................................................................... ........................... Phone Fax E-mail [ ] I shall particpate in FLAMOC'96. Enclosed is my payment of the Registration Fee (AUS$450 before 15.11.1995; AUS$500 after 15.11.1995; AUS$100 for students). I shall participate in tutorials: [ ] one selected lecture only : (please, indicate the number of the tutorial from the above list of tutorials) [ ] two selected tutorials: (please, indicate which numbers of the tutorials from the above list of tutorials) [ ] three selected tutorials: (please, indicate the corresponding numbers) [ ] four or more tutorials. Enclosed is my payment for the tutorials (AUS$100 for one lecture; AUS$180 for two lectures; AUS$250 for three lectures; AUS$300 for 4 lectures or more; [ ] Students' fee: AUS$50 full day) [ ] I shall exhibit fuzzy products. Enclosed is my payment for the exhibition space: AUS$2000. I shall participate in: [ ]Golf tournament (AUS$50) [ ]Conference Dinner &Harbourcruise (AUS$100) Enclosed is my payment for the golf tournament or/and the Harbourcruise. I am interested in Post Conference Activities such as: [ ] Great Barrier Reef Tour [ ] Blue Mountains Excursion [ ] Others: ________________________________________________________________________________ ________________________________________________________________________________ PAYMENT 1. CREDIT CARD ________Visa ________Master Card __________American Express ________ Card No.......................................................................... .......................... Expiration Date........................................................................ ................ Name of the cardholder:................................................................. ........ Signature of the cardholder:................................................................. .. Please, mail or fax the above form to: FLAMOC'96 POBox 91, Richmond 2753, AUSTRALIA Fax: +61(47) 761616 2. CHEQUE - made payable in Australian Dollars to: FLAMOC'96 The University of Sydney - sent to: POBox 91, Richmond 2753, AUSTRALIA ____________________________________________________________________________ ________ From kevin@research.nj.nec.com Wed Aug 16 01:22:55 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 16 Aug 95 01:22:52 -0500; AA28518 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 16 Aug 95 01:22:50 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa09866; 15 Aug 95 18:00:51 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa09864; 15 Aug 95 17:50:43 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa19496; 15 Aug 95 17:50:13 EDT Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa12522; 15 Aug 95 14:51:13 EDT Received: from zingo.nj.nec.com by EDRC.CMU.EDU id aa25989; 15 Aug 95 14:50:38 EDT Received: by zingo (940816.SGI.8.6.9/YDL1.4-910307.16) id OAA27703(zingo); Tue, 15 Aug 1995 14:49:12 -0400 Received: by doghein (5.52/cliff's joyful mailer #2) id AA20840(doghein); Tue, 15 Aug 95 14:49:12 EDT Date: Tue, 15 Aug 95 14:49:12 EDT From: Kevin Lang Message-Id: <9508151849.AA20840@doghein> To: connectionists@cs.cmu.edu Subject: learning 5-bit parity: RMHC vs. GP vs. MLP Comments on "A Response to ..." [Ko95] Kevin J. Lang NEC Research Institute kevin@research.nj.nec.com July 1995 These remarks constitute a brief technical reply to [Ko95], which was handed out at the ML-95 and ICGA-95 conferences. SCALING FROM 3-BIT TO 5-BIT PARITY: hill climbing still wins [Ko95, section 8] extends the experimental comparison in [La95] of the search efficiency of random mutation hill climbing and genetic search to the task of learning 5-bit parity. It is shown that any given run of RMHC is four times less likely than genetic search to find a circuit which computes 5-bit parity. However, when RMHC manages to find a solution, it does so about fifty times faster than genetic search. Hence, by using RMHC in an iterated manner (i.e. multiple independent runs), it is possible to generate solutions much more cheaply than with genetic search. For example, by restarting each run of RMHC after 75,000 candidates one could obtain a candidate to solution ratio of about 300,000. This compares well with the value of 2,913,583 candidates per solution reported for genetic search. The above argument could be formalized by calculating performance curves for the two algorithms, as discussed in [Ko92, chapter 8]. [Ko95] asserts that these curves would have permitted a meaningful comparison of the algorithms to be made, but does not provide them, citing the large computational expense that would supposedly have to be incurred. Actually, the relevant portion of the I(M,i,z) curve for RMHC could be estimated in one day by doing fifty runs out to 100,000 candidates. Since this is roughly the same amount of work as a single run of genetic search on this problem, estimating the performance curve for genetic search _would_ be a daunting task. DON'T USE RMHC: the power of multi-layer perceptrons I would like to emphasize that hill climbing algorithm described in [La95] was not intended to be either novel or good, and that I do NOT advocate its use. By deliberately using a bad version of an old algorithm, I sought to underline the negative character of my results. My positive advice is this: when learning functions, use multi-layer perceptrons. By adopting this representation for hypotheses one can exploit the powerful gradient-based search procedures that have been developed by the numerical analysis community. To illustrate the advantages of this approach, I invested 7 seconds of computer time in ten runs of conjugate gradient search for MLP weights to compute 5-bit parity. The resulting candidate to solution ratio was 393. This is roughly 750 times better than RMHC on boolean circuits, and 7400 times better than genetic search on boolean circuits. candidates found a examined solution? ---------- ---------- 31 yes 43 yes 62 yes 151 yes 196 no, stuck in local optimum 239 no, stuck in local optimum 271 no, stuck in local optimum 274 no, stuck in local optimum 308 no, stuck in local optimum 392 yes Unlike RMHC and genetic search, the conjugate gradient search procedure used here has no control parameters to tweak, and should yield good results in the hands of any user. Also, it detects when it is stuck in a local optimum, thus permitting an immediate restart to be made from random weights. This transforms the iterated methodology from an after-the-fact accounting system into a truly useful algorithm. Notes on the MLP network used in the above experiment: 5 input units 5 hidden units computing tanh 1 output unit computing tanh random initial weights drawn uniformly from the interval [-1,+1] true and false encoded by +1 and -1 REFERENCES [Ko95] John Koza, "A Response to the ML-95 Paper entitled "Hill Climbing Beats Genetic Search on a Boolean Circuit Synthesis Task of Koza's"", informally published and distributed document. [La95] Kevin Lang, "Hill Climbing Beats Genetic Search on a Boolean Circuit Synthesis Task of Koza's", The Twelfth International Conference on Machine Learning, pp. 340-343, 1995. Note: a copy of this paper can be obtained by anonymous ftp from ftp.nj.nec.com:/pub/kevin/lang-ml95.ps [Ko92] John Koza, "Genetic Programming", MIT Press, 1992, pp. 205-236. From walter.fetter@mandic.com.br Wed Aug 16 12:48:05 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 16 Aug 95 12:47:57 -0500; AA06662 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 16 Aug 95 12:47:54 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa10362; 16 Aug 95 2:44:10 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa10359; 16 Aug 95 2:33:58 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa19871; 16 Aug 95 2:33:26 EDT Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa23210; 16 Aug 95 2:32:37 EDT Received: from rjo04.embratel.net.br by RI.CMU.EDU id aa02309; 16 Aug 95 2:31:43 EDT Received: (from uumandic@localhost) by rjo04.embratel.net.br (8.6.12/8.6.12) id DAA21667 for connectionists@cs.cmu.edu; Wed, 16 Aug 1995 03:30:35 -0300 Subject: subscription From: WALTER FETTER Message-Id: <8AF40CA.012C010C4D.uuout@mandic.com.br> Date: Wed, 16 Aug 95 03:22:00 -0300 Organization: MANDIC BBS - Via EMBRATEL - Sao Paulo, Brazil (011) 816-3911 Reply-To: WALTER FETTER References: <8ACB049.012C0082FD.uuout@mandic.com.br> X-Mailreader: PCBoard Version 15.21 X-Mailer: PCBoard/UUOUT Version 1.10 Content-Type: text Content-Length: 165 Apparently-To: connectionists@cs.cmu.edu SUB connectionists Walter Fetter Lages []s Walter Fetter Lages on 08/16/95 at 02:30 (GMT - 3) Internet: w.fetter@ieee.org walter.fetter@mandic.com.br From jbower@bbb.caltech.edu Wed Aug 16 12:48:06 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 16 Aug 95 12:48:03 -0500; AA06667 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 16 Aug 95 12:47:57 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa10377; 16 Aug 95 2:52:25 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa10367; 16 Aug 95 2:34:53 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa19877; 16 Aug 95 2:33:46 EDT Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa19865; 15 Aug 95 21:37:09 EDT Received: from smaug-gw.caltech.edu by CS.CMU.EDU id aa17415; 15 Aug 95 21:36:23 EDT Received: from bbb.caltech.edu (smaug.bbb.caltech.edu) by gateway.bbb.caltech.edu (4.1/SMI-4.0) id AA11874; Tue, 15 Aug 95 18:44:29 PDT Received: from mordor.bbb.caltech.edu by bbb.caltech.edu (4.1/SMI-4.1) id AA06021; Tue, 15 Aug 95 18:44:08 PDT Date: Tue, 15 Aug 95 18:44:07 PDT From: Jim Bower Message-Id: <9508160144.AA06021@bbb.caltech.edu> To: connectionists@cs.cmu.edu Subject: GENESIS 2.0 ------------------------------------------------------------------------ This is to announce the release of GENESIS 2.0, a major revison of the GENESIS simulator. GENESIS is a general purpose simulation platform which was developed to support the simulation of neural systems ranging from complex models of single neurons to simulations of large networks made up of more abstract neuronal components. GENESIS has provided the basis for laboratory courses in neural simulation at both Caltech and the Marine Biological Laboratory in Woods Hole, MA, as well as several other institutions. Most current GENESIS applications involve realistic simulations of biological neural systems. Although the software can also model more abstract networks, other simulators are more suitable for backpropagation and similar connectionist modeling. The current version of GENESIS and its graphical front-end XODUS are written in C and run under UNIX on Sun (SunOS 4.x or Solaris 2.x), DECstation (Ultrix), Silicon Graphics (Irix 4.0.1 and up) or x86 PC (Linux or FreeBSD) machines with X-windows (versions X11R4, X11R5, and X11R6). Within the next two months, we expect to complete the port of GENESIS to DEC Alphas (OSF1 v2 and v3), IBM RS6000s (AIX) and HPs (HPUX). Other platforms may be capable of running GENESIS, but the software has not been tested by Caltech outside of these environments. The GENESIS ftp site also contains the first release of Parallel GENESIS, designed for networks of workstations (NOW), symmetric multiprocessors (SMP) and massively parallel processors (MPP). This release is known to run on SGI/Irix and has run on these workstations: Sun4/Solaris, Alpha/OSF1.3, DecStation/Ultrix4.3, Sun4/SunOS. It has run on these SMPs: SGI-Challenge/Irix 5.3, Sun4MP/Solaris. It will soon be ported to the Cray T3D/E MPP and later to Intel Paragon and IBM SP2 MPPs. GENESIS 2.0 also now runs on 486 and Pentium PC's under Linux and FreeBSD. In addition to these extensions, 2.0 includes a number of changes and improvements in the source code for portability, stablity, and consistency. Numerous changes in the GENESIS objects also add flexibility, especially when constructing network simulations. In addition, the XODUS graphical interface has been completely rewritten and is now independent of the Athena widget set. This allows greater interaction with simulations using the mouse (rescaling of graphs by click and drag, restructuring of simulation elements with drag and drop operations, etc.). A script converter is included which translates GENESIS 1 scripts to GENESIS 2. The GENESIS 2.0 release also includes updates of the simulation scripts for all tutorials and examples used in ``The Book of GENESIS'' (see below). This release includes a completely revised and expanded manual and on-line help. Acquiring GENESIS via free FTP distribution: We have made the current release of GENESIS (ver. 2.0, August 1995) available via FTP from genesis.bbb.caltech.edu (131.215.5.249). The distributed compressed tar file is about 3 MB in size. The current distribution includes full source code and documentation for both GENESIS and XODUS as well as fourteen tutorial simulations. Documentation for these tutorials is included along with online GENESIS help files and postscript files for generating the newly revised printed manual. To acquire the software use 'ftp' to connect to genesis.bbb.caltech.edu and login as the user "anonymous", giving your full email address as the password. You can then 'cd /pub/genesis' and download the software. Be sure to download the files LATEST.NEWS and README for information about new features of the current GENESIS version, the files on the system, and installation instructions. A detailed guide to the GENESIS neuroscience tutorials and to the construction of GENESIS simulations is given in: The Book of GENESIS: Exploring Realistic Neural Models with the GEneral NEural SImulation System, by James M. Bower and David Beeman, published by TELOS/Springer-Verlag -- ISBN 0-387-94019-7 For ordering information, contact info@telospub.com, or phone (in the US) 1-800-777-4643. BABEL - GENESIS users group Serious users of GENESIS are advised to join the users group, BABEL. Members of BABEL are entitled to access the BABEL directories and email newsgroup. These are used as a repository for the latest contributions by GENESIS users and developers. These include new simulations, libraries of cells and channels, additional simulator components, new documentation and tutorials, bug reports and fixes, and the posting of questions and hints for setting up GENESIS simulations. As the results of GENESIS research simulations are published, many of these simulations are being made available through BABEL. New developments are announced in a newsletter which is sent by email to all members. Members are able to access the BABEL directories and transfer files to and from their host machines using a passworded ftp account. Inquiries concerning GENESIS should be addressed to genesis@bbb.caltech.edu. Inquiries concerning BABEL memberships should be sent to babel@bbb.caltech.edu. Other information concerning GENESIS, including "snapshots" of GENESIS simulations and descriptions of research which has been conducted with GENESIS may be found on the GENESIS World Wide Web Server: http://www.bbb.caltech.edu/GENESIS From wgm@santafe.edu Wed Aug 16 22:50:43 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 16 Aug 95 22:50:41 -0500; AA12861 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 16 Aug 95 22:50:39 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa11366; 16 Aug 95 17:55:22 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa11364; 16 Aug 95 17:42:26 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa20615; 16 Aug 95 17:41:55 EDT Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa08601; 16 Aug 95 16:05:07 EDT Received: from sfi.santafe.edu by CS.CMU.EDU id aa24936; 16 Aug 95 16:04:12 EDT Received: from yaqui (yaqui.santafe.edu) by sfi.santafe.edu (4.1/SMI-4.1) id AA20822; Wed, 16 Aug 95 14:03:37 MDT Date: Wed, 16 Aug 95 14:03:37 MDT From: Bill Macready Message-Id: <9508162003.AA20822@sfi.santafe.edu> Received: by yaqui (4.1/SMI-4.1) id AA22075; Wed, 16 Aug 95 14:03:37 MDT To: Connectionists@cs.cmu.edu In his recent posting, Kevin Lang continues a contest with John Koza of the "my search algorithm beats your search algorithm according to the following performance measure for the following (contrived) fitness function" variety. Interested readers of connectionist should know that over the space of all fitness functions, no matter what the performance measure, any two search algorithms have exactly the same expected performance. This is discussed in the paper mentioned below. That paper goes on to discuss other more interesting issues, like head-to-head minimax distinctions between search algorithms, the information theoretic and geometric aspects of search, time-varying fitness functions, etc. Interested readers should also know that there was a (very) lengthy thread on this topic on the ga-list several months ago. In particular, some articles following up on the paper mentioned below are announced there. For example, an article by Radcliffe and Surry is announced there, as is an article by Macready and Wolpert on intrisically hard fitness functions (as opposed to functions that are hard with respect to some particular search algorithm). *** We are also currently involved in research with a student to exhaustively characterize the set of fitness functions for which searh algorithm A does much better than algorithm B and how that set differs for the set for which B outperforms A. In particular, we are doing this for the case where the algorithms are hill-climbers and GA's, as in Lang's debate with Koza. In addition, search is (from a formal perspective) almost identical to active learning, optimal experimental design, and (a less exact match) control theory. We are also currently investigating what those other fields have to offer for search. Anyone interested in receiving the results of that work as it gets written up can email us at wgm@santafe.edu or dhw@santafe.edu Bill Macready *** Here are the original paper announcements: ftp-file-name: nfl.ps No Free Lunch Theorems for Search D.H. Wolpert, W.G. Macready We show that all algorithms that search for an extremum of a cost function perform exactly the same, when averaged over all possible cost functions. In particular, if algorithm A outperforms algorithm B on some cost functions, then loosely speaking there must exist exactly as many other functions where B outperforms A. Starting from this we analyze a number of the other a priori characteristics of the search problem, like its geometry and its information-theoretic aspects. This analysis allows us to derive mathematical benchmarks for assessing a particular search algorithm's performance. We also investigate minimax aspects of the search problem, the validity of using characteristics of a partial search over a cost function to predict future behavior of the search algorithm on that cost function, and time-varying cost functions. We conclude with some discussion of the justifiability of biologically-inspired search methods. ------------------------------------------------------------- ftp-file-name: hard.ps What Makes an Optimization Problem Hard? W.G. Macready, D.H. Wolpert We address the question, ``Are some classes of combinatorial optimization problems intrinsically harder than others, without regard to the algorithm one uses, or can difficulty only be assessed relative to particular algorithms?'' We provide a measure of the hardness of a particular optimization problem for a particular optimization algorithm. We then present two algorithm-independent quantities that use this measure to provide answers to our question. In the first of these we average hardness over all possible algorithms for the optimization problem at hand. We show that according to this quantity, there is no distinction between optimization problems, and in this sense no problems are intrinsically harder than others. For the second quantity, rather than average over all algorithms we consider the level of hardness of a problem (or class of problems) for the algorithm that is optimal for that problem (or class of problems). Here there are classes of problems that are intrinsically harder than others. To obtain an electronic copy of these papers: ftp ftp.santafe.edu login: anonymous password: cd /pub/wgm get quit Then at your system: lpr -P If you have trouble getting any of these papers electronically, you can request a hard copy from publications (wp@santafe.edu), Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM, USA, 87501. From bap@scr.siemens.com Thu Aug 17 08:06:25 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 17 Aug 95 08:06:23 -0500; AA16209 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 17 Aug 95 08:06:20 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa11854; 17 Aug 95 1:54:09 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa11850; 17 Aug 95 1:46:33 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa00342; 17 Aug 95 1:46:06 EDT Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa16952; 17 Aug 95 1:13:55 EDT Received: from scr.siemens.com by CS.CMU.EDU id aa28707; 17 Aug 95 1:13:17 EDT Received: from gull.scr.siemens.com (gull.scr.siemens.com [129.73.7.53]) by scr.siemens.com (8.6.10/8.6.6) with ESMTP id BAA24203; Thu, 17 Aug 1995 01:13:14 -0400 Received: (from bap@localhost) by gull.scr.siemens.com (8.6.10/8.6.6) id BAA21287; Thu, 17 Aug 1995 01:13:12 -0400 Date: Thu, 17 Aug 1995 01:13:12 -0400 From: Barak Pearlmutter Message-Id: <199508170513.BAA21287@gull.scr.siemens.com> To: wgm@santafe.edu Cc: connectionists@cs.cmu.edu In-Reply-To: <9508162003.AA20822@sfi.santafe.edu> (message from Bill Macready on Wed, 16 Aug 95 14:03:37 MDT) Subject: Strawman: 2, GP: 0 Reply-To: Barak.Pearlmutter@scr.siemens.com It might be true that in a universe where everything was equally likely, all search algorithms would be equally awful. But that does not appear to be the universe that we live it, so it is not unreasonable to ask whether some particular search algorithm performs well in practice. But this point is somewhat moot, because Kevin Lang's recent post was not claiming that "his" "new" algorithm was better than current algorithms. Lang published a well reasoned and careful scientific paper, which cast doubt on the practicality and importance of a particular algorithm (GP) by showing that on a very simple problem *taken from the GP book* it compares unfavorably to a silly strawman algorithm. This paper passed scientific muster, and was accepted into ML, a respected refereed conference. John Koza responded by distributing a lengthy unrefereed screed, the bulk of which consisted of vicious invective and distorted half-truths. A small part of Koza's monograph had some actual technical content: it gave some new numbers on a scaled-up version of the problem in question, and interpreted them as showing that GP scaled better than the algorithm Lang had compared it to. However, Koza's interpretation was wrong: looking carefully at the raw numbers shows that even in Koza's hands, GP is scaling horribly worse than the silly strawman algorithm Lang had compared it to. That is the point of Lang's recent post. From workshop@Physik.Uni-Wuerzburg.DE Thu Aug 17 13:39:04 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 17 Aug 95 13:39:01 -0500; AA20743 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 17 Aug 95 13:38:57 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa12594; 17 Aug 95 12:39:30 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa12591; 17 Aug 95 12:29:52 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01030; 17 Aug 95 12:29:28 EDT Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa22798; 17 Aug 95 9:53:10 EDT Received: from wptx01.physik.uni-wuerzburg.de by EDRC.CMU.EDU id aa04605; 17 Aug 95 9:52:05 EDT Received: from wptx14.physik.uni-wuerzburg.de (wptx14.physik.uni-wuerzburg.de [132.187.40.14]) by wptx01.physik.uni-wuerzburg.de (8.6.10/8.6.10) with ESMTP id PAA10330 for ; Thu, 17 Aug 1995 15:49:06 +0200 Received: (from workshop@localhost) by wptx14.physik.uni-wuerzburg.de (8.6.10/8.6.10) id PAA17261 for Connectionists@cs.cmu.edu; Thu, 17 Aug 1995 15:55:33 +0200 From: workshop Message-Id: <199508171355.PAA17261@wptx14.physik.uni-wuerzburg.de> Subject: workshop and autumn school in Wuerzburg/Germany To: Connectionists Date: Thu, 17 Aug 95 15:55:33 MESZ Mailer: Elm [revision: 70.85] Second Announcement and Call for Abstracts INTERDISCIPLINARY AUTUMN SCHOOL AND WORKSHOP ON NEURAL NETWORKS: APPLICATION, BIOLOGY, AND THEORY October 12-14 (school) and 16-18 (workshop), 1995 W"urzburg, Germany INVITED SPEAKERS INCLUDE: M. Abeles, Jerusalem A. Aertsen, Rehovot J.K. Anlauf, Siemens AG J.P. Aubin, Paris M. Biehl, W"urzburg C. v.d. Broeck, Diepenbeek M. Cottrell, Paris G. Deco, Siemens AG B. Fritzke, Bochum Th. Fritsch, W"urzburg J. G"oppert, T"ubingen L.K.Hansen, Lyngby M. Hemberger,Daimler Benz L.van Hemmen, M"unchen J.A. Hertz, Copenhagen J. Hopfield, Pasadena I. Kanter, Ramat-Gan P. Kraus, Bochum B. Lautrup, Copenhagen W. Maass, Graz Th. Martinetz, Siemens AG M. Opper, Santa Cruz H. Scheich, Magdeburg S. Seung, AT&T Bell-Lab. W. Singer, Frankfurt S.A. Solla, Copenhagen H. Sompolinsky, Jerusalem M. Stemmler, Pasadena F. Varela, Paris A. Weigend, Boulder AUTUMN SCHOOL, Oct. 12-14: Introductory lectures on theory and applications of neural nets for graduate students and interested postgraduates in biology, medicine, mathematics, physics, computer science, and other related disciplines. Topics include neuronal modelling, statistical physics, hardware and application of neural nets, e.g. in telecommunication and biological data analysis. WORKSHOP, Oct. 16-18: Biology, theory, and applications of neural networks with particular emphasis on the interdisciplinary aspects of the field. There will be only invited lectures with ample time for discussion. In addition, poster sessions will be scheduled. REGISTRATION: Recommended before AUGUST 31, per FAX or (E-)MAIL to Workshop on Neural Networks, Inst. f"ur Theor. Physik, Julius-Maxmimilians-Universit"at Am Hubland, D-97074 W"urzburg, Germany Fax: +49 931 888 5141 E-mail: workshop@physik.uni-wuerzburg.de The registration fee is DM 150,- for the Autumn school and DM 150,- for the Workshop, due upon arrival (cash only). Students pay DM 80,- for each event (student ID required). ABSTRACTS: Participants who wish to present a poster should submit title and abstract together with their registration, preferably by E-mail. Deadline for the registration of poster contributions is AUGUST 31. ACCOMMODATION: Please use the appended form to contact directly the W"urzburg Tourist Office for room reservations (Fax +49 931 37652). NOTE THAT THIS NUMBER WAS WRONG IN THE FIRST ANNOUNCEMENT!!!!!!!!! We strongly recommend to arrange accommodation as soon as possible as various other conferences are scheduled for the same period of time. FTP-SERVER: Updated information (program, abstracts etc.) is available via anonymous ftp from the site ftp.physik.uni-wuerzburg.de, directory /pub/workshop/. Retrieve file README for further instructions. ORGANIZING COMMITTEE: M. Biehl, Th. Fritsch, W. Kinzel, Univ. W"urzburg. SCIENTIFIC ADVISORY COUNCIL: D. Flockerzi, K.-D. Kniffki, W. Knobloch, M. Meesmann, T. Nowak, F. Schneider, P. Tran-Gia, Universit"at W"urzburg. SPONSORS: Peter Beate Heller-Stiftung im Stifterverband f. die Deutsche Wissenschaft, Research Center of Daimler Benz AG, Stiftung der St"adt. Sparkasse W"urzburg. --------------------------------cut here---------------------------------- Registration Form Please return to: Workshop on Neural Networks Institut f"ur Theoretische Physik Julius-Maximilians-Universit"at Am Hubland D-97074 W"urzburg, Germany Fax: +49 931 888 5141 E-mail : workshop@physik.uni-wuerzburg.de I will attend the Autumn School Oct. 12-14 [ ] * (Reg. fee DM 150,- [ ] / 80,- [ ] due upon arrival) * Workshop Oct. 16-18 [ ] * (Reg. fee DM 150,- [ ] / 80,- [ ] due upon arrival) * * Please mark, reduced fee applies only for participants with valid student-ID. I wish to present a poster [ ] (If yes, please send a title page with a 10-line abstract!) Name: Affiliation: Address: Phone: Fax: E-mail: (please provide full postal address in any case!) Signature: --------------------------cut here, print out and fill in ------------------ To the Congress and Tourismus Zentrale Am Congress Centrum D- 97079 Wuerzburg Fax: ( +49) 931 37652 PLEASE NOTE: In case of room reservations the Tourist Office only proceeds as an agent. Your request should arrive here early enough to allow us to accommodate you and send you a reply (about one week) For this reservation we will charge you DM 5,- which you will have to pay with your hotel bill. REF: Workshop on Neural Networks Institut fuer Theoretische Physik Universitaet Wuerzburg Am Hubland D-97074 Wuerzburg - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - I hereby reserve ....... single rooms ....... double rooms LOCATION of the accommodation [ ] inner city [ ] city [ ] suburbs PRICE (per night, including breakfast) single double room with bath/shower/WC [ ] from DM 90,- [ ] from DM 130,- room with bath/shower/WC [ ] from DM 130,- [ ] from DM 180,- room with bath/shower/WC [ ] from DM 180,- [ ] from DM 250,- DATE of arrival ............. for ........... night(s) TIME of arrival (approximately) .............. h by car/train SIGNATURE and date : SENDER: (print letters) Company/institute : Mrs/Mr : Street : Zip Code : City : Phone : Fax : From kruschke@croton.psych.indiana.edu Thu Aug 17 21:20:36 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 17 Aug 95 21:20:34 -0500; AA25753 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 17 Aug 95 21:20:32 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab12594; 17 Aug 95 12:40:35 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab12591; 17 Aug 95 12:29:57 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01035; 17 Aug 95 12:29:45 EDT Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa23230; 17 Aug 95 10:16:01 EDT Received: from belize.ucs.indiana.edu by EDRC.CMU.EDU id aa04712; 17 Aug 95 10:15:41 EDT Received: from croton.psych.indiana.edu (croton.psych.indiana.edu [129.79.164.67]) by belize.ucs.indiana.edu (8.7.Beta.11/8.7.Beta.11/1.8IUPO) with SMTP id JAA06620 for ; Thu, 17 Aug 1995 09:14:02 -0500 (EST) Received: by croton.psych.indiana.edu; (5.65/1.1.8.2/07May94-0233PM) id AA12437; Thu, 17 Aug 1995 09:16:30 -0500 From: John Kruschke Message-Id: <9508171416.AA12437@croton.psych.indiana.edu> Subject: report announcement: Five principles of category learning To: connectionists@cs.cmu.edu Date: Thu, 17 Aug 1995 09:16:29 -0500 (EST) Cc: John Kruschke X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit ========================================================== Kruschke, J. K. & Erickson, M. A. (to appear). Five principles for models of category learning. Invited chapter in: Z. Dienes (ed.), Connectionism and Human Learning. Oxford, England: Oxford University Press. ABSTRACT: The primary goal of this chapter is to report a connectionist model that integrates five principles of category learning previously implemented separately. Previous work by Kruschke (1995) modeled the generalization phase of the ``inverse base-rate effect'' (Medin and Edelson, 1988), but did not address performance in the learning phase. That work emphasized the principles of rapid attention shifts and consistent use of base rate knowledge. Subsequent work by Kruschke and Bradley (1995) addressed the learning phase of a simpler categorization task, and emphasized the principles of short-term memory and strategic guessing. The present chapter integrates principles from both previous reports, and applies the integrated model to both the learning and generalization phases of the inverse base-rate effect. PostScript for this chapter, and for the previous papers cited in the abstract, may be retrieved from the Research section of my Web page, which has address (URL) listed below. John K. Kruschke e-mail: kruschke@indiana.edu Dept. of Psychology office: (812) 855-3192 Indiana University lab: (812) 855-9613 Bloomington, IN 47405-1301 USA fax: (812) 855-4691 URL= http://silver.ucs.indiana.edu/~kruschke/home.html ========================================================== From bert@mbfys.kun.nl Thu Aug 17 21:20:39 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 17 Aug 95 21:20:36 -0500; AA25759 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 17 Aug 95 21:20:34 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa12846; 17 Aug 95 15:58:06 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa12840; 17 Aug 95 15:49:49 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01221; 17 Aug 95 15:49:09 EDT Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id ac26018; 17 Aug 95 12:48:56 EDT Received: from septimius.mbfys.kun.nl by EDRC.CMU.EDU id aa04909; 17 Aug 95 11:04:47 EDT Received: from dontcare by septimius.mbfys.kun.nl via galba.mbfys.kun.nl [131.174.82.73] with SMTP id RAA14729 (8.6.10/2.4) for ; Thu, 17 Aug 1995 17:05:13 +0200 Date: Thu, 17 Aug 1995 17:05:13 +0200 From: Bert Kappen Message-Id: <199508171505.RAA14729@septimius.mbfys.kun.nl> To: Connectionists@cs.cmu.edu Subject: Paper dynamic linking, paper active vision The following two papers are available by anonymous FTP. DO NOT FORWARD TO OTHER GROUPS No hardcopies available Dynamic linking in Stochastic Networks Hilbert J. Kappen, Marcel J. Nijman To be presented at the International Conference on Brain Processes, Theories and Models to take place in Las Palmas de Gran Canaria, Spain, Nov 12-17, 1995. FTP-host: ftp.mbfys.kun.nl FTP-file: /snn/pub/reports/Kappen.Dyn.Link.ps.Z Abstract: It is well established that cortical neurons display synchronous firing for some stimuli and not for others. The resulting synchronous subpopulation of neurons is thought to form the basis of object perception. In this paper this 'binding' problem is formulated for Boltzmann Machines. Feed-forward connections implement feature detectors and lateral connections implement memory traces or cell assemblies. We show, that dynamic linking can be solved in the Ising model where sensory input provides local evidence. The lateral connections in the hidden layer provide global correlations between features that belong to the same stimulus and no correlations between features from different stimuli. Learning Active Vision Hilbert J. Kappen, Marcel J. Nijman, Tonnie van Moorsel To be presented at ICANN'95, October 1995, Paris FTP-host: ftp.mbfys.kun.nl FTP-file: snn/pub/reports/Kappen.Active.Vision.ps.Z Abstract: In this paper we introduce a new type of problem which we call active decision. It consists of finding the optimal subsequent action, based both on partial observation and on previously learned knowledge. We propose a method for solution, based on Boltzmann Machine learning of joint input-output probabilities and on an entropy minimization criterion. We show how the method provides a basic mechanism for active vision tasks such as saccadic eye movements. From byuhas@ucs.att.com Thu Aug 17 21:20:41 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 17 Aug 95 21:20:38 -0500; AA25761 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 17 Aug 95 21:20:36 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa12858; 17 Aug 95 16:04:55 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa12852; 17 Aug 95 15:52:34 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01234; 17 Aug 95 15:50:19 EDT Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa29027; 17 Aug 95 15:43:01 EDT Received: from gw2.att.com by EDRC.CMU.EDU id aa06096; 17 Aug 95 15:41:47 EDT Received: from engbak.ucs.att.com by ig2.att.att.com id AA02726; Thu, 17 Aug 95 14:18:34 EDT Received: from ucsbak.ucs.att.com (ucsbak.ucs.att.com [135.53.254.246]) by engbak.ucs.att.com (8.6.9/8.6.12) with SMTP id OAA17921 for ; Thu, 17 Aug 1995 14:16:24 -0400 From: byuhas@ucs.att.com Received: from netlink.ucs.att.com (netlink2.ucs.att.com) by ucsbak.ucs.att.com (4.1/SMI-4.1) id AA06929; Thu, 17 Aug 95 14:16:26 EDT Received: from cc:Mail by netlink.ucs.att.com id AA808665603; Thu, 17 Aug 95 14:14:18 EST Date: Thu, 17 Aug 95 14:14:18 EST Message-Id: <9507178086.AA808665603@netlink.ucs.att.com> To: connectionists@cs.cmu.edu, byuhas@ucs.att.com Subject: ********************************************************************** ************ PLEASE DO NOT REPOST ************** ********************************************************************** JOB OPPORTUNITY: NEURAL NETWORK FINANCIAL APPLICATIONS An opportunity exists for someone interested in applying neural networks to problems in the credit-card industry, including:risk modeling, marketing, fraud detection and related areas. This neural network effort will coincide with two other groups: one using more traditional statistical approaches and the other using AI and expert systems. Our goal is to combine and compare these methods to achieve the best possible results. Many of the problems we are interested in have two characteristics: 1) two-class discrimination is often the primary goal, with one class being significantly more prevalent than the other 2) huge amounts of data are available for modeling To be successful, candidate should have significant experience applying neural networks and be able to develop and implement novel algorithms, specialized error criteria and lever learning strategies to attack these problems. Qualifications of interest are: * a Masters degree, with a Ph.D. preferred * knowledge and experience with neural network modeling * 'C' programming in a UNIX environment * experience with SAS or S-Plus Compensation is competitive if not generous. ATT-UCS is located about 12 miles from the beach, just south of Jacksonville, FL. (The #1 medium-size city in Money's best places to live list out this week. ) If you have any questions or want to apply, you can contact me by: email: byuhas@ucs.att.com or phone: 904-954-7376 or fax: 904-954-8718. or Ben Yuhas ATT-UCS 8787 Baypine Road Jacksonville, FL 32256 From esann@dice.ucl.ac.be Thu Aug 17 21:20:44 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 17 Aug 95 21:20:40 -0500; AA25770 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 17 Aug 95 21:20:37 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa12869; 17 Aug 95 16:10:59 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa12867; 17 Aug 95 16:02:57 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01229; 17 Aug 95 15:49:45 EDT Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa28334; 17 Aug 95 14:54:48 EDT Received: from [130.104.2.80] by CS.CMU.EDU id aa04153; 17 Aug 95 14:54:23 EDT Received: from [130.104.31.81] by ns1.dice.ucl.ac.be (mp-14/10/92/SMI-4.1) id UAA05881; Thu, 17 Aug 1995 20:55:26 +0200 Message-Id: <199508171855.UAA05881@ns1.dice.ucl.ac.be> X-Sender: esann@ns1.dice.ucl.ac.be. Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Date: Thu, 17 Aug 1995 20:57:26 +0200 To: NEURO1-L@UICVM.CC.UIC.EDU, neuron-request@CATTELL20.psych.upenn.edu, Connectionists@cs.cmu.edu, enns-list@dcs.kcl.ac.uk, neur-sci@dl.ac.uk, comp-neuro@smaug.bbb.caltech.edu, frog-net@rana.usc.edu, echos@dmi.ens.fr, cogni-info@univ-lyon1.fr From: esann@dice.ucl.ac.be Subject: Neural Processing Letters Vol.2 No.4 Neural Processing Letters: new issue Vol.2 No.4 ----------------------------------------------- You will find enclosed the table of contents of the July 1995 issue of "Neural Processing Letters" (Vol.2 No.4). The abstracts of these papers are contained on the below mentioned FTP and WWW servers. We also inform you that subscription to the journal is now possible by credit card. All necessary information is contained on the following servers: - FTP server: ftp.dice.ucl.ac.be directory: /pub/neural-nets/NPL - WWW server: http://www.dice.ucl.ac.be/neural-nets/NPL/NPL.html If you have no access to these servers, or for any other information (subscriptions, instructions for authors, free sample copies,...), please don't hesitate to contact directly the publisher: D facto publications 45 rue Masui B-1210 Brussels Belgium Phone: + 32 2 245 43 63 Fax: + 32 2 245 46 94 Neural Processing Letters, Vol.2, No.4, July 1995 _________________________________________________ - Evolving neural networks with iterative learning scheme for associative me= mory Shigetaka Fujita, Haruhiko Nishimura - Combining Singular-Spectrum Analysis and neural networks for time series forecasting F. Lisi, O. Nicolis, Marco Sandri - A simplified MMC model for the control of an arm with redundant degrees of freedom U. Steink=FChler, W.-J. Beyn, H. Cruse - On the statistical physics of radial basis function networks Sean B. Holden, Mahesan Niranjan - Accelerated training algorithm for feedforward neural networks based on least squares method Y.F.Yam and Tommy W.S.Chow - On the search for new learning rules for ANNs Samy Bengio, Yoshua Bengio, Jocelyn Cloutier _____________________________ D facto publications - conference services 45 rue Masui 1210 Brussels Belgium tel: +32 2 245 43 63 fax: +32 2 245 46 94 _____________________________ From mkearns@research.att.com Thu Aug 17 21:20:45 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 17 Aug 95 21:20:42 -0500; AA25773 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 17 Aug 95 21:20:40 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa12884; 17 Aug 95 16:25:47 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa12882; 17 Aug 95 16:19:28 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01263; 17 Aug 95 16:18:56 EDT Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa29798; 17 Aug 95 16:16:25 EDT Received: from research.att.com by CS.CMU.EDU id aa05095; 17 Aug 95 16:16:00 EDT Received: by research.att.com; Thu Aug 17 16:13 EDT 1995 Received: from radish.research.att.com by hunny.research.att.com with smtp (Smail3.1.29.1 #2) id m0sjBK5-000d8KC; Thu, 17 Aug 95 16:13 EDT Received: by radish.research.att.com (Smail3.1.29.1 #9) id m0sjBJW-000q3IC; Thu, 17 Aug 95 16:13 EDT Message-Id: Date: Thu, 17 Aug 95 16:13 EDT From: "Michael J. Kearns" To: Connectionists@cs.cmu.edu Subject: COLT '96 Call for Papers Cc: mkearns@research.att.com ______________________________________________________________________ PRELIMINARY CALL FOR PAPERS---COLT '96 Ninth Conference on Computational Learning Theory Desenzano del Garda, Italy June 28 -- July 1, 1996 ______________________________________________________________________ The Ninth Conference on Computational Learning Theory (COLT '96) will be held in the town of Desenzano del Garda, Italy, from Friday, June 28, through Monday, July 1, 1996. COLT '96 is sponsored by the Universita` degli Studi di Milano. We invite papers in all areas that relate directly to the analysis of learning algorithms and the theory of machine learning, including neural networks, statistics, statistical physics, Bayesian/MDL estimation, reinforcement learning, inductive inference, knowledge discovery in databases, robotics, and pattern recognition. We also encourage the submission of papers describing experimental results that are supported by theoretical analysis. ABSTRACT SUBMISSION. Authors should submit fifteen copies (preferably two-sided) of an extended abstract to: Michael Kearns --- COLT '96 AT&T Bell Laboratories, Room 2A-423 600 Mountain Avenue Murray Hill, New Jersey 07974-0636 Telephone(for overnight mail): (908) 582-4017 Abstracts must be RECEIVED by FRIDAY JANUARY 12, 1996. This deadline is firm. We also anticipate allowing electronic submissions. Details of the electronic submission procedure will be provided in an updated Call for Papers, and will be available from the web site http://www.cs.cmu.edu/~avrim/colt96.html which will also be used to provide other program-related information. Authors will be notified of acceptance or rejection on or before Friday, March 15, 1996. Final camera-ready papers will be due by Friday, April 5. Papers that have appeared in journals or other conferences, or that are being submitted to other conferences, are not appropriate for submission to COLT. An exception to this policy is that COLT and STOC have agreed that a paper can be submitted to both conferences, with the understanding that a paper will be automatically withdrawn from COLT if accepted to STOC. ABSTRACT FORMAT. The extended abstract should include a clear definition of the theoretical model used and a clear description of the results, as well as a discussion of their significance, including comparison to other work. Proofs or proof sketches should be included. If the abstract exceeds 10 pages, only the first 10 pages may be examined. A cover letter specifying the contact author and his or her email address should accompany the abstract. PROGRAM FORMAT. At the discretion of the program committee, the program may consist of both long and short talks, corresponding to longer and shorter papers in the proceedings. The short talks will also be coupled with a poster presentation. PROGRAM CHAIRS. Avrim Blum (Carnegie Mellon University) and Michael Kearns (AT&T Bell Laboratories). CONFERENCE AND LOCAL ARRANGEMENTS CHAIRS. Nicolo` Cesa-Bianchi (Universita` di Milano) and Giancarlo Mauri (Universita` di Milano). PROGRAM COMMITTEE. Martin Anthony (London School of Economics), Avrim Blum (Carnegie Mellon University), Bill Gasarch (University of Maryland), Lisa Hellerstein (Northwestern University), Robert Holte (University of Ottawa), Sanjay Jain (National University of Singapore), Michael Kearns (AT&T Bell Laboratories), Nick Littlestone (NEC Research Institute), Yishay Mansour (Tel Aviv University), Steve Omohundro (NEC Research Institute), Manfred Opper (University of Wuerzburg), Lenny Pitt (University of Illinois), Dana Ron (Massachusetts Institute of Technology), Rich Sutton (University of Massachusetts) COLT, ML, AND EUROCOLT. The Thirteenth International Conference on Machine Learning (ML '96) will be held right after COLT '96, on July 3--7 in Bari, Italy. In cooperation with COLT, the EuroCOLT conference will not be held in 1996. STUDENT TRAVEL. We anticipate some funds will be available to partially support travel by student authors. Details will be distributed as they become available. From wgm@santafe.edu Fri Aug 18 16:12:54 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 18 Aug 95 16:12:53 -0500; AA05174 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Fri, 18 Aug 95 16:12:51 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa14211; 18 Aug 95 14:40:40 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa14203; 18 Aug 95 14:22:30 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02216; 18 Aug 95 14:22:03 EDT Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa16726; 18 Aug 95 13:43:35 EDT Received: from yaqui.santafe.edu by CS.CMU.EDU id aa12647; 18 Aug 95 13:42:34 EDT Received: by yaqui (4.1/SMI-4.1) id AA00176; Fri, 18 Aug 95 11:42:01 MDT Date: Fri, 18 Aug 95 11:42:01 MDT From: wgm@santafe.edu Message-Id: <9508181742.AA00176@yaqui> To: Connectionists@cs.cmu.edu In response to the recent postings of Kevin Lang and Bill Macready and David Wolpert, Barak Pearlmutter writes: >>> But this point is somewhat moot, because Kevin Lang's recent post was not claiming that "his" "new" algorithm was better than current algorithms. >>> Nor did we ever assume he was making this claim. We simply wanted to point to our work that resulted from many of the same motivations and investigates such issues from the most general perspective. >>> Lang published a well reasoned and careful scientific paper, which cast doubt on the practicality and importance of a particular algorithm (GP) by showing that on a very simple problem *taken from the GP book* it compares unfavorably to a silly strawman algorithm. This paper passed scientific muster, and was accepted into ML, a respected refereed conference. >>> Again we want to stress we were not attacking Lang's work *at all*. We couldn't agree more that the GP community needs more comparisons of its results to other techniques. For those interested in such comparisons I would also point to the papers of Una-May O'Reilly. They can be found at http://www.santafe.edu/sfi/publications/94wplist.html *** As regards No Free Lunch issues in optimization and supervised learning, >>> It might be true that in a universe where everything was equally likely, all search algorithms would be equally awful. >>> It is NOT true that "in a universe where everything was equally likely, all search algorithms would be equally awful." For example, there are major head-to-head minimax distinctions between algorithms. (The true magnitude of those distinctions is coming to light in the work with our student we mentioned in our previous post.) This is a crucially important point. There is a huge amount of structure distinguishing algorithms even in "a universe where everything was equally likely". All that would be "equally awful" in such a universe is expected performance. And of course in supervised learning there are other major a priori distinctions between algorithms. E.g., for quadratic loss, if you give me a learning algorithm with *any* random component (backprop with random initial weights anyone?), I can always come up with an algorithm with assuredly (!) superior performance, *independent of the target*. (And therefore independent of the "universe that we live in".) Bill Macready and David Wolpert From pazzani@super-pan.ICS.UCI.EDU Fri Aug 18 17:26:38 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Fri, 18 Aug 95 17:26:33 -0500; AA05792 Received: from paris.ics.uci.edu by lucy.cs.wisc.edu; Fri, 18 Aug 95 17:26:16 -0500 Received: from super-pan.ics.uci.edu by paris.ics.uci.edu id aa13755; 18 Aug 95 11:46 PDT To: ML-LIST:; Subject: Machine Learning List: Vol. 7, No. 14 Reply-To: ml@ics.uci.edu Date: Fri, 18 Aug 1995 11:29:57 -0700 From: Michael Pazzani Message-Id: <9508181146.aa13755@paris.ics.uci.edu> Machine Learning List: Vol. 7, No. 14 Friday, August 18, 1995 Contents: Genetic Programming and Hill Climbing Genetic Programming and Hill Climbing Response to Lang's July 31, 1995 "Comments" NNCM-95 PRELIMINARY PROGRAMME call for papers, IEEE Journal on SELECTED AREAS IN COMMUNICATIONS COLT '96 Call For Papers (preliminary version) new papers available via www; comments and feedback sought publications available Paper available on incremental local learning The Machine Learning List is moderated. Contributions should be relevant to the scientific study of machine learning. Mail contributions to ml@ics.uci.edu. Mail requests to be added or deleted to ml-request@ics.uci.edu. Back issues may be FTP'd from ics.uci.edu in pub/ml-list/V/ or N.Z where X and N are the volume and number of the issue; ID: anonymous PASSWORD: URL- http://www.ics.uci.edu/AI/ML/Machine-Learning.html ---------------------------------------------------------------------- From: John Koza Date: Thu, 17 Aug 95 12:59:47 PDT Subject: Genetic Programming and Hill Climbing [Note: John Koza submitted one message, which I have broken into two messages: This one and another following Lang's reply to the paper that is summarized by this message. Further commentary is welcome by either author (or anyone) in the next ML-LIST. ] FROM: John R. Koza Computer Science Department Stanford University Stanford, California 94305 Koza@CS.Stanford.Edu http://www-cs-faculty.stanford.edu/~koza/ 1. BACKGROUND In a paper entitled "Hill Climbing Beats Genetic Search on a Boolean Circuit Synthesis Problem of Koza's" that was accepted and published by the 1995 International Machine Learning Conference, Kevin Lang proposed a hillclimbing algorithm employing two kinds of random mutation of program trees that, he claimed, "beat" genetic programming (Lang 1995a). Lang's claim that his hillclimbing algorithm "beat" genetic programming was based on a comparative experiment based on 100 runs of all 80 distinct 3-argument Boolean functions for the two algorithms. I responded in a 24-page document distributed, by hand, on July 11, 1995 at the ML-95 conference in Tahoe City, California entitled "A Response to the ML-95 Paper Entitled 'Hill Climbing Beats Genetic Search on a Boolean Circuit Synthesis Problem of Koza's' " (Koza 1995). 2. SUMMARY OF LANG'S ML-95 PAPER AND MY JULY 11 "RESPONSE" Readers already familiar with Lang's ML-95 paper and my hand-distributed July 11 "Response" can skip this summary of the "first round" in sections 2.1 and 2.2 below. 2.1 LANG'S ML-95 PAPER Lang's ML-95 paper presents a specific alternative to the population-based search method employed by the genetic algorithm. His proposed search algorithm starts with a single random individual and, at each time step, remembers the single best individual ever seen. Then, on each time step, one candidate new individual is probabilistically created and compared with the current single best individual. If the new candidate is at least as good as the current best individual, the candidate becomes the new best individual. The candidate is created differently on alternate time steps. On the even-numbered steps (starting with time step 0), the new candidate is a randomly created entirely new individual. On the odd- numbered steps, the new candidate is the offspring produced by "crossing" the current best individual with a randomly created new individual. In this single-parent "crossing" operation (which is equivalent to the mutation operation of genetic programming), a randomly chosen subtree from an especially created new random individual is inserted at a randomly chosen point of the current best individual (thereby replacing the subtree currently located at that point). 2.2 MY JULY 11 RESPONSE One would think that most researchers in automated learning would be very surprised to see a tabulation of 100 runs of the family of 80 3-argument Boolean functions proffered as a suitable testbed for reaching a generalizable conclusion about machine learning. My "Response" asserted that "When a scientific claim must be established by experimental evidence (as opposed to proof), the persuasive quality of the experimental testbed is necessarily a threshold issue." I went on to say, "Boolean functions of ANY arity are ... of questionable value for basing generalizations about the performance of learning algorithms. (Emphasis in original). The inadequacy of Lang's testbed consisting of 80 3-argument Boolean functions can be seen merely by increasing the number of Boolean arguments to 4 and 5. 100% of the runs of genetic programming solved the only-slightly-less-trivial Boolean even-4-parity and the even-5-parity problems (in an average of about a half million and three million steps, respectively). In contrast, only 75% of a dozen runs of the Lang's mutation-based hillclimbing algorithm solved the even-4-parity problem and only 25% solved the even- 5-parity problem. The remaining runs of Lang's hillclimbing algorithm wandered around at various suboptimal fitness levels for the final 99% or so of the 4,800,000 steps allowed in each run. Lang's hillclimbing runs didn't succeed after giving it a generous number of steps (4,800,000) because of the well known tendency of probabilistic hillclimbing algorithms to not make any progress for very long stretches of time. (Note that stochastic hillclimbing algorithms don't ever get stuck. Given enough time, Lang's random mutation of program trees will, eventually, solve any problem whatsoever). The even-3-parity problem dramatizes the fallacy inherent in the Lang's bombastic proclamation that probabilistic hillclimbing "beats" genetic programming. Referring to a histogram showing how well 1,000,000 randomly created programs perform the even-3-parity task (table 26.1 of Koza 1994), 67.5% have fitness 4 (i.e., produce the correct answer for half of the 8 combinations of the 3 Boolean inputs); 15.6% have fitness 5 and 3 each; 0.5% have fitness 2 and 6; only 0.33% have fitness 7 or 1; and none (out of 1,000,000) have fitness 8 (the best) or 0 (the worst). Because two thirds of the random population have fitness 4, a run of Lang's proposed hillclimbing algorithm will typically start at time step 0 with a randomly created individual with a fitness of 4. On step 1 (and every succeeding odd-numbered step), a random GP mutation of the current best individual will be executed. On step 2 (and every succeeding even-numbered step), a randomly created entirely new individual will be generated. This entirely new random individual (and every succeeding entirely new random individual generated on these even-numbered steps) will have a 2/3 probability of having fitness 4 and a 1/6 probability of having fitness 5 (as shown in the histogram). After alternating these odd and even time steps a few times, the current best individual will probably work its way up to a fitness value of 5 and later to 6. Once the current best individual reaches a fitness level of 6, the even-numbered steps starts to become irrelevant since there is only a 0.33% (1 in 300) chance of creating a randomly created entirely new individual with fitness 7. The reason is that the even- numbered steps always draw from the same well (i.e., the known random distribution). When the fitness of the current best individual reaches the level of 7 (most likely due to an odd- numbered step), the chance that an even-numbered step will create an entirely new individual with fitness 8 is considerably less than one in a million. These odds are essentially "never" in relation to the total of small number of steps involved (i.e., 1,250). In other words, as soon as the fitness of the current best individual moves just a few standard deviations away from the mean value for randomly created new individuals, the even-numbered steps in the proposed hillclimbing algorithm become totally ineffective and irrelevant in advancing the progress of the search. In fact, the even-numbered steps matter only when the best current individual in this hillclimbing algorithm is very close to the mean (e.g., at the very beginning of the run). The irrelevance of the even-numbered steps would have been obvious to the paper's author had not made the threshold error (uncritically accepted by the 1995 International Machine Learning Conference) of employing the 80 very trivial 3-argument Boolean functions as its chosen testbed and had, instead, studied the ever-so-slightly less trivial 4-argument or 5-argument Boolean functions. For example, 100% of 1,000,000 randomly created individuals for the even-4-parity problem have fitness between 4 and 12. None of these 1,000,000 randomly created individuals even come close to the best level of fitness of 16 because a 16 is many standard deviations away from the mean of 8. Similarly, 100% of the 1,000,000 randomly created individuals for the even-5-parity task have fitness between 12 and 20. Again, a perfect score of 32 is many, many standard deviations away from the mean value of 16. Thus, once the proposed hillclimbing algorithm moves even slightly above the mean, there is essentially no chance that the even-numbered steps will produce a new individual that will dethrone the current best individual. The entire burden of advancing the progress of the search falls, almost immediately, upon the odd-numbered steps (i.e., the ordinary GP operation of mutation). Having eliminated the even-numbered steps from the picture, the question then becomes whether the odd-numbered steps can successfully search a complicated search space. We need not speculate on the answer to this question because the answer is already in Lang's ML-95 paper. Lang apparently first tried an approach consisting only of the odd- numbered steps and apparently discovered that this approach didn't work even in the nanoworld of 3- argument Boolean functions. The paper says, "Fully half of our candidates circuits were random, drawn from the same distribution that yielded poor performance for [random-generation- and-test]." Tellingly, the paper continues, "While this seems like a waste of resources, WE NEEDED TO HAVE SOME MECHANISM for escaping from the local optima that would have resulted from keeping only one good circuit around at a time. (Emphasis added)." In other words, Lang's paper concedes the inability of the odd-numbered steps alone for searching a non- linear space because of the well-known problem of lingering for long stretches of time at various suboptimal levels of fitness. Given that the paper concedes the inability of the odd-numbered steps to work and given the provable inability of the even-numbered steps to play any role in the search (once the search breaks out of the immediate neighborhood of the mean value of the initial random distribution), it is clear that NEITHER PART OF LANG'S PROPOSED HILLCLIMBING ALGORITM WORKS. The proposed hillclimbing algorithm is a pasting together of an approach that Lang's paper concedes doesn't work with an ineffective and irrelevant approach that provably can't work. Presumably, Lang came to the realization that he "needed to have some mechanism" from his own preliminary experimentation on the family of 3- argument Boolean functions. Then, because of the threshold error of conducting these experiments in the trivial testbed of 3-argument Boolean functions (an glaringly obvious error accepted uncritically by the International Machine Learning Conference), all the search activity of consequence was concentrated in the highly confined interval between a fitness level of 6 and 8. In this highly misleading and trivial world, the author of the paper managed to convince himself that his even-numbered steps (i.e., the resort to totally random "generation 0" individuals) actually contributed something to the progress of the search. ------------------------------ From: Kevin Lang Date: Fri, 28 Jul 95 15:17:14 EDT Subject: Genetic Programming and Hill Climbing Comments on "A Response to ..." [Ko95] Kevin J. Lang NEC Research Institute kevin@research.nj.nec.com July 1995 These remarks constitute a brief technical reply to the document [Ko95] which was handed out at the ML-95 conference and elsewhere. SCALING FROM 3-BIT TO 5-BIT PARITY: Hill Climbing Still Wins [Ko95, section 8] extends the experimental comparison in [La95] of the search efficiency of random mutation hill climbing and genetic search to the task of learning 5-bit parity. It is shown that any given run of RMHC is four times less likely than genetic search to find a circuit which computes 5-bit parity. However, when RMHC manages to find a solution, it does so about fifty times faster than genetic search. Hence, by using RMHC in an iterated manner (i.e. multiple independent runs), it is possible to generate solutions more cheaply than with genetic search. For example, by restarting each run of RMHC after 75,000 candidates one could obtain a candidate to solution ratio of about 300,000. This compares well with the value of 2,913,583 candidates per solution reported for genetic search. The above argument could be formalized by calculating performance curves for the two algorithms, as discussed in [Ko92, chapter 8]. [Ko95] asserts that these curves would have permitted a more meaningful comparison of the algorithms to be made, but does not provide them, citing the large computational expense that would supposedly have to be incurred. Actually, the relevant portion of the I(M,i,z) curve for RMHC could be estimated in one day by doing fifty runs out to 100,000 candidates. Since this is roughly the same amount of work as a single run of genetic search on this problem, estimating the performance curve for genetic search _would_ be a daunting task. DON'T USE RMHC: The performance of backpropagation I would like to emphasize that the particular version of random mutation hill climbing that was employed in my experiment was not intended to be either novel or good, and that I do NOT advocate its use. By deliberately using a bad version of an old algorithm, I sought to underline the negative character of my results. My positive advice is this: when learning functions, use multi-layer perceptrons. By adopting this representation for hypotheses one can exploit the powerful gradient-based search procedures that have been developed by the numerical analysis community. To illustrate the advantages of this approach, I invested 7 seconds of computer time in ten runs of conjugate gradient search for MLP weights to compute 5-bit parity. The resulting candidate to solution ratio was 393. This is roughly 750 times better than RMHC on boolean circuits, and 7400 times better than genetic search on boolean circuits. candidates found a examined solution? ---------- ---------- 31 yes 43 yes 62 yes 151 yes 196 no, stuck in local optimum 239 no, stuck in local optimum 271 no, stuck in local optimum 274 no, stuck in local optimum 308 no, stuck in local optimum 392 yes Unlike RMHC and genetic search, the conjugate gradient search procedure used here has no control parameters to tweak, and should yield good results in the hands of any user. Also, it detects when it is stuck in a local optimum, thus permitting an immediate restart to be made from random weights. This transforms the iterated methodology from an after-the-fact accounting system into a truly useful algorithm. Notes on the MLP network used in the above experiment: 5 input units 5 hidden units computing tanh 1 output unit computing tanh random initial weights drawn uniformly from the interval [-1,+1] true and false encoded by +1 and -1 REFERENCES [Ko95] John Koza, "A Response to the ML-95 Paper entitled "Hill Climbing Beats Genetic Search on a Boolean Circuit Synthesis Task of Koza's"", informally published and distributed document. [La95] Kevin Lang, "Hill Climbing Beats Genetic Search on a Boolean Circuit Synthesis Task of Koza's", The Twelfth International Conference on Machine Learning, pp. 340-343, 1995. [Ko92] John Koza, "Genetic Programming", MIT Press, 1992, pp. 205-236. ------------------------------ From: John Koza Date: Thu, 17 Aug 95 12:59:47 PDT Subject: Response to Lang's July 31, 1995 "Comments" On July 31, 1995, Lang wrote "Comments on 'A response to ... ' " subtitled "Hill Climbing Still Wins" (Lang 1995b) and submitted these "Comments" to the ML mailing list (and distributed them on another mailing list). Mike Pazzani, moderator of the ML mailing list, invited me to respond to Lang's latest "Comments" claiming that "Hill Climbing Still Wins." 3. TRACKING THE MOVING TARGET OF LANG'S JULY 31 COMMENTS I believe my July 11 "Response" succeeded in demonstrating the fallacy of Lang's claim in his ML- 95 paper that hillclimbing algorithm "beat" genetic programming ("of Koza's," as he put it). I believe my "Response" showed that, as the testbed is scaled up to these only-slightly-harder Boolean problems, there is a decreasing ability of the hillclimbing algorithm (on a dramatically decreasing fraction of the runs) to work, within any reasonable amount of time (while recognizing that Lang's algorithm will always work, on any problem, when given enough time). The best evidence of the success of my July 11 "Response" is the fact that Lang's July 31 "Comments" claiming that "Hill Climbing Still Wins" start off by ABANDONING the specific algorithm described in his published ML-95 paper. Indeed, in "Hill Climbing Still Wins," Lang unveils two new algorithms not even hinted at in his published ML-95 paper. Lang's first new algorithm is an "iterative" probabilistic hillclimbing algorithm that runs for 75,000 steps and is then restarted. Note that time is the essence in this debate. The only claim made in Lang's ML-95 paper is that hillclimbing "beats" genetic programming. And, the only claim made by Lang's July 31 iterative algorithm is that "Hill Climbing Still Wins" as to time. There is no dispute that both of Lang's algorithms will eventually work, on any problem, when given enough time. Thus, the essence of Lang's new "iterative" algorithm is the number 75,000 that defines a series of jack-rabbit starts (instead of letting the algorithm run to its natural and eventual guaranteed successful conclusion at some very distant time). What is the general principle by which one could possibly apply Lang's new iterative algorithm to an arbitrary problem in order to yield this time advantage? Does the cutoff number of 75,000 work for the 6- parity or 7-parity problem? The robotic box-pushing problem? The intertwined spiral problem (Lang and Witbrock 1989)? The transmembrane segment identification problem? Is 75,000 a universal constant - much like e? Where does it come from? The number 75,000 apparently comes from the fact that the three (25%) of MY dozen reported runs of the 5-parity problem each happened to solve just before step 75,000! The fact is that the only thing that permits Lang's new iterative algorithm to "still win" over genetic programming is a hand-crafted problem- specific cutoff number that saves his new algorithm from consuming what would otherwise be an exceedingly large amount of time to reach its guaranteed eventual successful conclusion. 4. ERRONEOUS CALCULATION Skipping past the origin of the magic cutoff number of 75,000, there is another important conceptual error in Lang's July 31 "Comments." Lang says, "by restarting each run of RMHC after 75,000 candidates, one could obtain a candidate-to- solution ratio of about 300,000." Apparently he arrives at 300,000 because of his prior statement that "RMHC is four times less likely than genetic search to find a {solution]" (with this "four" coming from dividing 100% by 25%). But, the number of steps needed to "solve a problem" using a probabilistic algorithm is not obtained by multiplying this "4" by 75,000! A probabilistic algorithm that yields a solution 25% of the time does not necessarily produce at least one solution in 4 tries. One reasonable way (Koza 1992, 1994) (but, by no means, the only way) to compute the needed number of steps for a probabilistic algorithm is to ask the following question: "How many steps must be performed to get a 99% probability of yielding at least one solution to the problem? The answer to this particular question is 16 times 75,000 (i.e., 1,200,000). This 1,200,000, of course, depends directly on the choice of the percentage (99%) and also assumes the statistical validity of the observed 25% success rate (i.e., 3 out of 12 runs). Before we get yet another missive from Lang (perhaps to be entitled "Hill Climbing Beats GP Yet Again") pointing out that 16 times 75,000 is less than 3 million, let me just say that I think that it is clear that Lang's new "iterative" algorithm based on the fixed magic number of 75,000 will also again crumble as the number of Boolean arguments is again slightly scaled up. 5. RETURNING TO "THE THRESHOLD ISSUE" In any event, the more important issue here still concerns what I correctly called "the threshold issue" of the suitability of Boolean functions as the tested in the first place. As I said in my July 11 "Response," "Boolean functions of ANY arity are ... of questionable value for basing generalizations about the performance of learning algorithms." "Boolean functions are unusual in that both the range and domain are DISCRETE and their performance can be efficiently represented by FINITE TRUTH TABLES. (Emphasis in original). There is nothing new about observed good performance by stochastic hillclimbing algorithms on problems from the discrete and idiosyncratic world of Boolean functions. Juels and Wattenberg (1994) describe a stochastic hillclimbing algorithm (operating, effectively, in the same way as Lang's hillclimbing algorithm, after removal of Lang's largely useless even-numbered steps). Their stochastic hillclimbing algorithm outperforms genetic programming on the Boolean 11- multiplexer problem. However, Juels and Wattenberg provide a clear explanation and insight as to what is really happening when Boolean functions are involved: "It is interesting to note -- perhaps partly in explanation of the [stochastic hillclimbing] algorithm's success on this problem -- that the [stochastic hillclimbing] algorithm formulated here defines a neighborhood structure in which there are NO STRICT LOCAL MINIMA. " (Emphasis in original). Juels and Wattenberg continue, "This property holds not only for the 11- multiplexer problem, BUT FOR ANY PROBLEM WHICH INVOLVES THE REALIZATION OF A SPECIFIC BOOLEAN FORMULA." (Emphasis in original). Juels and Wattenberg then prove this property in a proof that is specifically tailored to the search space of computer programs used by genetic programming (and by Lang's ML-95 hillclimbing algorithm). So the bottom line is that hillclimbing may indeed sometimes work well WHERE THE FITNESS LANDSCAPE HAS NO STRICT LOCAL MINIMA. That is to say, the threshold failure of Lang's ML- 95 paper (and his July 31 "Comments") is his continued use of a manifestly trivial testbed. (Juels and Wattenbergs' proof can be easily reworded in terms of finite truth tables to prove that a hillclimbing algorithm can learn any 5-argument Boolean function in only 32 steps using the tabular representation peculiar to discrete-valued functions over discrete-valued domains). 6. THE "NO FREE LUNCH" ISSUE Bill Macready (on August 16, 1995 on the Connectionists mailing list) has raised the issue of the no-free-lunch principle (Wolpert and Macready 1995; Macready and Wolpert 1995) concerning any claims as to whether one machine learning algorithm can be said to "beat" another: "In his recent posting, Kevin Lang continues a contest with John Koza of the 'my search algorithm beats your search algorithm ...' variety." Barak Pearlmutter (on August 17 on the Connectionists mailing list) then defended Lang from the no-free-lunch criticism by saying, "It might be true that in a universe where everything was equally likely, all search algorithms would be equally awful. But that does not appear to be THE UNIVERSE THAT WE LIVE IN, so it is not unreasonable to ask whether some particular search algorithm performs well in practice." (Emphasis added). But, regardless of whether you agree with Macready or Pearlmutter, Lang's originally published ML-95 claim that hillclimbing "beat" genetic programming was based on a comparative experiment based on 3-argument Boolean functions and Lang's July 31 claim that "Hill Climbing Still Wins" is based on the testbed of Boolean 4-parity and 5-parity functions. None of these Boolean problems are "the universe that we live in." 7. "PARAMETER-FREE" MULTI-LAYER PERCEPTRONS Lang closes his July 31 "Comments" by introducing yet another subject that was not mentioned at all in his published ML-95 paper, namely multi-layer perceptrons. He tells us: "My positive advice is this: when learning functions, use multi-layer perceptrons" because this approach, in conjunction with the conjugate gradient search, "has no control parameters to tweak." Then, in almost the next sentence, he mentions that the multi-layer perceptron had 5 hidden units; that the hyperbolic tangent function was used; and that the random initial weights drawn uniformly from the interval [-1,+1]. Presumably, all of this discussion about perceptrons has been introduced to recast Lang's published ML-95 paper as something that it never was, namely, now, a discussion about how perceptrons can also "beat" genetic programming. The fact is that Lang's ML-95 paper made a very strong (and wrong) claim that hillclimbing "beats" genetic programming based on an inadequate testbed. Lang's July 31 "Hill Climbing Still Wins" retrospectively recasts his first strong (and wrong) claim with a second strong (and wrong) claim. 8. THE PROBLEM WITH CROSS-PARADIGM COMPARISONS IN NANOWORLDS The fact that Lang's ML-95 paper was uncritically accepted and published by the Machine Learning conference is, in my opinion, a symptom of a serious problem in the field of machine learning. In my view, the present challenge for machine learning is how to get non-trivial results on non-trivial problems -- NOT the relative speed by which trivial and useless results can be obtained by various alternative paradigms. Cross-paradigm comparisons (if they are ever useful) can necessarily only be done on problems that can be solved by several different paradigms. Given the severe structural limitations inherent in many commonly-studied machine learning paradigms and their acknowledged inability to solve any significant breadth of problems, cross-paradigm comparisons are necessarily limited to the "least common denominator" of problem domains. In practice, that means some completely trivial problem domain. Once we get away from the trivial problem domains, the fact is that cross-paradigm time comparisons between genetic programming and other methods are (at least at the present time) a comparison between some finite amount of time and NEVER. Even if it were true that genetic programming were provably slower than N other machine learning methods on M toy problems (and, I strongly emphasize, I don't the slightest reason to believe that GP is either at the slow OR FAST end of such a scale), the point is that GP is capable of working on a breadth of non-trivial problems that cannot even be touched by other ML methods. The reason for this is that GP operates in the right search space (namely, the search space of computer programs). I believe that time will prove out my expectation that operating in the right search space will give GP the ability to solve some real problems. As I said in my first book (in what I repeatedly tell people is simultaneously the most trite and most important sentence in the book), " ... if we are interested in getting computers to solve problems without being explicitly programmed, the structures that we really need are COMPUTER PROGRAMS" (Emphasis in original). References Juels, Ari and Wattenberg, Martin. 1994. Stochastic Hillclimbing as a Baseline Method for Evaluating Genetic Algorithms. University of California Computer Science Department technical report CSD-94-834. September 28, 1994. Koza, John R. 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection. Cambridge, MA: The MIT Press. Koza, John R. 1994. Genetic Programming II: Automatic Discovery of Reusable Programs. Cambridge, MA: The MIT Press. Koza, John R. 1995. "A response to the ML-95 paper entitled 'Hill climbing beats genetic search on a Boolean circuit synthesis problem of Koza's' ". Document distributed, by hand, on July 11, 1995 at the 1995 International Machine Learning Conference in Tahoe City, California. Lang, Kevin J. 1995a. Hill climbing beats genetic Search on a Boolean circuit synthesis problem of Koza's. Proceedings of the Twelfth International Conference on Machine Learning. San Francisco, CA: Morgan Kaufmann. Lang, Kevin J. 1995b. "Comments on 'A response to the ML-95 paper entitled "Hill climbing beats genetic search on a Boolean circuit synthesis problem of Koza's" ' ". Dated July 31, 1995 and distributed on Connectionist mailing list on August 16, 1995 and on ML mailing list on August ---, 1995. Lang, Kevin J., and Witbrock, Michael J. 1989. Learning to tell two spirals apart. In Touretzky, David S., Hinton, Geoffrey E., and Sejnowski, Terrence J. (editors). Proceedings of the 1988 Connectionist Models Summer School. San Mateo, CA: Morgan Kaufmann Pages 52-59. Macready, W.G. and Wolpert, D. H. 1995. What Makes an Optimization Problem Hard? Santa Fe Institute Working Paper. Wolpert, D. H., and Macready, W.G. 1995. No Free Lunch Theorems for Search. Santa Fe Institute Working Paper. ------------------------------ From: Jerry Connor Date: Fri, 4 Aug 1995 19:26:05 BST Subject: NNCM-95 PRELIMINARY PROGRAMME NNCM 95 - PRELIMINARY PROGRAMME October 11, 12, and 13 1995 WWW site http://www.lbs.lon.ac.uk/desci/nncm.htm This years' Neural Networks in the Capital Markets Conference will be held in two parts. The Tutorials day offers two tracks. The Finance track is a series of 2 two-hour sessions designed to give engineers, mathematicians, and neural network practitioners an overview of the financial markets, pricing models for derivative securities, and the dynamics of price behaviour in high frequency markets. The Statistics & Neural Networks track is a series of 2 two-hour sessions designed to give finance professionals an overview of nonlinear techniques and mathematics that can be applied to data analysis, predictive modelling and analysis of financial markets. It will be held at London Business School, Sussex Place, London NW1 4SA on Wednesday 11 October. The Main conference will offer eight plenary sessions with invited speakers and original research contributions on Derivative & Term structure models, Equity & Commodity models, Foreign Exchange, Corporate Distress & Risk Models, Macroeconomic & Retail Finance applications, and two sessions on Advances in Methodology. Overall, the Main conference includes over 50 oral and poster paper presentations. It will be held on Thursday and Friday October 12 - 13 at the Langham Hilton, 1 Portland Place, London W1N 4JA, which is a short walk from London Business School. ------------------------------ From: Jianchang Mao 927-1932 Date: Fri, 11 Aug 95 18:31:52 -0800 Subject: call for papers, IEEE Journal on SELECTED AREAS IN COMMUNICATIONS CALL FOR PAPERS IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS COMPUTATIONAL AND ARTIFICIAL INTELLIGENCE IN HIGH SPEED NETWORKS Recent research in high speed networks has resulted in key architectural trends which are likely to fundamentally influence all facets of the communications infrastructure. A major opportunity is now the integration of diverse services on these networks. Unlike traditional teletraffic, many of these current and emerging services have poorly understood traffic parameters and user behaviors. These networks must be self-managing and self-healing and be able to maintain their quality of service, deal with congestion and failures, and allow dynamic reconfiguration with minimal intervention. This has led many researchers to investigate algorithms that have adaptive and even learning behaviors. There is a consensus among many researchers that to manage these new workloads and their workloads a class of techniques exhibiting some form of computational intelligence will be needed. Despite some progress, key challenges remain. One problem is that these resource management algorithms need to be able to respond anticipatively or preventatively to problems, since the time-bandwidth product of these networks does not always allow for reactive behavior. A second problem is that in some cases, decisions must be made on a very rapid (sometimes submicrosecond) time scale in order to optimize switching behavior. Thirdly, while the network adapts, its traffic sources and sinks are also adapting intelligently to the network's behavior, making for added complexity. Computational intelligence encompasses the information processing paradigms of adaptive systems such as neural networks and fuzzy logic. Examples of Artificial intelligence include expert systems and search techniques. Computational intelligence paradigms have the ability to learn from experience and to predict future behaviors. In some cases these learning rules are explicit, but in other cases the learning algorithms are implicit in a more general structure such as a neural network. In particular, neural networks have been shown to have properties that can help in managing congestion in networks, dealing with changing workloads, etc. Analog circuits that implement neural networks have been shown to be capable of solving optimization problems in submicrosecond timescales, fast enough to make on-the-fly switch routing decisions. Original papers are solicited on the applications of any technique in computational or artificial intelligence to the following topics (but not limited to): Fast packet switching Fault tolerant and dynamic routing Multimedia source measurement and modeling Call admission control Traffic policing Congestion and flow control Estimation of quality of service parameters Wireless and mobile networks Disconnectible terminal management Authors wishing to submit papers, should send six copies to Prof. Ibrahim Habib at the address below. The following schedule shall be applied: Submission Deadline: January 15, 1996 Notification of acceptance: May 15, 1996 Final manuscript due: July 1, 1996 Publication: 1st AQuarter, 1997. GUEST EDITORS: Prof. Ibrahim Habib Department of electrical engineering City University of New York, City College 137 street at Convent Avenue New York, N.Y. 10031 email: ibhcc@cunyvm.cuny.edu Dr. Robert Morris Manager, Data Systems Technology IBM Almaden Research Center San Jose, CA 95120 email: rjtm@almaden.ibm.com Dr. Hiroshi Saito Distinguished Technical Member NTT Telecommunications Networks Laboratories 3-9-11, Midori-chi, Musashino-shi Tokyo 180, Japan email: saito@hashi.ntt.jp Prof. Bjorn Pehrson Royal Institute of Technology Electrum 204 S-164 Kista, Sweden email: bjorn@it.kth.se ------------------------------ Received: from ics.uci.edu by paris.ics.uci.edu id aa15121; 17 Aug 95 14:25 PDT Received: from ics.uci.edu by q2.ics.uci.edu id aa12262; 17 Aug 95 14:25 PDT Received: from blum.pc.cs.cmu.edu by q2.ics.uci.edu id aa12258; 17 Aug 95 14:25 PDT From: Avrim Blum Date: Thu, 17 Aug 95 17:24:32 EDT To: ml@ics.uci.edu Subject: COLT '96 Call For Papers (preliminary version) Message-ID: <9508171425.aa12258@q2.ics.uci.edu> ______________________________________________________________________ PRELIMINARY CALL FOR PAPERS---COLT '96 Ninth Conference on Computational Learning Theory Desenzano del Garda, Italy June 28 -- July 1, 1996 ______________________________________________________________________ The Ninth Conference on Computational Learning Theory (COLT '96) will be held in the town of Desenzano del Garda, Italy, from Friday, June 28, through Monday, July 1, 1996. COLT '96 is sponsored by the Universita` degli Studi di Milano. We invite papers in all areas that relate directly to the analysis of learning algorithms and the theory of machine learning, including neural networks, statistics, statistical physics, Bayesian/MDL estimation, reinforcement learning, inductive inference, knowledge discovery in databases, robotics, and pattern recognition. We also encourage the submission of papers describing experimental results that are supported by theoretical analysis. ABSTRACT SUBMISSION. Authors should submit fifteen copies (preferably two-sided) of an extended abstract to: Michael Kearns --- COLT '96 AT&T Bell Laboratories, Room 2A-423 600 Mountain Avenue Murray Hill, New Jersey 07974-0636 Telephone(for overnight mail): (908) 582-4017 Abstracts must be RECEIVED by FRIDAY JANUARY 12, 1996. This deadline is firm. We also anticipate allowing electronic submissions. Details of the electronic submission procedure will be provided in an updated Call for Papers, and will be available from the web site http://www.cs.cmu.edu/~avrim/colt96.html which will also be used to provide other program-related information. Authors will be notified of acceptance or rejection on or before Friday, March 15, 1996. Final camera-ready papers will be due by Friday, April 5. Papers that have appeared in journals or other conferences, or that are being submitted to other conferences, are not appropriate for submission to COLT. An exception to this policy is that COLT and STOC have agreed that a paper can be submitted to both conferences, with the understanding that a paper will be automatically withdrawn from COLT if accepted to STOC. ABSTRACT FORMAT. The extended abstract should include a clear definition of the theoretical model used and a clear description of the results, as well as a discussion of their significance, including comparison to other work. Proofs or proof sketches should be included. If the abstract exceeds 10 pages, only the first 10 pages may be examined. A cover letter specifying the contact author and his or her email address should accompany the abstract. PROGRAM FORMAT. At the discretion of the program committee, the program may consist of both long and short talks, corresponding to longer and shorter papers in the proceedings. The short talks will also be coupled with a poster presentation. PROGRAM CHAIRS. Avrim Blum (Carnegie Mellon University) and Michael Kearns (AT&T Bell Laboratories). CONFERENCE AND LOCAL ARRANGEMENTS CHAIRS. Nicolo` Cesa-Bianchi (Universita` di Milano) and Giancarlo Mauri (Universita` di Milano). PROGRAM COMMITTEE. Martin Anthony (London School of Economics), Avrim Blum (Carnegie Mellon University), Bill Gasarch (University of Maryland), Lisa Hellerstein (Northwestern University), Robert Holte (University of Ottawa), Sanjay Jain (National University of Singapore), Michael Kearns (AT&T Bell Laboratories), Nick Littlestone (NEC Research Institute), Yishay Mansour (Tel Aviv University), Steve Omohundro (NEC Research Institute), Manfred Opper (University of Wuerzburg), Lenny Pitt (University of Illinois), Dana Ron (Massachusetts Institute of Technology), Rich Sutton (University of Massachusetts) COLT, ML, AND EUROCOLT. The Thirteenth International Conference on Machine Learning (ML '96) will be held right after COLT '96, on July 3--7 in Bari, Italy. In cooperation with COLT, the EuroCOLT conference will not be held in 1996. STUDENT TRAVEL. We anticipate some funds will be available to partially support travel by student authors. Details will be distributed as they become available. ------------------------------ From: Steven Salzberg Date: Fri, 28 Jul 95 15:50:25 EDT Subject: new papers available via www; comments and feedback sought The following paper is newly available, and comments from other machine learning researchers would be greatly appreciated. To retrieve it, go to http://www.cs.jhu.edu/salzberg/home.html. A number of other recent and not-so-recent papers are also available at that site. Title: On Comparing Classifiers: A Critique of Current Research and Methods Author: Steven Salzberg Abstract: Experimental machine learning research needs to scrutinize its approach to experimental design. If not done very carefully, comparative studies of classification algorithms can easily result in statistically invalid conclusions. This paper describes several phenomena that can, if ignored, invalidate an experimental comparison. It also divides machine learning research into several different types, and discusses why comparative analysis is more important for some than for others. ------------------------------ From: Vasant Honavar Date: Sat, 29 Jul 1995 21:34:40 CDT Subject: publications available The following recent publications of the Artificial Intelligence Research Group at Iowa State University (URL http://www.cs.iastate.edu/~honavar/aigroup.html) can be accessed on WWW via the URL http://www.cs.iastate.edu/~honavar/publist.html 1. Chen, C-H. and Honavar, V. (1995). A Neural Memory Architecture for Content as well as Address-Based Storage and Recall: Theory and Applications Paper under review. Draft available as ISU CS-TR 95-03. 2. Chen, C-H. and Honavar, V. (1995). A Neural Network Architecture for High-Speed Database Query Processing. Paper under review. Draft available as ISU CS-TR 95-11. 3. Chen, C-H. and Honavar, V. (1995). A Neural Architecture for Syntax Analysis. Paper under review. Draft available as ISU-CS-TR 95-18. 4. Mikler, A., Wong, J., and Honavar, V. (1995). Quo-Vadis - Adaptive Heuristics for Routing in Large Communication Networks. Under review. Draft available as ISU CS-TR 95-10. 5. Mikler, A., Wong, J., and Honavar, V. (1995). An Object-Oriented Approach to Modelling and Simulation of Routing in Large Communication Networks. Under review. Draft available as: ISU CS-TR 95-09. 6. Balakrishnan, K. and Honavar, V. (1995). Evolutionary Design of Neural Architectures - A Preliminary Taxonomy and Guide to Literature. Available as: ISU CS-TR 95-01. 7. Parekh, R. & Honavar, V. (1995). An Interactive Algorithm for Regular Language Learning. Available as: ISU CS-TR 95-02. 8. Balakrishnan, K. and Honavar, V. (1995) Properties of Genetic Representations of Neural Architectures. In: Proceedings of the World Congress on Neural Networks. Washington, D.C., 1995. Available as: ISU CS-TR 95-13. 9. Chen, C-H., Parekh, R., Yang, J., Balakrishnan, K. and Honavar, V. (1995). Analysis of Decision Boundaries Generated by Constructive Neural Network Learning Algorithms. In: Proceedings of the World Congress on Neural Networks. Washington, D.C., 1995. Available as: ISU CS-TR 95-12. The following publications will be available on line shortly (within the next few weeks): 1. Kirillov, V. and Honavar, V. (1995). Simple Stochastic Temporal Constraint Networks. Draft available as: ISU CS-TR 95-16. 2. Mikler, A., Wong, J., and Honavar, V. (1995). Utility-Theoretic Heuristics for Routing in Large Telecommunication Networks. Draft available as: ISU CS-TR 95-14. 3. Parekh, R., Yang, J., and Honavar, V. (1995). Constructive Neural Network Learning Algorithms for Multi-Category Pattern Classification. Draft available as: ISU CS-TR 95-15. 4. Yang, J., Parekh, R., and Honavar, V. (1995). Comparison of Variants of Single-Layer Perceptron Algorithms on Non-Separable Data. Draft available as: ISU CS-TR 95-19. The WWW page also contains pointers to other older publications some of which are available on line. Those who don't have access to a WWW browser can obtain ISU CS tech reports by sending email to almanac@cs.iastate.edu with BODY (not SUBJECT) "send tr catalog" and following the instructions that you will receive in the reply from almanac. Sorry, no hard copies are available. Best regards, Vasant Honavar Artificial Intelligence Research Group 226 Atanasoff Hall Department of Computer Science Iowa State University Ames, IA 50011-1040 email: honavar@cs.iastate.edu www: http://www.cs.iastate.edu/~honavar/homepage.html ------------------------------ From: Stefan Schaal Date: Tue, 1 Aug 95 13:31:27 JST Subject: Paper available on incremental local learning http://www.hip.atr.co.jp/~sschaal/pub/publications.html contains a number of papers, including the following new paper: FROM ISOLATION TO COOPERATIONION: AN ALTERNATIVE VIEW OF A SYSTEM OF EXPERTS Stefan Schaal and Christopher G. Atkeson submitted to NIPS'95 We introduce a constructive, incremental learning system for regression problems that models data by means of locally linear experts. In contrast to other approaches, the experts are trained independently and do not compete for data during learning. Only when a prediction for a query is required do the experts cooperate by blending their individual predictions. Each expert is trained by minimizing a penalized local cross validation error using second order methods. In this way, an expert is able to adjust the size and shape of the receptive field in which its predictions are valid, and also to adjust its bias on the importance of individual input dimensions. The size and shape adjustment corresponds to finding a local distance metric, while the bias adjustment accomplishes local dimensionality reduction. We derive asymptotic results for our method. In a variety of simulations we demonstrate the properties of the algorithm with respect to interference, learning speed, prediction accuracy, feature detection, and task oriented incremental learning. The paper is 8 pages long, requires 2.3 MB of memory (uncompressed), and is ftp-able as: ftp://ftp.cc.gatech.edu/people/sschaal/schaal-NIPS95.ps.gz or can be accessed through: http://www.cc.gatech.edu/fac/Stefan.Schaal/ http://www.hip.atr.co.jp/~sschaal/ Comments are most welcome. ------------------------------ End of ML-LIST (Digest format) **************************************** From bap@scr.siemens.com Sat Aug 19 01:58:07 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 19 Aug 95 01:58:05 -0500; AA08508 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 19 Aug 95 01:58:03 -0500 Received: by TELNET-1.SRV.CS.CMU.EDU id ab14289; 18 Aug 95 15:24:43 EDT Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa14239; 18 Aug 95 15:03:09 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa14237; 18 Aug 95 14:56:10 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02264; 18 Aug 95 14:55:49 EDT Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id ab17631; 18 Aug 95 14:45:46 EDT Received: from scr.siemens.com by RI.CMU.EDU id aa13080; 18 Aug 95 14:45:20 EDT Received: from gull.scr.siemens.com (gull.scr.siemens.com [129.73.7.53]) by scr.siemens.com (8.6.10/8.6.6) with ESMTP id OAA22546; Fri, 18 Aug 1995 14:45:16 -0400 Received: (from bap@localhost) by gull.scr.siemens.com (8.6.10/8.6.6) id OAA22582; Fri, 18 Aug 1995 14:45:14 -0400 Date: Fri, 18 Aug 1995 14:45:14 -0400 From: Barak Pearlmutter Message-Id: <199508181845.OAA22582@gull.scr.siemens.com> To: connectionists@cs.cmu.edu Cc: Tony Zador Subject: paper announcement Reply-To: Barak.Pearlmutter@scr.siemens.com Ftp-Host: archive.cis.ohio-state.edu Ftp-Filename: /pub/neuroprose/pearlmutter.vcdifn.ps.Z The following paper, to appear in Neural Computation, is available via ftp to archive.cis.ohio-state.edu:/pub/neuroprose/pearlmutter.vcdifn.ps.Z. VC Dimension of an Integrate-and-Fire Neuron Model Anthony M. Zador Barak A. Pearlmutter Salk Institute Siemens Corporate Research 10010 N. Torrey Pines Rd. 755 College Road East La Jolla, CA 92037 Princeton, NJ 08540 zador@salk.edu bap@scr.siemens.com ABSTRACT We compute the VC dimension of a leaky integrate-and-fire neuron model. The VC dimension quantifies the ability of a function class to partition an input pattern space, and can be considered a measure of computational capacity. In this case, the function class is the class of integrate-and-fire models generated by varying the integration time constant and the threshold, the input space they partition is the space of continuous-time signals, and the binary partition is specified by whether or not the model reaches threshold at some specified time. We show that the VC dimension diverges only logarithmically with the input signal bandwidth. We also extend this approach to arbitrary passive dendritic trees. The main contributions of this work are (1) it offers a formal treatment of the computational capacity of a dynamical system; and (2) it provides a framework for analyzing the computational capabilities of the dynamical systems defined by networks of real neurons. ---------------------------------------------------------------- Thanks for Jordan Pollack for maintaining the neuroprose archive. From lorincz@iserv.iki.kfki.hu Sat Aug 19 01:58:24 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Sat, 19 Aug 95 01:58:02 -0500; AA08506 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Sat, 19 Aug 95 01:58:00 -0500 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa14201; 18 Aug 95 14:33:05 EDT Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa14199; 18 Aug 95 14:21:39 EDT Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02210; 18 Aug 95 14:21:22 EDT Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa09091; 18 Aug 95 5:51:07 EDT Received: from iserv.iki.kfki.hu by EDRC.CMU.EDU id aa08640; 18 Aug 95 5:49:18 EDT Received: by iserv.iki.kfki.hu (AIX 3.2/UCB 5.64/4.03) id AA17280; Fri, 18 Aug 1995 11:22:36 +0200 Date: Fri, 18 Aug 1995 11:22:36 +0200 From: Andras Lorincz Message-Id: <9508180922.AA17280@iserv.iki.kfki.hu> To: Connectionists@cs.cmu.edu The following recent publications of the Adaptive Systems Lab of the Attila Jozsef University of Szeged and the Hungarian Academy of Sciences can be accessed on WWW via the URL http://iserv.iki.kfki.hu/asl-publs.html and http://iserv.iki.kfki.hu/qua-publs.html. Olah, M. and Lorincz, A. (1995) Analog VLSI Implementation of Grassfire Transformation for Generalized Skeleton Formation ICANN'95 Paris, accepted Abstract A novel analog VLSI circuit is proposed that implements the grassfire transformation for calculating the generalized skeleton of a planar shape. The fully parallel VLSI circuit can perform the calculation in real time. The algorithm, based on an activation spreading process on an artificial neural network, can be implemented by an extended nonlinear resistive network. Architecture and building blocks are outlined, the feasibility of this technique is investigated. Szepesvari, Cs. (1995) General Framework for Reinforcement Learning, ICANN'95 Paris, accepted Abstract We set up a general framework for the investigation of decision processes. The framework is based on the so called one step look-ahead (OLA) cost mapping. The cost of a policy is defined by the successive application OLA mapping. This way various decision criterions (e.g. the expected value criterion or the worst-case criterion) can be treated in a unified way. The main theorem of this article says that under minimal conditions optimal stationary policies are greedy w.r.t. the optimal cost function and vice versa. Based on this result we hope that previous results on reinforcement learning can be generalized to other decision criterions that fit the proposed framework. Marczell, Zs., Kalmar, Zs. and Lorincz, A. (1995) Generalized skeleton formation for texture segmentation Neural Network World, accepted Abstract An algorithm and an artificial neural architecture that approximates the algorithm are proposed for the formation of generalized skeleton transformations. The algorithm includes the original grassfire proposal of Blum and is extended with an integrative on-center off-surround detector system. It is shown that the algorithm can elicit textons by skeletonization. Slight modification of the architecture corresponds to the Laplace transformation followed by full wave rectification, another algorithm for texture discrimination proposed by Bergen and Adelson. Szepesvari, Cs. (1995) Perfect Dynamics for Neural Networks Talk presented at Mathematics of Neural Networks and Applications Lady Margaret Hall, Oxford, July, 1995 Abstract The vast majority of artificial neural network (ANN) algorithms may be viewed as massively parallel, non-linear numerical procedures that solve certain kind of fixed point equations in an iterative manner. We consider the use of such recurrent ANNs to solve optimization problems. The dynamics of such networks is often based on the minimization of an energy-like function. An important (and presumably hard) problem is to exclude the possibility of "spurious local minima". In this article we take another starting point and that is to consider perfect dynamics. We say that a recurrent ANN admits perfect dynamics if the dynamical system given by the update operator of the network has an attractor whose basin of attraction covers the set of all possible initial solution candidates. One may wonder whether neural networks that admit perfect dynamics can be interesting in applications. In this article we show that there exist a family of such networks (or dynamics). We introduce Generalised Dynamic Programming (GenDyP) for the government of the dynamics. Roughly speaking a GenDyP problem is a 3-tuple (X,A,Q), where X and A are arbitrary sets and Q maps functions of X into functions of X x A. GenDyP problems derive from sequential decision problems and dynamic programming (DP). Traditional DP procedures correspond to special selections of the mapping Q. We show that if Q is monotone and satisfies other reasonable (continuity and boundedness) conditions then the above iteration converges to a distinguished solution of the functional equation u(x)=inf_a (Q u)(x,a), that is the generalization of the well known Bellmann Optimality Equation. The proofs relies on the relation between the above dynamics and the optimal solutions of generalized multi-stage decision problems (we give the definitions in the next section). Kalmar, Zs., Szepesvari, Cs. and Lorincz, A. (1995) Generalized Dynamic Concept Model as a Route to Construct Adaptive Autonomous Agents Neural Network World 3:353--360 Abstract A model of adaptive autonomous agents, that (i) builds internal representation of events and event relations, (ii) utilizes activation spreading for building dynamic concepts and (iii) makes use of the winner-take-all paradigm to come to a decision is extended by introducing generalization into the model. The generalization reduces memory requirements and improves performance in unseen scenes as it is indicated by computer simulations J.Toth, G., Kovacs, Sz, and Lorincz, A. (1995) Genetic algorithm with alphabet optimization Biological Cybernetics 73:61-68 Abstract In recent years genetic algorithm (GA) was used successfully to solve many optimization problems. One of the most difficult questions of applying GA to a particular problem is that of coding. In this paper a scheme is derived to optimize one aspect of the coding in an automatic fashion. This is done by using a high cardinality alphabet and optimizing the meaning of the letters. The scheme is especially well suited in cases where a number of similar problems need to be solved. The use of the scheme is demonstrated on such a group of problems: on the simplified problem of navigating a `robot' in a `room'. It is shown that for the sample problem family the proposed algorithm is superior to the canonical GA. J.Toth, G. and Lorincz, A. (1995) Genetic algorithm with migration on topology conserving maps Neural Network World 2:171--181 Abstract Genetic algorithm (GA) is extended to solve a family of optimization problems in a self-organizing fashion. The continuous world of inputs is discretized in an optimal fashion with the help of a topology conserving neural network. The GA is generalized to organize individuals into subpopulations associated with the neurons. Interneuron topology connections are used to allow gene migration to neighboring sites. The method speeds up GA by allowing small subpopulations, but still saving diversity with the help of migration. Within a subpopulation the original GA was applied as the means of evolution. To illustrate the this modified GA the optimal control of a simulated robot-arm is treated: a falling ping-pong ball has to be caught by a bat without bouncing. It is demonstrated that the simultaneous optimization for an interval of height can be solved, and that migration can considerably reduce computation time. Other aspects of the algorithm are outlined. Amstrup, B., J.Toth, G., Szabo, G., Rabitz, H., and Lorincz, A. (1995) Genetic algorithm with migration on topology conserving maps for optimal control of quantum systems Journal of Physical Chemistry, 99, 5206-5213 Abstract The laboratory implementation of molecular optimal control has to overcome the problem caused by the changing environmental parameters, such as the temperature of the laser rod, the resonator parameters, the mechanical parameters of the laboratory equipment, and other dependent parameters such as time delay between pulses or the pulse amplitudes. In this paper a solution is proposed: instead of trying to set the parameter(s) with very high precision, their changes are monitored and the control is adjusted to the current values. The optimization in the laboratory can then be run at several values of the parameter(s) with an extended genetic algorithm (GA) wich is tailored to such parametric optimization. The extended GA does not presuppose but can take advantage and, in fact, explores whether the mapping from the parameter(s) to the optimal control field is continuous. Then the optimization for the different values of the parameter(s) is done cooperatively, which reduces the optimization time. A further advantage of the method is its full adaptiveness; i.e., in the best circumstances no information on the the system or laboratory equipment is required, and only the success of the control needs to be measured. The method is demonstrated on a model problem: a pump-and-dump type model experiment on CsI. The WWW pages also contain pointers to other older publications that are available on line. Papers are available also by anonymous ftp through iserv.iki.kfki.hu/pub/papers Best regards, Andras Lorincz Department of Adaptive Systems also Department of Photophysics Attila Jozsef University of Szeged Institute of Isotopes Dom ter 9 Hungarian Academy of Sciences Szeged Konkoly-Thege 29-33 Hungary, H-6720 Budapest, P.O.B. 77 Hungary, H-1525 email: lorincz@iserv.iki.kfki.hu