From rajubapi@school-computing.plymouth.ac.uk Tue Apr 2 00:36:45 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id AAA11239 for ; Tue, 2 Apr 1996 00:36:26 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id AAA29800 for ; Tue, 2 Apr 1996 00:36:20 -0600 Received: by TELNET-1.SRV.CS.CMU.EDU id aa17230; 1 Apr 96 22:41:14 EST Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa16724; 1 Apr 96 17:19:52 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa16722; 1 Apr 96 16:50:38 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa03295; 1 Apr 96 16:49:58 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa22962; 1 Apr 96 13:48:03 EST Received: from sun2.nsfnet-relay.ac.uk by EDRC.CMU.EDU id aa24843; 1 Apr 96 11:45:53 EST Via: uk.ac.plymouth.school-of-computing; Mon, 1 Apr 1996 17:45:16 +0100 Via: zeus.soc.plym.ac.uk; Mon, 1 Apr 96 17:47:18 BST Date: Mon, 1 Apr 1996 17:47:18 +0100 (BST) From: Raju Bapi Subject: Re: What is a "hybrid" model? To: jonathan_stein Cc: connectionists@cs.cmu.edu In-Reply-To: <9602298281.AA828133262@hub1.comverse.com> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Fri, 29 Mar 1996 Jonathan_Stein@com.comverse.hub1 wrote: > Next, it has been demonstrated in psychophysical experiments that there > are two types of learning. The first type is gradual, with slowly > improving performance, while in primates there is also "sudden" learning, > where the subject (EUREKA!) discovers a symbolic representation > simplifying the task. Thus not only is the basic hardware different for > the two processes, different learning algorithms are used as well. Could you (or any one on the list) please give references to this "sudden" or "Eureka" type of learning in animals ? Thanks Raju Bapi ---------------------------------------------------------- Neurodynamics Research Group School of Computing University of Plymouth Plymouth PL4 8AA United Kingdom email: rajubapi@soc.plym.ac.uk From skemp@gibbs.oit.unc.edu Wed Apr 3 04:13:08 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id EAA20883 for ; Wed, 3 Apr 1996 04:13:01 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id EAA23744 for ; Wed, 3 Apr 1996 04:12:59 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa18828; 3 Apr 96 1:53:29 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa18806; 3 Apr 96 1:29:03 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa04500; 3 Apr 96 1:27:09 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa17704; 3 Apr 96 1:24:23 EST Received: from gibbs.oit.unc.edu by EDRC.CMU.EDU id aa04912; 3 Apr 96 1:23:55 EST Received: from localhost by gibbs.oit.unc.edu (8.6.4.3/10.1) id BAA00875; Wed, 3 Apr 1996 01:23:47 -0500 Date: Wed, 3 Apr 1996 01:23:46 -0500 From: Steve Kemp To: Raju Bapi cc: connectionists@cs.cmu.edu Subject: Re: What is a "hybrid" model? In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Mon, 1 Apr 1996, Raju Bapi wrote: ..snip.. > On Fri, 29 Mar 1996 Jonathan_Stein@com.comverse.hub1 wrote: > > > Next, it has been demonstrated in psychophysical experiments that there > > are two types of learning. The first type is gradual, with slowly > > improving performance, while in primates there is also "sudden" learning, > > where the subject (EUREKA!) discovers a symbolic representation > > simplifying the task. Thus not only is the basic hardware different for > > the two processes, different learning algorithms are used as well. ..snip... > Could you (or any one on the list) please give references to this > "sudden" or "Eureka" type of learning in animals ? > > Thanks > > Raju Bapi Happy to oblige. The sudden learning was demonstrated in studies of human problem solving where it was eventually dubbed the "Aha!" effect. (I believe that there is a book by that name, but I don't have that reference.) In animal learning, it is known as one-trial learning. (I am unaware of the "Eureka" nomenclature.) The earliest reference I have for a study of this effect in humans is Maier (1930;1931). Six classic articles are excerpted in Wason & Johnson-Laird (1968). That should be a good source of background info. The mention of the demonstration of this effect in primates almost certainly refers to Wolfgang Kohler's (1925) classic study, THE MENTALITY OF APES, (Kegan-Paul, also reprinted by Penguin, 1957). That is the study where Kohler hung a banana from the top of a cage and placed several blocks in the cage. With all the blocks placed on one another, the resultant stack was tall enough for the ape to reach the banana. After some "contemplation," the ape would stack the blocks, climb to the top and retrieve the banana. Another Penguin book of readings, Riopelle (1967) includes a number of later articles on primates that discuss and followup on the Kohler work. That collection also includes the classic studies of animal problem-solving by Romanes (1888), Lloyd Morgan (1909), and Thorndike (1898). Single-trial learning is not restricted to apes, nor to cognitive learning alone. The Garcia Effect (Garcia, McGowan, & Green, 1972), a type of Pavlovian conditioning wherein animals as simple as baby chicks learn to avoid foods that have been associated with nausea, can be demonstrated after a single exposure. Indeed, Skinner (1932) demonstrated single-trial learning by reinforcing behavior in a pigeon. (Obviously, learning such simple tasks may not be "sudden" in the same sense of learning far more complex tasks in the studies cited above.) As to whether one-trial learning or the Aha! effect genuinely constitutes a distinct *type* of learning, it is most certainly distinct in that different experimental procedures are required to elicit such behavior. As to whether different brain processes are involved, brain scan studies, such as PET scan, single neuron monitoring, etc. will eventually answer such questions. I would imagine that such studies have already begun in the last few years, particularly with Pavlovian conditioning, but I am not up to date on that research. Perhaps someone else on the list is. I am not sure what Stein means by "psychophysical" in this context, but there is a relatively recent study by Metcalfe (1986) that attempts to measure the speed of sudden learning. For those interested in searching for further materials the keyword "insight" should get you pointed in the right direction on a computer search. Be warned however, that insight studies of REASONING will not be of much interest in this context. You might try INSIGHT and (PROBLEM SOLVING or LEARNING). steve kemp references: Garcia, J., McGowan, B. K., & Green, K. F. (1972). Biological constraints on conditioning. In Classical Conditioning, vol. 2.,, ed. by A. H. Black & W. H. Prokasy. New York: Appleton-Century-Crofts. Kohler, W. (1925). The Mentality of Apes. Kegan Paul. Lloyd Morgan, C. (1909). Introduction to Comparative Psychology. 2nd edition. Scribners Maier, N.R.F. (1930). "Reasoning in humans I: On direction." Journal of Comparative Psychology, vol. 10, pp.115-143. Maier, N.R.F. (1931). "Reasoning in humans II: The solution of a problem and its appearance in consciousness." Journal of Comparative Psychology, vol. 12, pp.181-194. Metcalfe, J. (1986). Feeling of knowing in memory and problem solving. Journal of Experimental Psychology: Learning Memory & Cognition, vol. 12, pp. 288-294. Riopelle, A. J., ed. (1967). Animal Problem Solving. Harmondsworth: Penguin Books. Romanes, G. J. (1888). Animal Intelligence. New York: D. Appleton. Skinner, B. F. (1932). On the rate of formation of a conditioned reflex. Journal of General Psychology. vol. 7, pp.274-286. Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. Psychological Review Monograph Supplements, vol. 2, pp. 1-9. Thorndike, E. L. (1911). Animal Intelligence: Experimental studies. New York: MacMillan. Wason, P. C. & Johnson-Laird, P. N., eds. (1968). Thinking and Reasoning. Harmondsworth: Penguin Books. (Please note that Wason & Johnson-Laird also have another book on reasoning with a very similar title. The book cited here is the Penguin book of Readings. Paperback only, but probably found in your local University library. Accept no substitutes. smk) Steven M. Kemp | Department of Psychology | email: steve_kemp@unc.edu Davie Hall, CB# 3270 | University of North Carolina | Chapel Hill, NC 27599-3270 | fax: (919) 962-2537 >>>>>>>>>>>>>>>>>>>>> <<<<<<<<<<<<<<<<<<<<<<<< New Left slogan from the Sixties: "Just because you're paranoid doesn't mean no one's out to get you." New Age slogan for the Nineties: "Just because you're schizophrenic doesn't mean no one's sending you messages." From WHYTE@VM.TEMPLE.EDU Wed Apr 3 04:15:10 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id EAA20885 for ; Wed, 3 Apr 1996 04:13:03 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id EAA23748 for ; Wed, 3 Apr 1996 04:13:00 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa18987; 3 Apr 96 2:37:04 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa18985; 3 Apr 96 2:28:55 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa04494; 3 Apr 96 1:26:09 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa13541; 2 Apr 96 18:18:13 EST Received: from vm.temple.edu by RI.CMU.EDU id aa25687; 2 Apr 96 18:17:08 EST Received: from VM.TEMPLE.EDU by VM.TEMPLE.EDU (IBM VM SMTP V2R2) with BSMTP id 2553; Tue, 02 Apr 96 18:17:13 EST Received: from VM.TEMPLE.EDU (NJE origin WHYTE@TEMPLEVM) by VM.TEMPLE.EDU (LMail V1.2a/1.8a) with BSMTP id 8180; Tue, 2 Apr 1996 18:17:14 -0500 Date: Tue, 02 Apr 96 18:12:10 EST From: WHYTE@VM.TEMPLE.EDU Organization: TEMPLE UNIVERSITY Subject: Post-doctoral fellowship opportunity To: Connectionists@cs.cmu.edu Message-Id: <960402.181713.EST.WHYTE@VM.TEMPLE.EDU> The Moss Rehabilitation Research Institute, at MossRehab Hospital, in Philadelphia is seeking post-doctoral fellows for a 2-year fellowship. There are several theoretical and applied topics in our research laboratories that would benefit from collaboration with someone with experience in neural network modelling. Potential topics include: simulation of language comprehension and production systems and the effects of "lesioning" them, in comparison to the data from patients with acquired language disorders; modelling the types of postural and other motoric compensations made by individuals with focal weakness (as in polio and similar disorders); and prediction of functional outcomes in rehabilitation given a variety of s complex and interacting impairments. Interested individuals should send a resume and cover letter to: John Whyte, M.D., Ph.D. Moss Rehabilitation Research Institute 1200 W. Tabor Rd. Phila. PA 19141 Fax: 215-456-9514 From terryd@dali.cit.gu.edu.au Thu Apr 4 01:00:02 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id AAA15393 for ; Thu, 4 Apr 1996 00:59:52 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id AAA14861 for ; Thu, 4 Apr 1996 00:59:50 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa20481; 3 Apr 96 23:48:28 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa20472; 3 Apr 96 23:30:12 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa00364; 3 Apr 96 23:29:22 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa05589; 3 Apr 96 21:47:15 EST Received: by CS.CMU.EDU id aa07213; 3 Apr 96 21:45:43 EST Received: from [132.234.5.2] by CS.CMU.EDU id aa07160; 3 Apr 96 21:36:23 EST Received: by dali.cit.gu.edu.au id AA18846 (5.67b/IDA-1.5 for connectionists@cs.cmu.edu); Thu, 4 Apr 1996 12:33:06 +1000 Date: Thu, 4 Apr 1996 12:33:06 +1000 From: Terry Dartnall Message-Id: <199604040233.AA18846@dali.cit.gu.edu.au> To: skemp@gibbs.oit.unc.edu Subject: Re: What is a "hybrid" model? Cc: connectionists@cs.cmu.edu Steve Thanks for that useful overview. You say >The sudden learning was demonstrated in studies of human problem solving > where it was eventually dubbed the "Aha!" effect. (I believe that there >is a book by that name, but I don't have that reference.) In animal >learning, it is known as one-trial learning. >. >. > As to whether one-trial learning or the Aha! effect genuinely constitutes >a distinct *type* of learning ... I know pretty much nothing about the area, but I would have thought that one-trial learning and the "Aha!" effect were different. I learnt not to stick my fingers in a power socket when I was a kid - and it only needed one trial! - but I wouldn't have though this was an "Aha!" situation. (It was a "Yow!" situation.) This applies to animals other than people, I'm sure. And you can have the "Aha!" effect after many trials, as with Koehler's apes. In fact I would have thought this is when you usually get it - after lots of frustrating failures. So one-trial learning is neither necessary nor sufficient for the "Aha!" effect. Isn't the "sudden learning problem" that, after a number of unsuccessful trials or trials, the answer suddenly comes to us? Best wishes Terry Dartnall ============================================== Terry Dartnall School of Computing and Information Technology Griffith University Nathan Brisbane Queensland 4111 Australia Phone: 61-7-3875 5020 Fax: 61-7-3875-5051 ============================================== From tibs@utstat.toronto.edu Thu Apr 4 01:00:04 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id AAA15395 for ; Thu, 4 Apr 1996 00:59:54 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id AAA14863 for ; Thu, 4 Apr 1996 00:59:52 -0600 Message-Id: <199604040659.AAA14863@lucy.cs.wisc.edu> Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab20481; 3 Apr 96 23:50:11 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab20472; 3 Apr 96 23:30:14 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa00369; 3 Apr 96 23:29:33 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa05628; 3 Apr 96 21:49:19 EST Received: from utstat.toronto.edu by RI.CMU.EDU id aa04111; 3 Apr 96 21:48:20 EST From: tibs@utstat.toronto.edu Date: Wed, 3 Apr 96 21:48 EST To: connectionists@cs.cmu.edu Subject: ew paper available Bias, variance and prediction error for classification rules Robert Tibshirani University of Toronto We study the notions of bias and variance for classification rules. Following Efron (1978) and Breiman (1996) we develop a decomposition of prediction error into its natural components. Then we derive bootstrap estimates of these components and illustrate how they can be used to describe the error behaviour of a classifier in practice. In the process we also obtain a bootstrap estimate of the error of a ``bagged'' classifier. Available at: http://utstat.toronto.edu/reports/tibs/biasvar.ps ftp: //utstat.toronto.edu/pub/tibs/biasvar.ps Comments welcome! ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Rob Tibshirani, Dept of Preventive Med & Biostats, and Dept of Statistics Univ of Toronto, Toronto, Canada M5S 1A8. Phone: 416-978-4642 (PMB), 416-978-0673 (stats). FAX: 416 978-8299 computer fax 416-978-1525 (please call or email me to inform) tibs@utstat.toronto.edu. ftp: //utstat.toronto.edu/pub/tibs http://www.utstat.toronto.edu/~tibs +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ From schlimme@eecs.wsu.edu Thu Apr 4 03:35:59 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id DAA16688 for ; Thu, 4 Apr 1996 03:35:37 -0600 Received: from cs.uwa.oz.au (bilby.cs.uwa.oz.au [130.95.1.11]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id DAA16192 for ; Thu, 4 Apr 1996 03:35:30 -0600 Received: from (mafm@parma.cs.uwa.oz.au [130.95.1.7]) by cs.uwa.oz.au (8.6.8/8.5) with SMTP id MAA15789; Thu, 4 Apr 1996 12:00:52 +0800 Message-Id: <199604040400.MAA15789@cs.uwa.oz.au> From: schlimme@eecs.wsu.edu (Jeffrey C. Schlimmer) To: reinforce@cs.uwa.edu.au (Reinforcement List) Subject: CFP: C on Machine Learning Date: Wed, 3 Apr 1996 19:14:29 -0800 ****************************************************** Preliminary Announcement ****************************************************** ICML'96 13th International Conference on Machine Learning Bari, Italy, July 3-6, 1996 ****************************************************** Early Registration Deadline is May 24, 1996 Hotel Reservation Deadline is May 24, 1996 (As the city of Bari has recently decided to host, in the same week of the conference, the World Youth Games, it is highly recommended to reserve your rooms as soon as possible.) The 13th International Conference on Machine Learning (ICML'96) will be held in Bari, Italy, during July 3-6,1996, with informal workshops on July 3rd. The conference will include presentation of refereed papers, and three invited lectures by: * Heikki Mannila (University of Helsinki, Helsinki, Finland) * Andrew Moore (Carnegie Mellon University, Pittsburgh, PA) * Vladimir Vapnik (AT&T Research, Murray Hill, NJ) Please note that the 9th Conference on Computational Learning Theory (COLT-96) will be also held in Italy, precisely in Desenzano sul Garda, on June 28th-July 1st, 1996. We offer reduced registration fees to attendees who register for both conferences. This preliminary announcement, which omits the final technical program, is being provided so that travel arrangements can be made as early as possible. An updated announcement, including the technical program, will be distributed sometime in April. General Information ------------------- Location -------- Bari (450.000 inhabitants) is the capital of Apulia (or Puglia in Italian), one of the most active regions in Southern Italy. Apulia is characterized by the presence of both plains and hills and extended contact with the sea. Apulia faces both Adriatic and Ionian seas, and this factor determines its climate, characterized by the long, pleasant, airy summers typical of the central Mediterranean areas. Its coast stretches for 800 kilometers. Bari is certainly one of the busiest meeting points in Southern Italy. The old part of the town, dating back to the Middle Age, is particularly interesting for its architecture and traditions, while the new part is a modern business city. Bari's surroundings present some of the most interesting historical and touristic sites in Southern Italy. How to reach Bari ----------------- Bari is accessible by air, train and car. Traveling by air, Bari is connected to Rome (1 h) and Milan (1.20 h), by means of direct flights to/from Bari's Palese Airport. Traveling by train, Bari is directly connected to Naples (4-5 h), Rome (6-7 h), Bologna (7-9 h), Milan (9-11 h), Turin (12 h) and Venice (10 h). Traveling by car, Bari is served by two highways, namely A16, which links Bari to Rome, and A14 (Adriatic line) which links Bari to Bologna. Bologna can be easily reached from all cities in Northern Italy. Climate ------- Bari enjoys a typical Mediterranean climate common to the whole of Southern Italy. There is very little rain during the summer months (the average annual rainfall is between 500 and 700 mm), and average temperatures are 25-26 degrees (Centigrades) with long stable periods of fine warm weather. Registration ------------ Please complete the attached registration form, and return it with payment for the full amount. The early registration (postmark) deadline is May 24, 1996 (Registration must be received by this date in order to qualify for the discounted rate). Registration fees include admission to all the technical sessions (including the workshops), one copy of the conference proceedings, conference kit, lunches on July 4-6 for the conference registrants, and also the lunch on July 3rd for the registrants to both the conference and to at least one workshop. Please notice that the student registration fees only include admission to the technical sessions and one copy of the proceedings. In order to have the printed Notes of the workshops, additional Lit. 15,000 are required for each set of notes. Housing ------- Please complete the attached hoter reservation form. Special rates for the conference are guaranteed for reservation arrived in Bari no later than May 24, 1996. As the city of Bari has recently decided to host, in the same week of the conference, the World Youth Games, it is higly recommended to reserve your rooms as soon as possible. Meals ----- During July 3-6 the lunch are included in the conference registration fees. Further Information ------------------- If you have any questions or problems, please send email to icml96@di.unito.it. -------------------------- cut here ------------------------------------ ICML '96 13th International Conference on Machine Learning July 3 - 6, 1996 - Bari (Italy) CONFERENCE REGISTRATION FORM Please return this form along with the deposit to: CENTRO INTERNAZIONALE CONGRESSI, II Trav. Via S. Matarrese, 3 - 70124 Bari, Italy Telephone Number: (+39) 80-5617299, Fax Number: (+39) 80-5614533 Please type or print clearly Name:_________________________________________________________________________ Last First Title (M/F) Institution or Company: ____________________________________________________ Address: ________________________________________________________________ _________________________________________________________________ City Post Code Country Telephone No: ___________________________ Fax No:_________________________ E-mail:______________________________________ REGISTRATION FEES (All Amounts Include Vat) ----------------- CONFERENCE By May 24, 1996 After May 24, 1996 -------------------------------------------------------------------- Regular Registrant Lit. 420.000 Lit. 520.000 |Lit. ________ | Student * Lit. 120.000 Lit. 150.000 |Lit. ________ | AI*IA Member (regular) Lit. 370.000 Lit. 470.000 |Lit. ________ | AI*IA Member (student) Lit. 100.000 Lit. 120.000 |Lit. ________ | COLT-96 registrant (regular) Lit. 370.000 Lit. 470.000 |Lit. ________ | COLT-96 registrant (student) Lit. 100.000 Lit. 120.000 |Lit. ________ | WORKSHOPS W 1 W 2 W 3 W 4 W 5 Lit. 15.000 per workshop notes |Lit. ___________ | Total amount Lit. ________________ *Students must enclose a document certifying their status. I am sending the payment in Italian Lire for the amount of Lit.__________________ payable to Centro Internazionale Congressi s.r.l. Method of payment ----------------- |_| Foreign draft/eurocheques payable to: Centro Internazionale Congressi s.r.l. |_| Bank transfer to: Centro Internazionale Congressi - Account no: 27/995 - Banco di Napoli Ag. 6 - Via Giovanni XXIII 70124 Bari, Italy. (cod. ABI 1010 - cod. CAB 04006) Please enclose a copy of the bank transfer payment with your registration form. |_| Credit Card |_| VISA |_| EUROCARD / MASTERCARD |_| DINERS' CLUB Card No. |_ |_ |_ |_ |_ | _| _|_ | _|_ |_ |_ |_ |_ |_ |_ | Expiration date | _| _|/| _| _| Cardholder's name (please print)__________________ Signature______________ SETTLEMENT ----------- Please make the amount payable in Italian lira to: Centro Internazionale Congressi s.r.l. and send it to the Organizing Secretariat. Registration cannot be confirmed if the present form is not accompanied by the corresponding payment. The bank charges must be added to the above amount. Any difference will be settled locally. In case a cancellation becomes necessary, it should be notified to the organising secretariat in writing. The following refund policy will be observed: Cancellation of your participation before May 24, 1996: refund of paied amount, less Lit. 50.000 Cancellation by June 24, 1996: 50% refund Cancellation after this date: no refund. All refunds will be forwarded after the Conference. I have duly read the registration and cancellation regulations and accept the conditions. Date: ___________________ Signature: ______________________________________ --------------------- cut here -------------------------------- 13th International Conference on Machine Learning ICML '96 July 3 - 6, 1996 - Bari (Italy) HOTEL ACCOMMODATION FORM Deadline: May 24, 1996 Please return this form along with the deposit to: CENTRO INTERNAZIONALE CONGRESSI, II Trav. Via S. Matarrese, 3 - 70124 Bari, Italy - Telephone Number: (+39) 80-5617299, Fax Number: (+39) 80-5614533 Please type or print clearly Name:_________________________________________________________________________ Last First Title (M/F) Institution or Company: ____________________________________________________ Address: ________________________________________________________________ _________________________________________________________________ City Post Code Country Telephone No: ___________________________ Fax No:_________________________ E-mail:_________________________ Name(s) of Accompanying Person(s):____________________________________________________________ Last First Hotels Category Rates per night Single* Double |_| 4 Star Lit.170.000/180.000 Lit. 220.000/250.000 |_| 3 Star Lit. 60.000/120.000 Lit. 100.000/200.000 ________________________________________________ | Deposit | | Single Double | | | |Lit. 170.000 Lit. 220.000 Lit. _________ | | | |Lit. 60.000 Lit. 100.000 Lit. _________ | |_______________________________________________| Total amount Lit. _______________ No. ___ Single Room(s) - No. ___ Double Room(s) - Date of arrival: ______ Date of departure: ______ Number of Night(s): ____ I am sending a hotel deposit (price of 1 night per room), in Italian Lire for the amount of Lit.__________________ payable to Centro Internazionale Congressi s.r.l. Method of payment |_| Foreign draft/eurocheques payable to: Centro Internazionale Congressi s.r.l. |_| Bank transfer to: Centro Internazionale Congressi - Account no: 27/995 - Banco di Napoli Ag. 6 - Via Giovanni XXIII - 70124 Bari, Italy - cod. ABI 1010 - cod. CAB 04006. Please enclose a copy of the bank transfer payment with your reservation form. |_| Credit Card |_| VISA |_| EUROCARD / MASTERCARD |_| DINERS' CLUB Card No. | _| _|_ | _| _| _| _| _| _| _| _| _| _| _| _| _| Expiration date |_|_|/|_|_| Cardholder's name (please print)______________________ Signature_____________________________________ HOTEL ACCOMMODATION INFORMATION *Since the number of single rooms is limited, if necessary the organization will reserve a double room for single occupancy, and the rate will correspond approximately to the double room rate less at least one continental breakfast. The above mentioned rates include breakfast, service and VAT. The organizing secretariat will send a hotel confirmation with all necessary information to the applicant after having made the hotel reservation. Rooms at various hotels will be allotted on a first-come, first-served basis. If all available rooms in the desired hotel category are occupied, a room will be reserved in the nearest possible category to the applicant's choice. Should booking dates not be indicated on the hotel accommodation form, the organization shall presume arrival July 2, 1996 and departure July 7, 1996. After May 24, 1996, hotel reservations may not be guaranteed. SETTLEMENT Please make the hotel deposit payable in Italian lira to: Centro Internazionale Congressi s.r.l. Reservations cannot be accepted without the hotel deposit. The deposit will be deducted from the hotel bill which must be settled by the registrant on departure. The bank charges must be added to the above amount. Any difference will be settled locally. In case a cancellation becomes necessary, it should be notified to the organising secretariat in writing. The following refund policy will be observed: if notification of cancellation is received by: Up to 30 days before the first night of stay: refund of 70% of the deposit 29 - 10 days before: refund of 20% of the deposit less than 9 days before or if no notice given: no refund. All refunds will be forwarded after the Conference. I have duly read the registration and cancellation regulations and accept the conditions. Date: ______________________ Signature: ___________________________________ -- Jeffrey C. Schlimmer, Asst. Prof., School of EE & CS, Washington State University, Pullman, WA 99164-2752, (509) 335-2399, (509) 335-3818 FAX http://www.eecs.wsu.edu/~schlimme/ PGP key: ftp://ftp.eecs.wsu.edu/pub/pgp/schlimmer.hqx, .txt powerPen Faculty Advisor, powerPen@eecs.wsu.edu http://www.eecs.wsu.edu/~schlimme/newton/index.shtml ftp://ftp.eecs.wsu.edu/pub/newton/ From lksaul@psyche.mit.edu Thu Apr 4 14:56:07 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id OAA27712 for ; Thu, 4 Apr 1996 14:55:50 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id OAA25965 for ; Thu, 4 Apr 1996 14:55:48 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa20470; 3 Apr 96 23:40:40 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa20468; 3 Apr 96 23:28:33 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa00358; 3 Apr 96 23:28:19 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa28277; 3 Apr 96 13:18:31 EST Received: from PSYCHE.MIT.EDU by CS.CMU.EDU id aa01171; 3 Apr 96 13:18:07 EST Received: from gubbio.mit.edu.mit.edu by psyche.mit.edu (4.1/DA1.0.4) id AA29806; Wed, 3 Apr 96 13:18:01 EST Date: Wed, 3 Apr 96 13:18:01 EST From: Lawrence Saul Message-Id: <9604031818.AA29806@psyche.mit.edu> Received: by gubbio.mit.edu.mit.edu (4.1/SMI-4.1) id AA18640; Wed, 3 Apr 96 13:17:58 EST To: connectionists@cs.cmu.edu Subject: paper announcement FTP-host: psyche.mit.edu FTP-file: pub/lksaul/mdplc.ps.Z WWW-host: http://web.mit.edu/~lksaul/ ---------------------------------------------------- The following paper, to appear at COLT'96, is now available on-line. It contains a statistical mechanical analysis of a simple problem in decision and control. ---------------------------------------------------- Title: Learning curve bounds for a Markov decision process with undiscounted rewards Authors: Lawrence Saul and Satinder Singh Abstract: The goal of learning in Markov decision processes is to find a policy that yields the maximum expected return over time. In problems with large state spaces, computing these returns directly is not feasible; instead, the agent must estimate them by stochastic exploration of the state space. Using methods from statistical mechanics, we study how the agent's performance depends on the allowed exploration time. In particular, for a simple control problem with undiscounted rewards, we compute a lower bound on the return of policies that appear optimal based on imperfect statistics. This is done in the thermodynamic limit where the exploration time and the size of the state space tend to infinity at a fixed ratio. ---------------------------------------------------- From sylee@eekaist.kaist.ac.kr Thu Apr 4 14:56:15 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id OAA27720 for ; Thu, 4 Apr 1996 14:55:59 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id OAA25967 for ; Thu, 4 Apr 1996 14:55:52 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa20726; 4 Apr 96 0:41:42 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa20724; 4 Apr 96 0:33:01 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa00475; 4 Apr 96 0:32:48 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa07109; 4 Apr 96 0:31:29 EST Received: from eekaist.kaist.ac.kr by CS.CMU.EDU id aa07954; 4 Apr 96 0:30:48 EST Received: (from sylee@localhost) by eekaist.kaist.ac.kr (8.6.12h2/8.6.12) id OAA25599 for Connectionists@cs.cmu.edu; Thu, 4 Apr 1996 14:28:59 +0900 Date: Thu, 4 Apr 1996 14:28:59 +0900 From: Soo-Young Lee Message-Id: <199604040528.OAA25599@eekaist.kaist.ac.kr> To: Connectionists@cs.cmu.edu Subject: Graduate Scholarship Content-Length: 1286 GRADUATE STUDENT POSITION A graduate student position is available at the Department of Electrical Engineering at Korea Advanced Institute of Science and Technology (KAIST) to study neural network modelling, speech and control appliations, and hardware (VLSI and optics) implementation. Bacholar degree is required for Master course students, and Master degree is required for Ph.D. course students. The positions are available from September, 1996. The KAIST is the top-ranked research-oriented engineering school in Korea, which belongs to Ministry of Science and Engineering. The Deaprtment of Electrical Enginnering consists of 48 professors, about 500 graduate students. Annual research fund is more than 15 million US dollars. Full scholarship may be provided. For those from other countries we also have Korean language classes. Applicants should send their CV, list of publications, a letter describing their interest, and name, address and phone number of two references to: Prof. Soo-Young Lee Computation and Neural Systems Laboratory Department of Electrical Engineering Korea Advanced Institute of Science and Technology 373-1 Kusong-dong, Yusong-gu Taejon 305-701 Korea (South) Fax: +82-42-869-3410 E-mail: sylee@ee.kaist.ac.kr Subject: Graduate Scholarship From skemp@gibbs.oit.unc.edu Thu Apr 4 14:56:17 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id OAA27723 for ; Thu, 4 Apr 1996 14:56:03 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id OAA25972 for ; Thu, 4 Apr 1996 14:56:01 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa20842; 4 Apr 96 2:08:44 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa20837; 4 Apr 96 1:49:58 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa00550; 4 Apr 96 1:49:15 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa08131; 4 Apr 96 1:48:06 EST Received: from gibbs.oit.unc.edu by EDRC.CMU.EDU id aa11884; 4 Apr 96 1:47:47 EST Received: from localhost by gibbs.oit.unc.edu (8.6.4.3/10.1) id BAA26156; Thu, 4 Apr 1996 01:45:37 -0500 Date: Thu, 4 Apr 1996 01:45:35 -0500 From: Steve Kemp To: Terry Dartnall cc: connectionists@cs.cmu.edu Subject: Re: What is a "hybrid" model? In-Reply-To: <199604040233.AA18846@dali.cit.gu.edu.au> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Thu, 4 Apr 1996, Terry Dartnall wrote: > > Thanks for that useful overview. You say > > >The sudden learning was demonstrated in studies of human problem solving > > where it was eventually dubbed the "Aha!" effect. (I believe that there > >is a book by that name, but I don't have that reference.) In animal > >learning, it is known as one-trial learning. > > I know pretty much nothing about the area, but I would have thought that > one-trial learning and the "Aha!" effect were different. I learnt not to > stick my fingers in a power socket when I was a kid - and it only needed one > trial! - but I wouldn't have though this was an "Aha!" situation. (It was a > "Yow!" situation.) This applies to animals other than people, I'm sure. > A good point. The original post was contrasting the sudden learning found with the Aha! effect with what the poster called "gradual" learning. If the distinction (that makes for the two types) is between gradual and sudden learning, then one-trial learning, while perhaps distinct from insight learning, seems to be sudden rather than gradual. That is, there are other non-gradual types of learning besides insight learning. > And > you can have the "Aha!" effect after many trials, as with Koehler's apes. In > fact I would have thought this is when you usually get it - after lots of > frustrating failures. So one-trial learning is neither necessary nor > sufficient for the "Aha!" effect. > > Isn't the "sudden learning problem" that, after a number of unsuccessful > trials or trials, the answer suddenly comes to us? > Another way of looking at it is that if insight learning occurs on the very first trial, it is very hard to distinguish such a case from one-trial learning, at least from an empirical perspective. The banana problem was quite a challenge for the mental capacity of the apes involved. The power socket "problem" was quite easy for you. If we were to suppose that certain types of learning *are* sudden, then doesn't it make sense that the sudden onset of learning would occur on an early trial for "simple" or "easy" problems and on a later trial for more "complex" or "harder" problems? In that case, the Aha! effect would just be the natural result of being presented with a difficult problem. In fact, in the mathematical learning theory literature, a number of Markov-based models were constructed after just such an assumption. It was assumed that all learning was "all-or-none" in character. Apparent gradual change was modeled as "random" correct guessing by subjects who had not yet "learned," plus artifacts of emprical measures used by experimenters that averaged across subjects or trials where learning had occurred in some instances and not in others. A remarkably large number of learning phenomena, including many apparently gradual ones, were successfully modeled. Finally, "sudden" or "gradual" is measured with respect to the number of trials. It is essential to Kohler's conception that some sort of ongoing internal "contemplative" process was occurring all through the process, during and between trials. More trials more rapidly presented allow a gradual process to appear gradual. If the subject is gradually catching on and we present fewer trials less often, then the gradual learning may appear sudden because enough learning occurred in the long interval between trials to become noticeable all at once on the following trial. In sum, my point is that it is difficult, if not impossible, to establish the existence or non-existence of genuinely different *types* of learning solely from behavioral phenomena, however augmented by theory or mathematics. One of the truly exciting things about the recent advances in the various technologies of brain monitoring is that they provide a second type of empirical evidence that can be correlated with behavioral evidence to discover if apparently distinct learning phenomena involve genuinely different brain mechanisms. regards, steve K Steven M. Kemp | Department of Psychology | email: steve_kemp@unc.edu Davie Hall, CB# 3270 | University of North Carolina | Chapel Hill, NC 27599-3270 | fax: (919) 962-2537 From thrun+@heaven.learning.cs.cmu.edu Fri Apr 5 03:06:18 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id DAA05573 for ; Fri, 5 Apr 1996 03:06:11 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id DAA06289 for ; Fri, 5 Apr 1996 03:06:09 -0600 Message-Id: <199604050906.DAA06289@lucy.cs.wisc.edu> Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa22387; 4 Apr 96 23:23:35 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa22385; 4 Apr 96 23:13:48 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01430; 4 Apr 96 23:13:27 EST Received: from HEAVEN.LEARNING.CS.CMU.EDU by B.GP.CS.CMU.EDU id aa26047; 4 Apr 96 22:11:44 EST To: connectionists@cs.cmu.edu Subject: Book announcement: EBNN, Lifelong Learning Date: Thu, 4 Apr 96 22:10:54 EST From: thrun+@heaven.learning.cs.cmu.edu Sender: thrun+@heaven.learning.cs.cmu.edu I have the pleasure to announce the following book. EXPLANATION-BASED NEURAL NETWORK LEARNING: A Lifelong Learning Approach Sebastian Thrun Carnegie Mellon University & University of Bonn published by Kluwer Academic Publishers ---------------------------------------------------------------------- Lifelong learning addresses situations in which a learner faces a series of different learning tasks, providing the opportunity for synergy among them. Explanation-based neural network learning (EBNN) is a machine learning algorithm that transfers knowledge across multiple learning tasks. When faced with a new learning task, EBNN exploits domain knowledge accumulated in previous learning tasks to guide generalization in the new one. As a result, EBNN generalizes more accurately from less data than comparable methods. This book describes the basic EBNN paradigm and investigates it in the context of supervised learning, reinforcement learning, robotics, and chess. ``The paradigm of lifelong learning - using earlier learned knowledge to improve subsequent learning - is a promising direction for a new generation of machine learning algorithms. Given the need for more accurate learning methods, it is difficult to imagine a future for machine learning that does not include this paradigm.'' -- from the Foreword by Tom M. Mitchell ---------------------------------------------------------------------- FOREWORD by Tom Mitchell ix PREFACE xi 1 INTRODUCTION 1 1.1 Motivation 1 1.2 Lifelong Learning 3 1.3 A Simple Complexity Consideration 8 1.4 The EBNN Approach to Lifelong Learning 13 1.5 Overview 16 2 EXPLANATION-BASED NEURAL NETWORK LEARNING 19 2.1 Inductive Neural Network Learning 20 2.2 Analytical Learning 27 2.3 Why Integrate Induction and Analysis? 31 2.4 The EBNN Learning Algorithm 33 2.5 A Simple Example 39 2.6 The Relation of Neural and Symbolic Explanation-Based Learning 43 2.7 Other Approaches that Combine Induction and Analysis 45 2.8 EBNN and Lifelong Learning 47 3 THE INVARIANCE APPROACH 49 3.1 Introduction 49 3.2 Lifelong Supervised Learning 50 3.3 The Invariance Approach 55 3.4 Example: Learning to Recognize Objects 59 3.5 Alternative Methods 74 3.6 Remarks 90 4 REINFORCEMENT LEARNING 93 4.1 Learning Control 94 4.2 Lifelong Control Learning 98 4.3 Q-Learning 102 4.4 Generalizing Function Approximators and Q-Learning 111 4.5 Remarks 125 5 EMPIRICAL RESULTS 131 5.1 Learning Robot Control 132 5.2 Navigation 133 5.3 Simulation 141 5.4 Approaching and Grasping a Cup 146 5.5 NeuroChess 152 5.6 Remarks 175 6 DISCUSSION 177 6.1 Summary 177 6.2 Open Problems 181 6.3 Related Work 185 6.4 Concluding Remarks 192 A AN ALGORITHM FOR APPROXIMATING VALUES AND SLOPES WITH ARTIFICIAL NEURAL NETWORKS 195 A.1 Definitions 196 A.2 Network Forward Propagation 196 A.3 Forward Propagation of Auxiliary Gradients 197 A.4 Error Functions 198 A.5 Minimizing the Value Error 199 A.6 Minimizing the Slope Error 199 A.7 The Squashing Function and its Derivatives 201 A.8 Updating the Network Weights and Biases 202 B PROOFS OF THE THEOREMS 203 C EXAMPLE CHESS GAMES 207 C.1 Game 1 207 C.2 Game 2 219 REFERENCES 227 LIST OF SYMBOLS 253 INDEX 259 ---------------------------------------------------------------------- More information concerning this book: http://www.cs.cmu.edu/~thrun/papers/thrun.book.html http://www.informatik.uni-bonn.de/~thrun/papers/thrun.book.html From hochreit@informatik.tu-muenchen.de Fri Apr 5 03:06:21 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id DAA05571 for ; Fri, 5 Apr 1996 03:06:09 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id DAA06287 for ; Fri, 5 Apr 1996 03:06:07 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa21962; 4 Apr 96 18:52:48 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa21957; 4 Apr 96 18:32:56 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01255; 4 Apr 96 18:32:09 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa16293; 4 Apr 96 11:47:30 EST Received: from papa.informatik.tu-muenchen.de by CS.CMU.EDU id aa00714; 4 Apr 96 11:46:45 EST Received: from gnom.informatik.tu-muenchen.de by papa.informatik.tu-muenchen.de id <116186>; Thu, 4 Apr 1996 18:46:02 +0200 From: Josef Hochreiter To: connectionists@cs.cmu.edu Subject: Flat Minima Message-Id: <96Apr4.184602+0200_met_dst.116186+506@papa.informatik.tu-muenchen.de> Date: Thu, 4 Apr 1996 18:45:52 +0200 FTP-host: flop.informatik.tu-muenchen.de (131.159.8.35) FTP-filename: /pub/articles-etc/hochreiter.fm.ps.gz FLAT MINIMA Sepp Hochreiter Juergen Schmidhuber To appear in Neural Computation (accepted 1996) 38 pages, 154 K compressed, 463 K uncompressed We present a new algorithm for finding low-complexity neural networks with high generalization capability. The algorithm searches for a ``flat'' minimum of the error function. A flat minimum is a large connected region in weight-space where the error remains approximately constant. An MDL-based, Bayesian argument suggests that flat minima correspond to ``simple'' networks and low expected overfitting. The argument is based on a Gibbs algorithm variant and a novel way of splitting generalization error into underfitting and overfitting error. Unlike many previous approaches, ours does not require Gauss- assumptions and does not depend on a ``good'' weight prior - instead we have a prior over input/output functions, thus ta- king into account net architecture and training set. Although our algorithm requires the computation of second order deri- vatives, it has backprop's order of complexity. Automatically, it effectively prunes units, weights, and input lines. Expe- riments with feedforward and recurrent nets are described. In applications to stock market prediction, flat minimum search outperforms conventional backprop, weight decay, ``optimal brain surgeon'' / ``optimal brain damage''. We also provide pseudo code of the algorithm (omitted from the NC-version). To obtain a copy, cut and paste one of these: netscape http://www7.informatik.tu-muenchen.de/~hochreit/pub.html netscape http://www.idsia.ch/~juergen/onlinepub.html Sepp Hochreiter, TUM Juergen Schmidhuber, IDSIA P.S.: Info on recent IDSIA postdoc job opening: http://www.idsia.ch/~juergen/postdoc.html From horne@research.nj.nec.com Fri Apr 5 15:48:27 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id PAA15198 for ; Fri, 5 Apr 1996 15:48:19 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id PAA15483 for ; Fri, 5 Apr 1996 15:48:15 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa21955; 4 Apr 96 18:43:46 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa21953; 4 Apr 96 18:32:05 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa01249; 4 Apr 96 18:32:01 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa16231; 4 Apr 96 11:44:39 EST Received: from zingo.nj.nec.com by RI.CMU.EDU id aa06630; 4 Apr 96 11:44:03 EST Received: from telluride (telluride [138.15.158.83]) by zingo.nj.nec.com (8.7.4/8.7.3) with SMTP id LAA09093 for ; Thu, 4 Apr 1996 11:43:51 -0500 (EST) Received: by telluride (940816.SGI.8.6.9/cliff's joyful mailer #2) id LAA26729(telluride); Thu, 4 Apr 1996 11:46:25 -0500 Date: Thu, 4 Apr 1996 11:46:25 -0500 From: Bill Horne Message-Id: <9604041146.ZM26727@telluride> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: Connectionists@cs.cmu.edu Subject: Spectral Radius TR Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii The following technical report is now available Lower bounds for the spectral radius of a matrix Bill Horne NEC Research Institute 4 Independence Way Princeton, NJ 08540 NECI Technical Report 95-14 In this paper we develop lower bounds for the spectral radius of symmetric, skew-symmetric, and arbitrary real matrices. Our approach utilizes the well-known Leverrier-Faddeev algorithm for calculating the coefficients of the characteristic polynomial of a matrix in conjunction with a theorem by Lucas which states that the critical points of a polynomial lie within the convex hull of its roots. Our results generalize and simplify a proof recently published by Tarazaga for a lower bound on the spectral radius of a symmetric positive definite matrix. In addition, we provide new lower bounds for the spectral radius of skew-symmetric matrices. We apply these results to a problem involving the stability of fixed points in recurrent neural networks. The report can be obtained from my homepage http://www.neci.nj.nec.com/homepages/horne.html Or directly at ftp://ftp.nj.nec.com/pub/horne/spectral.ps.Z -- Bill Horne Senior Research Associate Computer Science Division NEC Research Institute, 4 Independence Way, Princeton, NJ 08540 horne@research.nj.nec.com PHN: (609) 951-2676 FAX: (609) 951-2482 http://www.neci.nj.nec.com/homepages/horne.html From nq6@columbia.edu Sat Apr 6 00:45:02 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id AAA20871 for ; Sat, 6 Apr 1996 00:44:54 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id AAA19864 for ; Sat, 6 Apr 1996 00:44:51 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa23610; 5 Apr 96 15:27:19 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa23597; 5 Apr 96 15:03:28 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02511; 5 Apr 96 15:02:30 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa07296; 5 Apr 96 12:30:15 EST Received: from merhaba.cc.columbia.edu by CS.CMU.EDU id aa10309; 5 Apr 96 12:29:29 EST Received: (from nq6@localhost) by merhaba.cc.columbia.edu (8.7.3/8.7.3) id MAA07569; Fri, 5 Apr 1996 12:29:16 -0500 (EST) Date: Fri, 5 Apr 1996 12:29:16 -0500 (EST) From: Ning Qian Message-Id: <199604051729.MAA07569@merhaba.cc.columbia.edu> To: Connectionists@cs.cmu.edu Subject: postdoc position at Columbia Reply-to: nq6@columbia.edu Postdoctoral Position in Computational Vision Center for Neurobiology and Behavior Columbia University New York, NY A postdoctoral fellowship position in computational neuroscience is available immediately for a recent Ph. D. The postdoc will participate in an NIMH-funded project that applies mathematical analyses and computer simulations to investigate the neural mechanisms of stereoscopic depth perception and motion-stereo interactions. Opportunities for modeling other neural systems are also available. The details of our research interests and the PostScript files of some of our publications can be found at the web site listed below. Other systems neuroscience faculty members in the Center with closely related research interests include Drs. Vincent P. Ferrera, Claude P. Ghez, John Martin and Irving Kaupfermann. The funding for the position is available for two years with the possibility of renewal. Applicants should have a strong background in mathematics and computational modeling (in the Unix/X-windows/C environment). Previous experience in vision research is desirable but not required. Please send a CV, statement of research interests and experience, along with names/phone numbers/email addresses of three references to: Dr. Ning Qian Center for Neurobiology and Behavior Columbia University 722 W. 168th St., A730 New York, NY 10032 nq6@columbia.edu (email) 212-960-2213 (phone) 212-960-2561 (fax) ********************************************************************* For the details of our research interests and publications, please visit our World Wide Web home page at: http://brahms.cpmc.columbia.edu Selected Papers (available on line): A Physiological Model for Motion-stereo Integration and a Unified Explanation of the Pulfrich-like Phenomena, Ning Qian and Richard A. Andersen, submitted to Vision Research. Binocular Receptive Field Profiles, Disparity Tuning and Characteristic Disparity, Yudong Zhu and Ning Qian, Neural Computation, 1996 (in press). Computing Stereo Disparity and Motion with Known Binocular Cell Properties, Ning Qian, Neural Computation, 1994, 6:390-404. Transparent Motion Perception as Detection of Unbalanced Motion Signals III: Modeling, Ning Qian, Richard A. Andersen and Edward H. Adelson, J. Neurosci., 1994, 14:7381-7392. Generalization and Analysis of the Lisberger-Sejnowski VOR Model, Ning Qian, Neural Computation, 1995, 7:735-752. From goldfarb@unb.ca Sat Apr 6 00:45:04 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id AAA20869 for ; Sat, 6 Apr 1996 00:44:51 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id AAA19862 for ; Sat, 6 Apr 1996 00:44:49 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa23595; 5 Apr 96 15:16:46 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa23593; 5 Apr 96 15:02:29 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa02505; 5 Apr 96 15:02:17 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa05258; 5 Apr 96 10:26:10 EST Received: from hermes.csd.unb.ca by EDRC.CMU.EDU id aa18647; 5 Apr 96 10:25:24 EST Received: (from goldfarb@jupiter.sun.csd.unb.ca [131.202.3.10]) by unb.ca (8.7.5/960123-14:25) id LAA08901; Fri, 5 Apr 1996 11:25:12 -0400 (AST) Received: from localhost by jupiter.sun.csd.unb.ca (8.7.4/950215-16:05) id LAA27072; Fri, 5 Apr 1996 11:25:11 -0400 (AST) Date: Fri, 5 Apr 1996 11:25:10 -0400 (AST) From: Lev Goldfarb X-Sender: goldfarb@jupiter.sun.csd.unb.ca To: Steve Kemp cc: connectionists@cs.cmu.edu Subject: Re: What is a "hybrid" model? In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Thu, 4 Apr 1996, Steve Kemp wrote: > learning. If the distinction (that makes for the two types) is between > gradual and sudden learning, then one-trial learning, while perhaps > distinct from insight learning, seems to be sudden rather than gradual. > That is, there are other non-gradual types of learning besides insight > learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > In sum, my point is that it is difficult, if not impossible, to establish > the existence or non-existence of genuinely different *types* of learning > solely from behavioral phenomena, however augmented by theory or > mathematics. In view of this, why do then most of us ignore the scientific experience of the last four centuries that strongly suggest the scientific parsimony (in that case - one basic learning "mechanism")? Are we ready (i.e. adequately "educated") to deal with the greatest scientific challenge of cognitive science? Lev Goldfarb Tel: 506-453-4566 Fax: 506-453-3566 http://wwwos2.cs.unb.ca/profs/goldfarb/goldfarb.htm From shrager@neurocog.lrdc.pitt.edu Sat Apr 6 16:15:14 1996 Received: from lucy.cs.wisc.edu (lucy.cs.wisc.edu [128.105.2.11]) by sea.cs.wisc.edu (8.6.12/8.6.12) with ESMTP id QAA27929 for ; Sat, 6 Apr 1996 16:15:08 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU (TELNET-1.SRV.CS.CMU.EDU [128.2.254.108]) by lucy.cs.wisc.edu (8.6.12/8.6.12) with SMTP id QAA25698 for ; Sat, 6 Apr 1996 16:15:06 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa25172; 6 Apr 96 15:10:53 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa25170; 6 Apr 96 14:54:29 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa03487; 6 Apr 96 14:53:57 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa21500; 6 Apr 96 9:14:56 EST Received: from neurocog.lrdc.pitt.edu by EDRC.CMU.EDU id aa22768; 6 Apr 96 9:14:13 EST Received: by neurocog.lrdc.pitt.edu (1.37.109.10G/16.2) id AA248110042; Sat, 6 Apr 1996 09:14:02 -0500 Date: Sat, 6 Apr 1996 09:14:02 -0500 (EST) From: Jeff Shrager To: Lev Goldfarb Cc: Steve Kemp , connectionists@cs.cmu.edu Subject: Re: What is a "hybrid" model? In-Reply-To: Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Fri, 5 Apr 1996, Lev Goldfarb wrote: > > In sum, my point is that it is difficult, if not impossible, to establish > > the existence or non-existence of genuinely different *types* of learning > > solely from behavioral phenomena, however augmented by theory or > > mathematics. > > In view of this, why do then most of us ignore the scientific experience > of the last four centuries that strongly suggest the scientific parsimony > (in that case - one basic learning "mechanism")? > Are we ready (i.e. adequately "educated") to deal with the greatest > scientific challenge of cognitive science? I'm sorry, but this is all noise. The brain is a complicated machine. Saying that a car runs on "one basic principle" of chemistry (or physics) isn't saying anything important about a car as pertains to most people's interactions with it (except maybe people who are hit by its momentum :-) The "scientific experience of the last four centuries" (at least that little (though important!) spec of it that Lev is apparently referring to) explicitly eschews complexity, or turns it into abstract complexity (such as chaos theory), neither of which approach tells you very much about the real McCoy. If you care about the real brain, the abstract and general theories are important, interesting, and useful, but they are NOT the whole story. I'm sorry to say that this is going to quickly turn into the same old relogious war, and I'd really like to propose that we take it offline. -- Jeff