From zhuh@helios.aston.ac.uk Mon Dec 18 14:47:55 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Mon, 18 Dec 95 14:47:53 -0600; AA08637 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Mon, 18 Dec 95 14:47:50 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa00566; 18 Dec 95 13:44:42 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa00564; 18 Dec 95 13:17:42 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa13054; 18 Dec 95 13:17:21 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa05712; 18 Dec 95 8:13:27 EST Received: from hermes.aston.ac.uk by RI.CMU.EDU id aa02745; 18 Dec 95 8:12:35 EST Received: from email.aston.ac.uk by hermes.aston.ac.uk with SMTP (PP); Mon, 18 Dec 1995 13:16:50 +0000 Received: from darwin by email.aston.ac.uk with SMTP (PP) id <07890-0@email.aston.ac.uk>; Mon, 18 Dec 1995 13:16:29 +0000 From: zhuh Date: Mon, 18 Dec 1995 13:11:50 +0000 Message-Id: <4332.9512181311@sun.aston.ac.uk> To: Connectionists@cs.cmu.edu Subject: Re: NFL and practice X-Sun-Charset: US-ASCII Content-Length: 5881 I accidentally sent my reply to Joerg Lemm, instead of Connnetionist. Since he replied to the Connectionist, I'll reply here as well, and include my original posting at the end. I quite agree with Joerg's observation about learning algorithms in practice, and the priors they use. The key difference is Is it legitimate to be vague about prior? Put it another way, Do you claim the algorithm can pick up whatever prior automatically, instead of being specified before hand? My answer is NO, to both questions, because for an algorithm to be good on any prior is exactly the same as for an algorithm to be good without prior, as NFL told us. For purely cosmetic reasons, it might be helpful to translate the useless "No free lunch theorem" :-) Without specifying a particular prior, any algorithm is as good as random guess, into the equivalent, but infinitely more useful, "You have to pay for lunch Theorem" :-) For an algorithm to perform better than random guess, a particular prior should be specified. On a more practical level, > E.g. in nearly all cases functions are somewhat smooth. Do you specify the scale on which it is smooth? > This is a prior which exists in reality (for example because > of input noise in the measuring process). If you average smoothness over all scales, in a certain uniform way, you get a prior which contains no smoothness at all. If you average them in a non- uniform way, you actually specify a non-uniform prior, which is the crucial piece of information for any algorithm to work at all. > And the situation would we hopeless > if we would not use this fact in practice. It would still be hopeless if we only used the fact of "somewhat smooth", instead of specifying how smooth. See the following for theory and examples: Zhu, H. and Rohwer, R.: Bayesian regression filters and the issue of priors, 1995. To appear in Neural Computing and Applications. ftp://cs.aston.ac.uk/neural/zhuh/reg_fil_prior.ps.Z My original posting is enclosed as the following: ----- Begin Included Message ----- >From zhuh Fri Dec 15 13:04:17 1995 To: lemm@xtp141.uni-muenster.de Subject: Re: NFL and practice Joerg Lemm wrote > One may discuss NFL for theoretical reasons, but > the conditions under which NFL-Theorems hold > are not those which are normally met in practice. Exactly the opposite. The theory behind NFL is trivial (in some sense). The power of NFL is that it deals directly with what is rountinely practiced in the neural network community today. > 1.) In short, NFL assumes that data, i.e. information of the form y_i=f(x_i), > do not contain information about function values on a non-overlapping test set. > This is done by postulating "unrestricted uniform" priors, > or uniform hyperpriors over nonumiform priors... (with respect to Craig's ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > two cases this average would include a third case: target and model are > anticorrelated so anticrossvalidation works better) and "vertical" likelihoods. > So, in a NFL setting data never say something about function values > for new arguments. > This seems rather trivial under this assumption and one has to ask > how natural is such a NFL situation. This is indeed a very trivial and unnatural assumption, which has been criticised by generations of statisticians over several centuries. However, it is exactly what is practiced by a majority of NN researchers. Consider the claim: "This is an algorithm which will perform well as long as there is some nonuniform prior". If such a claim could ever be true, then the algorithm would also be good for a uniform hyperprior over nonuniform priors. But this is in direct contradiction to NFL. According to NFL, you have to say:"This is an algorithm which will perform well on this particular nonuniform prior, (hence it will perform badly on that particular nonuniform prior)". Similarly, with the Law of Energy Conservation, if you say "I've designed a machine to generate electricity", then you automatically imply that you have designed a machine to consume some other forms of energy. You can't make every term positive in your balance sheet, if the grand total is bound to be zero. > Joerg continued with examples of various priors of practical concern, including smoothness, symmetry, positive correlation, iid samples, etc. These are indeed very important priors which match the real world, and they are the implicit assumptions behind most algorithms. What NFL tells us is: If your algorithm is designed for such a prior, then say so explicitly so that a user can decide whether to use it. You can't expect it to be also good for any other prior which you have not considered. In fact, in a sense, you should expect it to perform worse than a purely random algorithm on those other priors. > To conclude: > > In many interesting cases "effective" function values contain information > about other function values and NFL does not hold! This is like saying "In many interesting cases we do have energy sources, and we can make a machine running forever, so the natural laws against `perpetual motion machines' do not hold." These general principles might not be quite obviously interesting to a user, but they are of fundamental importance to a researcher. They are in fact also of fundamental importance to a user, as he must assume the responsibility of supplying the energy source, or specifying the prior. -- Huaiyu Zhu, PhD email: H.Zhu@aston.ac.uk Neural Computing Research Group http://neural-server.aston.ac.uk/People/zhuh Dept of Computer Science ftp://cs.aston.ac.uk/neural/zhuh and Applied Mathematics tel: +44 121 359 3611 x 5427 Aston University, fax: +44 121 333 6215 Birmingham B4 7ET, UK ----- End Included Message ----- From imlm@tuck.cs.fit.edu Tue Dec 19 00:21:09 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 19 Dec 95 00:21:04 -0600; AA14296 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 19 Dec 95 00:21:02 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa00709; 18 Dec 95 17:02:36 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa00707; 18 Dec 95 16:53:27 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa13138; 18 Dec 95 16:52:21 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id ab11975; 18 Dec 95 16:42:30 EST Received: from tuck.cs.fit.edu by EDRC.CMU.EDU id aa01310; 18 Dec 95 16:41:21 EST Received: (from imlm@localhost) by tuck.cs.fit.edu (8.6.11/8.6.11) id QAA10740; Mon, 18 Dec 1995 16:39:40 -0500 Date: Mon, 18 Dec 1995 16:39:40 -0500 Message-Id: <199512182139.QAA10740@tuck.cs.fit.edu> From: IMLM Workshop (pkc) To: ml@ics.uci.edu, kdd@gte.com, INDUCTIVE@hermes.csd.unb.ca, DAI-List@ece.sc.edu, GA-List@AIC.NRL.NAVY.MIL, Connectionists@cs.cmu.edu, ai-stats@watstat.uwaterloo.ca, dbworld@lucy.cs.wisc.edu Cc: imlm@tuck.cs.fit.edu, sal@cs.columbia.edu, dhw@santafe.edu Subject: CFP: AAAI-96 Workshop on Integrating Multiple Learned Models CALL FOR PAPERS/PARTICIPATION INTEGRATING MULTIPLE LEARNED MODELS FOR IMPROVING AND SCALING MACHINE LEARNING ALGORITHMS to be held in conjunction with AAAI 1996 Portland, Oregon August 1996 Most modern machine learning research uses a single model or learning algorithm at a time, or at most selects one model from a set of candidate models. Recently however, there has been considerable interest in techniques that integrate the collective predictions of a set of models in some principled fashion. With such techniques often the predictive accuracy and/or the training efficiency of the overall system can be improved, since one can "mix and match" among the relative strengths of the models being combined. The goal of this workshop is to gather researchers actively working in the area of integrating multiple learned models, to exchange ideas and foster collaborations and new research directions. In particular, we seek to bring together researchers interested in this topic from the fields of Machine Learning, Knowledge Discovery in Databases, and Statistics. Any aspect of integrating multiple models is appropriate for the workshop. However we intend the focus of the workshop to be improving prediction accuracies, and improving training performance in the context of large training databases. More precisely, submissions are sought in, but not limited to, the following topics: 1) Techniques that generate and/or integrate multiple learned models. In particular, techniques that do so by: * using different training data distributions (in particular by training over different partitions of the data) * using different output classification schemes (for example using output codes) * using different hyperparameters or training heuristics (primarily as a tool for generating multiple models) 2) Systems and architectures to implement such strategies. In particular: * parallel and distributed multiple learning systems * multi-agent learning over inherently distributed data A paper need not be submitted to participate in the workshop, but space may be limited so contact the organizers as early as possible if you wish to participate. The workshop format is planned to encompass a full day of half hour presentations with discussion periods, ending with a brief period for summary and discussion of future activities. Notes or proceedings for the workshop may be provided, depending on the submissions received. Submission requirements: i) A short paper of not more than 2000 words detailing recent research results must be received by March 18, 1996. ii) The paper should include an abstract of not more than 150 words, and a list of keywords. Please include the name(s), email address(es), address(es), and phone number(s) of the author(s) on the first page. The first author will be the primary contact unless otherwise stated. iii) Electronic submissions in postscript or ASCII via email are preferred. Three printed copies (preferrably double-sided) of your submission are also accepted. iv) Please also send the title, name(s) and email address(es) of the author(s), abstract, and keywords in ASCII via email. Submission address: imlm@cs.fit.edu Philip Chan IMLM Workshop Computer Science Florida Institute of Technology 150 W. University Blvd. Melbourne, FL 32901-6988 407-768-8000 x7280 (x8062) 407-984-8461 (fax) Important Dates: Paper submission deadline: March 18, 1996 Notification of acceptance: April 15, 1996 Final copy: May 13, 1996 Chairs: Salvatore Stolfo, Columbia University sal@cs.columbia.edu David Wolpert, Santa Fe Institute dhw@santafe.edu Philip Chan, Florida Institute of Technology pkc@cs.fit.edu General Inquiries: Please address general inquiries to one of the co-chairs or send them to: imlm@cs.fit.edu Up-to-date workshop information is maintained on WWW at: http://cs.fit.edu/~imlm/ or http://www.cs.fit.edu/~imlm/ From ces@negi.riken.go.jp Tue Dec 19 00:21:12 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 19 Dec 95 00:21:06 -0600; AA14298 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 19 Dec 95 00:21:03 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa00888; 18 Dec 95 20:55:33 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa00886; 18 Dec 95 20:46:01 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa13285; 18 Dec 95 20:45:20 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa14257; 18 Dec 95 20:30:41 EST Received: from negi.riken.go.jp by CS.CMU.EDU id aa18598; 18 Dec 95 20:29:48 EST Received: by negi.riken.go.jp; id AA21982; Tue, 19 Dec 1995 10:36:46 +0900 Message-Id: <9512190136.AA21982@negi.riken.go.jp> To: Connectionists@cs.cmu.edu Subject: PhD Thesis Announcement : nonlinear filters Date: Tue, 19 Dec 95 10:36:45 +0900 From: ces@negi.riken.go.jp X-Mts: smtp  FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/Thesis/chng.thesis.ps.Z Dear fellow connectionists, the following Ph.D. thesis is now available for copying from the neuroprose archive: (Sorry, no hardcopies available.) - ----------------------------------------------------------------------- Applications of nonlinear filters with the linear-in-the-parameter structure Eng-Siong CHNG Department of Electrical Engineering University of Edinburgh, U.K. Abstract: The subject of this thesis is the application of nonlinear filters, with the linear-in-the-parameter structure, to time series prediction and channel equalisation problems. In particular, the Volterra and the radial basis function (RBF) expansion techniques are considered to implement the nonlinear filter structures. These approaches, however, will generate filters with very large numbers of parameters. As large filter models require significant implementation complexity, they are undesirable for practical implementations. To reduce the size of the filter, the orthogonal least squares (OLS) algorithm is considered to perform model selection. Simulations were conducted to study the effectiveness of subset models found using this algorithm, and the results indicate that this selection technique is adequate for many practical applications. The other aspect of the OLS algorithm studied is its implementation requirements. Although the OLS algorithm is very efficient, the required computational complexity is still substantial. To reduce the processing requirement, some fast OLS methods are examined. Two major applications of nonlinear filters are considered in this thesis. The first involves the use of nonlinear filters to predict time series which possess nonlinear dynamics. To study the performance of the nonlinear predictors, simulations were conducted to compare the performance of these predictors with conventional linear predictors. The simulation results confirm that nonlinear predictors normally perform better than linear predictors. Within this study, the application of RBF predictors to time series that exhibit homogeneous nonstationarity is also considered. This type of time series possesses the same characteristic throughout the time sequence apart from local variations of mean and trend. The second application involves the use of filters for symbol-decision channel equalisation. The decision function of the optimal symbol-decision equaliser is first derived to show that it is nonlinear, and that it may be realised explicitly using a RBF filter. Analysis is then carried out to illustrate the difference between the optimum equaliser's performance and that of the conventional linear equaliser. In particular, the effects of delay order on the equaliser's decision boundaries and bit error rate (BER) performance are studied. The minimum mean square error (MMSE) optimisation criterion for training the linear equaliser is also examined to illustrate the sub-optimum nature of such a criterion. To improve the linear equaliser's performance, a method which adapts the equaliser by minimising the BER is proposed. Our results indicate that the linear equalisers performance is normally improved by using the minimum BER criterion. The decision feedback equaliser (DFE) is also examined. We propose a transformation using the feedback inputs to change the DFE problem to a feedforward equaliser problem. This unifies the treatment of the equaliser structures with and without decision feedback. ----------------------------------------------------------- Criticism, comments and suggestions are welcome. Merry Christmas everyone! Eng Siong - -------------------------------------------------------------------------- Eng Siong CHNG Lab. for ABS, Frontier Research Programme, RIKEN, email : ces@negi.riken.go.jp 2-1 Hirosawa, Wako-Shi, Saitama 351-01, JAPAN. - -------------------------------------------------------------------------- RETRIEVAL INSTRUCTIONS: FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/Thesis/chng.thesis.ps.Z File size : 1715073 bytes Number of pages : 165 pages unix> ftp archive.cis.ohio-state.edu Connected to archive.cis.ohio-state.edu. 220 archive.cis.ohio-state.edu FTP server ready. Name: anonymous 331 Guest login ok, send ident as password. Password:neuron 230 Guest login ok, access restrictions apply. ftp> binary 200 Type set to I. ftp> cd pub/neuroprose/Thesis 250 CWD command successful. ftp> get chng.thesis.ps.Z 200 PORT command successful. 150 Opening BINARY mode data connection for chng.thesis.ps.Z 226 Transfer complete. ftp> quit 221 Goodbye. unix> uncompress chng.thesis.ps.Z unix> lpr chng.thesis.ps (postscript printer) Contact me if there are any problems with retrieval and or printing. ------- End of Forwarded Message From hag@santafe.edu Tue Dec 19 00:21:13 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 19 Dec 95 00:21:08 -0600; AA14304 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 19 Dec 95 00:21:05 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa01010; 18 Dec 95 22:05:19 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa01007; 18 Dec 95 21:57:57 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa13309; 18 Dec 95 21:57:51 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa14720; 18 Dec 95 21:26:49 EST Received: from sfi.santafe.edu by RI.CMU.EDU id aa06084; 18 Dec 95 21:26:00 EST Received: from grasshopper (grasshopper.santafe.edu) by sfi.santafe.edu (4.1/SMI-4.1) id AA29140; Mon, 18 Dec 95 19:22:57 MST From: "Howard A. Gutowitz" Message-Id: <9512190222.AA29140@sfi.santafe.edu> Subject: Exploring the Space of CA To: Connectionists@cs.cmu.edu Date: Mon, 18 Dec 1995 19:22:57 -0700 (MST) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 927 Announcing: "Exploring the Space of Cellular Automata" Cellular automata can be thought of as a restricted kind of neural net, in which the cells take on only a finite set of values, and connections are local and regular. This is set of interactive web pages designed to help you learn about CA, and the use of the lambda parameter to find critical regions in the space of CA. Credits: Concept: Chris Langton CA simulation program: Patrick Hayden. cgi interface: Eric Carr. Text: Chris Langton , Howard Gutowitz, and Eric Carr. Available from: http://alife.santafe.edu/alife/topics/ca/caweb -- Howard Gutowitz | hag@neurones.espci.fr ESPCI | http://www.santafe.edu/~hag Laboratoire d'Electronique | home: (331) 4707-3843 10 rue Vauquelin | office: (331) 4079-4697 75005 Paris, France | fax: (331) 4079-4425 From hicks@cs.titech.ac.jp Tue Dec 19 17:12:38 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Tue, 19 Dec 95 17:12:31 -0600; AA26779 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Tue, 19 Dec 95 17:12:29 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa01458; 19 Dec 95 2:16:54 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa01456; 19 Dec 95 2:08:27 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa13484; 19 Dec 95 2:07:32 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa16234; 19 Dec 95 0:01:17 EST Received: from euclid.cs.titech.ac.jp by CS.CMU.EDU id aa19618; 18 Dec 95 23:59:34 EST Received: by euclid.cs.titech.ac.jp (8.6.11+2.4W/3.2W-titmx-2.0g); Tue, 19 Dec 1995 13:58:07 +0900 Date: Tue, 19 Dec 1995 13:58:07 +0900 From: hicks@cs.titech.ac.jp Message-Id: <199512190458.NAA28669@euclid.cs.titech.ac.jp> To: zhuh@helios.aston.ac.uk, connectionists@cs.cmu.edu Cc: hicks@cs.titech.ac.jp, lemm@LORENTZ.UNI-MUENSTER.DE, cherkaue@lucy.cs.wisc.edu Subject: Re: NFL, practice, and CV Huaiyu Zhu wrote: >You can't make every term positive in your balance sheet, if the grand >total is bound to be zero. There ARE functions which are always non-negative, but which under an appropriate measure integrate to 0. It only requires that 1) the support of the non-negative values is vanishingly small, 2) the non-negative values are bounded So the above statement by Dr. Zhu is not true. In fact I think this ability for pointwise positive values to dissapear under integration is key to the "zero-sum" aspect of the NFL theorem holding true, despite the fact that we obviously see so many examples of working algorithms. My key point: A zero-sum (infinite) universe doesn't require negative values. ---- There is another important issue which needs to be clarified, and that is the definition of CV and the kinds of problems to which it can be applied. Now anybody can make whatever definition they want, and then come to some conclusions based upon that definition, and that conclusion may be correct given that definition. However, there are also advantages to sharing a common intellectual currency. I quote below from "An Introduction to the Bootstrap" by Efron and Tibshirani, 1993, Chapter 17.1. It describes well what I meant when I talked monitoring prediction error in a previous posting, and describes CV as a method for doing that. ================================================== In our discussion so far we have focused on a number of measures of statistical accuracy: standard errors, biases, and confidence intervals. All of these are measures of accuracy for parameters of a model. Prediction error is a different quantity that measures how well a model predicts the response value of a future observation. It is often used for model selection, since it is ensible ot choose a model that has the lowest prediction error among a set of candidates. Cross-validation is a standard tool for estimating prediction error. It is an old idea (predating the bootstrap) that has enjoyed a comeback in recent years with the increase in available computing power and speed. In this chapter we discuss cross-validation, the bootstrap, and some other closely related techniques for estimation of prediction error. In regression models, prediction error refers to the expected squared difference between a future response and its prediction from the model: PE = E(y - \hat{y})^2. The expectation refers to repeated sampling from the true population. Prediction error also arises in th eclassification problem, where the repsponse falls into one of k unordered classes. For example, the possible reponses might be Republican, Democrat, or Independent in a political survey. In classification problems prediction error is commonly defined as the probability of an incorrect classification PE = Prob(\hat{y} \neq y), also called the misclassification rate. The methods described in this chapter apply to both definitions of prediction error, and also to others. ================================================== Craig Hicks Tokyo Institute of Technology From lucas@scr.siemens.com Wed Dec 20 01:33:10 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 20 Dec 95 01:33:08 -0600; AA01810 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 20 Dec 95 01:33:00 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa03067; 20 Dec 95 0:24:09 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa03056; 20 Dec 95 0:08:23 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa14024; 20 Dec 95 0:06:54 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa26123; 19 Dec 95 13:03:29 EST Received: from scr.siemens.com by EDRC.CMU.EDU id aa05039; 19 Dec 95 13:03:15 EST Received: from owl.scr.siemens.com (owl.scr.siemens.com [129.73.10.76]) by scr.siemens.com (8.7.1/8.6.6) with ESMTP id MAA18011 for ; Tue, 19 Dec 1995 12:42:10 -0500 (EST) Received: (from lucas@localhost) by owl.scr.siemens.com (8.7.1/8.6.6) id MAA04146 for Connectionists@cs.cmu.edu; Tue, 19 Dec 1995 12:26:15 -0500 (EST) Date: Tue, 19 Dec 1995 12:26:15 -0500 (EST) From: Lucas Parra Message-Id: <199512191726.MAA04146@owl.scr.siemens.com> To: Connectionists@cs.cmu.edu Subject: Preprint: Symplectic Nonlinear Component Analysis Dear fellow connectionists, a preprint of the following NIPS*95 paper is available at: ftp://archive.cis.ohio-state.edu/pub/neuroprose/parra.nips95.ps.Z Symplectic Nonlinear Component Analysis Lucas C. Parra Siemens Corporate Research lucas@scr.siemens.com Statistically independent features can be extracted by finding a factorial representation of a signal distribution. Principal Component Analysis (PCA) accomplishes this for linear correlated and Gaussian distributed signals. Independent Component Analysis (ICA), formalized by Comon (1994), extracts features in the case of linear statistical dependent but not necessarily Gaussian distributed signals. Nonlinear Component Analysis finally should find a factorial representation for nonlinear statistical dependent distributed signals. This paper proposes for this task a novel feed-forward, information conserving, nonlinear map - the explicit symplectic transformations. It also solves the problem of non-Gaussian output distributions by considering single coordinate higher order statistics. From minton@ISI.EDU Wed Dec 20 01:33:12 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 20 Dec 95 01:33:09 -0600; AA01812 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 20 Dec 95 01:33:07 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id ab03067; 20 Dec 95 0:25:12 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id ab03056; 20 Dec 95 0:08:29 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa14029; 20 Dec 95 0:07:07 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa27758; 19 Dec 95 14:55:57 EST Received: from venera.isi.edu by CS.CMU.EDU id aa24800; 19 Dec 95 14:55:17 EST Received: from sungod.isi.edu by venera.isi.edu (5.65c/5.61+local-22) id ; Tue, 19 Dec 1995 11:54:55 -0800 Date: Tue, 19 Dec 95 11:53:27 PST From: minton@ISI.EDU Posted-Date: Tue, 19 Dec 95 11:53:27 PST Message-Id: <9512191953.AA11913@sungod.isi.edu> Received: by sungod.isi.edu (4.1/4.0.3-4) id ; Tue, 19 Dec 95 11:53:27 PST To: ai-stats@watstat.uwaterloo.ca, kdd@gte.com, connectionists@cs.cmu.edu Subject: JAIR article Readers of this mailing list may be interested in the following JAIR article, which was just published: Weiss, S.M. and Indurkhya, N. (1995) "Rule-based Machine Learning Methods for Functional Prediction", Volume 3, pages 383-403. PostScript: volume3/weiss95a.ps (527K) compressed, volume3/weiss95a.ps.Z (166K) Abstract: We describe a machine learning method for predicting the value of a real-valued function, given the values of multiple input variables. The method induces solutions from samples in the form of ordered disjunctive normal form (DNF) decision rules. A central objective of the method and representation is the induction of compact, easily interpretable solutions. This rule-based decision model can be extended to search efficiently for similar cases prior to approximating function values. Experimental results on real-world data demonstrate that the new techniques are competitive with existing machine learning and statistical methods and can sometimes yield superior regression performance. The PostScript file is available via: -- comp.ai.jair.papers -- World Wide Web: The URL for our World Wide Web server is http://www.cs.washington.edu/research/jair/home.html -- Anonymous FTP from either of the two sites below: CMU: p.gp.cs.cmu.edu directory: /usr/jair/pub/volume3 Genoa: ftp.mrg.dist.unige.it directory: pub/jair/pub/volume3 -- automated email. Send mail to jair@cs.cmu.edu or jair@ftp.mrg.dist.unige.it with the subject AUTORESPOND, and the body GET VOLUME3/FILE-NM (e.g., GET VOLUME3/MOONEY95A.PS) Note: Your mailer might find our files too large to handle. Also, note that compressed files cannot be emailed, since they are binary files. -- JAIR Gopher server: At p.gp.cs.cmu.edu, port 70. For more information about JAIR, check out our WWW or FTP sites, or send electronic mail to jair@cs.cmu.edu with the subject AUTORESPOND and the message body HELP, or contact jair-ed@ptolemy.arc.nasa.gov. From zhuh@helios.aston.ac.uk Wed Dec 20 14:19:25 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 20 Dec 95 14:19:17 -0600; AA12958 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 20 Dec 95 14:19:14 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa03054; 20 Dec 95 0:16:49 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa03051; 20 Dec 95 0:07:32 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa14017; 20 Dec 95 0:06:24 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa23404; 19 Dec 95 10:15:32 EST Received: from hermes.aston.ac.uk by EDRC.CMU.EDU id aa04364; 19 Dec 95 10:14:47 EST Received: from email.aston.ac.uk by hermes.aston.ac.uk with SMTP (PP); Tue, 19 Dec 1995 15:19:09 +0000 Received: from darwin by email.aston.ac.uk with SMTP (PP) id <27720-0@email.aston.ac.uk>; Tue, 19 Dec 1995 15:19:02 +0000 From: zhuh Date: Tue, 19 Dec 1995 15:14:20 +0000 Message-Id: <8208.9512191514@sun.aston.ac.uk> To: connectionists@cs.cmu.edu Subject: Re: NFL, practice, and CV X-Sun-Charset: US-ASCII Content-Length: 5629 This is in reply to the critisism by Craig Hicks and Kevin Cherkauer, and will be my last posting in this thread. Craig Hicks thought that my statement (A) > >You can't make every term positive in your balance sheet, if the grand > >total is bound to be zero. is contradictory to his statements (B) > There ARE functions which are always non-negative, but which under > an appropriate measure integrate to 0. > It only requires that > > 1) the support of the non-negative values is vanishingly small, > 2) the non-negative values are bounded But they are actually talking about different things. There is a big difference between positive and non-negative. For all practical purposes, the functions described by (B) can be regarded as identically zero. Translating back to the original topic, statement (B) becomes (C) There are algorithms which are always no worse than random guessing, on any prior, provided that 1) The priors on which it performs better than random guessing have zero probability to occur in practice. 2) It cannot be infinitely better on these priors. It is true that something improbable may still be possible, but this is only of academic interest. In most of modern treatment of function spaces, functions are only identified up to a set of measure zero, so that phrases like "almost everywhere" or "almost surely" are redundent. I suspect that due to the way NFL are proved, even (C) is impossible, but this does not matter anyway, because (C) itself is of no practical interest whatsoever. > ---- Considering cross validation, Craig wrote > > There is another important issue which needs to be clarified, and that is the > definition of CV and the kinds of problems to which it can be applied. Now > anybody can make whatever definition they want, and then come to some > conclusions based upon that definition, and that conclusion may be correct > given that definition. However, there are also advantages to sharing a common > intellectual currency. Risking a little bit over-simplification, I would like to summarise the two usages of CV as the following (CV1) A method for evaluating estimates, (CV2) A method for evaluating estimators. The key difference is that in (CV1), a decision is made for each sample, while in (CV2) a decision is made for all samples. If (CV1) is applied on two algorithms A and B, then we can always define a third algorithm C, by always choosing the estimate given by either A or B which is favoured by (CV1). But my previous counter-example shows that averaging over all samples, C can be worse than A. One may seek refuge in statements like "optimal decision for each sample does not mean optimal decision for all samples". Well, such incoherent inference is the defining characteristic of non-Bayesian statistics. In Bayesian decision theory it is well known that A method is optimal iff it is optimal on almost all samples, (excluding various measure zero anomolies.) The case of (CV2) is quite different. It is of a higher level than algorithms like A and B. It is in fact a statistical estimator mapping (D,A,f) to to a real number r, where D is a finite data set, A is a given algorithm, f is an objective function, and r is the predicted average performance. It should therefore be compared with other such methods. This appears not to be a topic considered in this discussion. -------------- Kevin Cherkauer wrote > > You forgot > > D: Anti-cross validation to choose between A and B, with one extra data > point. Well, I did not forget that, as you have quoted below, point 6. > > I don't understand your claim that "cross validation IS harmful in this case." > You seem to equate "harmful" with "suboptimal." See my original answer, points 1. and 4. > Cross validation is a technique > we use to guess the answer when we don't already know the answer. This is true for any statistical estimator. > You give > technique A the benefit of your prior knowledge of the true answer, but C must > operate without this knowledge. The prior knowledge is that the distribution is a unit Gaussian with unspecified mean, the true answer is its mean. No, they are not the same thing. C also operates with the knowledge that the distribution is a unit Gaussian, but it refuses to use this knowledge (which implies A is better than B). Instead, it insists on evaluating A and B on a cross validation set. That's why it performs miserably. > A fair comparison would pit C against D, not C > against A. As you say: > > >6. In any of the above cases, "anti cross validation" would be even > >more disastrous. If the definition was that "An algorithm is good if it is no worse than the worst algorithm", then I would have no objection. Well, almost any algorithm would be good in this sense. However, if the phrase "in any of the above cases" is droped without putting a prior restriction as remedy, then it's also true that all algorithm is as bad as the worst algorithm. Huaiyu PS. I think I have already talked enough about this subject so I'll shut up from now on, unless there's anything new to say. More systematic treatment of these subjects instead of counter-examples can be found in the ftp site below. -- Huaiyu Zhu, PhD email: H.Zhu@aston.ac.uk Neural Computing Research Group http://neural-server.aston.ac.uk/People/zhuh Dept of Computer Science ftp://cs.aston.ac.uk/neural/zhuh and Applied Mathematics tel: +44 121 359 3611 x 5427 Aston University, fax: +44 121 333 6215 Birmingham B4 7ET, UK From jlm@crab.PSY.CMU.EDU Wed Dec 20 21:52:38 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Wed, 20 Dec 95 21:52:36 -0600; AA17358 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Wed, 20 Dec 95 21:52:32 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa04715; 20 Dec 95 18:34:30 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa04712; 20 Dec 95 18:20:37 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa14850; 20 Dec 95 18:20:26 EST Received: from EDRC.CMU.EDU by B.GP.CS.CMU.EDU id aa20050; 20 Dec 95 18:18:27 EST Received: from ANDREW.CMU.EDU by EDRC.CMU.EDU id aa11125; 20 Dec 95 18:16:56 EST Received: from crab.psy.cmu.edu.psy.cmu.edu (CRAB.PSY.CMU.EDU [128.2.248.139]) by andrew.cmu.edu (8.7.1/8.7.1) with SMTP id SAA13138 for ; Wed, 20 Dec 1995 18:16:34 -0500 Received: by crab.psy.cmu.edu.psy.cmu.edu (4.1/SMI-4.1) id AA19275; Wed, 20 Dec 95 18:16:31 EST Date: Wed, 20 Dec 95 18:16:31 EST From: "James L. McClelland" Message-Id: <9512202316.AA19275@crab.psy.cmu.edu.psy.cmu.edu> To: connectionists@cs.cmu.edu Subject: Technical Report Available The following Technical Report is available electronically from our FTP server or in hard copy form. Instructions for obtaining copies may be found at the end of this post. ======================================================================== On the Time Course of Perceptual Choice: A Model Based on Principles of Neural Computation Marius Usher & James L. McClelland Carnegie Mellon University and the Center for the Neural Basis of Cognition Technical Report PDP.CNS.95.5 December 1995 The time course of information processing is discussed in a model based on leaky, stochastic, non-linear accumulation of activation in mutually inhibitory processing units. The model addresses data from choice tasks using both time-controlled (e.g., deadline or response signal) and standard reaction time paradigms, and accounts simultaneously for aspects of data from both paradigms. In special cases, the model becomes equivalent to a classical diffusion process, but in general a more complex type of diffusion occurs. Mutual inhibition counteracts the effects of information leakage, allows flexible choice behavior regardless of the number of alternatives, and contributes to accounts of additional data from tasks requiring choice with conflict stimuli and word identification tasks. ====================================================================== Retrieval information for pdp.cns TRs: unix> ftp 128.2.248.152 # hydra.psy.cmu.edu Name: anonymous Password: ftp> cd pub/pdp.cns ftp> binary ftp> get pdp.cns.95.5.ps.Z # gets this tr ftp> quit unix> zcat pdp.cns.95.5.ps.Z | lpr # or however you print postscript NOTE: The compressed file is 567,075 bytes long. Uncompressed, the file is 1,768,398 byes long. The printed version is 53 total pages long. For those who do not have FTP access, physical copies can be requested from Barbara Dorney . For a list of available PDP.CNS Technical Reports: > get README For the titles and abstracts: > get ABSTRACTS From dhw@santafe.edu Thu Dec 21 12:24:38 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 21 Dec 95 12:24:33 -0600; AA26052 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 21 Dec 95 12:24:30 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa04899; 20 Dec 95 21:24:33 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa04897; 20 Dec 95 21:13:33 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa14969; 20 Dec 95 21:13:19 EST Received: from RI.CMU.EDU by B.GP.CS.CMU.EDU id aa21353; 20 Dec 95 20:04:04 EST Received: from sfi.santafe.edu by RI.CMU.EDU id aa15394; 20 Dec 95 20:03:42 EST Received: from atalaya (atalaya.santafe.edu) by sfi.santafe.edu (4.1/SMI-4.1) id AA06007; Wed, 20 Dec 95 18:00:48 MST Date: Wed, 20 Dec 95 18:00:48 MST From: David Wolpert Message-Id: <9512210100.AA06007@sfi.santafe.edu> To: Connectionists@cs.cmu.edu Subject: NFL once again, I'm afraid First and foremost, I would like to request that this NFL thread fade out. It is only sowing confusion - people should read the papers on NFL to understand NFL. [[ Moderator's note: I concur. We've had enough "No Free Lunch" discussion for a while; people are starting to protest. Future discussion should be done in email. -- Dave Touretzky, CONNECTIONISTS moderator ]] Full stop. *After* that, after there is common grounding, we can all debate. There is much else that connectionist is more appropriate for in the meantime. (To repeat: ftp.santafe.edu, pub/dhw_ftp, nfl.1.ps.Z and nfl.2.ps.Z.) Please, I'm on my knees, use the time that would have been spent thrashing at connectionist in a more fruitful fashion. Like by reading the NFL papers. :-) *** Hicks writes: >>> case 1: * Either the target function is (noise/uncompressible/has no structure), or none of the candidate functions have any correlation with the target function.* Since CV provides an estimated prediction error, it can also tell us "you might as well be using anti-cross validation, or random selection for that matter, because it will be equally useless". >>> This is wrong. Construct the following algorithm: "If CV says one of the algorithms under consideration has particularly low error in comparison to the other, use that algorithm. Otherwise, choose randomly among the algorithms." Averaged over all targets, this will do exactly as well as the algorithm that always guesses randomly among the algorithms. (For zero-one loss, either OTS error or IID error with a big input space, etc.) So you cannot rely on CV's error estimate *at all* (unless you impose a prior over targets or some such, etc.). Alternatively, keep in mind the following simple argument: In its uniform prior(targets) formulation, NFL holds even for error distributions conditioned on *any* property of the training set. So in particular, you can condition on having a training set for which CV says "yep, I'm sure; choose that one". And NFL still holds. So even in those cases where CV "is sure", by following CV, you'll die as often as not. >>> case 2: * The target (is compressible/has structure), and some the candidate functions are positively correlated with the target function.* In this case CV will outperform anti-CV (ON AVERAGE). >>> This is wrong. As has been mentioned many times, having structure in the target, by itself, gains you nothing. And as has also been mentioned, if "the candidate functions are positively correlated with the target function", then in fact *anti-CV wins*. READ THE PAPERS. >>> By ON AVERAGE I mean the expectation across the ensemble of samples for a FIXED target function. This is different from the ensemble and distribution of target functions, which is a much bigger question. >>> This distinction is irrelevent. There are versions of NFL that address both of these cases (as well as many others). READ THE PAPERS. ***** Lemm writes: >>> 1.) In short, NFL assumes that data, i.e. information of the form y_i=f(x_i), do not contain information about function values on a non-overlapping test set. >>> This is wrong. See all the previous discussion about how NFL holds even if you restrict yourself to targets with a lot of structure. The problem is that the structure can hurt just as easily as help. There is no need for the data set to contain no information about the test set - simply that the limited types of information can "confuse" the learning algorithm at hand. READ THE PAPERS. >>> This is done by postulating "unrestricted uniform" priors, or uniform hyperpriors over nonumiform priors... >>> This is wrong. There is (obviously) a version of NFL that holds for uniform priors. And there is another version in which one averages over all priors - so the uniform prior has measure 0. But one can also restrict oneself to average only over those priors "with a lot of structure", and again get NFL. And there are many other versions of NFL in which there is *no* prior, because things are conditioned on a fixed target. Exactly as in (non-Bayesian) sampling theory statistics. Some of those alternative NFL results involve saying "if you're conditioning on a target, there are as many such targets where you die as where you do well". Other NFL results never vary the target *in any sense*, even to compare different targets. Rather they vary something concerning the generalizer. This is the case with the more sophisticated xvalidation results, for example. READ THE PAPERS. >>> There is much information which is not of this "single sharp data" type. (Examples see below.) >>> *Obviously* if you have extra information and/or knowledge beyond that in the training set, you can (often) do better than randomly. That's what Bayesian analysis is all about. More generally, as I have proven in [1], the probability of error can be written as a non-Euclidean inner product between the learning algorithm and the posterior. So obviously if your posterior is structured in an appropriate manner, that can be exploited by the algorithm. This was never the issue however. The issue had to do with "blind" supervised learning, in which one has no such additional information. Like in COLT, for example. You're arguing apples and oranges here. >>> 4) Real measurements (especially of continuous variables) normally do also NOT have the form y_i=f(x_i) ! They mostly perform some averaging over f(x_i) or at least they have some noise on the x_i (as small as you like, but present). >>> Again, this is obvious. And stated explicitly in the papers, moreover. And completely irrelevent to the current discussion. The issue at hand has *always* been "sharp" data. And if you look at what's done in the neural net community, or in COLT, 95% of it assumes "sharp data". Indeed, there are many other assumptions almost always made and almost never true that Lemm has missed. Like making a "weak filtering assumption": assume the target and the distribution over inputs are independent. But again, just like in COLT, we're starting simple here, with such assumptions intact. READ THE PAPERS. >>> This shows that smoothness of the expectation (in contrast to uniform priors) is the result of the measurement process and therefore is a real phenomena for "effective" functions. >>> To give one simple example, what about with categorical data, where there is not even a partial ordering over the inputs? What does "locally smooth" even mean then? And even if we're dealing with real valued spaces, if there's input space noise, NFL simply changes to be a statement concerning test set elements that are sufficiently far (on the scale of the input space noise) from the elements of the training set. The input space noise makes the math more messy, but doesn't change the underlying phenomenon. (Readers interested in previous work on the relationship between local (!) regularization, smoothness, and input noise should see Bishop's Neural Computation article of about 6 months ago.) >>> Even more: situations without "priors" are VERY artificial. So if we specify the "priors" (and the lesson from NFL is that we should if we want to make a good theory) then we cannot use NFL anymore.(What should it be used for then?) >>> Sigh. 1) I am a Bayesian whenever feasible. (In fact, I've been taken to task for being "too Bayesian".) But situations without obvious priors - or where eliciting the priors is not trivial and you don't have the time - are in fact *very* common. A simple example is a project I am currently involved on for detecting phone fraud for MCI. Quick, tell me the prior probability that a fraudulent call arises from area code 617 vs. the prior probability that a non-fraudulent call does... 2) Essentially all of COLT is non-Bayesian. (Although some of it makes assumptions about things like the support of the priors.) You haven't a prayer of really understanding what COLT has to say without keeping in mind the admonitions of NFL. 3) As I've now said until I'm blue in the face, NFL is only the starting point. What it's "good for", beyond proving to people that they must pay attention to their assumptions, be wary of COLT-type claims, etc. is: head-to-head minimax theory, scrambled algorithms theory, hypothesis-averaging theory, etc., etc., etc. READ THE PAPERS. **** Zhu writes: >>> I quite agree with Joerg's observation about learning algorithms in practice, and the priors they use. The key difference is Is it legitimate to be vague about prior? Put it another way, Do you claim the algorithm can pick up whatever prior automatically, instead of being specified before hand? My answer is NO, to both questions, because for an algorithm to be good on any prior is exactly the same as for an algorithm to be good without prior, as NFL told us. >>> Yes! Everybody, LISTEN TO ZHU!!!! David Wolpert [1] - Wolpert, D. "The Relationshop Between PAC, the Statistical Physics Framework, the Bayesian Framework, and the VC Framework", in "The Mathematics of Generalization", D. Wolpert (Ed.), Addison-Wesley, 1995 From terry@salk.edu Thu Dec 21 12:24:39 1995 Received: from lucy.cs.wisc.edu by sea.cs.wisc.edu; Thu, 21 Dec 95 12:24:35 -0600; AB26054 Received: from TELNET-1.SRV.CS.CMU.EDU by lucy.cs.wisc.edu; Thu, 21 Dec 95 12:24:33 -0600 Received: from TELNET-1.SRV.CS.CMU.EDU by telnet-1.srv.cs.CMU.EDU id aa04908; 20 Dec 95 21:32:58 EST Received: from DST.BOLTZ.CS.CMU.EDU by TELNET-1.SRV.CS.CMU.EDU id aa04901; 20 Dec 95 21:14:23 EST Received: from DST.BOLTZ.CS.CMU.EDU by DST.BOLTZ.CS.CMU.EDU id aa14974; 20 Dec 95 21:13:38 EST Received: from CS.CMU.EDU by B.GP.CS.CMU.EDU id aa21580; 20 Dec 95 20:34:24 EST Received: from helmholtz.salk.edu by CS.CMU.EDU id aa05767; 20 Dec 95 20:34:17 EST Received: by salk.edu (4.1/SMI-4.1) id AA16333; Wed, 20 Dec 95 17:34:15 PST Date: Wed, 20 Dec 95 17:34:15 PST From: Terry Sejnowski Message-Id: <9512210134.AA16333@salk.edu> To: connectionists@cs.cmu.edu Subject: Senior Position at GSU Forwarded to Connectionists: Date: Mon, 18 Dec 1995 15:00:23 -0500 (EST) From: Donald Edwards Subject: job Dear friends and colleagues, I am writing to let you know of a senior position in computational neuroscience available here in the Department of Biology at Georgia State University. This person would join neurobiologists, physicists, mathematicians and computer scientists in the newly established Center for Neural Communication and Computation, and would participate in the graduate program in Neurobiology in the Department of Biology. This person would also help guide the construction, equipping and staffing of a Laboratory for Computational Neuroscience for which funds have already been obtained from the George Research Alliance. Georgia State University is located in downtown Atlanta. For more information, please contact me at this address, or call at (404) 651-3148. To apply, please send a letter of intent, c.v., and two letters of reference to Search Committee for Computational Neuroscience, Department of Biology, Georgia State University, Atlanta, GA 30302-4010. FAX: (404) 651-2509. Please share this message with anyone who might be interested. Thanks for your consideration, Don Edwards