DarienHacker Wrote: ------------------------------------------------------- > I’m saying that both neural nets and genetic > algorithms are cast as alternatives to more > traditional, if less flashy, techniques. Their > performance and behavior are often poorer or not > understood at all. In the neural net world I > can’t remember the last time I saw an application > that wasn’t better served by a straightforward > statistical analysis. I’d say similar for genetic > algorithms. Well, I would enjoy convincing you otherwise. Two examples from my life for these: 1) I had a student (http://research.microsoft.com/~aselag/) who did his senior thesis on controlling a prosthetic hand using electrodes placed on the wrist (move your fingers while holding your wrist with the other hand - there’s lots of info in the wrist about what you are trying to do with your fingers). I did the stats side with him and an EE prof did the neural nets side. It was astounding how each picked up different kinds of signals and the combination produced some great control info. My introduction to neural nets being a useful tool in life. This was the best piece of undergraduate work I ever saw. Asela has a formidable brain. 2) The NRMP has a couples matching problem that is NP-hard with gajillions of candidates. I wrote a genetic algorithm that solves the whole problem in minutes (as you say, smart thinking makes the problem lots easier than doing a straight genetic algorithm), 10 years ago. No idea what’s become of that now.
NN and GA have their useful applications. Strictly speaking, NN is not even an optimization technique: it is more akin to an advanced regression, black-box predictive technique. You input a set of learning data, and you get your link weights and hidden nodes etc, and then you use it like you would a regression model. NN as a comp science product, having lived gloriously for a decade or so in the 1980s, is in sun-set mode simply because it takes so long to “learn”. NN has found a second life of sort in statistical learning, but enthusiasm remains luke-warm. See the excellent stats learning text by Friedman, Hastie and Tibrashani(?) for nn as stat learning tool. GA, on the other hand, is an optimization technique, albeit an unique one. GA is, succinctly, a stochastic heuristic search algorithm. Stochastic necessarily implies that the “optimal” solution may not be unique on each run. They may not even be the global optimum, and there is no way to know if you are near or at it. Heuristic because it follows a set of well-defined rules when doing the search. Despite its “heuristic” nature, GA works wonderfully on large-scale, data intensive or NP-hard problems that traditional technique have no hope in. And it is relatively fast. The bugbears are of course, the difficulty in setting optimal crossover and mutation parameters, and the inability to handle constraints. I once did an interesting project that marries the strength of GA with constraint programming to solve american/bermudan range accrual options in the binomial/trinomial grid setting.
I’ve hear about neural networks for a long time and am casually familiar with them. Am I right in thinking that they are basically “black boxes” that can (or might be able to) learn or be trained to generate a set of outputs given a training set of inputs, but they can’t really be “interpreted” the way a set of regression coefficients can (usually) be interpreted? Genetic algorithms have also interested me from time to time, but here I always run into the question of how does one design the algorithm’s “chromosomes”. For example, if you are trying to find an optimal equation for modeling something, and use a genetic algorithm to search for zillions of specifications, how do you write the permitted combinations of factors and specifications in ways that don’t artificially limit your possible results?
I think you are spot-on with the “black-box” definition about nn. it is a blacbox not in the computational sense (the algorithm is well-defined, or the computer cannot execute nn) but in the sense that the end results are not amenable to meaningful interpretations like a std regression model is. I took a course from Friedman on statistical learning (he is still teaching it) and there’s useful note on the math of neural network in the statistics sense. http://www.stanford.edu/class/stats315b/handouts/neural.doc Also, designing a good chromosome representation is perhaps the most important aspect of using GA. There’s no hard and fast rule, and it is up to your creativity and imagination.
geez you guys are nerdy. j/k
That’s the trick of course. You need to think of ways that the problem can be characterized that would allow partial good solutions to make complete solutions. A really nice place to start on that is just to check out genetic solutions to traveling salesmen problems.
MFE Wrote: ------------------------------------------------------- > geez you guys are nerdy. > > > > > > > > > j/k I do nerdy fine.
I have little to no clue about what you guys are discussing, but all this stuff sounds so intriguing. Hats off to Asela. He definitely has some impressive research. Makes me glad to see another Sri Lankan being mentioned on this forum; that too by the Great Joey D.