Due: Friday, January 30, 2004. Please put your
assignment in my mailbox in the departmental office. I believe the office
closes at .
Paper and Pencil: Exercise 3.2, 3.4,
4.7. Bonus question: 4.9. Note: 4.9 is a “thinking question”, which
can be treated at various levels of precision, from some relevant informal
observations to a full-blown formal proof. I will grade it as a bonus question.
I’d like you to give some thought to it because it’s quite
instructive about neural networks, but if you find it difficult to get your
head around it, don’t sink too much time into it (e.g. > 2 hours).
The purpose of this assignment is for you to get a sense of various standard
machine learning technology, by looking at some
implementations of the algorithms rather than a general definition. To pass,
I'd like to see some evidence that you've managed to run an algorithm and that
you had some understanding of the output you saw. The evidence should be in the
form of a print-out of screen displays or program outputs, and usually a brief
discussion of the output (things you noticed or that strike you.) Specifically,
this week I'd like you to run an implementation of one or more neural
nets. What follows are some specific instructions and suggestions for
experimentation. Feel free to explore and share with us if you find something
Check Out the Web
Enter in google,
"neural net software", "neural net companies". Take a
look at what comes up, check out a few links. You'll notice that there are
quite a few companies selling some kind of neural net software - perhaps a
future employer? http://www.emsl.pnl.
Neural Net Software
There are a number of
companies that offer evaluation copies of their neural net software. One
that I've tried is NeuroDimensions.
You can get an evaluation copy of their software and run their demos (or
more runs if you like). They have something for the PC - I'm not sure about
A number of research groups
offer Java demos. One nice site is http://neuron.eng.wayne.edu/software.html
which has a number of neural net architectures for various problems. The function
approximation is nice to explore. Which functions seem to be easy for the
network, and which are hard (e.g., compare linear with sin). Let the
network train a number of times, and see what happens - does the error
always go down with training? How quickly? - A downside to this site is
that they don't seem to offer ways of inspecting the network, but some of
the documentation has nice pictures.
Aaron Hunter recommends two
applets for handwriting recognition by neural nets, namely
http://members.aol.com/Trane64/java/JRec.html, which allows you to
experiment with your own handwriting (rather, "mouse writing"),
and http://www.geocities.com/SiliconValley/2548/ochre.html. Julia Birke has tried out
http://www-dsi.ing.unifi.it/neural/software.html which has a number of
machine learning software, including neural nets, and
www.geocities.com/CapeCanaveral/1624/ for a number of illustrative tasks
solved by neural nets. Kimberley Voll worked
with EasyNN, a fairly small-scale commercial package.
The text book has neural
net code (in C, to be run on a Unix system)
and face recognition training data. Instructions and questions are posted
as an assignment. If you use this, YOU DO NOT HAVE TO WORK THROUGH ALL THE
QUESTIONS. Just get it to run on the input files; basically, you would
look at Part I, Questions 1-3 and 5-6. As always, you are of course
welcome to play around. Some familiarity with this C code might be useful
if you are thinking of doing your own programming with neural nets.
GaborMelli has used the NNinExcel tool which allows you to run a neural net
with in an Excel spreadsheet. As a neural net package, it’s a bit
slow, but it has a nice user interface and you can use the Excel graphing
tools to visualize your findings.
There are other network
simulators and packages on the web. GlendonHolst recommends Lutz Prechelt's website which has frequently asked
questions, pointers to literature and an annotated list of free software.