(Last Update: September 2010) I have worked on a number of topics in different areas: computational learning theory, computation theory, inductive inference, philosophy of science, belief revision, game theory: foundations, algorithms, evolutionary analysis, and machine learning for particle physics. The aim of this note is a high-level, informal introduction to my work that does not assume background in computer science. For detailed descriptions, you can look at the annotations on my publications. (See also the thesis topics I've listed for graduate students if they're up on the web).

Biography

I was born in Toronto. My family moved to Germany when I was six. After finishing high school there, I came back to the University of Toronto to do a Bachelor's Degree in Cognitive Science. The Cognitive Science program in Toronto combines computer science, philosophy and psychology. For my Ph.D., I went to the philosophy department at Carnegie Mellon University, in Pittsburgh, Pennsylvania. Carnegie Mellon is a private university founded by Andrew Carnegie.

Research Interests

My work is mostly concerned with an issue that receives different formulations and labels in different areas: how to use observations to adapt to an external world. In computer science, we talk about machine learning and optimal design for learning systems. In philosophy of science, this leads to questions about scientific reasoning and method. Epistemology considers how to form beliefs on the basis of observations, especially inductive generalizations. In biology, the topic is adaptation. Mostly I've worked in the context of machine learning, scientific reasoning and epistemology, which aim to find optimal ways of learning and adapting, rather than to describe how humans or animals actually do learn. My guiding principle is that good reasoning methods or algorithms are those that lead us towards our cognitive aims, especially towards theories that get the observations right. Though not everybody agrees with this, it's hardly a new idea. What's new in my work is a systematic attempt to work out the details. For example, what are relevant cognitive goals? What are the most powerful reasoning methods like? How hard can it be to attain some goal? Are some learning aims more difficult to realize than others? Which empirical questions are easy, which are hard, which are impossible, and what makes them so? Together with several other people working on these questions, we have found some precise, systematic and often surprising answers.

In considering the virtues and vices of learning methods, I often resort to general principles of rational choice. The idea is to think of an inference-method as something that you can choose, and you can apply rational choice principles to help you figure out how to do so. After applying ideas from the analysis of rational choice, I've become interested in the foundations of that theory. I've also worked on methods for representing rational choice models in logic and using methods from computational logic to carry out game theoretic reasoning.

Applications and Projects

I believe it was Goethe who said that "nothing is as practical as a good theory". That's my motto too (in Germany you are raised to believe everything that Goethe said). So if my theories about learning and inductive inference and empirical inquiry are on the right track, they ought to have concrete applications.

I have worked a lot on one such application: a computer program that postulates conservation principles and hidden particles for elementary particles. My analysis says that the program is in principle optimal for this problem. We have implemented the program and run it on the current data from particle physics. The program matches exactly the conservation laws that physicists have found, and it rediscovers the need for an unobserved neutrino, the electron anti-neutrino, which is one of the main concerns in current particle research.
Another learning problem that I have worked on, together with my student Wei Luo, is the learning-theoretically optimal way to infer Bayes nets (aka causal graphs) from information about statistical correlations between a potentially large set of variables. You can think of Bayes nets as a compact graphical way to represent a probability distribution over a set of variables that makes explicit which variables directly depend on which others. The learning-theoretic analysis has led to a new method for learning Bayes nets. The way Bayes net researchers would describe it is that it is a hybrid method that combines correlation testing with model selection. Generally our method finds Bayes nets that are closer to the true causal model and make more accurate probabilistic predictions. However, it takes more computation time than previous methods.
Kevin Kelly has recently made a big advance in the theory of mind-change optimal inductive inference by showing how it can be applied to statistical data and model selection directly, without first extracting correlation information (see his UAI conference paper). He shows that with model selection scores, mind changes happen as follows. Consider two causal variables X and Y, e.g. X = watching video games and Yaggression. It is possible to have a causal scenario where X causes Y, the statistical model selection method when given small samples correctly infers the causal direction, when given medium-sized samples it incorrectly flips the causal direction from Y to X , and finally with large samples it changes its mind again to settle on the correct causal direction.
I've recently become interested in combining logic and probability in order to get the best of both worlds and use both for capturing relevant knowledge. I'm thinking about ways to use Bayes nets to represent and find probabilistic dependencies among attributes in a relational database. A related topic is assigning probabilities to assertions and rules that are expressed in a logically rich language (e.g., "people with high income that have neighbours whose children go to expensive schools tend to send their children to expensive schools too").