Thoughts on Bayesian Inference and Reasoning

Dynamics and Transport were both mechanical engineering major requirements. I also took two classes that fulfilled general graduation requirements. The first of these was Bayesian Inference and Reasoning, which counted as my probability and statistics course.

Bayesian was taught by Sanjoy, who was my professor for Waves last spring. I took this class because it fulfilled the ProbStat requirement, and I've covered traditional (or frequentist) probability and statistics a couple of times before. Bayesian inference is a bit different, and some friends from Mathcamp had told me that it seemed like a much more reasonable way to do statistics, so I was excited for the class.

All Sanjoy classes are somewhat alike. Homework is generally minimal, there's an in-class final exam, and everything is graded on effort. The subject matter covered in Bayesian was a lot narrower than that covered in Waves, and I think that was why I enjoyed Bayesian less. It often seemed to go slowly. Almost all of the class focused on solving probability and statistics problems with Bayes' Law; the rest was just details. The basic process of solving problems with Bayesian reasoning involves listing all the possible hypotheses related to some event, giving each hypothesis an initial probability, and then revising those probabilities based on data. We spent half the class on problems with a finite number of hypotheses, and then in what was left of the class we covered infinite hypotheses, comparing how effective two medical treatments are, and a little bit of frequentist statistics. Sanjoy often came up with interesting problems for us to do, but then we would do the same types of problems over and over. I learned in the class, but it seemed like I could have learned the material just as well with a quarter of the class time.

For homework, Sanjoy had us read The Theory That Would Not Die by Sharon Bertsch McGrayne, which is about the history of Bayes' Law. We didn't actually get very far in the book, and we never talked about it in class; we just had to read it and comment on it on a class forum. I didn't find what we did read to be particularly enlightening. The book was very light on technical details, a bit jumbled, and very pro-Bayesian while framing the history in a Bayesian vs. frequentist way. I liked seeing how various people reinvented Bayesian reasoning through history and how they used it, but I don't think I'll finish the book. I'm also not entirely sure why we read the book at all, since the reading was separate from everything else in the course.

I put an average of half an hour of work a week into Bayesian outside of class except for the last two or so weeks of the semester. In the last week and a half, though, there was a homework assignment, a reading assignment, a final project, and the final exam. While that still only came to about eight hours outside of class, that was more than I had put in during the rest of the semester all together. The work was definitely not well-distributed through the semester.

The final project was one of my favorite parts of the class. We all split up into groups and wrote programs that played Mastermind. My partner was Eleanor, who lives in my hallway. We wrote most of our program in one night after spending a few hours in front of a whiteboard talking about different strategies. Our strategy was based on information theory, an area of math that uses Bayesian inference, but our approach didn't really rely on probability. With six possible peg colors and codes that were four pegs long, our program usually needed 4 guesses to correctly guess the Mastermind code. (When all the programs "competed" in class, the programs that did use probability needed more guesses, but they ran faster and would have been able to deal with far longer codes and more possible colors of pegs.)

Overall, I thought the problems in Bayesian were interesting, but the class went too slowly and felt too disjointed. It was a better option for me than taking a more traditional probstat class, though, so I don't regret taking Bayesian.