Bayesian textbook list
Exactly what the title says.
Without coming on too strong as a zealot, let me say that Bayesian theory is both highly practical and has deep theoretical foundations. Regardless of where you end up landing personally on your data science philosophy, familiarizing yourself earnestly with the scholarly work surrounding Bayesian ideas is, IMO, certainly worthwhile.
To that end, here are some of the main books that influenced my own thinking. The order is more-or-less arbitrary. (After some feedback I include a very few that I’ve not yet read — I’ll indicate where this is the case.)
David Blackwell, "Basic Statistics".
An absolute gem of a book, teaching intro stats the Bayesian way.
Jim Berger, "Statistical Decision Theory and Bayesian Analysis".
Ironically, one of the most famous Bayesian is actually a frequentist in some sense (at least I concluded so the more conversations I had with him).
3. Mark Schervish, "Theory of Statistics".
This book gives equal coverage to the two dominant paradigms and in so doing, gives readers a great basis for deciding for themselves what Bayes is all about.
4. John Hartigan, "Bayes Theory"
A bit more mathematical in places, but the introduction alone is worth the price of admission. (I love it so much I'll probably dedicate a whole post to it in the near future.)
5. Peter Hoff, "A First Course in Bayesian Statistical Methods"
I've used this text in a course I teach for many years. Highly practical and concise.
6. Christian Robert, "The Bayesian Choice"
Great coverage of the foundational ideas in a relatively accessible style.
7. Bernardo and Smith, "Bayesian Theory”
This was "the Bible" during my Duke Statistics grad school days.
8. Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference.
A go-to reference for modern Bayesian computation.
9. West and Harrison, "Bayesian Forecasting and Dynamic Linear Models".
This book is so good that the Appendices alone are fantastic.
10. Rossi, Allenby, and McCulloch, "Bayesian Statistics and Marketing”.
A nice introduction to applied Bayesian econometrics.
11. Gelman, et al. "Bayesian Data Analysis"
Many people's first and last book on Bayesian inference, but not my personal favorite.
12. Richard T. Cox, "The Algebra of Probable Inference".
And now we start to get into the hardcore subjectivist stuff.
13. E.T. Jaynes, "Probability Theory: The Logic of Science".
Brilliant and polemical. Don't start here unless you want to start a fight.
14. Richard Jeffrey, "Subjective Probability: The Real Thing".
A philosophy book with insights for anyone wanting to learn from data under uncertainty.
15. Leonard "Jimmie" Savage, "The Foundations of Statistics".
Proof that some very smart people thought a Bayesian approach made a lot of sense. David Blackwell has explained that it was Savage that converted him :-)
16. I.J. Good, “Good Thinking: The Foundations of Probability and Its Applications”.
In which the author famously enumerates all the different flavors of Bayesian that he recognizes as possible.
17. Box and Tiao, "Bayesian Inference in Statistical Analysis".
18. Edward Leamer, "Specification Searches: Ad Hoc Inference with Nonexperimental Data".
One of my personal favorites, from the author of the famous “Taking the Con out of Econometrics.”
19. Raiffa and Schlaifer, "Applied Statistical Decision Theory".
In which conjugate models are developed thoroughly because Harvard B-school students couldn’t be bothered with calculus.
20. Morris DeGroot, “Optimal Statistical Decision”.
I used this one heavily studying for my prelim exam (both times lol).
Bruno de Finetti, “Theory of Probability”.
I mostly learned my de Finetti from second-hand expositions, but going to the source is never a bad idea.
Anthony O'Hagan, “Bayesian Inference”.
Another one I kept on my desk during graduate school.
Richard McElreath, “Statistical Rethinking: A Bayesian Course with Examples in R and STAN”.
This one is added to the list by popular demand and is one I need to be sure to read myself now that I know about it. Definitely a crowd favorite.