
Information Theory, Inference & Learning Algorithms
David J.C. MacKay · 2003
Machine learning, statistics and probability › Probability, causality and information theory
#404 most recommended
Hacker News
111
mentions by 93 people · top 3% of 15,732 books
Open Library
4.00★★★★★★★★★★
1 rating · 21 want to read
Wikipedia
—
No article yet
About the book
Shannon's information theory, covering entropy, source coding and the noisy-channel coding theorem, alongside the probabilistic inference methods that connect it to learning. It spans practical systems such as arithmetic coding for compression and sparse-graph codes for error correction, and applies the same tools to clustering, independent component analysis and neural networks. MacKay treats compression, error correction and learning as one problem: representing and reasoning about uncertainty with probability, so that a good code and a good model are two views of the same thing. Its central claims are that Shannon's limits can be approached in practice by sparse-graph codes decoded with message-passing algorithms, and that Bayesian inference, carried out with Monte Carlo methods and variational approximations, gives a single toolkit for both communication and machine learning. The book also argues that information theory and inference should be taught together rather than as separate subjects.
Written by Claude from what it knows of this book.
What Hacker News says 83% recommend · 0% critical
In the commenters’ own words
I came to some of these conclusions in 1997: http://www.canonical.org/~kragen/real-news.html In 2008 I savaged Digg and Reddit for the same reasons…
Go download David MacKay's Information Theory, Inference and Learning Algorithms (free book). Go through the part on Bayes and the part on Neural Nets (and…
What Open Library readers say 4.00 from 1 rating
- 50
- 41
- 30
- 20
- 10
- 21 want to read
- 2 reading
- 1 have read
Only 1 rating: treat the average as a hint.
What critics say, via Wikipedia
No Wikipedia article about this book yet, so no critics to quote.
Reddit communities — coming later
The threads it came up in
- Ask HN: What books changed the way you think about almost everything?
- Ask HN: Which book can attract anyone towards your field of study?
- Ask HN: Math books that made you significantly better at math?
- Ask HN: What books have you read in the first half of 2016?
- Ask HN: The book that did it for you in math and/or CS?
- Ask HN: Serious mathematics books that can replace a good teacher?
- Reading list for those who love to learn
- Ask HN: Math books like SICP?
Details
- First published
- 2003
- ISBN
- 9780521644440
- Editions known
- 4 at Open Library
- Open Library
- OL8325677W
003/.54AlgorithmsAlgoritmenAprendizado computacionalBayesianDat 708fInferenceInferenzInferenz (künstliche intelligenz)Inferenz <künstliche intelligenz>InformatietheorieInformation theoryInformation, The orie de l'Information, Theorie de l'
Read next: its neighbours on the mosaic
Bayesian reasoning and machine learningDavid Barber · 2011 · recommended by 15 people · 4 people recommended both
Elements of Information TheoryT. M. Cover · 2006 · recommended by 20 people · 6 people recommended both
The Elements of Statistical LearningTrevor Hastie · 2001 · recommended by 144 people · 11 people recommended both
Pattern Recognition and Machine LearningChristopher M. Bishop · 2008 · recommended by 80 people · 7 people recommended both
All of StatisticsLarry Wasserman · 2004 · recommended by 40 people · 6 people recommended both
Probability TheoryE. T. Jaynes · 2003 · recommended by 77 people · 6 people recommended both
Probabilistic Graphical ModelsDaphne Koller · 2009 · recommended by 16 people · 3 people recommended both
Introduction to Statistical LearningGareth James · 2021 · recommended by 85 people · 5 people recommended both
Pattern classificationRichard O. Duda · 2001 · recommended by 8 people · named together in 3 comments
Learning From DataYaser S. Abu-Mostafa · 2012 · recommended by 31 people · 3 people recommended both
Convex optimizationStephen P. Boyd · 2006 · recommended by 40 people · 3 people recommended both
Bayesian data analysisAndrew Gelman · 1995 · recommended by 27 people · named together in 3 comments
Bayesian methods for hackersCameron Davidson-Pilon · 2015 · recommended by 24 people · 2 people recommended both
Algorithm designJon Kleinberg · 1922 · recommended by 30 people · 2 people recommended both
Introduction to AlgorithmsThomas H. Cormen · 1990 · recommended by 274 people · 6 people recommended both
Deep LearningIan Goodfellow · 2016 · recommended by 44 people · similar by subject and what people say about it
The Art of ProbabilityRichard Hamming · 1990 · recommended by 7 people · similar by subject and what people say about it
Visual complex analysisTristan Needham · 1997 · recommended by 38 people · 2 people recommended both
The Master AlgorithmPedro Domingos · 2015 · recommended by 23 people · 2 people recommended both
Game TheoryDrew Fudenberg · 1991 · recommended by 3 people · similar by subject and what people say about it
Statistical RethinkingRichard McElreath · 2015 · recommended by 81 people · 3 people recommended both
Quantum Computing Since DemocritusScott Aaronson · 2013 · recommended by 73 people · 3 people recommended both
Computer Age Statistical InferenceBradley Efron · 2016 · recommended by 14 people · similar by subject and what people say about it
Machine learningKevin P. Murphy · 2012 · recommended by 15 people · similar by subject and what people say about it