DeepRead
Cover of Information Theory, Inference & Learning Algorithms

Information Theory, Inference & Learning Algorithms

David J.C. MacKay · 2003

Machine learning, statistics and probability › Probability, causality and information theory

#404 most recommended

About the book

Shannon's information theory, covering entropy, source coding and the noisy-channel coding theorem, alongside the probabilistic inference methods that connect it to learning. It spans practical systems such as arithmetic coding for compression and sparse-graph codes for error correction, and applies the same tools to clustering, independent component analysis and neural networks. MacKay treats compression, error correction and learning as one problem: representing and reasoning about uncertainty with probability, so that a good code and a good model are two views of the same thing. Its central claims are that Shannon's limits can be approached in practice by sparse-graph codes decoded with message-passing algorithms, and that Bayesian inference, carried out with Monte Carlo methods and variational approximations, gives a single toolkit for both communication and machine learning. The book also argues that information theory and inference should be taught together rather than as separate subjects.

Written by Claude from what it knows of this book.

What Hacker News says 83% recommend · 0% critical

01020200720102015202020262016 peak · 15
This bookAverage book in Probability, causality and information theory

In the commenters’ own words

What Open Library readers say 4.00 from 1 rating

  1. 50
  2. 41
  3. 30
  4. 20
  5. 10
  • 21 want to read
  • 2 reading
  • 1 have read

Only 1 rating: treat the average as a hint.

Open Library’s page for the book

What critics say, via Wikipedia

No Wikipedia article about this book yet, so no critics to quote.

Reddit communities — coming later

The threads it came up in

Details

First published
2003
ISBN
9780521644440
Editions known
4 at Open Library
Open Library
OL8325677W

003/.54AlgorithmsAlgoritmenAprendizado computacionalBayesianDat 708fInferenceInferenzInferenz (künstliche intelligenz)Inferenz <künstliche intelligenz>InformatietheorieInformation theoryInformation, The orie de l'Information, Theorie de l'

Explore in the mosaic

Read next: its neighbours on the mosaic