
Elements of Information Theory
T. M. Cover, Joy A. Thomas · 2006
Machine learning, statistics and probability › Probability, causality and information theory
#1,524 most recommended
Hacker News
34
mentions by 28 people · top 10% of 15,732 books
Open Library
5.00★★★★★★★★★★
1 rating · 10 want to read
Wikipedia
—
No article yet
About the book
Elements of Information Theory is a graduate-level textbook on the mathematical theory of information, built on probability and statistics. It covers entropy, relative entropy and mutual information, then applies them to data compression, communication over noisy channels, rate distortion, and networks of sources and channels. The second edition (2006) also treats Kolmogorov complexity, gambling and portfolio theory, and the Kelly criterion. Its central claim is that a few information measures, chiefly entropy and mutual information, capture the fundamental limits of compressing data and transmitting it reliably. The book develops these quantities from basic definitions and proves coding theorems showing that the limits can be approached or reached. It also presents information as a quantity with operational meaning beyond communication, including its role in statistical inference and in the growth rate of investment wealth.
Written by Claude from what it knows of this book.
What Hacker News says 71% recommend · 0% critical
In the commenters’ own words
OK, here ya go. This list is, for now, just a raw dump of titles and authors. When I have some more time, I'll turn this into a proper blog post with more…
Mike Jordan at Berkeley sent me his list on what people should learn for ML. The list is definitely on the more rigorous side (ie aimed at more researchers…
What Open Library readers say 5.00 from 1 rating
- 51
- 40
- 30
- 20
- 10
- 10 want to read
- 0 reading
- 2 have read
Only 1 rating: treat the average as a hint.
What critics say, via Wikipedia
No Wikipedia article about this book yet, so no critics to quote.
Reddit communities — coming later
The threads it came up in
Details
- First published
- 2006
- ISBN
- 9780471241959
- Editions known
- 2 at Open Library
- Open Library
- OL11674990W
Information theory
Read next: its neighbours on the mosaic
Information Theory, Inference & Learning AlgorithmsDavid J.C. MacKay · 2003 · recommended by 77 people · 6 people recommended both
An introduction to Kolmogorov complexity and its applicationsMing Li · 1993 · recommended by 5 people · named together in 3 comments
CausalityJudea Pearl · 2000 · recommended by 17 people · similar by subject and what people say about it
Convex optimizationStephen P. Boyd · 2006 · recommended by 40 people · 2 people recommended both
Kelly Capital Growth Investment CriterionLeonard C. MacLean · 2010 · recommended by 3 people · similar by subject and what people say about it
Game TheoryDrew Fudenberg · 1991 · recommended by 3 people · similar by subject and what people say about it
A user's guide to measure theoretic probabilityPollard, David · 2002 · recommended by 2 people · loosely similar by subject
The Elements of Statistical LearningTrevor Hastie · 2001 · recommended by 144 people · 2 people recommended both
Introduction to the theory of computationMichael Sipser · 1997 · recommended by 66 people · similar by subject and what people say about it
Pattern Recognition and Machine LearningChristopher M. Bishop · 2008 · recommended by 80 people · 2 people recommended both
Probability TheoryE. T. Jaynes · 2003 · recommended by 77 people · named together in 2 comments
The mathematical theory of communicationClaude Elwood Shannon · 1949 · recommended by 47 people · similar by subject and what people say about it
Introduction to Statistical LearningGareth James · 2021 · recommended by 85 people · 2 people recommended both
The InformationJames Gleick · 2011 · recommended by 61 people · similar by subject and what people say about it
Fortune's FormulaWilliam Poundstone · 2005 · recommended by 32 people · similar by subject and what people say about it
Information rulesCarl Shapiro · 1999 · recommended by 18 people · similar by subject and what people say about it
An introduction to information theoryJohn Robinson Pierce · 1980 · recommended by 15 people · similar by subject and what people say about it
Symbols, signals, and noiseJohn R. Pierce · 1961 · recommended by 12 people · similar by subject and what people say about it
Introduction to probability theoryPaul Gerhard Hoel · 1971 · recommended by 7 people · similar by subject and what people say about it
ChaosJames Gleick · 1987 · recommended by 45 people · similar by subject and what people say about it
The nature of computationCristopher Moore · 2009 · recommended by 10 people · similar by subject and what people say about it
An introduction to probability theory and its applicationsWilliam Feller · 1950 · recommended by 8 people · similar by subject and what people say about it
Information TheoryJames V Stone · 2015 · recommended by 2 people · similar by subject and what people say about it
Signals and SystemsAlan V. Oppenheim · 1983 · recommended by 12 people · similar by subject and what people say about it