
Markov Decision Processes
Martin L. Puterman · 1994
Machine learning, statistics and probability › Bayesian and probabilistic modeling
#11,062 most recommended
Hacker News
3
mentions by 3 people · top 64% of 15,732 books
Open Library
—
No ratings yet
10 want to read
Wikipedia
—
No article yet
About the book
Markov decision processes, a framework for sequential decision-making under uncertainty in which outcomes are partly random and depend on the current state and the action chosen. The book covers the theory, computation and applications of these models, with its main focus on infinite-horizon discrete-time models with discrete state spaces. It also treats arbitrary state spaces, finite-horizon models and continuous-time models with discrete states. The book organises the subject around optimality criteria and presents its results in theorem-proof form, using a common framework built on the optimality (Bellman) equation. Its central aim is a unified and rigorous treatment of theory, algorithms and applications, drawn together from a literature that was then scattered. It illustrates results and algorithms repeatedly with a two-state model, and it covers recent work on average-reward countable-state models, constrained models, risk-sensitive criteria, modified policy iteration and sensitive optimality.
Written by Claude from what it knows of this book.
What Hacker News says 67% recommend · 0% critical
In the commenters’ own words
The theory of multi-armed bandits arises from the study of Markov decision processes (MDP). This field is also closely studied in and related to Stochastic…
You should read more about Markov Decision Processes (MDPs) and Game Theory in general. The word "solve" is well defined in this context, and is used…
What Open Library readers say
No Open Library reader has rated it yet.
- 10 want to read
- 0 reading
- 0 have read
What critics say, via Wikipedia
No Wikipedia article about this book yet, so no critics to quote.
Reddit communities — coming later
Details
- First published
- 1994
- ISBN
- 9780471727828
- Editions known
- 12 at Open Library
- Open Library
- OL3904991W
31.70 probabilityDiskreter Markov-ProzessDynamic programmingDynamische OptimierungDynamische programmeringEntscheidungstheorieLinear ProgrammingMarkov ChainsMarkov processesMarkov-beslissingsproblemenMarkov-processenMarkovscher ProzessPrise de décision (Statistique)Processus de Markov
Read next: its neighbours on the mosaic
Dynamic programming and optimal controlDimitri P. Bertsekas · 1995 · recommended by 2 people · similar by subject and what people say about it
Approximate dynamic programmingWarren B. Powell · 2011 · recommended by 1 person · similar by subject and what people say about it
Controlled Markov processesE. B. Dynkin · 1979 · recommended by 1 person · similar by subject and what people say about it
Prospect theoryPeter P. Wakker · 2010 · recommended by 1 person · similar by subject and what people say about it
Thinking and decidingJonathan Baron · 1988 · recommended by 6 people · similar by subject and what people say about it
Stochastic optimal controlDimitri P. Bertsekas · 1978 · recommended by 0 people · similar by subject and what people say about it
Behavioral Game TheoryColin F. Camerer · 2003 · recommended by 2 people · similar by subject and what people say about it
Adventures in stochastic processesSidney I. Resnick · 1992 · recommended by 1 person · similar by subject and what people say about it
Modern graph theoryBéla Bollobás · 1998 · recommended by 3 people · loosely similar by subject
Stochastic ProcessesRobert G. Gallager · 2013 · recommended by 2 people · similar by subject and what people say about it
Choices, values, and framesDaniel Kahneman · 1985 · recommended by 2 people · similar by subject and what people say about it
Markov LogicDaniel Lowd · 2009 · recommended by 1 person · similar by subject and what people say about it
Statistical decision theory and Bayesian analysisJames O. Berger · 1985 · recommended by 2 people · similar by subject and what people say about it
Micromotives and MacrobehaviorThomas C. Schelling · 1978 · recommended by 3 people · similar by subject and what people say about it
Probability and ComputingMichael Mitzenmacher · 2005 · recommended by 4 people · similar by subject and what people say about it
Machine learningKevin P. Murphy · 2012 · recommended by 15 people · similar by subject and what people say about it
Multiagent systemsYoav Shoham · 2008 · recommended by 1 person · similar by subject and what people say about it
Behavioral mathematics for game AIDave Mark · 2009 · recommended by 2 people · similar by subject and what people say about it
The art and theory of dynamic programmingStuart E. Dreyfus · 1977 · recommended by 1 person · similar by subject and what people say about it
Statistical RethinkingRichard McElreath · 2015 · recommended by 81 people · similar by subject and what people say about it
Probabilistic Graphical ModelsDaphne Koller · 2009 · recommended by 16 people · similar by subject and what people say about it
The psychology of judgment and decision makingScott Plous · 1993 · recommended by 4 people · similar by subject and what people say about it
Practical Probabilistic ProgrammingAvi Pfeffer · 2016 · recommended by 4 people · loosely similar by subject
Thinking StrategicallyAvinash K. Dixit · 1991 · recommended by 9 people · loosely similar by subject