
Controlled Markov processes
E. B. Dynkin · 1979
Machine learning, statistics and probability › Bayesian and probabilistic modeling
#6,649 most recommended
Hacker News
10
mentions by 4 people · top 29% of 15,732 books
Open Library
—
No ratings yet
Wikipedia
—
No article yet
About the book
The subject is the optimal control of Markov processes, in which a decision-maker picks actions that influence the transition law of a random process and aims to optimise an expected reward or cost. The scope covers discrete-time Markov decision models and controlled processes on general measurable state spaces, together with the probability and measure-theoretic foundations they need. The book is co-authored with A. A. Yushkevich. The book treats control as a problem over strategies and builds the theory around the dynamic programming principle and the functional equations it yields. Its central claim is that, under suitable conditions, the value of a control problem satisfies such equations and that optimal or near-optimal policies can be found among simple Markov policies. It examines criteria such as total, discounted and average reward, and sets out the regularity conditions (semicontinuity, compactness of action sets, measurable selection) needed to justify these results.
Written by Claude from what it knows of this book.
What Hacker News says 25% recommend · 0% critical
In the commenters’ own words
Project planning and estimating is a big field. In a nutshell, the key to making such an effort very effective is just really good knowledge of and experience…
There is what 'dynamic programming' is, and there is the 'word segmentation' problem of this thread. The problem is interesting, and so are algorithms to…
What Open Library readers say
No Open Library reader has rated it yet.
What critics say, via Wikipedia
No Wikipedia article about this book yet, so no critics to quote.
Reddit communities — coming later
Details
- First published
- 1979
- ISBN
- 9780387903873
- Editions known
- 2 at Open Library
- Open Library
- OL16348307W
Control theoryMarkov processesMathematical optimization
Read next: its neighbours on the mosaic
Deterministic and Stochastic Optimal ControlWendell H. Fleming · 1975 · recommended by 1 person · named together in 7 comments
Foundations of optimal control theoryE. B. Lee · 1967 · recommended by 1 person · named together in 2 comments
The art and theory of dynamic programmingStuart E. Dreyfus · 1977 · recommended by 1 person · named together in 9 comments
Brownian motion and stochastic calculusIoannis Karatzas · 1988 · recommended by 3 people · similar by subject and what people say about it
Stochastic optimal controlDimitri P. Bertsekas · 1978 · recommended by 0 people · similar by subject and what people say about it
Statistics of random processesRobert Shevilevich Lipt︠s︡er · 1977 · recommended by 1 person · similar by subject and what people say about it
Introduction to Stochastic ProcessesErhan Cinlar · 2013 · recommended by 4 people · similar by subject and what people say about it
Introduction to stochastic processesE. Çınlar · 1974 · recommended by 4 people · similar by subject and what people say about it
Time Series AnalysisGeorge E. P. Box · 1970 · recommended by 2 people · loosely similar by subject
Optimization by Vector Space MethodsDavid G. Luenberger · 1968 · recommended by 7 people · loosely similar by subject
Convergence of Probability MeasuresPatrick Billingsley · 1968 · recommended by 2 people · loosely similar by subject
Linear algebra and matrix theoryEvar D. Nering · 1963 · recommended by 1 person · loosely similar by subject
ProbabilityLeo Breiman · 1968 · recommended by 5 people · loosely similar by subject
Dynamic programming and optimal controlDimitri P. Bertsekas · 1995 · recommended by 2 people · similar by subject and what people say about it
A Course in Probability TheoryKai Lai Chung · 1968 · recommended by 3 people · loosely similar by subject
Mathematical foundations of the calculus of probabilityJ. Neveu · 1965 · recommended by 3 people · loosely similar by subject
Mathematical Theory of Optimal ProcessesPontryagin Ls · 1962 · recommended by 0 people · similar by subject and what people say about it
Markov Decision ProcessesMartin L. Puterman · 1994 · recommended by 2 people · similar by subject and what people say about it
Integer and combinatorial optimizationGeorge L. Nemhauser · 1988 · recommended by 2 people · loosely similar by subject
Computer solution of linear algebraic systemsGeorge E. Forsythe · 1967 · recommended by 2 people · loosely similar by subject
Linear statistical inference and its applicationsRao, C. Radhakrishna · 1965 · recommended by 1 person · loosely similar by subject
Optimal Control TheoryDonald E. Kirk · 1970 · recommended by 1 person · similar by subject and what people say about it
Stochastic ProcessesJoseph L. Doob · 1953 · recommended by 1 person · similar by subject and what people say about it
Counterexamples in analysisBernard R. Gelbaum · 1964 · recommended by 11 people · loosely similar by subject