Marty Puterman's Avatar

Marty Puterman

@martypute

Researcher on MDPs and RL. Retired prof. #orms #rl

279
Followers
373
Following
34
Posts
23.11.2024
Joined
Posts Following

Latest posts by Marty Puterman @martypute

Behold, the beautiful result of downloading NeurIPS 2023 code and running it on MNIST!

27.02.2026 05:00 πŸ‘ 7 πŸ” 1 πŸ’¬ 2 πŸ“Œ 0

Hillary Clinton: β€œYou have compelled me to testify, fully aware that I have no knowledge that would assist your investigation, in order to distract attention from President Trump’s actions and to cover them up despite legitimate calls for answers”

26.02.2026 20:19 πŸ‘ 2294 πŸ” 639 πŸ’¬ 48 πŸ“Œ 22
Preview
GitHub - martyput/MDP_book Contribute to martyput/MDP_book development by creating an account on GitHub.

Just about finished copyediting "Markov Decision Processes and Reinforcement Learning" by Puterman and Chan. Expect it out in June. All chapters of the pre-copyedited version and more are still available at github.com/martyput/MDP.... #RL #OR

24.02.2026 18:10 πŸ‘ 1 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0

Well done Alan and David.

23.02.2026 17:17 πŸ‘ 0 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0

Happy to just have deleted my twitter/x account

16.02.2026 03:39 πŸ‘ 6 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0

But its helpful to students and researchers if you share them.

02.02.2026 01:39 πŸ‘ 0 πŸ” 0 πŸ’¬ 0 πŸ“Œ 1

It takes alot of effort to turn them into a text. I speak from experience.

01.02.2026 23:43 πŸ‘ 2 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0

The bee sting and great expectations

14.01.2026 01:13 πŸ‘ 2 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0
Preview
GitHub - martyput/MDP_book Contribute to martyput/MDP_book development by creating an account on GitHub.

Preprints of ALL chapters of "Markov Decision Processes and Reinforcement Learning" by Puterman and Chan are now posted at github.com/martyput/MDP....

All feedback and typos are greatly appreciated and will impact the final version. Links will be removed when the book is published. #AI #orms

30.09.2025 17:40 πŸ‘ 7 πŸ” 2 πŸ’¬ 0 πŸ“Œ 0
Post image

Chapters 1-7 of our forthcoming book β€œMarkov Decision Processes and Reinforcement Learning” by Puterman and Chan can be dowloaded at:
github.com/martyput/MDP....
They cover model foundations, and MDPs. Please share comments and/or typos ASAP.
#orms #AI #ReinforcementLearning

25.09.2025 17:06 πŸ‘ 5 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0

An important point about this site is that if you have deep MDP questions, instead of asking an influencer on twitter, you can ask @martypute.bsky.social which is uh, a serious step up

22.09.2025 04:09 πŸ‘ 15 πŸ” 1 πŸ’¬ 1 πŸ“Œ 1

How to reach them?

22.09.2025 04:04 πŸ‘ 2 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0

Will post details and link to preprint on bluesky asap. I want to get off twitter too.

17.09.2025 03:36 πŸ‘ 1 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0

I hope our forthcoming MDP/RL text fills that gap.

16.09.2025 22:10 πŸ‘ 1 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Eh Canada - Rick  Scott
Eh Canada - Rick Scott YouTube video by RickScottKids

Rick Scott; We'll miss your amazing music and presence in the Vancouver music scene.
youtu.be/nlyRiUIAByM?...

07.08.2025 18:28 πŸ‘ 2 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0

Our textbook on Markov decision processes and reinforcement learning is shortly going to press.

08.07.2025 05:46 πŸ‘ 1 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0

A key philosophy in our new book.

23.05.2025 14:55 πŸ‘ 2 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0
Post image
30.04.2025 04:16 πŸ‘ 2321 πŸ” 566 πŸ’¬ 65 πŸ“Œ 31

Stay safe

16.03.2025 00:34 πŸ‘ 1 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0

I strongly agree

09.03.2025 00:11 πŸ‘ 2 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0
Post image
08.03.2025 23:23 πŸ‘ 6063 πŸ” 1433 πŸ’¬ 244 πŸ“Œ 90
Preview
AI pioneers who channeled 'hedonistic' machines win computer science's top prize Teaching machines in the way that animal trainers mold the behavior of dogs or horses has been an important method for developing artificial intelligence and one that was recognized Wednesday with the...

Congrats Andrew and Rich, well deserved!! apnews.com/article/turi...

06.03.2025 03:40 πŸ‘ 6 πŸ” 3 πŸ’¬ 0 πŸ“Œ 0
We are Canadian.
We are Canadian. YouTube video by Average Joes

WE ARE CANADIAN. β€οΈπŸ‡¨πŸ‡¦πŸ’ͺ

youtu.be/1LzhCLzfJFg

06.03.2025 01:38 πŸ‘ 331 πŸ” 91 πŸ’¬ 6 πŸ“Œ 13

A new Trump resort?

26.02.2025 05:31 πŸ‘ 3 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0

Hell no!

09.02.2025 18:53 πŸ‘ 0 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0

In the early part of my career, many said to me: all your work in RL & MDPs will never see applications. They were so wrong...most modern robots are based on RL. And about 1/3rd of the talks at the last INFORMS were on RL and the Bellman equation.

08.02.2025 15:05 πŸ‘ 2 πŸ” 1 πŸ’¬ 1 πŸ“Œ 0

I would like to give a tutorial on MDPs.

05.02.2025 16:29 πŸ‘ 2 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0

Our forthcoming mdp/rl book should help! #rl #orms

25.12.2024 00:31 πŸ‘ 4 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0

Chinese food with family.

24.12.2024 19:17 πŸ‘ 0 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0

I’m planning to attend. Would like to speak about mdps and our new book.

13.12.2024 23:52 πŸ‘ 0 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0