Dr Carl Boettiger
The University of California, Berkeley
From Theory to Practice: Reinforcement Learning for Realistic Ecosystem Management
Ecological, environmental, and social systems are complex, incompletely understood, and only partially observed. How then do we make effective management decisions in the face of such multiple and often irreducible uncertainty? In this course, we will introduce fundamental concepts of decision theory to address these challenges. While this approach has long had a compelling theoretical basis, practical implementation has to date relied almost entirely on highly stylized models and simplifying assumptions to make that theory computationally tractable. Today, advances in an important area of artificial intelligence known as reinforcement learning are suddenly making this theory readily applicable to vastly more complex and realistic models.
We will begin with classic sequential decision problems in conservation and ecosystem management as well as traditional algorithms for exact solutions such as stochastic dynamic programming. We will see how these algorithms buckle under the ‘curse of dimensionality’ for more realistic problems, before turning our attention to draw on recent methods in AI that can break this curse, but will also sacrifice the promise of optimality for something far more slippery. As these approaches frequently rely on intensive computing, we will also introduce fundamentals of reproducible, portable, and scalable cloud computing using modern, secure, and user-friendly toolchains.
Topics:
- Markov Decision Problems
- Optimal Harvesting Theory
- Stochastic dynamic programming
- Deep reinforcement learning in the context of conservation policy
- Low-friction cloud computing (containers, environments, dependency management, codespaces)
- GitHub Version control / collaboration / workflow
Pre-requisites
Participants will benefit from some familiarity with dynamical systems (differential or difference equations, stochastic models) and prior exposure to programming in R or Python. Tutorials will include some use of the Python programming language.
Relevance
Structured decision-making and adaptive management have long played a central role in conservation policy and practice, from fisheries management to the location of protected areas, yet the biodiversity crisis has taken on ever increasing urgency in face of climate and other anthropogenic global change. Meanwhile, AI based decision-making algorithms are reshaping industry and challenging societal structures. Understanding both the potential and limitations of these tools to leverage their strengths and avoid their pitfalls is thus a growing imperative for conservation and ecosystem management.
Dr Carl Boettiger
The University of California, Berkeley
Carl Boettiger is an Associate Professor in the Department of Environmental Science, Policy, and Management and a faculty advisor to the Schmidt Center for Data Science and Environment at the University of California, Berkeley. He is an active contributor to open source projects, including being co-founder of the rOpenSci and Rocker projects.
Carl works on problems in ecological forecasting and decision making under uncertainty, with applications for global change, conservation and natural resource management. He is particularly interested in how we can predict or manage ecological systems that may experience regime shifts: sudden and dramatic changes that challenge both our models and available data. The rapid expansion in both computational power and the available ecological and environmental data enables and requires new mathematical, statistical and computational approaches to these questions. Ecology has much to learn about what are and are not useful from advances in informatics & computer science, just as it has from statistics and mathematics. Traditional approaches to ecological modeling and resource management such as stochastic dynamic systems, Bayesian inference, and optimal control theory must be adapted both to take advantage of all available data while also dealing with its imperfections. Carl’s approach blends ecological theory with the synthesis of heterogeneous data and the development of software – a combination now recognized as data science.