ADAPTIVE DECISION MAKING
/ADAPTIVE MANAGEMENT
We are now going to
combine the 2 major themes we’ve been discussing today: (1) using models to
make optimal resource decisions, and (2) reducing uncertainty via monitoring and
feedback of information into model predictions.
As before, we are trying to make an optimal management decision, but we
are faced with 2 or more alternative models about how the system works. When we do this, and also conduct
monitoring to follow up on the response of our system to management and then
feed back this information into future decision making, we are engaging in Adaptive Resource Management (ARM).
ARM has these key
features:
ARM thus ordinarily
requires some type of sequential decision making process. That is, if a decision will be made once, and
then never again, there is no possibility of adaptation. We’ll see that this isn’t a particularly
restrictive assumption, and that in fact most natural resource decision
problems can be viewed sequentially, and thus ARM is appropriate. But, if the decision problem is truly one
time, we can (and should) still use the first 3 features:
ADAPTATION THROUGH TIME –
Example: Adaptive harvest management for mid-continent mallards
· Decision: What
should the harvest rate (harvest regulations) be each year?
· Predictions:
Under additive (AMH) and compensatory (CMH) mortality hypotheses
· Monitoring:
Aerial surveys of breeding ducks (Bpop) and habitat (Ponds)
This is a problem
that clearly fits into ARM, because (1) there is a clear objective, (2)
decisions are being made sequentially, (3) there is uncertainty and prediction
under alternative models, and (4) we have a monitoring program to allow
adaptive feedback. We’ll illustrate this
example using a simplified version of the AHM models for mallards, in which we
are considering only population size (Bpop) at 3 levels: 5, 10, or 15 million
ducks, and 3 possible harvest decisions: restrictive (10% harvest rates),
moderate (20%), or liberal (30%) regulations.
The Netica program sets up this simple problem, in which
next year’s population is predicted given the current Bpop, and each of the 3
possible harvest levels. The “model” box
sets the weights on each model: 100% on CMH, 100% on AMH, or 50% weight on each
(if ‘unknown’ is selected). This model
takes into account both the immediate (this year’s harvest= current value)
value of harvest, and potential future harvest (future value) according to a
model which assigns probabilities to each potential population level next year,
for each decision, and under each model.
The total value (the right hand column in the ‘harvest rate’ box is the
current harvest value, plus the expected future value (gotten by
averaging values over each possible future population size). For example, if we currently have 10 million
ducks, and harvest at 20%, we will get a harvest this year of 2 million; the harvest
next year depends on next year’s Bop (20% chance of 5million , 50% of 10
million, and 30% of 15 million). The
expected value of this future (in this example, over the next 4years) harvest
will be 12.58 million, making the total expected value of that decision 14.58
million. If we compare that decision to
the other 2 (10% or 30% harvest) we see that 20% provides the highest
value. This result depends on Bpop: at 5
million ducks the optimal decision is also to harvest at 20%, but at 15 million
ducks it is 30%.
If we switch models
(100% weight on AMH) we get different results: 10% harvest when Bpop=5 million,
and 20% harvest at Bpop=10 or 15 million Finally, if we give equal weight to
each model (right-click ‘unknown’ on “Model”) then we get 10% harvest when Bpop=5 million, 20%
harvest at Bpop=10, and 20% at Bpop=15 million. This gives us a means of taking
into account model uncertainty in decision making.
To update our model
weights, we simply have to make a decision (presumably, the optimal one) and
then see what next year’s Bpop turns out to be.
This is essentially the reverse problem: we know where we started, what we decided, and where
we ended. Now we want to see how likely the competing models are, given these
outcomes.
Going back to
the harvest decision problem, assume that we:
(1) Started
with some relative belief in the alternative models (e.g., 50-50 for AMH-CMH),
(2) Observed that the population was in
a particular state (e.g., Current_Bpop=15)
(3) Based on (1), (2) and our model we
made a decision about what harvest rate should be in order to reach our
objective (maximize expected long-term harvest). In this example for
Current_Bpop=15 the Netica
program provides us with the optimal decision Harvest_rate=0.3 (notice that
it’s a close call!).
Now we come back a year later and survey
the population and find that Next_bpop=5 (i.e., the population has declined
from 15 to 5). Since we now have this information, our question no longer is
"what’s the probability of going from Current_pop=15 to Next_bpop=5, given
our decision Harvest_rate=0.3 and 50-50 probabilities for each model". The
question is turned around: given that we indeed have made this particular
transition, how does this affect our relative belief in the 2 models? It turns
out that this outcome is more likely for one of the models than it is for the
other, and we can use Bayes’ Theorem to adjust our model weights using these
likelihoods.
If we call L(5|M) the likelihood
of this occurrence (going from 15 to 5 under a harvest decision of 0.3) we have
L(5| CMH)= 0.28571 and L(5|AMH) = 0.
71429 (we’ll see where these come from in a minute). Given this, we can use
Bayes’ Theorem to get a new probability for AMH as
P(AMH|5) = L(5|AMH) P(AMH) / [L(5|AMH)
P(AMH)+L(5|CMH) P(CMH)]
Where do we get P(AMH) and P(CMH)? These
are just our prior beliefs (before the new survey came along) in each
hypothesis, or 0.5 each. Putting all this together we get
P(AMH|5) = 0.71429 (0.5) / [0.71429
(0.5)+0.28571 (0.5)] = 0.71429 = 71.4%
and P(CMH|5) = 1- P(AMH|5)= 0.28571 =
28.6%
That is, our belief has now jumped to
over 2/3 in favor of AMH.
In turns out that we can get these
values for any survey outcome by the approach suggested above: essentially
reversing the direction of the problem. Using the previous
Netica file accomplishes this. Simply click on S5=15, A5=0.3, and S4=5 to
set these as "knowns" , keeping "MODEL" as the unknown.
Notice that what has happened is that the model probabilities have shifted– to
the ones calculated above (Netica expresses these as percentages).
Where did the "likelihoods"
come from? Essentially what has happened is that Netica "knows" that
the conditioning has been reversed: what was previously unknown (the future
state of the system) and predicted conditional on the model weights (known), is
now known (observed), and it is the model weights that are predicted. This is
not "magic" ; it is Bayes' Theorem, where recall that
P(a|b)P(b) = P(b|a)P(a)
Netica automatically
applies Bayes’ Theorem, to solve for the conditional probability which is
unknown: in the first case, the conditional probability of the observed
population size at t+1, given the model, and in the second case, the
conditional probability of the model, given the observed population size.
As an important aside, the above is a
simplification of the "updating" process that is actually used. It is
simpler because we are focused entirely on the probabilities of changing states
under each model, and are totally neglecting sampling issues. That is, we
essentially assumed that the state of the system was perfectly observable at
each time period. In actuality we must take sampling error into account, and
this considerably complicates things. Nonetheless, the above illustrates the
basic ideas behind updating, and for that reason is useful (and necessary to
understand before moving on to more complex problems!).
Exercise : Find the new
probabilities for CMH and AMH if
Current_pop=15, Harvest_rate=0.3, but Next_yr_pop=10.
Using the New Information in Decision-making
What use can we
make of this new information? First, we apply the new information to our next
round of decision making, by changing the model probabilities in the decision
model to reflect the update. This has been done already in a revised Netica program
Exercise
: Find the optimal decision and its value, given Current_pop =5 and these new
model probabilities. What can you say about the value of this decision, compared
to 50-50 model uncertainty?
Moving these probabilities away from 50%
(to 100% or 0) should improve decision making. It thus has a value, tied
directly to the management objective. Formally methods like the above can be
used to determine just how much theoretically could be gained by collecting the
best information possible, know as the Expected Value of Perfect Information
(EVPI).
DON’T STOP NOW! It is unlikely (!) that in a single
year, perfect knowledge about a system will be gained. Knowledge tends to
increase incrementally, and the gain is slowed down by environmental variation,
imperfect management control, and sampling error. The updating process should
continue, by continually making new predictions about the impact of management,
using the current model probabilities (so, knowledge gained to date) as the
starting point. Use the Netica file with the new probability weights (revised program), which P(AMH) = 71.4% and P(CMH) =
28.6%
Exercise: Find the new model probabilities using the revised Netica program if Current_pop =10,
Harvest_rate =0.2, and Next_yr_pop =15.
AHM SUMMARY
Adaptive Harvest Management calls for:
(1) Starting with some relative belief
in the alternative models (e.g., 50-50 for AMH-CMH),
(2) Observing that the population is in
a particular state (e.g., Current_bpop=15)
(3) Based on (1), (2) and our models
making e a harvest decision in order to maximize the objective
(4) Collecting survey data next year to
compare to each model prediction and calculate new model probabilities.
(5) Going to step (1) using the new
model probabilities.
A Note on Simplicity Versus Complexity
Perhaps you are
bothered by the fact that we modeled a harvest system with a single state
variable (abundance), without consideration for habitat or other factors. You
might also not like the fact that the population state is considered rather
crudely (5, 10, 15 of whatever units– perhaps millions of ducks).
In fact, most of the major elements of
adaptive harvest management (AHM) are recognizable in this example, albeit
simplified. The key thing to remember is that we do not have to include
all the information about the system, in order to use a model for decision
making. The first thing to ask is: what are the elements of the system
essential to the decision maker, relevant to determining the future value of
the decision? The second thing to ask is, what of these system states can be
practicably assessed, in a timely manner, so as to be useful to decision
making? For North American migratory birds, the key pieces of information are
(1) the abundance of the stock (as determined by annual breeding ground
surveys) and (2) the abundance of suitable breeding habitats (determined also
by spring surveys). A number of alternative, empirical models are used to
predict fall flight and the impacts of harvest (e.g, under AMH and CMH). The
system states are considered at a bit finer scale of resolution than our
examples above, and the models are a bit more complex (although fundamentally
reducable to some simple probability tables, like the examples), and the
computational methods a bit more complicated (e.g., SDP as noted earlier). But
the basic pieces are the same.
For a more complete coverage of this
topic, see the FORS
8390 lab on harvest decision analysis.
ADAPTATION THROUGH SPACE
As mentioned
earlier, sometimes the decision that is made is one that is not likely to be
revisited any time soon. For instance, the decision might be whether (or how)
to build a reserve, cut a forest stand, or take out a dam on a river. Apparently, this type of decision problem is
not amenable to adaptation, because the decision will not repeat soon. However, if we think about this problem a
bit, we will realize that adaptation can be used, if we generalize the
problem to one of using decision making and monitoring at 1 spatial location to
inform us about a decision that has not yet been made at some other spatial
location. This information sharing is
appropriate, if the locations are part of an overall system thought to behave
according to a common set of dynamics (which are imperfectly understood).
Example: Reserve design
In this example, we
have a decision problem involving the question of whether to build or not build
a reserve to protect a particular species from local extinction. The species
currently exists at each of 2 sites, and we have alternative models that make
different predictions about whether or not the species will persist following
each decision (one basically says it makes no difference, the other says
building the reserve increases the probability of persistence). Following each decision, we monitor the
population and observe whether it persists or goes locally extinct. Finally, each of these outcomes has a value
to the decision maker (see the FORS
8390 lab on conservation decision analysis for more details, including the
rationale behind these values).
The first decision is whether to build a reserve on the first site, followed by a decision as to whether to build the reserve on the second site. Let's suppose that at the first decision, we have (perhaps based on previous empirical work) greater (70%) belief in Model 2 than in Model 1. This is the scenario programmed into the spatial updating with Netica example.
In this case, the expected value of "Reserve" for Decision 1 is slightly higher than "Nothing" (100.5 vs. 100.25), so we build the reserve. Suppose we then observe that the species persists (Status 1= persists). Notice that the model weights have changed: they are now 24.8 - 75.2 in favor of Model 2. Notice too that the expected value of Decision 2 now changes: evidence as a result of the previous decision has "fed back" into decision making. We again make the apparently optimal decision (Reserve) and observe what happens (persistence or extinction). For example, if the species persists at the second reserve, the model weights now change to 20.2 - 79.8.
Exercise : Describe
the impact of the following sequence of decisions and observations on the
relative belief in Model 1 and Model 2 (following all decisions and
observations):
|
Decision 1 |
Status 1 |
Decision 2 |
Status 2 |
P(Model1) |
P(Model1) |
|
Reserve |
Persist |
Nothing |
Persist |
|
|
|
Reserve |
Persist |
Reserve |
Extinct |
|
|
|
Reserve |
Extinct |
Reserve |
Extinct |
|
|
|
Nothing |
Persist |
Reserve |
Extinct |
|
|
|
Reserve |
Persist |
Nothing |
Extinct |
|
|
|
Reserve |
Persist |
Reserve |
Persist |
|
|
For a more complete coverage of this topic, see the FORS 8390 lab on conservation decision analysis.