On Model Parameterisation

Presentation on theme: "On Model Parameterisation"— Presentation transcript:

1 On Model Parameterisation
with special reference to 3-PG Peter Sands CSIRO Forestry and Forest Products and CRC for Sustainable Production Forestry

2 Major topics covered Calibration or parameterisation?
Sensitivity analysis Thoughts on parameterisation Parameterisation of 3-PG

3 Calibration or parameterisation?
What’s the difference? Example of calibration

4 What’s the difference? Calibration Parameterisation
an empirical relation is developed to improve match of model output to observed data analogous to calibration of an instrument does not affect model, but operates on its outputs Parameterisation parameters assigned from independent data or estimated by adjusting them to optimise fit of model output to observed data operates directly on the model

5 Model calibration A model has been parameterised using a certain set of data (r2=0.97) but exhibits systematic errors under new conditions it might not be possible or appropriate to re-parameterise the model could calibrate the model for new conditions by plotting observed data v. model output and developing an empirical calibration relationship

6 Model calibration (continued)
Plot of observed data v. model output shows strong quadratic relationship ycal = ymdl ymdl2 Use this as a calibration relationship Application to model output gives adequate prediction of observed data under the new conditions (r2=0.91)

7 Sensitivity Analysis What is sensitivity analysis?
Formal definition of sensitivity Examples: PROMOD

8 What is sensitivity analysis?
Structural sensitivity how sensitive is the performance of the model to structural assumptions or processes included? Parameter sensitivity how sensitive is the performance of the model to the values of parameters that characterise relationships included in the model? Input sensitvity how sensitive is the performance of the model to variations in the data required to drive the model?

9 What is ... ? (continued) Extensive structural, parameter and input sensitivity analyses performed for PROMOD Results have been invaluable strengthened our acceptance of model’s structure enhanced understanding of model behaviour facilitated parameterisation of PROMOD for E. nitens & P. radiata can elucidate GxE interactions No similar analysis yet published for 3-PG or CABALA but come to EUCPROD in November

10 Definition of sensitivity
Define sensitivity Xp of output X w.r.t parameter p as i.e. ratio of fractional change in output to fractional change in parameter Calculate Xp by running model for range of p for each p use +ve & -ve values for of p to capture possible non-linearity repeat for sites with radically different conditions to capture environmental variability

11 PROMOD parameter sensitivity
Example of sensitivity of four parameters at four distinct sites  Darkan  Forcett  Northcliffe  Esperance Note following nonlinearity site variability value of a1 generally not critical

12 Ranking parameter sensitivity
Here PROMOD parameters are ranked by their sensitivity across range of sites   mean sensitivity dS or CofV  environmental sensitivity

13 Joint sensitivity wrt multiple p’s
Example is for MAI predicted by PROMOD by contour plots show sensitivity of MAI w.r.t. two parameters at a single site shows regions of high and low sensitivity

14 Joint sensitivity (continued)
Example again for MAI predicted by PROMOD combine joint sensitivity plots from two sites shaded region is range of parameters such that MAI is within 10% of observed NB: there might be no such range!

15 Thoughts on Parameterisation
What is parameterisation? Parameter assignment and estimation Importance of understanding the model Question the results!

16 What is parameterisation?
Parameterisation is the process where model parameters are assigned values so that the model in some way mimics the original system Two common techniques used in practice directly on the basis of experimental data obtained from a study of the relationship in question or estimated by adjusting parameters to optimise fit of model output to observed data

17 Example of parameterisation
The maths: where R is root partitioning and (1) (2) Above-ground biomass partitioning in 3-PG is based on allometric relationships between (1) stem mass and DBH (2) ratio of foliage to stem partitioning and DBH Can assign values for aS and nS by statistical analysis since (1) is based on observed data, but (2) is not accessible to experiment, so probably have to estimate values for ap and np by fitting model output to observed biomass data

18 Example … (continued) Parameter assignment
3-PG uses an allometric relationship between mean stem mass (wS, kg tree-1) and mean stem diameter (B, cm): Supporting data readily available. So assign values based on this data, rather than by fitting 3-PG output to observed stand data. Also seems to show little variation in aS and nS for same species

19 Example … (concluded) Parameter estimation
If relationship not experimentally accessible, estimate its parameters by varying them so as to optimise the fit of model output to observed data. The assumed allometric relationship between foliage to stem partitioning ratio and DBH is not easy to examine experimentally Since pFS affects both WF and WS , estimate values for ap and np by fitting 3-PG output to observed WF and WS

20 They stress the fundamental importance of understanding the model
Some rules of thumb ... The following “rules of thumb” are from experience with various models - from dynamic simulations models, to simple, static non-linear regressions. They stress the fundamental importance of understanding the model Experience with 3-PG will illustrate many of these points!

21 Know thy Model! Parameterisation of a model is aided by intimate knowledge of the model: The model structure its submodels and feedbacks Input and output data the quality of inputs how outputs are derived from state variables What the parameters are and do sensitivity of key outputs to parameters biological or physical limits on values

22 Understand model structure
Parameter estimation can be strongly affected by the structure of a model. Sub-models ... to properly test or parameterise a model, work at the sub-model level if no observed outputs from a sub-model, do not estimate parameters in that sub-model, or do so with care! … and feedback loops feedback loops imply sub-models are not independent errors propagate and magnify through feedback loops

23 Understand the data The available data determines which parameters can be estimated and strongly affects estimation process. Quality of input data Errors in input data cause errors in outputs and hence errors in parameter estimates Correlations in input data can cause problems Derived output variables Some output variables are derived from state variables, and these relationships are possibly site or species specific Fit to state variables wherever possible

24 Understand the parameters
Parameters characterise relationships in a model, and their values can drastically change model behaviour. Can a parameter be assigned a value a priori? Whenever possible assign parameters from independent experiments which directly pertain to the parameter rather than by fitting model output to observed data. Sensitivity Parameter (+structural) sensitivity analyses provide insight into how parameters affect a model’s functioning Also facilitates adaptation of the model to novel situations or species

25 Understand the parameters (concluded)
Compounded parameters Independent parameters often combine into a compound parameter which is estimated, not the original parameters Biological or physical meaning What is the significance of a parameter? Estimation is aided if a parameter has an intuitive meaning What are its units? What are the biological limits on its values? How sensitive is the model to the parameter?

26 Verify or validate the resulting model output!
Question your results! Unwary estimation quickly generates a mountain of garbage … or give good fit for wrong reasons. Does the parameter look reasonable, e.g. is it within the biological range? Is it within its standard error of a “nice” value, e.g. 0? Can any parameters be formally identified? Are any parameters correlated? Did the estimation seem to take ages to run? Verify or validate the resulting model output!

27 Parameterisation of 3-PG
Introductory comments re 3-PG Growth Fertility effects Specific leaf area What can be done with what data? Stand initialisation

28 Parameterisation of 3-PG
Many parameters determine stem growth essentially multiplicatively Must separately consider all components and state variables of model

29 The Essence of 3-PG (or of any other forest growth model)
The basic equations for this system are:

30 Data to parameterise 3-PG
Based more on empirical observations than on physiological data but physiological data can be a guide Allometric data for stem biomass v DBH Observed data from a broad range of sites site & soil factors, climate data to run 3-PG observed foliage, stem and root biomass pools litterfall and root-turnover observed stem numbers, volume and DBH Seasonal variation of ASW

31 Explicit age-dependence in 3-PG
3-PG has several explicitly age dependent relationships age factor in  canopy structure – SLA, canopy closure stem volume - branch & bark fraction, basic density litterfall rate These empirical relationships are assigned parameters by analysing the corresponding observed data

32 Growth rates in 3-PG Examination of 3-PG growth rates show some parameters combine to form a compound parameter e.g. stem growth is given by where pFS depends on WS through stem diameter Because the product (1-R)aCY appears in both DWF and DWS, estimation based on WF and WS will not determine individual parameters in this product

33 Growth rates in 3-PG… (continued)
Now examine components of the product (1-R)aCY in detail The maths: If working with long-term data, growth conditions are given and the average fx and  can be assumed constant from year to year So: all parameters in fT, fN and hR are compounded with aCx and Y

34 Growth rates in 3-PG… (concluded)
This has significant implications for estimation when only WF and WS are observed: Estimation at a single site Can only estimate (1-R)aCY - can’t separate hR, aC or Y ! If appropriate experimental data is available can assign values to some individual parameters in this product. Estimation at multiple sites If significant environmental differences between sites, then may estimate parameters in the various fx and j. If WR is also observed, then can also estimate hR.

35 The Essence of 3-PG (or of any other forest growth model)
The basic equations for this system are:

36 Effects of site fertility
The fertility rating (FR) in 3-PG potentially affects both biomass allocation and light use efficiency high FR  low allocation to roots and higher volume growth high FR  higher light use efficiency and overall higher biomass production But it is NOT possible to distinguish these two affects without root & stem biomass data see next slide! Again – danger of right answers for wrong reasons!

37 Effects of FR… (continued)
Another issue is the effect of site fertility on canopy quantum efficiency and on root partitioning: Site fertility and growth rates FR affects growth rates through fN in aC and m in hR This expression has a rich repertoire of behaviour w.r.t. its parameters, and especially w.r.t. site fertility. This makes it easy to adjust FR to get a good fit, and also means it’s easy to get “right” answer for wrong reason.

38 Effects of FR… (concluded)
Different parameters – same results! Very similar above ground growth rates can be obtained with very different parameters FR has full affect on m and none on  - the default FR has no affect m, some on  FR full affect on m, some on  Legend m0 fN0 hRx hRn Full/none None/some Full/some

39 Specific Leaf Area Specific leaf area an important parameter:
affects rate of canopy development & early biomass production affects light interception and hence growth rates declines during early canopy development. During early canopy development WF is small and hence net production is but around canopy closure WF is large and stable and

40 The Essence of 3-PG (or of any other forest growth model)
The basic equations for this system are:

41 could estimate these when fitting to observed data
Initial Conditions Dynamic simulations start at some initial time with the system in an initial state - called the initial conditions. The precise time-course the system then follows depends on the initial conditions, sometimes quite dramatically. The initial conditions can be arbitrarily assigned, or may be known. In the context of 3-PG could start simulation at planting with a nominal seedling biomass, or at time of first observation with observed stand data. Because observations are subject to error, so are the initial conditions  could estimate these when fitting to observed data

42 Initial Conditions (continued)
Simulation of canopy development sensitive to initial stem and foliage biomass, stem growth rate is insensitive to initial conditions.

43 Initial Conditions (concluded)
If stand initialised at planting with typical seedling data, canopy development sensitive to initial biomass, but stem growth rate is insensitive to initial biomass.  Observed LAI  Observed stem biomass Initialising with nominal stand data is the normal for most applications

44 What can be done with what data?
The parameters that can be estimated, or submodels that can be tested, depend on 3-PG output variables for which data has been observed the number of distinct sites from which data is available the range of environmental conditions covered by these sites

45 What can be done… ? (concluded)
Available data Stem& foliage biomass (or LAI) Root biomass Leaf litter Soil water, sap flow Stem numbers Stem diameters Stem volumes Principal sub-models Light interception, net production, above-ground partitioning Root partitioning Canopy development Water balance, canopy conductance Mortality Stem allometrics Branch + bark fraction, density Key parameter groups aC, s, pFS hR gF gC, fVPD, fSW wSx1000 aS, nS pB, r

46 What can be done… ? (concluded)
It is highly unlikely that we will have all the data needed to separate all state variables in 3-PG! I am concerned that unsound parameterisation and questionable applications of 3-PG are out there! Beware: There is a real danger that we are getting the right answers for the wrong reasons, or just plain wrong answers!

47 Concluding remarks … Does it all mean anything? Learn by experience.
Important to bear guidelines in mind. Remember: The proof of the pudding is in the eating.