Statistical or Dynamical? Which Model To Choose

In an earlier post ‘Model This and Model That’, we looked at some of the models used in the modelling and prediction of ENSO, collated and plotted by the IRI/CPC. We learnt a little about what statistical models and what dynamical models are, and touched on some of the reasons for these different types of models. The modeller's knowledge and experience, and the basis of what the model is for, all contribute to the type of model that is built. Another large factor in deciding what type of model to build comes down to resources. These resources include man power, computer power, time and availability of parameter data. For example, temperature is the most widely available meteorological data, as well as having the longest record of observations, whereas solar radiation is harder to obtain.

You may have noticed that there are more dynamical than statistical models used in the IRI/CPC plots. To be precise, there are 17 dynamical, and 9 statistical models used. So what does this say..? Is it as simple as dynamical models are better than statistical ones because there are more? Here are some of the advantages and disadvantages of both statistical and dynamical models to give you more of an understanding in why one would be built and used over the over.

Statistical

Advantages:

• Models are relatively quick and simple to build and run
• Little to no knowledge of underlying physical principles is required
• Simple analytical methods allows fairly easy model inversion

Disadvantages: 

• Model incorporates many parameter assumptions under certain observation conditions, meaning confidence of extrapolation is hard to justify
• The model does not enable further understanding of the physical processes

Dynamical

Advantages:

• Can be applied to a wide range of conditions
• Incorporates great complexity of processes via the use of numerical solutions
• Increases understanding of physical processes

Disadvantages: 

• Needs powerful computers to run complex models, which still can take long time to run
• All relevant processes  and corresponding variables need to be accounted for in the model
• Complicated to invert model due to difficulty in obtaining analytical solutions

Over the past few decades, a large proportion of models have improved in their ability to predict El Niño episodes (Guilyardi et al., 2012). These improvements are down to a combination of reasons including more advanced technology to observe and record data at a higher spatial resolution. Such improvements ultimately lead to more understanding of the physical processes of ENSO, which also enables improvements to the models to be made. It is thought that the reason behind those who haven’t significantly improved their skill is due to the additional processes that the models are now simulating. Such processes include; the carbon cycle; ecosystems; the indirect effect of aerosols; and, the interaction between stratosphere and troposphere (Guilyardi et al., 2012). However, the short term consequence of these additional processes and model complications not initially adding anything to the model, gives potential areas to explore and improve understanding of in the near future. This is where the conflict between over simplifying or over complicating a model comes into play. Simple models can be very powerful tools, but sometimes maybe they are just missing the mark when it comes to usefulness. Statistical models are usually simpler than dynamical models, as we found in the advantages and disadvantages of the model types above.

In the early 1990’s the statistical and dynamical models used for the study of ENSO showed comparable skills (Barnston et al., 1994). This was reinforced later in the 90’s by model predictions of the exceptionally strong El Niño in 1997/98. The forecasts from twelve statistical and dynamical models were studied, with results concluding that skills were again of similar levels (Landsea and Knaff 2000). None of the dynamical models conclusively performed better than the El Niño–Southern Oscillation Climatology and Persistence (ENSO–CLIPER) model, a simple statistical model, which was used as the baseline for comparison of model skill levels. Hence, it could be argued that statistical models where the preferred type due to the ease and lower associated cost of development.

A more recent study of the statistical and dynamical models used in the IRI/CPC plots from 2002 to 2011 was undertaken, with 8 statistical and 12 dynamical. The study found that despite only analysing the models over a short period of time (9 years), the skill of dynamical models have now exceeded those of statistical models, specifically for months March to May when shifts in ENSO are most likely, therefore making predictions most difficult (Barnston et al., 2012). Yet, it is also acknowledged that the short period of time in which the models were studied means that it is hard to prove the findings statistically robust, but within its limited time frame intriguing results nonetheless. Dynamical models have received greater funding than statistical ones over the past few decades, meaning that the majority of statistical models analysed have not been drastically altered in many years. This may play a part in why the dynamical models have shown to have better predictive power than their statistical equivalents. However, dynamical models are proving the potential they hold due to their capability of modelling the non linearity and rapid change of state of ENSO (Barnston et al., 2012). Without denying the value of statistical models, it seems that dynamical models will be at the forefront of modelling El Niño, especially with the continuing development in technology meaning the power of computers increase, and the associated costs decrease.

So I’ll finish by going off topic, but to wish you a very Merry Christmas! Without even doing any scientific analysis, I can (unfortunately) say that to a 5% significance level there is sufficient evidence to reject the hypothesis that it will snow on Christmas day of 2015 (if you’re in England that is). 

Source: Buzzfeed

Here’s hoping you didn’t place a bet that I advertised near the beginning of this blog... however if you did, remember, I did warn you that I wasn’t to blame if you lost! Enjoy the festivities wherever you may be, and whatever you may do, and we’ll catch up again in the New Year!