The skill of multimodel seasonal forecasts of the

Authors: F J DoblasReyes V Pavan D B Stephenson

Publish Date: 2003/09/27

Volume: 21, Issue: 5-6, Pages: 501-514

PDF Link

Abstract

The skill assessment of a set of wintertime North Atlantic Oscillation NAO seasonal predictions in a multimodel ensemble framework has been carried out The multimodel approach consists in merging the ensemble hindcasts of four atmospheric general circulation models forced with observed sea surface temperatures to create a multimodel ensemble Deterministic ensemblemean based and probabilistic categorical NAO hindcasts have been considered Two different sets of NAO indices have been used to create the hindcasts A first set is defined as the projection of model anomalies onto the NAO spatial pattern obtained from atmospheric analyses The second set obtains the NAO indices by standardizing the leading principal component of each singlemodel ensemble Positive skill is found with both sets of indices especially in the case of the multimodel ensemble In addition the NAO definition based upon the singlemodel leading principal component shows a higher skill than the hindcasts obtained using the projection method Using the former definition the multimodel ensemble shows statistically significant at 5 level positive skill in a variety of probabilistic scoring measures This is interpreted as a consequence of the projection method being less suitable because of the presence of errors in the spatial NAO patterns of the models The positive skill of the seasonal NAO found here seems to be due not to the persistence of the longterm decadal variability specified in the initial conditions but rather to a good simulation of the yeartoyear variability Nevertheless most of the NAO seasonal predictability seems to be due to the correct prediction of particular cases such as the winter of 1989 The higher skill of the multimodel has been explained on the basis of a more reliable description of largescale tropospheric wave features by the multimodel ensemble illustrating the potential of multimodel experiments to better identify mechanisms that explain seasonal variability in the atmosphereThis study was undertaken when the first author worked at the Centre Nationale de Recherches Météorologiques MétéoFrance Toulouse France VP has received support from the Progetto Strategico SINAPSI funded by the Ministero dellIstruzione dellUniversitae della Ricerca MIUR and Consiglio Nazionale di Ricerca CNR The authors wish to thank David Anderson Magdalena Balmaseda Michel Déqué Thomas Jung Alexia Massacand Laura Ferranti and Tim Palmer for reviews of early drafts and constructive advice Richard Graham and an anonymous reviewer are especially acknowledged for their significant contribution to the improvement of the scientific quality and readability of the paper This work was in part supported by the EUfunded DEMETER project EVK2199900197A tool commonly used to evaluate the association between ensemblemean hindcasts and verification is the time correlation coefficient This measure is independent of the mean and variance of both variables As in the rest of the study different climatologies for hindcasts and verification were computed using the crossvalidation technique making the correlation estimator unbiased Déqué 1997A set of verification measures has been used to assess the quality of the probabilistic hindcasts the ranked probability skill score RPSS the receiver operating characteristic ROC area under the curve the Peirce skill score PSS and the odds ratio skill score ORSS Most of them along with estimates of the associated error are described in Stephenson 2000 Zhang and Casey 2000 and Thornes and Stephenson 2001 where the reader is referred to for more specific definitions and propertiesThe ROC Swets 1973 is a signaldetection curve plotting the hit rate against the false alarm rate for a specific event over a range of probability decision thresholds Evans et al 2000 Graham et al 2000 Zhang and Casey 2000 Basically it indicates the performance in terms of hit and false alarm rate stratified by the verification The probability of detection is a probability decision threshold that converts probabilistic binary forecasts into deterministic binary forecasts For each probability threshold a contingency table is obtained from which the hit and false alarm rates are computed For instance consider a probability threshold of 10 The event is forecast in those cases where the probability is equal to or greater than 10 This calculation is repeated for thresholds of 20 30 up to 100 or whatever other selection of intervals depending mainly on the ensemble size Then the hit rate is plotted against the false alarm rate to produce a ROC curve Ideally the hit rate will always exceed the false alarm rate and the curve will lie in the upperlefthand portion of the diagram The hit rate increases by reducing the probability threshold but at the same time the false alarm rate is also increased The standardized area enclosed beneath the curve is a simple accuracy measure associated with the ROC with a range from 0 to 1 A system with no skill made by either random or constant forecasts will achieve hits at the same rate as false alarms and so its curve will lie along the 45° line and enclose a standardized area of 05 As the ROC is based upon a stratification by the verification it provides no information about reliability of the forecasts and hence the curves cannot be improved by improving the climatology of the system The skill score significance was assessed as in the case of RPSS by Monte Carlo methods

Keywords: