Researchers and forecasters are always looking for ways to improve weather forecasts. After several years of development, the NOAA Hydrometeorological Testbed at the Weather Prediction Center (HMT-WPC) recently unveiled a new online verification resource that evaluates how well computer models are predicting the intensity and location of precipitation. This tool, called MODE (Method for Object-Based Diagnostic Evaluation), was developed by NOAA’s Developmental Testbed Center (DTC) and uses an object-oriented (shape-recognition) approach to forecast evaluation. By comparing the observed precipitation to the model forecast feature-by-feature, this tool can help identify deficiencies in weather prediction that can be addressed through future research.
“This innovative tool provides a fairer representation of the forecast model’s skill because it allows the user to compare predicted and observed precipitation over a region where precipitation occurs, rather than just at a specific location,” says Allen White, HMT Program Director.
Object-oriented verification techniques such as MODE are designed to better account for spatial discrepancies between forecast and observed areas of precipitation. This avoids the “double penalty” problem that can arise in traditional verification methods in which a forecast is penalized once for missing the observed area of precipitation and a second time for giving a false alarm. In addition, these techniques can provide more detailed information about forecast quality than can be obtained through traditional verification measures alone.
MODE verification determines the degree of similarity between regions of precipitation in the forecast and observed fields by evaluating characteristics such as the size of the precipitation areas, the angle of orientation, and the distance between two areas. Precipitation areas that are found to be similar to one another are considered “matched”, while those that are not similar are considered “unmatched”. Unmatched areas in the forecast field are equivalent to a false alarm, while unmatched areas in the observed field are equivalent to a miss.
"Since MODE is designed to try to replicate a forecaster’s subjective evaluation of forecast quality", says Wallace Hogsett, WPC’s Science and Operations Officer, "We are hoping that over time this technique will help us quantify some of the model biases frequently observed by our forecasters."
WPC's MODE website displays maps comparing 24 hr precipitation forecasts to the corresponding observed precipitation. Preliminary verification is based on radar-derived precipitation observations (NSSL MRMS), and is available the same day. The final verification is based on a combination of radar and gauge precipitation observations (NCEP Stage IV) and is available two days later. In addition to the graphical comparisons, the page also displays statistical information for each matched object. Verification is available for Day 1, Day 2, and Day 3 forecasts of 0.50, 1.0, and 2.0 inches of precipitation from WPC and several numerical models, including the North American Mesoscale Forecast System (NAM) and the Global Forecast System (GFS).