“Never make predictions, especially about the future.” Yogi BerraA couple of weeks ago we had what we call a forecasting bust. During that spell of very warm weather at the end of April we forecasted on Saturday a Sunday high of 69. We ended up at 80 degrees -- an 11 degree error 24 hours out in time. In this case it was a matter of a cold front ending up about 30 miles farther north than we were thinking.
One of the things we constantly do in the weather office is to look at our errors. Specifically, we look at our high temperature errors when we forecast a day or two in the future. It may seem a rather simplistic way to monitor how we are doing but it really takes into account a lot of our forecasting. When we make a temperature forecast we are asking ourselves questions like “Can we get enough sunshine to warm into the 50s?” or “will it start raining in the morning which will hold back temperatures?” In that way a temperature forecast is also a cloud forecast, a precipitation forecast, a frontal forecast etc all rolled into one.
Anyhow, each year I take a look at how we did with our next day temperature forecast. I’m not perfect, I miss recording a few days each year but it’s still a pretty large sample. For example, in 2008 I used 357 days (out of 366 total days). First off, you might be interested to know that we are right on with a next day temperature forecast (0 error) just under 15% of the time (52 out of 357 days). Hearing that might spark the old debate that meteorologists are paid for getting it wrong most of the time (in this case 85% of the time!). Let me put a different spin on this. When you’re outside on a summer day can you tell the difference (without a thermometer by your side) between 80 degrees and 82 degrees? I know I can’t. So what if we look at our error if we are within 2 degrees (plus/minus) of the actual high. In that case we are within that range almost 2/3 of the time (64%). Fully 90% of our next day forecasts fall into a plus or minus 5 degree range. So the next time you hear us give a temperature forecast for the next day think of it as a probability forecast: we are saying that we are 90% sure of being within 5 degrees of the actual high we are forecasting.
How many times do we have a day like that Sunday in late April when we are more then 10 degrees off? Last year it didn’t happen at all. In 2007 3 times. In 2006 2 times. Over the course of a year that is less than 1% of the time.
The next question is did our forecasts improve in 2008? The answer is yes. Our next day forecast error in 2007 was 2.40 degrees and last year it was down to 2.27 degrees. One of the things we are also constantly watching is how we do versus our computer model forecast temperatures. Both the NAM and GFS models that we mention in the blog from time to time will statistically determine high and low temperatures for Syracuse with each run of the model. The good news here is that we are almost always better than the computers. There is no big celebration when this happens, we aren’t bumping our chests or jumping up and down saying “We’re number 1!” It’s just nice to know that we can objectively look at the model data and still improve on what the computer says. When the computers start beating us on a regular basis then I’ll know it’s time to get into a new profession
