What to do in the heat? Compare weather forecasts. I’ve recently been trying to see if any of the major weather forecasts have a bit of accuracy. This is unscientific, and limited in scope and location.
The 5 online weather reports I tested were from the National Weather Service (NWS); Weather.com (The Weather Channel – TWC); AccuWeather; WeatherUnderground; and Intellicast. For one of my first tests, I also checked using Yahoo, Google, and msn. Those 3 were thrown out for subsequent tests, as Yahoo is just reporting TWC data, and iGoogle uses WeatherUnderground; msn was tossed because it was so far out of the ballpark it wasn’t worth checking ever again, probably because it’s from a location nearly 20 miles from the target location.
(Notes: 1. It appears that TWC and Intellicast use the same forecast data – in nearly every instance their temperature predictions were identical. 2. I based this test on a location I’m familiar with [a small town in a different climate environment] but which is 100 miles distant, so I would not have a personal look-out-the-window bias. 3. I used the reported high temperatures from both the city and county weather stations as my “official” temps in all instances. It’s always possible that some services do not use those locations, rather using a different place such as the small regional airport about 5 miles from the city/county weather stations. 4. AccuWeather issues a 15-day forecast – longer than anyone else – but given the inaccuracies in even medium-range forecasts, as you’ll see below, that seems pretty useless.)
My first test was of high temperature predictions for 3 consecutive days, based on the forecast high from each service the same day. On two of the days, both Intellicast and TWC were the most accurate, within 2 degrees of the actual high for the day. On one day, both the NWS and AccuWeather called the high temperature exactly.
During the first 3-day test, I also looked at how forecasts changed for the third day, over the course of the 3 days. Most services kept their forecasts within 1 degree of their 3-day-previous and 2-day-previous guesses.
My second test was for a medium-term forecast, again for 3 consecutive days (this testing was done about a week after test 1, above). I recorded forecasts for 4, 5, and 6 days out, and then checked to see if those forecasts changed over time until each target day arrived. For the 4-day-out forecast, no service came closer than 3 degrees of accuracy. When that first test day arrived, the actual recorded high temperature was 74 – the day-of forecasts for highs that day were 80-83-86-86-88. Interestingly, AccuWeather was “closest” with their forecasts from 2 and 3 days previous (79-77), but on day-of they were the worst, with a forecast high of 88. The closest day-of forecast (80, still off by 6 degrees) was from the NWS.
For all intents, we had no real winners that first day.
When target day 2 arrived, the day-of predictions ranged from 65 to 78 – an amazing 13-degree difference between forecasts. (The previous-days-out forecasts had ranged from 71 to 81.) The actual high temperature that day was 67. WeatherUnderground was dead on with a day-of guess of 67, and the NWS came in second with a guess of 65. But they, like every other service, had been predicting mid to upper 70s just a couple of days previous.
So for a day-of forecast, NWS and WU were the best, but since this was a test of a medium-range forecast, I’d again say we had no winners.
For my final target day, the 1- to 5-day-previous forecasts had ranged from 70 to 80, with the day-of forecasts ranging from 69 to 79 – still a full 10 degrees of difference. Actual temperature that final day of the test was 66. The closest day-of calls were again by the NWS (69, off by 3 degrees), WeatherUnderground (72, off by 6 degrees), and AccuWeather (73, off by 7 degrees).
(An interesting observation: For the final target day, Intellicast forecast a high of 79 – and they had forecast that temperature every day from 5 days out through the day-of. Ya gotta admire conviction, even when it’s consistently and dramatically wrong.)
Once again we had no long-term winners. TWC and Intellicast were the worst, calling for highs of 78-80 in every one of their 5-days-previous forecasts. AccuWeather was a little bit better, with forecasts ranging from 73-76. The NWS did predict 68 (from 1 day before) and 69 (day-of), and two other forecasts were at 70 and 71. WeatherUnderground called for 70 once, and 72 twice. But overall, another big OOPS for all of them.
Final Conclusions:
From a trend basis, the NWS, WeatherUnderground, and (lastly) AccuWeather seemed a tiny bit closer (when I averaged temperature predictions over the entire multi-day trial period), but none were close enough to take to the bank. Day-of, the NWS’s guess had a slight but significant edge most days, with WeatherUnderground a close second. Yet in fairness, there were a few days that Intellicast/TWC came closer for day-of guesses. For current conditions – actually happening on the ground – WeatherUnderground has an excellent array of weather station sites (at least for the region I was testing). A really odd observation was that if the high for the day turned out to be higher than the average for all 5 forecasts, Intellicast and TWC were usually (slightly) closer; if the high was lower than the average forecast, NWS and WeatherUnderground were closer.
Personally, when I want a forecast, I’ll probably check all services just for grins (if I have the time). Right now, I can’t say I’d rely on any one site for medium- or long-range forecasts. My two go-to sites will probably be the NWS and WeatherUnderground (and remember, if iGoogle is your taste, it also uses WU information).
Next up: As soon as we get into a rainy/stormy period, I’m going to test forecasts about precipitation.
Posts
