How to Clean Errors from Technological Systems

Satellites and high-tech weather instruments are just two examples of the range of sensitive technological systems that our civilization is heavily reliant upon today. Like everything else in this world, regardless of the amount of work that goes into creating these systems, they all have errors, caused either by glitches, or by oversight on the part of the people overseeing their construction. Now, in an attempt to make that a thing of the past, a team of experts has learned how to combine statistics, imaging and other disciplines to create an innovative computational technique that is able to identify errors in all of these sensitive systems, the US National Science Foundation (NSF) reports.

Details of this amazing innovation, which has the ability to impact a wide variety of research fields, appear in the latest issue of the respected Journal of Atmospheric and Oceanic Technology. The Intelligent Outlier Detection Algorithm, or IODA, technique has already been patented, and is currently awaiting implementation in existing systems. One of the most remarkable things that IODA can do is alert human operators whenever faulty, or failing, sensors begin to produce erroneous readings that could provide the foundation for other technological or scientific processes. This would help reduce the number of errors that would get transmitted to the secondary processes, and ultimately result in better end-products.

The new analysis technique was developed by researchers at the University of Colorado in Boulder (UCB), who collaborated closely with colleagues from the National Center of Atmospheric Research (NCAR). The Program Director for the Center is NSF expert Steve Nelson. “This technology will have broad applicability in many new areas,” he believes. “This could, at least in theory, enable operators to keep a system performing even while it's failing. When a system starts to fail, it's absolutely critical to be able to control it as long as possible. That can make the difference between disaster or not,” NCAR software engineer Andrew Weekley, the leader of the effort to develop the algorithms behind IODA, adds.

“Our thought was to organize a sequence of data as an image and apply image processing techniques to identify a failure unambiguously. We thought that, by using image processing, we could teach the system to detect inconsistencies, somewhat like a person would,” Weekley reveals. “I would say the approach we report in the paper is a radical departure from the usual techniques found in the time series literature. The image processing and other techniques are not new, but the use of these images and techniques together in a time series application is new. IODA is able to characterize good and bad points very well in some commonly encountered situations,” UCB mathematician Kent Goodrich, also a co-author of the new paper, concludes.