The shutdown of Blaine's water system this year can be chalked up to old age, poor upkeep and communications failures, according to an engineering firm hired by the city to investigate the matter.
A system failure in January caused low water pressures that forced schools to close, left residents and businesses temporarily without water and led to an advisory to boil water once it was restored. Low water pressure can result in the back flow of bacteria and chemicals. Another incident in February led to the same results and a public outcry.
Barr Engineering Co. found that Blaine's water woes were prompted by failures of the communication system that controls and monitors the city's water and sewage systems. The firm's report, made public at a work session of the City Council, says the computer system that runs Blaine's water and sewage treatment facilities was installed in the 1990s. The hardware was obsolete, and the documentation needed to maintain and troubleshoot the equipment was missing or incomplete, the report says.
The city hired In Control Inc. in 2015 to upgrade the computer system but that project is still underway.
"Everything ages over time and things start to deteriorate and rust," Mayor Tom Ryan said. "We know we have to keep it up."
The water and sewage systems have sites with local controls, but they are supervised by a single master-control system, according to the engineering report. When that system fails, it disables automatic control of all city wells.
The incident in February was caused when the central automation system failed to send a signal to city wells to compensate for lost water pressure. Barr found that the system lacked a method for monitoring communication failures, had obsolete control panel hardware and could not adequately receive data, according to the report.
The master system receives information from a radio network, an Ethernet cable and fiber optic lines extended to two water treatment plants. The radio network was using outdated technology that was impractical for monitoring remote sites due to slow communication speeds and weak signals, the consultants said. They also found that some of the radio antennas were too high, violating the city's Federal Communications Commission license. The Ethernet connection was unreliable and is not robust enough to run the water and wastewater systems. And the fiber optic lines, though reliable, were too costly for the city to install systemwide. Many of the records needed to maintain the system were outdated or incomplete, Barr said.
The city is working with In Control to fix the problems, Ryan said. "We're not going to get to it all in the next month," he cautioned.
Meantime, Blaine has expanded the hours for operators to check on the water system during nights and weekends.
"I don't foresee this happening again," Ryan said of the system failures. "It better not."
Gabriel Sanchez is a University of Minnesota student on assignment for the Star Tribune.
