History of Water/Wastewater SCADA Expenses

Since we suddenly need to spend over $40,000 for yet another SCADA (Supervisory Control and Data Acquisition) computer system for the water/wastewater system, I did some research to find out just what the history of expenses are for this.

Here is what I found going back to 2013 (as far as I was able to search):

August 9th, 2013: City places an ad in the Dispatch asking for proposals and qualified bids for a new SCADA unit. Yes, they put this out for bid back then. The public post was again made in the Dispatch on August 13th, 2013.

Records are incomplete and sketchysee page 105 as my proof. No numbers were given but “Phil Andreas, Public Works Director, explained that proposals were sent out for the installation of a SCADA system for the Electric Department and for an upgrade of the W/WW SCADA system. The proposal costs came in considerably higher than expected. City staff is asking that Council reject the W/WW bids received” [from page 14]

The closet thing to “W/WW SCADA” I can find is at the end of 2013 [page 67] – awarding a $18,252 contract to AWD for “lift station monitoring” and “SCADA additions”. They talk of integrating the equipment into the City’s “existing Wonderware SCADA system”.

July 14, 2017: Council unanimously approves the installation of Supervisory Control and Data Acquisition equipment at the sewage lift station on FM 580 East, just past the Key Avenue intersection. The cost of labor, materials, programming and installation is $25,250 through Trac-N-Trol, the city’s SCADA system contractor. [see page 127 for invoice and explanation]

March 30, 2018: According to the Dispatch: “Council voted 6-0 to replace Supervisory Control and Data Acquisition system equipment at four lift stations”

“In 2013, the city approved a bid for SCADA equipment that did not exactly match the existing SCADA system for the sewage treatment plant and the water system, Water/Wastewater Operations Manager Van Sims said. He said the two systems have not been communicating properly, and he said the only way to fix the problem is to ensure all units are the same brand. The council awarded the equipment replacement project to Trac-N-Trol for $15,900.”

According to the invoice [see page 151], they had to “remove existing Siemens PLC” and “install Allen Bradley PLC” instead. So, rip out all the old stuff that was previously bought and put in new stuff.

[Whoops! Sounds like a pretty big fuckup. Did anyone’s head roll for this? Did the contractor make things good for free? Did the City get any money back for the mistake? I’m guessing not]

August 2019: A mere 17 months after everything was now supposedly running smoothly thanks to $15,900 spent in March of 2018, a ransomware attack was allowed by Monica Wright’s IT Department which rendered the W/WW SCADA unit “inoperable”.

The City (Finley) informed us that the SCADA was “approaching end of life” anyways. Yes, you read that properly: the brand new SCADA that was just installed in April 2018 was now “end of life” in August of 2019.

Ok…if you say so, Finley.

The City then spent ANOTHER $15,990 in September of 2019 to install an ENTRIELY NEW SCADA under the premise that the previous one was “old” and only had Windows 7, when actually they needed Windows 10.

Fun fact: Windows 10 was released way back in 2015 – THREE YEARS before the March 2018 SCADA installation. So it begs the question: why didn’t the idiots at TraC-n-trol install a Windows-10-supported SCADA system in March 2018??

October 2021: Barely two years after the City paid $15,990 (a second time) to upgrade everything to Windows 10-supported SCADA, they are back asking for $40,530 for a new system. This is now supposedly an “upgrade” for an entirely new system.

How long will this one last before disaster strikes?

Don’t get me wrong – water treatment is right up there with law enforcement as an important and necessary function of local government. I’d rather they spend a million dollars on a proper water treatment system than spend $1 on stupid shit like a “business” park or water tanks for a community garden that already has water.

But heads should have rolled big time back in 2019 when a BRAND NEW SYSTEM was wrecked by a ransomware attack. Instead, Finley and Talbert brushed it under the rug as a Windows 10 upgrade and then THANKED the IT Department for their hard work!!!