Obsolete SCADA systems are a common industry problem. Parts aren’t supported, you’ve used your last spare, and you’re left to search eBay to get your system running again. Or, you have so many alerts that technicians have stopped paying attention to them at all. You want to modernize your system, but your funds are tapped out addressing quick fixes. How do you eliminate SCADA obsolescence for good?
It Starts With Technical Debt
What is Technical Debt? Technical debt is like any other kind of debt. It usually builds up while you’re not paying attention, more focused on the short term wins than the long term plan. In this case, technical debt leads to SCADA obsolescence.
Here’s how this usually happens: A facility completes a major system overhaul, with all of the newest, most leading-edge technologies. The system works great, but maybe the documentation provided at the project’s end wasn’t comprehensive or complete. Over time, changes or upgrades have to be made. An alarm goes off one night, and someone comes in to fix it and get the system back online as soon as possible. In the haste to get things running, they complete the fix quick and dirty – with no documentation. Later, another change needs to be made. Because the first fix wasn’t documented, it takes longer to make the change. And, maybe this next change needs to be completed in an unorthodox way in order to accommodate the first, shoddy fix. These “one-off”, undocumented modifications compound for years until the system becomes more work to use and maintain than it should. Replacement becomes the best option, but now, because the system was not properly maintained, it takes several extra weeks of work to untangle the spaghetti of code and reverse engineer the system. The project now feels monumentally harder and more expensive – which often leads companies to put it off for longer.
Getting out of technical debt requires discipline, but it results in a system that functions and takes a weight off your chest.
Change Management Plan
The first move you will want to make is to stop the bleeding. By implementing a clear change management program – and adhering to it – you can prevent messy changes and complications in the future.
A change management plan includes a clear process for making and documenting changes. It also would include creating a well-thought-out plan for access. Only those who must have read/write access to particular parts of a system should have access. Blanket access for all to everything runs the risk of inexpert technicians making accidental changes with unforeseen implications. By limiting access, changes are well-managed by those that are very familiar with the system and the process for making changes.
A regular preventative maintenance program is important to get ahead of problems and prevent the need for late-night emergency fixes.
Typically, routine maintenance should be performed at least every 4-6 months. Our preventative maintenance program, for example, includes
- Software installations such as updates and patches
- Checking for operating system problems such as reaching storage limits and looking into the event logs for errors
- Hardware inspection, such as checking for warning lights on the server and PLC speed
- Electrical panel inspection
This type of maintenance can prevent huge headaches. For example, if you wait too long to replace the seals on the electrical panel doors, water could get in and fry the entire panel! A few dollars spent on a new seal would prevent an extended outage and a serious system overhaul.
Long Term Hardware/Software Plan
Putting a long-term hardware and software plan together allows you to focus on the future and turn maintenance and upgrade costs into planned, capital expenses rather than emergency operating expenses.
On its own, a SCADA system has a 10-20 year span before the actual platform will be obsolete. Physical servers usually have a 5-7 year lifespan. Operating systems and applications may only be supported for 5-10 years and will need to be upgraded over time. None of this information is a secret; it is published on the manufacturer’s website. By researching this information, you can put together a plan. For example, in 5 years, we will replace this hardware and in 10 years, we will upgrade this element. That way, you can budget the costs. Aside from hardware and software, you can plan ahead to get a decent labor estimate. Then with a budget for the equipment and labor costs, you can make sure money is available. If you don’t, you will slide further into technical debt.
By digging your system out of technical debt, you will be well on your way to avoiding SCADA obsolescence – forever. You will have a clear plan with regular updates so that you’re never falling far behind. These systems create far fewer unexpected problems and far fewer headaches for the staff.