The agency responsible for distribution of natural gas and water throughout the metropolitan area of Omaha, Nebraska, USA, cannot tolerate failure in its scada system. Not only could a failure threaten the drinking water supply of over four hundred thousand residents, it might cost the district hundreds of thousands of dollars in penalties from the natural gas pipelines on which it depends. They have installed a failsafe Citect scada system that uses built-in redundancy to protect data and to ensure continued monitoring in case of hardware downtime.
The Metropolitan Utilities District (MUD) provides water and natural gas to the metropolitan area of Omaha. Water comes from two water treatment plants while the natural gas is purchased from a portfolio of suppliers where volume purchases keep user rates low. Gas is transported via pipeline to town border stations around the metropolitan area, then it is distributed through 3476 km of mains to 172 500 customers. The district annually delivers over 113 million m3 of water on average, and 850 million m3 of natural gas.
Delivery of the natural gas from the pipelines is monitored remotely at six town border stations where the district takes possession of the gas. The flow is metered at these points, and an odorant added. In addition to these sources of gas, the district owns two propane air plants and an LNG (liquefied natural gas) operation.
A centralised control station monitors the 'take' flow, and the 'make' flow from their local gas sources, and adds them together to be compared against the projected daily requirement. As shortfalls or surpluses occur, the flow from local sources can be adjusted and pipeline allotments sold or purchased as required.
Production of water from the two water plants, which have their own separate scada systems, is monitored and adjusted for predicted demand changes. Most water is pumped a second time at one of the fourteen remotely operated pump stations. Total storage of 439 000 m3 helps to balance production and demand.
Prior to the system upgrade, all control and monitoring was done via a Unix-based system that had reached the end of its economic life. Support costs were expensive and rising because the district's system was essentially an orphan technology. Also the system redundancy was rudimentary, providing a basic version of hardware back-up in case the primary workstation failed. The application of redundancy in this way suffered from a number of deficiencies. It required manual switching in the case of failure, and refused to operate predictably while the original primary remained off line. In addition, the workstations were saturated by data flow, with the CPUs running at 95% of capacity, so that staff had to slow the rate at which I/O was polled.
The district has a separate LNG (liquefied natural gas) operation that comes on line to balance gas loads during the winter months. This peak saving operation had already selected and installed a Citect scada as the result of competitive bidding. In their request for bids on the distribution scada system, the district specified that Citect would be required on any proposed solution.
"We decided to standardise on a common software package throughout the entire operation," explains James Knight, then the Superintendent of Gas Supply & Production. "The district already has Citect in place at the LNG operation and we knew that it would work reliably. It also had the added benefit that our engineers and operators were already familiar with it."
Scope of project
The MUD scada communicates with 28 Fisher ROC RTUs (remote terminal units) in the field via 19 leased lines. Leased lines are used because they provide faster data polling than would be permitted by dial-up modems. Speed of polling is limited primarily by the internal modem in the RTU. A new driver for the RTU hardware had to be written and installed by Ci Technologies to support the engineering effort.
The leased line modems are connected to rack-mounted fallback switches via RS232; the switches are connected to the RS232 ports on both I/O servers. An Ethernet hub for the scada system connects both I/O servers, a display client for the water department operator, a display client for the gas department operator, a development/file server node, and the MUD LAN router. Three manager nodes (display only) are connected via the MUD LAN.
The I/O servers handle all I/O, alarms, trends, and reports, with the A unit being the primary and the B unit acting as the secondary node, prepared to take over seamlessly in case of primary failure.
Although the field instrumentation has only 750 I/O points, measurement and control requires nearly 7000 tags:
* 5046 variable tags.
* 275 trend tags.
* 187 digital alarm tags.
* 234 analog alarm tags.
* 940 advanced alarm tags.
Despite the relatively large number of tags in the system, the district needed to purchase only a small number of Citect licences:
* Two 1500-point I/O servers.
* Two display clients.
* Three manager nodes.
* One development node.
Because the manager nodes are floating licences, multiple users throughout the district can access the system from their office computers via the network. Five Pentium III 450 MHz PCs running Windows NT 4.0 were installed in the district control room. They are networked peer-to-peer, and connected to the district's Novell-based LAN.
There are 332 Citect accumulators configured in the project. Many water and gas flows are totalised on an hourly, daily, or monthly basis, with the accumulators updated every second. All operator commands (setpoint changes, pump start/stops, etc) are logged for an historical sequence of events so that the information can be available for troubleshooting.
A supergenie popup screen with many powerful features was created for analog I/O points. When an operator clicks on any analog I/O point on any screen, an analog popup screen appears. It displays additional details such as the minimum and maximum range of the selected point. Properties of the point can also be viewed and modified from the analog popup screen. The selected analog point can be placed OFF scan and a manual value can be entered. The alarming for the selected point can be disabled, an historical trend for the point can be displayed, and the alarm setpoints and the point's zero cutoff value can be modified. A note for the point can also be entered from the analog popup screen.
Additional alarming details are worthy of note. The alarms are configured such that water alarms appear only on the water operator's screen and the gas alarms only appear on the gas operator's screen. There are 32 alarm categories configured. All alarms, disabled alarms, and alarms for each ROC can all be individually displayed.
Custom Cicode written for this project, especially on the gas side where many complex calculations were required. "This is worth noting," commented the MUD project engineer. "It describes the powerful functions available with Cicode. For example, the projection factor data is retrieved from a table each hour, and is then used in a calculation. This table contains a value for every hour of the day, for each day of the week."
MUD management has concluded that when results are compared with their original objectives, all objectives were accomplished by installation of the new scada system.
Operational oversight
Departmental managers needed to be able to view operations from their own offices in realtime to plan for the production of water and the purchase or manufacture of natural gas in response to changing weather conditions. Failure to purchase in advance the correct amount of natural gas can cause the district to pay higher than necessary amounts for same day purchases, and incur penalties from the pipelines.
Result: Manager nodes and floating licences now allow department heads and other personnel to check operational status while seated at their desks. Natural gas is purchased or made as required without interrupting operator activities. Both water and gas managers routinely import directly from the system into spreadsheets for operational analysis.
Failsafe system with redundancy
Loss of monitoring and control capability for any long period of time would create multiple problems: major penalties from the gas pipelines which could run into hundreds of thousands of dollars, overtime for meter reading, and costly disruption of district operations. Computer downtime has been all but eliminated, and testing shows that switching between the primary and secondary I/0 server is seamless.
Data sharing
Operational data should be electronically archived, to enable the district's administrative and accounting staff to check production and purchases against data generated in the field so that they can more quickly resolve billing questions.
Analog points and accumulator totals are logged to an Access database every 15 minutes and this data is used to generate hourly, daily and monthly reports. Accounting personnel routinely check data from the archived Access databases over the office network, and use it to resolve billing questions. Citect readily transfers data into other business systems and improves the core operation.
Reduction of paperwork
Electronic archiving should eliminate the reams of printouts generated by the old system, which printed each alarm acknowledgment, event, flow reading and report.
Multiple displays
Operators needed to be able to view more than one window at the same time. This had been a useful feature of the old system. Engineers used the Citect window resizing feature and Cicode programming to devise a toolbar icon so that an operator can fit up to four windows on a single display in any arrangement they like. In addition, the water distribution operator has two 21" monitors connected to his PC, which sees the two as one single display. It lets him drag and drop a window from one monitor to the other seamlessly. This makes monitoring of alarms during other operations much more efficient.
Flexibility
The district needed to be able to replace PCs with commercial off-the-shelf Windows-based units in order to upgrade their scada incrementally and inexpensively. The district was also able to use the existing 19" rack enclosure for the I/O servers with no modification. Standardisation on Windows-based software ensures an economical path for future upgrades.
In-house modifications and documentation
To reduce technical support costs, staff needed the ability to make minor modifications to the system without requiring an outside firm. Documentation of system status was also a requirement; for example, an operator needed to be able to note that a particular pump was out of service. The system now allows technical personnel to note the status of sensors or control elements on the popups used by the operators. Operators can immediately see the reason for any discrepant input from the field, or notify maintenance when a unit appears to be out of service. An extensive Help system was created in Microsoft Word, complete with illustrations, published as an HTML document and can be accessed in the Citect scada system as an ActiveX window using Internet Explorer.
Compatibility with existing control hardware
No existing control hardware had to replaced or modified. The MUD engineering and operations staff continues to experiment with the capabilities of the new scada system.
Football, beer and water pressure
At heart, Omaha remains an All American city, most notably in its devotion to the Cornhuskers football team. When Husker games are broadcast, everybody in town watches. Yes it is true; MUD sees the results on their new scada system! Gene Siadek, who took over as water supply engineer in the late phases of this project, noticed that the system pumps seemed to kick on with unusual regularity during the games. When he pulled up historical pressure data from a game and compared it to the pressure trend from a typical day, the spikes during commercials showed up clearly. Toilets flush all around the city when there is a time out, with the biggest drop in pressure coming right around half time. "It only took five minutes," said Siadek, who found that the system was so easy to use that he could do this kind of analysis before he had even learned much about it. "Being able to get your hands on that kind of data and to import it into spreadsheets means that we can do all sorts of analysis that we could never even consider previously. "Siadek takes care to point out that he primarily uses this capability for more serious purposes, such as improving overall distribution efficiency and reducing water loss.
Xycom South Africa
(011) 462 6671
© Technews Publishing (Pty) Ltd | All Rights Reserved
printer friendly version