Q: What happens when a bank's electronic data center takes a direct lightning hit to its primary power line? Equipment toast? Data lost? Business shuts down for repairs?
A: None of the above…when the building's electrical system is protected by a proper copper grounding system. Here's what actually took place at a major Florida bank.
Suncoast Schools Federal Credit Union is the ninth largest organization of its type in the country, serving more than 215,000 members through 28 branches in 15 counties along Florida's Gulf Coast. It offers a full line of banking services and, like any modern financial institution, it conducts just about all of its business electronically. Its ATMs alone handle more than 1.5 million transactions per month.
The bank operates a state-of-the-art data center (Figure 1) near its Tampa headquarters (above). The center is housed in a single-story, 5,000-sq ft (500-sq m) cinderblock building that had once been a warehouse. Before moving in, Suncoast gutted the old structure completely and rewired it top-to-bottom for sensitive electronic equipment. Telephone equipment is housed in a separate building to diminish the risk of total system failure in the event of a severe lightning storm.
Figure 1. Suncoast Schools Federal Credit Union is the ninth largest organization of its type in the USA, serving more than 215,000 members in 15 Florida counties. The bank's Tampa annex building is closely linked with a nearby data processing center (above) and with 28 branch offices throughout the Gulf Coast region. Because it is located in Florida's notorious "lightning alley," the data center is equipped with a robust, all-copper grounding system and multiple layers of both surge protection and uninterruptible power supply devices.To make certain that its data system would be as close to 100% reliable as possible, Suncoast had its electrical system installed according to recommended power-quality practices: dedicated branch circuits for computers, six or fewer outlets per branch circuit, green wires in all circuits and oversized neutrals in computer circuits and their feeders. All telecommunications wire - and there are hundreds of miles of it in the data center and next-door headquarters building - is Category 5E or better. (The two buildings are linked by fiber optic cable.)
Suncoast management was also aware that the data center is located in the most lightning-prone city in the country and that one hit could bring their entire system down. To minimize that risk, the bank installed a low-resistance, heavy-gage copper grounding system, along with multi-layer surge protection. The cost of added protection was minimal, especially when compared with the value of the equipment being protected - not to mention the risk of lost data and possible downtime. But no matter what the cost, the investment in a little extra copper paid off handsomely for Suncoast on the day the lightning struck…
Back to Top…And no one noticed!
Earl Brendle, Suncoast's vice president, facilities, was working in one of the bank's branches when he got a call from the data center. As Brendle recalls, "The data center has a 250-kW emergency generator and transfer switch to keep us on line whenever we lose utility power for extended periods. Last year, one of the center's technicians stepped outside about half an hour after a storm passed and noticed that the generator was running, and would I please stop by to reset it.
"When I got to the data center, the generator was still powering the building. That seemed odd, because if the generator had kicked in due to an outage, it should have shut down once utility power was restored.
"Then I saw the meter, or rather, the mass of burned metal and insulation that was left of the meter socket (Figure 2a). Lightning had struck our 480-V service entrance cable. It blew the meter off the wall (we never did find it) and headed straight for the transfer switch and generator.
Figure 2a. A direct lightning strike to the data center's overhead service entrance cable destroyed the electric meter and meter socket but did no other damage, thanks to a solid-copper, low-resistance grounding system and multiple layers of surge protection.
Figure 2b. Shows the destroyed unit along with the replacement meter and socket."Fortunately, our electrical consultant had recommended that we install a good grounding system, and it was that system, coupled with the cascading transient voltage surge suppressors (TVSSs) he had installed, that saved us. Because it was well grounded, the primary TVSS shunted the lightning surge to earth before it could do any damage downstream. The TVSS survived, as did all of our equipment.
"The amazing thing is that no one in the center even knew we were hit! The lights flickered a bit, but that's not unusual around here. Thinking back on it, what was really scary was that all of our very expensive data processing equipment and a lot of important financial data could have been destroyed in an instant. Instead, the center just kept on working as if nothing had happened."
Back to TopSystem Designed by Florida Company
Figure 3. Basic elements of the Suncoast data center's electrical system. Also shown are several of the up-to-seven layers of transient voltage surge protection installed by the Orlando-based electrical consulting company. The TVSS at upper left was directly in the path of a nearby lightning strike, but it shunted the current to ground and survived the hit without damage.Figure 3 shows a partial schematic of the system PSI designed for Suncoast. Starting from the ground up, its main elements include:
- Three 20-ft x 5/8-in (6-m x 16-mm) copper-clad-steel grounding electrodes located at the service entrance (Figure 4a), between the emergency power generator and transfer switch (Figure 5a), and in a lot behind the building, respectively. Ordinarily, PSI would have preferred to install a buried, heavy-gage copper ring-ground around the building, but property constraints dictated the electrodes' location. The three electrodes PSI installed replaced a single galvanized rod left over from the building's warehouse days. The electrodes are exothermically bonded to a buried, bare 4/0 copper grounding electrode conductor leading from the main service entrance panel (Figure 4b). Other buried 4/0 copper outside the building connects the electrodes and the service entrance with the A/C panel and dual A/C units (Figure 6).
Figure 4a. The main grounding electrode conductor extends from the electrode array to the meter box (vertical cable) and to the service entrance panel seen at upper left. The 4/0 grounding cable leading to the right connects to the air-conditioning equipment subpanel.
Figure 4b. Exothermic welds connect the grounding electrode conductor to other outdoor equipment.
Figure 5a. View of the 4/0 grounding conductor for the 200-kW emergency generator. This conductor is exothermically bonded to the buried grounding electrode system described in the text.Figure 5b. Shows the corrosion-resistant bronze clamp that bonds the 4/0 grounding conductor to the generator base.
- The grounding system's resistance to earth as measured by fall-of-potential testing is 4.3 ohms. That's comfortably within the 1- to 5-ohm range considered suitable for commercial buildings under the Institute for Electrical and Electronic Engineers Standard 142, IEEE Recommended Practice for Grounding Industrial and Commercial Systems - The Green Book.
Figure 6a. The transfer switch (left), service entrance and meter location. One 20-ft x 5/8-in copper-clad-steel electrode is located between this transfer switch and the emergency generators, another at the service entrance, and a third behind the building. All are exothermically bonded to a buried 4/0 copper grounding electrode conductor.Figure 6b. shows the grounding connection at the transfer switch.
- Another 4/0 copper grounding conductor connects the ground-neutral bus in the service entrance panel to the ground bus in a 480-V distribution panel in the building's electrical room (Figure 7). The bus serves the building's two 480-V air conditioning units (one is a back-up), a 277-V lighting circuit and the primary of the building's 480Δ - 208Y/120-V transformer.
Figure 7. The 480-V distribution panel shown here feeds a 480Δ- 208Y/120-V transformer (lower right), the 277-V lighting circuit and the two air conditioning units. Connected to the 480-V bus at the right side of the panel is a 200-kA-rated TVSS that provides both line-ground and neutral-ground protection.- The transformer feeds two 120/208-V, 300-A distribution panels (Figure 8). One of the panels supplies power to the data center's main computer room, which houses the center's host computer, several servers and a variety of other sensitive equipment (Figure 9). The other panel feeds branch circuits to workstations and remaining outlets throughout the building.
Figure 8. Two 120/208-V branch circuit panels supply the data center's computer room and other locations in the building, respectively. Each panel is protected by TVSSs rated at 100-kA (blue enclosures). Fire alarm and fire call circuits are additionally protected by TVSS devices rated at 20-kA. There are as many as seven layers of surge protection in critical portions of the data center's electrical system.- Multiple uninterruptible power supplies (UPSs): Two standalone 16-kVA UPS units serve the main computer room (Figure 9). Smaller units are found at individual workstations. The primary UPS is redundant in both electronics and batteries.
Figure 9. Suncoast Schools FCU's main computer room and the reason for the extensive grounding and surge protection described in this publication. Continuous power to the host computer (a) and various servers and other equipment (b) is assured by means of the stand alone 16-kVA UPS seen at the lower right. Smaller UPS devices are located at individual workstations.
- Up to seven layers of voltage surge protection. A TVSS is installed on the transfer switch; another is located at the main 480-V bus. Additional TVSSs protect each of the two120/208-V subpanels, including circuits serving the UPS units for the main computer. (Thus, both primary and secondary sides of the transformer are surge-protected.) Each of the air conditioning disconnects is individually surge-protected, and two branch circuits serving fire alarm and fire call circuits, respectively, are protected by additional TVSS units at the distribution panel. Surge strips are plugged into wall outlets where needed in the building. Finally, small units located at workstations contain surge suppression circuitry to protect individual computer terminals.
TVSS devices are scaled to match or exceed the size of the upstream risk. The large units on the main switch are rated at 200-kA; the smaller suppressors on the subpanels at 100-kA; the units for fire alarms at 20-kA; and so forth down the line. The main TVSSs provide both line-neutral and line-ground protection on all phases. The 100-kAand 20-kAunits are also multimodal. It is important to note that all surge suppressors are securely connected to the building's equipment grounding circuits.
That's essential, as PSI's president, John West, points out. "High-quality surge suppressors are very effective protection devices," he says. "But even the most expensive TVSS you can buy is absolutely useless unless it sees a high-quality, low-resistance ground. And by high quality, I don't just mean meeting Code requirements for low impedance between outlets and the ground-to-neutral bus. Our specifications call for less than five ohms total resistance to earth, especially where there's sensitive equipment involved. We won't accept a job unless our customer agrees to that."
In addition to low-resistance grounding and heavy-gage copper conductors, PSI's specifications require that all bonds be made either with exothermic welds or copper-base (brass or bronze) mechanical clamps, as appropriate to the location. Copper and copper alloys, such as brass and bronze, resist corrosion, so joint resistance remains low over long periods of time. For added protection, PSI grinds all mating joint surfaces to bare metal and coats them with conductive grease before clamping. Mechanical bonds made with copper connectors also remain tight indefinitely and don't have to be re-torqued periodically to avoid loss of continuity.
"We also prefer copper-clad electrodes except in locations where the soil is very acidic," says PSI's West. "We never use galvanized steel electrodes, and when it comes to conductors, we never use anything but copper. It's one reason why we haven't had any of our clients lose a single piece of equipment during the ten years we've been in business."
Earl Brendle knows that the Suncoast Schools FCU is one of those satisfied clients. "If someone asks me how they should protect their equipment, I tell them to do what we did: install a good grounding system and multiple layers of surge protection. Then, if their facility gets hit by lightning, they'll probably be as fortunate as we were, and no one will even know it happened."
The Principals
Earl Brendle, vice president, facilities for Suncoast Federal Credit Union, is responsible for new construction as well as all existing Suncoast installations in 15 Florida counties. It was Mr. Brendle who discovered that Suncoast's data center had taken a direct lightning strike, an event that went completely unnoticed inside the building, thanks to extensive, copper-based grounding and surge-protection systems. Suncoast can be reached at www.suncoastfcu.org.
John West, Sr. is president of Power Systems & Innovations, Inc. (PSI), Orlando, Florida. Founded in 1992, PSI provides products, services and consulting related to power quality, power protection, surge protection, grounding, lightning protection and other anomalies that can damage equipment and/or cause data loss. PSI also provides on-site consulting services. In addition, the company designs, installs and services a broad range of power conditioning equipment, including transient voltage surge suppressors for all domestic and commercial electrical systems. Further information about PSI can be found at www.psihq.com or by calling (800) 260-2259.