10 Steps to Create Stronger, Cost Effective Substation Physical Security

Enhanced physical security can be a smart business decision if threats are identified before physical security requirements are specified. This article describes how nimble electric utilities can cost effectively eliminate the possibility that motivated threat actors can compromise the electric energy grid.

Security Professionals’ 10 Essential Steps

A decade after the April 16, 2013, Metcalf substation attack, electric utilities have yet to develop nationwide physical security protocols. This concern can be resolved by following the example of the nuclear power industry.

Nuclear power physical security teams include system engineers who define the threat; design engineers who specify features that deter, delay and detect intruders; and security professionals who respond to intruders. As a team, they discuss threats, weak points, response time, etc. and then implement systems to deter, detect, and delay intruders. Their approach to physical security can be explained as a ten step process that begins with identifying design base threats:

1. Define the threat and understand the vulnerability
2. Identify weak points
3. Reconnoiter facilities
4. Obstruct line of sight
5. Harden components
6. Separate commingled systems
7. Surveil the installation
8. Develop recovery plans
9. Perform intrusion drills
10. Deter, detect, deny, delay, defend

When implemented by electric utilities, these ten steps would greatly improve the physical security of the electric energy grid. To be most effective, physical security professionals who implement the ten steps must also engage individuals who design and operate the electric energy grid.

Basic Security is Lacking at Today’s Substations

Very few of the 75,000 substations in the United States have been equipped with basic physical security protection features, such as keycards with personal identification at controlled entry and exit points, equipment operating handles that are stored in locked areas, and surveillance systems that detect intruders. More than 60,000 neighborhood substations have no industry approved requirements for any type of physical security. Evidence of the need for enhanced physical security is the ongoing theft of copper wire, a problem that nearly every electric utility has experienced.

“Threat Actors” Seek to Damage Substation Components

In the case of substation security, the term “threat actors” refers to saboteurs who wish to damage or destroy substation components. Electric utilities must maintain an awareness that with today’s physical security at substations, a single, motivated threat actor could easily create a short term wide area blackout across multiple states, or a weeklong, localized neighborhood blackout.

Threat actors may recognize substation hazards and attempt to create damage with small arms fire from outside the fence. Ideologically driven threat actors may not be concerned about their personal safety and will enter a substation to maximize their efforts. Some threat actors may be willing to harm electric utility employees and first responders. Electric utilities must prepare for every type of potential threat actor.

Understanding the 10 Steps to Improved Physical Security

The first three items in the list are an absolute necessity. The remaining seven are equally important, though offer room for flexibility. In my experience, most items in this list have been glossed over by electric utilities, as the main threat on the electric energy grid is perceived to be theft rather than vandalism or sabotage. Let’s take a minute to understand each item.

1. Define the Threat and Understand the Vulnerability

Defining threats is the first step in establishing physical security protocols. This step must be completed by individuals who monitor and analyze the electric energy grid on a continuing basis. Reducing vulnerability by hardening components can be more effective than enhancing physical security.

At regional substations, a threat actor can create a short term, multi-state, wide area blackout by creating a single, three phase fault on the grid. Threat actors may use explosives or drones to attack substation components; those with insider knowledge may operate ground switches. At regional substations, the threat that needs to be evaluated is:

Threat actors compromise components in a substation and then place a three phase fault on the electric grid.

This threat is more challenging than faults that occur when lightning strikes a line or when insulators flash over.

At neighborhood substations, a threat actor can create a weeklong neighborhood blackout by using small arms fire to damage several transformers or circuit breakers.

The difference between threats at regional and neighborhood substations is how the electric energy grid responds. On the regional level, faults should be cleared in less than 100 milliseconds and consequential effects should be mitigated in a few hours. If faults at regional substations are theorized to persist for more than 250 milliseconds, enhanced physical security is essential. At neighborhood substations, damaged transformers can fail catastrophically and take days to replace.

2. Identify Weak Points

After threats are defined, identifying weak points is relatively easy. This step must be completed by individuals who select components, design substations, and understand physical separation requirements for redundant facilities.

Obvious weak points are transformers and circuit breakers. Less obvious weak points are control houses, battery rooms, DC systems, ground switches, and disconnect switches. Motivated threat actors will use internet research to locate critical facilities, to obtain copies of technical reports developed by NERC, and to learn how others attempted to compromise the electric energy grid in the past.

Electric utility professionals would benefit from thinking like threat actors when seeking to identify weak points. They must identify how a threat actor would gain access to each weak point, and create methods to deny access to those points. Building walls and bullet resistant fences around substations provides minimal protection for weak points. Additional protection must be implemented.

3. Reconnoiter Facilities

Security professionals, accompanied by electric utility engineers, must perform substation reconnaissance by walking around the outside of substations to observe weak points. Walk arounds should be performed at the perimeter fence or wall, as well as at a distance of 100 feet from the perimeter. Due to their different backgrounds, engineers and security professionals may notice different weak points. Engineers should present security professionals with a list of essential weak points based on the functionality of each component. Both professionals should point out each observable weak point as they walk around the perimeter.

4. Obstruct Line of Sight

Electric utilities should identify low cost alternatives that can obstruct the line of sight to weak points in substations. Many electric utilities have retrofitted substations with animal guards to prevent outages caused by birds, squirrels, snakes, and racoons. A somewhat similar approach can be used to obstruct the line of sight of potential threat actors.

5. Harden Components

Transformers and other components can be hardened to eliminate threats. In neighborhood substations, transformers can be equipped with motor operated valves that automatically close on loss of oil when a threat actor damages coolers with small arms fire. Ground switches can be secured by removing operating handles and placing them in locked cabinets. The list of methods to harden components is extensive. The key is to recognize the need.

6. Separate Commingled Systems

Separating commingled systems in existing substations is a herculean task. A better approach is to require that redundant systems in all new substations be physically separated. In regional substations, primary and secondary protective relay systems must be located in separate buildings and supplied by separate DC systems. Cable and raceways must be constructed so that redundant systems are not compromised if cables in a single trench are damaged.

7. Surveil the Installation

Roving patrols should survey regional substations every day and walk the perimeter of regional substations once each week. Roving patrols should survey neighborhood substations once each month to identify suspicious activities, new weak points, and other abnormalities. Creating a known security presence will dissuade threat actors, who recognize that guards are watching for threats.

8. Develop Recovery Plans

The amount of physical security must be commensurate with the estimated recovery time and expenses. If all loads can be transferred to a nearby neighborhood substation in two hours, additional physical security is hard to justify. If, on the other hand, recovery will take days or weeks, additional physical security can be justified. Developing a thorough recovery plan facilitates decision making for physical security enhancements.

9. Perform Intrusion Drills

Intrusion drills test and verify physical security improvements. Drills must answer the following questions: Can a motivated threat actor gain access to a substation in less than one minute? Will the threat actor be detected? Can the threat actor compromise a substation in less than 10 minutes?

Simulated intrusions should be performed as if the intrusion were planned and implemented by a motivated threat actor with reasonable background knowledge. For example, it should be assumed that the intruder has used the internet to identify critical components, survey the substation, and develop an action plan. In addition, simulations should use only tools available in a hardware store.

10. Deter, Detect, Deny, Delay, Defend

This traditional approach must be enhanced to deal with motivated threat actors. Making the site appear well-protected to discourage threat actors must be balanced with the understanding that motivated threat actors can acquire critical knowledge via the internet. Identifying suspicious activity using motion sensors, CCTV, video analytics, and intrusion detection systems must be balanced with response time. Neutralizing threats is difficult when threat actors can complete their mission in a few minutes.

Preventing access with physical barriers and improved, hardened components is essential. Slowing down threat actors minimizes the number of components that can be damaged before first responders arrive.

Prudent Physical Security Improvements Minimize Costs

At new regional substations, enhanced physical security can reduce costs by changing the focus from installing perimeter walls and bullet resistant fences ($10 million per substation) to installing more cost effective and physically secure components, such as:

• Purchasing transformers with plug connectors rather than bushings and motor operated valves that automatically close on loss of oil ($50,000 per transformer).
• Building two control houses ($200,000 for the second building).
• Installing two batteries ($100,000 for the second battery).

The cost of a new regional substation with hardened components and enhanced physical security can be $5 million less than a traditional regional substation with today’s physical security.

At new neighborhood substations, enhanced physical security should reduce costs by changing the focus from installing perimeter walls and bullet resistant fences ($3 million per substation) to purchasing transformers with plug connectors rather than bushings and motor operated valves that automatically close on loss of oil ($25,000 per transformer). The cost of a new neighborhood substation with hardened components and enhanced physical security can be $2 million less than a traditional neighborhood substation with today’s physical security.

At both existing regional and neighborhood substations, when periodic maintenance is performed, micarta “doghouses” could be placed around transformer bushings, motor operated valves could be inserted in transformer cooler flow paths, and lids on cable trenches could be secured so that lids are not easily removed. The cost impact, about $200,000 at each regional substation and $100,000 at each neighborhood substation, would occur over five to ten years.

Anticipate New Threat Types

Physical security teams need to anticipate threats. Threat actors can be motivated by personal issues or societal goals. Historically, threat actors were copper thieves. Recently, threat actors damaged neighborhood substations using small arms fire. Will future threat actors follow the lead of Luigi Mangione and perform due diligence before they act? We may not know the motivation, but we can anticipate threats, begin reducing the visibility of substations, and begin hardening substation components. The time for change is now.

Interested in having your system’s physical security strategies assessed and enhanced? Prescient offers Power System Physical Security Analysis as one of our many services. Our expert staff will evaluate your current physical security practices and provide a report outlining the level of risk at your substations. We also provide strategies to implement the 10 steps outlined above. Contact us for a free, in depth whitepaper or consultation.