Consolidating Cybersecurity Systems at Cheyenne Mountain SFS

Asset Management

By Tim Belz, M.SAME, Matthew Kinney, Jason Cook, P.E., PMP, M.SAME, Scott Ytzen, M.SAME, W. Jeff Robertson, CISSP, M.SAME, and William Hoffman

In a first for the Department of the Air Force, Cheyenne Mountain SFS recently integrated dozens of supervisory control and data acquisition and facility-related control systems, as well as related hardware components, into a single enterprise network architecture.
A cybersecurity project carried out at Cheyenne Mountain SFS, Colo., marked the first time a risk management framework watermarked “High” system had been migrated to a unified network in the Department of the Air Force. DOD photo by Airman 1st Class Alexus Wilcox.

Located in central Colorado, Cheyenne Mountain SFS has been fully operational for 57 years, playing a pivotal role in national defense and most recently serving as a training center and alternate command center for U.S. Northern Command and North American Aerospace Defense Command, based at Peterson SFB, Colo.

To enhance mission resilience, the installation recently pursued an upgrade to its facility-related control systems (FRCS), driven by the deterioration of equipment, an increased frequency in attempted cyberattacks, and subsequent updates to code requirements. The base worked with the U.S. Air Force to identify a team for the design and installation of a cyber-hardened, enterprise unified network.

The new unified network at Cheyenne combines the installation’s supervisory control and data acquisition and facility-related control systems into a single enterprise network architecture. Photo courtesy Tetra Tech.

Tribal One, the general contractor, partnered with Tetra Tech as the designer of record, and Frontline Cyber Solutions Corp., for a commercially available off-the-shelf network on a series of task orders to create the unified network. This program converged FRCS and supervisory control and data acquisition (SCADA) into a single enterprise network architecture. It marked the first time that a risk management framework watermarked “High” system had been migrated to a unified network, cyber-hardened platform in the Air Force.

Top to Bottom Coverage

The extensive program at Cheyenne Mountain encompassed the integration of 34 facilities, lift stations, programmable logic controllers/direct digital controls for HVAC and power generation/distribution, SCADA operator workstations, CCTV cameras, access control nodes, and telecommunications enclosures. It also covered over 5-acres of outside plant cable infrastructure, an intra-building cable backbone, and more than 10,000-ft of horizontal distribution communication cabling.

By utilizing current and relevant cyber security techniques and standards throughout the unified network, this system is the first of its kind across the Department of the Air Force.

Over the course of three task orders, a new fiber backbone was installed along with the related panels, racks, and hardware. This was designed to accommodate future growth and provide cyber resiliency across the installation. All the systems were tied into security badging, cameras, and other systems. Finally, the project connected, tested, and replaced all the cabling throughout the facility before cutting over all the different devices or controls.

Integral Functionality. FRCS are vital to the function of a building. Their devices need to be tracked like any other building system. Watcher, the commercial security sensor utilized in this project, continuously monitors the network for anomalies, compiles asset inventories, and performs asset discovery. This allows personnel to ensure that all pieces of the network are accounted for and documented while also being able to detect anything new. The tool alerts users to any unauthorized assets on the network so that they can be dealt with immediately.

The new system at Cheyenne Mountain also can do predictive maintenance analytics, which helps identify performance issues or failing equipment before it creates a problem. Users can now get ahead of unexpected outages and schedule the downtime to replace the equipment so it does not negatively impact the base’s mission.

As part of the project at Cheyenne Mountain, a new fiber backbone was installed and tied into existing security systems to increase cyber resiliency and enable future growth. Photo courtesy tetra tech.
Planning for Success

The SCADA/FRCS project required advanced planning because of the facility’s underground, constrained location—along with the need to uphold the integrity of the installation. With the age of the facility, the team sometimes would run into old conduit that was left in place or found it difficult to install new pathways. Lead time on equipment proved to be a challenge due to working in this secure environment, but effective communication prevented the majority of issues.

The indoor, underground working conditions created numerous design considerations, such as the natural water table. To overcome this challenge, the project team used watertight cabinets to prevent water leakage from reaching devices and connections in the network. Some areas even necessitated a dehumidifier to avoid degradation of equipment.

Operational Importance. Due to the critical nature of the missions at Cheyenne Mountain, the new systems (especially the FRCS) needed to remain operational throughout. To avoid interruptions, the team worked to create a new system alongside the existing platform, with the intent of doing it very rapidly. The goal was to have a seamless transition, so that the operations could be consistent across the legacy system before everything was transferred.

Once the new FRCS was in place, the team had to ensure everything needed to protect, maintain, and monitor alerts was in place and operational before the switchover.

Unified Environment

A cyber-hardened infrastructure like what was delivered for Cheyenne Mountain can be tailored to each end user, creating an environment that is secure, resistant, and resilient. It manages those systems and monitors the traffic to allow for identification of any threats. In addition, the infrastructure has the ability to isolate threats. A compromised device can be enclaved to prevent an attack from spreading.

Over the course of three task orders, a new fiber backbone was installed along with the related panels, racks, and hardware. This was designed to accommodate future growth and provide cyber resiliency across the installation.

This unified network provides a converged architecture. What that means is it can host several different automation systems, FRCS, access control, closed circuit television, HVAC controls, and automated electric metering. A single, integrated infrastructure can host all those different control systems in a cyber-secure environment.

By utilizing current and relevant cyber security techniques and standards throughout the unified network, this system is the first of its kind across the Department of the Air Force. This backbone could be used for any FRCS to integrate a locking system with an electrical switching system, a battery monitoring system for uninterruptable power supplies, and HVAC controls. Even though these systems are independent and sometimes proprietary, they can all operate cohesively over the same backbone.

With the unified network’s machine learning element, installations can actively monitor system vulnerabilities so it is protected not just from outside dangers, but from insider threats and sabotage. That same machine learning can also provide condition-based maintenance, alerting users when equipment is operating outside its normal parameters, to allow for quick repairs.

Repeatable Approach

As the industry keeps pushing toward cyber hardness and resiliency for all platform systems, military installations need to consider the best way to upgrade their networks. Industrial controls are a key element to keep any base operational, and the system implemented for Cheyenne Mountain SFS has been designed from the ground up for the most secure and critical missions.

Cybersecurity does not always require a system to be completely re-engineered for it to be truly effective. The Cheyenne project demonstrates that in operational technology and FRCS, security can be added without affecting the performance of the underlying system. The new system utilizes network sensors throughout the infrastructure to look for abnormalities in the security system—a solution that can be bolted on to any operational technology or FRCS out there today.

While the work at Cheyenne Mountain may seem like an extreme case, it is scalable and implementable elsewhere. The architectural integration of its system can be applied to any other installation to protect the industrial control systems or operational technology infrastructure of those locations.

The cyber hardening of operational technology and FRCS is a nationwide problem. It is critical that installations recognize the importance of securing their infrastructure against cyberattacks.

Tim Belz, M.SAME, is Project Manager and Matthew Kinney is Low-Voltage Foreman, Tribal One. They can be reached at timbeltz@tribal.one; and matthewkinney@tribal.one.

Jason Cook, P.E., PMP, M.SAME, is Senior Project Manager; Scott Ytzen, M.SAME, is Project Manager; and W. Jeff Robertson, CISSP, M.SAME, is Senior Manager/Engineer, Operational Technology & Cybersecurity, Tetra Tech. They can be reached at jason.cook1@tetratech.com; scott.ytzen@tetratech.com; and jeff.robertson@tetratech.com.

William Hoffman is Chief Security Officer, Frontline Cyber Solutions Corp.; whoffman@frontlinecyber.us.

Article published in The Military Engineer, September-October 2024

More News from TME