Cyber attacks from foreign adversaries have accelerated significantly on the Department of Defense (DoD) networks and systems, creating the need to identify threats and vulnerabilities at untenable speed. For incident responders, time is the most valuable metric. Navy Cyber Defense Operations Command (NCDOC), responsible for coordinating, monitoring, and overseeing the defense of all Navy computer networks and systems, knows all too well how time impacts the execution of defensive cyber operations (DCO).
In the Information Age, many leaders look to software, networks, and data to deliver a competitive advantage. However, we cannot rely solely on technology but rather more on how we leverage technology, and more importantly, cultivate the talent that can build the capabilities and the teams to defend DoD’s networks and systems.
NOTE: This is not an endorsement of any specific technology. We identify an industry partner due to their critical role in assisting us in building the capability that made a significant impact on how we operate and execute the Navy’s DCO mission.
Opportunity for Change
In 2016, two named operations occurred in which NCDOC responded to foreign adversaries who compromised computer networks at the Navy’s Research Laboratory and attacked the Joint Staff network with attempts to compromise the Navy’s portion of the DoD network. We were also supporting efforts in the Petya/NotPetya malware and WannaCry ransomware events. NCDOC was on speed dial for many General and Flag officers for daily situational awareness briefings, creating a stressful operational tempo as analysts and leaders struggled to keep up with the demands of accurate situational reporting (SITREP).
It was clear that the current culture of compliance and complacency needed to shift. Yet, we faced challenges of resistance and complex processes and policies.
Over the years, many tools had been purchased or developed internally by analysts who have since departed the command. Network visibility was gleaned primarily from what we refer to as the Global Sensor Grid that includes Intrusion Detection System (IDS) and Intrusion Prevention System (IPS) managers, and Host-Based Security System (HBSS). Our Security Information and Event Management (SIEM) tools included Splunk, Atlassian, Navy Cert Database, Tanium, and Intelligence Feeds. Despite all these tools, network visibility was still poor, and much of the Navy’s attack surface was still left unmonitored. We did not have the capabilities to aggregate, automate, and accelerate reports. Analysts were spending the majority of their time learning about the various tools and the disparate workflows, not to mention the staff churn to verify details and duplication of efforts. These disconnected workflows and complex processes resulted in important details often left out of analysts’ reports, and an associated low confidence in briefing an accurate SITREP to senior leadership.
Leadership needed a common, shareable, and accurate tactical picture of operations and intelligence during these briefings. Also, analysts needed 1) a solution that allowed flexible queries against multiple data sources, 2) the ability to search all data sets, and 3) the ability to process correlations and provide ad hoc predictive analysis. The data platform needed to aggregate and automate information flows from various tools to accelerate accurate reporting.
Fortunately, Splunk was already being used for data collection. A few analysts saw the platform could potentially provide the capabilities needed; however, they faced resistance as they proposed changes over the years. The culture of compliance and complacency such as “stay in your lane,” “follow SOP checklists,” and the mentality of “that’s how we’ve always done it” was widespread.
Most capability development is outsourced to either research and development labs, or the private sector. Most often, leaders in operational units are not allowed to take risks associated with developing capabilities or innovating in a production environment. Years of advocating for a dev-test environment so that cyber defenders could develop and test new capabilities in-house had been a lost battle as funding was diverted to higher priorities. And very little, if any, new capability development on operational units occurs. Despite this, NCDOC is unique in what we have been able to do as an operational unit.
Building Capabilities and Cross-Functional Teams
During this time, there was also a slight cultural shift occurring at NCDOC. A dedicated cadre of leaders focused on shifting the culture of compliance and complacency to delivering valuable and meaningful outcomes. The Commanding Officer appointed an Innovation Officer (IO), a mid-level civilian leader who was familiar with the needs of the Operations (N3) department and tensions with the Communications and Information Systems (N6) department. The IO’s role was to identify and enable innovators, break down barriers, and provide the time, trust, and top-cover, as needed. He understood that building a cross-functional team was necessary and time-consuming. Considering the current operational tensions, it was especially difficult for network engineers and system administrators to trust Enlisted Information Warfare Specialist (EIWS) Sailors on production networks.
Cryptologic Technician Networks Second Class Dillon Saylor and Cryptologic Technician Networks Second Class Aaron Kohler knew we were not using our current SIEM technology to its potential. They purposefully set out to tackle this issue. The IO championed their work, entrusting them to build what leaders had been looking for. The IO and Sailors worked tirelessly to break down the barriers between departments and the military-civilian divide. After several months, the engineers gave the Sailors the level of access and permissions needed to develop the Navy’s first DCO solution on the Enterprise network.
Saylor and Kohler possessed some digital hard skills such as basic coding and programming; however, they were not that advanced compared to other Sailors of this rate with similar backgrounds and training. They also had a desire, drive, and determination – the desire to do more than their assigned jobs, the drive to contribute to the mission in a significant way, and the determination to build their digital skills leveraging the free hours of training provided from Splunk. As their digital skills grew, they also developed better critical thinking skills, including how to ask the right questions and use inductive reasoning to solve potential problems.
By mid-2017, the cross-functional team was working synchronously and resulted in Saylor’s first “NCDOC Watchfloor” leadership dashboard. Due to Splunk’s built-in alerting system, it allowed the engineers to become more confident in their ability to fix the affected systems immediately should the Sailors’ cause a network or system outage. This initial consolidation effort of multiple tools into a common operating picture or “a single pane of glass” was complete with automated alerting mechanisms based on rules set within Splunk. These rules are known as “playbooks.” This platform was the first visualization to provide network visibility from all sensors and tools for analysts and leadership in one intuitive user interface.
Delivering Value and Meaningful Outcomes
The SIEM team, which by now included another EIWS Sailor and a civilian, briefed executive leadership on the Splunk capability they built with a full demonstration of the dashboard. With support from leadership, the SIEM team continued to evolve the workflow automation and expedite network management detections with advanced correlation.
In 2018, the “Operational Data Interface for NCDOC” (ODIN) dashboard replaced the NCDOC Watchfloor dashboard. ODIN’s additional features included automation of up to 85 percent of Cyber Event Reports (CER), and reports included detailed investigation information compiled via Splunk’s continuous automated queries. CER automation reduced the time analysts spent on creating reports by 50 percent, increased the number of reports, and the quality of reporting increased significantly. The SIEM team used the saved time to develop a Splunk Bootcamp to train other analysts.
Current Status
Today, we have full automation of particular CERs that bypass the Network Forensics pipeline and go directly to Incident Handling, more than 400 queries automated to set alerts for the watchfloor, and all are visible from the ODIN. This capability has increased the efficiency and effectiveness of our daily operations and our mission.
While these are important metrics and progress continues, we also look to the results often overlooked and hard to measure. Trust between SIEM analysts and civilian engineers, a cross-functional team that works well together, are not easily measurable. This teamwork is now embedded in their roles and responsibilities. Additionally, the Splunk Bootcamps are part of the training pipeline for all analysts.
This story is just one example of how leaders, even those most junior, can shift the culture and deliver significant value to the mission that lasts long after they depart the command. To build the capabilities DoD needs to defend networks, knowing what technology to use is only part of the solution. We will gain a competitive advantage against our adversaries when leaders at all levels take calculated risks and cultivate the talent of our junior personnel. Many of them have the desire, drive, and determination to build the capabilities and cross-functional teams needed to deliver meaningful outcomes in support of cyber warfare.
The mission of NCDOC is to execute defensive cyberspace operations and enable global power projection through proactive network defense and reports operationally to U.S. Fleet Cyber Command/U.S. 10th Fleet.
U.S. Fleet Cyber Command serves as the Navy component command to U.S. Strategic Command and U.S. Cyber Command, and the Navy’s Service Cryptologic Component commander under the National Security Agency/Central Security Service. Fleet Cyber Command also reports directly to the Chief of Naval Operations as an Echelon II command.
U.S. 10th Fleet is the operational arm of Fleet Cyber Command and executes its mission through a task force structure similar to other warfare commanders. In this role, C10F provides operational direction through its Maritime Operations Center located at Fort George Meade Md., executing command and control over assigned forces in support of Navy or joint missions in cyber/networks, electronic warfare, cryptologic/signals intelligence and space.
For more news and information from NCDOC/CTF1020 visit www.navycyberwarriors.com and follow us on Facebook @NavyCyberWarriors and Twitter @NCDOC_PAO.