Two students at the Naval Postgraduate School (NPS), U.S. Navy Lts. JaMerra Turner and Joanna Cruz, are exploring the application of emerging technologies in augmented reality and how Sailors can use them to advance the decision cycle. As the thesis for their master’s degrees at NPS, titled “Supporting Mission Planning with a Persistent Augmented Environment,” Cruz and Turner’s research provides insights into the feasibility and effectiveness of this novel form of data representation and user interaction, and its capability to support faster and improved situational awareness and decision-making in a complex operational technology environment.
Turner, now a graduate of the NPS Cyber Systems and Operations program with a master’s in computer science, was looking for a partner for her thesis. While combing through the research of other students, she found Cruz, a student in the computer science program through the Modeling Virtual Environments and Simulations (MOVES) Institute, who wanted to focus her research on augmented reality.
“Joanna had this brilliant idea of combining her expertise in virtual environments with my proficiency as an information professional officer,” said Turner. “Using my knowledge of communications networks, and her ability to create augmented realities, we could develop something that would drastically improve the decision-making capabilities of the senior decision makers in the field.”
A ‘persistent augmented environment’ is a concept the students’ thesis advisor Dr. Amela Sadagic has been working on for some time, and consists of two major components. ‘Persistent’ means that a diverse set of data are being collected and visualized in real-time, and the data sets are updated and corrected as quickly as possible, ensuring the most accurate information is available to operators who need it to support their decision making. An ‘augmented environment’ refers to superimposed, computer-generated visual elements over visual information that comes from the real-world environment. Cruz described it as “Pokémon Go on ships” or “tactical Pokémon Go.”
With their thesis, the two students wanted to use simulated data of communications systems and cyber network operations, reporting, and resource management decision-making to demonstrate their product. Using augmented reality displays allowed the operators to see each other and discern key non-verbal cues typically used in group discussions. At the same time, they could also see the elements of the simulated operational environment and interact with them.
When a carrier strike group goes on deployment, the ships’ lines of communications are of critical importance. Between several forms of radio frequencies, types of satellites and the litany of other ways ships need to talk to one another, there is bound to be confusion. One ship might have a certain system down, and the other vessels need to be aware of that failure and in a timely manner.
“Imagine every ship has a system where they could input data automatically in real time, and every ship in the strike group would know immediately [through an augmented reality interface],” said Cruz. “With our program, the communications officer on every ship could have a headset with that data constantly updating, and all they would have to do to determine which systems are down and which systems are functional would be to look at the ship they are trying to reach.”
Turner and Cruz designed and developed a straightforward 3D visualization prototype of how the augmented reality interface would look and then tested its usability with 27 Naval officers. They focused on users’ performance and comprehension of the prototype that depicted augmented reality-enhanced Naval cyber battlespace onboard multiple ships. An additional objective was to acquire a better understanding of how that type of system could be used to assist in effective mission planning at the tactical level. And the results were very positive and encouraging, they said.
“[Users] found the interface easy to understand and operate, and the prototype was characterized as a valuable alternative to their current practices,” said Turner.
Although the research just focused on communications, Cruz and Turner suggest the concept could be expanded to cover several systems onboard Navy vessels. From damage control conditions to weapons systems, key personnel on every ship could simply put on a headset and be instantly updated on the rest of the strike group.
“It can be used by different warfare areas, and it can go towards different readiness conditions,” said Cruz. “We can input data regarding air warfare, surface warfare, submarine warfare, ballistic warfare, strike warfare, and the list goes on and on.”
Both students were very satisfied with their final thesis, and its potential to support a key initiative in NAVPLAN 2022. The interdisciplinary approach, capitalizing on each student’s unique area of expertise to advance a Navy-relevant topic of study, is a rare opportunity, except for students at NPS.
“The opportunity for two people in two different communities to combine efforts on a MOVES related thesis and to actually come up with a brand-new idea and start implementing it was a very rewarding and satisfying experience,” said Cruz. “The world is changing, technology is improving at an incredible rate, and NPS is doing all it can to stay at the very tip of innovation.”