Editor’s Note: Richard Boyd, CEO of Tanjo, authored this article. Over the past 26 years, Richard has created some of the industry’s most innovative game technology companies. He has served as a game technology consultant for a wide variety of industries including energy, healthcare, education, and motion pictures. In this article, he discusses how these games were applied to America’s defense forces to help train the warfighter for the future. Featured illustration courtesy of the University of North Carolina Chapel Hill. Provided by Tanjo.
In 2015, Secretary of Defense, Ash Carter, launched an initiative called the Force of the Future. In June of that year, I was invited to participate on the Red Team of a war game in Washington, D.C. The purpose of the war game, was to help think through the issues of increasing automation and cyber threats and aid the Pentagon in preparing the Force of the Future. There were a lot of lessons learned from the games that apply to leveraging emerging technologies to help train the Force of the Future.
As our team faced cyber attacks across all of our systems we realized that we had a shortage of talent and faced arduous policies that slowed our ability to recruit the technology expertise we needed from industry into the public sector. During the exercise we learned that:
- Policy trails technology by at least 10 years. We need to close that gap to meet new challenges such as information warfare and cyber warfare. The network-centric battlefield, like the rest of the world, is increasingly automated, sub-automated, interconnected, and data-driven.
- Increased complexity on the battlefield, think of what was soon to come in Syria, has become too much for stand-alone human management. This is reflected in the saying, “The human brain has not had an upgrade since the Pleistocene.”
- Because we will be giving more personnel these awesome powers, we will need to recruit, train, and retain with programs like those we have for fighter pilots. Programs will need to be highly selective, with tremendous investment in their training and health and mental well-being, because they will be operating very expensive systems with tremendous powers.
To address these challenges, we found that we needed to recruit and train from a new perspective.
Train for a Human/Machine Partnership: Guidance from Science Fiction
But following the threads of AI, autonomous systems, and the evolving nature of information warfare, we went on to flex the Silicon Valley futuristic muscle of our team, which included Michael Jones from Google Earth and Alan Kay of Xerox, among others. We found that the Force of the Future would need only about 20% of the human fighting force we have today but would have three to five times the strike capacity by leveraging emerging technologies.
There were a few different references that we used as we considered how to recruit and train the future force. Frank DiGiovanni, the director of Training Readiness and Strategy for the Office of the Under Secretary for Personnel and Readiness, advised us to “be provocative” and told us to “bring the outside perspective from [our] fields in Silicon Valley software, Hollywood film-making, and computer gaming to get this group thinking outside the box.”
We naturally ended up drawing on science fiction. Yes, there were the obligatory mentions of films like “Terminator” and “Ex Machina,” but we spent a bit more time talking about two specific works of science fiction. The first was a book called Daemon, by Daniel Suarez that showed that rules-based AI can be harnessed to wreak a large amount of havoc on infrastructure and systems, especially if humans can be tricked, incentivized, or threatened into performing simple sub actions. The second was Jonathan Scalzi’s Old Man’s War where a space force was created and showed how the human has to develop a fluency and relationship with this embedded AI and learn where and when to cede control to the system. I encourage each of you to read these pieces. These two fictional constructs guided our thought experiments during the two days of wargaming. We thought through how we would need to recruit people carefully to invest them with some of these new powers, like artificial intelligence decision support and targeting systems, that can fuse large volumes of battlefield sensor data, process it with complex hierarchical decision trees from the rules of engagement, and make micro-second decisions.
Combining human intuition and creativity with machine memory and retrieval, computation, pattern detection and distance sensing to compress matter energy space and time isn’t a new idea. But now we are having to wrestle with the implications of this merger and answer questions like: What is a soldier when they are amplified with all of this capability? What new considerations compel our thinking when giving a solitary human this much power?
As we discussed this idea, many in the room insisted that AI, autonomy, and other technologies – like robotics and nanotechnology and CRISPR – were simply better ways to do the old things. Our Red Team position was that they represented much more of a shift than that. As we continue the merger of human and machine, we actually find ourselves identifying new training requirements. The philosophers in the room began to argue that we were indeed directing our own evolution and becoming something more than human; transhuman, protean. One example cited was where F-35 pilots began having dreams where the aircraft was an extension of them. They called this “super-proprioception.”
A New Consideration for Procurement Now and in the Future
The driving problem of the century is how to achieve the right balance between humans and automation to optimize outcomes. This means that every organization should be looking at absolutely every activity to determine what belongs in the human domain of effort and attention, and which efforts should be turned over to automation. The answer to this question changes almost monthly and must be constantly updated.
Should there be a new position? Perhaps something like Human Automation Resource Officer?
The next consideration is whether to build that automation internally, procure it as a product, or have it custom built by a third party? Once machine learning and artificial intelligence technologies are understood, it becomes clear that, unlike other software, it cannot be trusted at all to black box operation by third parties. It works best when the technology can be deeply trained on organizational knowledge and intelligence, or domain area and remains deeply connected to all of the data lakes and ponds in an organization as well as feeding off of the data exhaust from human effort. The power of this technology and the intimacy needed to leverage it effectively means that new levels of care will be required during acquisition to ensure that the resulting system remains completely under the control of the acquirer, with transparency and traceability down to the source code level.
A question that was asked as we wrapped up the event remains with me: “What happens when the AI is defeated by EMP, or other means, and goes offline? What is the fallback or mitigation?”
The answer: “There is no mitigation. If you have any reason to believe your adversary still has connection, the only correct move is to abandon the field. Otherwise you risk a Pickett’s Charge or Charge of the Light Brigade situation with similar results.” On the battlefield as well as elsewhere in this accelerating information age, being a stand-alone un-augmented human being is insufficient.