My talk delivered on 10th of April 2014 in Bristol at ACCU Conference.
This is the combination of a few talks I delivered over 2012 and 2013 with some latest updates.
This is an experience report based on the work of many developers from Atlassian and Spartez working for years on Atlassian JIRA.
If you have (or going to have) thousands of automated tests and you are interested how it may impact you, this presentation is for you.
business model, business model canvas, mission model, mission model canvas, customer development, hacking for defense, H4D, lean launchpad, lean startup, stanford, startup, steve blank, pete newell, bmnt, entrepreneurship, I-Corps, JTAC, Computer vision, NSIN
Accessibility Testing: Mileage May Vary Sean Kelly
Karen Herr and Sean Kelly present "Accessibility Testing: Mileage May Vary"
When accessibility audits of the same digital property differ, our stakeholders have questions. How can we be assured our findings are valid?
business model, business model canvas, mission model, mission model canvas, customer development, hacking for defense, H4D, lean launchpad, lean startup, stanford, startup, steve blank, pete newell, bmnt, AI, Machine Learning, ML
My talk delivered on 10th of April 2014 in Bristol at ACCU Conference.
This is the combination of a few talks I delivered over 2012 and 2013 with some latest updates.
This is an experience report based on the work of many developers from Atlassian and Spartez working for years on Atlassian JIRA.
If you have (or going to have) thousands of automated tests and you are interested how it may impact you, this presentation is for you.
business model, business model canvas, mission model, mission model canvas, customer development, hacking for defense, H4D, lean launchpad, lean startup, stanford, startup, steve blank, pete newell, bmnt, entrepreneurship, I-Corps, JTAC, Computer vision, NSIN
Accessibility Testing: Mileage May Vary Sean Kelly
Karen Herr and Sean Kelly present "Accessibility Testing: Mileage May Vary"
When accessibility audits of the same digital property differ, our stakeholders have questions. How can we be assured our findings are valid?
business model, business model canvas, mission model, mission model canvas, customer development, hacking for defense, H4D, lean launchpad, lean startup, stanford, startup, steve blank, pete newell, bmnt, AI, Machine Learning, ML
business model, customer development, department of defense, dod, h4d, hacking for defense, innovation, lean launchpad, lean startup, stanford, startup, steve blank, Pete Newell, Joe Felter, Jackie Space
Team Networks - 2022 Technology, Innovation & Great Power CompetitionStanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, networks
Team LiOn Batteries - 2022 Technology, Innovation & Great Power CompetitionStanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, LiOn Batteries
Team Quantum - 2022 Technology, Innovation & Great Power CompetitionStanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, Quantum
Team Disinformation - 2022 Technology, Innovation & Great Power CompetitionStanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, Disinformation
Team Wargames - 2022 Technology, Innovation & Great Power CompetitionStanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, Wargames
Team Acquistion - 2022 Technology, Innovation & Great Power Competition Stanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, Acquistion
Team Climate Change - 2022 Technology, Innovation & Great Power Competition Stanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, climate
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...Stanford University
Technology, Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, unmanned, autonomy, Army venture capital
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...Stanford University
Technology, Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve Blank, Army Venture capital
Team Catena - 2021 Technology, Innovation & Great Power CompetitionStanford University
Technology, Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, unmanned, autonomy, economic coercion,
business model, customer development, department of defense, dod, h4d, hacking for defense, innovation, lean launchpad, lean startup, stanford, startup, steve blank, Pete Newell, Joe Felter, Jackie Space
Team Networks - 2022 Technology, Innovation & Great Power CompetitionStanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, networks
Team LiOn Batteries - 2022 Technology, Innovation & Great Power CompetitionStanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, LiOn Batteries
Team Quantum - 2022 Technology, Innovation & Great Power CompetitionStanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, Quantum
Team Disinformation - 2022 Technology, Innovation & Great Power CompetitionStanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, Disinformation
Team Wargames - 2022 Technology, Innovation & Great Power CompetitionStanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, Wargames
Team Acquistion - 2022 Technology, Innovation & Great Power Competition Stanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, Acquistion
Team Climate Change - 2022 Technology, Innovation & Great Power Competition Stanford University
Technology Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, climate
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...Stanford University
Technology, Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, unmanned, autonomy, Army venture capital
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...Stanford University
Technology, Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve Blank, Army Venture capital
Team Catena - 2021 Technology, Innovation & Great Power CompetitionStanford University
Technology, Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, unmanned, autonomy, economic coercion,
Team Apollo - 2021 Technology, Innovation & Great Power CompetitionStanford University
Technology, Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, unmanned, autonomy, space force
Team Drone - 2021 Technology, Innovation & Great Power CompetitionStanford University
Technology, Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, unmanned, autonomy, c3i, command and control
Team Short Circuit - 2021 Technology, Innovation & Great Power CompetitionStanford University
Technology, Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, unmanned, autonomy, semiconductors
Team Aurora - 2021 Technology, Innovation & Great Power CompetitionStanford University
Technology, Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, unmanned, autonomy, Army venture capital
Team Conflicted Capital Team - 2021 Technology, Innovation & Great Power Comp...Stanford University
Technology, Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, unmanned, autonomy, venture capital
Lecture 8 - Technology, Innovation and Great Power Competition - CyberStanford University
Technology, Innovation and Great Power Competition,TIGPC, Gordian knot Center, DIME-FIL, department of defense, dod, hacking for defense, intlpol 340, joe felter, ms&e296, raj shah, stanford, Steve blank, AI, ML, AI/ML, china, unmanned, autonomy, Michael Sulmeyer, cybercom,USCYBERCOM
This presentation includes basic of PCOS their pathology and treatment and also Ayurveda correlation of PCOS and Ayurvedic line of treatment mentioned in classics.
A workshop hosted by the South African Journal of Science aimed at postgraduate students and early career researchers with little or no experience in writing and publishing journal articles.
A review of the growth of the Israel Genealogy Research Association Database Collection for the last 12 months. Our collection is now passed the 3 million mark and still growing. See which archives have contributed the most. See the different types of records we have, and which years have had records added. You can also see what we have for the future.
The simplified electron and muon model, Oscillating Spacetime: The Foundation...RitikBhardwaj56
Discover the Simplified Electron and Muon Model: A New Wave-Based Approach to Understanding Particles delves into a groundbreaking theory that presents electrons and muons as rotating soliton waves within oscillating spacetime. Geared towards students, researchers, and science buffs, this book breaks down complex ideas into simple explanations. It covers topics such as electron waves, temporal dynamics, and the implications of this model on particle physics. With clear illustrations and easy-to-follow explanations, readers will gain a new outlook on the universe's fundamental nature.
Executive Directors Chat Leveraging AI for Diversity, Equity, and InclusionTechSoup
Let’s explore the intersection of technology and equity in the final session of our DEI series. Discover how AI tools, like ChatGPT, can be used to support and enhance your nonprofit's DEI initiatives. Participants will gain insights into practical AI applications and get tips for leveraging technology to advance their DEI goals.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
Introduction to AI for Nonprofits with Tapp Network
Introduction to the DOD 101 workshop - narrative H$D Stanford 2016
1. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 1 of 20
Hacking for Defense @ Stanford
DOD/IC 101 Workshop
Read this narrative along with the slides at
http://www.slideshare.net/sblank/h4d-dod-101-workshop-040516
Slides 1 -11 Joe Felter:
In the spirit of doing more work than any of your other classes,
it's 7pm and its not a scheduled class session and you're all here.
We appreciate all of you attending and we’ll not waste your time.
I'm going to give you a little background, and a few vignettes and
context to help frame and understand the nature of the
contemporary and emerging challenges we are facing.
This is a brave new world and a threat environment unlike those we have faced in previous
decades. I’d like to describe the complexities and challenges of this environment to help
you get you motivated to address the problems your teams are taking on. Next, Pete Newell
is going to give you some background on where our DOD and IC problems come from. And
our guest, Jackie Space is going to talk to us about some of the practical applications of the
acquisition process and the challenges and opportunities it presents.
(Slide 2)
This is a scene from West Point in 1963. You may recognize the
speaker, Douglas MacArthur, just before he passed away. This is
his famous duty honor country speech.
…the “big idea”- to use Steve Blank’s vernacular - from this
speech was that, “the mission of West Point cadets and the entire
DOD is to win our nation's wars”. How do we do this? Think about the types of wars that
General MacArthur was thinking of when he said "Our mission is to win our wars."
(Slide 3)
Back in his day, wars were not easy but much simpler to
understand how to fight. When a state was attacked, they knew
the source of the attack and it was usually another nation state
with the capacity to project power. For example, consider the
aftermath of Pearl Harbor. We knew who attacked us and in
general terms how to respond. The US mobilized its resources
and industrial base, raised powerful military forces and
projected power - directing it at a defined enemy and the enemies industrial base. In
conventional state-on-state warfare, the operational and tactical level activities that
support a strategy to win are often clear. You mass fire power on objectives. You destroy
the enemy’s military and industrial capabilities and seize terrain. All those things are
missions that the military can get their head around. Conventional wars against well
defined enemies are not easy but are simple to appreciate what it takes to win them.
2. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 2 of 20
(Slide 4)
If you fight these wars effectively you eventually win. In
MacArthur’s time we actually defeated our enemies, drove them
to unconditional surrender. We had victory parades- kissed the
nurse, went home and demobilized. The US and their allies’ back-
to-back world war champions. …We're the winning team. It was
good to be king then, right?
We used to win our wars, and we knew how do it. We not only won conventional wars like
WWI and WWII, we were pretty good at winning low-intensity conflicts, unconventional
wars, limited wars. There's no low-intensity conflict when you're at the other end of an
AK47, trust me, but we were even good at leveraging our technology and our capabilities to
win low-intensity conflicts and small wars. We were dominant, but today's it's something
that's changed.
(Slide 5)
This is a picture at the tail end of the cold war. It’s taken in
December 1989 in Operation Just Cause in Panama where we
were conducting a night combat airborne assault into Panama
and ultimately seized former Panamanian President Manuel
Noriega. Pete Newell and I were both junior officers, I was
with the 3d Ranger Battalion, Pete was in the 82nd Airborne
division. We knew the mission was dangerous. We were
parachuting in in the middle of the night to hostile territory under fire. We were not cocky,
but we had a certain level of confidence going in to the mission. We were the winning team
- nobody could stand up to our powerful military. We had aircraft carriers and ICBMs, but
we also had a pretty good tech when it comes down to the small unit, the individual level
technology.
Let me tell you about some of the technology we had back then. (Jokingly) It's going to just
blow you away - nobody but the US and other powerful states’ militaries had access to this
level of tech. We had night vision goggles you could put on your head and you could see in
the dark. No kidding. See at night! It was phenomenal. We landed on the airfield, people are
shooting at you, they don't know where we are but they're spraying trying to hit people, we
could see them. We had night vision scopes on some of our rifles. We could put a crosshair
on the white T-shirts they were wearing. It is “good to be the king” and have access to these
game changing technologies.
Before we went in we had overhead images from satellites taking pictures of the earth and
giving us maps so we knew how to plan, and knew where our objectives were. These
images obtained from national assets were so sensitive that the satellite imagery was
classified material. I remember tying my imagery with parachute cord onto my cargo
pocket because if you lost it, it was losing a sensitive item and I would be subject to an
investigation.
3. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 3 of 20
I literally was jumping out of a C-130 at 450 feet which was pretty low. Getting shot at. My
map rips out of my pocket. All I can think of is, “oh my God I'm in big trouble because I just
lost my overhead imagery,” which is pretty sensitive stuff. At the same time we also had
satellites up in orbit triangulating to tell us within 10 meters of accuracy where we were
standing. GPS. It was crazy.
We had communications, where every individual soldier in my platoon could talk to each
other. I mean little tiny radios that could talk to each other. These little pens that could
shoot a laser to direct fire and maneuver. Never before in the history of warfare has an
organization been able to direct its operations and fire and maneuver with that kind of
technology. And all of this cost thousands and thousands and thousands of dollars, really
expensive stuff that few other states if any could afford to deploy.
There’s a saying that, “the strong do what they want to do and the weak do what they have
to do.” We're one of the strong states. What do strong states like the United States care
about? Who do they worry about? Other strong states, right? So who was bigger than the
U.S. back then?
But this was the end of an era. This is 1989, the cold war was about over. In some ways,
despite the nuclear menace, those seem in retrospect like the good old days. When it was
good to be king – where power and resources translated almost directly in to battlefield
dominance.
What's changed? Who are these folks?
(slide 6)
I was joking about the advanced technology we had in
1989. Today with a credit card and Internet
connection and you can get any one of those things
described as game changing advantages for us when
Pete and I jumped in to Panama for a $100. Night
vision goggles are cheap and available, Google Maps
with images of almost any part of the planet are
online, you can just get that and download it. All of this was formerly only in the hands of
the very strongest states, the very strongest super powers.
The following slides highlight just a few examples of how the proliferation and diffusion of
technology changes the face of the battlefield today and why this matters. Let's look at
some examples of the asymmetries that groups like ISIS, al-Qaeda and other groups take
advantage of to do us harm.
4. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 4 of 20
(Slide 7)
ISIS is able to use overhead imagery and drone surveillance
to plan and execute their operations. It's not so hard to do,
they've got their own drones. They're using this against us.
This is an example of technology that's now in the hands of
our adversaries that formerly was only in the hands of the
strongest states like the United States.
(Slide 8)
Terrorists and other nation states are using social media
against us. I remember going to SERE school as a special forces
officer candidate. We were trained to develop a cover story to
hide our identify if captured. This was a whole convincing
cover story that you give so you don't get found out. Today, if
you Google me or call up my Facebook page it would be pretty
clear who I was and my background.
(Slide 9)
Imagine what our our enemies can do with readily available
technology like 3D printing? The instructions for making the
semi-automatic rifle in this photo are available online.
(Slide 10)
Cyber Threats to Critical Infrastructure. Think about how the
internet and social media have changed the ability to recruit, to
train, to radicalize, motivate and inspire our enemies. It's just
unbelievable what this technology is doing to empower our groups
intent on doing us harm.
One of the big advantages of being a strong superpower-like state was that you got to
project power. Now anyone sitting anywhere in the world can project power … so now the
ability to project power is not only the purview of strong states and superpowers, it's kids
who are just logging on and they're able to attack us from afar, with cyber threats.
(Slide 11)
Back to MacArthur. … when he told the West Point Corps of
Cadets in 1963 that their mission was to win our he was talking
about defeating strong states which were our biggest threats at
the time. Think about what the threats are now. International
relations theory states that said, strong states need to be most
5. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 5 of 20
concerned by threats posed by other strong states, has been turned on its head. Today,
weak states, sub-state actors, even individuals are now empowered through increasingly
available technology to do us harm. Addressing these emerging and lethal threats is now
also required to “win our wars”,- this is definitely a different kind of threat environment
that we're in.
So what do we do about it? How do we win our wars going forward? At a minimum we
need to stack the deck again in our favor. We used to be back-to-back world war champs,
able to mobilize our resources effectively and take it to our enemies. Well I would argue
that in many ways we're not doing well in todays changed threat environment. Not only do
we need to stack the deck in our favor, in some cases we are at a disadvantage and just
need to level the playing field. Ironically we aren't there yet. We've got to continue to
defend against high-intensity threats. This isn't just a war against ISIS and terrorism, we've
got a resurgent China, pushing us in the South China Sea. We have to basically defend on
both fronts.
ISIS doesn't have federal acquisition regulations; in Steve Blank’s definition ISIS is a Lean
Organization. They are pivoting, learning and adapting and they rapidly react to evolving
threat environments. This is a tough adversary - an adaptive enemy. This is a threat
environment where our adversaries have access to technologies and they're able to adapt,
to overcome, and improvise and pivot and deploy that technology in ways that are way
ahead of us.
I’m trying to emphasize that this is a different type of threat environment, different type of
adversaries. The formerly linear relationship between state power and military
effectiveness has forever changed. States don’t automatically generate the power predicted
by their resources – choices matter. Some of the advantages that we enjoyed back in the
day when we were “back-to-back world war champs” don't really apply today.
The stakes are high and we're losing ground on many fronts. We need to do something
about it. You can help.
Pete Newell is going to give you the background of how we develop our requirements,
where our missions come from and then our guest Jackie Space is going to talk about some
of the challenges and opportunity of the acquisition process.
Slides 12-29 Pete Newell
Obviously, things have changed over the years. The
environment has changed, our country has changed, the
economy has changed.
Each of your teams has a problem that was generated by a
government sponsor for a reason. The rest of the discussion
tonight is peeling back the layers to figure out what that reason is, where that problem
might have come from and who else shares that problem.
6. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 6 of 20
To begin the search you must understand how our National Strategies are built and how
they come together to drive this massive organization called the Department of Defense.
What we're going to try and do here is fast as we possibly can, is give you a one over the
world of how strategy and funding effect how things are bought.
(Slides 13-15)
On this first slide is a big, complicated chart that would take
me 30 years to explain to you. So we're not going to do that.
We'll cut it down from 2000 slides to about 15.
Here's the chart. This is the defense acquisition system. Way
up there in the upper left hand corner, is where the
requirements for military systems come from and what gets things started. To make it
simple if you start on the top left you have the Joint Capabilities and Integration
Development System, (JCIDS). JCIDS is how we handle the collision between two many
requirements and not enough money.
We're going to split this talk. I'm going to take care of the tanks and airplane side of this
and then we're going to hand it off to Jackie Space and she's going to talk through the
islands of innovation where a lot of these rules don't apply, or they've condensed them into
much tighter acquisition cycles.
Looking at the red boxes at the top of the chart I’ve highlighted the key activities you need
to understand, beginning with how requirements are developed. Next we develop
prototypes. Then we figure how we're going to produce something. Then we figure out how
we're going to sustain it. That's kind of the life cycle of the acquisition system. It's not real
complicated until you dig in to who does what to who and how and when they make
decisions. For the purposes of this class what is in the red boxes at top of the slide are the
things we've got to keep in mind.
(Slides 16-17)
Here's what you've really got to remember, there are
three key activities that take place. In the bottom left
corner of this slide is the planning, programming, and
budget execution work - no different than any major
corporation out there. You have to figure out what funds
are coming in and where you're going to prioritize and
send those funds out to. This is largely driven by a process
of strategy documents that we'll talk through in just a second. To the right in yellow is
JCIDS, the system used to develop requirements. Finally, in red at the top of the slide is the
acquisition process of how we buy things to fill the gaps that were identified in original
documents.
Jackie Space: Do the people in the audience know what we mean by requirements?
Basically requirements are what are used to build a system, it’s the technical parameters by
which the system is being built. So there is a whole process of people that develop the
7. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 7 of 20
requirements and the cost requirements, the technical requirements around what system
is actually being built.
Joe Felter: For example, defending against dismounted IEDs (roadside bombs) in
Afghanistan versus the IED threat in Iraq, which were against vehicles. The lag time that it
took for the military to address the dismounted IEDs threat in Afghanistan caused
thousands of casualties. Pete made that a very poignant point last week.
Pete Newell:
(Slide 18)
All right so here we go. Let's start with the strategy documents.
The strategy document that starts it all is the National Security
Strategy (NSS). This is a document that the President of the
United States produces that lays out America's enduring
interests, not just militarily but also economically, and socially.
The NSS lists four things that are considered to be our enduring interests.
1. The security of the United States, its citizens, and US allies and partners.
2. A strong innovative and growing US economy.
3. An open international economic system that can produce opportunity for
prosperity, respect for universal values at home and around the world, and
4. An international order advanced by US leadership that promotes peace, security,
and opportunity through cooperation to meet global challenges.
(Slide 19)
The Secretary of Defense uses the NSS as a basis for issuing
his strategy. The Department of Defense calls it the National
Defense Strategy (NDS). The NDS takes the President’s
National Security Strategy, looks at it from a defense
perspective, and says here are the things that we need do
that meet those Presidential objectives. The current NDS
includes things like counter-terrorism, irregular warfare,
defer and defeat aggression, project power, counter weapons of mass destruction, provide
a stabilizing presence, conduct stability, and counter insurgent operations, and
humanitarian disaster relief and other operations.
There's a bunch more in there but broadly it details what the Department of Defense is
going to do to meet the President's National Security Strategy.
(Slide 20)
Then comes the quadrennial defense review (QDR). It’s
done every four years. You notice the dates of these are out
of sync, because several years goes between each one of
them. So one document may be issued in 2015 but we're still
operating off one from 2012 another one from 2013 and one
8. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 8 of 20
more. The quadrennial defense review is really the document that starts to take those
strategies and look at them in terms of what can we really afford.
The QDR describes what are we really going to do in order to meet the President’s and the
Secretary of Defense’s requirements. The document has some very broad statements like,
“Protect the homeland” and “Deter and defeat attacks on the United States,” “Build security
globally to preserve regional stability.” “Project power and win decisively and defeat
aggression.” But underneath there's a nice line that says "At the President's budget level
the military will be able to defend the homeland, conduct sustained distributed counter
terrorist operations, and deter aggression and assure allied in multiple regions before
presence and engagement." You know what it doesn't say anymore? That we're going to
fight and win two wars simultaneously.
Seriously. At the President's budget level, which means the President's already told him
here's what we're going to pay for in terms of defense this year, they determine that these
are the best that they can do to meet the key objectives that were set up by the National
Defense Strategy in order to meet the President's National Security Strategy.
I think you're starting to see the where the gaps might start to appear. As we change one
word in one line in the QDR and suddenly the Strategy documents mean something
radically different. Are we buying tanks now? Or are we doing more humanitarian
operations? More aircraft carriers or more airplanes? Depending on where you sit in that
big system it can very rapidly change based on how you perceive the best answer to those
problems.
What the three documents end up giving us, and what falls out of it, is called the Defense
Planning Guides. It really is how we develop the budgets that drive the organization of the
military and eventually the activities of these agencies.
(Slide 21)
In the U.S we have four Military Services: the Army, Navy, Air
Force, and Marines. (The Coast Guard gets to act like the fifth
service but they actually belong to another government
agency.)
The military services are responsible for providing people
and equipment to combatant commanders. It's their job to
raise the army, navy, air force and marines. It's their job to train and equip them. It’s their
job to provide trained equipped ready forces to combatant commanders who are the guys
out here who actually fight the wars.
In between the Military Services we have a number of Defense Agencies. The easy way to
know if your talking about an agency is the last word in their name says agency. It's things
like the National Geospatial Intelligence Agency, the National Security Agency. There's only
agency who doesn't have the word agency is their name is the National Reconnaissance
Office. For some reason they got to be different than everybody else.
9. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 9 of 20
The agencies out there all have consolidated portfolios of activities that they do at the
behest of the rest of the government for DOD, but they're also strategic force providers for
the combatant commanders. I'll show you a couple of slides that actually makes this a little
easier to understand.
Finally, we have a bunch of DOD field activities. It's interesting what shows up as field
activities. For example, Washington Headquarters Services is responsible for the military
installations of Washington DC. Another example of a field activity is the Defense POW
mission personnel office. 60 some odd years after the end of the Korean war we are still
looking for persons missing in action or remains that were never recovered. We still have
an activity that actually goes to Vietnam, Laos and some other places looking for the
remains of service members who never came home.
There are lots of other small activities that don't neatly fit anywhere within someplace on
that chart up there.
Student: I noticed that the Central Intelligence Agency is not on there. Why?
Pete Newell: The Central Intelligence Agency is not a DOD activity or agency. Although, I'll
talk a bit later about how some of the other government agencies actually have interest
that are embedded in DOD. As a deployed brigade commander in Iraq in southern Iraq in
2010 I had members of the CIA, DIA, the NSA, the FBI, the secret service, all who were part
of my organization or part of my footprint that I was responsible for ensuring that they
could do whatever the government sent them there to actually get done.
Student: When there's a group that's made up of DOD personnel and say CIA personnel,
how does that work in terms of coordination?
Pete Newell: There are first a series of standing inter-agency agreements that account for
cross-agency activities. Then there are a series of contingency plans that are a reaction to
something, that automatically enact. One example is what happened on 9/11. On 9/11
little known to anybody there was a 1960’s agreement that allows the United States Air
Force to take control of all of the air space in the United States. About 30 minutes after the
plane hit the second tower, there was a young watch officer at NORAD, which is in
Colorado, who reads a one-line sentence over the phone one line sentence. "We at NORAD
are enacting ... " and what you heard on the phone was dead silence while people were
focused on figuring out what he was talking about.
The first guy on the phone is a guy from the FAA who says "So if I understand this right you
want us to turn off all the navigational aids in the United States." There was a pregnant
pause and this guy's flipping through the book, and says "No I want you to do X, Y, and Z."
In some cases these are agreements and rules go back years. In this case there was a staff
and decision makers who zeroed in on that regulation who understands what's supposed
to happen.
10. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 10 of 20
Student: To what extent is the structure here embraced by personnel within DOD? Like
how easy would it work this time?
Pete Newell: What you have to understand is every time an agency gets their name up
here on this chart, they have a budget that comes with it from Congress. It takes almost an
act of Congress to change one of those things. If you're a new organization and you want to
become an official agency, it takes an act of Congress to get your name up here. From a
structural context the last major change that was made was based on the Goldwater-
Nichols Act that mandated the formation of combat commanders and a number of other
things. It changes from time to time, but not frequently.
(Slides 22-24)
As I mentioned, the services – the Army, Navy, Air Force, Marines job
is to provide people and equipment to the combatant commanders
who are responsible for the world.
On the bottom of this chart are the combatant commands. The
African Command (AFRICOM) is a command responsible for 53 African countries. The
Central Command (CENTCOM) is responsible for the Middle East, North Africa and Central
Asia. They are also responsible for prosecuting one war in Iraq, one in Afghanistan, and
also dealing with pirates outside of Yemen. The combatant commanders get all the bad
stuff that goes on in the world. To help deal with the world they get people, equipment,
assets from all of these Services, and from all of those agencies that listed on the previous
slide.
Based on their missions they have a series of contingency plans that are developed in
response to the strategies, that say, “if this happens we apportion these kinds of forces to
you.” Each one of those things also comes with a budget. And the budget come with specific
“types of money” called “titles of money.” Title X money belongs to the services. For
instances, as the Director of the Army's Rapid Equipping Force, I had a $200 million budget
which was title X money.
I'll give you a scenario. In Afghanistan, if a Marine element working as part of an Army
organization came to us and said “hey we're having a problem and we need to find a
solution for,” I couldn't do it. I couldn't give it to them because I couldn't spend title 10
money directed to the Army by Congress to provide equipment to the Marines. However, I
could provide that equipment to the Army element that controlled them, who could then
assign it down to them.
There are titles of money directed at Reserves and National Guard forces strictly for their
use in the United States. There are titles of money for combatant commanders. There is a
different title of money that's directed to them for their contingency operations to do
things and then there's all kinds of colors of money related on what activities they place in
the say of things. We’ll come back to talk about the “colors of money” in Slide 27.
11. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 11 of 20
The key takeaway here is the world's broken up into regions, that are run by combatant
commands. And it’s these combatant commands start to identify gaps based on their
operations.
For example, In Central command, the Army and Department of Defense felt that getting
MRAPS into Iraq would save soldiers in Humvees from getting blown up by IEDs. MRAPs
were the best thing since sliced butter, but they were as big as an elephant, armored but
saved lots of lives. We moved to Afghanistan and MRAPS there were like driving in
quicksand. Very shortly afterward the CENTCOM Commander was telling the Department
of Defense is the MRAPs they were providing in Afghanistan were inadequate to meet the
conditions of the environment and that created a gap.
(Slide 25)
That gap then, leads us back to that JCIDS process I mentioned
on the initial slide. In JCIDs a gap is assigned a priority for
somebody then to go figure out how to solve it. That solution
then turns into a requirement. The solution in this case was
adifferent kind of armored vehicle which were a smaller version
of the MRAP. They were lighter, could handle the sand a little
better.
Here we go. Gaps, not requirements, gaps. Those top three the competition with what these
guys say they have to do in order to actually achieve something. So lets take one of the
teams in class here, distributed ISR. Is it a gap or a requirement?
Student: It is a gap,
Pete Newell: Who's gap is it?
Student: The gap is lack of a capability to rapidly, to have eyes on most of the domain the
7th Fleet is supposed to be keeping track of, and simultaneously be able to deploy
something quickly to, if they wanted to see something rapidly.
Pete Newell: Correct. If I'm responsible for hunting Russian nuclear submarines in the
Pacific and I can't find them because the ocean's too big, it's a gap. If I am responsible for
securing the high speed access to the western coast from drug and illegal people
immigration, those kinds of things, that's a gap. Which combatant command is it?
It’s PACOM where 7th Fleet is assigned as a Navy component to the command. We'll walk
through a little bit, but you can see where from multiple perspectives the same gap may
produce different requirements. The gap is still the center, we can't manage to do X, Y, or Z.
(Slide 26)
Obviously there is friction there. I'll tell you this happens a lot.
These guys get a budget and they plan on a five-year cycle. I
started building a tank in year 1, we're coming up with a solution
12. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 12 of 20
that I'm still deploying in year 5. However, in Year 2, the combatant commander comes in
and says whatever you were building doesn't solve my problem any more. The guy up at
the top says, well if you want me to start over it's going to be another five years before I get
your solution to you.
In some cases the combatant commands will use their money to actually procure material
solutions that fill immediate gaps. When they're done with the material, or when it gets
really expensive to maintain, they'll look at the service, because they have all the
authorities to maintain that stuff, and say here you go I need the maintenance package that
goes with this stuff. There's friction between the two over how they solve those kinds of
problems. When it happens it can be really ugly.
Student: 8 problems were chosen for the teams in this room to work on. Were any of them
requirements, or were they all gaps?
Pete Newell: None of them are full-fledged requirements. A lot of people misuse the
terminology. I have a “requirement” to fix this problem. The only way you get a
requirement is to come out the bottom end of this massive chart up here. We very
specifically told our sponsors not to give us requirements. Your gap came from the part of
the chart up here under joint operating concepts.
Eventually gaps turn into JCIDS recommendations that say we're going buy a tank that
requires three people that has to be trained and sustained, and a long list of things that
goes with it. Eventually that comes out a list of requirements.
Once it comes out as a real requirement it's very, very hard to change because you have to
go back through the entire process. In our case, for this class we very specifically vetted
some of the problems to ensure that they were more on the gap side, that they hadn't
determined what the requirement was going to be and it wasn't well within that process.
All of the problems for this class fall within recognized gaps that are out there …
Student: What's the authority that certifies that a requirement has been met? For instance,
if I have a requirement that's out there and I'd like to be a little bit flexible with it. Who
certifies that yes the requirement is met and the way that it's met?
Pete Newell: I'll give you the book answer and then I'll tell you the politics.
At one time they set a standard for measuring battery usage: 9 people operating for 72
hours away from the base. This makes sense. 72 hours means we carry a lot of weight, a lot
of water, food and a lot of batteries. At the time depending upon what mission we were
performing in Afghanistan those my guys had to distribute 247 pounds of batteries to
operate for 72 hours.
The first standard that is set is not really a standard for the requirements. it used for the
standard for the measurement of performing effectively for 72 hours. So the first guy says
why is it 72 hours and not 120 hours. Nuclear power guys who have it in their heads that
they want to deploy small nuclear devices to power this. You know what those nuke dudes
13. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 13 of 20
have? They have a really good powerful lobbyist. That caused a debate for 18 months over
the capabilities manager and the other guys who said 72 hours is standard. They had to go
back over 10 years worth of documents to articulate why the standard was 72 not 120 or
24, or something else. Because as soon as you open a can of worms everybody comes in and
says well my solution would be perfect if it was 24 hours. But if it's 36 then I'm useless. I've
got a lobbyist, I'm going to attack the system to try and get the standard changed.
People play a lot of games when it comes to who determines the requirement. Within the
Army, the Training and Doctrine Command is responsible for actually training people and
determining how many pieces of equipment we need to perform a mission is responsible
for writing the initial requirement document. That document then goes to the Pentagon
where it goes through a series of boards. The boards then decide that yes is it a valid
requirement written correctly, the parameters fit within our priorities, it doesn't violate
the law, etc. I can't tell you how many of those boards there are. Eventually it comes out the
other end and poof we have a requirement.
It depends on what you're touching and how many different complexities there are to it,
but if you listen carefully you'll start to hear vendors and other people who have things to
sell starting to talk about how their thing does better than something else. They'll start
quoting numbers. I've done it with sensors. I'm actually working on a vehicle at one point at
Georgia Tech Research Institute validating the sensors placed inside striker vehicles to
provide data logging against IED blasts. We wanted to know what happened inside the
vehicle. We came up with a set of parameters for that sensor. It was a pounds per square
inch reading that it had to withstand. Literally I had a vendor start a Congressional
investigation over why it was 10 pounds and not 8 pounds - because it meant his versus
another vendor’s sensor.
You run into those things all the time. Which is why sometimes somebody is willing to say
exactly what it is because as soon as you said it, and you can't retract it. I know that's a long
answer but I just want to tell you it's not simple.
Student: for a program like space-based radar, led by Air Force, do they consider the needs
of other agencies and take requirements from those people?
Pete Newell: If they're smart they will. They'll not only consider what those other people
need, they'll consider what those people are already doing. If you want to be successful you
avoid duplicating something else.
Student: Who was the main sponsor of this class? I was just wondering what's their
expectation from the outcome of the class. …
Pete Newell: Do you mean, “What's the gap that this class meets?” It’s the Lack of DOD/IC
innovation. There is a lack of opportunity for young technologist to perform a national
public service. There is a massive gap between the military and the civilians that they're
charged with protecting. A shrinking military in a very complex world where technology is
blowing past them at light speed who has no access to the intellect that you have on a
14. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 14 of 20
recurring basis. Those are all gaps. Nobody's written a requirement, although they're
talking all over the place about how do you deal with those issues. For folks like us that
recognize that there's a gap our hope is that we can step in and prototype something to
help validate that the gap is real and that there maybe a potential pathway to solving it.
This class is exactly what's going to happen. We'll prototype it. It's a pilot. It's going to
grow. Somebody in the government is going to decide to put a whole lot of money in it, you
know $20-$30 million a year to ensure that it grows into another 30 colleges across the
country. Somebody else from another agency says that's really cool I want to connect my
program to it. And eventually we'll be at the sustainment mode. …
Student: We spoke with some folks from a big system integrator who said said that part of
the reason why the sensors are so expensive and hard to deploy is because we have to
source everything through the US facilities and assemble in secure facilities, and so on and
so forth. If our solution goes down this pipeline, at what point might we be expect for
someone to say hey we need to source everything in the US.
Jackie Space: You are too early in the process to judge where you need to source it.
Because they have to go through the traditional defense industrial base for the most part its
true. But I think that there are certainly sensors or other technologies potentially that are
being built on the outside. Eventually at some point, if you're going to continue
development on something big you will have to consider that.
Student: …I guess you're saying in commercial sites their tolerance for sensor error are a
lot higher because they're not used to approaching it from the ‘hey we can fix this area.’
They need to have a much lower error. …
Pete Newell: I would say there's a fundamental difference when you apply them with
public mind that has to be accounted for to the nth level. My experience in dealing with
these folks, and it's not because they're bad people but the system creates bad behaviors. It
dis-incents people to take risk. Not only are they not rewarded for accepting risk and
failing but they're actually punished for not being perfect. So in the goal of trying to
perfectly acquire something we tend to fail bigger, more frequently than we would
otherwise. Part of the beauty of what Steve Blank has done with Lean LaunchPad is
essentially build a framework by which, with a little bit of translating, we've been able to
provide a strategy by which we can fail in this system much earlier and much less
expensively.
At least that's our hypothesis. That's where we're at today. …
Jackie Space: There are places in the government that are always going to remain military
grade. The billion-dollar satellite that they launch that enables somebody to read a license
plate? That's going to stay a military grade system, but there's this awareness now in the
military and government that there's a lot of other things out there that are good enough,
with a much lower resolution that you can find the private industry.
15. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 15 of 20
(Slide 27)
Pete Newell: One of the things that you have to understand is
money has different colors. There's different types of money
to say what it was used for. Believe it or not the Department
of Defense caused itself this problem. It doesn't have to be
this way. If Congress would quite honestly change it if DOD
said “we want to change it.” But here's what happens.
Operations money is essentially one-year money that is used
for everything from buying toilet paper in a barracks to paying for gas for vehicles or for
buying things that cost less than $250,000 per item. I can buy something for $200,000,
even if it's a piece of equipment, using operational dollars.
The problem that you run into with operational money though is on the 30th of September
it goes away. It doesn't go back into the government's bank, it doesn't go back into some
magic bag to pull out later. It is gone. Disappeared.
If you are working on a problem and a guy suddenly shows up in July and says I've got a
million dollars I can spend it today, what can I get. My guess is he's probably using end of
year money and he's in a panic because he knows it disappears the 30th of September.
Here is what happens. The folks in the Pentagon who manage a massive budget say on the
30th of August if you have money left in your account subordinate to us we're taking it all
and we're going to spend it on our un-forecasted requirements. So the next layer down,
they say “if you have money left in your budget on the 30th of July we're taking it all for our
use. On it goes until you have the poor guy, who on the 1st of April, is in a panic because he
hasn't spent his money yet.
Jackie Space: If you don't spend your money you get your budget slashed the next year.
They say oh you didn't spend your money. … another point on these colors of money is that
if you have a project that you're working on you should be asking beneficiaries what type of
money are you working with. How you end up putting your proposal and projects together
can actually fit in all three of those depending on how you write it. Really having an
understanding beforehand of what type of money you're working with is very important
Pete Newell: Here are the questions you start to ask so you can figure out where they are.
First, procurement dollars. I will tell you procurement is not my expertise. I probably
violated more rules than not. Procurement process was designed to buy much larger
things. That's how we buy tanks. That's how we buy sustainment. The life cycle cost of
putting something out there. Unfortunately, you can't use procurement dollars to buy toilet
paper or anything like that. So it’s designed to buy major items.
It does last for three years. Which means that we're in the middle of 2016 so at the end of
September 2016, 2014 procurement money is going to expire. Why is that important to
know. If somebody's telling you “hey come do this for us we've got $3 million,” You might
want to ask what the expiration year is. Because if something's dragging along and you're
16. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 16 of 20
about to hit the 30th of September, and you don't know if the contract is going to be signed
yet, if it's expiring money you're not going any further.
Jackie Space: The smart money manager will have his money spent by July 1st just to avoid
losing it or his contracts. A lot of people don't, so another smart manager knows how to go
around get money to use in their program from people that can't get their money spent.
One of the questions when you're talking to your program sponsor, especially program
managers, is how do you handle your end of year budget. You want to know if they are
good at planning money at end of year to support your programs.
Pete Newell: I'm going to hit the last one, RDT&E money, research, development, testing
and evaluation money. With this money you can build prototypes, you can test, etc. Most of
you working on your solutions would likely first touch RDT&E money.
There are special cases of money starting with purple money which means they can do
whatever they want with it. In some of these cases organizations have been given special
authorities to allow them to essentially break all the rules. The problem sometimes though
is when you're given special authorities you also get special oversight. Those organizations
are probably the most scrutinized organizations out there in terms of Congressional
oversight. Not that they're going to do something wrong but because Congressmen want to
know that that money's being spent where they think it's being spent.
The Secretary of Defense has a Rapid Acquisition Authority (RAA). When I was with the
Rapid Equipping Force, the Secretary of Defense could authorize the up to $200 million a
year in RAA that would allow us to change the way money was used. To use that authority
we would simply write a memo that says we are going to use RDT&E money to buy gas or
something like that and staff it through the Secretary. If he approved the request we were
able to do what we needed to. It wasn’t easy but it wasn’t hard either. It was fairly quick but
it also required Congressional notification because we were doing something other than
what Congress directed with the money.
The following type of money is one you have to pay attention to. Overseas Contingency
Operating money. This is the thing they slap on top of the budget every year because we
keep having conflicts someplace that weren’t planned for as part of the budget. What you
have to know about OCO is that if you're working on something and somebody's offering
you OCO money to do something, what you're doing has to be focused on solving a gap
overseas. You can't use OCO dollars everywhere, because right now I think OCO is still
limited to CENTCOM which means that we can't solve the distributed ISR problem in
PACOM using these dollars - even if that's all we have at our disposal.
The last thing you ought to pay attention to, continual resolutions (CR’s). For example, if
we're not going to pass a defense budget in September because Congress wants to see what
how the election turns out first, we will likely end up with a continuing resulution to fund
the DoD. What happens with a continuing resolution is that the budget folks will say “you
are not allowed to program spending more than 65% of last year's budget.” That poor guy
who's supposed to spend all his money by April or if it’s already taken away, is now told
17. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 17 of 20
you've got to stretch 65% of your budget to last all year. When the budgets finally passed in
January, what happens? He suddenly has this massive (influx) of cash and he's got how long
to spend it? Two months.
Which means he has to figure out what he's going to buy, he has to contract it, he has to do
everything in a very short period of time.
Student: So they literally cannot spend 35% of that budget on anything over this period? …
Pete Newell: No, in fact they have to turn in a budget that does not exceed 65% of the
previous year’s budget and show how that will cover them for the entire year. Once there is
a Defense budget passed, they have to submit a whole new budget. This means they're
always playing with two budgets. This is sometimes what causes expensive programs to
triple in expense. Failure of politicians to do their job is one of the biggest expenses we
have.
(Slide 28)
Let’s go back to the mission model canvas; Value
propositions, buy in, advocates … according to who?
There are a lot of people out here who will tell you they
fall in all three of these categories. On this slide is a list
of folks you should look for. On it I've also left you
some questions you ought to be asking whoever you
are talking to.
The first one is:
• who are you?
• Where do you fall in here? Beneficiary, advocate, are you both?
• Are you from a service? Are you from a COCOM?
• Where do you fit in this massive list of things?
• What's your mission?
• What are you supposed to do? What are you not supposed to do and not allowed?
Make sure you've got that clear in your head, what are they supposed to do, what are they
not supposed to do. Where'd your funding come from? What kinds do you have? What are
you talking to me about? I would ask about types of contracts if it's appropriate. Finally,
who do you work with? Then finally, who else do you know that has this problem that
might be working this area?
This what I call an asset inventory. Everybody you meet exists in an ecosystem. You're
trying to figure out where they belong and how they connect. Here are 8 questions you can
ask that will help you clarify where they are. The answers to those questions, or the way
they answer them will allow you to figure out where they belong in your ecosystem of
things that you might need eventually.
18. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 18 of 20
(Slide 29)
Using one of our teams working on Distributed ISR as an
example, the sponsor is the 7th Fleet. The 7th Fleet is part of the
US Navy, however 7th Fleet is assigned to PACOM as the Navy
component of the Pacific command.
So who has the problem? Is it the Navy's problem or is it
PACOMs problem? Or is it both? Might they have difference of
opinion on how to solve that problem based on how expensive it is to train people and how
many people it costs. Do you see where the friction might come from? You can be given
conflicting guidance when you're talking to people, one side versus the other side.
All these agencies who provide assets to PACOM, probably have some amount of equity in
that problem. By equity we mean they have authority and they have budgets to take action
against parts of the problem. They may also have five different opinions on the scope of the
problem that are different than that of PACOM or 7th Fleet.
Now, the CIA, the Coast Guard, other folks. Don’t you think that they might have a vested
interest in how this is solved? Are they a user? Are they a capability provider? Are they an
advocate? Are they trying to kill the effort?
Other services. If the Navy component of PACOM says this is a problem rest assured there's
an Army component, there's a Marine component, there's an Air Force component who all
think it is as well, but for different reasons. Do you think they might have a vested interest
in this?
Finally, SOCOM is a combatant command, but SOCOM also provides folks to PACOM. So
SOCOM has a vested interest in this right? Slide 28 is what you've got to pay a lot of
attention to. Money, contract, strategy and all that other stuff, it's interesting but not
necessarily going to help you for the next 8 weeks. Slide 28 will.
Slides 30-36 Jackie Space:
So Pete just talked about the acquisition system at large and I want
to just drill down really quickly into the more practical aspects of
what it means for your teams and the program sponsors that you're
working with, the problem statements, and how it fits into this. Also
for you're beneficiaries and how you should be talking to them as
you're working through their problems.
(Slide 31)
Quickly about me, I'm a systems engineer by trade, I went to the
Air Force Academy. I spent the majority of my career doing
government acquisitions. I've worked very large programs like
GPS and I've also worked very small programs in terms of getting
things rapidly fielded into the field from a technology
perspective.
19. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 19 of 20
(Slide 32)
The purpose of this slide is to to convey to you that even though
you're all working with different government organizations and
they all fall under this big system of acquisitions.
Every organization is quite different in culture and so when you're
working with them you need to figure out, 1) do they move
rapidly in how they do business, 2) how they develop technology, 3) how they field them.
Are they focused on larger weapons systems and is it's going to be ten years before they
actual develop and field capability. When I say rapid I mean like less than 2 years, versus
10-year time lines. Part of your work is to figure out where does your organization
program sponsor reside in that time frame.
(Slide 33-34)
The whole acquisition process is designed to develop big
programs - like the F35, ballistic missile defense, helicopters, -
that sort of thing. All of this exists to be able to field big weapons
programs. A lot of the friction ends up when we have
requirements or when we have needs that emerge that don't fit
into the time line.
On slide 33 we've got the DARPA’s and the Air Force Research Lab and the other labs that
exist in the government that are developing technology that will probably never ever leave
the lab. Or in some cases it will take 10 years before it actually does anything. On this slide
I put some of the organization that the teams in the class are working with to show where
they fit in technology maturity and how they're different from both ends of it.
If you look at two of the sponsors in this class, the Asymmetric Warfare Group and the Joint
Improvised Threat Defeat Agency, they operate differently. They're not concerned about
weapon systems or developing things in support of those weapon systems. These guys are
on a rapid time line and they want to field capabilities to user requirements in less than 2
years. All of the problem statements for teams in this class fit within this category. This is
where the opportunities reside to actually make an impact because you have a shorter time
line to fielding and delivery.
Then you have other organizations like SOCOM, NSA, CIA, and the service components that
play in all parts of the timeline. Even with the team that's working with SOCOM I would bet
that they are more on this 2-year time line for fielding a solution. But that's something that
you should be asking your program sponsors when you're working with them, “What is
your traditional time line for fielding technologies?”
(Slide 35)
For entities outside of the traditional defense industrial base
to engage (ie. Startups), it's really in the R&D area as well as
prototype development. One caution, when you talk about
government R&D, if you bring a capability you want the
20. This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License
DOD/IC Workshop: Hacking for Defense Joe Felter, Pete Newell, Jackie Space Page 20 of 20
government to give you money to help develop, you will eventually have to deal with IP
issues. Because anytime the government gives you money to develop something, they now
own it. In other cases where you have a capability that's more developed that you bring
that to the government and demo it, that's when you can actually maintain the majority of
your IP.
(Slide 36)
I want to talk more about the questions that you should be
talking to your beneficiaries about. …Most of you have
talked to the potential users of your system and you're
getting feedback from them, which is really valuable.
However, you need to find the program managers and/or
the people that are actually executing a program that is
either similar or has parts of the problem that you're dealing with. That’s because the
program manager understands the time line it takes to field something, ie. What are all the
things that they have to go through to actually bring on a capability like that? How do they
engage with industry? How do they actually participate with people outside of their
program?
Finding that individual or set of individuals is really important will be really important for
this class.
In terms of the culture, I think it's incredibly important to as you're talking to your
sponsors to ask the questions about the track record of their organization in actually
getting something like your solution fielded? How long did it take? How many times have
they actually attempted this particular technology set? Do you know who the mission
partners around you that may be able to contribute to that?
To get more information go to FedBizOpps.gov. FedBizOpps.gov is the main mechanism by
which the government will post or solicit for industry feedback. They'll post their problems
saying, we want a proposal or we want some information around this actual technology
that we're looking at. If you go to FedBizOpps.gov and you type in a keyword around your
particular problem, what comes up will be really interesting because you might find that
there's five or six other organizations that are actually soliciting for the same technology.
Thanks,
Joe Felter, Pete Newell, Jackie Space