Discussions with DTRA: Episode 2

Episode 2: Agile Development at DTRA

Length: 25:59

DTRA is transforming how it acquires and manages the development of capabilities to keep up with quickly evolving technologies and warfighter requirements. This is especially true for software development and Artificial Intelligence/Machine Learning/Data Science efforts. Acquisition, Contracts, and Logistics Directorate is taking the lead to facilitate adopting the agile mindset and culture change at DTRA.

Interviewer:
Michael R. Howard
Chief, Acquisition Systems, Training and Support/PM/COR Acquisition
Management Department Defense Threat Reduction Agency

Interviewee:
Ennis C. (Jim) Whitehead III, Ph.D.
Professor of Systems Engineering
Defense Acquisition University

Public Affairs Facilitator:
Darnell P Gardner
Public Affairs Specialist
Defense Threat Reduction Agency

Transcript

You bring the joint force. You bring the users in to talk to the RD folks because that's an Agile principle, right? Getting feedback.

Welcome to Detect, Deter, Defeat with DTRA, where the Defense Threat Reduction Agency brings together subject matter experts to discuss meeting today's challenges of WMD and emerging threats, increase awareness, and deliver morale boosting information. And now today's show.

Greetings and welcome to Detect, Deter, Defeat with DTRA. I am Darnell Gardner from DTRA PA, and I will be facilitating this event. Today's podcast host will be Mr. Michael Howard, Chief for Acquisition Systems, Training and Support at DTRA, and he will be accompanied by special guest speaker, Dr. Jim Whitehead, Professor of Systems Engineering at the Defense Acquisition University. Our topic for today will be Agile Software Development. Gentlemen, take it away.

Darnell, thank you very much for that intro. Yes. I'm Mike Howard with the Acquisition Management Department and I'm sitting here with Dr. Jim Whitehead. And before we get started into the meat of the subject, Dr. Whitehead, would you share more of your background with us...

Sure [crosstalk 00:01:30].

With our audience?

Happy to Mike. I appreciate it. I appreciate the opportunity to be here. Yeah. I spent 15 years. I've been at DAU for three years as a Professor of System Engineering. I teach the IT courses and DAU hired me because I spent 15 years at the National Geospatial-Intelligence Agency as an Acquisition Professional in IT, in program management, in engineering, and in IT operations. And so as a result of my experience there, DAU hired me and I've enjoyed teaching for the last three years at DAU.

Thank you very much for that introduction. Why we're here today talking about Agile, is DTRA is on a journey, at this time, of pursuing artificial intelligence, machine learning, and data science as a way forward with our mission set and from what I've gathered, in my readings and studies, that Agile is a way to do better software and therefore, it's the best approach for getting to where we want to go with artificial intelligence, machine learning, and data science at DTRA. We're trying to develop this Agile mindset. However, I'm not sure everyone at DTRA knows what Agile is or is about. Although, we have many software development projects going on right now, I think we need some help. And that's why we're here with you today. So, my first question to you Dr. Whitehead is what is Agile?

Agile is really a set of principles and best practices. It was established originally for software development, but it can be used for project management as well. Those principles and best practices. The bottom line on Agile is if you're doing software development and you're not doing Agile software development, you're doing it the wrong way. You should only use Agile principles practices because it's the best practice. It's the best practice for software development and you should be using it. So Agile is an iterative approach where you're delivering valuable software on an incremental basis every two to four week sprints, every two to four weeks, you deliver software. You demonstrate that software to the users. You get feedback on what they see, whether it meets their needs, doesn't meet their needs. Do they need to make adjustments? Do they have additional requirements that they add on based on those reviews?

So, it's iterative, it's incremental and it's constant feedback. And that's the beauty of Agile. When I first came to NGA, I was supposed to be certified at Program Management. They sent me off to DAU to take classes, to be Certified Program Management. And I learned all about Waterfall and that's the development we've been doing before. Even for software before Agile came along and Waterfall, we would gather up all the requirements and we'd send it off to whoever won the bid for that particular system. We'd get the system requirements, we'd send it off to a Lockheed Martin or to General Dynamics. And they would take it. In 18 months or even two years later, they would deliver the system. And then we'd show the system to the users. In our case at NGA, was imagery analysts. Generally, a lot of these systems were to help imagery analysts do their development of intelligence products based on the imagery that our satellites take.

So, we would present the system to the imagery analysts and they'd say, "It's not what we want." And we'd say, "Well, that's what you asked for." And they said, "Yeah, two years ago, we asked for it. The world has changed in two years. That's not what we need anymore." That's what I learned at DAU, the Waterfall. And Waterfall is still being used for major programs like F35 and so on as it was being developed. But for software development, you should use Agile principles because NGA learned that and NGA pivoted from Waterfall for software to Agile for software. And with Agile today, in fact, when we got started... When I learned about Agile, I said, "Hey, we need to send imagery analysts down to Plano, Texas where Lockheed is developing this new system for them. And we need those image analysts to talk to the software developers and let them know, is this what they need and so on. Are they developing what they need?"

And everybody looked at me like I had three heads. Whitehead, go back in the corner, forget it. Our imaging analysts are too valuable. Too valuable to waste their time talking software development, talking to software developers. And then, they realized that, no, we weren't providing systems in a timely fashion. We weren't providing systems that the users wanted, we needed to change. And so NGA did change. They reorganized their software development organization into four, what you would call PEOs with a senior executive in charge of each one, but they also assign a senior executive for each of those PEOs from the analysis department. And they brought their staff with them, their image analyst staff with them to be those product owners to go down and talk to the software developers. So, NGA went 360 and as a result, software development has a much better approach. NGA pivoted to Agile to fix the problem they were having. Takes too long to do acquisition and was delivered. It wasn't what the customer needed.

Excellent. Excellent. Well, you've actually touched on my next question a bit. So, I'm going to be agile and pivot to my next. What are some of the key guiding principles of Agile?

Agile best practices talk about developing software, small teams that work together in the same room, hopefully, in the same room where they can help out each other. Face to face communication is a big deal with Agile principles. Small teams. They also talk about doing things in sprints. In other words, two to four week sprints that you focus for two to four weeks on part of the problem. And you work on that part of the problem. And at the end of those two to four week sprints, you actually do.

And let me step back a second. Let me explain what happened at NGA and how I really learned Agile. I was called in by the Chief Acquisition Executive whose new boss, who just came in from the outside, was giving her a hard time. And he said, "Everybody is telling me acquisition takes too long. And when we deliver it, the systems we deliver are not meeting our users needs, not meeting our customers needs."

And so I know she came to me and said, "Hey, my boss said this," and I kept thinking, I've been here for 10 years. I've been thinking that the entire 10 years, but, okay. So, she said, "We need to improve the acquisition process." That was her job, right? The acquisition process. Use Agile principles to do process improvement. And that's where I learned Agile.

I became the product owner. A product owner of an Agile team is the government person who understands what the stakeholders need. And this is all part of the Agile environment. Product owner, government, sets the priorities, and helps figure out what's getting done next sprint versus the following sprint versus the following sprint, prioritize the requirements, prioritizes the work for the Agile team.

So, you have a product owner, government. Most of the time we contract out for Agile Software Development, right? We contract out to contractors develop. Used to be, we would select a company that understood, perhaps understood the system, perhaps understood the domain area. Today, that's not what you do with Agile. With Agile, you're hiring Agile Software Development capacity. And you're hiring that software development capacity with fixed price contracts. And you don't even have to do award fee or incentive fees because the incentive for those software development companies is to get the next contract or get the next task order. And so, you’re hiring capacity. And for example, at NGA, we had a contract for a company to provide 36 software developers, six scrum teams. Usually a scrum team is six to nine developers.

And, basically, their incentive. They had a one year contract with an option year for a follow-on. They did a good job, they got another year. If they didn't do good of a job, go to somebody else. That was their incentive. So, the contracting officers loved it, fixed price contracts, no incentive fee, lower overhead to take care of from a contract officer perspective. It was all positive. So, it's perfect for that type of thing, where you're the product owner. The government product owner says, "Okay, these three enhancements are most important. That's what we're going to do in the next sprint. These three level one bugs that we need to fix or defects that we need to fix. They're also important. We're going to do that in that sprint as well." And that became the sprint plan, to do these enhancements and fix these bugs, decided by the government.

And as things come up, maybe there is a major cyber threat that comes up that's been discovered for that system. That suddenly becomes a very important thing to work in the next sprint, for example, to solve that problem. So, that's the advantage of Agile, that you can react. You get feedback.

Let me continue the story of how it works. So, we decided on two week sprints and we really had nine day sprints because every other Friday, we would do our Retrospective, which means, and that's another Agile principle, continuous improvement. So, we would do what we called Retrospective. You might call it After Action Review, where we would ask three questions. What went well? What didn't go so well? And how can we improve in the next sprint?

And so we would meet on Friday morning and do that, do a Retrospective. Then we had something, we call a Review. It took me a long time to figure out what's a Retrospective? What's a Review? The review is with external stakeholders. And that's where we would present, in my case, to my boss and to other stakeholders, the Chief Acquisition Executive, we tried to pull in the Senior Procurement Officer who runs all the contract people, as well as leadership from the program management side. Tried to bring them in in these reviews and explain where we were. And then we would get feedback from them and they might say, "Yep, you're on track." Or they might say, "Well, wait a minute. I'm concerned about this particular thing. I need this other thing." And so we would take that and put it in our requirements list, which is called a Product Backlog.

And then we would prioritize it for a future sprint. And so constant feedback every two weeks. And then we would plan for the next sprint, based on that Review that we just had with the customers and stakeholders, and with the Retrospective, we did the After Action Review that we just did, so that we can make improvements for the next sprint. Every time we had a sprint, the idea is improve it. Every time we had a sprint, we might be adjusting based on customer feedback, stakeholder feedback. And that's the real advantage of Agile is you get the constant feedback and you have the ability to make adjustments.

That's great information. I'm sure our audience is learning a lot today. I know you mentioned the NGA example, but tell me other examples that if you're familiar of how Agile is being implemented within DOD, if you have some others to share with us?

Well, another example is, and this was when I was at DAU, I was asked to go do an After Action Review for a new program, the Integrated Visualization Augmentation System. It was a heads up display for combat soldiers and Marines because the Secretary of the Defense at that time felt that we needed to give our combat soldiers and Marines the best technology to give them an edge in combat. And the idea is, a heads up display would provide a lot of information for those soldiers and Marines that they didn't have today. And so, the Secretary of Defense actually went to Congress, mid-year, he got mid-year appropriations for $1.88 billion for this program. The largest probably mid-year program in the history of the Congress and a program office was born. And so the program office was working hard.

They did a review, one of the Deputy Secretaries of Defense asked them, are you doing After Action Reviews of your program? And the program manager was very honest saying, "We're too busy. We don't have time to do that. Everybody's full bore." And, so that Deputy Secretary of Defense called up the president of DAU said, "Hey, can you provide some people?" And they did and said, "Whitehead, get over there to the IVAS program and do an After Action Review." So, a team of us did. And what we found is this. First of all, in six months, they put 13 different contracts on contract, which is pretty good in the acquisition world.

Impressive.

Very impressive. So, they put 13 competitively sourced contracts on contract in six months. And, of course, the Secretary of Defense told them, "I want a prototype in one year." And so, the only way they can do that is quickly get people on contract and quickly get them working.

And in Agile, we talk about Minimum Viable Product. That's another principle, Minimum Viable Product. What do you have to produce at the minimum first, before adding things more? So, you figure out the Minimum Viable Product. Well, the Minimum Viable Product for the IVAS program was they had to have night vision and thermal sites because today combat soldiers and Marines have pieces of equipment they carry that give them thermal sites and night vision. And so, that was the minimum. That was the Minimum Viable Product. You need to produce that Minimum Viable Product in the first year. And so, they did that, but they also did the right thing. Microsoft won the bid to be the integrator for this IVAS system.

And so, the Defense Department sent soldiers and Marines up to Redmond, Washington, to sit down with the software developers to give them feedback on what they're developing. And, for example, developed a way to put Blue Force Tracker, which is an intelligence thing that's usually on a display. Blue Force Tracker, where all the good guys are. Red Force Tracker, where all the bad guys are. There were two intelligence displays and they put them on this IVAS, the heads up display. And they said, "What do you think?" And the soldier said, "Hey, I would like to actually see what's going on in the front. Can you move those displays to the side? And I can look right or left if I want to see Blue Force Tracker and Red Force Tracker, can you do that?" And the developer said, "Sure."

The developer didn't think about the fact that, yeah, I'd like to see the bad guy, the real world in front of me. And I can always look to the side, up or down, to see the intelligence information that's being displayed. That was an example of a user sitting down with a software developer, making things better. That's an example of an Agile Review, where you review with, as you go along, as you develop things, you review your progress with your stakeholders and get feedback. And in some cases, maybe this is wonderful. In other cases, it may be, "Hey, I'd like to be able to see the real world in front of me. Can you move it to the side?" "Sure. We can do that." Another soldier went up and said, "Hey, I have an idea. Can you put the sight of my N4 rifle in front of my display, regardless of where my rifle is pointed?"

And they said, "Sure, we can do that." And they did. That wasn't a requirement ever for the... Up to that point, that was never a requirement for the Integrated Visualization Augmentation System, the heads up display. But a soldier talking to software developer came up with a great thing because I've spent 33 years in the Army. I know how important it is that you don't always have time to bring your weapon up, to get a good sight picture before you have to fire. With a sight picture in front of you all the time, it may save lives. That was a soldier talking to a software developer getting feedback. And that's one of the principles of Agile, constant feedback.

Those both were great examples. Really. Fantastic. It sounds like Agile might be a silver bullet. However, does embracing an Agile mindset in culture represent a promise to solve all IT and program management problems?

Yeah. You brought up several good things in that question. First of all, you do need to have the right culture and policies within the organization to support Agile. Also, you need to have leadership who understands Agile principles, what Agile is, to support Agile in their area's responsibility.

I spent 20 years in marketing before I came to the Federal Government. And so let me market DAU a little bit. We provide senior executive courses. We tailor them from 45 minute, I personally have given, to new PEOs, from 45 minutes to four hours. Some PEOs have four hours to talk about Agile. Some only have 45 minutes. And we've tailored that training to them. So, one thing I recommend is if any leader wants to know more about Agile, DAU's happy to provide that type of training to meet their needs, to meet their timeframes.

We also have two and a half day training that we can tailor for DTRA anytime for your managers and other folks in DTRA that want to know more about Agile. And we're happy to provide that. We have open enrollment courses that people can come, or we can provide tailored training to DTRA, anytime you want it. You can just reach out to me or anybody at DAU.

But, going back to the fact is Agile a silver bullet? And the answer is, there was a study of 10,000 software development projects a few years ago, and that wasn't just DOD project, not just Federal Government, but also across industry as well. And the results of the study were that Agile Software Development projects were three times as successful as Waterfall projects. Three times.

And so, a database of 10,000 is certainly enough to say, to me, that, yeah, it probably is three times more successful. But is Agile a silver bullet? It's not a Panacea. 9% of Agile projects in this study failed. So, not every Agile project is successful, but Agile projects were three times as successful as Waterfall projects. And so, basically, my message is that if you're doing software development and you're not doing Agile software development, then you're not doing software development correct. If you're contracting for software development, you contract for those who are best at doing Agile software development, that's who you pick.

Sounds like a great plan and recipe for success. So, then I wanted to ask, are there IT and program management cases where Agile may not apply?

That's another good question and we actually talk about that in our course. If you have a project, well, with software, you probably should be using Agile regardless. But if you have a project that's fixed. That your requirements are fixed, they won't change. Perhaps you've already done a similar project before where everything is fixed. You don't need to talk to the customer. You don't need to talk to a stakeholder to do. You don't need feedback. You can just do the project. Then, perhaps, you don't need to use Agile principles and practices. So, it's not for every project, but if you're doing software development, yes. If you're doing other projects, if everything is fixed, you don't need feedback. Nothing change. The requirements will not change regardless. Then you don't necessarily need to follow Agile principles.

Okay. Great. Great. Thank you for that.

Mm-hmm (affirmative).

Well, lastly, I'm going to conclude with this question. What can our senior leaders at DTRA do to support Agile adoption agency-wide?

Well, I think they're already doing that. It sounds like when we came over here, you were talking about the research and development where you were trying to bring what I call war fighters, the joint force, whatever the term is today, at DAU we say, war fighter. You bring the joint force. You bring the users in to talk to the RD folks because that's an Agile principle, right? Getting feedback, showing them the development that's been done. It may not be software development, it may be other types of R&D. Showing them what you produced and getting their feedback. That's an Agile principle. You're already doing that and it sounds like leadership is already on board with Agile.

But with that, thank you very much, Dr. Whitehead, it has been quite a learning experience. I learned a lot and I'm sure our DTRA audience learned a lot as well. Thank you again.

Thank you. Appreciate it Mike.

Thanks for listening to Detect, Deter, Defeat with DTRA. To hear more podcasts, don't forget to subscribe on Apple Podcast, Google Play, or wherever you listen. You can find out more about DTRA at DTRA.mil