Mr. Paul Croll, VP Technical and Conference Activities
At 25th year, this is a long running conference. Air Force lost funding for the program last summer, IEEE took over stewardship
Mr. Lloyd Moseann, former Asst Secretary of the Air Force for Acquisition (Communications, Computers and Support Systems, and STC founder)
Moseann: "I'm a bit of an antique; at the 1st conference, it was all government; now it's all industry"
Moseann: 12 yrs ago: 1988: different world: "gallon of gas 98 cents; sw engineering non-existant; not many cell phones"
Moseann: "Dept of Defense has extremely high reliance on software; now $1.5 trillion portfolio value annually
Moseann: "1985: birth of Software Tech Conf: 7/10 weapon systems in trouble b/c of software: projects late b/c sw as art
Moseann: "NATO coined term: 'Software Crisis': projects over budget/schedule/inefficient/low quality/or never delivered"
Moseann: "2nd STC: 4/19/1989: Pentagon told world: C-17 program over budget/schedule b/c software; plane is ready, but no sw
Moseann: "Fear was C-17 would be on runway, but still waiting for software to load on it; major factor was cultural/sociological
Moseann: "You can't see, touch or feel software; made it difficult to convince generals of importance" (no diff than now w/CEOs!)
Moseann: "1980s was a lost decade for software dev; wondered back then whether 1990s wd be better: thot 're-use' was ray of hope
Moseann: "Pentagon wasn't concerned abt sw cost: instead, it was on predictability: could orgs ever deliver on their promises?"
Moseann: "2002: stated world of software was still governed by faith based mgmt, bumper sticker mgmt
Moseann: "'best practices' does not exist with defense contractors; trying to convince that contractors are well trained"
Moseann: "1991: Paul Strassman: '#1 priority is to turn software from cottage industry into modern industrial method of production
Moseann: "2012: Joint Strike Fighter: 24MM lines of code; sw has delayed testing/training & added costs; capabilities being delayed
Moseann: "Apparently, Air Force things software problem is fixed, since Bold Stroke program has been canceled
Moseann: "We must acquire product as well as process; we need transform doctrine and policy into reality in the acquisition & development of complex systems dependent on software"
Moseann: "Must happen first at Military Dept Level, and again at Acquisition Program Manager Level: and must include Architecture"
Moseann: "Architecture Driven programs: embedded in acquisition and RFP process: Army Integrated Battle Command Systems; AF Joint Mission Planning Systems; Navy Common Link Integration Processing; Navy DDG-1000; Army Warfighter Information Network Tactical
Moseann: "Most hopeful: Army's Strategic Software Improvement Program (ASSIP): in 2010, mandated software architect in every program
Moseann: "Let's hope that there's enough software architects out there"
Moseann: "from SEI: 1/3 of reporting orgs failed to have CMM Level 3; 58% USA software orgs > 3 or above; 78% non-USA > 3 (!!)
Moseann: "5x are many CMMI Level 3 or above are outside of the US"
Moseann: "It saddened me to hear friend say, 'CMMI no longer has traction in OSD, you shouldn't even talk about it"
Moseann: "Orgs, govt & mgrs must recognize difference betw Performance they value & Discipline needed to achieve results
Gretzky: "What helped me most in becoming a good hockey player was learning to skate to where the puck was going, rarther than where it was or where it had been"
Boehm: "Reflection in action" or asking how could we have done our last project better" is actually merely optimizing bureaucracy
Boehm: "8 surprise free trends: increasing integration of syse and swe; user value focus; software criticality and dependability; rapid, accelerating change; distribution, mobility, globabliztation; complex systems of systems; COTS and open source and reuse; computational plenty"
Boehm: "Wild cards: autonomy software; combinations and biology and computation"
Boehm: "Biggest 2013 trends missed in 2006: nanotech sesnors; search and mining of ultralarge data aggregations; multicore cips; cloud computing; social networking"
Boehm: "No longer any clear boundary between dev and ops
Boehm: "We can no longer pre-specify what we want. Doesn't match current DoD acq process w/master schedule & master plan
Boehm: "Worse, sched/cost overruns typically result in project cancellation; erasing all data on emergent processes/outcomes
Boehm: "Current university students will be practicing SysE into 2050s; so, we must teach them timeless principles
Boehm: "When deciding Agile vs. Plan Driven, ask 'where are the change hotspots?; think UI vs. dealing w/device drivers"
Q: "as engineer, I don't see contract until it's signed. how do I influence contract manager?" A: "DoD should do more competitive prototyping, to see what creative companies can do, instead of seeing what integrators propose; they're doing more draft RFPs, but not as much as they should"
Dr. Vic Basili: "Aligning An Org's Goals & Strategies Thru Measurement: GQM*Strategies"
Professor Emeritus of Computer Science, University of Maryland
Basili: "need to develop & connect goals to all levels of the org"
Baynes: "In this planning process, we're constituting the team, identifying any training, tooling needs, etc. Transition planning allows team to hit ground running on Day 1
Baynes: "I have 4 scrum teams working concurrently: 3 ceremonies: sprint planning, daily scrum, sprint review, sprint retrospective
Baynes: "daily scrums: what did you do since last mtg; what are obstacles; what will you do before next mtg?
Baynes: "managing backlog: individuals sign up for work; estimated work remaining updated daily
Baynes: "Our automated build process and continuous integration; every commit goes into CI, static code analysis, unit test databases for all sorts of Oracle versions and schemas, all unit tests run, code coverage analysis on unit testing; generate online help and API docs from comments in code; create CD image; then deploy to test clusters, update dashboards,
Baynes: "Sprint review and retro: I explain to my management, see working software; retrospective (went well, improve, develop actions & priorities)
Baynes: "Readiness: our Release or Hardening Sprint: finalize remaining tests (perf/load testing, regression; platform test is extensive); docs are finalized, prepare deployment CD and artwork; release retrospective is planned; QA manager leads our "total product readiness" process
Baynes: "Testing techniques: for section 508: take away mouse and see whether you can use software
Baynes: "Agile testing: test early, test often, automate
Baynes: "You'll spend 50% of dev cycles understanding reqs in order write testable software
Baynes: "It's better to get 80% of the features 100% done, than 100% of the features 80% done"
Baynes: "Transition strategy: Understand: it's important to first understand what it is that you're trying to change
Baynes: "Educate: educate them so they can break comfort for doing things status quot; Execute
Baynes: "Even US govt cited need for chg: 2012 GAO Report: cited $76B spent on IT that did not advance mission, recommends Agile
Baynes: "Even DoD CIO very concerned, asking all contractors & integrators to change; President Obama enacted
Hantos: "To deliver a 4700K KDSI ground system to support satellites is a 47 month project
Hantos: "Mission assurance risks composed of (mission risk: satellite blows up; program risk is cost and schedule)
Hantos: (Asserts Iron Triangle negates posited benefits of Lean and Agile; pretty sure he's wrong, despite FEARSOME intelligence)
Hantos: (Asserts that mission assurance is not valued by Dev; and that there's no fluff to cut. Wrong.)
this guy is fearsome debater, I think he's wrong)
Hantos: (citing all the non-functional requirements; "the Lean question: which ones do you cut? which ones don't add value?)
Hantos: "Sign on my dentist's wall: Brush only those teeth you wish to keep" (Haha)
Hantos: "Typical 3-4 year development process and min 5-10 yr long ops/sustainment for space vehicle req strong tools support"
Hantos: (fascinating watching practitioners and academics jump on Hantos' assertions)
Hantos: "Turnover in IT sector almost 2x higher than federal employees
Hantos: "When facing situation where employees are let go due to funding, extreme need for documention
Hantos: asserts that Agile values doesn't scale up for large projects. True? False?
Hantos: "JROC, DoD, Congress are all high-inertia orgs w/complex, bureaucratic processes for interaction"
Hantos: "It's unfortunate fact of life that when things don't go well, collaborative resolution becomes less and less feasible
Hantos: "Yogi Berra: if you don't know where you are going, you will wind up somewhere else
Hantos: "Recoverable vs non-recoverable software failures:
Hantos: "It's really bad when Congress legislates a software development process: guaranteed to backfire
Hantos: "In 1988, spiral development introduced, one of most powerful adaptive methods; 2003, declared as preferred methodology
Hantos: "Every govt contract submission insisted they did spiral dev; 5 yrs later, they took it out, b/c they learned what it was
Hantos: "Agile is an extreme adaptive process model: being pushed blindly
Hantos: "Govt is pushing for Fixed Price Contracts; pursuing agile development and fixed price contracts at same time is not feasible
Hantos: "The temptation to cut corners, even in the name of being efficient or expedient, is ever present, especially in a global business that is economically forgiving
Hantos: "Dr Wanda Austin, President CEO Aerospace Corp: "That is why 'getting it right' must be a 24/7 commitment"
Phyllis Marbach, Boeing: Case Study Of DoD Program Using Agile Software Dev Process
Marbach: "Introduction to agile, major defense acquisition program description, JTRS enterprise network management (JENM), contract specifics, systems engineering role, agile and earned value management, program reviews and contract requirements, metrics and productivity, continuous integration
Marbach: "plan review: srs, irs, sad, all plans; release reviews: hand over test scripts, etc.;
Marbach: "at Boeing project, we went from 2 to 3 week sprints to enable time for program reviews & contract documentation
Marbach: "organized sprints so that teams could complete enough functionality to enable testing
Marbach: "Major Defense Acquisition Program defined as: those that cost more than $365MM/year in 2000 dollars
Marbach: "JTRS JENM was a major project to enable unified network management for warfighter: IDIQ awarded 4/2010
Marbach: "Proposal written w/Agile mentioned: Had to support lots of legacy radios, field testing (in mud), lots of priority chgs
Marbach: "In contract: priority of work items may change, but not the requirements
Marbach: "807 requirements -> 1607 features
Marbach: "Systems Engineering Role: Test; SysE and SwTestE didn't work in iterations; received software at end of each sprint"
Marbach: "SysE Role: Deliveries: SwE maintains reqs, design & test scripts within iterations
Marbach: "Must get infrastructure set up in Sprint 0/Planning: infrastructure, dev & continuous integration test env" (Yes!)
Marbach: "At Boeing, we added a 4th member to Agile team: technical owner, in addition to product owner"
Marbach: Q: "this obviously has hardware, where does it fit?" A: "we tested at end of each sprint in lab"
Marbach: Q: "how did you prevent testers from being left behind?" A: "we'd enforce testability of UI to database and back; at first, we had 3 weeks to qualification tests, and finished a day early, b/c we were finding/fixing as we go
Marbach: "Agile and Earned Value Management: each software release was a milestone deliverable; progress reported weekly on % complete story points for each release's planned total story points
Marbach: "Story points are estiated effort to copmlete a backlog or user story; business value is in the completion of a feature
Marbach: "A release might have a partial feature delivered that will be finished in the next release
Marbach: "hybrid Agile: after PER, your design and how you satisfy infosec requirements; forced many team members to decouple from team to satisfy security testing requirements via detailed design; then went back to Agile
Marbach: "This was an example where three letter agency tested design, traceability requirements
Marbach: "when you have heavy information assurance requirements, maybe you can't do iterative building;
Marbach: "You need to have architecture and modules, which enables iterations; between PDR and CDR; then customer changed everything; but with prioritized backlog, we could absorb it
Marbach: "Used Version ONe;
Marbach: "We hit the schedule, and we had assurance based on velocity reporting that we were going to hit the date
Marbach: "Customer attended demos every 3 weeks; the more we showed, the more showed up
Marbach: "Challenges: learning the process (recommended training b/c it created champions), planning work, maintaining the design, too many conflicting stakeholders, customer waterfall schedule, defining the backlog to trace the requirements, Agile and EVM
Marbach: "What did they like? liked deliverable product every 3 wks, do builds quicker, agile testing rather than whole thing at once, keeps everyone accountable, build relationships w/our suppliers, stakeholders impressed with demonstrations
Marbach: "What would you change: reduce the amt of change, maintain the design betterc
Marbach: "Biggest surprise: how rigorous it was, it's working,
Marbach: "Our customer kept changing requirements, using Agile as the rationale of why we could absorb them" (haha)
Suzanne M. Miller & Mary Ann Lapham, SEI: "Ready & Fit: Understanding Agile Adoption Risks in DoD & Other Regulated Settings
"DoD is not the only regulated setting that constrains approaches; compliance du jour is everywhere"
"Other regulated envs: Dept of Energy, FDA, Financial Services (pharama & medical devices), healthcare records, international trade
"Probs: inherently conflicting regs:
"Regulatory ghosts in the system (still cutting off ends of roasts: becuase mom did it, not knowing it was because roast was bigger than the pan
"Regs often have unwritten laws: written law protects you & persecute those who violate unwritten law (eg, don't trust contractors)
"Problems of the past become the regulations we have to live with today"
"Are we fit? Meaning, do we have the culture and processes to get things done?"
"Why do I care about Agile? B/c methods are based on explicit set of principles: incr learning, trust & roles working side by side