2017/11/13: Day 1: DOES17 San Francisco

by Gene Kim on

#Does17

2017/11/13: Day 1: DOES17 San Francisco

Scott Prugh, Erica Morrison

  • @scottprugh: “CSG: 35 years serving world’s most respected CSP; 3300 employees”
  • @DOESUSA: What is @csgi? @ScottPrugh sharing now live at #does17 https://t.co/oaAUDRajSi
  • @scottprugh: “
  • @electriccloud: RT @DOES_USA: Presentation slides available each day on Dropbox and GutHub! 👏🏻👏🏻 #does17 https://t.co/vS9za0qQkL
  • @scottprugh: “Mode 2: servers/apps run safely w/speed & quality; tech debt is actively managed; be waiver free”
  • @scottprugh: “Mode 1: get rid of all the other stuff, servers destroyed in the parking lot with a sledgehammer”
  • @scottprugh: “For 5 years, we had to deal with legacy servers that would take 4 hours to reboot; 4 hours to patch/reboot; dangerous! Late nights!”
  • @scottprugh: “We went thru 48MM subscribers in 2012 to 61MM in 2017; 750 TPS to 4000 TPS; incredible capacity growth
  • @scottprugh: “2012: 1495 incidents/month; then by driving service ownership, by 2017, 633 incidents/month; build it/run it
  • @scottprugh: “We have 26 people from CSG here; thank you for being on this journey together!”
  • @RosalindRad: Service owner model @ScottPrugh bringing teams together #DOES17 https://t.co/wkyFOX4ZC9

- @scottprugh: “Even the necessary batching all the changes into the periodic ITIL CAB worsens outcomes”

  • @peterscraig: Collapse teams to get feedback, but realize that fixing is moving forward says @ScottPrugh #DOES17 https://t.co/1VpS3VMRFt
  • @scottprugh giving kudos to @jonhall about his work on swarming in ITSM
  • #Does17 @scottprugh giving kudos to @jonhall_ about his work on swarming in ITSM
  • @paula_thrasher: @ScottPrugh presenting a better CCB/CAB model - be decentralized but automated change management - not meetings #DOES17
  • @ericamorrison: “in Dev, we were often blissfully unaware of production issues; no more; thru After Action Summary, blameless review
  • @ericamorrison: “
  • @DOES_USA: @ScottPrugh reviewing the Swarm Model 👍vs 3 Tier Support Model 👎#DOES17 https://t.co/mdBb8JZBig
  • @ericamorrison: “Spotlight: Team handling load balancer: IaaC, created synthetics, small batch release cadence, evolving to self-svc
  • @ericamorrison: “Load balancer changes could take 6 hours: lots of clicking, complex non-standard changes”
  • @ericamorrison: “Rollback is great; but for global infrastructure chgs, often not obvious what we should rollback to!” (Ha!)
  • @ericamorrison: “Big problem: in middle of big deployments, teams kept asking ‘how’s it looking?’ Driving towards self-service”
  • @ericamorrison: “A great metric: how much sleep I get when we do load balancer changes; in beginning, never slept thru night, constantly checking

Up: Stephanie Gillespie< SVP, Digital Retail Delivery; John Rzestzotarski SVP, Cont Delivery/Feedback @rzesz, @gilles22

  • @gilles22: “Key Bank: 15 states; 1200 branches; 20K employees; $135B assets; $5B revenue”
  • @rzesz: “We never fired anyone at Key for adding another layer of security” (Ha!)
  • @rzesz: “2015: one user logging in required 225+ network hops between two data centers”
  • @rzesz: “Digital 17: started in 2015, designed to be a 2 yr project: more than ‘create online experience that didn’t suck’
  • @DOES_USA: What it looks like on the backend when a @keybank user logs in 😳😳 @rzesz #does17 #devops https://t.co/y3gQqYXVkV
  • @gilles22: “The goal was to entirely redesign digital experience for 2MM+ custs; allow experimentation, fast/cheap; changed everything”
  • @gilles22: “Then acq First Niagara; 24 month project to 17 months! Digital17 will now ship in Q2 2016
  • @gilles22: “Speed was everything; req new decision making process; answers needed within 24 hours”
  • @rzesz: “left #does15 w/3 takeaways: exec support; metrics driven / MTTR; focus on removing bottlenecks
  • @gilles22: “Couldn’t wait for a top-down initiative; we purposely picked where to start; automated testing, containers, CI/CD
  • @gilles22: “
  • @rzesz: “I love term ‘containergeddon’: they will take over the world..” Haha
  • @rzesz: “We didn’t use Kubernetes to built PaaS and help devs; we did it for reliability; Google is always up; we want that too
  • @rzesz: “Netflix Hysterix helps protect us from all the services we rely on, like MC/Visa and lots of svcs that every bank uses”
  • @gilles22: “Legacy testing took 20 hours; went down to 18 min”
  • @gilles22: “Day of launch; UX miscue; users kept locking themselves out; 2+ hrs in call center; social media.. but tech was working fine, cust very unhappy
  • @gilles22: “a pround moment: Beth Mooney called out DevOps as key to success” (Wow!)
  • @gilles22: “Shifting our mix of contractors vs. employees; not just looking for devs; looking for end/end engineers: deployment/containers/proxy
  • @rzesz: “New CTO: we need generalists: Linux ppl: use mouse; Windows ppl: use command line” (Ha!)
  • @Valien: lol. New CTI at @KeyBank: Windows engineers now using a keyboard, and Linux engineers now using a mouse. #DOES17 @rzesz
  • @rzesz: “We have 8am daily post-mortem; not about blame; almost entire tech org dials into this call; very popular; created converts

  • Again, congrats to @gilles22 and @rzesz for that amazing Beth Mooney quote!!!!

  • @RealGeneKim: That amazing quote from Beth Mooney, CEO of KeyBank, from @rzesz @sgilles22 #does17 https://t.co/USXBsx4Xun

Up: Scott Nasello, Columbia Sportswear (@scottnasello)

  • @ravi_kalaga: Yes, DevOps is a game-changer! Mad money says it as well. #liatrio #DOES17 https://t.co/vSfBUgPNx4
  • @scottnasello: “we’re unlikely place for DevOps: Windows shop; bought our software; VMware; no budget; no mandate
  • @scottnasello: “Founded 80 yrs ago; values innovation; almost ran out of money, so incredible financial conservatism;
  • @scottnasello: “Here’s the platforms team: support day/day, break/fix; storage, load balancer, Windows admins; change is coming
  • @scottnasello: “Let’s find a win/win: let’s do something great for your career & help Columbia Sportswear too
  • @scottnasello: “As an engineering org, we forgot how to engineer; we have 100 pg install docs w/screenshots; single-threaded; everyone is Brent
  • @scottnasello: “Great book: Turn This Ship Around; Captain Marquet”
  • @scottnasello: “Marquet: dangerous combo; a crew who follows order and a captain who doesn’t know anything” (Ha!!)
  • @scottnasello: “Our strategy: grow talent we have; happy talent attracts more talent;
  • @scottnasello: “Army taught me; Be a servant leader, adaptive learner, role model; Know yourself/people/profession; do right things
  • @scottnasello: “2016: threw away all year end review: new Qs: what & how did you learn? how did you learn/teach others?
  • @DOES_USA: RT @matthewjhoward: #devops is surely main stream when @jimcramer talks about it on TV and highlights a 100 year old bank as the poster chi…
  • @DOES_USA: RT @matthewjhoward: #devops is surely main stream when @jimcramer talks about it on TV and highlights a 100 year old bank as the poster chi…
  • @scottnasello: “500 artifacts in version control; amazing for org that used to email scripts to self, save script.bak, etc
  • @scottnasello: “started using Powershell DSC, despite WYSIWIG basis; why? My team didn’t have dev background” (thx to @jsnover!)
  • @scottnasello: “Over 1500 servers put into version control, automatically provisioned”
  • @scottnasello: “Data center move: 19 racks of gear; no downtime; 655 servers moved”
  • @scottnasello: “Rotation caused everyone on teams to constantly have beginner mindset; weekly retrospective; prevents Brents
  • @scottnasello: “ChatOps for SAP BASIS team to self-service monitor health of their environments” (Wow!)
  • @DOES_USA: .@Columbia1938 lessons learned: We moved from confident and complacent to being more skilled at change… https://t.co/qje25lcV6x
  • @scottnasello: “this is a once or twice in a career opportunity to make changes of this magnitude”

Up: Dr. Steven Mayner (@stevemayner)

  • @stevemayner: “Kotter: 70% of change initiatives failed; Smith: 19% of change initiatives involving culture fail” (uh oh)
  • @stevemayner: “
  • @Josh_Atwell: “Orgs today exist in one of the most dynamic times in history” “pace of change will accelerate, not slow down”
  • @stevemayner: “Change often leads to anxiety: will I make it? Will I be able to provide for my family? Now we’re at bottom of Maslow pyramid: survival mode
  • @stevemayner: “Transformational leader: characteristics of biography of JFK, WWII combat photographer;
  • @stevemayner: “What’s amazing about 2017 State of DevOps Report: actually found predicative capabilities of Transformational Leadership
  • @stevemayner: “
  • @RebeccaFitzhugh: This. No one is a "natural born leader." Leadership is a skill that is taught and honed with practice. https://t.co/mZeuwvwb5Y
  • @stevemayner: “Often most resistant group to change is middle-management; most accountability, but least authority to lead change
  • @stevemayner: “...often least confident in their ability to change to new ways, eg, Agile, DevOps;
  • @stevemayner: “Middle managers: do they direct work? No; define work? No; do the work? No; but they’re still valuable
  • @stevemayner: “Middle managers have institutional knowledge, have relationships inside/outside the company; know where skeletons are buried;
  • @stevemayner: “Experience never gets old” (Ha!)
  • @stevemayner: “Leaders have little do with your title; leaders are needed at all levels of the organization”
  • @stevemayner: “I heard, ‘this is the first time that I ever felt the company cared about me.’”
  • @stevemayner: “Results were so successful, merged with CSRA; rated #1 provider of services to Fed Govt”

Up: Robin Yeman, Fellow, Lockheed Martin; Suzette Johnson, Fellow, Northrop Grumman

  • Johnson: “2005: program select/process/disseminate data in physical situation awareness to our analysts/warfighter; security req
  • Johnson: “125 engineers; 10 small agile teams; each building a component; big integration challenges; each change breaks things
  • Johnson: “Now running 15K automated tests/day; never satisfied; trained everyone on how to build quality in; make goals clear
  • Yeman: “2002: customer required us to Agile in RFP; less than 1 yr after Manifesto was signed; 8 teams of 10; read lots of books
  • Yeman: “when customers asks you do do something in RFP, you do it. I decided to learn about Agile immediately.”
  • Yeman: “If you can spell something, like Agile, you are now the subject matter expert” (Ha!)
  • Yeman: “Over the years, helped support AEGIS radar/weapons control, F-35 program, Orion heavy rocket launch
  • @colinwfletcher: Really impressed w/real #DevOps collab spirit at #DOES17 ->leaders from competitors presenting together - love it! https://t.co/9aNANDYTqe
  • Johnson: “In large org, we don’t want teams to kep building things over and over again; thus the COE, to enable, not be impediments
  • Johnson: “over the years, we’ve trained over 15K people: engineers, managers, and leaders”
  • Johnson: “one of our goals: get teams able to be up and running immediately after an RFP is won and awards;
  • @Carter_McHugh: "Build more champions than you think you need. Define the path for growth" #DOES17 https://t.co/Ndx0vo5wYp
  • Yeman: “DevOps Handbook is now one of my favorite books” (Woot! :)
  • Yeman: boring, daily grind, people always asking them, are things working? What do you think you should do next?
  • Yeman: “we started measuring employee happiness; some ask ‘why do we care?’; they’re building our products; we need them engaged”
  • Yeman: “We need psychological safety; otherwise, people won’t learn and won’t extend themselves
  • Yeman: “