2017/09/27: DevOpsDays Detroit Day 1

by Gene Kim on

#devopsdays

2017/09/27: DevOpsDays Detroit Day 1

Up: Dr. Richard Cook (@ri_cook)

  • NTSC 30 FPS
  • splitters and lumbers
  • @ri_cook: “paraphrasing Woods: as complexity of system increases, our knowledge of system decreases”
  • @ri_cook: “Learning gets truncated at organizational boundaries”
  • @ri_cook: “Conroy Law (haha): after accident inquiries model the org factors that produced the event” (inquiry models org causes)
  • @ri_cook: “The curse of devops: continuous deployment requires continuous scrutiny” (always poised to deploy)
  • @ri_cook: “For past 30 yrs, my colleagues & I studied systems; military; surgery; transportation; now tech systems”
  • @ri_cook: “Resilience is now buzzword; like Lean; same as accountability, visibility; widespread use of them diminishes value/clarity
  • @ri_cook: “Woods desc 4 facets of resilience: 1) rebound 2) robustness 3) graceful extensibility; 4) sustained adaptability
  • @ri_cook: “- @RealGeneKim:
  • RT @bsnugent: #devopsdays 4 concepts of resilience @ri_cook https://t.co/FzU2MHIM2h
  • @ri_cook: “Some of you know John Allspaw @allspaw; he’s looks like he’s 19 years old” hahaha
  • @ri_cook: SNAFU Catchers: “an inquiry oriented consortium”
  • @ri_cook: “SNAFU: capturing the notion that normal condition is broken, but it’s broken in a familiar (normal) way
  • @ri_cook: “PBY-5:flying boat rescue downed WWII airmen in Pacific Ocean, init designed for reconaissance
  • @ri_cook: “Making use of the PBY’s unique capability req a whole organization deliberately tuned to the mission
  • @ri_cook: “Entire org had to be created: construting runways, fueling infrastructure, tasking, scheduling
  • @ri_cook: “People go to museum to see the PBY-5 plane, but the magic/excitement/innovation is in the org that supported it
  • @ri_cook: “Amazing story: US submarine sank; sailors climbed onto PBY-5 plane, too heavy to take off; used it as a life raft, waiting for rescue
  • @ri_cook: “Report from STELLA SNAFUcatcher Workshop coming out soon
  • @RealGeneKim: #devopsdays https://t.co/ly21PoriFo
  • @ri_cook: “Tech debt vs. dark debt; above the line vs below the line
  • @ri_cook: “
  • @dlaulusa: Dr Richard Cook says that we, in DevOps, are "gearheads". @devopsdaysdet #devopsdays https://t.co/2jIUz0N599
  • @ri_cook: “The line of representation: in this case: above line, activities of people: actions, speech, clicks, signals
  • @ri_cook: “I study cognition: the thought bubbles that result in the observed actions: eg: why, what is happening, what does this mean?
  • @ri_cook: “I.e., what matters, and why does what matters matter?”
  • @ri_cook: “Technologists tend to be orders of magnitude better at ‘below the line’ (the tech) vs. above line (actions, cognitive models)
  • @ri_cook: “The shock: there is nothing below the line: all we have is a representation of the system, it’s not the system
  • @ri_cook: “You can’t know the entire system; even assertions of what kernel version, etc. it’s changing all the time
  • @ri_cook: maybe what he’s suggesting that digital complexity so far outstrips
  • @ri_cook: “We only have impoverished models of the systems that we are working on”
  • @ri_cook: “Major coping mechanism: post-mortems: failure is because our perception of reality is incorrect
  • @ri_cook: “The purpose of a post-mortem is not to improve the system; it’s to improve our understanding of system as it exists in reality
  • @bsnugent: Brilliant keynote from @ri_cook "As the complexity of a system increases, any agent's model of the system decreases… https://t.co/8mKjIylni2
  • @botchagalupe: I want to shout out an "Amen Brother" right now... @ri_cook - below the line is hidden... doesn't exist... #devopsdays...
  • @RealGeneKim: #devopsdays The post-mortem arc https://t.co/9v4MndBzSn
  • @ri_cook: “Often OH in PMs: I had no idea the system worked that way, everything is so brittle, incr dependence on others & their stuff
  • @RealGeneKim: #devopsdays @ri_cook: areas of hope: https://t.co/G5DAmZ6WqW

Up:

  • Why do we make more mistakes when things ar moving fast?
  • eye tells you that bottom line is shorter, but brain is overriding your answer, b/c it knows the right answer is that they’re same
  • Kahneman: considered the founder of behavioral economics
  • only Nobel Prize for psychology
  • Planning Fallacy: plans & forecasts are unrealistically close to best-case scenario; can be improved by consulting past data
  • @RealGeneKim: #devopsdays Scottish Parliament Building; 10x initial cost estimates https://t.co/sPxUpqU1Vw
  • @RealGeneKim: #devopsdays Software estimates are often way off: Healthcare dot gov: initial cost estimate was $95MM; actual: $1.5B https://t.co/arQRmqy19D
  • @bsnugent: #devopsdays Amin Yazdani talking about the psychology of estimation with lots of references to work of Kahneman and Tversky
  • @RealGeneKim: #devopsdays System 1 is always on; System 2 defaults to off, requires energy, focus, turns off after problem is sol… https://t.co/GJLWKgS5tk
  • Yazdani: Force 1: information availability: what you see is all there is
  • Yazdani: Force 2: prediction by association
    • even though probability is correct, System 2 is so lazy, it defers to System 1; Tom W psych profile is irrelevant
  • Yazdani: Force 3: anchoring
    • asked “what % of African nations in UN?”
    • wheel of fortune experiment: wired to only show 10 or 65
    • grad students who saw 10: avg guess 25%; those who saw 65, avg guess was 45%
  • Yazdani: what you will hear:
    • that will slow us down; just give me a ballpark estimate; i can’t wait until we have all the details
  • Yazdani: Countermeasures: more detailed user stories; definition of done; acceptance criteria
    • these are ways to kick in System 2, to cause more deliberation
  • Yazdani: Prediction by association: what goes wrong: estimate multiple tasks in a row; not looking at data
    • Yazdani: countermeasures: compare historical actual vs estimates, causes System 2 to kick in; similar projects/tasks
  • Yazdani: Anchoring risks: “I think we should be able to do it in a month”; you are being anchored, introducing bias into your estimate
    • Yazdani: countermeasures: planning poker, estimate separately before the meeting, randomize order of user stories (don’t do easy first! Ha!)

Up: Avleen Vig: Adding and Retiring Tech in Your Stack (congrats on the new gig at Facebook!)

  • just started at Facebook
  • Vig: Example scenario: moving from Pregresql to YourSQL: enumerate prob & scope; realize the scope is huge; it needs to happen sooner than you think; estimate time and tests; then triple them
  • #devopsdays Vig: Example scenario: Pregresql > YourSQL: estimate time and tests; then triple them; almost always much diff than we think
  • Vig: “how many people still using Graphite in your infrastructure?” (5 ppl); “new tech like Prometheus has many advantages
  • Vig: “nothing against graphite: it’s just an example of the cost and opportunity cost of using tech that’s 10+ yrs old”
  • @edyesed: 👏nice👏to👏see👏this👏said👏out👏loud👏: humans need to leverage composition to reason about complex work. : Even if u don… https://t.co/zAyzLfs6Av
  • @RealGeneKim: #devopsdays Vig: how to define a problem associated with old tech https://t.co/sqexsNhiJ0
  • @RealGeneKim: #devopsdays Vig: and evaluating the new possibility… https://t.co/fTBWv2qZz4
  • @RealGeneKim: #devopsdays: Vig: fictitious strengths/weakness of tech: https://t.co/2hzwue0d85
  • Vig: making the switch; design; what currently works, keep those features;
  • Vig: Gaining consensus: early conversation; architecture review; production launch review
  • @RealGeneKim: RT @botchagalupe: Least we not forget Senge #devopsdays https://t.co/GAnp3KC7Hb
  • @RealGeneKim: RT @botchagalupe: Also good follow on to Kahneman #devopsdays https://t.co/ljffZ7odv6
  • Vig: removing old technology: assume everything needs to be replaced; plan for obsolesence; be are of emotional ties;
  • Vig: Hard to plan to replace centralized databases;
  • @RealGeneKim: #devopsdays Vig: an example of a burn down graph, showing reduction of calls over time to zero, as it was replaced https://t.co/v9xkNSl7YY
  • Vig: “just remember, removing old stuff is actually much easier than adding new stuff” (???)
  • Vig: “by the way, I’ve been using Nagios since 1999, back when it was called NetSaint.. I keep deciding that switching from Nagios is not for me”

TODO

  • does RT not pull into notes?!
  • Pull into notes: put in new line and “- “
  • find way to display t.co pictures: @RealGeneKim: #devopsdays @ri_cook: areas of hope: https://t.co/G5DAmZ6WqW
  • would be super great to have tweet mentions in tweet stream
  • when tweeting, trim off left spaces: “ -“
  • Command-S to save scribe; I’d use it all the time; and Command-E to get into Editor