2016/08/09: DevOpsDays Portland

by Gene Kim on

#devopsdays

DevOpsDays Portland: 2016/08/09

  • Super cool. For people on-call, #devopsdays PDX has table in back w/power for people on call who get paged.
  • #devopsdays @kelseyhightower: Ops: "Go away, or I will replace you with a very small shell script"- RT @fuzzychef: "You know how I said that you make this event possible? That's a lie. It's actually sponsor money." #devopsdays
  • RT @jasonhand: "When you've been replaced by a shell script ... that's unfortunate." - @kelseyhightower #devopsdays https://t.co/dc8Lcyl9MO
  • Ha! You know you're at a DevOpsDays when "legacy ticketing system" is JIRA. @kelseyhightower
  • #devopsdays @kelseyhightower: "you know you're in an enterprise and you here, 'what, you let anyone push that button?"
  • #devopsdays @kelseyhightower is telling ultimate Ops horror story: updating and maintaining certificates. :)
  • RT @esigler: "In the future of operations youre going to have to move closer to the actual codebase" @kelseyhightower #devopsdays https://t.co/dsO8glwD2a
  • #devopsdays @kelseyhightower: "Future of Ops: put our expertise in the codebase; No Ops? No problem."
  • #devopsdays @kelseyhightower talking about his personal success stories in Dept of Transportation; "ghost stories of audit, security"
  • #devopsdays @kelseyhightower describing absence of PaaS like going to rental car agency w/o self-checkout, which is angry-making. :)

Up: @nellshamrell: History of DevOps

  • That moment when you realize you're so old that you can no longer deal with 11pt text on 12" screen
  • OMG. So funny. @nellshamrell citing server/sysadmin ratio from Visible Ops (2004) creating fears of mass unemployement. Awesome.
  • I remember how much controversy the stat of 100:1 server/sysadmin ratios created when we put that book out in 2004! @nellshamrell
  • - @wholemilk: Loving this history of industrial standardization, and really? Nothing ever changes! https://t.co/B4dU6imZJ1
  • @nellshamrell: Phoenix Project: "WIP is the silent killer" "In college, worked in pizza kitchen; broken oven meant lots of ruined piles of pizzas"
  • @nellshamrell: "even in DOD-STD-2167, waterfall was described as very imperfect, but no better way at the time"
  • .@nellshamrell: Schwaber on Scrum: "development interval is inherently unpredictable; must create ctrls to control variance; timeboxing, etc."
  • .@nellshamrell: Huzzah. citing Extreme Programming, one of less heralded contribs to modern sw development & DevOps; @kentbeck
  • @RealGeneKim: .@nellshamrell: Huzzah. citing Extreme Programming, one of less heralded contribs to modern sw development & DevOps; @kentbeck #devopsdays
  • @ericpassmore: Living through waterfall today it's still painful @nellshamrell #devopsdays
  • RT @gitbisect: .@nellshamrell: What is your hated #DevOps buzzword? respondant: DevOps! #devopsdays @DevOpsDaysPDX
  • RT @richburroughs: Shoutout to @patrickdebois and his 2008 Agile Operations and Infrastructure #devopsdays
  • .@nellshamrell: "a key DevOps mindshift change: Ops cannot own all the risk caused by change"
  • @esigler: @nellshamrell: "Ops and Dev have the same ultimate job: to enable the business." 100%! #devopsdays
  • @jasonhand: @nellshamrell 's talk on the History of #Devops should be required viewing/listening by anyone new to the movement. Well done. #Devopsdays
  • @thegrandyoni: Technology is easy, culture is hard. @nellshamrell #devopsdays https://t.co/IBgs8n67aR
  • @releaseteam: RT @ericpassmore: #devops bringing #kaizen back to software development #devopsdays @nellshamrell
  • @ericpassmore: We always aspire to work this way. #risk and #value owned by Dev and Ops #devopsdays @nellshamrell
  • @mikebarton: Surely... https://t.co/pQFL4IcXoe
  • @techlahoma: RT @michaelgorsuch: takeaway from @nellshamrell's talk: success comes first and foremost from the way in which your coworkers treat one ano…
  • @michaelgorsuch: takeaway from @nellshamrell's talk: success comes first and foremost from the way in which your coworkers treat one another #devopsdays
  • @ericpassmore: 2010 #devops takes off in Twitter. Did @nellshamrell predict peak #devops in 2016?! #devopsdays
  • Kudos to @nellshamrell on a wonderfully scholarly history of the bodies of knowledge that contributed to DevOps movement!

  • RT @richburroughs: @DevOpsDaysPDX I think @alicegoldfuss is totally free associating these intros and they are cracking me up #devopsdays

  • @richburroughs: @DevOpsDaysPDX I think @alicegoldfuss is totally free associating these intros and they are cracking me up #devopsdays

DevOps vs SRE

  • @DevOpsDaysPDX: DevOps vs SRE and obscure comic book characters with @otterbook #devopsdays https://t.co/D1eBBSzSBT
  • @ericpassmore: Entering the land of SRE , Site Reliability Engineering @otterbook #devopsdays
  • @otterbook: Link to Ben Treynor's SREcon talk: https://www.youtube.com/watch?v=n4Wf14e2jxQ
  • @ericpassmore: SRE answer the question, what does it mean to be up and healthy and what is a reasonable goal? I live this everyday @otterbook #devopsdays
  • @otterbook: Agreed that Ben Treynor defined a superb self-correcting system that balances agility and taret reliability
  • - @ericpassmore: SRE and #devops Shared love for monitoring, automation, shared responsibility @otterbook #DevOpsdays @ericpassmore: Brilliant, overflow in system outages events SRE team escalates to devs teams @otterbook #devopsdays
  • @jasonhand: "Error Budgeting" is hands down the most intriguing concept in the @OReillyMedia Site Reliability Engineering book #SRE #devops #devopsdays
  • @jasonhand: "You can't fire people to make your service more reliable. Keep the postmortems 'blameless'." - @otterbook #devopsdays
  • @otterbook: Key SRE principle that every DevOps should embrace: "we must engineer our problems away"; yes, like @scottprugh!
  • - @esigler: RT @ericpassmore: Blameless post mortems because you can't fire your way to reliability @otterbook #devopsdays
  • @otterbook: Q: "how do you enforce error budgets?" A: "must have the political will"
  • @otterbook: "we must engineer failures out of production; can't just throw more servers/capabity at the problem and call it done"
  • One of the most interesting things about Google SRE is functional orientation; Treynor has 1300+ SREs reporting to him
  • @otterbook: RT @ericpassmore: Brilliant, overflow in system outages events SRE team escalates to devs teams @otterbook #devopsdays

Sponsors

Caskey L. Dickson: Metrics, Gauges, Counters and Ratios

  • @toddvernon: RT @jasonhand: Yuuuuuup! (Alerts should always be actionable)
  • @nellshamrell #Devopsdays https://t.co/Q5rX3gnk0B @DevOpsDaysPDX: Digging into the data with @caskey #devopsdays https://t.co/zybmiLKRwj
  • @ericpassmore: Counters, gauges, time stamped all types of metrics. I would include the color red Ex: cluster just went red ;) @caskey #devopsdays
  • @aneel: RT @richburroughs: Now it's @caskey dropping knowledge on metrics @DevOpsDaysPDX #devopsdays https://t.co/FUwrbbKlrW
  • @caskey slides: https://goo.gl/pZDGSp
  • - @esigler: Fantastic breakdown of what goes into a metric. https://t.co/jWWTzZIKhe
  • @RandomNoun7: I have no proprietary knowledge of math - @caskey πŸ˜‚ #devopsdays

Open Spaces

  • It's @joenuspl describing Open Spaces rules of the road; we were dorm mates at Purdue in 1990! :)
  • #devopsdays It's @joenuspl describing Open Spaces rules of the road; we were dorm mates at Purdue in 1990! :)
  • Great open spaces topics! DevOps for SAP (contact @dhauge), DevOps and DBAs and more!

After I left...

  • @cacorriere: RT @jasonhand: Re: The Q to @kmugrage on how to move away from the fear of failure..
  • All advancements & innovations come from trial & erro…
  • @filler: Nailed it. https://t.co/u0FhUmPSPv
  • @EverydayKanban: Lots of thanks to @botchagalupe in @kmugrage talk at @DevOpsDaysPDX #devopsdays
  • @chyatt: RT @grendzy: Psychological safety the #1 predictor of team effectiveness at Google. https://t.co/bGII0tdhx9 #burnout #devopsdays
  • @ericpassmore: Burnout: Do not try to guess cause and effect, professionals are best @kmugrage #devopsdays
  • RT @chyatt: RT @grendzy: Psychological safety the #1 predictor of team effectiveness at Google. https://t.co/bGII0tdhx9 #burnout #devopsdays
  • RT @EverydayKanban: Lots of thanks to @botchagalupe in @kmugrage talk at @DevOpsDaysPDX #devopsdays
  • @RealGeneKim: RT @richburroughs: I just can't escape @petecheslock #devopsdays https://t.co/7EW10p4hZN
  • @RealGeneKim: .@dapperdanforth Yes, SAP DevOps! You should check out @dhague talk on DevOps at SAP. It’s amazing! https://t.co/etWpuIxuGX #devopsdays
  • @RealGeneKim: RT @dapperdanforth: Notes from Legacy/SAP DevOps open space breakout #devopsdays https://t.co/AcfkdgqdIU
  • @RealGeneKim: RT @jasonhand: @kmugrage giving what is probably the most important talk of the day at #devopsdays #burnout https://t.co/aNfkYucw3L

Day 2

  • @scottnasello: RT @DevOpsDaysPDX: .@nullfinch, @benjammingh, & @mrzarquon up next w/ three perspectives on building, configuring, and securing software."…
  • @DevOpsDaysPDX: "What can't JIRA fix?" #devopsdays https://t.co/xV4EqVS7uE
  • @jasonhand: RT @gitbisect: Huge respect to all the #devopsdays attendees who are on-call at @DevOpsDaysPDX. Thanks for coming, while being ready to res…
  • OH: "don't do security bug bounties until you've fully engaged internally talent; otherwise, giving $$ away"
  • @DevOpsDaysPDX: RT @gitbisect: " #devopsdays isn't about tooling... so let's talk about tools!" I think every ops team needs a pancake bot. @DevOpsDaysPDX
  • What is a DevOpsDays without a little puppet/chef banter? :) @nullfinch, @benjammingh @mrzarquon
  • Brilliant. "having a secure server doesn't necessarily make you money; but having vulnerable server makes it much more likely that you'll lose money"
  • RT @gitbisect: Having a secure server won't make you money, but having an insecure server can certainly cost you a lot of money! - @nullfinch #devopsdays
  • @nullfinch: "developers need to be security-conscious; you can't prevent [entire category of attacks like timing] with a firewall
  • @nullfinch et al Q: "what if infosec breaks stuff?" A: "hmmm. do you work for Delta? game days; triage teams;
  • @gitbisect: Ops/Sec can't always mitigate insecure code. Devs need to take more responsibility & interest in sec #devopsdays

Up: Julia Wester, LeanKit, @everydaykanban: Chaos Driven Development

  • @wholemilk: Some day I hope to be introduced by @alicegoldfuss #devopsdays
  • @VictorOps: RT @LuckyLefky: Ops, Dev, AND ITSec sharing the stage discussing breaking down silos as πŸ”‘ πŸ—’at #devopsdays PDX https://t.co/BAlcJzJM7c
  • @everydaykanban: "endless cycle of pain: too much WIP, nothing gets done, excessive escalations"
  • @everydaykanban: "scheduling routines ala OS: FIFO works best for homogeneous work (standardized, narrow variances curves)
  • @everydaykanban: "Shortest Job First: trouble is we don't know what the job duration, ppl bad at estimating; should short pre-empt long?
  • @everydaykanban: "Schedule by 'priority': this is most common; most often set by Highest Paid Person In Room (HIPPO), easily gamed
  • @everydaykanban: "Round Robin: rarely encounted in human systems; work on job for one quantum, then switch to next; we start everything..
  • @everydaykanban: "I use multilevel feedback queue: prioritze buckets of work: at CNN: breaking news, news stories, archives...
  • @gitbisect: .@EverydayKanban explains Multilevel Feedback Queues for work processing... Fascinating! Talks like this are what make #devopsdays great!
  • @goforcd: What a fun day at Day 1 of #DevOpsDays #PDX! And so good to meet all the #GoCD fans 😊😊 We're all set for Day 2!! https://t.co/JBfCzQIczT
  • @arafatm: RT @kylog: .@everydaykanban at #devopsdays: we must ask "is our goal to keep people busy or deliver quickly?"
  • @toozinator: RT @rsaturns: "Artisanal tire fires." You can't get more Portland than that. #devopsdays

- @MorellatoSimone: RT @michaelgorsuch: takeaway from @otterbook's talk: normalized reliability metrics, SLAs and error budgets are cool #devopsdays

@jasonhand: RT @gitbisect: Is our goal to keep people busy or deliver value quickly? - @EverydayKanban #devopsdays @DevOpsDaysPDX
- @gitbisect: No matter what workflow system you're using (scrum, agile, etc), you can always examine it for potential improvements. #devopsdays
- @julieapetersen: #Learning toolkit to tame the chaos #devopsdays Thanks! @EverydayKanban https://t.co/AJS7yfhjDI
- @crollo: Hey @EverydayKanban how would you handle the "need info" challenge? Back to the backlog? Or ??? #devopsdays
- @RandomNoun7: RT @JoeNuspl: #devopsdays Portland. @EverydayKanban summer reading list! https://t.co/QPJ3LGM11a
- @nibalizer: Making friends at #DevOpsDays pdx with @scottnasello and a @RealGeneKim photobomb https://t.co/vvN4aAvL0D
- @esigler: .@EverydayKanban "Policies (e.g. WIP) allow for shared understanding...If you need to break the rules, everyone has the context" #devopsdays
- @kitchen: Agreed. I have lots of homework to do now. https://t.co/uTqOlkny1W
- @RandomNoun7: The number of people taking pictures of @EverydayKanban's slides is testament to how it speaks to people. #devopsdays
- @stevenlyons: RT @gitbisect: There is nothing so useless as doing efficiently that which should not be done at all - Drucker #devopsdays @DevOpsDaysPDX
- @michaelgorsuch: I think @EverydayKanban's talk about how we can reduce team chaos by more intentional scheduling is my favorite so far. #devopsdays

- RT @JoeNuspl: RT @gitbisect: There is nothing so useless as doing efficiently that which should not be done at all - Drucker #devopsdays @DevOpsDaysPDX

@RandomNoun7: Love this slide. Keep your task count down to these limits so you can actually FINISH! #devopsdays @EverydayKanban https://t.co/NjO0F0E6CG
- RT @crollo: Great reminder that by keeping tasks/stories etc small you can get more DONE! Thx @EverydayKanban #devopsdays
- @JoeNuspl: #devopsdays Portland. @EverydayKanban "FizzGood" https://t.co/qfRXpLqwze
- @gitbisect: Does partially-done work provide value?
- Put a premium on work-in-progress. Only introduce chaos if there's value
@EverydayKanban #devopsdays

Up: Ben Hughes, DevSecOps

  • Ben Hughes with his pink hues is talking about Security and DevOps @benjammingh #devopsdays https://t.co/vvQWHQYmq5
  • @benjammingh: "the reason you get security in early because that's when we can do the most"
  • Ha! @benjammingh: "typical org: 100 devs, 10 ops, 1 infosec; that 1 infosec has to deal with 110 ppl making bad life decisions"
  • @joshrivers: Security is a priority. Relevant security is essential. #DevOpsDays with @CDKGlobal https://t.co/3A45XjDOmC
  • @benjammingh: "we bring devs to BlackHat, DefCon, SummerCon; they leave terrified about infosec risks"
  • @benjammingh: "At Etsy: we have Designated Hackers for each product team; they hang out in their chat channels, standups, etc."
  • @LuckyLefky: Include ITSec early in #devops - @benjammingh from @Etsy. Reminds me of pulling testing forward in dev. #devopsdays https://t.co/uSSOeBqEDA
  • @crollo: Designated != Dedicated #devopsdays
  • @EverydayKanban: You scale security by making security everyone's responsibility @benjammingh @DevOpsDaysPDX #devopsdays
  • @DevOpsDaysPDX: RT @EverydayKanban: Security people need a bit more humility - need to be approachable @benjammingh @DevOpsDaysPDX #devopsdays
  • @DevOpsDaysPDX: RT @richburroughs: Shoutout to the best film ever by @benjammingh @DevOpsDaysPDX #devopsdays https://t.co/kD67kDGcKb
  • @benjammingh: On blamelessness: "Infosec: why do you keep doing this?!? Dev: Fine, I'll do a better job at hiding it next time."
  • RT @necrobuffalo: RT @EverydayKanban: The job of a recruiting team is to open random files from unknown people all day. #security #devopsdays @benjammingh @D…
  • RT @esigler: "You tell people not to open things from random people on the Internet. You also have a recruiting team." @benjammingh #devopsdays
  • RT @fuzzychef: "GPG is a tire fire designed in the 1800's" #devopsdays
  • Ha! @benjammingh on the problem of DevOpsSec: "Infosec tends to be bigger a**holes than Ops people"

Sponsors

  • RT @bryanbrake: #infosec people... #devOpsDays sounds like a great place to send a talk that will reach real people, instead of preaching to the choir...
  • OH: "Here at Microsoft, over 1/3 of our workloads in Azure are on Linux. Microsoft loves Linux. We all love Linux."

Up: Jennifer Davis @sigje

  • RT @DevOpsDaysPDX: πŸ¦„πŸŒˆβœ¨ Up next at #devopsdays Portland, @sigje talks about monitoring the unmeasurable. πŸ’–
  • Ha! @sigje: "Hopefully my soul stays with me.. at least until I die."
  • @sigje talking about how left-pad broke entire JS/Node community: http://www.theregister.co.uk/2016/03/23/npmleftpad_chaos/
  • @sigje: "other examples of left-pad-like problems include Heartbleed: solution: Chef encourages engrs to improve libraries we depend on
  • @gitbisect: Monitor consumers to producers. Donate time towards your fav projects to help improve the ratio & decrease bottleneck @sigje #devopsdays
  • @DevOpsDaysPDX: RT @nellshamrell: .@sigje's talk is totally a technical talk @DevOpsDaysPDX #devopsdays
  • @sigje: "companies can't lose money by supporting community; we had this problem at Yahoo; had to retire Zimbra b/c it competed w/Yahoo Mail
  • @JoeNuspl: RT @gitbisect: .@sigje encouraging orgs to help pay project contributors. Not all companies can dedicate an employee, but you can pitch in!…