2013/06/20 Velocity Day 3

by Gene Kim on

#velocityconf

My favorite conference!

(Umm, the twitter handles on these tweets are wrong -- it's pulling in the real name instead of handle. Sorry! We're working on it!)

Souders/Allspaw: Opening

  • @Colin Bendell: "Gotta check the mainframe" movie montage at #VelocityConf ! (Sadly, I think I recognize all those movies)
  • @Jonathan Thorpe: Awesome mainframe video in the keynote. Must get a copy!
  • @Ernest Mueller: I want a mainframe
  • @kate matsudaira: super excited about my talk at #velocityconf at 9:55 - you can view it live: http://t.co/Ekfn4VSmfy
  • @Sevan Janiyan: Oh “10 million dollars in second round of funding, do you know how many ssds that could buy” - @allspaw on @fastly #VelocityConf
  • @Joshua Timberman: “I love the UI of tcpdump”
  • @Benjamin Krueger: @markimbriaco Shout out for you at the #velocityconf keynote. :-)
  • @James Wickett: release it scored as one of the top 2 books for ops, congrats @mtnygard
  • @James Wickett: release it scored as one of the top 2 books for ops, congrats @mtnygard
  • @Ernest Mueller: You can watch the #velocityconf keynotes via live stream - http://t.co/TuXOqqkWJ7

  • Just curious. How did they "fix the wifi?" Roll in more access points? Bigger pipe to hotel? Just turned the knob to 11?

  • @Gene Kim: OMG. Coffee just came out my nose. Hahaha. “@mattjay: @RealGeneKim turned it off and started it again. #velocityconf

  • @Matt Johansen: @RealGeneKim turned it off and started it again.

  • @Karthik Gaekwad: @RealGeneKim I'd like to find out as well. Also, there is a conf Internet dashboard on one of the TV's outside the ballroom.

  • @Don Roeber: How to fix WiFi: RT @mattjay: @RealGeneKim turned it off and started it again.

Dylan Richards: Practicing Failure, on resilience in Team Obama

  • me@dylanr.com
  • @O'Reilly Velocity: You have 18 months to build software that has to work for 4 days. @dylanr knows all about it http://t.co/USbePCAMKr
  • .@dylanr: "Like all political technologists, I worked at Crate & Barrell doing point of sales systems"
  • .@dylanr: "40 engineers; 300 repos; 7 teams; 3000 servers (AWS for almost everything); millions views/day, 1M+ volunteeers
  • .@dylanr: "8000 staff onboarded in less than 18 months; only 1 metric mattered: 270 (# of electoral votes needed to win)
  • .@dylanr: "A campaign: plan, build, execute, GOTV (get out the vote)"
  • .@dylanr: "@Jonathan Thorpe: A guy came up to me at the jobs board and stated "it's great to work in DevOps". Like it as a job title or not, it's hot!
  • .@dylanr: "Execution phase: last 6 months where we actually use all the stuff we built; keep building + real feedback!
  • .@dylanr: "GOTV: last 4 days: Sat-Tue: if we don't do well in GOTV, people don't vote! for tech, this requires SCALE
  • .@dylanr: "Call support app: in three days, hundreds to 100Ks to 1M+ calls per day. Planning for this scale is hard
  • .@dylanr: "Tech could have lost the election." (Haha. Showing pic of cake saying 'Don't Fuck This Up')
  • .@dylanr: "We had 1 shot to get this right. That means making the apps resilient. Execute even if terrible things happen
  • .@dylanr: "@Joel Barciauskas: RT @douglas: 6 months of 1000-2000 calls with OFA call tool, to 1.2 million calls the day before election day — @dylanr
  • .@dylanr: "No feature is cool enough not to have a working app; most don't work when it matters most
  • .@dylanr: "Tech fails@kate matsudaira: wow @dylanr is great - its going to be a tough act to follow
  • @kate matsudaira: wow @dylanr is great - its going to be a tough act to follow
  • .@dylanr: "Only option: Practice, Practice, PRACTICE; 6 weeks before election, we held Game Days; sat everyone done...
  • .@dylanr: "Built Staging to match Prod, then we'll break things, and see how we'll react
  • .@dylanr: "No features need to be developed for 2 weeks; 2 days for final prep; then we start breaking things
  • .@dylanr: "We had a scripted agenda: Turn of slave, then master, etc. But I was lying; disasters don't have real scripts
  • .@dylanr: "For a week, I tortured 40 engineers; we asked DevOps ppl to make chgs, and then didn't do it (it happens!);
  • .@dylanr: "We were the Agressors" (wow, love this Red/Blue team, like infosec! Blowing my mind. cc @joshcorman)
  • .@dylanr: "We broke things we thot we had fixed; when master db when down, all user facing apps broke; this was huge prob
  • @James Wickett: campfire was used for collaboration, google docs for run log for the game day test
  • @Douglas Back: .@dylanr didn’t just test technological failures, but also human ones — devops lied about making some changes during gameday
  • @Sebastian: Awesome idea from the Obama campaign: game days which are basically an epic fire drill.
  • @Douglas Back: .@dylanr didn’t just test technological failures, but also human ones — devops lied about making some changes during gameday
  • @James Wickett: campfire was used for collaboration, google docs for run log for the game day test
  • @Douglas Back: .@dylanr didn’t just test technological failures, but also human ones — devops lied about making some changes during gameday
  • @Sebastian: Awesome idea from the Obama campaign: game days which are basically an epic fire drill.
  • @Wilfried Schobeiri: “Real disasters aren’t scripted” @dylanr lied about simulated event order to keep team on their feet
  • .@dylanr: "We put together run books, like an instr manual, & put it into the repo w/code; every engr knows how to fix
  • .@dylanr: (knowledge bases prevent Brents! #phoenixproject)
  • @Langdon White: @dylanr plotted against his dev team (claims he feels bad) on the team "gameday" to test the env during #obama #campaign
  • @Jack Steadman: Keeping identity up was critical - Quick Donate (and everything else) depended on it. @dylanr
  • @Wilfried Schobeiri: “Put your runbook in the repo.” @dylanr Love this - I put deployment info in the repo as well.
  • @Wilfried Schobeiri: “Put your runbook in the repo.” @dylanr Love this - I put deployment info in the repo as well.
  • @Eric McCraw: awesome talk by @dylanr right now on resilience!! so glad he didnt inject polarizing political views in his TECHNICAL talk at
  • @Douglas Back: Practice failing, learn from that failure, and build your instruction manual, use it, and revise it. — @dylanr
  • @James Wickett: the game day testing came in handy the next day when AWS actually went out
  • @Wilfried Schobeiri: “Practice failing, and learn from that failure” @dylanr
  • @Eric McCraw: awesome talk by @dylanr right now on resilience!! so glad he didnt inject polarizing political views in his TECHNICAL talk at
  • @Jeff Sussna: RT @asf: Awesome & funny presentation on Reslient Apps and Teams #velocityconf @dylanr
  • @James Wickett: build resilient teams, practice failure, write an instruction manual to handle failure @dylanr great points!
  • @Stephen Thair: @dylanR - so Game Day is a big game of D&D with you as the Dungeon Master? :-)
  • @Gene Hughson: RT @jeffsussna: "Fail to what matters. Talk to stakeholders to know what it is that matters." via @dylanr at

  • @Colin Bendell: I think you could use WiFi utilization has a proxy for audience engagement. I would love to see a timescale correlation #VelocityConf

  • @Colin Bendell: I think you could use WiFi utilization has a proxy for audience engagement. I would love to see a timescale correlation #VelocityConf

  • @Stephen Thair: RT @nicj: Easy to zone out during the #velocityconf keynote ads. Need a Velocity Tivo.

  • @kellan: Having WiFi problems? Bring out @dylanr again!

  • @Phil Sorber: RT @jespi: Check The Mainframe! http://t.co/h9JEjdp7Gm

  • @Daniel Schauenberg: RT @allspaw: Some live WiFi metrics for #velocityconf, thanks to @circonus and @postwait : http://t.co/hedxKMPKTH

Update From The W3C Performance Working Group Arvind Jain (Google)

Eliminating Website Performance Theft Joseph Loveless (Neustar)

If You Don't Understand People, You Don't Understand Ops Kate Matsudaira (popforms)

  • Advice from @allspaw: "Subscribe to Kate's Tech Leadership Newsletter. Most important thing you'll do today." (Agreed!)
  • .@katematz talking on how to influence, communicating, generating excitement, build teams;
  • @gWaldo: This happened: CC: @katemats #VelocityConf http://t.co/W1vVTNWT0C
  • .@katemats: "Accountability: swagger is something you can learn; it's not about title and # of people reporting to me"
  • .@katemats: Observing that in mtgs, there's often a person who everyone listens to, despite no reports; that's influence
  • .@katemats: "He was known by hundreds of people of being able to 'get it done,' 'he followed thru on his commitments'
  • @petecheslock: "If you want to be an influencer you need to be accountable for what you say and do." @katemats
  • @ErnestMueller: Yay @katemats on stage at #velocityconf about leading gangster style!
  • .@katemats: "Accountability is more than hitting estimates/deliverables; it's being present, engaged, on time"
  • .@katemats: "Everyone is an ally/homie;
  • @1teration: The @katemats presentation has a lot of golden nuggets about how to work well with others, trust, accountability and respect.
  • .@katemats@gWaldo: "Beinng Late is a Selfish Act." --@katemats #VelocityConf
  • @LangdonWhite: every day, every month: hustlin' @katemats
  • @Wickett: Accountability is crucial to success, doing what you say you are going to do @katemats
  • .@katemats reminding me of my fave quote: "You're only as smart as top 5 people you hang out with"
  • @1angdon: respond to anger with "help" and/or positive response, take ownership of other people's problems #howToBeAGangster
  • @Michael Leinartas: luckily in this crowd instead of annoyed looks i just get knowing nods when my pager sound goes off
  • .@katemats: "Make every interaction positive"; (Yes, but often difficult for cynical Ops ppl who are burned out, yaknow?)
  • .@katemats: "Reciprocity": (Another one of my favorite philosophies; relentless giving)
  • .@katemats: "Reciprocity" (Random acts of kindness)
  • @James Wickett: Everyone wants to feel special and you have power to make that happen, you wield a powerful tool
  • @Douglas Back: “Be the person that is making the world better for other people, and it will pay off in the long run.” — @katemats
  • @John Allspaw: I wonder how many attendees recognize that @katemats topic is actually "devops" without mentioning the word once. Yay!
  • @Pete Cheslock: "People and success go hand in hand" @katemats #velocityconf http://t.co/kGzAMpumaI
  • @Gary Pedretti: RT @RealGeneKim: .@katemats reminding me of my fave quote: "You're only as smart as top 5 people you hang out with"
  • @Kiran reddy: @katemats great talk!!
  • @Brad Ledford: I'm seeing If You Don't Understand People, You Don't Understand Ops at #velocityconf. http://t.co/W4ReruxLgR
  • @ginablaber: RT @pkdotnet: @katemats speaks truth at #velocityconf. great stuff about accountability, being real and genuine and helpful.
  • @Mark: "How can I help you?"
  • @gWaldo: @asf Dealing with people (communication) is by far the hardest part of Engineering. And the most overlooked. #VelocityConf
  • @gWaldo: RT @mleinart: luckily in this crowd instead of annoyed looks i just get knowing nods when my pager sound goes off
  • @Mario Lopez: Best talk at #velocityconf so far from @Katematz, Looking forward to subscribing to her newsletter.
  • @Karthik Gaekwad: The principles that @katemats just shared reminds me a lot of #devops- #communication, #culture #sharing
  • @jeffsussna: RT @courtneynash: I talked to @katemats about how Ops is like Soylent Green: it's made out of people. http://t.co/nThJwONuO9
  • @pereiragoncalo: very interesting for trends http://t.co/z0IcDXraJd
  • @chrisblackburn_: @katemats culture talk at #velocityconf is by far the best so far. If she hasn't done a Ted talk yet, she needs to.

Lightning Demos Ilya Grigorik (Google), Patrick Meenan (Google)

  • Did anyone else use @igi
  • Did anyone else follow @igrigorik's work competing in the $1MM Netflix Prize? Thank you, @igrigorik! You rock! cc @adrianco
  • @qwade: I love public shaming of websites!! Google showing off HTTP archive at #velocityconf http://t.co/k3qu6BZ0oE
  • @douglas: …and if that’s not enough, @igrigorik is showing off Google Docs scripts to populate a spreadsheet with data from Big Query
  • @collidr: HttpArchive data can be queried directly with bigquery?!? #mindblown
  • @datacenter: RT @pingdom: What has been down in the last hour around the Internet? Animated map: http://t.co/LaEUwiBEBX
  • @_losh: RT @douglas: Try it out yourself — HTTP Archive data available at Big Query, http://t.co/XqCKL7sOWd — no download required
  • @souders: Better URL for #HTTPArchive on BigQuery from @igrigorik: http://t.co/7z33T95Wha #VelocityConf
  • @kitesurfer: hum you can add custom script to google docs. Me not know that. Cool integration of google docs with bigquery @igrigorik
  • @GiantEvolving: Every time that @igrigorik speaks I get smarter!
  • @marcelduran: reached new level of awesomeness by @igrigorik : http://t.co/rHq6fY15Vo #speechless

It's 10PM: Do You Know Where Your Code Is? Patrick Lightbody (New Relic)

It’s 11 PM, Do You Know What Your Code Is Doing? Steve Riley (Riverbed Technology)

Optimizing Development Efficiency Through API Design Jonathan LeBlanc (PayPal)

Misc

  • Woohoo! Great book! RT @James Wickett: release it scored as one of the top 2 books for ops, congrats @mtnygard
  • Thank you @velocityconf! RT @Ernest Mueller: You can watch the #velocityconf keynotes via live stream - http://t.co/TuXOqqkWJ7
  • RT @douglas: 6 months of 1000-2000 calls with OFA call tool, to 1.2 million calls the day before election day — @dylanr
  • You'll do awesome! :) @kate matsudaira:
  • @Douglas Back: .@dylanr didn’t just test technological failures, but also human ones — devops lied about making some changes during gameday
  • RT @Wilfried Schobeiri: “Put your runbook in the repo.” @dylanr Love this - I put deployment info in the repo as well.
  • @Wilfried Schobeiri: “Put your runbook in the repo.” @dylanr Love this - I put deployment info in the repo as well.
  • RT @Wickett: the game day testing came in handy the next day when AWS actually went out
  • @John: RT @ernestmueller: Ops School from Etsy/O'Reilly! http://t.co/RJMYYDoW0Z
  • @Cheyenne T Greatorex: RT @unixdaemon: Weapon of choice is excellent walk on music.
  • @Eventifier: Pics, tweets, videos, slides of #velocityconf is archived here, have a look. http://t.co/4CAcffkK8f @petecheslock @brianlmoon @techmjh TY!
  • * @James Wickett: I'm writing and tweeting about 'Velocity Conference - June 20th' with @tweetscriber! http://t.co/Ti2xVPEuoI RT @gWaldo: "Beinng Late is a Selfish Act." --@katemats #VelocityConf @Gene Kim: Right on, @katemats! Subscribe to her kickbutt newsletter here: http://t.co/ytVh2g7Ufn