2013/09/19: OSS4B Conference

by Gene Kim on


"NoSQL and the OpsEx Business Plan," Joe Drumgoole, MongoDB (@jdrumgoole)

  • Drumgoole: "NoSQL is a generational change, the likes we haven't seen since relational databases"
  • Drumgoole: "Innovation: reduce inputs to create outputs, or increase output from given input"
  • Drumgoole: "Trials reqd to develop: WD-40: 39; Angry Birds: 51 flopped games before hit; Dyson vacuum: 5161; Lightbulb: 10K
  • Drumgoole: "investment areas: Dev, HW, SW; in old days, it was all on right end (Dev was clerical); now all on left end
  • Drumgoole: "Now, cost of Dev is extremely high, b/c of talent required
  • Drumgoole: "Old waterfall method is horrible, but it works, as long as u're implementing X.400: specs are frozen
  • Drumgoole: "Waterfall sucks b/c it assumes technology is the problem; reality is that problem is external
  • Drumgoole talked about DECNet team: 3 yr dev cycles, never shipped anything, b/c TCP/IP kept changing
  • Drumgoole: "Hypothesis: people would buy addl services at airport; took 6 months to validate; experimentations only begin after you ship first version
  • Drumgoole: "Tragedy of huge failed govt projs: sw vendors got paid 1st, regardless of weather proj shipped! (ORCL, MS, etc)
  • Drumgoole: "MongoDB: store JSON like you use JSON; start on desktop; then deploy 6K shards on AWS
  • Drumgoole: "Relational DBs were built to optimize for storage, back when it was really expensive; 50MB disk used to cost $50K
  • Hilarious: Drumgoole: "This is how parking attendant would store parked cars, if using relational DB"
  • Drumgoole: "Relational DB creates impedance mismatch; code, XML configs, DB schemas
  • Drumgoole: "Marrying apps to db is horrible; I knew DB mgr who begged to be taken off the project"
  • Drumgoole: "Publishing company estimating they'll be uploading 1TB per day, every day."
  • @hertling @_flynn @jeffweiss: Drumboole making very convincing case for lower dev cognitive load of NoSQL/MongoDB vs. mySQL
  • Drumgoole: "Proj took 3 months in old way (Reqd 2 dev + DBA); took only 1 week to replicate that project in MongoDB"
  • @lefred: Am I in the right room?? #humour #oss4b http://t.co/ssb7p98WIf
  • Haha. Drumgoole: "MetLife data center is like a museum; decades of technology on display, row after row"
  • Drumgoole: "NoSQL allows you to leave legacy systems alone; layer on top, to allow innovation
  • @RealGeneKim lol, think you mean @jdrumgoole :-)

"Kanban for Portfolio Management," Gaetano Mazzanti

  • Mazzanti showing horrendous reality of tech demand >> capacity on kanban:
  • Mazzanti: "just by putting all the crap on a wall, wonderful stuff happens"
  • Mazzanti: "Bottlenecks are often gates; committees constipate flow; huge queue in front, nothing in next lane
  • Holy cow. Great example of real typical complex flows in IT on kanban board:
  • Mazzanti: "Complexity and uncertainty are intrinsic to knowledge work --David J Andersen"
  • Mazzanti: "We hate uncertainty; we want to balance risk/return; maintenance/growth; short/long term
  • Mazzanti: "Selection: what to do next?; what to finish next?" (note how he doesn't use word 'prioritize')
  • Mazzanti: "How to select: fit with goals vs. timeliness (urgency, cost of delay, etc.)
  • Mazzanti: "ROI, IRR, NPV... most important, often overlooked, is Cost of Delay; Cost of Delay for 1 mo: $5K/mo vs $30K/mo"
  • Mazzanti: "On cumulative flow, flat horizontal lines show bottlenecks"

"From Problem To Solution, Faster: Using Interviews To Improve Your Process & Accelerate Delivery," Melissa Doerken, Thoughtworks

  • .@melissadoerken: "I'm business analyst on the Mingle team, based in SF, targeting project management tools
  • .@melissadoerken: "The problem interview vs. solution interview
  • .@melissadoerken: "Our Mingle app deployed mult times/day, but lead times varied: 5.5 wks for Mingle+ vs. 12 wks for WYSIWIG. why?
  • .@melissadoerken: "Problem Interview: technique for prob discovery and undersatnding
  • .@melissadoerken: "If I had 1 hr to save world, spend 59 min defining the problem & 1 min to find solution. --Einstein"
  • .@melissadoerken: "1. Do homework; capture assumptions & identify the riskiest
  • .@melissadoerken: "Ex: People perferred using word processors over using our tool; they want rich images to convey stories
  • .@melissadoerken: "2. Limit number of assumptions; know when to move on (e.g., when 3-5 validate problem, move on)
  • .@melissadoerken: "3. One person at a time; especially when different roles, consensus will never happen; 1:1 interview
  • .@melissadoerken: "4. Start with open-ended questions; close-ended questions limit learning
  • .@melissadoerken: "5. Listen more than u speak" (My fave quote: "We were given two ears & only one mouth for a reason")
  • .@melissadoerken: "6. Separate product feedback from behavior; why? product feedback is less important than understanding behavior
  • .@melissadoerken: "Product Principles; conversation over documentation invalidated assumption 'people prefer WYSIWIG
  • .@melissadoerken: "On the dangers of refinement: will often miss the best solution"
  • .@melissadoerken: "On dangers of refinement: will often miss the best solution" #oss4b (similar to A/B testing danger @littleidea)
  • .@melissadoerken: "Instead of Refining, try Exploring" #oss4b (similar to avoiding A/B testing danger @littleidea)
  • .@melissadoerken: "The Solution Interview: a technique for early and continuous feedback (takes 30m or less)
  • .@melissadoerken: "In Problem Interview, user is showing you; in Solution Interview, you're showing user" (Nice)
  • .@melissadoerken: "1. Understand
  • .@melissadoerken: "2. Start as early as possible; ideally when you have minimum interactable mockup
  • .@melissadoerken: "3. Make sure they're comfortable; (have them use their preferred browser)
  • .@melissadoerken: "4. Quiet, distraction-free space
  • .@melissadoerken: "5. Record; can review nonverbal cues, create shared understanding
  • .@melissadoerken: "6. Exit chat

"Raise The Bar!" Alessandro Franceschi

  • @lefred: @avagante is starting his presentation about infrastructure automation #oss4b http://t.co/2mwjDCvy1v
  • Franceschi: "Scenario 1: brand new project: easiest; most freedom of tech choices; brand new OS/stack; clean start, no tech debt, faster/smoother deployment times
  • Franceschi: "Scenario 2: Infrastructure migration: existing systems may not be centrally managed; migration of existing stacks to new systems
  • Franceschi: "Scenario 2: Infrastructure migration: downsides: existing systems may not be centrally managed; migration of existing stacks to new systems, all new systems should at least be fully managed
    • Define the standard baseline; create stacks/roles you need
    • Enlarge coverage of application stacks
    • Start from what is more used and needed
    • Evalulate
      • how easy/quickly can it be done?
      • how stable are systems?
      • what maintenance efforts are required?
      • number of systems?
      • migration risks/impact?
      • what's benefit of automating?
      • what is the roadmap for the platform/app? (maintenance or high rate of change?
  • Franceschi: "Scenario 3: infrastructure update: hardest
    • Problems
      • harder and more dangerous, because it's production
      • probably different OS to manage
      • undetermined existing setup procedure
      • manual configs accumulated over time
    • Mitigations
      • Evaluate agent setup on older systems
      • evaluate effort and benefits
      • evaluate the migration alternative
  • Franceschi: "Priorities: automate svr deploys; automate common system configs; automate most important stacks; run testing and app deploys, then automate; automate or delegate monitoring, integrate what already works well
  • Franceschi: "Config rollouts
    • notify users of the ongoing changes
    • have a test environment
    • test effects on any single different OS
    • propogate the configs
    • watch the logs and reports
    • don't be surprised of skeletons: review & patch uncovered configs
  • Franceschi: "Mindset chg for sysadmins; when infrastructure is code, it's versioned, tested, re-used; oh, u're a developer now

Up: "Benefits Of Using Kanban In IT Ops", Dragos Dumitriu, DJAA (@netheart)

  • .@netheart retelling amazing XIT story improving flow at MSFT in 2005 after they offshored IT work
  • .@netheart: "I inherited a team that was first Agile team at MSFT; 'they're anything but Agile'
  • .@netheart: "Because teams were always late, biz demanded ROMs (rough order of magnitude); more lateness, more ROMs"
  • .@netheart: "Soon, team was spending all of its time estimating work & building PPTs, instead of doing work." (hahaha)
  • .@netheart: "I calculated lead time for system was 155 days; ie, ask for anything, it'll take 5 months to complete"
  • .@netheart: "And yet, every team estimate never exceeded 20 days; despite lead time was 155 days." (Haha)
  • .@netheart: "Q: where was the 130 days going; conclusion seemed to be 'team not doing estimates well'"
  • .@netheart: "Reln between Redmond & Indian teams very strained. Blue: good relationship; Red: conflicts
  • .@netheart: "Oh, by the way, at completion of all work, had to be approved by SOX team, b/c apps were financial reporting
  • .@netheart: "Mystery: where did 130 missing days go? Answer: it was all in queue/handoffs; 20 days was only work time
  • .@netheart: "Found it only by accident; stopped deleting timestamps from ticketing system
  • .@netheart: "Changed ticketing system, created new stage called 'Waiting for Dragos', to capture when work is queued/blocked
  • .@netheart: "70% of team's time was in queue; 30% was spent chasing reqs, handling incomplete reqs; that's 6% efficiency
  • .@netheart: "Unfortunately, even in good teams, even 12% is considered good; what is we could double efficiency; it's like doubling size of team
  • .@netheart: "To eliminate waste, we must first figure out where we're spending the time
  • .@netheart: "We had to make the work, workflow and queue/wait times visible (kanban);
  • .@netheart: "Data said that they were delivering 3 items/month; 3 years of work in backlog;
  • .@netheart: "We replaced ROMs with a fixed estimate based on historical data; waste no more time doing estimates; data was very consistent
  • .@netheart: "Historical data was very accurate; had 80 man-years of work history; immed 30% boost to Dev & Test capacity
  • .@netheart: "Cost accounting was replaced with ROI based on budget contribution; immed 20% boost to PM capacity
  • .@netheart: "PM took over some Dev tasks; 20% boost to Dev capacity (and they were happier; coding instead of estimating)
  • .@netheart: "Leveraged SME Ops to avoid rework; established reputation for building the right things right
  • .@netheart: "Used usability expert to change chg req form; to get a glass of water, had to fill out 4 page form; new: 2 page, lots of open spaces, but provided full kit requirement
  • .@netheart: "Created high-trust relationships; when offshore team apologized that work would be 1 wk late, customer laughed. 'No prob!'
  • .@netheart: "Next phase: created buffer of work, so that any blocked Dev or Test could work on; SLA was guaranteed 25 days (down from 155 days)
  • .@netheart: "Accused of doing just the easy work; asked customer, who said, 'no, we're picking the work. you're delivering
  • .@netheart: "For every 2 days of Dev work, reqd 1 day of Test;
  • .@netheart: "Asked testers for their aspirations: wanted to move to other team;
  • .@netheart: "Eliminated entire 3 yrs of backlog in 9 months; demand went up. Delivered everything that was asked each month; no one got fired, instead people got promoted
  • .@netheart: "Ended up at 22 day lead time; every week, had 40-60 open items; went down to 5; (WIP)
  • .@netheart: "Lesson: when dealing w/mgmt, stick to 1 slide. If u hv 6 slides, u'll never get past slide 2 in 30 min mtg
  • .@netheart: "Success happened when we focused on lead time, vs Dev & Test optimizing for themselves
  • .@netheart: "Next big productivity surge: promoted bored testers to Dev, bringing Dev/Test ratio from 1:1 to 2:1
  • .@netheart: "Lead time went from 155 -> 22 days; WIP went from 40-60 work items; went down to 5 #oss4b
  • .@netheart: "Ops needed slack, b/c of production issues; that was hidden
  • .@netheart: "Kanban has been shown to increase productivity by 2-8x in weeks
  • .@netheart: "By reducing lead time, cost per feature went from $12K to $2K
  • .@netheart: "10 years ago, we didn't have kanban boards; we only had ticketing system (bug mgmt that we used at MSFT); created views: top 5 commited items, Done or WIP; like Excel spreadsheet
  • .@netheart: "After we reduced backlog to zero, other MSFT GMs demanded to know how we did it; replicated in 5 other areas
  • .@netheart: "Wife's background was in ballet; 25 yrs; in Romanian, lightswitch and fuse are same word

Up: "Success is 99% Persistence," Simon Riggs

  • Riggs: "Licensing model wasn't appropriate funding model for PostgreSQL
  • Riggs: "My Big Data defn: ability to store detailed data at event level; enables making biz processes visible and improvable
  • Riggs: "Database also record transactional history that allows us to analyst what happened; BI, data warehouse, JIT, efficient manufacturing/distribution

"Why We Need DevOps Now", Gene Kim

  • @alvagante: Always a pleasure to see @RealGeneKim talking. At #oss4b http://t.co/tIYKarG4CK
  • @simonemartelli: RT @OSS4B: @RealGeneKim "When you break a promise you compensate with even bigger promises that can't be met" #technicaldebt #devops #oss4b
  • @lefred: From @RealGeneKim 's presentation #oss4b #devops http://t.co/I6PUt5BHjd
  • @lefred: Keynote on #devops by Gene Kim started
  • @lefred: Real coffee! #oss4b #italia http://t.co/MMbbKNfJzO