10/25 AppSecUSA

by Gene Kim on

#appsecusa

Gene's keynote

  • @wickett: Infosec insurgency recommended by @RealGeneKim at #AppSecUSA http://t.co/iIu50VfI
  • @jeffsussna: RT @wickett: @RealGeneKim is dropping some #devops knowledge at #AppSecUSA << Good!
  • @jeffsussna: RT @wickett: We need systems thinking that moves from dev to ops, from biz to customer. @RealGeneKim #AppSecUSA
  • @appsecusa: RT @CabForward: We're learning more about rugged software development (application security, scalability, availability, etc) at #AppSecUSA
  • @wickett: Synonyms for DevOps is manufacturing and work processing #AppSecUSA #OWASP #devops @RealGeneKim
  • @brianaengle: @RealGeneKim @appsecusa: If you wake developers up at 2am, bugs get fixed faster than ever.
  • @transvasive: Enjoying the @RealGeneKim keynote at #AppSecUSA on #DevOps: the two key qualities of successful IT departments are discipline and rigor.
  • @wickett: Google makes devs support the app for 6 mo. before ops will take it. @RealGeneKim at #DevOps #AppSecUSA #OWASP
  • @appsecusa: RT @brianaengle: @RealGeneKim @appsecusa: If you wake developers up at 2am, bugs get fixed faster than ever.
  • @chriseng: RT @wickett: Google makes devs support the app for 6 mo. before ops will take it. @RealGeneKim at #DevOps #AppSecUSA #OWASP
  • @brianaengle: @RealGeneKim @appsecusa: Break things early and often. Do painful things more frequently so that they become less painful.
  • @jonathanmarcil: First keynote at #AppSecUSA is about Rugged DevOps. It's about processes but still uses technical terms. I like that. http://t.co/JbSiF7DO
  • @dakami: RT @wickett: Google makes devs support the app for 6 mo. before ops will take it. @RealGeneKim at #DevOps #AppSecUSA #OWASP
  • @wickett: Reserve 20% of your cycles for technical debt reduction #AppSecUSA #DevOps @RealGeneKim
  • @mhat: RT @wickett: Google makes devs support the app for 6 mo. before ops will take it. @RealGeneKim at #DevOps #AppSecUSA #OWASP
  • @brianaengle: @RealGeneKim @appsecusa If you don't reserve 20% of cycles buying down tech debt you will spend 100% of time dealing w/tech debt
  • @wickett: Amazing quote by intuit founder on release speed. #AppSecUSA http://t.co/md2muPvm
  • @todb: really happy I'm able to catch @realgenekim at #appsecusa after all
  • @wickett: RT @todb: really happy I'm able to catch @realgenekim at #appsecusa after all

Josh Corman

  • @brianaengle: @joshcorman @appsecusa Cynicism is an infosec professional core competency. Perceived Self Efficacy contributor to Sec-Burnout.
  • Signs of burnout: fatigue, cynicism, illusions of adequacy RT @brianaengle: @joshcorman Cynicism is an infosec core competency
  • Signs of burnout: fatigue, cynicism, illusions of adequacy RT @brianaengle: @joshcorman Cynicism is an infosec core competency #appsecusa
  • .@joshcorman: "Call to action: End SQL Injection in our lifetime; let's do OWASP Top 1 -- why bother with Top 10?"
  • .@joshcorman: "Strength of zero talent attackers follows Metasploit; Most sites not tall enough to clear Metasploit bar"
  • .@joshcorman: "2009: told Jeff Williams, OWASP doomed to repeat OS vuln failure: too many scanners, not enough fixing"
  • .@joshcoman: "We must end the vulnerability-industrial complex; how do we fix root cause?"
  • .@joshcoman: "Rugged Software manifesto: My code will be used in ways never designed for, and for longer than ever intended"
  • .@joshcoman: "

  • PS: Screenshot of @TweetScriber on my iPad: take notes, tweet, & curate other people's tweets for my notes

  • .@joshcorman: Gorlich Detroit BSides Rugged DevOps success story: "qtrly releases to daily releases, quicker TTF!"

  • .@joshcorman: "Rugged Summit defined 'Rugged by Role': CIOs, infosec analysts, architects, dev, test, pm"

  • .@joshcoman: "If we find PHP defects, maybe we stop putting out new PHP features [until we learn how to stop it]"

  • .@joshcoman: Implied switching from PHP is a solution; but, many CTOs choose PHP for ease of design & maximize Dev skills

  • .@joshcorman: Awesome article on "Why PHP Won" by @ericries: http://www.startuplessonslearned.com/2009/01/why-php-won.html

Putting Your Robots To Work, From The Twitter Security Team

  • Next up: "Putting Your Robots To Work" From The Twitter Security Team!
  • Collins: "Twitter grew quickly/publicly: hiccups: The Fail Whale; @barackobama account hacked, exposing admin interface"
  • Haha. Collins: "After @barackobama breach, FCC required company to be secure for next 15 years; thus all of us were hired"
  • Collins: (Oops, that's FTC injunction, not FCC. My mistake)
  • Collins: "We have a Hack Week: 1 week per quarter, resulting in demo to entire company"
  • Collins: "Guideline: get the right info to the right people": write secure code isn't tech, it's about people & tech
  • Collins: "Therefore, communicating about vulnerabilities is just as important as finding them; reinforce social processes"
  • @wickett: After Obama's account was hacked Twitter was told by the FTC that they had to have a security program for the next 20 years. #AppSecUSA
  • RT @wickett: After Obama's acct was hacked Twitter was told by FTC that they had to have security program for the next 20 years. #AppSecUSA
  • Collins: "The best predictor of the next bug is the last bug"
  • Collins: "We need Dev to trust us, so they need to be able to tell us about false positives"
  • Collins: "The point is: automate dumb work; anything that doesn't require judgement or creativity is game"
  • Collins: "Manual security: code review, pen testing, external reports; automated: static analysis, dynamic analysis, content security"
  • Collins: "Problem: Manual security: run tool, wait for it, interpret reports, repeat; Needs automation"
  • Collins: "We run static analysis when code is committed; dynamic tools are running all the time in background"@jaymclaughlin: @jaymclaughlin: Listening to @Twitter guys @alsmola & @presidentbeef talk about security automation #sadb
  • RT @jaymclaughlin: Listening to @Twitter guys @alsmola & @presidentbeef talk about security automation #sadb
  • Collins: "By doing this, we do less dumb button-pushing tasks, doing more stuff w/creativity and judgement"
  • Twitter: "Using Jenkins CI was unsatisfactory. Build #sadb: Security Automation Dashboard." Haha. Logo is a frowning bee!
  • Twitter: "Inputs: brakeman, phantom gang, csp, threatdeck, roshambo; Outputs: email dev or infosec" #saddb
  • Twitter: "Brakeman: static analysis for Ruby on Rails: @presidentbeef is primary author: 25 releases in last yr"
  • Wow. Twitter: @presidentbeef: "When does Brakeman run? When dev saves code: find bugs as quickly as possible"
  • Twitter: @presidentbeef: "Dev gets emailed when Brakeman finds security defect; also gets a CONGRATS email when fixed"
  • Twitter: @presidentbeef: "@wickett: Twitter: Uses mesos to run brakeman jobs after each git push - @presidentbeef
  • RT @wickett: Twitter: Uses mesos to run brakeman jobs after each git push - @presidentbeef
  • Twitter: @presidentbeef: defect testing in the CI deployment pipeline is great; Catching at file save is brilliant. @jezhubmble
  • Twitter: @presidentbeef: "When Dev gets email, they get explanation, avoiding anger/confusion; False pos button builds trust"
  • Twitter: @presidentbeef: (Showing video of demo; from back room, from defect introduction to fix; crowd applauds. Nice. :)
  • @mattjay: Looks like Twitter is doing the Secure SDLC right with their Breakman tool. Testing code early and often. #AppSecUSA
  • RT @mattjay: Looks like Twitter is doing the Secure SDLC right with their Breakman tool. Testing code early and often. #AppSecUSA
  • Twitter: @presidentbeef: "Ideal future: #saddb could halt deployments; expand from ruby to java, scala; breakman demo tomorrow!
  • Twitter: @presidentbeef: "
  • @TheSuggmeister: LOL - the false positive button in Twitter's #sadb is labeled "Bullshit (False Positive)". Beautiful
  • RT @TheSuggmeister: LOL - the false positive button in Twitter's #sadb is labeled "Bullshit (False Positive)". Beautiful
  • Twitter: "Phantom Gang looks for mixed content (thru dynamic crawler), sensitive forms posting over non-HTTPS, old jquery"
  • @jdarrelthomas: #phantomgang: dynamic analysis to complement #brakeman. #sadb #AppSecUSA
  • Twitter: "Phantom Gang: series of node.js processes; spins up headless browser instances to see what users see"
  • Twitter: "Output of Phantom Gang goes to JIRA, not developer directly; (hard to trace to individual developer)"
  • @dharmatel: The Twitter gang had one of the smoothest and most understandable mid-talk app demos I've seen at a conference. #AppSecUSA #sadb
  • Twitter: "+1 RT @dharmatel: Twitter gang had smoothest & most understandable mid-talk app demos I've seen at a conference. #AppSecUSA #sadb
  • (Did I hear the Twitter people mention Etsy? Jeez, Etsy people are everywhere. @ngalbreath)
  • Twitter: "CSP module: has policy of what can or can't be on any given page;
  • @mattjay: Looks like twitter is using an internal DAST tool called Phantom Gang that spins up NodeJS instances to do browser emulation #AppSecUSA
  • RT @mattjay: twitter using an internal DAST tool called Phantom Gang to spin up NodeJS instances to do browser emulation #AppSecUSA
  • Twitter: "We use Big Data capabilities to look for site spikes, often indicator of XSS attack"
  • Twitter: "We use CSP to enforce HTTP Strict Transport Security, protecting against firesheep attacks; Dev&Ops loves this"
  • Twitter: "Now we have security headers for entire Twitter site"@mattjay: Twitter is also one of the few that is willing to brave implementing CSP on their sites. Kills XSS. #AppSecUSA
  • RT @mattjay: Twitter is also one of the few that is willing to brave implementing CSP on their sites. Kills XSS. #AppSecUSA
  • Twitter: "ThreatDeck: came from TweetDeck, where columns set up to look for attack patterns; now built into #sadb;"
  • Haha. Twitter: "Roshambo: (rock papers scissors game to pick loser who had to manually find & review code); now automated
  • Twitter: "@edu_mayoral: RT @wickett: Google makes devs support the app for 6 mo. before ops will take it. @RealGeneKim at #DevOps #AppSecUSA #OWASP
  • Awesome Limoncelli Usenix preso. RT @wickett: Google makes dev support app for 6 mo. before ops will take it @RealGeneKim #DevOps #AppSecUSA
  • Twitter: "Our journey: manual > automated; low visibility > trending/reports; late discovery > auto notification"
  • Twitter: "All tools described in this presentation are open source (or tools we built)"
  • Twitter Roshambo lesson: manual code reviews reqd after big chgs were so tedious, no one wanted to do it. so they automated it.
  • Twitter: "Q: how long did it take you to get to where you are? A: went from constant emergency to more strategic in yrs"
  • Twitter: "8 infosec staff covering hundreds of Dev"
  • Twitter: "We want to open source #sadb, but it's very Twitter specific; don't want to get ur hopes up" (crowd sighs sadly)
  • Twitter: "
  • @bhitchcock247: The presentation on #sadb at #appsecusa was the best talk off the day. Lots of creative innovation going on at twitter
  • @SuperSerch: RT @wickett: Your code looks like your people and the communication between them. That's deep @bwsr_sr #AppSecUSA
  • RT @0xcharlie: [Check out awesome Twitter talk] @presidentbeef @nilematotle @alsmola #appsecusa: http://t.co/drUXxwbN

  • Next up: the famous Nick Galbreath (@ngalbreath) on 'Infosec & Continuous Delivery"

  • Line operations safety audit: John Benninghoff

  • .@ngalbreath: "Talking up #BSidesLA, because LA is beautiful. :) At Etsy, was Director of Engr, did enterprise, fraud, fun"

  • .@ngalbreath: "SQLi is still OWASP #1. Why?

  • .@ngalbreath: "Old, unhealthy paradigm: high SW distrib costs: retail CDs, embedded systems, safety/medical, OS (phone/server)

  • .@ngalbreath: "Necessitates low deploy rates, with high impact and risk. Nothing wrong, but it doesn't have to be this way..."

  • (Haha) @ngalbreath: "Circa 2000: True story: CTO wouldn't allow spelling fix error b/c too risky! Could crash site!"

  • .@ngalbreath is giving awesome thesis on deriving #devops from first principles, showing how world has changed in last 20 yrs

  • .@ngalbreath: "@shai_saint: RT @0xcharlie: Wonder what I do at Twitter? Check out my co-workers talk today @presidentbeef @nilematotle @alsmola at #appsecusa: http://t.co/drUXxwbN

    • RT @0xcharlie: [Check out awesome Twitter talk] @presidentbeef @nilematotle @alsmola #appsecusa: http://t.co/drUXxwbN
  • .@ngalbreath: "The way to break the doom loop/endless hamster wheel of pain? Continuous Deployment. Successful companies do it"

  • .@ngalbreath: "Goal: have Dev responsible & confident with their code. In Production." (vs. not caring about production)

  • .@ngalbreath: "How to change culture of caring Dev: make a Deploy button that anyone can push, pushing trunk to Production"

  • HAHA!!! Awesome. .@ngalbreath: "Response #1: FEAR! Dev afraid to push the button!!! Afraid to deploy their own code!"

  • .@ngalbreath: "Attempt #2: Someone brave pushes button, but then someone else tells them that their code broke production"

  • .@ngalbreath: "Attempt #3: With graphs: everyone can see how releases impact Prod: error logs, performance, etc. Graphite etc

  • .@ngalbreath: "Seeing graphs of page views and logins drop to zero is awesome."

  • .@ngalbreath: "Attempt #4: Now Dev does their deploy, is aware of problems, fixes it to completion; small understandable chgs

  • .@ngalbreath: "No one wants to do code reviews of 50 pgs of code diffs. They pick random, trivial stuff. Defies understanding

  • .@ngalbreath: "Attempt #7: dark pushes: push out all supporting files for feature but don't enable feature; enable w/config chg

  • .@ngalbreath: "When everything is in trunk (vs a branch), everyone can see it; vs people having to check out special branch

  • .@ngalbreath: "Attempt #8: Ramp-up: turn on features selectively to small # of machines/users/etc. Reduces risk!"

  • .@ngalbreath: "Start w/employees only, then 1%, 2%, 10%; after gaining confidence, roll out to 100% users"

  • .@ngalbreath: "Addl benefit: when only 1% of users have feature, infosec can catch up while exposure is low"

  • .@ngalbreath: "Attempt #9: Eliminate stupid bugs w/commit or pre-commit static code analysis"

  • .@ngalbreath: "Goal: make as much of infosec available as Self-service to Dev. Get out of the way"

  • .@ngalbreath: "Attempt #10: Increase ability to communicate: IRC works well. Etsy releases are completely self organized

  • @stevewerby: Though I didn't make it to #appsecusa despite being an hour away, I feel like I'm there thanks to @RealGeneKim. :-)

  • .@ngalbreath: "All sorts of bad things happen when Dev is trained not to care about Production"

  • .@ngalbreath: "Here's what it feels like when @mrembetsey calls you at 3am, telling you not to do that again" [everyone laughs]

  • .@ngalbreath: "Result? Dev now understands consequences of their changes. And Infosec can find/fix issues fast"

  • .@ngalbreath: "Another result:

  • yes! .@ngalbreath: "Infosec can bridge multiple disciplines: dev, qa, ops, release, business"

  • .@ngalbreath: "Infosec has special powers: When breach happens, we need to patch fast."

  • Yes. @ngalbreath: "Continuous deployment: Trust me, you want it."

Panel

  • @brianaengle: @RealGeneKim If u can't tie a [code function] to an active responsible employee it should be killed. Culture changing DevOps
  • @brianaengle: @RealGeneKim on DevOps. While humans sleep at night machines laugh at us for all of the things we do manually.
  • @brianaengle: @mortman on separation of duties within DevOps: Utilize automated systems with consistent process and checks.
  • @chriseng: It was suggested by @NGalbreath that reach-arounds can be a catalyst for getting devs to help with security. #appsecusa #hiswordsnotmine
  • RT @chriseng: @NGalbreath sez reach-arounds can be a catalyst for getting devs to help with security. #appsecusa #hiswordsnotmine
  • @iteration1: A @mrembetsy shout out from @RealGeneKim during the #devops panel talk at #appsecusa. We miss you man!
  • RT @iteration1: A @mrembetsy shout out from @RealGeneKim during the #devops panel talk at #appsecusa. We miss you man!
  • @brianaengle: At devops panel #appsecusa: Can InfoSec champion the DevOps movement within an organization? @wickett says no, infosec is irrelevant auditor
  • @brianaengle: At devops panel #appsecusa: Can InfoSec champion the DevOps movement within an organization? @wickett says no, infosec is irrelevant auditor

  • @RealGeneKim: #appsecusa This @mrembetsy talk is among the best I've seen: he describes the Etsy transformation: 2nd video down: http://t.co/VCd90yc1