Testing at cloud speed

by James Wickett on


Talk by @matt_tesauro

The Problem

  • Talking about securing apps in a DevOps world (API, CD, CI, QE,...)
  • The problem surrounding security and DevOps is don't be the stop sign guy
  • If you try to stop the biz then the will just avoid you
  • New programming languages like ruby and python make old skool security practices like static analysis more difficult

The Solution(s)

  • The solution to testing in a DevOps world is automate all the security testing (apps, ops, config mgmnt)
  • Run short tests all the time (ssl expiry, other ssl ops)
  • Smoke tests versus full regression test so you can check early and often (static analysis via grep as a smoke test)
  • If developers hear you speaking developer, they actually will like you... #infosec #wisdom from @matt_tesauro

Test Driven Security (ala TDD)

  • It's time to set the security snail on fire .@matt_tesauro
  • be agile. Period. Add in manual testing when you can, but get a percentage of your security testing running every commit

Config Management Security

  • Once you harden one chef cookbook, all people who use the cookbook benefit
  • review cookbooks for security
  • run a HIDS on the nodes you spin up with a config management, a third party agent
  • nice to have agents on the nodes that report what is happening

Vulnerability Scanning doesn't scale

  • Add values to ops teams by signing up to email lists with automated alerting (gmail plus filters) to notify people that use a version of the software
  • Recommendation for sequinia VIM
  • devs don't care about PDFs... Submit vulns and security findings to the bug tracker
  • security issues can get stuck in death by backlog tho...
  • Try do a security sprint every now and then to take care of the less serious vulns

Reports = findings + automation

  • @beto_atx: Threadfix mention at #OWASP mtg by Matt Tesauro from @Rackspace http://t.co/3SD5gOA7un
  • use markdown format in dradis and role a report from that (use pandoc)

Start with devs

  • make the security testing easily reproducible for devs
  • build a static page with all the links for reflective cross site scripting and hand it off to devs (fix until the alerts stop)

How to test APIs

  • threat models are great to key off of
  • look at the XML stuff for the API cuz devs all love json more so the XML part might have vulns
  • focus on connections with external systems
  • focus on format translations XML to JSON
  • if you run Nessus, print to PDF and hand that off you provide anti-value

Quiet is better than wrong

  • befriend dev teams
  • don't dictate implementation, instead give a requirement
  • @c3llardoor: "Your real value is filtering out results and only handing actionable items over." @matt_tesauro at Austin #OWASP
  • remember fast deploys also mean fast fixes so your exposure window is smaller if you can get the people is right


  • chef security? They review the repo, check versions, rely on HIDS agent to check what is happening and if old version is good
  • cloud passage has a hard time dealing with package version and software regression and the distros sometimes back port the tool
  • virage on github as monitoring things from rackspace