Testing all the way to Production - Sam Adams @LMAX
They have grown fast, in 2010
Keep the build always releasable @LMAX
When you commit you are saying "I believe this code is safe to go to production"
Release schedule every two weeks and they have the luxury that financial markets stop over the weekend
Over 7,000 automated acceptance tests, card/story not done until green ATs
@markmadsen: #2 continuous testing starting at point of commit, using same exact binaries, scripts, kit as production
@LMAX they have a build sheriff and everyone gets to be sheriff for a day
They had painful releases and they did retrospectives... knowing the full health of the system turned out to be the goal
@chrisgonyea: “Long releases = tired unhappy dev/ops = more human error”
Testing-in-Live, not functional testing, not perf testing, ... tests the plumbing and communication between components
Test entire workflow through the system and make sure everything works
@HSilatani: #velocityconf LMAX Sam Adams - test data isolation in prod and venues ! Game changer in my opinion.
@ThomasatKeynote: "... production is just another release ..." that is modern development by .@LMAX
@roman_pavlyuk: "There's nothing special about production -- it is just another release!". The meaning of CD in one phrase!
My notes: I like the direction they went here. This seems exactly like what we did at NI and at Mentor Graphics. We sent in real jobs and verified results, created new users, test web workflows,... Basically we tried to walk the entire stack with each transaction or test. A little more high stakes at @LMAX as they have real money there. We called this smoke tests but here they call it Testing-in-live. Basically write tests that mock out the user experience with as many pathways through the system
Most of the time we measure the performance of others
@markmadsen: Third party content makes up large portion of many pages, example of page with data from 142 domains, 25sec load time